JPS63157593A - Automatic tracking image pickup device - Google Patents

Abstract

PURPOSE:To extract a face region with a simple processing by detecting a moving part from a differential between a delayed video signal and a video signal not delayed, binarizing its moved part and using the number of picture element having a level over the threshold value in a block so as to apply the processing. CONSTITUTION:An inter-frame difference signal from a picture element difference arithmetic circuit 13 is binarized by using a predetermined threshold value to all picture element of n1Xn2 at a binarizing circuit 14 and converted into a binarizing differential signal. The binarizing differential signal is fed to an effective block discrimination circuit 15, where the signal is divided into blocks (n1/l)X(n2/l) and the number of picture elements having a threshold value while being binarized at every block is counted and a block having a total number being a predetermined number or over is decided to be an effective block, where (l) is a common measure of n1, n2. The block differential signal is fed to a face region deciding circuit 16, where the face region of a person is decided. Then when it is decided to be the face region, an extraction signal is outputted and the face region is extracted from a face region circuit 18.

Description

[Detailed description of the invention] "Industrial application field" This invention can be used, for example, in a video telephone device.

The present invention relates to an automatic tracking imaging device that extracts an area including the face of a person from an imaging screen.

``Prior art'' In a video telephone device that uses a low bit rate transmission line 7 to realize a face-to-face conversation between two distant parties, the number of pixels per screen is For this reason, when a wide area is imaged, the degree of resolution of a person's face is insufficient.

It is clearly visible (two is clearly visible). For this reason, it is necessary to display the face of the person as large as possible on the screen.

When a person moves, the part of the person's face may be out of alignment with the viewing angle of the screen.

One way to solve this problem is to install a monitor device and always project your own image, but the monitor device C
Second, you have to constantly be aware of whether you are visible or not, which makes operation extremely difficult.

In addition, when a person moves, the TV camera that is capturing the image of the person can be tracked (= a method of physically moving it is also possible, but in this case, the mechanical part such as the electric pan head for the TV camera, the electric zoom lens, etc.) The problem is that it becomes too large.

An object of the present invention is to provide, for example, a video telephone device without having a mechanical part such as an electric pan head.

Another object of the present invention is to provide an automatic tracking and imaging device that solves the problem of 1: falling off the screen when a person's face moves without providing a monitor device.

"Means for Solving Problems" According to the present invention, a pixel screen of sxrlz is imaged by an imaging device, and its output video signal is delayed by N frames (N is an integer of 1 or more) by a frame memory, The pixel-by-pixel difference between the delayed video signal in the frame memory and the undelayed video signal from the imaging device is calculated by a pixel difference calculating means, and each of the difference signals is binarized by a binarizing means. Ru.

The binarized image is divided into (mouth, /)) x (n2/)) block screens (l is a common divisor of mouth1.n2).

If the number of pixels in each block that is equal to or greater than the binarization threshold is on a predetermined Itfl la, the block is determined to be a valid block by the valid block determining means.

The facial area of the person is determined by the facial area determining means based on the position and continuity of the determined effective block, and the area including the detected facial area of the person is smaller than the n1X rlz screen by km, xkm2 Screen (nl > km ml, n2) k rr12
. In the case of a video telephone device, the pixels of the video signal extracted from the face area extracting means (2) are thinned out by the thinning means as necessary to produce a video signal of ml×m2 pixels.

As described above, according to the present invention, the difference between the delayed video signal and the undelayed video signal is detected, a moving part is detected based on the difference, the moving part is binarized, and the threshold value in the block is converted into a block screen. This is because processing is performed using a number of pixels greater than the value.

Facial regions can be extracted with relatively simple processing.

Embodiment FIG. 1 is a diagram illustrating an embodiment of the automatic tracking and photographing one-image device of the present invention. Imaging device 1 having the number of pixels of nlx mouth 2
The f-th frame (FIG. 2A) of the output video signal from 1 is stored in the frame memory 12. The accumulated f-th frame video signal and the output video signal of the next f+1-th frame (FIG. 2B) from the imaging device 11 are subjected to subtraction processing for each pixel by the pixel difference calculation circuit 13.
A differential signal between both frame video signals is extracted. In other words, a region with movement between the f-th frame and the f+1-th frame (the shaded area in FIG. 2C) is extracted.

The difference signal from the pixel difference calculation circuit 13 is sent to the binarization circuit 1
In 4, all 11x02 pixels (2) are binarized using a predetermined threshold and converted into a binarized difference signal.The threshold used here is based on the contrast between the background and the human part. Considering this, select a value that allows the differential signal to be detected correctly.

The binary and converted difference signals from the binarization circuit 14 are processed by the valid block determination circuit 15 to determine the area around the image of n1Xn2 (
n1/])×(mouth2/)) block (i is nl.

For each block, count the number of pixels that are equal to or greater than the threshold value when binarized, and determine that the block whose total number is equal to or greater than a predetermined value is a valid block. and outputs a block difference signal. Note that the numerical values used for the determination here are selected so that noise such as flicker can be sufficiently removed.

The block difference signal from the valid block determination circuit 15 is
The facial area determination circuit 16 determines the facial area of a person using the following method. The output of the imaging device 11 is also supplied to a synchronization extraction circuit 17, where a vertical synchronization signal and a horizontal synchronization signal are extracted, and these are supplied to a facial area determination circuit 16. As shown in Figure 3 (= each block (X, Y)
It is marked with coordinate points, and the block at the top left of one screen is (1, 1).

First, when y=1, block in the X direction from 1 to n, /J
Scan from left to right until 2 and check whether there is a valid block.

Thereafter, similarly, the presence or absence of a valid block is checked by sequentially scanning in the X direction from y=2 to n 2 /.

For example, in a videophone device that captures an image of a person,
The y coordinate ya at the top of the screen of the effective block is considered to be the top of the person's head, and the part below that is considered to be the face. This y coordinate ya
From the next line, the X coordinate Pt (1<i<(nx
/J) L! :When scanning the right side and the left Cff1, the first (= X coordinate of the valid block that appears Qi (1<i<(n1/)
)), and calculate the effective block width in the X direction for each block line (scanning) Wj (j >a)" Qi Pi'a
'demand. The effective block width Wi is determined by sequentially scanning each block line, and if the value of the effective block width Wi of consecutive lines shows a constant f&i over a predetermined number of block lines, what is the value? It is determined that the width of the face is W. Note that the number of lines used here is selected in consideration of the block size so that the width of the facial area can be detected correctly.

Also, let the X coordinates of the final block line determined to be the width W of the face part be Pi - Qi and Xq, and its center coordinate X
(=: (Xp + Xq) / 2 is determined. The vertical length T of the face part is determined by the width W (
=Xq
t is a positive real number) (obtained from 2. Facial area determination circuit 16
Then, the mouth including the face, k r smaller than the ×02 pixel area
A region extraction signal is output that specifies a region consisting of 2 pixels (m1', m2 are predetermined values, positive real numbers). For example, xmi of the extraction area in the block coordinate system when the width:height ratio including the face part is 4=3. # xmax + Ymrn, ymax
The value of (Figure 3) can be found from the following equation (2).

xmin=xo-2/3・T xmax=xo+2/3・T ymjn=Ya Ymax ”Ya+T From the above values, ×nkm1xkm on a 2-pixel screen
Extraction coordinates of 2-pixel area Xm1n −Xmax + Ym
in +Y□aX (Fig. 2C) is determined by the following equation.

Xm1n = (xmin 1) × xmax:
xmaxxノYinin = (Ymin 1) XノYmaX
= Area extraction coordinate instruction signal Xm1n determined by ymax x or more #X
max * Ymin, Ymax synchronous extraction circuit 17
The facial area extraction circuit 18 uses the vertical synchronization timing extracted in
(;Given.) The face area determination circuit 16 determines the screen area of the person based on the position of the effective block and its continuity.

Is FIG. 4 an example of the facial area determination circuit 16? FIG. The block difference signal from the valid block determination circuit 15 is sent to the ya detection circuit 31. IH delay circuit 32. The signal is input to the IH delay reverse scanning circuit 33.

When the ya detection circuit 31 detects the y coordinate ya of the valid block at the top of the screen, it notifies the flip-flop 34C2 and sets the data 3/a in the ya register 35. From the output signal of the IH delay circuit 32, Pl is detected by the Pi detection circuit 36, and from the output signal of the IH delay reverse scanning circuit 33, Qi is detected by the Qi detection circuit 37.

The W calculation circuit 38 calculates (
QiPi) and stores the calculation result in the W register 39. Note that the W calculation circuit 38 uses the ya detection circuit 31 to
A flip-flop 41 is controlled by the output of the flip-flop 34 in order to enable operation when the a-mirror is released.

The W comparison circuit 42 compares the data in the W register 39 with the next data from the W calculation circuit 38.

If the comparison results are equal, the counter 43 is incremented by one. Further, if the comparison results are not equal, the counter 43 is reset. When the count value of the counter 43 reaches a predetermined value of 1 (=), the second outputs a set signal, the P resistor, 14 receives the Pi signal from the Pi detection circuit 36, the Qi register, and 15 the Q1 detection circuit 37. Qi' from
& Set each. By the way, once the tank set signal is output, the flip-flop 46 is controlled by the output of the counter 43 in order to stop the counter 43 from operating until the first screen appears (until the next vertical synchronization signal or later).

In the extraction area designation circuit 47, the ya register 35, Pi
Based on each of the registers 44 and Qi register 45, the area extraction coordinate instruction signal Xmin-X is generated by the method described above.
The facial area extraction circuit 18 that calculates and outputs max e Ymin + Ymax calculates km1 from the input video from the frame memory 12 based on the area extraction coordinate instruction signal from the facial area determination circuit 16 obtained in this way.
An image of only the xkm2 area (FIG. 2D) is created and input to the thinning circuit 19 (FIG. 1).

The thinning circuit 19 thins out the pixels of the created video.
It is output to the output terminal 20 as a video signal of the screen (FIG. 2E) consisting of Ixm2. Since this video signal has been thinned out, the number of pixels is reduced compared to the input video signal, and the resolution is lowered. Therefore, when a normal video signal is used, two systems are possible in which the signal is transmitted at a low bit rate and with a reduced number of pixels.

In addition, if a normal image signal is required as an output, an imaging device with approximately 1000 scanning lines or high-quality images should be used (
It can be obtained from two. In this case, the thinning circuit 19 may not be provided.

′? In FIG. 51, the delay in the frame memory 12 is not limited to l frames, but may be delayed by an integer number of frames.

``Effects of the Invention'' As explained above (2. The device of this invention images a wide area including the face area of a person, and extracts the area 7 surrounding the person's face from within the imaged screen using the movement of the person. From using a new video signal.

It does not require any special mechanical parts or the use of a monitor device, and can always capture images centered on the face of the person.
Another advantage is that even if the person approaches or moves away from the imaging device, the image of the beggar can be captured at approximately the same size.

[Brief explanation of the drawing]

Fig. 1 is a schematic diagram showing an embodiment of the device of the present invention, Fig. 2 is a diagram showing input/output screen collapse of each part of Fig. 1, and Fig. 3
The figure is a diagram for explaining the operation of the facial area determining circuit, and FIG. 4 is a block diagram showing a specific example of the facial area determining circuit 16 in FIG. Patent applicant: Nippon Telegraph and Telephone Corporation Representative: Takashi Kusano /'17
1 Second entry Δ Off frame B Tree (
L+1) Frame Meka Erection Output Thinning Output Sweat 5 times

Claims (1)

[Claims] (1) An imaging device that images an n_1×n_2 pixel screen, a frame memory that delays the output video signal of the imaging device by N frames (N is an integer of 1 or more), and the delayed video signal of the frame memory and the image pickup device. A pixel difference calculating means for calculating a pixel difference with the undelayed video signal from the device, a binarizing means for binarizing the pixel difference signal from the pixel difference calculating means, and a binarized image (n_1/ Divide into l) x (n_2/l) block screens (l is a common divisor of n_1 and n_2),
Valid block determining means for determining a valid block when the number of pixels in each block that is equal to or higher than a binarization threshold is equal to or higher than a predetermined value, and a person based on the position and continuity of the determined valid block. facial area determining means for determining the facial area of the person, and n_1×n_ including the facial area of the person detected by the determination.
km_1×km_2 screen (n_1>
km_1, n_2>km_2, m_1, m_2 are predetermined values, and k is a positive real number). Facial area extraction means extracts only the video signal of the area from the output video signal of the imaging device. .