JPH06303601A - Image communication device - Google Patents

Abstract

PURPOSE:To display an optimum face picture capturing a visual line independently of a position and a direction of a face being an object within a permissible range by providing a camera A receiving a sender side picture through a translucent mirror and a camera B receiving a transmission picture and arranging the camera B at a position to which the visual line is easily set form a paper material at hand. CONSTITUTION:The picture communication equipment comprising a camera A21 receiving a sender side picture through a translucent mirror 11 is provided with a camera B22 receiving a transmission picture and a picture changeover switch 31 is used to select a picture from the camera A21 or the picture from the camera B22. The camera B22 consists of a small sized CCD camera and a tripod whose height is adjustable and located at an optional position to receive the transmission picture. For example, when the operator makes an explanation for an opposite party while observing a paper material at hand, the camera B22 is located at a location to which a visual line is easily directed from the paper material at hand, or plural cameras B are provided and a control section is used to select any of pictures detecting the visual line preferably.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for line-of-sight matching in an image communication device such as a video telephone device, a video conference device, and a multimedia distributed conference device.

[0002]

2. Description of the Related Art As an example of a conventional image communication apparatus, it is described in a document [Kunihiro Suetake, Kenichi Yoshikawa, "Development of eye-coincidence type small-size videophone apparatus", 1988, Shin-Gaku Autumn Fall, D-38]. I will explain what you have. For example, the image display unit (1) displays a communication unit that transmits and receives an image signal to and from a communication terminal on the partner side and a face image of a moving image or a still image on the partner side received by the communication terminal. The image display unit (1) is installed with its display surface facing upward, and a translucent mirror is located directly above the image display unit (1). The semitransparent mirror is tilted at 45 °, reflects the image of the other party displayed by the image display section (1), and displays the reflected image of the other party on the image display section (2). On the other hand, the camera located just before the semi-transparent mirror is
The image on the transmission side in front of the image display unit (2) is input through the semitransparent mirror. As a result, the transmission side has the image display unit (2).
There has been reported an image communication apparatus capable of transmitting a transmission side image while seeing the image of the other side displayed in FIG.

[0003]

However, in the above-mentioned line-of-sight matching type small-sized videophone device, since the camera is installed in the back of the image display section (2), the range of capturing the subject of the camera is narrow, and the operator's range is small. The position of the face is quite limited.
On the other hand, the operator often looks through the keyboard and the materials at hand, and the movement of the face is not constant. As a result, there is a problem that many images whose eyes do not match are displayed on the image display section (2) of the other party.

Therefore, the present invention solves such a problem, and an object of the present invention is to optimize an optimum face in which a line of sight is captured without being influenced by the position or orientation of the face of a subject within a certain allowable range. It is intended to display images.

[0005]

In order to achieve the above object, a communication means for transmitting and receiving an image signal to and from a communication terminal of a partner through a communication line, and a moving image or a still image of the partner received by the communication terminal. Image display unit (1) that displays the face image of
Through a semitransparent mirror that reflects the image of the other party displayed by the image display unit (1), an image display unit (2) that displays the image of the other party reflected by the semitransparent mirror, and the semitransparent mirror In an image communication apparatus including a camera A for inputting a transmission side image, the first invention image communication apparatus includes a camera B for inputting a transmission image and an image changeover switch for switching the images of the camera A and the camera B. It is characterized by having.

The image communication apparatus of the second invention comprises a face image detecting means for detecting a face image from the image on the transmission side input by the camera B, and a camera of the camera B when the face image is detected from the image of the camera B. It is characterized by comprising image selecting means for selecting an image and selecting an image of the camera A when a face image cannot be detected from the image of the camera B.

The image communication apparatus of the third invention comprises at least one camera B for inputting a transmission image, eye image detecting means for detecting an eye image from the images input by the camera A and the camera B, and the eye image detecting means. A calculation means for calculating the image area of the eye from the image and an image having a face image with the largest eye image area value is selected from the images of the camera A and the camera B. If there is no eye image, the image of the camera A is selected. An image selecting means for selecting is provided.

An image communication apparatus according to a third aspect of the invention is characterized in that it has delay means for performing the number of times of detecting a face image at a certain fixed time.

[0009]

According to the image communication apparatus of the first invention, the camera B for inputting the transmission image can be placed at any place. In addition, even if the position or orientation of the operator's face changes, it is easier to capture than the camera A. For example, when the operator is looking through the material at hand and explaining it to the other party, it is easy to direct the line of sight from the material at hand. By placing the camera B on the camera, the operator can explain with a natural motion without returning to the camera A.

According to the image communication device of the second invention,
Face image detecting means for detecting a face image from the image on the transmission side input by the camera B, and when the face image is detected from the image of the camera B, the image of the camera B is selected, and the face image is selected from the image of the camera B. When the image cannot be detected, the image selecting means for selecting the image of the camera A can transmit an important face image for a meeting or a meeting, and can remove a useless image having no face image. Further, when the operator wants to explain the figure of the material to the other party, he or she can explain it to the other party by selecting the image of the camera A.

According to the image communication device of the third invention,
The camera A and at least one camera B for inputting the transmission image broaden the range captured by the camera, and the face image can be easily detected even if the position or orientation of the operator's face changes. Further, by an eye image detecting means for detecting an eye image from the images input by the camera A and the camera B,
The image in which the line of sight is detected can be selected. Further, the area of the eye image detected by the calculation means is obtained, and the image selecting means for selecting the image having the largest eye image area value from the images of all the cameras can further select the image in which the line of sight is captured.

According to the image communication device of the fourth invention,
By the delay unit that performs the number of times the face image is detected every certain period of time, it is possible to prevent the face of the other party from becoming difficult to catch due to continuous face image detection, and it is possible to transmit the optimum face image. .

[0013]

(Embodiment 1) An image communication apparatus will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a conventional image communication apparatus. Communication terminal 10 and communication line 1 of the other party
The image display unit (1) 12 displays the communication means for transmitting and receiving the image signal via the communication terminal 4 and the face image of the moving image or the still image of the other party received by the communication terminal 10. Image display section (1)
The display surface 12 is installed upward, and the semitransparent mirror 11 is located directly above the image display unit (1) 12. Translucent mirror 11
Is inclined at 45 ° and reflects the face image of the partner displayed by the image display unit (1) 12 and displays the reflected face image of the partner on the image display unit (2) 13. On the other hand, semi-transparent mirror 1
The camera 21 (camera A) positioned immediately before 1 inputs the face image on the transmission side in front of the image display unit (2) 13 through the semitransparent mirror 11. As a result, the transmitting side transmits the face image of the transmitting side while looking at the face image of the other side displayed on the image display unit (2) 13, and the image communication apparatus for holding a conference in which the line of sight coincides is shown in FIG. FIG. 3 is a configuration diagram of an example of an image communication apparatus in Example 1. The camera 22 (camera B) is composed of a small CCD camera and a tripod that adjusts the height, and is placed in an arbitrary place to input a transmission image.
Is. For example, when the operator explains to the other person while looking through the material at hand, it is preferable to place the camera 22 at a place where it is easy to direct the line of sight from the material at hand. Further, the images of the camera 21 and the camera 22 are switched by the image changeover switch 31, and as a result, an image communication apparatus in which the lines of sight coincide with each other is realized.

(Second Embodiment) An image communication apparatus will be described below with reference to the drawings. FIG. 3 is a block diagram of an embodiment of the image communication apparatus according to the second embodiment, and FIG.
FIG. 7 is a diagram showing an example of a control unit 30 including the face image detecting means of the present invention. Control unit 3 provided with face image detection means
0 is connected to the camera 21 (camera A) and the camera 22 (camera B). The face image detecting means of the control unit 30 is
The method already known in JP-A-1-314385 will be referred to. For example, if the image of the camera is Cn = Cn + 1
(N is the number of images) 40, the image of the camera 22 is C1
Therefore, the skin color area extraction 41 is performed, and if the skin color area cannot be extracted, the image selection means 51 selects the image 51 of the camera 21.
I do. It is impossible to determine whether the image is a face image only by performing the skin color region extraction 41. Therefore, the non-skin color region extraction 4 surrounded by the skin color region is performed.
Do 2. If the non-skin-colored area surrounded by the skin-colored area cannot be extracted, the image selection means 51 performs the image selection 51 of the camera A. Next, from the non-skin color area extraction 42 surrounded by the skin color area, it is determined 43 whether there is an eye image candidate, a nose image candidate, or a mouth image candidate that are the features of the face. FIG. 5 is a diagram showing an embodiment of the face image feature matching means. Eye image matching 50 is performed to match the shape of the eye image candidate with the general shape of the eye, and when the image candidates match, the nose image matching 53 is performed as the eye image determination 52.
If they do not match, the image selection means 51 selects the image of the camera A 51. The nose image matching 53 is performed to match the shape of the nose image candidate with the general shape of the nose, and when the image candidates match, the nose image determination 54 is performed as the mouth image matching 55. The image selection 51 of A is performed. The mouth image matching 55 is performed to match the shape of the mouth image candidate with the general shape of the mouth, and when the image candidates match, the mouth image determination 56 is made. That is, all matching is performed, and when the image candidates match, the face image detection 57 (see FIG.
(The same as the face image detection 44). Next, the image display switching of the face image detection 57 is performed from the image transmitted so far 48 by the image selection means of the control unit 30. As a result, it is possible to realize an image communication device in which the lines of sight coincide with each other.

(Third Embodiment) An image communication apparatus will be described below with reference to the drawings. FIG. 6 shows a block diagram of an embodiment of the image communication apparatus of the present invention. The camera 21 (camera A) inputs the front face image of the operator 15 on the transmission side, and among the cameras 22 to 25 (at least one camera B),
The camera 22 inputs the face image facing the left side of the operator, the camera 23 inputs the face image facing the right side of the operator, and the camera 2
4 inputs a face image facing the upper side of the operator, camera 25 inputs a face image facing the lower side of the operator, and all cameras are connected to the control unit 30. FIG. 7 is an overall view showing an embodiment of the control unit 30 that controls the face image with the same line of sight.
All the cameras 21 to 21 are controlled by the eye image detection means of the control unit 30.
The face image is detected from the image input by 25, and then the eye image is detected. Regarding the method of detecting the face image and the eye image, the method already known in Japanese Patent Laid-Open No. 1-314385 is referred to. For example, when the image from at least one camera is Cn = Cn + 1 (n is the number of images) 40, for example, in the case of the camera 21 and the cameras 22 to 25, it is C5, and the skin color area extraction 41 is performed. At this time, if there is no skin color area, the image selection means 51 selects an image from the camera 21. Since it is not possible to determine whether the image is a face image only with the skin color region extraction 41, the non-skin color region extraction 42 surrounded by the skin color region is performed. Next, the presence / absence of eye image candidates, nose image candidates, and mouth image candidates, which are facial features, is checked by the face image feature matching 43 from the non-skin color region extraction 42 surrounded by the skin color regions. FIG. 5 shows an embodiment of performing face image feature matching 43. Eye image matching 50 is performed to match the shape of the eye image candidate with the general shape of the eye. If the image candidates match, nose image matching 53 is performed as eye image determination 52, and if they do not match, image selecting means 51.
The image from the camera 21 is selected by. The nose image matching unit 53 that matches the shape of the nose image candidate with the general shape of the nose is performed. If the image candidates match, the nose image determination unit 54 proceeds to the mouth image matching unit 55, and if they do not match, the image selecting unit. The image from the camera 21 is selected by 51. Mouth image matching 5 that matches the shape of the mouth image candidate with the general shape of the mouth
5 is performed, and when the image candidates match, the mouth image determination 56 is made. That is, the case where the image candidates match in all the matching is set as the face image 57 (the same as the face image detection 44 in FIG. 4). Next, a calculation means calculates the image area of the eye 45 from the eye image, and the image selection means compares the eye image areas.
6, the face image having the largest image area of the eyes is selected 47 from the face images of all the cameras 21 to 25, and the image display switch 48 is performed from the images transmitted so far by the image changeover switch. If the image selecting unit 51 cannot detect a face image from the image, the image from the camera 21 is selected. As a result, it is possible to realize an image communication device in which the lines of sight coincide with each other.

(Fourth Embodiment) An image communication apparatus will be described below with reference to the drawings.

According to the fourth aspect of the present invention, FIG. 8 is a diagram showing an embodiment of the control unit 30 having the delay means of the present invention. The control unit 30 includes a camera 21 (camera A) and a camera 2
The image of 2 (camera B) is input and a face image is detected. The face image detecting means of the control unit 30 is disclosed in JP-A-1-314385.
Reference is made to the method already known in the publication. For example,
When the image of the camera is set to Cn = Cn + 1 (n is the number of images) 40, the face image detection is continuously performed by the delay unit 60 which performs the number of times of detecting the face image at a certain fixed time. To prevent the face of the other party from becoming difficult to capture, the image of the camera 22 is C1, the skin color area extraction 41 is performed, and if the skin color area cannot be extracted, the image selection means 51 selects the image of the camera 21. Since it is not possible to determine whether the image is a face image by only the skin color area extraction 41, the non-skin color area extraction 42 surrounded by the skin color area is performed. If the non-skin color area surrounded by the skin color area cannot be extracted, the image of the camera A is selected by the image selecting means 51. Next, from the non-skin color area extraction 42 surrounded by the skin color area, it is determined 43 whether there is an eye image candidate, a nose image candidate, or a mouth image candidate that are the features of the face. Next, the image display switching of the face image detection 57 is performed from the images transmitted so far by the image selection means 48. As a result, it is possible to realize an image communication device in which the lines of sight coincide with each other.

[0019]

As described above, according to the first aspect of the present invention, the camera B for inputting the transmitted image can be placed at any place. In addition, even if the position or orientation of the operator's face changes, it is easier to capture than the camera A. For example, when the operator is looking through the material at hand and explaining it to the other party, it is easy to direct the line of sight from the material at hand. Placing the camera B in the position has the effect that the operator can explain with a natural motion without the motion of returning the line of sight to the camera A.

In the second aspect of the present invention, a face image detecting means for detecting a face image from the image on the transmitting side input by the camera B, and an image of the camera B is selected when the face image is detected from the image of the camera B. Then, when the face image cannot be detected from the image of the camera B, the image selecting means for selecting the image of the camera A transmits an important face image for a meeting or a meeting, and removes unnecessary images without the face image. You can In addition, when the operator wants to explain the figure of the material to the other party, he or she can select the image of the camera A to explain to the other party.

According to the third aspect, the camera A for inputting the transmission image and the at least one camera B broaden the range captured by the camera, so that the face of the operator can be changed even if the position or orientation of the face changes. Easy to detect images. Furthermore, the image in which the line of sight is detected can be selected by the eye image detecting means that detects the eye image from the images input by the cameras A and B. Further, the area of the eye image detected by the calculating means is obtained, and the image selecting means for selecting the image having the largest eye image area value from the images of all the cameras can further select the image in which the line of sight is captured. Have.

According to the fourth aspect of the present invention, it is difficult to capture the face of the other party because the face image detection continuously occurs due to the delay means for performing the number of times of detecting the face image at a certain fixed time. It is possible to prevent the above and transmit an optimum face image.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a conventional image communication device.

FIG. 2 is a configuration diagram showing an embodiment of an image communication apparatus of the present invention.

FIG. 3 is a configuration diagram showing an embodiment of an image communication apparatus of the present invention.

FIG. 4 is a diagram showing an embodiment of a face image detecting means of the control unit of the present invention.

FIG. 5 is a diagram showing an embodiment of a face image feature matching means of the present invention.

FIG. 6 is a configuration diagram of an embodiment of an image communication apparatus of the present invention.

FIG. 7 is an overall view showing an embodiment of a control unit of the present invention.

FIG. 8 is a diagram showing an embodiment of a control unit having a delay means of the present invention.

[Explanation of symbols]

 10 ... Communication terminal 11 ... Semi-transparent mirror 12 ... Image display part (1) 13 ... Image display part (2) 14 ... Communication line 15 ... Operator 21 ... Camera A 22 ... Camera B 23 ... Camera B 24 ... Camera B 25 ... Camera B 30 ... Control unit 31 ... Image changeover switch 40 ... Image Cn = Cn + 1 (n is the number of images) 41 ... Skin color region extraction 42 ... Non-skin color region extraction 43 surrounded by skin color regions 43 ... Face image feature matching means 44 ... Face image detection 46 ... Calculation of eye image area 47 ... Selection of image having the largest eye image area 48 ... Image display switching 50 ... Eye image matching 51 ... Image selection means 52 ... Eye image determination 53 ... Nose image Matching 54 ... Nose image determination 55 ... Mouth image matching 56 ... Mouth image determination 57 ... Face image detection 60 ... Delay means

Claims (4)

[Claims] 1. A communication means for transmitting and receiving an image signal to and from a communication terminal of a partner, and an image display unit (1) for displaying a face image of a moving image or a still image of the partner received by the communication terminal. A semitransparent mirror that reflects the face image of the other party displayed by the image display unit (1); an image display unit (2) that displays the face image of the other party reflected by the semitransparent mirror; An image communication apparatus including a camera A for inputting a face image on the transmission side through a transparent mirror, comprising a camera B for inputting a transmission image and an image changeover switch for switching between images of the camera A and the camera B. Image communication device. 2. The image communication apparatus according to claim 1, wherein a face image detecting means for detecting a face image from an image on the transmission side input by the camera B, and a camera when the face image is detected from the image of the camera B. An image communication apparatus comprising image selecting means for selecting an image of B and selecting an image of camera A when a face image cannot be detected from the image of camera B. 3. The image communication apparatus according to claim 1, wherein at least one camera B that inputs a transmission image, and an eye that detects an eye image from the images input by the camera A and the camera B. Image detecting means, calculating means for calculating the image area of the eyes from the eye images, and selecting an image having a face image with the largest eye image area value from the images of the camera A and the camera B, and there is no eye image. In the case, the image communication apparatus is provided with an image selecting means for selecting an image of the camera A. 4. Claim 1 or claim 2 or claim 3.
The image communication device according to the item 1, wherein the image communication device includes a delay unit that detects a face image at regular intervals.