JPH07131764A - Picture interactive device - Google Patents

Abstract

PURPOSE:To obtain an interactive presence from the picture interactive device in which positions of eyes of talkers are detected to match visual lines on a display device. CONSTITUTION:A liquid crystal display screen 11 is driven in the transparent state, a video camera 7 picks up a talker 13 from the front side and an infrared ray from an infrared ray source 12 arranged on other liquid crystal display screen 11 is reflected in eyes of the talker 13 and visual line position information is obtained by an infrared ray image sensor 8. A picture in which an eye position is matched with an optical axis of the video camera 7 is stored or sent by an image memory section 3 under the control of a write control circuit 9 based on visual line position information. On the other hand, a reception section 2 receives a picture sent in the similar operation and a 2nd picture memory section 4 fetches the picture and a read control circuit 10 reads the picture based on visual line position information of the talker 13 and the position of the talker 13 is displayed on a liquid crystal video projector 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image interactive apparatus capable of simultaneously communicating not only both voices but also images in a distant place such as a videophone and a video conference.

[0002]

2. Description of the Related Art Conventionally, an image interactive apparatus such as a videophone and a video conference has a structure capable of transmitting and receiving an image and a voice at the same time. In such an image dialogue device, an image pickup device, that is, a camera, for photographing a talking person is installed.

However, since this camera is arranged laterally with respect to the display, when talking while looking at the image of the other person on the display, the image on the camera shows the talker's face down or to the side. Since it will be an image facing
It was not a conversation in the situation of facing each other.

Therefore, in order to solve such a drawback, a device as shown in FIG. 5 has been proposed. (This is described on pages 123 to 125 of the Journal of Image Electronics Engineers, Vol. 22, No. 2, pp. 123-125, published on April 25, 1993. That is, in FIG. 5, 13 is a speaker and 11 is a speaker. A screen in which a transparent state and a light scattering state are alternately repeated on a liquid crystal screen, 5 is a video projector such as a liquid crystal, 7 is a video camera, 6 is a drive circuit for controlling the liquid crystal screen 11, and 20 is an image for storing the face image of the talker. A memory unit, 1 is a transmitting unit, 2 is a receiving unit, 14 is a transmitting side communication line, and 15 is a receiving side communication line.

Next, the operation of FIG. 5 will be described. First, the front of the face of the speaker 13 is photographed by the video camera 7 via the liquid crystal screen 11. The drive circuit 6 makes the liquid crystal screen 11 transparent so that the video camera 7 can photograph the speaker 13.

Then, the image is temporarily stored in the image memory unit 20.
The same image is read from the image memory unit 20 and transmitted from the transmission unit 1 via the communication line 14 until the liquid crystal screen 11 becomes transparent again by the drive circuit 6.

On the other hand, the image of the other party is sent to the receiving unit 2 via the communication line 15 from the other party in the above-described method and receives it. Then, the receiving unit 2 sends the image to the liquid crystal video projector 5 and displays the image on the liquid crystal screen 11.

According to such a system, the video camera can capture an image in a state of facing the front even if the speaker who is looking at the display speaks in the same direction as that of the speaker. You can talk in the state of.

However, even if such a system is used, the speakers are not always able to talk while looking at the lens of the video camera, and furthermore, the positions of the eyes of the speakers are slightly different from each other. , There is a problem that it is impossible to completely match the eye positions.

The simplest way to eliminate this is to mark the position of the optical axis of the video camera on the liquid crystal screen, but the speaker must always be aware of the marking when speaking. If the marking is of a size that can be easily recognized, there is a problem in that both display and photography are an obstacle.

[0011]

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and in an image interactive apparatus, it is possible to detect the positions of the eyes of the speakers and to align the line of sight on the display. The purpose is to gain a sense of realism in the dialogue.

[0012]

SUMMARY OF THE INVENTION According to the present invention, an image pickup device transmits an image and voice of a person on the transmitting side to a person with whom communication is to be performed via a communication line, while an image of a person with whom the person is to communicate is transmitted. In an image dialogue device that receives voice and voice through a communication line and displays an image on a display device, and performs communication between the two, a line-of-sight position detection unit that detects the line-of-sight position of a person who sends communication, and a line-of-sight position Transmission means for transmitting the line-of-sight position information obtained by the detecting means together with the image through the communication line, a receiving means for receiving the image of the other party communicating through the communication line and the line-of-sight position information, and the other party obtained by the receiving means From the line-of-sight position information of the eye and the line-of-sight position information of the communication sending side obtained from the line-of-sight position detecting unit, the line-of-sight position correcting unit that matches the line-of-sight position of the person image displayed on the display device with the line-of-sight position of the communication sending side. And from That is an image interaction device.

[0013]

According to the present invention, with the above-described configuration, the line-of-sight positions of the speakers are sent to each other, and the line-of-sight of the image is aligned with the line-of-sight position of the speakers looking at the display side. Match.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows a block diagram of an embodiment of the present invention. In this block diagram, the same drawing numbers as those shown in FIG. 5 of the prior art have the same functions, and therefore the description thereof will be omitted.

In FIG. 1, 3 is a first image memory unit,
4 is a second image memory unit, 8 is an infrared image sensor,
Reference numeral 9 is a write control circuit, 10 is a read control circuit, and 12 is an infrared light source.

Next, the operation of the present invention will be described.
First, as in the conventional technique, the video camera 7 captures an image of the speaker 13 via the liquid crystal screen 11, and the captured image is stored in the first image memory unit 3.

That is, in front of the speaker 13, the liquid crystal screen 1
1 is arranged, and the video camera 7 and the infrared image sensor 8 are arranged behind it.
Causes the shutter to be periodically opened for shooting, and outputs a control signal for driving the liquid crystal screen 11, and the drive circuit 6 receives the signal and keeps the liquid crystal screen 11 transparent while the shutter is open. Put in a state.

As a result, the video camera 7 has the speaker 1
The image of 3 will be photographed. Since today's video camera 7 realizes a shutter speed on the order of 1/1000, it takes only a short time to open the shutter, and if this is repeated at a cycle of 1/30 to 1/60 seconds, reproduction is performed. A moving image comparable to
It is stored in the image memory unit 3.

By the way, in order to align the line-of-sight positions of the speakers, the present invention is carried out as follows. That is, the infrared light source 12 is arranged above the liquid crystal screen 11, and the speaker 13
Irradiate against. Then, as shown in FIG. 4, the infrared rays are reflected by the eyes of the speaker 13 and pass through the liquid crystal screen 11, and the infrared image sensor 8 causes the liquid crystal screen 11 to move.
With respect to the line of sight position, the line of sight position information e is output.

The writing control circuit 9 stores the image on the basis of the line-of-sight position information. That is, as shown in FIG. 3, it is detected how much the optical axis of the video camera 7 is deviated, and the information is used as line-of-sight position information. In the case of FIG.
In A, the shift is c with respect to the optical axis, and in B, the shift is d with respect to the optical axis.

The writing control circuit 9 writes the line-of-sight position of the image in the first image memory unit 3 based on the line-of-sight position information so as to coincide with the optical axis. That is, at this time, the write control circuit 9
Receives the above-mentioned line-of-sight position information so that the image of the eye of the speaker 13 has the same positional relationship with the optical axis at a predetermined address among the addresses of the memory of the first image memory unit 3. Controls the timing of writing. The image corrected in this way is sent from the first image memory unit 3 to become image data, and the transmitting unit 1 puts this on the communication line 14 as communication data.

On the other hand, the receiving section 2 which has received the communication data from the communication line 15 sends it to the second image memory section 4. On the other hand, in the read control circuit 10, the speaker 13 is irradiated with infrared light by the infrared light source 12, and when the liquid crystal screen 11 is in the transparent state, not only the speaker 13 is photographed but also the infrared image sensor 8 is used. The infrared ray reflected by the eyes of the hand 13 is received, and the line-of-sight position information for indicating the position of the eyes is input.

Then, the memory read control is performed so that the line-of-sight position information and the address at which the image of the eye of the talker of the other party on the second image memory unit 4 transmitted is read out coincide with each other. To do.

Then, the image input signal from the second image memory unit 4 is sent to the liquid crystal video projector 5, and the liquid crystal video projector 5 projects the image of the other party on the liquid crystal screen 11. As mentioned above, the liquid crystal screen 11
Since the speed of changing to the transparent state is fast for humans, the image of the other party is always displayed on the liquid crystal screen 11 when seen from the speaker 13.

That is, as shown in FIG. 3, when A is the transmission side and B is the communication partner, the first image memory section 3 stores and transmits an image in which the sight line of A is aligned with the optical axis, An image in which the line of sight of B is aligned with the optical axis is transmitted and stored in the second image memory unit 4. Then, in the image of B, the line of sight is made to coincide with the line of sight of A so that the image looks like a dotted line.
On the receiving side of B, the image of A is set to be a dotted line so as to match the line of sight of B.

With the above operation, an image interactive apparatus in which an image corrected so that the positions of the eyes of the speaker 13 and the eyes of the speaker of the other party are displayed on both liquid crystal screens 11 is displayed. Can be realized.

Next, another embodiment will be described. FIG. 2 shows another embodiment. In this embodiment, the transmitting side includes a multiplexing circuit 16 that multiplexes the image and the line-of-sight position information, and the receiving side separates the image and the line-of-sight position information by a separating circuit 17. The writing control circuit 18 stores an image in the image memory unit 19.

The operation will be described below. The arrangement and operation of the speaker 13, the liquid crystal screen 11, the video camera 7, and the infrared image sensor 12 are the same as those in the embodiment described with reference to FIG. Further, in response to the control signal sent from the video camera 7, the drive circuit 6 brings the liquid crystal screen 11 into a transparent state, and at that time, the speaker 13
It is also the same that the image of is captured and sent as an image output signal.

The speaker 13 is irradiated with infrared rays from the infrared light source 12, and when the liquid crystal screen 11 is in the transparent state, the infrared image sensor 8 detects the position of the eyes of the speaker 13 and uses it as the line-of-sight position information. The same is true for sending.

First, after the line-of-sight position information is multiplexed on the image output signal by the multiplexing circuit 16, the transmitter 1 puts this on the communication line 14 as communication data.

On the other hand, the receiving unit 2 which receives the communication data of the communication partner from the communication line 14 sends this to the separation circuit 17, and the separation circuit 17 separates the image signal and the line-of-sight position information of the communication partner from the communication data. To do.

The image signal is written in the image memory unit 19, and the writing control circuit 18 is provided with the above-mentioned speaker 1.
The line-of-sight position information indicating the position of the third eye is also input, and based on this and the separated line-of-sight position information, the eye of the speaker of the other party is set to the address corresponding to the position of the eye of the speaker 13. The writing timing is controlled so that the positional relationship is such that the image is written.

Further, the image input signal sent from the image memory section 19 is inputted to the liquid crystal video projector 5,
The image of the other party is projected on the liquid crystal screen 11.

As a result of the above operation, the images of the talker of the other party corrected so as to match the eye positions of the talker 13 are displayed on both liquid crystal screens 11. It is possible to realize an image interactive apparatus in which the position of and the position of the eye of the other party always match.

In the above embodiments, the positions of the eyes of the speaker are detected by using the infrared light source and the infrared image sensor. However, the face is specified from the photographed image of the speaker and the characteristic points of each part of the face are identified. It is also possible to apply an image recognition technique of extracting and detecting the eye position from the extracted image.

[0036]

As described above, according to the present invention, in the image dialogue apparatus, it becomes possible to perform dialogue in a state in which both communicating parties face each other and eye positions always match.
A more realistic dialogue can be realized.

Further, there is no need to always be aware of the position of the video camera, and even if the head is slightly moved, the position of the eyes can be automatically followed and adjusted, so that it is more relaxed. The effect of being able to have a dialogue can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a block diagram showing another embodiment of the present invention.

FIG. 3 is a diagram for explaining the operation of the present invention.

FIG. 4 is a diagram for explaining detection of line-of-sight position information according to the present invention.

FIG. 5 is a block diagram showing a conventional technique.

[Explanation of symbols]

 1 Transmitter 2 Receiver 3 First Image Memory 4 Fourth Image Memory 5 Liquid Crystal Video Projector 6 Drive Circuit 7 Video Camera 8 Infrared Image Sensor 9 Write Control Circuit 10 Readout Control Circuit 11 Liquid Crystal Screen 12 Infrared Light Source 13 Speaker 14 Sender communication line 15 Receiver communication line

Claims (3)

[Claims] 1. An image pickup device transmits an image and a voice of a person on the transmitting side to a communication partner through a communication line, while an image and a voice of a person of a communication partner are transmitted through a communication line. In an image interactive device that receives and displays an image on a display device and communicates with each other, a line-of-sight position detection unit that detects the line-of-sight position of a person who is transmitting the communication, and a line-of-sight position obtained by the line-of-sight position detection unit. Transmission means for transmitting information through the communication line together with the image, a receiving means for receiving the image of the other party communicating through the communication line and the line-of-sight position information thereof, and the line-of-sight position information of the other party obtained by the receiving means. An image composed of a line-of-sight position correcting unit for adjusting the line-of-sight position of the person image displayed on the display device from the line-of-sight position information of the communication sending side obtained from the line-of-sight position detecting unit versus Apparatus. 2. An image pickup device transmits an image and a voice of a person on the transmission side to a communication partner through a communication line, while an image and a voice of a person of a communication partner are transmitted through a communication line. In an image interactive device that receives and displays an image on a display device and communicates with each other, a line-of-sight position detection unit that detects the line-of-sight position of a person who sends communication, and a position detection obtained from the line-of-sight position detection unit. According to the information, the image of the person sending the communication is corrected so that the line-of-sight position is adjusted to a certain display position, converted into a digital image signal, and transmitted through the communication line, and communication is performed through the communication line. Receiving means for receiving the digital image signal of the other party and its line-of-sight position information, and communicating the line-of-sight position of the person image displayed on the display device with the digital image signal of the other party obtained by the receiving means from the line-of-sight position information. Image interaction device comprising a line-of-sight position correcting means so as match the that side of the line-of-sight position. 3. An image pickup device transmits an image and a voice of a person on the transmitting side to a communication partner through a communication line, while an image and a voice of a person of a communication partner are transmitted through a communication line. In an image interactive device that receives and displays an image on a display device and communicates with each other, a line-of-sight position detection unit that detects the line-of-sight position of a person who is transmitting the communication, and a line-of-sight position obtained by the line-of-sight position detection unit. Transmission means for transmitting information together with the image through the communication line, receiving means for receiving the image of the other party communicating through the communication line and the eye gaze position information, and separating the image of the other party and the eye gaze position information Means, a storage means for storing the image of the other party as a digital image, and a person who sends the image of the other party from the line-of-sight position information of the other party and the line-of-sight position information obtained from the line-of-sight position detecting means. An image interactive apparatus comprising: a writing control unit that controls writing to the storage unit so that the line of sight of the object is displayed on the display unit.