JP2009081881A - Image processing apparatus and method, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a videoconference of enhanced presence by making the eyes of the participants mutually meet. <P>SOLUTION: A movement quantity detection device 51 is attached on the head or the like of each participant 24, and detects the direction of the movement of the face of the participant 24 and the quantity of the movement thereof. A computation device 52 outputs, to a transmission/reception device 53, the image portion in which the face of the participant 24 exists at the center thereof out of the image of the participant 24 imaged by a front video camera 31-21. The outputted image is determined based on the quantity of the movement detected by the movement quantity detection device 51. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, and a recording medium, and more particularly, to an image processing apparatus and method, and a recording medium that enable a video conference or the like to be performed more realistically.

  There has been proposed a remote conference system in which a plurality of conference rooms are connected using a network and a conference is held as if surrounding a single table. Each conference room in such a system is equipped with a display that displays a video of a participant other than myself (referred to as participant A) and a speaker that outputs the speech of the participant displayed on the display. . The same number of displays and speakers as the number of participants excluding participant A are installed.

Each conference room is also equipped with a video camera for capturing the participants in the conference room and a microphone for capturing audio. The video camera and microphone are installed in the vicinity (mainly at the top) of the display provided in each conference room. Participant A's video imaged by the video camera and audio captured by the microphone are output to a display and a speaker corresponding to the participant A provided at each venue. (For example, see Patent Documents 1 to 6)
JP-A-10-51755 Japanese Patent Laid-Open No. 4-249990 JP-A-10-257475 JP-A-6-343174 JP-A-10-153651 JP-A-5-205030

  However, in the above-described apparatus, the optical axis of the video camera and the line of sight of the participant A looking at the display are misaligned. In other words, the participant shown on the display and the participant A watching the participant Communication may become unnatural because the line of sight does not match.

  As one method for solving such a problem, a method has been proposed in which the optical axis of the video camera and the line of sight of the participant A viewing the display are matched (gaze matching) using a half mirror. In this method, it is assumed that Participant A sits in a predetermined position and moves his face so as not to come out of the predetermined range. Therefore, Participant A's face appears from the predetermined range. If this happens, there is a problem that the line of sight cannot be matched after all.

  The present invention has been made in view of such a situation, and detects the movement amount of a participant's face and drives an imaging device or processes a captured image according to the detected movement amount. By doing so, the line of sight is matched, thereby providing a more realistic video conference and the like.

  An image processing apparatus according to an aspect of the present invention detects an amount of movement of an object and an imaging unit that captures an image of a subject in an image processing apparatus that exchanges image data with another image processing apparatus via a network. Detection means; drive means for driving the imaging means so that the subject is positioned at the center of the image taken by the imaging means in accordance with the amount of movement detected by the detection means; and imaging by the imaging means Transmitting and receiving the image data to the other image processing apparatus, receiving and transmitting image data transmitted from the other image processing apparatus, and an image corresponding to the image data received by the transmission and reception means Display means for displaying.

  When the imaging unit and the display unit are not arranged at an optically equivalent position with respect to the subject, the driving unit has a shift between the line-of-sight direction of the subject and the optical axis of the imaging unit. The imaging means is driven so that the angle is 4.5 degrees in the left-right direction, 12 degrees in the upward direction, and within 8 degrees in the downward direction.

  An image processing method according to an aspect of the present invention is an image processing method of an image processing apparatus that exchanges image data with another image processing apparatus via a network, an imaging step of capturing an image of a subject, and movement of the subject A detection step for detecting an amount; a driving step for driving the imaging step so that the subject is positioned at the center of the image captured in the imaging step according to the movement amount detected in the detection step; The image data captured in the imaging step is transmitted to the other image processing apparatus, the transmission / reception step for receiving the image data transmitted from the other image processing apparatus, and the image data received in the transmission / reception step. Displaying a corresponding image.

  The recording medium according to one aspect of the present invention includes an imaging step of capturing an image of a subject in an image processing device that exchanges image data with another image processing device via a network, and detection for detecting the amount of movement of the subject. A step of driving the imaging step so that the subject is positioned at the center of the image captured in the imaging step according to the amount of movement detected in the detection step; Transmitting the received image data to the other image processing apparatus, receiving and transmitting the image data transmitted from the other image processing apparatus, and displaying an image corresponding to the image data received in the transmission / reception step A computer-readable program that executes a process including a display step is recorded.

  In the image processing apparatus and method and the recording medium according to one aspect of the present invention, an image of a subject is captured, the amount of movement of the subject is detected, and the subject is located at the center of the captured image according to the detected amount of movement. The imaging device is driven so that there is

 According to one aspect of the present invention, it is possible to match the lines of sight of participants in a video conference or the like, and thus it is possible to provide a conference with a sense of presence.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows the configuration of a video conference system to which the image processing apparatus of the present invention is applied. In the present specification, the system represents an overall apparatus composed of a plurality of apparatuses. As shown in FIG. 1, a plurality (four in this embodiment) of communication centers 1-1 to 1-4 are mutually connected via a network 2 such as an ISDN (Integrated Service Digital Network). It is connected. Each communication center includes one conference room as shown in FIG. 2, for example.

  In the conference room shown in FIG. 2, one table 10, one chair, and three display devices are provided. For example, in the conference room of the communication center 1-4, a chair is arranged at the position of the number 4 shown in FIG. 2, and a display device is arranged at the positions of the numbers 1 to 3. Further, in the conference room of the communication center 1-3, a chair is arranged at the position of the number 3 shown in FIG. 2, and a display device is arranged at the positions of the numbers 1, 2, and 4. Further, in the conference room of the communication center 1-2, a chair is arranged at the position of the number 2 shown in FIG. 2, and a display device is arranged at the positions of the numbers 1, 3, and 4. Further, in the conference room of the communication center 1-1, a chair is disposed at the position of number 1 shown in FIG. 2, and a display device is disposed at the positions of numbers 2 to 4.

  Further, in the display device arranged in the conference room of the communication center 1-4, an image obtained by imaging the participants of the communication center 1-1 is displayed on the display device arranged at the position of the number 1 shown in FIG. Then, the display device arranged at the number 2 position displays an image of the participants of the communication center 1-2, and the display device arranged at the number 3 position displays the communication center 1-3. An image obtained by imaging the participant is displayed. Similarly, in other communication centers, a display device arranged at a position other than the chair on which the participant sits displays an image obtained by imaging the corresponding communication center participant.

  As described above, the conference room of each communication center is provided with a chair for a participant to sit at a specific position of the communication center, and the others are display devices that display participants of other communication centers. Will be placed. Therefore, by configuring the conference room in this way, the participants are placed at the same position in the conference room of any communication center. That is, it is as if the four participants are actually placed at specific positions with the table 10 as the center. However, in the conference rooms of each communication center, all participants other than the actual participants themselves are participants by display display, but the same conference state is realized in any conference room.

  Next, details of each communication center will be described with reference to FIG. Each communication center has almost the same configuration although the arrangement of display devices is somewhat different. Therefore, only the communication center 1-4 will be described here, and the other communication centers 1-1 to 1-3 will be described. Description is omitted.

  First, in the conference room of the communication center 1-4, as shown in FIG. 2, a chair is arranged at the position of the number 4, and a display device is arranged at the positions of the numbers 1 to 3, respectively. Therefore, the participant 24 shown in FIG. 3 is seated in the chair arranged at the position of the number 4 in FIG. In addition, in each display device 21 to 23, in order to capture the video of the participant 24, the left side video camera 31-22 installed on the left side of the participant 24 and the front side installed in front of the participant 24. A video camera 31-21 and a right side video camera 31-23 installed on the right side of the participant 24 are provided. Furthermore, microphones 32-21 to 32-23 (hereinafter referred to as microphones 32-21 to 32-23) that capture the remarks of the participant 24 are simply referred to as microphones 32. In the same manner, the speaker units 33-21 to 33-23 that output audio supplied from other communication centers and the display units 34-21 to 34- that display images corresponding to the audio are also described. 23 is provided.

  The speaker unit 33 and the display unit 34 are configured to output images transmitted from the communication centers 1-1 to 1-3 and audio corresponding to the images, respectively. That is, for example, an image of a participant of the communication center 1-1 is displayed on the display unit 34-21 of the display device 21, and a speech of the participant is output from the speaker unit 33-21. ing. Further, an image of the participant of the communication center 1-2 is displayed on the display unit 34-22 of the display device 22, and a speech of the participant is output from the speaker unit 33-22. . Furthermore, an image of the participant of the communication center 1-3 is displayed on the display unit 34-23 of the display device 23, and the speech of the participant is output from the speaker unit 33-23. .

  Further, the front video camera 31-21 arranged in the display device 21 images the participant 24 of the communication center 1-4, and the microphone 32-21 captures the speech of the participant 24, and the participant 24 The images and remarks are supplied to the communication center 1-1. The left side video camera 31-22 installed in the display device 22 images the participant 24 of the communication center 1-4, and the microphone 32-22 captures the participant's remarks, and the participant 24 The images and utterances are supplied to the communication center 1-2. Furthermore, the video camera 31-23 installed in the display device 23 images the participant 24 of the communication center 1-4, and the microphone 32-23 captures the participant's remarks, and the image of the participant 24 is captured. Is sent to the communication center 1-3.

  And as shown in FIG. 3, the display devices 21 to 23 are shown in FIG. 2 so that the participants 24 can see the display units 34-21 to 34-23 of the display devices 21 to 23. It is arranged at a predetermined position.

  In the video conference system in which each conference room is configured in this way and connected by each network, it is as if four participants are actually having a meeting in the same room. Can hold a meeting.

  By the way, when the participant 24 talks with, for example, the participant of the communication center 1-1 among the other communication centers 1-1 to 1-3, the participant 24 faces the display device 21 in FIG. talk. That is, in this case, the participant 24 speaks while watching the display screen of the display unit 34-21 on which the video of the participant of the communication center 1-1 is displayed as described above.

  In this case, the video of the participant 24 captured by the front video camera 31-21 of the display device 21 is transmitted to the participants of the communication center 1-1. The front video camera 31-21 is shown in FIG. In Embodiment, it arrange | positions at the upper part of the display part 34-21 on which the image | video of the participant of the communication center 1-1 is displayed.

  Therefore, the video provided to the participant of the communication center 1-1 is not like the participant 24 watching himself (here, the participant of the communication center 1-1), and is somewhat lower. It looks like it is suitable. As a result, the participants of the communication center 1-1 may not see the line of sight of the participants 24 as if they are facing their own direction, that is, the lines of sight do not match and the sense of reality may be impaired.

  Although it is conceivable that the screen of the display unit is made smaller so that the line of sight is within the detection limit range, this reduces the size of the displayed participant's video, which also impairs the sense of reality.

  Moreover, if the front video camera 31-21 is arranged in front of the screen of the display unit 34-21, the video of the participant 24 whose line of sight is facing is provided to the participant of the communication center 1-1. However, in this case, the participant 24 is prevented from seeing the video of the participant of the communication center 1-1 displayed on the display unit 34-21.

  Therefore, the display device 21 (the same applies to other display devices) can be configured as shown in FIG. 4, for example. In the following, description of the part related to sound will be omitted.

  FIG. 4 shows the display device 21 as viewed from above the participant 24. In this embodiment, for example, at least a part of light such as a half mirror or PBS (polarized beam splitter) is used. An optical element 41-21, which is an optical system that transmits and reflects the remaining light, is newly provided. Here, a flat half mirror is used as the optical element 41-21. Then, the image displayed by the display unit 34-21 is transmitted to the optical element 41-21 and provided to the participant 24, and the light from the participant 24 is reflected by the optical element 41-21. A display unit 34-21, a front video camera 31-21, and an optical element 41-21 are arranged so as to enter the front video camera 31-21.

  That is, in the embodiment of FIG. 4, it is a straight line in the normal direction of the display screen of the display unit 34-21 and approximately the center of the display screen (for example, assuming that the display screen is a rectangle, the rectangle And the optical axis of the front video camera 31-21 are included in the same plane, and the optical axis of the front video camera 31-21 is included in the same plane. A display unit 34-21 and a front video camera 31-21 are arranged so as to be orthogonal to each other.

  The optical element 41-21 has a reflecting surface perpendicular to a plane including both the optical axis of the display unit 34-21 and the optical axis of the front video camera 31-21, and the optical axis or front surface of the display unit 34-21. The video camera 31-21 is arranged so that the angle formed between the optical axes of the video camera 31-21 is approximately 45 degrees. Therefore, according to the optical element 41-21, the optical axis of the front video camera 31-21 and the optical axis of the display unit 34-21 are equivalently matched.

  As a result, in the display unit 34-21, for example, as shown in FIG. 5, when an image of the participant of the communication center 1-1 is displayed at the center of the display screen, the image is displayed on the optical element 41. -21 is transmitted and enters the eyeball of the participant 24, whereby the participant 24 observes an image of the participant in the communication center 1-1. When the participant 24 talks to the participant of the communication center 1-1 and looks at the direction of the image, the external light (illumination light, etc.) reflected by the participant 24 is the optical element 41-. 21 is reflected by 90 degrees and enters the front video camera 31-21, whereby the participant 24 is imaged. That is, in the front video camera 31-21 that is equivalently arranged in the center of the screen of the display unit 34-21 on which the video of the participant of the communication center 1-1 is displayed, the participation of the communication center 1-1. Participant 24 watching the person's video is imaged.

  Accordingly, the video provided to the participants of the communication center 1-1 is as if the participant 24 is looking at the participants of the communication center 1-1. It can be prevented from being damaged.

  By the way, for example, in the conference room of the communication center 1-4, when the participant 24 sits on a chair placed at the position indicated by the number 4 in FIG. It is difficult to keep. That is, the participant 24 of the communication center 1-4 (the same applies to participants of other communication centers) may move during the video conference.

  Specifically, for example, as shown in FIG. 6, the participant 24 captures the face of the participant 24 at the original position (here, for example, approximately in the center of the image frame of the front video camera 31-21. The front video camera 31-21 images the face of the participant 24 at the left position of the image frame. As a result, on the display unit of the communication center 1-1, for example, the video of the participant 24 is displayed at the right end as shown in FIG. In this case, since the participant of the communication center 1-1 talks while watching the video of the participant 24, the line of sight is directed to the right end direction rather than substantially the center of the display unit of the communication center 1-1. In the video camera of the communication center 1-1, since the participant of the communication center 1-1 facing the right direction in this way is imaged through the optical element, the display unit 34- of the communication center 1-4. The video of the participant of the communication center 1-1 displayed on the screen 21 is also directed rightward.

  Since the display unit, the video camera, and the optical element are described above, the participant is arranged so that the line of sight is a so-called camera line of sight when the participant points his / her line of sight to the approximate center of the display unit. When a participant of the communication center 1-1 talks in the left direction, the video of the participant of the communication center 1-1 displayed on the display unit 34-21 of the communication center 1-4 faces so to speak. It will be like that. Therefore, in this case, the line of sight does not match and the sense of reality is impaired.

  Therefore, the display device 21 (the same applies to other display devices) can be configured as shown in FIG. 8, for example. In this configuration, a movement amount detection device 51 is provided above the participant 24. The movement amount detection device 51 is composed of, for example, a magnetic sensor, a gyroscope, an angular velocity sensor, and the like. Data output from the movement amount detection device 51 is input to the arithmetic device 52. Image data of the participant 24 captured by the front video camera 31-21 is also input to the arithmetic device 52.

  The arithmetic device 52 processes the image data input from the front video camera 31-21 according to the data input from the movement amount detection device 51, and outputs it to the transmission / reception device 53. The transmission / reception device 53 is connected to the network 2 (FIG. 1), encodes the data input from the arithmetic device 52 and transmits the encoded data to the communication center 1-1 and is transmitted from the communication center 1-1. Receive and decrypt data.

  The operation of the arithmetic device 52 will be described with reference to the flowchart of FIG. In step S <b> 1, the arithmetic device 52 performs initial setting of the movement amount detection device 51. In this initial setting, the participant 24 sits in front of the display device 21, watches the display device 21, and the position of the head at that time (the position where the movement amount detection device 51 exists) is calculated by the computing device 52. The initial value is set to 0. The amount of movement from the set initial value position is detected by the movement amount detection device 51 in step S2.

  Here, the movement amount detected in step S2 is the movement amount when moving in parallel with the display unit 34-21. In step S3, the arithmetic unit 52 corrects the image output from the front video camera 31-21 according to the detected movement amount.

  FIG. 10 is a diagram illustrating the image correction performed in step S3. The front video camera 31-21 captures an image larger than an image transmitted to the communication center 1-1. That is, the image captured by the front video camera 31-21 is an image in the imaging frame 61, and the image in the transmission frame 62 is transmitted to the communication center 1-1. The transmission frame 62 is a frame that is set smaller than the imaging frame 61 and is a frame that moves within the imaging frame 61.

  As shown in FIG. 10A, the range in which the participant 24 is imaged in the imaging frame 61 is set as the transmission frame 62, but this transmission is performed at the initial setting in step S1. The position of the frame 62 within the imaging frame 61 is also set. For example, at the time of initialization, when the transmission frame 62 is positioned in a state as shown in FIG. 10A, the position is set as a reference point, and as shown in FIG. If it moves to the right side in the figure, the movement amount d is detected by the movement amount detection device 51.

  The arithmetic device 52 calculates the movement amount d ′ of the participant 24 in the imaging frame 61 corresponding to the detected movement amount d, and moves the transmission frame 62 by the calculated movement amount d ′. In this way, the face of the participant 24 can always be positioned in the central portion of the transmitted image.

  As described above, the image corrected by the arithmetic device 52 in step S3 (the image in the transmission frame 62) is output to the transmission / reception device 53 in step S4. The transmission / reception device 53 transmits the input image to the communication center 1-1.

  Steps S2 to S4 are repeated during the meeting. As a result, the image of the participant 24 displayed on the display device installed in the communication center 1-1 is always located near the center, so that the line of sight can be matched and there is a sense of presence. A video conference system can be provided.

  As described above, the image is corrected based on the movement amount obtained by the movement amount detection device 51. However, by providing the imaging frame 61 and the transmission frame 62, the image captured by the front video camera 31-21 is obtained. It is possible to determine the amount of movement and correct the image.

  In the display device 21 shown in FIG. 11, an image captured by the front video camera 31-21 is supplied to the transmission / reception device 53 via the arithmetic device 52. As described with reference to FIG. 10, the front video camera 31-21 has the imaging frame 61 and captures an image within the frame. The computing device 52 sets a transmission frame 62 and outputs an image within the frame to the transmission / reception device 53.

  The operation of the arithmetic device 52 will be described with reference to the flowchart of FIG. First, in step S11, initialization is performed. At the initial setting stage, the transmission frame 62 is set at the center of the imaging frame 61, and the image of the participant 24 captured by the front video camera 31-21 when the participant 24 sits at a predetermined position. Is imaged in the transmission frame 62. The image of the participant 24 imaged at this time is used as a reference image and stored in the arithmetic device 52.

  In step S12, a region having a high degree of correlation is detected in order to determine an image to be transmitted. This detection will be described with reference to FIG. The reference image captured as the initial setting in step S11 is the image in the transmission frame 62 in FIG. 10A, and the image captured at a predetermined time is the image in the imaging frame 61 in FIG. 10B. In this case, a portion having a high degree of correlation with the image (reference image) in the transmission frame 62 in the state of FIG. 10A is detected from the image in the imaging frame 61 shown in FIG. What is detected as an image having a high degree of correlation is a portion of the image obtained by capturing the participant 24, and the transmission frame 62 in FIG. 10B is set so as to include that portion. The image existing in the transmission frame 62 set in this way is output to the transmission / reception device 53 in step S13.

  Steps S12 and S13 are repeated each time an image is captured by the front video camera 31-21. Alternatively, it may be repeated every time a predetermined number of images are captured.

  In the above-described embodiment, the image of the participant 24 displayed on the display unit of the communication center 1-1 is displayed at the center of the screen by processing the captured image. The same effect can be obtained by making the front video camera 31-21 movable. The configuration of such an apparatus is shown in FIG. In the configuration shown in FIG. 13, the data output from the movement amount detection device 51 is input to the arithmetic device 52. The arithmetic device 52 calculates the amount by which the front video camera 31-21 is moved based on the input data, and outputs the calculated data to the drive device 71.

  The drive device 71 translates (or pans) the front video camera 31-21 in a horizontal plane according to the input data. That is, it is possible to cope with the case where the participant 24 moves in parallel with respect to the display unit 31-21. The front video camera 31-21 outputs the captured image to the transmission / reception device 53, and the transmission / reception device 53 transmits the input image data to the corresponding communication centers 1-1 to 1-3. Yes.

  The operation of the apparatus shown in FIG. 13 will be described with reference to the flowchart of FIG. Since the initial setting in step S21 is the same as the initial setting performed in step S1 of FIG. 9, the description thereof is omitted. However, since the imaging frame 61 and the transmission frame 62 are not set in the front video camera 31-21 and the arithmetic device 52 shown in FIG. 13, these initial settings are not performed.

  In step S22, the movement amount is calculated. This calculation is performed by how much the front video camera 31-21 is moved according to the movement amount of the participant 24 detected by the movement amount detection device 51. As described above, the front video camera 31 is moved by the optical element 41-21. This is a process of calculating whether the optical axis of −21 and the optical axis of the display unit 34-21 are equivalently matched. For example, when the participant 24 moves to the left side in FIG. 13 (when the participant 24 is in the state shown in FIG. 6), the computing device 52 moves the front video camera 31-21 to the lower side in FIG. Calculate whether to move to.

  In this way, the movement amount calculated by the arithmetic device 52 is output to the drive device 71. In step S23, the driving device 71 drives the front video camera 31-21 in accordance with the input movement amount data. In step S24, the front video camera 31-21 outputs the captured image to the transmission / reception device 53, and the transmission / reception device 53 transmits the input image data to the communication center 1-1.

  Thus, by moving the front video camera 31-21 in accordance with the amount of movement of the participant 24, it is possible to realize line-of-sight matching, and thus to provide a realistic video conference system. It becomes.

  In the above-described embodiment, the optical elements 41-21 are used so that the optical axes of the display unit 34-21 and the front video camera 31-21 are matched. However, for example, as shown in FIG. 3, when the front video camera 31-21 is fixed on the display device 21 and the optical axis thereof does not coincide with the display unit 34-21, the front video camera 31 is used. The line of sight can be matched by driving -21 three-dimensionally.

  That is, as shown in FIG. 15, the front video camera 31-21 is installed above the display device 21 so as to be movable up and down, left and right, and front and rear. The drive device 71 controls the movement of the front video camera 31-21, and the drive device 71 is supplied with the amount of movement calculated by the calculation device 52. The computing device 52 is supplied with the movement amount of the participant 24 detected by the movement amount detection device 51 attached above the participant 24. In addition, the image of the participant 24 captured by the front video camera 31-21 is output to the transmission / reception device 53.

  As shown in FIG. 15, the participant 24 looks at the participant shown on the display unit 34-21 of the display device 21, and the front video camera 31-21 When it is located at the near side, the angle formed by the participant 24's line of sight and the front video camera 31-21 is θ1, and similarly, the front video camera 31-21 is shown on the right side (to the participant 24). When the angle formed between the eyes of the participant 24 and the front video camera 31-21 is θ2, the angle θ1 is greater than the angle θ2. Using this, the front video camera 31-21 is driven so that the angle θ formed by the participant 24's line of sight and the front video camera 31-21 becomes small.

  Here, for example, the Journal of Television Society, vol. 42, no. According to “Human Communication” described on pages 698 to 702 of 6 1992, the human detection limit is 4.5 degrees in the horizontal direction, 12 degrees in the upward direction, and 8 degrees in the downward direction. According to this principle, if the line of sight deviates beyond this range, the sense of reality is impaired.

  Therefore, in this example, the front video camera 31-21 is in the front-rear direction with respect to the participant 24 so that the angle θ is not 12 degrees in the upward direction and is within 8 degrees in the downward direction. Or it is driven in the vertical direction. Similarly, regarding the parallel movement of the participant 24 with respect to the display device 21, the front video camera 31-21 is driven in the left-right direction with respect to the participant 24 so that the deviation of the line of sight is within 4.5 degrees. The

  The operation of the apparatus in such a configuration is the same as that described with reference to the flowchart of FIG. However, the driving of the front video camera 31-21 performed in step S23 is performed in the vertical direction, the horizontal direction, and the front-back direction.

  In the above-described embodiment, the processing is performed so that the line of sight matches before transmitting the image data to another communication center. However, the image data may be processed on the receiving side.

  FIG. 16 is a diagram illustrating a configuration of the display device 21 that performs processing for performing line-of-sight matching from received data. In this configuration, the movement amount detected by the movement amount detection device 51 is output to the arithmetic device 52. The amount of movement input to the arithmetic device 52 is converted into data used when processing is performed on the image received on the receiving side and output to the transmitting / receiving device 53. The transmission / reception device 53 transmits the data output from the arithmetic device 52 and the image data output from the front video camera 31-21.

  The transmission / reception device 53 receives image data of participants in other communication centers and data related to the amount of movement of the participants when an image of the image data is captured. These received data are output to the arithmetic unit 52. The arithmetic device 52 performs processing on the input image data based on the input movement amount data. This process is performed so as to output an image in a frame corresponding to the transmission frame 62, for example, as described with reference to FIG. By doing in this way, the face of the participant 24 projected on the display unit installed in each communication center can always be projected at the center of the screen.

  In the above description, the present invention is applied to a video conference system. However, the present invention can also be applied to other devices. In the above description, the number of communication centers is four. However, the present invention is not limited to this, and the present invention can also be applied to cases where the number of communication centers is increased or decreased.

  In this specification, the medium for providing a computer program for executing the above processing to the user includes not only an information recording medium such as a magnetic disk and a CD-ROM, but also a transmission medium via a network such as the Internet and a digital satellite. .

It is a figure which shows the structural example of the video conference system to which the image processing apparatus of this invention is applied. It is a figure which shows the state of the meeting room of each communication center in a video conference system. It is a figure which shows the arrangement | positioning state of the display apparatus in the communication center in FIG. It is a figure which shows the structural example of a display apparatus. It is a figure which shows the example of a display of the display part of FIG. It is a figure explaining a gaze coincidence shift by a participant's position shifting. It is a figure which shows the example of a display of a display part when a line-of-sight agreement has shifted | deviated. It is a figure which shows the other structural example of a display apparatus. It is a flowchart explaining operation | movement of the arithmetic unit of FIG. It is a figure explaining the correction | amendment of the image performed by step S3. It is a figure which shows the further another structural example of a display apparatus. It is a flowchart explaining operation | movement of the arithmetic unit of FIG. It is a block diagram which shows the further another structural example of a display apparatus. It is a flowchart explaining operation | movement of the arithmetic unit of FIG. It is a figure which shows the further another structural example of a display apparatus. It is a figure which shows the further another structural example of a display apparatus.

Explanation of symbols

  1-1 to 1-6 Communication Center, 2 Networks, 21-23 Display Device, 35-21 Front Video Camera, 35-22 Left Side Video Camera, 35-23 Right Side Video Camera, 36-21 to 36-23 Microphone , 37-21 to 37-23 display unit, 38-21 to 38-23 speaker unit, 41-21 optical element, 51 movement amount detection device, 52 arithmetic unit, 53 transmission / reception device, 61 imaging frame, 62 transmission frame, 71 Drive device

Claims (4)

In an image processing apparatus that exchanges image data with other image processing apparatuses via a network,
Imaging means for capturing an image of a subject;
Detecting means for detecting the amount of movement of the subject;
Driving means for driving the imaging means so that the subject is positioned at the center of the image taken by the imaging means in accordance with the amount of movement detected by the detection means;
Transmitting / receiving means for transmitting image data captured by the imaging means to the other image processing apparatus and receiving image data transmitted from the other image processing apparatus;
An image processing apparatus comprising: display means for displaying an image corresponding to the image data received by the transmission / reception means. When the imaging unit and the display unit are not arranged at an optically equivalent position with respect to the subject, the driving unit has a shift between the line-of-sight direction of the subject and the optical axis of the imaging unit. The image processing apparatus according to claim 1, wherein the imaging unit is driven so that the horizontal direction is 4.5 degrees, the upward direction is 12 degrees, and the downward direction is within 8 degrees. In an image processing method of an image processing apparatus that exchanges image data with other image processing apparatuses via a network,
An imaging step for capturing an image of a subject;
A detecting step for detecting a movement amount of the subject;
A driving step for driving the imaging step so that the subject is positioned at the center of the image captured in the imaging step according to the movement amount detected in the detection step;
A transmission / reception step of transmitting the image data captured in the imaging step to the other image processing device and receiving the image data transmitted from the other image processing device;
A display step of displaying an image corresponding to the image data received in the transmission / reception step. To an image processing apparatus that exchanges image data with other image processing apparatuses via a network,
An imaging step for capturing an image of a subject;
A detecting step for detecting a movement amount of the subject;
A driving step for driving the imaging step so that the subject is positioned at the center of the image captured in the imaging step according to the movement amount detected in the detection step;
A transmission / reception step of transmitting the image data captured in the imaging step to the other image processing device and receiving the image data transmitted from the other image processing device;
A recording medium on which is recorded a computer-readable program that executes processing including a display step of displaying an image corresponding to the image data received in the transmission / reception step.

Images