JPH07162532A - Multipoint communication conference Communication support device - Google Patents

Abstract

(57) [Abstract] [Purpose] A conference without increasing the scale of the terminal device, reducing the degree of freedom of the installation position of the terminal device, and without increasing the amount of transmission data due to the connection form of the communication network. (EN) Provided is a multipoint communication conference communication support device capable of notifying a conference participant of motion information of each participant's communication. [Composition] A multipoint communication conference communication support device in which video and audio are transmitted and received between a plurality of terminal devices, and a plurality of participants hold a conference, wherein the terminal device inputs operation information of each conference participant. Motion information input means to
A motion information transmitting means for transmitting the motion information to the terminal device at each point, a motion information receiving means for receiving the motion information from the motion information transmitting means, and a motion information received by the motion information receiving means. And notification means.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a conference communication support device in a multipoint communication conference system.

[0002]

2. Description of the Related Art A conventional communication support device for a conference in a multipoint communication conference system is shown in FIG. In FIG. 8, A to D are conference participants, 81 to 84 are communication conference terminal devices (hereinafter, referred to as terminal devices) for executing a communication conference of conference participants A to D, 811, 82.
1, 831 and 841 are video input devices that capture the images of the conference participants A to D, 812, 822, 832 and 842 are video display devices that display the images of the conference participants A to D, 81
3, 823, 833, 843 are voice input devices for obtaining voices of conference participants A to D, 814, 824, 834, 8
Reference numeral 44 is a voice output device that outputs voices from the conference participants A to D.

In practice, the terminal device 8 at each point
1 to 84 are composed of a video / audio encoding / decoding device, a communication network interface, and the like, but they are omitted because they are not particularly necessary in the description of the multipoint communication conference communication support device in the present application.

In FIG. 8, the number of participants in the conference is four, but this may be any number. Further, in FIG. 8, the communication network 80 connecting each point is a mesh type, but this may be any type of connection such as a star type or a ring type.

In FIG. 8, for example, the image of the conference participant A input from the image input device 811 of the terminal device 81 is transmitted to and displayed on the other terminal devices 82 to 84.

Similarly, the images of the conference participants B to D are also input from the image input devices 821 to 841 of the respective terminal devices 82 to 84, and are input to the terminal devices 81 to 84 other than the own point among the terminal devices 81 to 84. Other video display devices 822 to
842 is displayed.

Further, in FIG. 8, for example, the terminal device 8
The image display device 812 of No. 1 shows an example of displaying the images of three conference participants on one display screen by using the multi-window method used in personal computers and workstations. Is divided into a plurality of vertical and horizontal areas, and a screen split display method for displaying video of different conference participants in each area, or one conference participation selected by a user of the terminal devices 81 to 84 using switches or the like. In some cases, a selection display method or the like for displaying only the image of the person is used.

In FIG. 8, for example, the voice of the conference participant A input from the voice input device 813 of the terminal device 81 is transmitted to the other terminal devices 82 to 84 and output. Similarly, the voices of the conference participants B to D are their own terminal devices 82 to 84.
Are input from the voice input devices 823 to 843, are transmitted to the terminal devices 81 to 84 other than the home point among the terminal devices 81 to 84, and are output to the other voice output devices 814 to 844.

In this way, the conference participants can hold the conference while seeing and hearing the faces / voices of other participants (and themselves).

However, when a conference is held using the communication support device for the conference in the conventional multipoint communication conference system, there are the following problems.

(1) In the case of a normal face-to-face meeting, when a utterance of a conference is approaching the end, the participant of the conference can take over the utterance by turning his / her eyes to the participant who wants to speak next. Can be urged. However, in the situation shown in FIG. 8, when the conference participant A is speaking, the utterance nears the end, and the conference participant A wants the conference participant B to take over the utterance, the conference participant A There was a problem that even if the line of sight of the image of the conference participant B on the image display device 812 was turned to, it was not possible for the conference participant B to know that the line of sight was directed to himself.

(2) Further, in the case of a normal face-to-face meeting, the conference participant who is making a speech determines whether or not the other participant is paying attention to his or her speech and the face orientation of the other participant. , Can be read from the direction of the line of sight, facial expression, etc. However, in the situation of FIG. 8, even if the conference participant A is talking and the conference participant A wants to know the reaction of the conference participant B, from the image of the conference participant B on the image display device 812, , It is not possible to know the direction of the face and the line of sight of the conference participant B. Moreover, although the facial expression can be read, the conference participant B does not know the line-of-sight direction and cannot know whether the conference participant B is paying attention to (the image of) the conference participant A. Participant A had a problem that he did not know whether the facial expression of conference participant B at that time was a reaction to his (conference participant A).

(3) Further, in the case of a normal face-to-face meeting, when a conference participant wants to get the attention of another conference participant (attracting the attention of the conference participant who is speaking and interrupting the speech) If you try), you can get the attention of the opponent by gestures and eye-gaze for that particular opponent. However, in the situation of FIG. 8, for example, even if the conference participant B makes some action to get the attention of the conference participant A, who is the speaker, Since it is not possible to determine who (the image of) is paying attention, the conference participant A knows whether the behavior of the conference participant B at that time is directed to himself (the conference participant A). There was a problem that I could not do it.

Conventionally, as a technique for solving such a problem, a plurality of video input devices and a plurality of video display devices are used for each terminal device so that the face and line of sight of the conference participants are different from each other. There is a method that can be identified.

This method will be described below with reference to Fig. 9 (Reference 1: Sellen A. Speech patterns in v).
ideo-mediated conversations.Proceeding of CHI'9
2. pp.49-59).

FIG. 9 is a diagram showing an example of a multipoint communication conference communication support device provided with a plurality of video input devices and a plurality of video display devices for each terminal device.

In FIG. 9, A to D are conference participants, 91 to 94 are terminal devices of conference participants A to D, and 9111 to 9113 are video input devices of the terminal device 91 of conference participant A. , 9121 to 9123 are video display devices of the terminal device 91 of the conference participant A.

In FIG. 9, the video input device 9111 is
The image of the conference participant A seen from diagonally left is captured and transmitted to the terminal device 92 of the conference participant B. The video input device 9112 captures a video viewed from the front of the conference participant A and transmits it to the terminal device 93 of the conference participant C.
The video input device 9113 captures a video image of the conference participant A viewed from the diagonally right front side, and captures the image from the terminal device 9 of the conference participant D.
Send to 4.

Similarly, the terminal device 91 of the conference participant A is
To the conference participant B, the image seen from the diagonally right front of the conference participant B, the image seen from the front of C, and the image seen from the diagonally left front of D are transmitted and displayed on the image display devices 9121 to 9123, respectively.

This configuration, when viewed from the conference participant A, uses a pair of point-to-point communication conference systems, one for each of the other conference participants B to D, when a face-to-face conference is held. At the position where the participants B to D sit, each conference participant B to
As if the terminal devices 92 to 94 that communicate with D are installed,
The video display devices 9121 to 9123 are arranged in the terminal device 91.

Here, the voice input / output device and the like are not shown because they have no particular relation to solving the above-mentioned problems (1) to (3) in communication.

Similar to FIG. 8, a video / audio encoding / decoding device, a communication network interface and the like are not shown in the figure as they are not particularly required for the description of the multipoint communication conference communication support device.

In FIG. 9, for example, the video input device 91
11 connecting the image display device 9223 and the image display device 9223.
Is a communication line, and similarly, the arrows connecting the other devices are also communication lines.

Here, the connection form of the communication network is a mesh (mesh) type, but this is a logical connection, and the physical connection may be a star (star) type or a ring (annular) type. Any connection form may be used.

In the configuration shown in FIG. 9, when the conference participant A is currently speaking, the utterance nears the end and the conference participant A wants the conference participant B to take over the utterance, When the participant A faces his / her face (or line of sight) to the image of the conference participant B on the image display device 9121, the image input device 9111 captures the face (or line of sight) seen from the front of the conference participant A and participates in the conference. It is transmitted to the terminal device 92 of the person B.

Therefore, the conference participant B is assigned to the conference participant A.
Tells you that you are being prompted to take over your speech. This solves the problem (1).

Further, in FIG. 9, when the conference participant A is talking and the conference participant A wants to know the reaction of the conference participant B, the image of the conference participant B on the image display device 9121 is viewed. Thus, it is possible to know from the face direction whether the conference participant B is paying attention to (the image of) the conference participant A. This solves the problem (2).

Similarly, for example, when the conference participant B attempts to get the attention of the conference participant A and performs some gesture,
The conference participant A determines whether or not the gesture is directed to himself (conference participant A) by his own terminal device 92.
It can be known from the face direction of the image of the conference participant B displayed in. This solves the problem (3).

As described above, by using the multipoint communication conference communication support device shown in FIG. 9 having a plurality of video input devices and a plurality of video display devices for each terminal device, the multipoint shown in FIG. (1) to (3), which have been problems in the communication support device
Can solve the communication problems shown in.

[0030]

However, in the multipoint communication conference communication support apparatus shown in FIG. 9, since the plurality of video input apparatuses and the plurality of video display apparatuses are provided, the scale of the terminal apparatus becomes large. There was a problem.

In addition, in the multipoint communication conference communication support device shown in FIG. 9, in order to transmit different images to each connection point, when the communication network is star type or ring type. In addition, there is a problem that the amount of data to be transmitted increases in proportion to the number of ground points.

In addition, in order to transmit different images to each connection ground in this way, there is a problem that each terminal device must have as many video encoding devices as the number of connection grounds (all In the case of transmitting the same video to the connection destination, each terminal device should have one video encoding device).

Also, since the scale of the terminal device becomes large, there is a problem that the number of connection points cannot be increased so much.

Further, in the multipoint communication conference communication support device shown in FIG. 9, since the participants of the communication conference must fit within the angles of view of the plurality of video input devices, the position where the terminal device is installed and the conference There is a problem that the position where the participant sits is limited (the degree of freedom of the installation position of the terminal device is low), and the range where the conference participant can move during the conference becomes narrow. About this problem
Will be explained.

FIG. 10A is a schematic diagram showing the positional relationship between the angle of view of the video input device and the conference participants in the terminal device using one video input device.

In FIG. 10A, 101 is a video input device, 102 is an area within the angle of view of the video input device 101, and 103 is a conference participant.

Conference participant 103 using a terminal device using one video input device 101 shown in FIG.
Is the video input device 1 anywhere in the area 102
01 allows the user to send his / her own video to other communication conference participants.

FIG. 10B is a schematic diagram showing the positional relationship between the angle of view of the video input device and the conference participants in the terminal device using the three video input devices.

In FIG. 10B, 104 to 106 are video input devices, and 107 to 109 are video input devices 10, respectively.
An area within the angle of view of 4 to 106, and 110 is a conference participant.

A conference participant 110 using a terminal device using the three video input devices 104 to 106 shown in FIG. 10B has an overlapping area 111 in which three areas 107 to 109 are overlapped. If it does not exist, you cannot send your video to all other teleconference participants.

That is, the overlapping area 111 in FIG.
Has a problem that it becomes narrower than the area 102 of FIG.

The present invention has been made to solve the above problems, and an object of the present invention is not to increase the scale of the terminal device and to reduce the degree of freedom of the installation position of the terminal device. In addition, a communication support device for a conference in a multipoint communication conference system capable of notifying the conference participants of communication operation information of each conference participant without increasing the amount of transmission data due to the connection form of the communication network. To provide.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[0044]

In order to achieve the above object, the present invention relates to a conference in a multipoint communication conference system in which video and audio are transmitted and received between a plurality of terminal devices and a plurality of participants hold a conference. A communication support device, wherein the terminal device includes motion information input means for inputting motion information of each conference participant, motion information transmitting means for transmitting the motion information to a terminal device at each point, and motion information transmitting means. It is provided with a motion information receiving means for receiving motion information from the means and a notifying means for notifying the motion information received by the motion information receiving means.

[0045]

According to the above means, each terminal device inputs the operation information of each conference participant, transmits the operation information to the terminal device at each point, and the transmitted operation information is sent to the terminal at each point. Since the device receives and informs the received operation information,
Communication information of each conference participant without increasing the scale of the terminal device, reducing the degree of freedom of the installation position of the terminal device, and without increasing the amount of transmission data due to the connection form of the communication network. Can be notified to the conference participants.

[0046]

Embodiments of the present invention will be described in detail below with reference to the drawings.

[Embodiment 1] FIG. 1 is an overall configuration diagram showing a configuration of an embodiment 1 of a multipoint communication conference communication support apparatus to which the present invention is applied.

In FIG. 1, A, B, C and D are conference participants, 11 to 14 are terminal devices, 15 and 16 are communication lines, and 1 is a communication line.
11 is a video input device, 112 is a video display device, 113 is a voice input device, 114 is a voice output device, 115 is a data input device, 116 is a data transmission device, 117 is a data notification device, 1121 to 1123 are image windows, 115
Reference numeral 1 is a pointer, 1152 is a mouse, 1153 is a pointer movement processing unit, 1154 is a pointer display unit, 1155 is a pointer position determination unit, 1161 is a target storage unit, 116.
2 is a data transmission unit, 1171 is a data reception unit, 1172.
Is a multi-window frame display section.

In FIG. 1, the conference communication support apparatus in the multipoint communication conference system according to the first embodiment, as shown in FIG. 1, inputs operation information for inputting attention targets of conference participants A to D. As means, a pointer 1151 and a mouse 115 displayed on the video display device 112 are displayed.
2 (pointing device) and the data input device 11
Using the pointer position determination unit 1155 of No. 5 as a notifying means (notifying means) for notifying the attention relationship between the conference participants based on the transmitted data, using the change of the display frame of the image of the conference participants. Is.

The terminal device 11 inputs a video input device 111 for capturing the video of the conference participant A, a video display device 112 for displaying the video of the other conference participants B to D, and the voice of the conference participant A. Voice input device 113, a voice output device 114 for outputting the voices of the other conference participants B to D, and a data input device 11 for specifying the attention target of the conference participant A.
5, a data transmission device 116 for transmitting data input by the conference participant A using the data input device 115 to the terminal devices 12 to 14 of other conference participants, and transmission from the other terminal devices 12 to 14. The received data is received, and the data of the other conference participants B to D is sent to the conference participant A as the video display device 11.
2 and a data notification device 117 for notifying via the data communication device 2.

The image windows 1121 to 1123 displayed on the video display device 112 display the videos of the conference participants B to D, respectively, and use the multi-window method.

Further, the data input device 115 determines the mouse 1152 for operating the pointer 1151 displayed on the video display device 112 and where on the video display device 112 the pointer 1151 is to be displayed based on the movement of the mouse 1152. The pointer movement processing unit 1153 to be determined, the pointer display unit 1154 for displaying the pointer 1151 at the location determined by the pointer movement processing unit 1153, and the location determined by the pointer movement processing unit 1153 are on the image windows 1121 to 1123. And if so, which image window (1121-11
23) and a pointer position determination unit 1155 that determines whether or not the pointer 1151 is above the pointer 1151.

Further, the data transmission device 116 stores the attention target of the conference participant A input from the data input device 115, and when a new input is made, what data is input to which conference participation point. The attention target storage unit 1161 that determines whether or not to be transmitted, and the data transmission unit 1162 that transmits data to the other terminal devices 12 to 14 according to an instruction from the attention target storage unit 1161.

Further, the data notification device 117 has a data receiving section 1171 for receiving data from the other terminal devices 12-14, and a multi-window frame display for displaying the frames of the image windows 1121-1123 based on the received data. And a section 1172.

Terminal devices 12-1 of other conference participants B-D
4 has the same configuration.

Further, the arrow 15 is a communication line for transmitting to the terminal devices 12 to 14 the data indicating which of the conference participants A to D the conference participant A is paying attention to, and the arrow 16 is It is a communication line for transmitting to the terminal device 11 data indicating who of the conference participants B to D is paying attention to the conference participant A.

In FIG. 1, only the communication lines 15 and 16 between the terminal device 11 and the other terminal devices 12 to 14 are shown, but similar communication lines are also provided between the other terminal devices 12 to 14. Exists.

In the first embodiment, the communication network connecting the points is of a mesh type, but may be any type of connection such as a star type and a ring type.

Further, in FIG. 1, there are four conference participants.
Although it is a person, this may be any number.

Actually, of course, the configuration of the terminal device at each point requires a video / audio encoding / decoding device, a communication network interface, etc. It is omitted because it is not necessary.

These are the same in all the embodiments described below.

Although omitted in FIG. 1, a communication line for transmitting video and audio signals exists in the same manner as that shown in FIG. 7, and even in this device configuration, it is one in a conventional multipoint communication conference system. Similarly, the video and audio signals are transmitted.

The communication lines 15 and 16 and the communication lines for video and audio signals, which are omitted here, are logical ones. As a physical reality, the same communication line is used by multiplexing data. May be.

The operation of the first embodiment shown in FIG. 1 will be described below.

In FIG. 1, when the conference participant A pays attention to the conference participant B, the conference participant A performs the following operation so that himself / herself (conference participant A) can identify himself as the conference participant B. Explicitly enter what you paid attention to.

First, the conference participant A uses the mouse 1152 to give an instruction to the data input device 115 to move the pointer 1
151 is moved to the image window 1121 in which the image of the conference participant B is displayed.

Then, the mouse 1152 moves to the conference participant A.
The pointer movement processing unit 1153 is notified of the movement amount by the operation of.

The pointer movement processing unit 11 that has received this notification
53 is the video display device 112 of the current pointer 1151.
New display coordinates of the pointer 1151 are determined from the display coordinates on the above and the movement amount notified from the mouse 1152,
Pointer display unit 1154 and pointer position determination unit 11
Notify 55.

The pointer display unit 1154 displays the pointer 1151 on the video display device 112 according to the determination made by the pointer movement processing unit 1153.

The pointer position determination unit 1155 determines whether the coordinates determined by the pointer movement processing unit 1153 are on the image windows 1121 to 1123, and if so, which image window (1121 to 1123). It is determined whether it is above, and the result is notified to the data transmission device 116. Here, "B" is notified as data.

Upon receiving the notification from the data input device 115, the data transmission device 116 performs the following processing.

First, the attention target storage unit 1161 stores the attention target of the conference participant A notified from the data input device 115. The stored data is compared, and if they are the same, the process ends.

If there is a difference between the two data and the old data is "B to D", the data transmission unit 1
162 to the terminal devices 12 to 14 of old data,
Instruct the participant A to convey that he / she has stopped paying attention.

If the new data is “B to D”, the data transmitter 1162 sends the new data to the terminal device 1
2 to 14 are instructed to inform that the conference participant A is paying attention.

Here, the data, that is, the object of interest is "B ~
The case where the pointer 1151 is not “D” is a case where the pointer 1151 is located in a region other than the image windows 1121 to 1123 on the video display device 112.

The data transmission unit 1162 follows the instructions from the attention target storage unit 1161 and sets the other terminal devices 12-14.
Send data to.

The data notification device 117 is the other terminal device 1
The data transmitted from 2 to 14 are received, and the conference participant A is notified of the data according to the following procedure.

First, the data receiving unit 1171 receives the data transmitted from the other terminal devices 12 to 14, and notifies the multi-window frame display unit 1172 of the data. The multi-window frame display unit 1172, when the received data informs that attention is paid, the image window (1121 to 1123 of the image window of 1121 to 1123) displaying the images of the conference participants B to D who transmitted the data. Display either frame in a special format (or special color).

If the data indicates that the attention has stopped, the image windows (one of 1121 to 1123) of the conference participants B to D displayed on the video display device 112 which is the source of the data. Or) is returned to the standard format and displayed.

The frame formats of the image windows 1121 to 1123 displayed in this way are held until new data is notified from the data receiving unit 1171.

In the first embodiment shown in FIG. 1, the frame of the image window 1122 displaying the image of the conference participant C on the terminal device 11 of the conference participant A is the image window 1121.
And those displayed thicker than the frames 1123 are that the conference participant C is paying attention to the conference participant A and the conference participant B is
And D indicate that the conference participant A is not paying attention.

Further, in FIG. 1, in the terminal device 11 of the conference participant A, the pointer 1151 is located on the image window 1121 displaying the image of the conference participant B. This means that the conference participant B is paying attention.

In this way, the conference participants A to D can inform other conference participants that they are paying attention by operating the mouse 1152 to identify their attention target. Becomes

[Second Embodiment] FIG. 2 is an explanatory diagram for explaining a second embodiment of a multipoint communication conference communication support apparatus to which the present invention is applied. In the second embodiment,
The change in the size of the image window is used as a data input means for specifying the attention target of the conference participants.

In the second embodiment, the operation on the notified side is the same as that in the first embodiment, so the description thereof will be omitted and only the different case will be described.

In FIG. 2, A to D are participants of the conference (conference participants B to D are displayed in the screen), 2
Reference numeral 1 is a communication conference terminal device (hereinafter, referred to as a terminal device) of the conference participant A, and 212 is a terminal device 21 of the conference participant A.
2151 is a pointer for performing various GUI (graphical user interface) operations in a workstation or the like, and 2152 is a mouse for moving the pointer 2151 on the video display device 212.

Further, reference numerals 2121 to 2123 are image windows displayed by the multi-window method, and display images of conference participants B to D, respectively. 215
Reference numeral 3 denotes a scaling detection device that detects scaling of the image windows 2121 to 2123.

In FIG. 2, only one point of the communication conference system between four points is shown, but as shown in FIG. 1, it is assumed that terminal devices having the same structure are present at the other three points. To do. This is the same also in the examples shown below.

Here, the video input device and the audio input / output device are omitted because they are not particularly required for explanation.

In FIG. 2, if the conference participant A now pays attention to the conference participant B, the conference participant A will use the mouse 215 as in the normal multi-window system.
2 is used to enlarge the image window 2121 in which the image of the conference participant B is displayed.

Then, the enlargement / reduction detecting device 2153 detects the operation of the mouse 2152 and notifies the terminal device (not shown) of the conference participant B.

The operation on the side receiving the notification is the same as in the first embodiment.

In FIG. 2, the terminal device 2 of the conference participant A is shown.
1, the image window 2121 displaying the image of the conference participant B is displayed larger than the image windows 2122 and 2123, indicating that the conference participant A is paying attention to the conference participant B. There is.

In this situation, when the conference participant A changes the attention target from the conference participant B to the conference participant C, the conference participant A uses the mouse 2152 or the like.
The image window 2122 displaying the image of is enlarged.

The enlargement / reduction detection device 2153 detects this enlargement operation, and transmits data indicating that the conference participant A has started to pay attention to the terminal device (not shown) of the conference participant C.

Further, the image window 2121 on which the image of the conference participant B is displayed is returned to the size before being enlarged,
Data indicating that the conference participant A has stopped paying attention is transmitted to the terminal device (not shown) of the conference participant B.

[Third Embodiment] FIG. 3 is an explanatory diagram for explaining a third embodiment of a multipoint communication conference communication support apparatus to which the present invention is applied. In the third embodiment,
As the data input means for specifying the attention target of the conference participant, the change in the volume of the voice signal of the conference participant is used.

In the third embodiment, the operation on the notified side is the same as that in the first embodiment, so the description thereof will be omitted and only the different case will be described.

In FIG. 3, four conference participants A to D are participating in the communication conference. 31 is a schematic diagram of the audio output system of the terminal device of the conference participant A, and 311 to 313 are communication lines for transmitting the audio signals of the conference participants B to D to the terminal device of the conference participant A 315, respectively. 317 is a volume for adjusting the volume when outputting the audio signals of the conference participants B to D to the speaker, 318 is a volume level detecting device for detecting the level of the volumes 315 to 317, and 319 is A voice signal adder for outputting the voice signals of conference participants B to D to one speaker.
Reference numeral 14 is a voice output device of the terminal device of the conference participant A.

In FIG. 3, if the conference participant A pays attention to the conference participant B, the conference participant A adjusts the volume 315 to increase the volume of the voice from the conference participant B.

Then, the volume level detecting device 318 detects the volume increasing operation, and the volume set to the maximum volume among the volumes 315 to 317 becomes the volume 315 for the audio signal from the conference participant B. At that time, the terminal device (not shown) of the conference participant B is notified.

The operation on the side receiving the notification is the same as in the first embodiment.

Fourth Embodiment FIG. 4 is an explanatory diagram for explaining a fourth embodiment of the multipoint communication conference communication support apparatus to which the present invention is applied. In the fourth embodiment,
The line-of-sight detection device 419 is used as a data input means for specifying the attention target of the conference participant.

In the fourth embodiment, the operation on the notified side is the same as that in the first embodiment, so the description thereof will be omitted and only the different case will be described.

In FIG. 4, A to D are conference participants, 41 is a terminal device of the conference participant A, 412 is a video display device of the terminal device 41 of the conference participant A, and 419 is a line-of-sight detection device. is there.

Further, reference numerals 4121 to 4123 are image windows displayed by the multi-window method, and display images of conference participants B to D, respectively.

In FIG. 4, the video input device and the audio input / output device are omitted because they are not particularly necessary for explanation.

In FIG. 4, when the conference participant A pays attention to the image window 4121 on the display device 412 on which the image of the conference participant B is displayed, the line-of-sight detection device 419 determines that the conference participant A The direction of the line of sight is detected and notified to the terminal device (not shown) of the conference participant B.

The line-of-sight detection device 419 may use a commonly used difference in reflection characteristics between the cornea and the sclera, or a device using ocular potential, or more simply, a conference. By causing the participants A to D to wear glasses or hairpieces having a characteristic reflection characteristic for infrared rays and ultrasonic waves, and detecting reflected waves from the wearing tool of infrared rays and ultrasonic waves emitted from the line-of-sight detection device 419, A device that detects the orientation of the faces of the conference participants A to D may be used.

The operation on the side receiving the notification is the same as in the first embodiment.

[Fifth Embodiment] FIG. 5 is an explanatory view for explaining a fifth embodiment of a multipoint communication conference communication support apparatus to which the present invention is applied. In the fifth embodiment,
A pointer 5151, a mouse 5152, a pointer position determination unit 1155 (not shown), and symbol displays 51541 to 51544 of the conference participants are used as data input means for specifying the attention target of the conference participants.

In the fifth embodiment, the operation on the notified side is the same as that in the first embodiment, so the description thereof will be omitted and only the different case will be described.

In FIG. 5, A to D are participants of the conference, 51 is a terminal device of the participant A of the conference, and 512 is a video display device of the terminal device 51 of the participant A of the conference. 51
Reference numeral 51 is a pointer for specifying the attention target of the conference participant A, and reference numeral 5152 is a mouse for moving the pointer 5151 on the video display device 512. Reference numeral 5153 is a pointer position detection device for detecting the position of the pointer 5151 on the video display device 512.

Further, reference numerals 5121 to 5123 are image windows displayed by the multi-window method, and display images of the conference participants B to D, respectively. 515
Reference numeral 4 is a conference status display window that allows a list of conference participants, and 51541 to 51544 are conference participants A, respectively.
It is a symbol display of the conference participants showing ~ D.

In FIG. 5, the video input device and the audio input / output device are omitted because they are not particularly necessary for the description.

In FIG. 5, if the conference participant A now pays attention to the conference participant B, the conference participant A uses the mouse 5152 to move the pointer 5151 to the conference participant B.
Is moved to the symbol display 51542 of the conference participant who represents. The pointer position detection device 5153 is a mouse 5152.
Is detected and the terminal device (not shown) of the conference participant B is notified.

The operation on the side receiving the notification is the same as in the first embodiment.

[Sixth Embodiment] FIG. 6 is an explanatory diagram for explaining a sixth embodiment of the multipoint communication conference communication support apparatus to which the present invention is applied. In the sixth embodiment,
The data receiving unit 6171, the conference status display unit 6172, and the symbol display 61541 of the conference participants are provided as a function of notifying the attention relationship between the conference participants based on the transmitted data.
And 61544 are used.

In FIG. 6, A to D are participants of the conference, 61 is a terminal device of the participant A, and 612 is an image display device of the terminal device 61 of the participant A. 61
21 to 6123 are image windows displayed by the multi-window method, and conference participants B to D, respectively.
The image of is displayed. Reference numeral 6154 is a conference status display window that allows a list of conference participants.
61544 is a symbol display of the conference participants representing the conference participants A to D, respectively.

Further, in FIG. 6, 617 is a data notifying device for receiving data from another terminal device (not shown) and notifying the conference participant A of the received data, and 6171 is another 6172 is a data receiving unit for receiving data from a terminal device (not shown) of
Is a conference status display unit that displays a conference status display window 6154 including symbol displays 61541 to 61544 representing conference participants based on the received data.

Note that, in FIG. 6, the video input device and the audio input / output device are omitted because they are not particularly necessary for explanation.

In FIG. 6, the data notification device 617 is
The data transmitted from another terminal device (not shown) is received, and the received data is notified to the conference participant A according to the following procedure.

First, the data receiving unit 6171 receives data transmitted from another terminal device (not shown),
The conference status display unit 6172 is notified.

Then, the conference status display portion 6172 firstly sends the notified data to other conference participants (B to B).
If any one of D) is the data notifying that the attendee A is paying attention, the symbol display (61542) representing the attendee (one of B to D) of the sender of the data. 61544) in a special format (or special color).

If the received data is data notifying that the other conference participants (B to D) stopped paying attention to the conference participant A, the conference participant (B to Symbol display (one of D) (6
1542 to 61544) in a standard format (or standard color).

In this way, as for the format (or color) of the symbol display 61541 to 61544 of the conference participants (A to D) displayed, new data is received by the data receiving section 617.
It is held until 1 is notified to the conference status display unit 6172.

Symbol display (61541-615)
44) is displayed in the special format (or special color) or the standard format (or standard color) by an image generally used in WS (workstation) or PC (personal computer). It is realized by using the display method.

In FIG. 6, assuming that the conference participant B pays attention to the conference participant A, the terminal device (not shown) of the conference participant B is as shown in the first to fifth embodiments. , Data is input by some data input means, the input data is detected, and the terminal device 61 of the conference participant A is notified.

In response to this notification, the terminal device 61 of the conference participant A displays the symbol of the conference participant B representing the conference participant B in the conference status display window 6154 displayed on the video display device 612 by the multi-window method. By changing 61542 to a special format (or a special color), conference participant B notifies conference participant A that he or she is paying attention to conference participant A (self).

In FIG. 6, the terminal device 6 of the conference participant A
1, the symbol display 61542 representing the conference participant B among the conference participant symbol displays 61541 to 61544 displayed in the conference status display window 6154 is highlighted. It shows that attention is paid to A.

[Seventh Embodiment] FIG. 7 is an explanatory diagram for explaining a seventh embodiment of a multipoint communication conference communication support apparatus to which the present invention is applied. In the seventh embodiment,
As a function of notifying the attention relation between the conference participants based on the transmitted data, the symbol displays 72541 to 72544 of the conference participants and the attention relation are used, and the system has the same configuration as that in FIG. Only the one corresponding to the conference status display window 6154 is shown.

In FIG. 7, reference numeral 7254 denotes a conference status display window which allows the conference participants to be listed.
1 to 72544 are symbolic representations of conference participants representing conference participants A to D, respectively. In FIG. 7, 72545 to 72547 are related symbols for identifying the attention target of the conference participant, and now the conference participant A is the conference participant B.
If attention is focused on, the conference participant A is
As shown in FIG. 5, data is input by some data input means, the input data is detected, and the data is input to the terminal devices (not shown) of the conference participant B and the other conference participants C and D. Notice.

The terminal devices (not shown) of the conference participants B to D receive this notification, and the conference in the conference status display window 7254 displayed by the multi-window method on the video display device (not shown). Between the conference participant symbol display 72541 representing the participant A and the conference participant symbol display 72542 representing the conference participant B, the relation symbol 72545 representing that the conference participant A is paying attention to the conference participant B. Is displayed to notify each of the conference participants B to D that the conference participant A is paying attention to the conference participant B.

In FIG. 7, the conference status display window 7
Symbol display 7 of conference participants B to D displayed in 254
1541 to 71544 and related symbols 71545 to 7
1547 shows that the conference participant A pays attention to the conference participant B, the conference participant B pays attention to the conference participant C, and the conference participants C and D mutually pay attention to each other. It means that they are meeting each other.

The seven embodiments of the communication support device for the conference in the multipoint communication conference system according to the present invention have been described above. The "target attention of conference participants" described in the first to fifth embodiments is shown. The "data input means for specifying" and the "function of notifying the attention relationship between conference participants" shown in the first embodiment and the sixth and seventh embodiments may be used in any combination.

Further, the "data input means for specifying the attention target of the conference participants" and the "function of notifying the attention relation between the conference participants" may be used individually or in combination. May be.

For example, as the "data input means for specifying the attention target of the conference participant", the method of using pointing to the image window of the first embodiment and the method of using the size of the image window of the second embodiment. By combining and, the following method can be realized.

The conference participant A inputs that he is paying attention to the conference participant B by enlarging the image window in which the image of the conference participant B is displayed, but temporarily joins the other participants. When the person C is focused, the pointer is moved onto the image window in which the image of the conference participant C is displayed to input the attention to the conference participant C.

Then, when the attention is returned from the conference participant C to the conference participant B, the pointer is moved to a place other than the image window.

In the above, the embodiments of the present invention have been focused on “data input means for specifying the attention target of the conference participants” and “function of notifying the attention relationship between the conference participants”.
Although specifically described, the requirement of the present invention is that the data input by the participant of the multipoint communication conference system by the “data input means for specifying the attention target of the conference participant” is used by the other conference participants. It is to support communication between conference participants in a multipoint communication conference system by transmitting it to the terminal device and notifying the conference participant by the "function of notifying the attention relationship between conference participants". The "data input means for specifying the attention target of the conference participants" and the "function of notifying the attention relation between the conference participants" may be realized by methods other than those shown in the embodiments.

As described above, the invention made by the present inventor is
Although the present invention has been specifically described based on the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments, and it goes without saying that various modifications can be made without departing from the scope of the invention.

[0142]

As described above, according to the present invention, each terminal device inputs the operation information of each conference participant, and transmits the operation information to the terminal device at each point. Since the terminal device at each point receives the received operation information and notifies the received operation information, the scale of the terminal device is not increased, the degree of freedom of the installation position of the terminal device is not reduced, and the communication network It is possible to notify the conference participants of the communication operation information of each conference participant without increasing the amount of transmission data due to the connection form.

With this, even in the multipoint communication conference system, the attention target of each conference participant can be transmitted and notified to other conference participants.

As a result, it becomes possible to provide a conference environment that is even closer to the ordinary face-to-face conference, and it is expected that a human interface that is kind to humans is realized.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram showing a configuration of a first embodiment of a multipoint communication conference communication support apparatus to which the present invention is applied.

FIG. 2 is an explanatory diagram for explaining a second embodiment of a multipoint communication conference communication support device to which the present invention is applied.

FIG. 3 is an explanatory diagram for explaining a third embodiment of a multipoint communication conference communication support device to which the present invention has been applied.

FIG. 4 is an explanatory diagram for explaining a fourth embodiment of a multipoint communication conference communication support device to which the present invention has been applied.

FIG. 5 is an explanatory diagram for explaining a fifth embodiment of a multipoint communication conference communication support device to which the present invention has been applied.

FIG. 6 is an explanatory diagram for explaining a sixth embodiment of a multipoint communication conference communication support device to which the present invention has been applied.

FIG. 7 is an explanatory diagram for explaining a seventh embodiment of a multipoint communication conference communication support device to which the present invention has been applied.

FIG. 8 is an explanatory diagram for explaining a conventional multipoint communication conference system.

9 is an explanatory diagram for explaining a system that solves the problems of the multipoint communication conference system of FIG. 8. FIG.

FIG. 10 is a schematic diagram showing the positional relationship between the angle of view of the video input device and the conference participants.

[Explanation of symbols]

A, B, C, D ... Conference participants, 11, 12, 13, 14
... Terminal device in Embodiment 1, 15, 16 ... Communication line,
111 ... video input device in the first embodiment, 112 ... video display device in the first embodiment, 113 ... audio input device in the first embodiment, 114 ... audio output device in the first embodiment, 115 ... data input device in the first embodiment, 116
... Data transmission device in Example 1 117 ... Example 1
A data notification device in 1151, a pointer in the first embodiment, 1152, a mouse (pointing device) in the first embodiment, 1153, a pointer movement processing unit,
1154 ... Pointer display unit, 1155 ... Pointer position determination unit, 1161 ... Target storage unit, 1162 ... Data transmission unit, 1171 ... Data reception unit, 1172 ... Multi-window frame display unit.

Claims (1)

[Claims] 1. A communication support device for a conference in a multipoint communication conference system in which video and audio are transmitted and received between a plurality of terminal devices and a plurality of participants hold a conference, wherein the terminal device is a conference participant. Motion information input means for inputting each motion information, motion information transmitting means for transmitting the motion information to the terminal device at each point, motion information receiving means for receiving the motion information from the motion information transmitting means, An inter-multipoint communication conference communication support device, comprising: an informing unit for informing the operation information received by the operation information receiving unit.