JP2019102032A - Communication system, method for communication, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem to be solved by the transmission source / transmission of a message (communication) at a distance visible to each other by combining information of an orientation sensor which is generally built in a terminal device. It is to provide a communication system that can post the previous direction relationship. SOLUTION: The communication system of the embodiment has a first terminal device including a direction acquisition unit for acquiring a direction in which the first terminal device is facing, and a first communication unit for transmitting the acquired direction. , The second communication unit that receives the direction from the first terminal device, the direction acquisition unit that acquires the direction that the second terminal device is facing, the direction that the first terminal device is facing, and the second A second terminal device including a direction calculation unit that calculates the direction of the first terminal device from the direction in which the terminal device is facing, a display unit that displays the calculated direction, and an audio output unit that presents by sound. And prepare. [Selection diagram] Fig. 1

Description

Embodiments of the present invention relate to a communication system, a communication method, and a program.

  2. Description of the Related Art Conventionally, a communication system has been known in which users of terminal devices can exchange messages with each other by using a terminal device connected to a network, such as a mobile phone, a smartphone, or a wireless income. In a typical communication system, voice messages are simply transmitted regardless of the other party's location, even if they are at a visible distance from each other, and the text message information does not indicate the direction of the sender user's name or It is displayed with information such as time.

  In the real world, if the distance between the source user and the destination user is as close as possible to direct voice, it is usually understood from which side the voice has been hung, and by facing each other, communication using gestures and facial expressions using body is used. You can do it, or you can get around each other. For this reason, for example, when waiting for a crowded station, when the user finds the other party in the meeting, he can not indicate the direction in which he is present to the destination user, and notices until the voice reaches close enough to reach the voice. It happens that you can not get it. In addition, workers wearing protective clothing and helmets and workers under noise have difficulty in knowing the direction of the other person because direct voices do not reach them, and communication using gestures and communication with each other make joint efforts. There was a problem that the work could not be performed smoothly.

  On the other hand, there is an apparatus that presents the direction of the transmission source to the transmission destination. For example, a system that displays the direction of the transmission source viewed from the transmission destination on the terminal by detecting the orientation of the face of the transmission destination viewed from the transmission source based on the acquired camera image and finding each other's positional relationship The direction of the transmission source and the distance from the transmission source are calculated by using the position of the own (transmission source), the position of the transmission destination, and the moving azimuth of the own, and the direction arrow to the transmission destination Systems for displaying and the like are known.

  However, in the prior art, since processing of a camera image for capturing the direction of the user is necessary, there is a problem that it can not be applied when the camera is not attached or when the image processing load can not be applied. In addition, there is also a problem that it is difficult to stably obtain the direction depending on the direction of the transmission destination, and a problem that batch transmission can not be performed to a plurality of transmission destinations facing different directions. Furthermore, when the distance between the terminal devices is not sufficient, or when the positioning conditions are not good because of indoors, buildings, etc., it is difficult to stably obtain the direction due to the influence of the GPS accuracy, so the transmission destination is visually There is a problem that it can not be applied under the condition that it is within the possible distance.

Patent No. 3952870 gazette Patent No. 3886923 WO 07/105436 JP, 2012-60285, A Patent No. 4327822 gazette

  The problem to be solved by the present invention is that the directions of the source and destination of the message (communication) at a distance visible to each other by combining the information of the orientation sensor as generally incorporated in the terminal device. It is to provide a communication system capable of posting a relationship. By posting each other's direction, in communication between workers wearing protective clothing and helmets and workers under noise, not only the voice of the sender can be heard through the wireless network but also the location of the sender It is also possible to face each other by recognizing it, and it becomes possible to communicate using gestures using the body and facial expressions together.

  In order to achieve the above-mentioned subject, the communication system of an embodiment transmits the direction where the direction acquisition part which acquires the direction where the first terminal device is facing, and the direction where the acquired first terminal device is facing is transmitted. A first terminal apparatus comprising: a communication unit; a second communication unit for receiving an orientation in which the first terminal apparatus faces from the first terminal apparatus; and a second terminal apparatus The second terminal apparatus viewed from the second terminal apparatus from an azimuth acquiring unit for acquiring an azimuth, an azimuth at which the received first terminal apparatus faces, and an azimuth at which the acquired second terminal apparatus faces A second terminal device comprising: a direction calculation unit that calculates a direction of one terminal device; and a display unit that displays the calculated direction of the first terminal device or an audio output unit that presents with sound Prepare.

FIG. 1 is a block diagram showing the configuration of a communication system according to a first embodiment. The block diagram which shows the structure of (a) 1st terminal device 110 (b) 1st terminal device 120 which concerns on 1st embodiment. FIG. 2 is a block diagram showing configurations of a wearable terminal 110a and a terminal main body 110b according to the first embodiment. (A) The mounting | wearing image figure of the wearable terminal 110a. (B) The figure which shows an example of the wearable terminal 110a. FIG. 2 is a block diagram showing the configuration of a server 200 according to the first embodiment. The figure which shows the processing flow of the 1st terminal device 110 which concerns on 1st embodiment. The figure which shows the processing flow of the server 200 which concerns on 1st embodiment. The figure which shows an example of the user master 231 which concerns on 1st embodiment. The figure which shows an example of the beacon master 232 which concerns on 1st embodiment. The figure which shows an example of the place interlocking | linkage type table 233 which concerns on 1st embodiment. The figure which shows an example of the nickname dictionary 234 which concerns on 1st embodiment. FIG. 5 is a sequence diagram showing an example of a communication processing procedure between devices of the communication system according to the first embodiment. FIG. 5 is a sequence diagram showing an example of a communication processing procedure between devices of the communication system according to the first embodiment. FIG. 5 is a sequence diagram showing an example of a communication processing procedure between devices of the communication system according to the first embodiment. The figure which shows an example of the transmission source direction calculation method in case the terminal device A (transmission source) calls with respect to the terminal device B (transmission destination = Mr. Suzuki). The figure which shows an example of the transmission source direction calculation method in case the terminal device D (transmission source) calls towards pole number 8 vicinity with a beacon. The figure which shows an example of the transmission source direction calculation method in case a terminal device H (transmission source) calls to the terminal device J (transmission destination) of the 25th user identified in the camera image. The figure which shows an example of the screen display of the terminal device of the smart phone type which concerns on this embodiment. The figure which shows an example of the screen display of the terminal device of a smart phone type when it turns to the diagonally left direction in the form which responds to Mr. Sato's calling. FIG. 6 is a view showing an example of screen display of the tablet type terminal device according to the present embodiment. The figure which shows an example of the screen display of the tablet-type terminal device when it turns to the diagonally left direction in the form which responds to Mr. Sato's calling. The block diagram showing the composition of the communication system concerning a second embodiment. The figure which shows the processing flow of the communication system which concerns on 2nd embodiment. The block diagram showing the composition of the communication system concerning a third embodiment. The figure which shows the processing flow of the communication system which concerns on 3rd embodiment. The block diagram showing the composition of the communication system concerning a fourth embodiment. The figure which shows the processing flow of the communication system which concerns on 4th embodiment.

  Hereinafter, embodiments of the present invention will be described.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of a communication system according to the first embodiment. In the communication system of the present embodiment, a terminal device group 100 including one or more terminal devices and a server 200 are connected via a network. Here, the network includes radios for mobile phones such as 4G, LTE, and 3G, wireless LANs, Bluetooth (registered trademark), and the like.

<Configuration of Terminal Device>
A block diagram of the terminal device according to the present embodiment is shown in FIG. In the present embodiment, the terminal device possessed by the user who has sent the message or the like is the first terminal device 110, and the terminal device possessed by the user designated as the transmission destination is the second terminal device 120. However, the configurations of the terminal devices in the terminal device group 100 are the same, and both can be source and destination terminal devices, so only the configuration of the first terminal device 110 will be described, and details regarding other terminal devices will be described. I omit it.

FIG. 2A shows a first terminal apparatus 110 which is a terminal apparatus of a transmission source. The first terminal device 110 is, for example, a dedicated device, a smartphone, a tablet, a PC, etc., and the network interface 111, the azimuth information acquisition unit 112, the position information acquisition unit 113, the display unit 114, the control unit 115, the input unit 116, and voice An acquisition unit 117 and an audio output unit 118 are provided.

  The network interface 111 transmits / receives data to / from a terminal device other than the own terminal or the server 200. The orientation information acquisition unit 112 acquires orientation information on the user of the first terminal device 110. The position information acquisition unit 113 acquires position information of the first terminal device 110.

  The orientation information acquisition unit 112 uses, for example, a geomagnetic sensor, a gyro sensor, a GPS, an acceleration sensor, or a combination thereof, but any method may be used as long as the orientation in two or three dimensions can be detected with appropriate accuracy. . Moreover, you may acquire with the azimuth | direction sensor incorporated in the 1st terminal device 110 main body. Furthermore, when the first terminal device 110 is in a direction different from the direction of the user, for example, with respect to the direction acquired by the first terminal device 110, the terminal device by an acceleration sensor, a gyro sensor or the like. The direction in which the user points may be corrected based on the information of the user's traveling direction by up and down, GPS or the like. Also, the angle (elevation angle) in the vertical direction of the terminal device may be included in the orientation information.

  The position information acquisition unit 113 acquires, for example, position information including latitude and longitude by GPS. The display unit 114 displays data received from another terminal device or the server 200. The control unit 115 controls each configuration in the first terminal device 110 and the exchange of the acquired / received data. The input unit 116 acquires an input signal from the user, such as selection of a mode to be described later. The voice acquisition unit 117 obtains the voice uttered by the user of the first terminal device 110. The voice output unit 118 reproduces voice data received from a terminal device other than the own terminal or the server 200. Here, the display unit 114 may double as a display capable of touch input, that is, the input unit 116.

The 2nd terminal device 120 which is a terminal device of a transmission place in Drawing 2 (b) is shown. The configuration is the same as that of the first terminal device 110, and includes a network interface 121, an azimuth information acquisition unit 122, a position information acquisition unit 123, a display unit 124, a control unit 125, an input unit 126, an audio acquisition unit 127, and an audio output unit 128. Since all of them are the same as the first terminal device, the details will be omitted.

  The terminal device according to the present embodiment may be divided in configuration using the wearable terminal 110a as shown in FIG. The wearable terminal 110a is a wearable device having, for example, a shape like glasses, and a microphone type voice acquisition unit 117, a binaural earphone type voice output unit 118, and an orientation for acquiring the direction in which the user who is wearing is facing An information acquisition unit 112 and a button switch type input unit 116 are provided, and the network interface 111, the position information acquisition unit 113, the display unit 114, and the control unit 115 are provided in the terminal main body 110b. The mounting image is shown in FIG. 4 (a). The display unit 114 of the terminal body 110b may be included in the configuration of the wearable terminal 110a as a head mounted display. The wearable terminal 110a may be a headphone-shaped device as shown in FIG. 4 (b). Furthermore, the orientation information acquisition unit 112 may be configured as the terminal main body 110b. In addition, the wearable terminal 110a may be provided with an image acquisition unit such as a camera. All or part of the transmission source terminal device and the transmission destination terminal device in the terminal device group 100 of the communication system according to the present embodiment may use the wearable terminal 100a or may not use the wearable terminal 100a. good.

<Configuration of Server 200>
Next, FIG. 5 shows the configuration of the server 200 according to the present embodiment. The server 200 is, for example, a server that provides an on-premises server, a cloud-type service, and the like, and includes a network interface 210, a processing unit 220, and a storage unit 230. The network interface 210 exchanges data with the terminal devices in the terminal device group 100. The processing unit 220 performs speech recognition processing, and controls each configuration in the server 200 and exchange of received data. The storage unit 230 stores necessary data tables.

«Processing flow of first terminal device 110 of communication system»
In the communication system according to the present embodiment, two types of modes, "call mode" and "tweet mode", are prepared. In the “call mode”, the direction of the transmission source is from the first terminal device 110 possessed by the transmission destination user or the member of the group by pointing the direction and making a message input (call) when the transmission destination is visible, etc. Is a mode in which a message is output. The "tweet mode" is a mode in which normal message output is performed without the direction of the transmission source.

  FIG. 6 is a process flow of the first terminal device 110 according to the present embodiment. First, when the user makes a request for system activation by activating a dedicated application or accessing a home page, the control unit 115 of the first terminal device 110 controls the display unit 114 to display an initial screen of the system. Is displayed (S601). At this time, for example, the user may log in to the system by inputting a login ID and a password. In addition, each user can transmit a system activation request to the server 200 together with identification information of the first terminal device 110 (hereinafter referred to as a terminal device identification ID) and manage flags in the user table of the storage unit 230. It may be possible to confirm whether or not the system is activated, that is, online or offline.

  Next, the user selects the user or group of the other party who sends the message, and the user name and group name are input from the input unit 116, whereby the control unit 115 of the first terminal apparatus selects the selected transmission. Get the other party. Also in the received message, only a message from the user or group selected here or a message sent to the group may be displayed. Subsequently, the user who wants to send a message selects whether to use the system in the “call mode” or the “tweet mode”, and inputs the signal from the input unit 116 to control the first terminal device 110 first. The unit 115 determines whether the signal acquired by the input unit 116 is in the “call mode” (S602).

<Processing Flow of “Calling Mode” in First Terminal Device 110>
When the transmission source user wants to convey the user's own direction as viewed from the transmission destination, for example, when the transmission destination user can view the user or the group as the transmission destination, the transmission source user selects the "call mode". The input unit 116 of 110 acquires a signal of the "call mode" (YES in S602). The user taps, for example, the “call mode” button displayed on the display unit 114 (in this case, the display unit 114 doubles as the input unit 116), or inputs “voice mode” as voice (in this case, voice The acquisition unit 117 doubles as the input unit 116 and executes voice recognition processing to select the “call mode”. In the case where the input unit 116 is a button switch, for example, the case where the button is pressed twice in succession may be used as the input start button in the call mode. Alternatively, the input unit 116 (voice acquisition unit 117) may call the name of the transmission destination (for example, “Mr. ○ ○”), or an utterance including a specific keyword set in advance corresponding to the call (for example, !), Voice recognition processing may be performed to detect a keyword, and call mode operation may be performed. This voice recognition process may be performed by the input unit 116 (voice acquisition unit 117) in the first terminal device 110, but may be performed by the processing unit 220 of the server 200 by transmitting the acquired voice to the server 200. good.

  Next, the orientation information acquisition unit 112 acquires the orientation in which the first terminal device 110 or the headset 110a or the like is facing (S603). At this time, in a state where the user of the first terminal device 110 (transmission source user) faces the first terminal device 110 or the face attached to the headset 110 a or the like to the transmission destination second terminal device 120 (transmission destination user) is there. Here, the position information acquisition unit 113 of the first terminal device 110 may also acquire position information of the first terminal device 110, for example, latitude and longitude.

  Note that the first terminal device 110 or the headset 110a is obtained by providing an image acquisition unit (not shown in FIGS. 2 and 3) such as a camera for photographing the same direction as the direction in which the first terminal device 110 or the headset 110a faces. By performing known image recognition processing on an image, it is possible to specify transmission destination information (a transmission destination user, a group or a terminal device) existing at a destination where the transmission source user is facing, or to face the transmission destination user It is also possible to determine the direction of movement. Here, instead of the camera, detection may be performed using an optical input device such as a directional infrared sensor. This image recognition process may be performed on the first terminal device 110 side, but may be performed by the processing unit 220 of the server 200 by transmitting the acquired image to the server 200.

  The identification of the camera image by the terminal device is performed by detection of an identification code by visible light or infrared light emitted from the second terminal device 120 possessed by the transmission destination user or the second terminal device 120 possessed by the transmission destination user. Further, by combining with the result acquired by the azimuth information acquisition unit 112, the azimuth and the transmission destination to which the user is directed may be derived more accurately.

  Also, if there is a wall surrounding the area where the terminal device group 100 exists, for example, information such as characters such as "SW" and "225", a two-dimensional code, an identification code by infrared light etc. Determine from the information such as the letters "SW" and "225" in the acquired image that the orientation to which the image acquisition unit (sender user) is facing is southwest (225 degrees clockwise with north at 0 degrees) And may be replaced by the orientation information acquisition unit 112.

  Subsequently, the transmission source user inputs a voice message to be transmitted to the transmission destination from the voice acquisition unit 117, or inputs a text message from the input unit 116 (in the case of tap input, the display unit 114) (S604). At this time, information that specifies the transmission destination user may be input. For example, the user may select a transmission destination user or group from the input unit 116 (in the case of tap input, the display unit 114), may input a name or a group name, , Group names etc. may be added.

  Next, the control unit 115 transmits the acquired information such as the input message, the input time of the message, the direction information, the identification ID of the first terminal device 110, and the like to the server 200 via the network interface 111 (S605). Here, if the transmission destination can be specified, transmission destination information (transmission destination user, group or terminal device) is also transmitted. The orientation information may be the orientation acquired by the source user at S602, or the first terminal device viewed from the second terminal device 120 by 180 degrees opposite to the acquired orientation. It may be transmitted after conversion to the direction of 110. Alternatively, only the direction information may be transmitted without inputting text or voice messages. Furthermore, the position information acquisition unit 113 may acquire position information of the first terminal device 110 and transmit it. The orientation information and the position information may be accompanied by information of measurement accuracy.

  Subsequently, the control unit 115 determines whether the signal acquired by the input unit 116 is a system termination request (S606). When the user wants to terminate the system, the user inputs a system termination request by tapping the “system termination” button displayed on the display unit 114 of the first terminal device 110 or logging out of the system. If there is no system termination request (NO in S606), the process returns to S604. At this time, for example, "Yes" may be selected in response to the message "Do you want to shut down the system?" Or the like may be input that the system is not to be shut down. When there is a termination request (YES in S606), the system is terminated, and the processing of the first terminal device 110 according to the present embodiment is terminated. Here, the system may be terminated when there is no input processing or operation by the user for a predetermined time or more. In addition, when online / offline management of each user is performed, a system termination request is transmitted to the server 200 together with the identification ID of the first terminal device 110.

  In the present embodiment, the azimuth information is acquired before the message input of the transmission source user, but may be acquired and transmitted at another timing. Also, the transmission source user may acquire orientation information multiple times during message input and sequentially transmit. Alternatively, the acquired direction information may be stored in the control unit 115 in association with the transmission destination, and the information may be continuously transmitted to the same transmission destination without being repeatedly acquired.

<Processing Flow of “Tweet Mode” in First Terminal Device 110>
If it is not necessary to transmit the direction in which the transmission source user is present to the transmission destination, in the present embodiment, the transmission source user does not select the “call mode” (NO in S602) and selects the “tweet mode”. The input unit 116 of the one terminal device 110 acquires a "tweet mode" signal (YES in S607). Here, the transmission source user, for example, taps the “Tweet mode” button displayed on the display unit 114 (in this case, the display unit 114 doubles as the input unit 116), or inputs “Tweet mode” as voice. (In this case, the voice acquisition unit 117 doubles as the input unit 116) to select the “tweet mode”. When the input unit 116 is a button switch, for example, a single press may be used as an input start button in the tweet mode. In addition, it is possible to select either “call mode” or “tweet mode”, or automatically “tweet mode” when it is not possible to acquire direction information of the transmission source or information specifying the transmission destination. It may be

  Then, it progresses to S604. In the "tweet mode", the azimuth information is not transmitted in S605, but is the same as the "call mode" except that the azimuth information is not transmitted, so the details of the subsequent processing will be omitted.

  If the "tweet mode" is not input (NO in S607), the process proceeds to S606, but the subsequent processing is the same as the "call mode" and thus will be omitted.

  The first terminal device 110 connects to the server 200 by, for example, the REST API via the wireless network and performs transmission. When voice message transmission starts, a voice transmission session start request (PUT voice / start_time) is issued, and information (session key) identifying the session is returned from the server 200. Voice messages are sent sequentially (POST voice). Finally, the end of the voice transmission session is transmitted (PUT voice / end_time). Also, when sending a text message, it sends text information (POST speech). In the case of a voice message, azimuth information, position information, and the like of the transmission source are transmitted as parameters for voice transmission session start request and text information transmission.

«Processing flow of server 200 of communication system»
Subsequently, FIG. 7 shows a process flow of the server 200 in the communication system according to the present embodiment. First, the network interface 210 of the server 200 receives a message or the like transmitted from the first terminal device 110 (S701). In the "call mode", in S701, the input message, the input time of the message, the direction information, the identification ID of the first terminal apparatus 110, and the like are received. In the "tweet mode", no azimuth information is included.

  Subsequently, the processing unit 220 refers to the user master 231 from the identification ID of the first terminal device 110, and identifies the user name of the first terminal device 110 (S702). The user master 231 associates and stores at least the terminal device identification ID and the name of the user who uses the terminal device, and is stored in the storage unit 230.

  An example of the user master 231 according to the present embodiment is shown in FIG. The user master 231 of the present embodiment is provided with a "terminal device identification ID", a "user name" using the terminal device, and a "group name" field to which the user belongs. For example, when the terminal device identification ID of the received first terminal device 110 is “ab01”, it can be read that the user of the first terminal device 110 is “Taro Sato”.

In addition, as composition of the user master,
(1) The "user ID" used by the system when internally discriminating the user, the "login ID" specified when the user logs in to the system, the "user name", and the "user ID" to which the user belongs Login ID master consisting of "group ID" and
(2) A group ID master consisting of a "group ID" and a "group name" used by the system to distinguish groups internally
(3) a terminal ID master comprising the "terminal device identification ID" and the "user ID" using the terminal device,
There may be a plurality of tables as in FIG.

  Next, the processing unit 220 identifies the second terminal apparatus 120 that is the transmission destination of the received message (S703). If the transmission destination information (transmission destination user, group or terminal device) is received from the first terminal device 110, the storage unit 230 is referenced to specify the second terminal device identification ID which is the transmission destination. For example, when the received transmission destination information is “Jiro Takahashi”, the processing unit 220 refers to the user master 231 in the storage unit 230 and specifies the corresponding terminal device identification ID “cd02”. When the received transmission destination information is "business", the processing unit 220 refers to the "group name" column of the user master 231 in the storage unit 230, and identifies the corresponding terminal device identification ID "ab01" "cd02". Do. Alternatively, terminal device identification IDs of users who belong to the sales group may be registered in advance.

  Also, for example, when the received transmission destination information “user ID” is “xy07”, the terminal ID master is referenced to specify the corresponding terminal device identification ID. If the transmission destination information “user name” is “Jiro Takahashi”, the login ID master is referenced, and the corresponding user ID “xy07” is identified, and then the terminal device identification ID is similarly identified. When the received transmission destination information "group ID" is "gh13", after referring to the login ID master and specifying the corresponding user ID "xx06" "xy07", the terminal ID master is referred to and the corresponding terminal Identify the device identification ID. When the transmission destination information "group name" is "business", the corresponding group ID "gh13" is specified with reference to the group ID master, and the terminal device identification ID is similarly specified.

  When using a group linked to a place (hereinafter referred to as a place group), install beacons using radio such as Bluetooth Low Energy (registered trademark) or infrared rays according to the place, and identify the beacon identification ID and the place beforehand. Are associated with each other and stored as the beacon master 232. A terminal device linked to a beacon is assumed to be within a predetermined distance from the beacon, for example, within a radius of 5 m. An example of the beacon master 232 according to the present embodiment is shown in FIG. In this example, it is a case where a beacon is installed for each pillar to which a number is given at a construction site or a manufacturing site, etc. The beacon master 232 of this embodiment is the "beacon identification ID" and the name of the place. And a field for storing "postal number".

  When a terminal device used by each user receives a radio wave from a beacon, it is determined that the control unit of each terminal device is within 5 m of the beacon, and the identification ID of the received beacon, its own terminal device identification ID, Are sent to the server 200. The processing unit 220 of the server 200 refers to the beacon master 232 based on the received identification ID of the beacon, identifies the name of the place, and stores the name in the place-linked table 233 in association with the terminal identification ID. Alternatively, terminal device identification IDs existing near each beacon are collectively registered as a place group. Further, the processing unit 220 may calculate and store the distance from the beacon using the position information acquired by the position information acquiring unit of the terminal device. Further, instead of managing the place group by the server 200, it is determined whether or not each of the terminal devices (the second terminal device 120 serving as the transmission destination) belongs to the transmission destination place group according to the detection status of the beacon. You may do it.

  An example of the location linked table 233 according to the present embodiment is shown in FIG. The location linked table of the present embodiment includes a field for storing "terminal device identification ID", "postal number" which is the name of the location, and "distance from the post" indicating the distance from each beacon. For example, it can be read that the user whose terminal device identification ID is "ef30" is near "pillar 25" and is separated from pillar 25 by "0.5 m".

  Furthermore, when the transmission destination is specified by the user's call name included in the message or the like (to be described in detail later), the call name dictionary 234 is prepared in advance. An example of the nickname dictionary according to the present embodiment is shown in FIG. In the name dictionary 234 of the present embodiment, a plurality of names indicating the user of the terminal device are stored in association with the terminal device identification ID. For example, when “Taro san” is included in the message or the like, it can be estimated that the user is the terminal device identification ID “ab01” by referring to the nickname dictionary 234. Here, after associating with the user ID etc. instead of the terminal device identification ID, the user master may associate with the terminal device identification ID.

  In this way, by storing correspondences in the storage unit 230 as data tables, such as names of users (such as "Mr. XXX"), names of groups to which the user belongs (such as sales groups), groups linked to places, etc. It is possible to set and specify various destinations. Note that all or part of the data table of the storage unit 230 of the server 200 may be provided on the transmission source terminal or the transmission destination terminal.

  Subsequently, the processing unit 220 performs recognition processing of the received message (S704). In the case of a voice message, the voice message is converted into text by performing known voice recognition processing. Here, if the transmission destination can not be identified in S703, the text-converted message or the received text message is analyzed, and if it is included, the user's name or name, group name, location, etc. Extract the name etc.

  As a result of analyzing the text message, when the user's name, name, group name, and place name are extracted, the processing unit 220 refers to the data table in the storage unit 230 to specify the transmission destination. For example, in the case where the name "Mr. Suzuki" is extracted, the terminal name ID "ef30" can be specified with reference to the name dictionary 234.

  If the transmission destination can not be specified or estimated, all terminal devices except the first terminal device 110 may be set as the distribution destination, or the like. Or you may send a voice or text message. Here, in the case of distribution, when the destination terminal apparatus and another terminal apparatus can not be distinguished, or when a message having the same content as the transmission destination terminal is sent to other terminal apparatuses that are not the transmission destination terminal apparatus. Etc., indicates a case where a terminal including another terminal is set as a message destination. In addition, when the text display of the message is not performed in the second terminal apparatus 120 which is the transmission destination, the voice message may be transmitted to the second terminal apparatus 120 as the voice data without being converted into text. Furthermore, in the case of the "calling mode", there may be no message (even information on the direction of the transmission source user). In addition, the first terminal apparatus 110 transmits the voice recognition result and the text analysis on the transmission source terminal, and temporarily sends the speech recognition result and the text message analysis result in the server 200 to the first terminal apparatus 110. May be selected.

  When the image acquisition unit exists in the first terminal device 110 and receives image data, the image recognition is performed to calculate the feature amount of the destination user's face, or the user wears clothes, a helmet, etc. The transmission destination may be determined by recognizing the number etc. described in the one attached to and collating with a database etc.

  Subsequently, the processing unit 220 determines whether or not the direction information is included in the information received from the first terminal device 110 (S705). If the direction information is included (YES in S705), the received mode is the "call mode", and thus the process proceeds to S706. If the direction information is not included (NO in S705), since the "tweet mode" is set, the process proceeds to S707.

  In S706, the processing unit 220 acquires orientation information of the second terminal device 120, and calculates the direction of the first terminal device 110 in the second terminal device 120 that is the transmission destination. The processing unit 220 calculates the difference between the direction 180 degrees opposite to the direction in which the first terminal device 110 is facing and the direction in which the second terminal device 120 is on the basis of the received direction information. Here, in the case where the azimuth information received from the first terminal device 110 is the azimuth that the first terminal device 110 is facing as it is, calculation is performed using an azimuth 180 degrees opposite to that. When the azimuth information received from the first terminal device 110 is converted to an azimuth 180 degrees opposite to the azimuth of the direction in which the first terminal device 110 faces, calculation is performed using the azimuth as it is. Here, the direction in which the second terminal device is facing may be 180 degrees opposite to calculate. The calculation of the direction of the first terminal device 110 may be performed by the control unit 125 of the second terminal device 120. In that case, the direction information of at least the first terminal device is received directly from the first terminal device 110 or from the server 200, and calculation is performed using the direction information of the second terminal device 120 acquired from the direction information acquiring unit 122. In this case, the server 200 transmits the direction information of the first terminal device to the second terminal device as it is or only by converting it to the direction opposite to 180 degrees.

<< Communication processing sequence of communication system >>
The sequence diagram of FIG. 12 illustrates an example of a communication processing procedure between the first terminal device 110, the server 200, the second terminal device 120, and other terminals of the communication system according to the present embodiment.

  First, when the user of the first terminal device 110 (sender user) inputs a text message, the first terminal device 110 acquires the text content of the message (S1201), and then the sender user presses a button By doing the same, the first terminal apparatus 110 receives a signal of "call mode" (S1202). Subsequently, the first terminal device 110 acquires the azimuth information from the azimuth information acquisition unit 112, and transmits the text content of the message and the azimuth information of the first terminal device 110 to the server (S1203).

  The server 200 transmits the text content and direction information of the message received from the first terminal device 110 to the second terminal device 120 which is the transmission destination (S1204). The second terminal device 120 acquires the azimuth information from the azimuth information acquisition unit 122, calculates the direction of the first terminal device 110, and displays the text content of the received message and the calculated direction (S1205).

  Also, the server 200 distributes the text contents of the message received from the first terminal device 110 to the terminal devices other than the transmission destination (S1206). The other terminal displays the text content of the received message (S1207). Here, the server 200 may distribute the azimuth information and the transmission destination information in addition to the text content, and the other terminal device determines that the own terminal is not the transmission destination based on the received transmission destination information, and simply receives it. Text content may be displayed. Furthermore, the first terminal device 110 transmits a session start request to the server (S1208), and receives a session ID from the server.

  Next, the transmission source user performs voice input of speech 1 and speech 2 toward the first terminal device 110, and the first terminal device 110 acquires voice data (S1209, S1210). The first terminal device 110 transmits the acquired voice data to the server 200 (1211, S1212). The transmission source user presses the “send” button after the end of the speech, etc., whereby the first terminal device 110 acquires a signal of the end of the speech (S1213). Subsequently, the first terminal device 110 notifies the server 200 of the end of the voice data (S1214), adds the acquired direction information to the session end request, and transmits it to the server 200 (S1215). Here, the azimuth information may be the azimuth information acquired in S1203, or may be the azimuth information acquired again.

  The server 200 sends a data transmission notification corresponding to the session ID to the second terminal apparatus 120 which is the transmission destination, together with the direction information of the first terminal 110 which is the transmission source (S1216). The second terminal device 120 that has received the notification acquires the azimuth information, calculates the direction of the first terminal device 110, and displays the direction (S1217). Here, the azimuth information may be the azimuth information acquired in S1205 or may be the azimuth information acquired again. Also, voice data of the session ID is acquired from the server 200 (S1218). Here, the audio data subjected to sound image control may be acquired by the server 200 by designating and requesting the direction of the sound image. Then, the acquired audio data is reproduced (S1219).

  The server 200 also sends a data distribution notice corresponding to the session ID to terminal devices (other users) other than the transmission destination (S1220). The other terminal device that has received the notification displays that there is distribution (S1221), and acquires voice data of the session ID from the server (S1222) and reproduces it (S1223). Thus, the second terminal device 120 may calculate the direction using the received azimuth information and the like of the first terminal device 110.

  The sequence diagram of FIG. 13 illustrates a different example of the communication processing procedure between the first terminal device 110, the server 200, the second terminal device 120, and other terminals of the communication system according to the present embodiment. .

  Since S1201 to S1203 are the same as FIG. 12, the details are omitted. Subsequently, the server 200 transmits a direction information request to the second terminal apparatus 120 which is the transmission destination (S1301). The second terminal device 120 acquires azimuth information from the azimuth information acquisition unit 122 and returns the azimuth information to the server 200.

  The server 200 calculates the direction of the first terminal device 110 viewed from the second terminal device 120 based on the received direction information of the first terminal device 110 and the second terminal device 120, and calculates the calculated direction information and the text content. It transmits to a 2nd terminal device (S1302). Since S1205 to S1207 are the same as those in FIG. 12, the details are omitted.

  Also in S1208 to S1214, the same as in FIG. In S1303, a session end request is transmitted to the server 200 without the direction information. The server 200 transmits, to the second terminal apparatus, a data transmission notification corresponding to the session ID with the calculated direction information indicating the direction of the first terminal apparatus 110 as viewed from the second terminal apparatus 120 (S1304). The second terminal device 120 that has received the notification displays the direction of the first terminal device 110 (S1305). The subsequent steps S1218 to S1223 are the same as in FIG.

  In S1301 and S1302, the second terminal device 120 returns the direction information of the second terminal device 120 in response to the direction information request from the server 200, and then receives the direction information and the text information from the server 200. However, after receiving the text information notification from the server 200, it may request the server 200 to specify the direction information of the second terminal apparatus 120 and receive the direction information and the text information. The second terminal device 120 can also request the server 200 by designating the direction of the sound image in S1218. However, the direction information is specified and requested, and the server 200 calculates the direction and then the sound image It is also possible to send out controlled voice data.

  Thus, the direction may be calculated by the server 200 acquiring orientation information from the first terminal device 110 and the second terminal device 120. In addition, it may be designated as a transmission destination terminal apparatus when the transmission source terminal apparatus exists within a predetermined angle (for example, within plus or minus 15 degrees) as viewed from a certain terminal apparatus, for example. Furthermore, the terminal device of the transmission destination and the other terminal devices display or reproduce the direction of the transmission source terminal device based on the direction calculated from the acquired direction information, or by sound image control.

  The sequence diagram of FIG. 14 illustrates an example of a communication processing procedure between the first terminal device 110 and the second terminal device 120 of the communication system according to the present embodiment.

  First, when the user of the first terminal device (sender user) inputs a text message, the first terminal device 110 acquires the text content of the message (S1401), and then the sender user presses a button, etc. By doing this, the first terminal device 110 receives a signal of the "call mode" (S1402). Next, the first terminal device 110 acquires orientation information from the orientation information acquisition unit 112, and transmits an orientation information request to the second terminal device 120 (S1403). The second terminal device 120 acquires orientation information from the orientation information acquisition unit 122 and returns the information to the first terminal device 110.

  Subsequently, the first terminal device 110 calculates the direction of the first terminal device 110 viewed from the second terminal device 120 based on the direction information of the first terminal device 110 and the second terminal device 120, and the input text content The direction information thus calculated is transmitted to the second terminal device 120 (S1404). The second terminal device 120 displays the received text content and direction (S1405), synthesizes the voice of the received text content and the word representing the direction, and reads out (S1406).

  Next, the user of the first terminal device 110 (sender user) performs voice input of the utterance 1 toward the first terminal device 110, whereby the first terminal device 110 acquires the voice data 1 (S1407) . The first terminal device 110 transmits the acquired voice data 1 to the second terminal device 120 (S1408). The second terminal device 120 performs sound image control in accordance with the direction of the first terminal device and reproduces the received voice data 1 (S1409). The same applies to the utterance 2 (S1410, S1411, S1412). When the transmission source user presses the “Talk” button or the like after the end of the speech, the first terminal device 110 acquires a signal of the speech end (S1413). Subsequently, the first terminal device 110 notifies the second terminal device 120 of the end of the audio data (S1414).

  Next, when the transmission source user inputs a text message to the first terminal device 110, the first terminal device 110 acquires the text content of the message (S1415). Next, when the transmission source user presses a button or the like, the first terminal device 110 receives a “tweet mode” signal (S1416). Then, the first terminal device 110 transmits the input text content to the second terminal device 120 (S1417). The second terminal device 120 displays the received text content (S1418) and, at the same time, synthesizes the voice of the received text content and reads it out (S1419).

  In addition, the transmission source user performs voice input of the utterance 3 toward the first terminal device 110, whereby the first terminal device 110 acquires the voice data 3 (S1420). The first terminal device 110 transmits the acquired voice data 3 to the second terminal device 120 (S1421). The second terminal device 120 normally reproduces the received audio data 3 (without sound image control etc.) (S1422). The transmitter presses the “Talk” button or the like after the end of the speech, whereby the first terminal apparatus 110 acquires a signal of the speech end (S1423). Next, the first terminal device 110 notifies the second terminal device 120 of the end of voice data (S1424).

As described above, the first terminal device 110 and the second terminal device 120 may communicate with each other without going through the server 200, and the calculation of the direction may be performed in the first terminal device or the second terminal device.

  Next, an example of the direction calculation method in the case where the terminal device A (the transmission source and the first terminal device 110) makes a call to the terminal device B (the transmission destination = Mr. Suzuki and the second terminal device 120) Is shown in FIG. In this example, the north direction is 0 degrees, and the clockwise angle is calculated as the positive direction.

  The direction of the terminal device A (that is, the transmission source user) 305 degrees (or the opposite 125 degrees of 180 degrees) is transmitted as the direction information of the transmission source, and the direction of the user of the terminal device B (that is, the transmission destination user) is In the case of 90 degrees, from 125-90 = 35, the front of the destination user is the reference and the right 35 degrees is the direction of the source user. Note that the other terminal device C is irrelevant, and the direction of the transmission source user is not presented.

  Also, for example, FIG. 16 shows a case where the terminal device D (transmission source) makes a call toward the vicinity of the pillar number 8 where the beacon is present. The direction of the transmission source user, which is 300 degrees (or 120 degrees opposite to 180 degrees thereof), is transmitted as direction information, and the terminal device E and the terminal device F, which are location groups in the vicinity of the 8th beacon, Become. When the direction of the user of the terminal device E is 90 degrees, from 120-90 = 30, the front of the transmission destination user is the direction of the transmission source user by 30 degrees right based on that. When the direction of the terminal apparatus F is 180 degrees, the left 30 degrees is the direction of the source user from 120-180 = -30 on the basis of the front of the destination user. As described above, when transmitting to a group, the direction of the transmission source user is calculated for each transmission destination terminal device. In addition, the terminal device G which is not the vicinity of a beacon is irrelevant to these, and the direction of a transmission source user is not shown.

  An example of the direction calculation method in the case where the image acquisition unit is present in the terminal device of the transmission source and image data is received is shown in FIG. It is assumed that the user of the terminal device J, which is the transmission destination, attaches a bib describing "25" to the clothes. When the terminal device H (sender) makes a call to the “25” user identified in the camera image, the direction of the sender user is 315 degrees (or the opposite 135 degrees of 180 degrees) is the azimuth information When the direction of the user of the terminal device J is 90 degrees, the front of the destination user is from 135-90 = 45, and the right of 45 degrees is the direction of the source user based on the reference. In addition, if the position of the transmission destination user in the camera image is 10 degrees left on the transmission source user front reference, the direction 305 degrees of the transmission source user (or 125 degrees opposite to 180 degrees) is transmitted as direction information When the direction of the user of J is 90 degrees, 35 to the right with respect to the front of the destination user from 125−90 = 35 is the direction of the source user. The terminal device K is irrelevant to these, and the direction of the transmission source user is not presented. When a camera image exists, correction of the direction of the transmission destination may be performed with respect to the direction in which the camera (transmission source) faces, according to the position of the transmission destination identified in the image.

  In the present embodiment, the heading information (heading) is a character string representing a numerical value of 0 to 359.99 (360 degrees clockwise with north as 0, for example, west facing with “270”). The location information is latitude (location_latitude) and longitude (location_longtitude), decimal degrees in degrees (negative numbers for south latitude and west longitude and positive numbers for north latitude and east longitude, for example "35.690300,139.691700", near Tokyo Shinjuku, "- 22.903400, -43.339300 "and the character string representing Rio de Janeiro). The latitude and longitude may be expressed as decimal numbers in 60-minute minutes. Moreover, these measurement accuracy (heading_accuracy, location_accuracy) may be accompanied as incidental information. Further, in the server 200, the orientation information may be stored in the storage unit 230 in association with the combination of the transmission source and the transmission destination, and the orientation information may be continuously assigned to the same set.

  In addition, the distance is calculated from the position information acquired by the position information acquisition unit of the first terminal device 110 and the position information acquired from the second terminal device 120, and the distance is greater than a predetermined distance (a correct direction is given by visual confirmation) When the distance to the source user is short and the distance from the center of the place group (corresponding beacon) is long (it is judged that the place group does not match the place group intended by the sender) In this case, the direction information may not be used, and the output may be made in the "tweet mode". Further, when the calculated distance is longer than the predetermined distance, the accuracy of the direction calculated based on the position information can be secured, so the direction calculated based on the direction information may be used instead. Although processing such as position information is performed by the server 200 in the present embodiment, the same applies to the case where the direction calculation is performed on the first terminal device 110 or the second terminal device 120 side.

  When calculating the distance and direction between two latitudes and longitudes using position information, it is possible to calculate with high accuracy based on the earth ellipsoid, but in general, it may be calculated by sphere approximation calculation . In the present embodiment, high accuracy in both distance and orientation is not required, and it is assumed to be used at a relatively close distance, so there is no problem even if approximate calculation is performed assuming that two points are on a plane.

  Furthermore, even if the time difference between the time information of the received message and the current time is large (it is determined that the time is likely to be out of the right after the transmission), no direction information is used, and the "tweet mode" is used. You may output it. Further, when distributing a message to other than the transmission destination user in the "call mode", when the transmission destination information does not match the own terminal, the communication mode is switched to the "tweet mode", and the output is performed without using the azimuth information. For example, when "call" addressed to "Mr. Takahashi" is delivered while all "sales" share messages and used in "tweet mode", all "sales" messages receive , The terminal other than "Mr. Takahashi" outputs as "Meeting mode", and the terminal of "Mr. Takahashi" outputs as "Call mode".

  Subsequently, the processing unit 220 specifies the message, the input time of the message, the name of the transmission source user, etc. via the network interface 210, the direction of the transmission source user as the transmission destination, and It transmits to the terminal device 120 (S707). In the "calling mode", when voice or text is transmitted only to the transmission destination, only the terminal device identification ID of the transmission destination may be designated, but when distributing the message to other terminal devices, the distribution destination In addition to the designation of, a terminal identification ID (heading_target) representing a transmission destination is also designated. At this point, the processing in the “call mode” in the server 200 ends.

<< Operation of the second terminal device 120 of the communication system >>
In the “call mode”, the second terminal device 120 receives, from the server 200, a message, a message input time, a sender user's name, a sender user's direction, and the like. After that, the received text message, the sender user name, the calculated direction, the message input time, etc. are displayed on the display unit 124, or in the case of the speaker or binaural earphone in which the audio output unit 128 can perform stereo reproduction. , And control the sound image control of the voice message (or the voice synthesis sound of the text message) and reproduce from the voice output unit 128. here. Both text display and audio reproduction may be performed. In addition, when there is no message, the direction may be displayed on the display unit 124, or the sound as a cue may be reproduced by sound image control. When sound image control can not be performed, such as in the case of a single-ear earphone, accompanied by a word indicating the direction, for example, "Mr. ○ ○ is calling from the diagonally right back." It is also possible to present the direction by speech synthesizing a word such as “from Mr. OO in the direction of”, and adding and reproducing it before or after the received voice message. Also, the direction may be indicated by vibration or touch.

  In the "tweet mode", a text message is simply displayed on the display unit 124 or a voice message is reproduced from the voice output unit 128 without any information on the direction of the transmission source.

  The transmission destination terminal is connected to the server 200 via the wireless network, for example, by the WebSocket API, and receives notification when a voice message or text message has been sent. In the case of a voice message or text message in "tweet mode", information identifying a session (session key), message input time, first terminal device identification ID (or user ID of first terminal device), second terminal device The message is acquired with the identification ID (or the user ID of the second terminal device or the group ID) or the like, and in the case of the voice message or text message in the “call mode”, the azimuth information or direction of the transmission source is also acquired. If there is a voice message, voice data corresponding to the session key can be obtained (GET voice). In the “call mode”, if the direction information or direction of the transmission source can be acquired and transmitted by display or voice, it can be recognized that someone is calling from the calculated direction.

  Subsequently, an example of the screen display of the terminal device of the smartphone type is shown in FIG. In this example, the "sales group" in which messages are currently exchanged is displayed at the top of the screen. In the middle of the screen, the history of texts and voices being transmitted and received is displayed. The balloon from the left is a message received, the balloon from the right is a message sent from the own terminal, and the lower part of the screen is the latest. In addition, according to the balloon, there is a display such as "Sato> Suzuki 13:25" and an arrow in the left diagonal direction. At 13:25, from Sato to Mr. Suzuki (own terminal), from the left diagonal It indicates that there was a call for "Saun".

  The screen display example of FIG. 19 is a screen display when the user of the terminal device turns diagonally to the left in response to Mr. Sato's call, and the arrow at Mr. Sato becomes the top direction of the screen, and the top row of the screen Also changed to "Sato-san". Then, it is displayed that the user responds by saying "Yes, I will go over there" by pressing the "calling" button.

  Next, FIG. 20 shows an example of the screen display of the tablet type terminal device. The call of Mr. Sato from the left diagonal direction is displayed, and the tweets of Mr. Sato and the like before that are also displayed to flow sequentially toward the left upper direction. On the other hand, previous ones such as Mr. Takahashi's call from the front direction flow in the upward direction one by one. Then, "Please wait a minute." In response to Takahashi-san's call is displayed at the bottom of the screen.

  The screen display example of FIG. 21 shows the screen display when the screen is turned diagonally to the left in a form that responds to Sato's call, the screen rotates, and Sato's remarks are directed upward on the screen by Mr. Takahashi's The remark is shifted to the upper right of the screen. Then, it is displayed that the user responds by saying "Yes, I will go over there" by pressing the "calling" button. The message of the own terminal flows down the screen.

  With such display and sound image control, the user of each terminal device designated as the transmission destination can know the direction of the transmission source of the call, and if the transmission source can be visually confirmed, communication by gesture or the like becomes possible. In addition, even if the direction information is acquired without acquiring the position information or the camera image, it is possible to present the direction of the terminal device of the transmission source only by performing a simple calculation process by the server or the terminal device.

  In the present embodiment, the first terminal device 110 and the second terminal device 120 communicate with the server 200, and mainly process the user master and name data table processing, mode determination, orientation information processing, etc. Although described as being performed by the processing unit 220 of 200, these processes can be performed on the terminal device side, and the present invention is not limited to the configuration in which the server 200 intervenes. For example, communication may be performed between the first terminal device 110 and the second terminal device 120 via a network, and the control unit 125 of the second terminal device 120 may perform mode determination and processing of orientation information.

  Moreover, in this embodiment, although it demonstrated as what does not transmit or receive azimuth | direction information in "tweet mode", discrimination | determination of "call mode" and "tweet mode" is not restricted to this. It is also possible to add information indicating the mode separately, including the direction information. Also, the mode may be identified by the presence or absence of the information indicating the transmission destination.

Second Embodiment
FIG. 22 is a block diagram showing the configuration of a communication system according to the second embodiment. The communication system of this embodiment is configured by connecting a terminal device group 100 including one or more image devices 130 and a server 200 via a network. However, the configuration of the terminal devices in the terminal device group 100 is the same as that of the first terminal device 110 or the image device 130 described in the first embodiment or the movable device 140 and the moving body 150 described later. Also, since the terminal device can be a transmission source and a transmission destination, only the configuration of the imaging device 130 will be described, and the details regarding other terminal devices will be omitted. Moreover, since the configuration of the server 200 is also the same as that of the first embodiment, the details will be omitted.

  The image device 130 includes a network interface 131, an orientation information acquisition unit 132, an image acquisition unit 133, a display unit 134, a control unit 135, and an input unit 136. The network interface 131, the direction information acquisition unit 132, the display unit 134, the control unit 135, and the input unit 136 are the same as in the first embodiment, and thus the details thereof will be omitted. The image acquisition unit 133 is an optical input device such as a camera or an infrared sensor, and acquires an image around the image device 130.

  Next, a processing flow of the communication system according to the second embodiment will be described with reference to FIG. However, since the identification of the transmission destination terminal device in the server 200 and the terminal device, the method of calculating the direction, and the like are the same as in the first embodiment, the details will be omitted.

  First, the button "Display this" is tapped from the input unit 116 of the first terminal device 110 which is the transmission source terminal device (in this case, the display unit 114 doubles as the input unit 116), A signal indicating that the apparatus 110 is to be displayed is acquired (S2001). Here, the user of the first terminal device 110 (transmission source user) is in a state in which the face with the first terminal device 110 or the headset 110 a or the like is directed to the image device 130 of the transmission destination. The input signal is not limited to that by the button, but may be anything as long as it can send a signal such as text input or voice. Subsequently, the direction information acquisition unit 112 of the first terminal device 110 acquires the direction in which the first terminal device 110 is facing, and the acquired direction of the first terminal device 110 via the network interface 111, the first terminal The terminal device identification ID and the like of the device 110 are transmitted (S2002).

  Next, the server device 200 or the image processing device 130 which is the transmission destination terminal device receives a signal from the terminal device 110 and the like, and the processing unit 220 of the server 200 or the control unit 135 of the image processing device 130 which is the transmission destination terminal device. From the received terminal device identification ID, the first terminal device 110, which is the transmission source terminal device, is identified 110 (S2003). Next, the orientation information acquisition unit 132 in the imaging device 130 acquires the orientation of the imaging device 130 (S2004). Subsequently, the processing unit 220 of the server 200 or the control unit 135 of the image device 130 calculates the direction of the first terminal device 110 viewed from the image device 130 based on the acquired orientation (S2005).

  Finally, the control unit 135 of the image device 130 changes the image shown on the display unit 134 based on the calculated direction (S2006). For example, the image is enlarged (digital zoom) centering on the direction of the first terminal device 110 as viewed from the image device 130. Alternatively, a movable unit (not shown) in FIG. 19 is provided, and the image acquisition unit 133 is mechanically rotated to face the first terminal device 110 to change the focal length of the lens to zoom the image (optical zoom). Furthermore, the control unit 135 of the image device 130 acquires the image information in which the direction of the first terminal device 110 is reflected in the direction of the first terminal device 110 in the direction information acquiring unit 132 of the image device 130 acquired. It transmits via the network interface 131 together with the direction information etc. of the movable part. Then, in the first terminal device 110, the received image information is output as that from the direction of the image device 130. In addition, when not providing a movable part, you may transmit the azimuth | direction 180 degree | times opposite to the azimuth | direction of the received 1st terminal device 110 as azimuth | direction information of the imaging device 130. FIG.

Alternatively, when the display unit 134 is provided with a plurality of display surfaces and the images acquired from the plurality of image acquisition units 133 are displayed, the image acquisition unit in the direction of the first terminal device 110 or the first terminal device It switches to the image display of only the display part of the direction of 110. Thus, the process of the communication system according to the second embodiment ends.

  With such a mechanism, the image device designated as the transmission destination can know the direction of the terminal device which is the transmission source of the signal, and an effective effect such as display of an image cooperated by users of a plurality of terminal devices Is possible. At this time, even if the position information is not acquired, if the azimuth information is acquired, the direction of the terminal device of the transmission source can be known only by performing simple calculation processing by the server or the terminal device.

Third Embodiment
FIG. 24 is a block diagram showing the configuration of a communication system according to the third embodiment. The communication system of the present embodiment is configured by connecting a terminal device group 100 including one or more movable devices 140 and a server 200 via a network. However, the configuration of the terminal devices in the terminal device group 100 is the same as that of the first terminal device 110 or the imaging device 130 or the movable device 140 described in the first embodiment, or the movable body 150 described later. Also, since the terminal device can be a transmission source and a transmission destination, only the configuration of the movable device 140 will be described, and the details regarding other terminal devices will be omitted. Moreover, since the configuration of the server 200 is also the same as that of the first embodiment, the details will be omitted.

  The movable device 140, for example, is a robot capable of changing the direction of the face or body, a light capable of changing the direction of illumination, a water gun capable of changing the direction of the nozzle, and reaching for a certain location It is an apparatus such as a robot arm capable of changing the acting direction, and includes a network interface 141, an orientation information acquisition unit 142, a movable unit 143, and a control unit 144. The network interface 141, the direction information acquisition unit 142, and the control unit 144 are the same as in the first embodiment, and thus the details will be omitted. The movable unit 143 performs a mechanical operation in the direction of the transmission source terminal device.

  Next, the processing flow of the communication system according to the third embodiment will be described with reference to FIG. However, since the identification of the transmission destination terminal device in the server 200 and the terminal device, the method of calculating the direction, and the like are the same as in the first embodiment, the details will be omitted.

  First, the button "Turn here" is tapped from the input unit 116 of the first terminal apparatus 110 which is the transmission source terminal apparatus (in this case, the display unit 114 doubles as the input unit 116), A signal indicating that the user wants the terminal device 110 to face is obtained (S2201). Here, the user of the first terminal device 110 (transmission source user) is in a state in which the face on which the first terminal device 110 or the headset 110a is attached is directed to the movable device 140 of the transmission destination. The input signal is not limited to that by the button, but may be anything as long as it can send a signal such as text input or voice. Subsequently, the direction information acquisition unit 112 of the first terminal device 110 acquires the direction in which the first terminal device 110 is facing, and the acquired direction of the first terminal device 110 via the network interface 111, the first terminal The terminal device identification ID and the like of the device 110 are transmitted (S2202).

  Next, the server apparatus 200 or the movable apparatus 140 which is the transmission destination terminal apparatus receives a signal or the like from the terminal apparatus 110, and the processing unit 220 of the server 200 or the control unit 144 of the movable apparatus 140 which is the transmission destination terminal apparatus. Identifies the first terminal apparatus 110 which is the transmission source terminal apparatus from the received terminal apparatus identification ID (S2203). Next, the orientation information acquisition unit 142 in the movable device 140 acquires the orientation of the movable device 140 (S2204). Subsequently, the processing unit 220 of the server 200 or the control unit 144 of the movable device 140 calculates the direction of the first terminal device 110 viewed from the movable device 140 based on the acquired orientation (S2205).

  Finally, the control unit 144 of the movable device 140 operates the movable portion 143 based on the calculated direction (S2206). For example, the robot's face or body faces in the direction of the first terminal 110 viewed from the movable device 140, the light rotates and illuminates, the water gun nozzle rotates and waters up, and the robot arm extends. To Furthermore, the control unit 144 of the movable device 140 generates, as a response to the first terminal device 110, text information and voice information such as “Mr. ○, found!” And the like, and is acquired by the orientation information acquisition unit 142. It transmits via the network interface 141 with the azimuth | direction information etc. of the movable apparatus 140. FIG. Thereafter, the first terminal device 110 outputs the received text information and voice information of the response while presenting the direction of the movable device 140. Also, in the case where a signal indicating that the user wants the user to face other than the direction of the first terminal device 110 is obtained by, for example, tapping the button "to look at" from the input unit 116 of the first terminal device 110, A direction different from the first terminal device 110 may be selected to operate (avoid the direction of the first terminal device 110). Thus, the process of the communication system according to the third embodiment ends.

  By such a mechanism, the mobile device designated as the transmission destination can operate based on the direction of the transmission source. Further, even if the azimuth information is acquired without acquiring the position information or the camera image, the direction of the terminal apparatus of the transmission source can be known only by performing simple calculation processing by the server or the terminal apparatus.

Fourth Embodiment
FIG. 26 is a block diagram showing the configuration of a communication system according to the fourth embodiment. The communication system of the present embodiment is configured by connecting a terminal device group 100 including one or more mobile units 150 and a server 200 via a network. However, the configuration of the terminal devices in the terminal device group 100 is the same as that of the first terminal device 110 or the imaging device 130 or the movable device 140 or the movable body 150 described in the first embodiment, and any terminal device transmits Since the terminal device can be a source / destination terminal, only the configuration of the mobile unit 150 will be described, and the details of other terminal devices will be omitted. Moreover, since the configuration of the server 200 is also the same as that of the first embodiment, the details will be omitted.

  The moving body 150 is a movable device on which, for example, wheels and the like are installed, and includes a network interface 151, an azimuth acquiring unit 152, a moving unit 153, and a control unit 154. The network interface 151, the direction acquisition unit 152, and the control unit 154 are the same as in the first embodiment, and thus the details are omitted. The moving unit 153 moves the moving body 150 in the direction of the transmission source terminal device.

  Next, a processing flow of the communication system according to the fourth embodiment will be described with reference to FIG. However, since the identification of the transmission destination terminal device in the server 200 and the terminal device, the method of calculating the direction, and the like are the same as in the first embodiment, the details will be omitted.

  First, the "coming here" button is tapped from the input unit 116 of the first terminal device 110 which is the transmission source terminal device (in this case, the display unit 114 doubles as the input unit 116), A signal indicating that the user wants to come in the direction of the device 110 is acquired (S2401). Here, the user of the first terminal device 110 (transmission source user) is in a state in which the face with the first terminal device 110 or the headset 110 a or the like is directed to the mobile unit 150 of the transmission destination. The input signal is not limited to that by the button, but may be anything as long as it can send a signal such as text input or voice. Subsequently, the direction information acquisition unit 112 of the first terminal device 110 acquires the direction in which the first terminal device 110 is facing, and the acquired direction of the first terminal device 110 via the network interface 111, the first terminal The terminal device identification ID and the like of the device 110 are transmitted (S2402).

  Next, the mobile unit 150 which is the server apparatus 200 or the transmission destination terminal apparatus receives a signal or the like from the terminal apparatus 110, and the control unit 154 of the mobile unit 150 which is the processing unit of the server 200 or the transmission destination terminal apparatus. The first terminal apparatus 110 which is the transmission source terminal apparatus is specified from the received terminal apparatus identification ID (S2403). Next, the orientation acquisition unit 152 in the mobile unit 150 acquires the orientation in which the mobile unit 150 is facing (S2404). Subsequently, the processing unit 220 of the server 200 or the control unit 154 of the mobile unit 150 calculates the direction of the first terminal device 110 viewed from the mobile unit 150 based on the acquired orientation (S2405).

  Next, the control unit 154 of the moving body 150 moves the moving unit 153 based on the calculated direction (S2406). Finally, the “stop” button is tapped from the input unit 116 of the first terminal device 110, and a signal to the effect that it wants to stop is acquired (S2407), and the mobile 150 receives the signal via the network. The control unit 154 stops the moving unit 153. (S2408). Here, for example, when the moving object 150 is a vehicle and the direction of movement and the direction of the exit are different, the orientation of the first terminal device 110 acquired in S2401 or the first acquired in S2408 again Based on the direction of the terminal device 110, the direction of the entrance is adjusted. When the mobile object is a robot, the face may be directed to the direction of the first terminal device 110 or the ear may be directed. Also, a signal indicating that the user wants to move other than in the direction of the first terminal device 110 by tapping the "open route" and "avoid to the left" button from the input unit 116 of the first terminal device 110, etc. In the case where the second terminal device 110 has acquired, it may be operated by selecting a direction different from that of the first terminal device 110 (for example, moving in the direction of ± 90 degrees with respect to the direction of the first terminal device 110). In addition, position information acquisition parts, such as GPS, and image acquisition parts, such as a camera and an infrared sensor, are provided in the 1st terminal device 110 or the moving body 150, and based on the position of the 1st terminal device 110 which acquired or analyzed Even if there is no stop signal, the mobile unit 150 may be controlled to stop, or the mobile unit 150 may be moved more accurately. Also, immediately to the other party's moving object 150 detected from the image (without tapping a button etc.) "Avoid to the left" (or "Avoid to the right" if detected on the right in the image) signal It may be sent out and evacuated. Thus, the process of the communication system according to the fourth embodiment ends.

  According to such a mechanism, the mobile designated as the transmission destination can know the direction of the transmission source of the call, and can move based on the direction of the transmission source. In addition, when the moving body is a vehicle, the direction of the moving body can be adjusted and stopped so that the user of the transmission source can easily ride. Furthermore, even if the direction information is acquired without acquiring the position information or the camera image, the direction of the terminal device of the transmission source can be known only by performing simple calculation processing by the server or the terminal device.

  The method described in the above embodiment can be executed as a program that can be executed by a computer, such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), magneto-optical disk (CD-ROM, DVD etc.) It can also be stored and distributed in a storage medium such as MO) or semiconductor memory.

  Here, as a storage medium, any storage format may be used as long as it can store a program and can be read by a computer.

  In addition, an operating system (OS) operating on a computer based on an instruction of a program installed in the computer from a storage medium, MW (middleware) such as database management software, network software, etc. realize this embodiment. Part of each process may be executed.

  Furthermore, the storage medium in the present embodiment is not limited to a medium independent of a computer, and includes a storage medium downloaded and stored or temporarily stored a program transmitted by a LAN, the Internet, or the like.

  Further, the storage medium is not limited to one, and the processing in the present embodiment may be executed from a plurality of media, and the storage medium in the present embodiment may have any configuration.

  Note that the computer in the present embodiment executes each process in the present embodiment based on a program stored in a storage medium, and an apparatus comprising a single personal computer or the like, and a plurality of apparatuses are connected via a network. It may be any configuration such as a system.

  Further, each storage device of the present embodiment may be realized by one storage device or may be realized by a plurality of storage devices.

  The computer in the present embodiment is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, etc. included in an information processing device, and generically refers to devices and devices capable of realizing the functions of the present embodiment by a program. ing.

  While certain embodiments of the invention have been described above, these embodiments have been presented by way of example only, and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the description. These embodiments and modifications thereof are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

100: Terminal device group 110: First terminal device 110a: Terminal main body 110b: Wearable terminal 120: Second terminal device 111, 121: Network interface 112, 122: Orientation information acquisition unit 113, 123: Position information acquisition unit 114, 124 ... display unit 115, 125 ... control unit 116, 126 ... input unit 117, 127 ... voice acquisition unit 118, 128 ... voice output unit 130 ... image device 131 ... network interface 132 ... orientation information acquisition unit 133 ... image acquisition unit 134 ... Display unit 135 ... Control unit 136 ... Input unit 140 ... Movable device 141 ... Network interface 142 ... Direction information acquisition unit 143 ... Movable unit 144 ... Control unit 150 ... Mobile body 151 ... Network interface 152 ... Orientation information acquisition unit 153 ... Moving unit 154 ... control unit 200 ... server 10 ... network interface 220 ... processing unit 230 ... memory 231 ... User Master 232 ... Beacon master 233 ... location-linked tables 234 ... nickname Dictionary

Claims (17)

A communication system including a first terminal and a second terminal connected by a network, the communication system comprising:
The first terminal device is
An azimuth acquisition unit that acquires an azimuth at which the first terminal device is facing;
A first communication unit that transmits the acquired direction in which the first terminal apparatus is facing;
Equipped with
The second terminal device is
A second communication unit that receives an azimuth from the first terminal device to the first terminal device;
An azimuth acquisition unit that acquires an azimuth at which the second terminal device is facing;
A direction calculation for calculating the direction of the first terminal device as viewed from the second terminal device from the received direction in which the first terminal device is facing and the direction in which the acquired second terminal device is facing Department,
A display unit for displaying the calculated direction of the first terminal device, and an audio output unit for presenting with sound;
A communication system comprising A communication system including a first terminal and a second terminal connected by a network, the communication system comprising:
The first terminal device is
An azimuth acquisition unit that acquires an azimuth at which the first terminal device is facing;
A first communication unit that transmits the direction of the first terminal device calculated by the reception and direction calculation unit of the direction in which the second terminal device is facing;
A direction calculation unit that calculates the direction of the first terminal device viewed from the second terminal from the direction in which the first terminal device is facing and the direction in which the second terminal device is received; ,
Equipped with
The second terminal device is
An azimuth acquisition unit that acquires an azimuth at which the second terminal device is facing;
A second communication unit that transmits the acquired direction in which the second terminal device is facing and receives the direction of the first terminal device calculated by the direction calculation unit of the first terminal device;
A display unit for displaying the direction of the first terminal apparatus received by the second communication unit, or an audio output unit for presenting with a sound;
A communication system comprising: The first terminal device comprises an input unit for acquiring a message;
The first communication unit transmits the obtained message.
The second communication unit receives the message from the first communication unit,
The communication system according to claim 1, wherein the display unit or the audio output unit presents the received message. The message is a voice message,
The communication system according to claim 3, wherein the voice output unit presents the direction of the first terminal device by performing sound image control on the voice message and reproducing the voice message. The communication system according to any one of claims 1 to 4, wherein the display unit displays the direction of the first terminal device by the direction of the arrow. The input unit acquires information on a transmission destination.
The first terminal device includes a control unit that specifies a transmission destination from the acquired information on the transmission destination,
The communication system according to any one of claims 1 to 5, wherein the first communication unit transmits data to the transmission destination specified by the control unit. A communication system comprising a plurality of terminal devices including a first terminal device and a server and a second terminal device connected by a network,
The first terminal device is
An azimuth acquisition unit that acquires an azimuth at which the first terminal device is facing;
An input unit for acquiring information on a transmission destination;
A first communication unit that transmits information on the direction in which the acquired first terminal device is facing and the transmission destination;
Equipped with
The server is
A server communication unit that receives from the first communication unit the direction in which the first terminal is facing and information on the transmission destination;
A processing unit that specifies a second terminal apparatus that is a transmission destination based on the received information on the transmission destination; and the server communication unit further transmits a second terminal apparatus specified by the processing unit. Get the direction that your terminal is facing,
Said server further
A direction calculation unit that calculates the direction of the first terminal device viewed from the second terminal device from the received direction in which the first terminal device is facing and the direction in which the second terminal device is facing And
The second terminal device is
An azimuth acquisition unit that acquires an azimuth at which the second terminal device is facing;
A second communication unit that transmits the acquired direction in which the second terminal apparatus is facing to the server and receives the calculated direction of the first terminal apparatus;
A display unit for displaying the calculated direction of the first terminal device or an audio output unit for outputting by sound;
A communication system comprising A communication system comprising a plurality of terminal devices including a first terminal device and a server and a second terminal device connected by a network,
The first terminal device is
An azimuth acquisition unit that acquires an azimuth at which the first terminal device is facing;
An input unit for acquiring a message,
A first communication unit for transmitting a direction in which the acquired first terminal apparatus is facing and a message;
Equipped with
The server is
A server communication unit that receives the direction in which the first terminal faces from the first communication unit and the message;
A processing unit that analyzes the received message, extracts information related to a transmission destination, and identifies a second terminal apparatus that is a transmission destination based on the extracted information related to the transmission destination; Furthermore, the direction in which the second terminal device is facing is acquired from the second terminal device specified by the processing unit,
Said server further
A direction calculation unit that calculates the direction of the first terminal device viewed from the second terminal device from the received direction in which the first terminal device is facing and the direction in which the second terminal device is facing And
The second terminal device is
An azimuth acquisition unit that acquires an azimuth at which the second terminal device is facing;
A second communication unit that transmits the acquired direction in which the second terminal apparatus is facing to the server and receives the calculated direction of the first terminal apparatus;
A display unit for displaying the calculated direction of the first terminal device or an audio output unit for outputting by sound;
A communication system comprising The second terminal device is
An image acquisition unit configured to acquire an image around the second terminal device;
The image acquisition unit changes the image displayed on the display unit based on the calculated direction of the first terminal device.
The communication system according to any one of claims 1 to 8. The second terminal device is
And a movable unit configured to change the orientation or position of the second terminal device based on the calculated direction or position of the first terminal device.
The communication system according to any one of claims 1 to 9. The communication system according to any one of claims 6 to 10, wherein the information related to the transmission destination is identification information of a transmission destination terminal apparatus or identification information of a user. The terminal device or the server includes a storage unit that associates and stores identification information of the terminal device and a user name, a group name to which the user belongs, or a user's nickname.
The communication system according to any one of claims 6 to 11, wherein the information on the transmission destination is a user name, a group name to which the user belongs, or a name of the user. The terminal device receives a signal from a beacon installed at each location;
The terminal device or the server includes a storage unit that associates a beacon with the identification information of the terminal device that has received the signal from the beacon and stores the associated information as a location group.
The communication system according to any one of claims 6 to 12, wherein the information on the destination is a place group. A program executed by a communication system including a first terminal and a second terminal connected by a network, the program comprising:
The first terminal device is
An azimuth acquisition function of acquiring an azimuth at which the first terminal device is facing;
A first communication function of transmitting the acquired direction in which the first terminal apparatus is facing;
Equipped with
The second terminal device is
A second communication function for receiving an azimuth from which the first terminal device faces from the first terminal device;
An azimuth acquisition function of acquiring an azimuth at which the second terminal device is facing;
A direction calculation for calculating the direction of the first terminal device as viewed from the second terminal device from the received direction in which the first terminal device is facing and the direction in which the acquired second terminal device is facing Function,
A display function for displaying the calculated direction of the first terminal device or an audio output function for presenting with sound;
A communication program that causes a computer to comprise. A program executed by a communication system including a first terminal and a second terminal connected by a network, the program comprising:
The first terminal device is
An azimuth acquisition function of acquiring an azimuth at which the first terminal device is facing;
A first communication function of transmitting the direction of the first terminal device calculated by the reception and direction calculation function of the direction in which the second terminal device is facing;
A direction calculation function of calculating the direction of the first terminal device viewed from the second terminal from the direction in which the first terminal device is facing and the direction in which the second terminal device is received; ,
Equipped with
The second terminal device is
An azimuth acquisition function of acquiring an azimuth at which the second terminal device is facing;
A second communication function of transmitting the acquired direction in which the second terminal apparatus is facing and receiving the direction of the first terminal apparatus calculated by the direction calculation function of the first terminal apparatus;
A communication program causing a computer to have a display function to display the direction of the first terminal device received by the second communication function or an audio output function to present with sound. A program executed by a communication system comprising a plurality of terminal devices including a first terminal device and a server and a second terminal device connected by a network,
The first terminal device is
An azimuth acquisition function of acquiring an azimuth at which the first terminal device is facing;
An input function to obtain information on the destination,
A direction in which the acquired first terminal device is facing, information on the transmission destination, and a first communication function to transmit;
Equipped with
The server is
A server communication function of receiving an azimuth in which the first terminal is facing and information on the transmission destination from the first communication function;
And a processing function of specifying a second terminal apparatus that is a transmission destination based on the received information on the transmission destination, and the server communication function further transmits a second terminal apparatus specified by the processing function. Get the direction that your terminal is facing,
Said server further
Direction calculation function of calculating the direction of the first terminal device viewed from the second terminal device from the received direction in which the first terminal device is facing and the direction in which the second terminal device is facing And
The second terminal device is
An azimuth acquisition function of acquiring an azimuth at which the second terminal device is facing;
A second communication function of transmitting the acquired direction in which the second terminal apparatus is facing to the server and receiving the calculated direction of the first terminal apparatus;
A display function for displaying the calculated direction of the first terminal device or an audio output function for outputting by sound;
A communication program that causes a computer to comprise. A program executed by a communication system comprising a plurality of terminal devices including a first terminal device and a server and a second terminal device connected by a network,
The first terminal device is
An azimuth acquisition function of acquiring an azimuth at which the first terminal device is facing;
With input function to get message,
A first communication function for transmitting a direction in which the acquired first terminal apparatus is facing and a message;
Equipped with
The server is
A server communication function for receiving the direction in which the first terminal faces from the first communication function and the message;
A processing function of analyzing the received message, extracting information on a transmission destination, and identifying a second terminal apparatus as a transmission destination based on the extracted information on the transmission destination; Furthermore, the direction in which the second terminal device is directed from the second terminal device specified by the processing function is acquired,
Said server further
Direction calculation function of calculating the direction of the first terminal device viewed from the second terminal device from the received direction in which the first terminal device is facing and the direction in which the second terminal device is facing And
The second terminal device is
An azimuth acquisition function of acquiring an azimuth at which the second terminal device is facing;
A second communication function of transmitting the acquired direction in which the second terminal apparatus is facing to the server and receiving the calculated direction of the first terminal apparatus;
A display function for displaying the calculated direction of the first terminal device or an audio output function for outputting by sound;
A communication program that causes a computer to comprise.

Images