JP2018067785A - Communication robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot call center providing service by dialog via a telepresence robot, by connecting a client with a proper operator.SOLUTION: A telepresence robot 101 performs call transmission for starting conversation of a client and a robot, on the basis of the information obtained from a media I/O device 104 and a position determination device 105. A call center device 102 has a multi-modal communication server 111 for assigning appropriate one of operator terminals 103 to a call received from an approach controller 106, and mediating and recording communication using audio video and telepresence robot body motion between a call source telepresence robot 101 and the assigned operator terminal 103. The operator terminal 103 has a voice I/O device 112, a remote operation device 113, and an operator communication controller 114 for controlling communication with the call center device 102.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system for constructing and operating a communication robot.

Robots that can communicate with humans are called communication robots. In particular, humanoid robots with human-like faces and bodies are expected to have a function that allows them to perform conversations on behalf of people by speaking with voice or using gestures like humans.

Among communication robots, robots called telepresence robots do not operate completely autonomously, but operate according to a remote operation or remote instruction by a person and act in a place away from the person on behalf of that person. It is characterized in that it can be performed (see Non-Patent Document 1).

The telepresence robot includes a communication device so that it can act in a remote place as a human agent, and includes a device that exchanges video and audio between remote locations. An example of a telepresence robot's function is to display the person's face image on the remote robot side and show it to the person in front of the robot, to output the person's voice on the robot side, and conversely There is a function to send the video and voice of the person in front of the person to the person.

Also, a person can move the telepresence robot closer to the person who wants to talk.

The telepresence robot can be regarded as an extension of the telephone and video conference system from the viewpoint of remote communication. People can send their voices to remote places using the telephone. Also, if you use a video conferencing system, you can deliver your video and voice to remote locations.

In particular, telephones have been widely used as a means of remote communication from daily use to business. For example, a call center system is a typical example of a telephone service used in business. By automatically connecting a call from a customer to an appropriate one of a large number of waiting operators, a call center system is connected between a customer and a company. Communication is made efficient (see Patent Document 1). In a call center system, it is a main process to distribute a call sent by a customer to call a call center to an appropriate operator, and an automatic call distribution device (ACD) is a device that performs this distribution automatically. being called.

On the other hand, a person can use a telepresence robot to have a robot that can act as a substitute for himself in a remote place. Telepresence robots not only deliver voices like telephones, but also deliver voices and images like video conferencing systems, and use the body to move, show facial expressions, and point at hands. You can also communicate.

Japanese Patent No. 3934543

Guizzo, E., “When my avatar went to work”, IEEE Spectrum, 2010, p.26-50

By applying the telepresence robot to the business for the purpose of providing services to customers, it becomes possible to create new services and solve problems that cannot be realized simply by using conventional telephone and video conferencing systems.

As an example of the service, there is a guide robot service that guides customers' directions. This is a service that provides guidance for customers who are in trouble at a tourist destination without knowing the route by using a telepresence robot with a guide at another location to speak, point and guide.

If the route guidance service can be provided via the telepresence robot, the guide does not need to be in the field, and can respond while at home or in the office. In particular, local operations such as road guidance have been impossible to do by telecommuting so far, but if this service can be provided by a telepresence robot, this problem can be solved.

Also, if the customer is in front of the telepresence robot and the guide responds remotely by remote operation, and if there is no customer, the remote operation can be separated and other work can be performed, and the unused time of the customer is wasted. The problem that has been solved can be solved and work efficiency can be improved.

In addition, if the remote operation of the telepresence robot can be automatically connected to a person who is free of hands among a plurality of guides instead of a specific one, the operation rate of the guides can be increased. it can.

The guide robot service for guiding customers as described above can be applied not only to sightseeing spots, but also to guidance work in a wide area such as a theme park, an exhibition hall, and a supermarket.

As an example of a service using a telepresence robot, a bell robot service that performs the work of a bell staff in a hotel can also be considered. The business required for Bell Robot and the advantage of using it are close to the above-mentioned guide robot service, but it differs in that Bell Robot is strongly required to learn the face of a hotel customer. By using a telepresence robot, the customer's face image can be transmitted to a human Belstaff in a remote location, so it is possible to judge whether this Belstaff is a good customer, and the robot has a face recognition function If you have a robot, you can determine who the robot is in advance, and then call the customer's dedicated bell staff at a remote location.

As an example of a service using a telepresence robot, a counter robot service that can act as a receptionist can be considered. Improve operational efficiency by automatically connecting to those who are available among multiple receptionists in operations such as bank receptions, public institutions, local government offices, and major company receptions. it can. Instead of having to move a lot during reception work, it is necessary to be able to present detailed text and images to the customer, such as documents that the customer needs to fill out, and a map or information board that shows where the customer should go. In this case, the display of the telepresence robot is required to be able to display data by remote operation.

As an example of a service using a telepresence robot, a robot concierge service is also conceivable. In places where libraries and museums, department stores, airports, large-scale home appliance stores, home centers, etc. have large and specialized functions and services, you can not only walk along the site together, but also hold collections and products. A concierge service that can respond to the customer's consultation in detail, such as details and details of necessary procedures, is required. Such a concierge service can also be realized by integrating the functions of the guide robot, bell robot, and counter robot.

The provision of services using the telepresence robot as described above contributes not only to the improvement of business efficiency but also to the enhancement of service expertise. Since the number of people with specialized knowledge is limited, there is a problem that the number of places where specialists are located is limited when the experts are actually located where the work is necessary. On the other hand, if a telepresence robot is used so that specialists do not have to be on site, it is possible to provide services to a large number of places with a small number of specialists.

One example of expert expertise is interpretation skills. For example, the number of guides who can interpret, especially those who can interpret languages other than English, is limited, and it is difficult to prepare in advance in tourist destinations nationwide. If there is a mechanism to call an expert only when necessary using a telepresence robot, this problem can be solved. Interpretation skills are considered to be required not only for guide robot services but also for bell robot services, counter robot services, and robot concierge services.

However, there are many problems to realize the call center type telepresence robot service as described above.

One of the challenges is that the conventional telepresence robot of the telephony / videoconference expansion type is a mechanism that assigns a specific person to the robot. It is mentioned that there is no mechanism to automatically assign vacant people or people with the necessary skills to robots.

One problem is that conventional automatic call distribution devices (ACDs) are for telephone use and are not sufficient for automatic assignment of telepresence robots. Compared to voice-only telephones, telepresence robots have various communication means such as facial expressions, gestures, and movements in addition to voice, so ACD processing corresponding to these is required. In addition, since the connection partner is always a telephone in a conventional telephone call center, the conventional telepresence robot with expanded telephony / videoconferencing need only be able to connect to a specific type of robot. The call center type telepresence robot assigns multiple people to multiple telepresence robots, but the shape and size of the connected robot, the number of arms, the presence or absence of moving means, etc. are different. It is desirable to be able to cope with cases. In this case, ACD processing that can be connected to an operator who can cope with the function of each robot is required.

One of the challenges is that in the case of a conventional telephone / video conferencing service, the customer calls the other party by pushing a number or menu close to his or her telephone or stationary communication terminal, while in the case of a telepresence robot, the customer It is expected that the conversation with the remote operator (operator) of the robot can be started by a procedure close to a person-to-person conversation rather than a mechanical operation that a customer calls by button operation.

As one of the procedures close to the person-to-person conversation described above, it is conceivable to bring the customer and the telepresence robot closer to a distance that can be used to start the conversation.

As one of the procedures close to the person-to-person conversation described above, when the customer appears at the location of the telepresence robot, the customer can start a conversation with the operator without any operation. Can be considered.

As one of the challenges, the conventional telephone / video conferencing service conveys the voice and facial image of the person providing the service to the customer as it is, but in the case of a service using a telepresence robot, the voice and facial image of the operator are used as is by the customer. It is thought that it is often unnatural to tell to. Considering development / manufacturing costs, necessary functions, and operation efficiency, it is not always desirable for a robot body to be a live-action image. Robots that are humanoid but not live-up to human beings may be dissatisfied in communication because there is a gap in talking with human voices or displaying operator images only on the face.

As one of the challenges, the telephone service often uses the function of transferring voice conversations to other parties, but in an integrated telepresence robot, switching to another voice operator on the way is a robot consistency. There is also a problem that the individuality of the integrated robot is impaired.

One of the challenges in the case of telephone services is that a third party on the service provider side is required to monitor the conversation content in order to support the service and check whether a correct conversation with the customer is possible. Then, in addition to the conversation content, it is necessary to be able to monitor the remote operation content such as the movement state, facial expression, and gesture of the robot as well.

As one of the issues, the conversation content is recorded as a log in the telephone service and used for future work, but in the telepresence robot, the remote operation log of the robot is recorded in addition to the conversation content, and there is no problem with the customer. It must be possible to play back and check later whether there was an appropriate exchange.

In order to solve the above-described problems, a communication robot system according to the present invention includes a telepresence robot, a call center device, and an operator terminal, and the telepresence robot includes a video / audio capture device that acquires a video / audio of a customer; , A position determination device for acquiring a positional relationship between the customer and the robot, and control of call transmission for starting a conversation between the customer and the robot based on information obtained from the video / audio capture device and the position determination device An approach control device, a robot voice output control device that outputs an input from an operator as a voice of the robot, a drive unit control device that receives a body control instruction from the operator, and a drive unit that is controlled by the drive unit control device; Display characters and icons according to remote operation by the operator A robot communication control device that controls communication between the bot display control device and the call center device, and the call center device is a suitable one of the plurality of operator terminals for the call received from the approach control device. A multi-modal communication server that relays and records communication between the audio and video between the ACD device to which the platform is assigned and the caller's telepresence robot and the assigned operator terminal, and the physical movement of the robot, The operator terminal includes a voice input device from the operator, a remote operation device for remotely operating the telepresence robot, an operator side voice output control device for outputting the customer's voice, and an operator for outputting the customer image. Side display system Having a device, the operator communication control unit for controlling the communication with the call center device, and wherein.

According to the program, the video and audio capture device, the position determination device, the approach control device, the robot audio output control device, the drive unit control device, the robot display control device, the robot communication control device, the ACD device, the above The multi-modal communication server, the voice input device, the remote operation device, the operator-side voice output control device, the operator-side display control device, and the operator communication control device may function.

According to the present invention, it is possible to realize a robot call center that connects a customer and an appropriate operator and provides a service through conversation via a telepresence robot.

In the above conversation, according to the present invention, an operator receives a video and audio of a remote customer using a telepresence robot, and an operator operates the telepresence robot to output audio to the customer, facial expression, gesture, movement, display By presenting information using, it can be realized as a real-time conversation between the operator and the customer via a telepresence robot.

In the present invention, by using the ACD, it becomes possible to automatically assign a person who has a free hand or a person having a necessary skill to a telepresence robot. In particular, by making it possible to process skills related to the remote operation of the robot with the ACD, an appropriate operator can be assigned to the robot.

In the present invention, by using the ACD, it is possible to assign an appropriate operator who can handle each robot by allowing the individual functions of different types of robots to be processed by the ACD.

In the present invention, the above approach control device brings the customer and the telepresence robot close to a distance where they can talk to start a conversation, automatically calls from the robot and connects to the operator, and the conversation between the customer and the operator becomes natural. Allows to start.

In the present invention, instead of transmitting the operator's video and voice to the customer as they are, only the voice is output as a robot-like voice by using the robot voice output control device without using the video, thereby enabling a conversation with less sense of discomfort with the customer. It becomes. Further, even if the operator changes during the conversation, the same robot-like voice is output, thereby solving the problem that the consistency of the robot and the individuality of the integrated robot are impaired.

In the present invention, by performing audio / video conversation and remote operation of the robot through the multimodal communication server, a third party connects to the multimodal communication, and in addition to the audio / video conversation, the telepresence robot moves or gestures, etc. Realize monitoring of remote operation.

In the present invention, by recording data on the multi-modal communication server, in addition to the audio-video conversation, the state of remote operation such as the movement and gesture of the telepresence robot is also recorded, and there is no inappropriate communication with the customer. It is possible to play back and check later.

In the present invention, the assignment of the robot and the operator is controlled using the ACD, so that the operator can be changed in the middle or the conversation with the user using a plurality of robots can be realized.

It is a schematic block diagram of the communication robot system which is one Embodiment of this invention. It is a figure which shows the structure of the body shape of a telepresence robot, a drive part, and a media input / output device in the communication robot system which is one Embodiment of this invention. It is a conceptual diagram of the position and connection of a telepresence robot, a call center device, and an operator terminal in a communication robot system according to an embodiment of the present invention. It is a figure which shows an example of an operation | movement flowchart in the case of a customer and an operator having a conversation via a telepresence robot in the communication robot system which is one Embodiment of this invention. It is a schematic block diagram of the position determination apparatus in the communication robot system which is one Embodiment of this invention. It is a schematic block diagram of the media input device in the communication robot system which is one Embodiment of this invention. It is a schematic block diagram of an operator terminal in the communication robot system which is one Embodiment of this invention. It is a schematic block diagram of the drive part control apparatus in the communication robot system which is one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
[First Embodiment]

FIG. 1 shows a schematic block diagram of a communication robot system 100 according to the first embodiment. The communication robot system 100 includes a telepresence robot 101, a call center device 102, and an operator terminal 103. The telepresence robot 101 is a media input unit that captures customer's video and audio, outputs audio from the operator, and outputs the display device. Based on the information obtained from the output device 104, the position determination device 105 for determining the positional relationship between the customer and the robot, and the media input / output device 104 and the position determination device 105, for starting the conversation between the customer and the robot. An approach control device 106 that controls call transmission, a drive unit control device 107 that receives physical control instructions from an operator, a drive unit 108 that changes facial expressions, gestures, and movements by physical control, and the call center device 102 Robot that controls communication The call center device 102 assigns an appropriate one of the operator terminals 103 prepared for the call to the ACD device 110 and the caller telepresence robot 101. A multi-modal communication server 111 that relays and records communication using a voice image and a robot's body motion with the operator terminal 103. The operator terminal 103 includes a voice input / output device 112, a remote operation It has the operator communication control apparatus 114 which controls communication with the apparatus 113 and the call center apparatus 102, It is characterized by the above-mentioned.

FIG. 2 shows the body shape of the telepresence robot 101 in FIG. 1 and the configuration of the drive unit 108 and the media input / output device 104.

The body of the telepresence robot 101 includes a head 200, a torso 201, wheels 203, and a pair of arms 202, and has a shape in which wheels 203 are attached instead of feet to the upper body of a person.

The head 200 includes a pupil 204, an eyelid 205, an eyebrow 206, a mouth 207, a robot side microphone 208, an environmental camera / GPS 209, and a Wifi communication device 210.

The pupil 204, the eyelid 205, the eyebrows 206, and the mouth 207 are movable so that facial expressions can be created.

The robot-side microphone 208 is installed at a position corresponding to a human right ear and left ear, and can collect sound with a stereo microphone.

The environment camera / GPS 209 is a camera capable of capturing 360-degree surroundings and GPS capable of acquiring position information, and is used to know the position and orientation of the robot and the surrounding conditions when the operator remotely operates the robot.

The Wifi communication device 210 is used to connect to the Internet wirelessly.

The body 201 includes an interpersonal camera 211, a robot side speaker 212, a robot side touch panel display 213, a gyro sensor 220, and an impact sensor 221.

The interpersonal camera 211 is a pan / tilt camera, and is used to acquire an image of a customer standing in front of the camera.

The robot-side speaker 212 is provided on the front surface of the trunk 201 and is used to make a voice from the robot. The reason why the voice is not output from the mouth 207 is to prevent the mouth 207 from opening and closing the mouth and preventing the voice output because the mouth 207 is movable and used to make a facial expression.

The robot-side touch panel display 213 can display characters and image information by remote operation, and accepts touch panel input from customers.

The gyro sensor 220 can detect the fall of the robot.

The impact sensor 221 can measure the impact received by the robot.

The neck 214 that connects the head 200 and the torso 201 is a biaxial rotation type, and the head 200 can be rotated horizontally with respect to the torso 201 to change the direction of the head 200 or perform a nodding gesture by rotating it vertically. it can.

The arm 202 has a shoulder joint 215, an elbow joint 216, a wrist 217, and a finger 218.

The shoulder joint 215 is a biaxial rotation type, and can perform a gesture of raising the arm or raising the arm forward by rotation.

The elbow joint 216 is a biaxial rotation type and can be bent by connecting the upper and lower portions of the arm 202.

The wrist 217 is biaxially rotatable and can be bent by connecting the lower part of the arm 202 and the finger 218.

The finger 218 has five fingers having a joint structure similar to a human, and the thumb can be bent inward by two joints and the other fingers by three joints.

The wheel 203 is composed of four wheels, and is a wheel having a size that can stably travel even for work such as road guidance. In addition to moving forward and backward, by rotating the right two wheels and the left two wheels at different speeds, it is possible to perform clockwise rotation and counterclockwise rotation.

In FIG. 2, the wheel 203, the pupil 204, the eyelid 205, the eyebrows 206, the mouth 207, the neck 214, the shoulder joint 216, the elbow joint 216, the wrist 217, and the finger 218 are connected to the drive unit 108. Configure.

In FIG. 2, the robot side microphone 208, the environmental camera / GPS 209, the interpersonal camera 211, the robot side speaker 212, and the robot side touch panel display 213 constitute the media input / output device 104.

In FIG. 2, the human sensor 219 is physically independent from the body of the telepresence robot 101, but is installed near the robot and can transmit a sensing result to the telepresence robot 101 via Wifi. it can.

FIG. 3 is a conceptual diagram of the positions and connections of the telepresence robot 101, the call center device 102, and the operator terminal 103 in the communication robot system 100 shown in FIG. A plurality of telepresence robots 101 are used and are placed in different places. The telepresence robot 101 can be connected to the Internet via Wifi. A plurality of operators 300 who remotely operate the telepresence robot 101 are also waiting. One operator terminal 103 is placed in front of one operator 300 and is connected to the Internet. A plurality of operators 300 may be gathered at the same place, or may be at different places. The call center device 102 is placed in an environment connected to the Internet, such as a data center, and the telepresence robot 101 connected to the Internet and the operator terminal 103 are automatically connected by the ACD 110, and audio and video and remote operation are performed by the multimodal communication server 111. Relay information.

FIG. 4 is a diagram showing an example of an operation flowchart from the start of the conversation to the end of the conversation when the customer and the operator have a conversation via the telepresence robot. Hereinafter, with reference to this FIG. 4, the operation | movement for implement | achieving this conversation is demonstrated.

Step S1 is a state in which the customer has not visited, and the telepresence robot 101 enters a standby state. In the standby state, the telepresence robot 101 does not communicate with the call center device and is not connected to the operator, so that the power saving state can be maintained. In the standby state, the position of the robot is at a predetermined installation position, and facial expressions and gestures are also stationary in the predetermined state.

In step S2, the telepresence robot 101 waits for a person to come near the robot. If a person comes, the process proceeds to step S3. If not, the telepresence robot 101 stays in step S2.

In step S2, the position determination device 105 is used to determine whether a person has come near the robot.

FIG. 5 shows a schematic block diagram of the position determination device 105. The position determination device 105 recognizes the distance between the person in the video and the telepresence robot 101 based on the video acquired by the human sensor result receiving unit 500 that receives the sensing result of the human sensor 219 and the environmental camera / GPS 209. And a video distance recognition device 501.

The human sensor 219 is installed at the entrance of the room where the telepresence robot 101 is installed or a nearby ceiling. When a person comes near the robot, the human sensor 219 transmits an approach signal to the human sensor result receiving unit 500.

When the human sensor 219 cannot be installed near the telepresence robot 101, the video distance recognition device 501 determines the approach of a person based on the video acquired by the environmental camera / GPS 209. The approach is determined by detecting the moving object and its shape based on the background difference of the video.

When the human sensor result receiving unit 500 receives an approach signal or the image distance recognition device 501 determines that an object is approaching, the position determination device 105 has come to the approach control device 106. Notify that.

In step S3, the telepresence robot 101 starts an approach action to a person who has come close. The details of the approach behavior will be described later, and only an outline is shown here. An approach action is an action that works from a robot to a person who has come close to the robot. A person who has come close may just pass by, or a person who has a purpose and approaches the robot. In the approach behavior, the robot tries to talk to a person who has come nearby by approaching, gesturing, or speaking to the person from the robot side.

Further, during the approach action, the telepresence robot 101 is connected to the operator terminal 103 by calling the call center device 102 at an appropriate timing, and is operated by the operator 300. Details of the timing will be described later.

In step S4, as a result of the approach behavior, if the person disappears from the front of the robot or if the conversation is not established due to refusal to speak, the approach fails and the process proceeds to step S5. If a conversation with a person is established, the approach is successful and the process proceeds to step S6.

In step S5, an approach end process is performed. When the telepresence robot 101 has moved from the original installation position due to the approach action, it automatically returns to the specified position. If the robot does not have a function to automatically return to the specified position, the operator returns to the original position by remote operation. Also, facial expressions and gestures automatically return to the default state. After finishing the above approach end processing, the process returns to step S1.

In step S6, a person in front of the telepresence robot 101 is set as a customer, and a response via the robot is started with the operator, and the process proceeds to step S7. During the reception, the operator receives the video and audio of the customer at the remote location via the robot, and the operator operates the robot to output the voice, facial expression, gesture, route guidance by movement, and information display using the display. By doing this, a real-time conversation is performed between the operator and the customer via a robot.

In step S7, processing is performed when there is a monitoring start / end request from the operator manager during the response. The operator manager is in a position to manage the operator's work, and in order to manage whether the operator can respond to the customer without any problems, the audio video content of the conversation using the telepresence robot, the facial expression of the robot, You must be able to monitor gestures and movements. The operator manager transmits a request for starting or ending monitoring of a specific operator's response from the operator terminal of the operator manager to the call center device. If this monitoring start / end request is received, the process proceeds to step S8. If there is no request, the process proceeds to step S9.

In step S8, if the request is for monitoring start, monitoring is started. If it is ended, monitoring is ended, and the process proceeds to step S9. During monitoring, the operator administrator can monitor the customer's response to a specific operator by outputting it to his / her operator terminal.

During monitoring, the operator administrator can send a text chat to the specific operator being monitored from his operator terminal to give advice on the content of the response in real time.

In step S9, it is determined whether or not the operator has requested transfer during the response. An operator is in charge of another operator when he / she is not able to deal with the situation during his / her interaction with the customer, for example, when a topic that requires specialized knowledge that he / she does not know or a problem that he / she does not have the authority to judge A request can be sent to the ACD 110 to forward the current response to. The transfer request is displayed on the transfer destination operator terminal, and if the transfer destination operator permits it, the transfer is successful and the process proceeds to step S10. If no transfer request is made or if the transfer destination operator does not permit, the process proceeds to step 11.

In step S10, the transfer source operator is disconnected from the telepresence robot 101, and instead, the transfer destination operator connects to enable remote operation. Note that the processing on the transfer source operator side ends here, but the transfer destination operator side processing newly starts from step S6.

In step S <b> 11, it is determined whether or not the response between the operator and the customer has been completed and the operator has requested a response end process by operating the operator terminal 103. If there is a request, the process proceeds to step S12. If there is no request, the process returns to step S7 to continue the response.

In step S12, a response end process is performed. If the telepresence robot 101 has moved from the original installation position, it automatically returns to the specified position. If the robot does not have a function to automatically return to the specified position, the operator returns to the original position by remote operation. Also, facial expressions and gestures automatically return to the default state. The audio video content of the conversation, the facial expression of the robot, the gesture, and the state of movement from the start of the response to the end of the response are recorded in the multimodal communication server 111 as a response log. After completing the above-described response end processing, the process returns to step S1.

A method of responding to the customer by the operator starting from step S6 will be described with reference to FIGS.

FIG. 6 shows a schematic block diagram of the media input device 104 shown in FIG. The media input / output device 104 includes a robot microphone 208, a robot voice capture device 600 that captures voice input from the robot side microphone 208, and transmits the voice to the operator terminal 103 via the robot communication control device 109. A robot voice output control device 601 that receives input and converts it into robot voice and outputs it, a robot side speaker 212 that receives and outputs voice from the robot voice output control device 601, an interpersonal camera 211, an environmental camera / GPS 209, The video capture device 604 that captures the video input from the interpersonal camera 211 and the environment camera / GPS 209 and transmits it to the operator terminal 103 via the robot communication control device 109, and receives the video from the video capture device 604 and performs face recognition. Having a face recognition apparatus 605 rows, the robot-side touch panel display 213, and a robot display controller 606 for controlling input and output of the robot-side touch panel display 213 communicates with the operator terminal 103 through the robot communication control device 109.

The robot voice capture device 600 has a function of capturing voice data and converting it into character data by voice recognition. The voice converted into character data is used in an automatic answering call described later.

The robot voice output control device 601 includes a voice synthesis device 602 and a voice conversion device 603. When the conversation input from the operator is a sentence, the speech synthesizer 602 performs speech synthesis from the sentence and outputs the voice. When the conversation input from the operator is a voice, the voice conversion device 603 converts the original human voice into a certain robot-like voice in real time and outputs the voice.

When the conversation input from the operator is a sentence, the real-time nature of the conversation is reduced compared to the case of speech, but the voice synthesized from the sentence differs from person to person when compared to the voice converted by the voice converter. There is an advantage that it is easy to make voice without.

In the event of an emergency, it is better to clearly indicate that the person behind him is speaking rather than the robot, the operator can stop the voice conversion process of the voice converter 603 by remote operation and output the original voice as it is.

FIG. 7 shows a schematic block diagram of the operator terminal 103. The operator terminal 103 has a voice input / output device 112 for an operator to have a voice conversation with a customer, and a remote operation device 113 for an operator to receive and remotely control video and position information from the telepresence robot 101. .

The voice input / output device 112 includes an operator microphone 700, an operator voice capture device 701 that captures voice input from the operator microphone 700, and transmits the voice to the telepresence robot 101 via the operator communication control device 114, and an operator speaker. 702, an operator side voice output control device 703 that receives a voice from the customer from the telepresence robot 101 via the operator communication control device 114 and outputs it from the operator side speaker 702, a keyboard 704, and a character conversation input from the keyboard 704. It has a character chat input device 705 that receives and transmits it to the telepresence robot 101 via the operator communication control device 114.

The remote operation device 113 includes an operator-side touch panel display 706, an operator-side display control device 707 that controls input / output of the operator-side touch panel display 706 and communication with the telepresence robot 101 via the operator communication control device 114, and an operator Side operation pad 712, and an operation signal encoding device 713 that encodes inputs from the operator side operation pad 712 and the operator side touch panel display 706 as operation signals and transmits the operation signals to the telepresence robot 101 via the operator communication control device 114.

The operator side display control device 707 includes a robot model display control device 708, a robot environment display control device 709, a customer video display control device 710, and a material presentation control device 711.

The robot model display control device 708 acquires information on the current driving unit 108 through communication from the telepresence robot 101, thereby displaying the current facial expression and gesture of the robot on the operator-side touch panel display 706 using the 3D model CG. Thus, the current state of the robot's body is transmitted to the operator.

The robot environment display control device 709 acquires the current robot position and orientation information and 360-degree video acquired by the telepresence robot 101 using the environmental camera / GPS 209 via communication, and displays an electronic map on the operator-side touch panel display 706. By displaying it in an overlapping manner, the operator is informed of the situation around the robot.

The customer video display control device 710 acquires the current video of the interpersonal camera 211 by communication from the telepresence robot 101 and displays it on the operator-side touch panel display 706 to convey the video of the customer in front of the robot to the operator. .

The material presentation control device 711 stores materials (electronic data such as photographs, documents, maps, videos, etc.) to be presented to the customer when the operator responds, and the operator operates the operator side touch panel display 706 to provide necessary materials. Is selected so that the selected material is displayed on the robot-side touch panel display 213 of the telepresence robot 101.

The operator can transfer the response or play back the response record by operating the touch panel display 706 on the operator side.

The operator can remotely operate the telepresence robot 101 using the operator side operation pad 712. The operator side operation pad is provided with two directional sticks and a plurality of buttons used in a game machine.

The operator can instruct the telepresence robot 101 to move forward, backward, rotate left, and rotate clockwise using the direction stick for movement of the operator side operation pad 712.

An operator can pan / tilt the interpersonal camera 211 of the telepresence robot 101 using the viewpoint changing direction stick of the operator side operation pad 712.

The operator can instruct the telepresence robot 101 to execute the facial expression registered in each button by pressing a button on the operator side operation pad 712. Examples of defined facial expressions include smiles and crying faces.

The operator can instruct the telepresence robot 101 to execute the gesture registered in each button by pressing a button on the operator side operation pad 712. Examples of defined gestures include pointing to the left and right, waving hands, shaking heads, and nodding.

The operator can also use the operator-side touch panel display 706 to instruct the telepresence robot 101 for facial expressions and gestures. Frequently used facial expressions and gestures are registered in the buttons on the operator side operation pad 712, and those that are not used can be selected from the operator side touch panel display 706 using a touch menu.

By using the operator-side touch panel display 706, the operator can directly operate each driving unit of the telepresence robot 101 instead of the predefined facial expression and gesture.

The response log can be replayed on the operator terminal to confirm the response details later.

FIG. 8 is a schematic block diagram of the drive unit control device 107. The drive unit control device 107 includes an operation signal receiving device 800 that receives a remote operation signal from an operator from the robot communication control device 109, and an operation signal decoding device 801 that decodes the operation signal and converts it into setting numerical value data of each drive unit. And a movement control device 802, a facial expression control device 803, and a gesture control device 804 that transmit the decoded set numerical values to each drive unit.

The movement control device 802 controls the operation of the wheel 203.

The movement control device 802 may move the robot autonomously according to the process of the approach control device 106 even when there is no remote operation instruction from the operator. Details will be described later.

When moving the robot autonomously as described above, the movement control device 802 uses the environmental camera / GPS 209 to track the target, avoids any obstacles in front of the eyes, and hits the obstacles. If it is detected, the impact sensor 221 moves to the destination while making a determination.

The facial expression control device 803 controls the operations of the pupil 204, the eyelid 205, the eyebrows 206, and the mouth 207.

The gesture control device 804 controls operations of the neck 214, the shoulder joint 215, the elbow joint 216, the wrist 217, and the finger 218.

The gesture control device 804 performs an autonomous operation such as sometimes slightly moving the neck and joints or nodding by moving the neck when the customer speaks even when there is no remote operation instruction from the operator. This is to prevent the robot from appearing like a life if it is stationary for a long time.

Hereinafter, the details of the approach behavior described briefly in step S3 of FIG. 4 and the timing at which the telepresence robot 101 calls the call center device 102 during the approach behavior will be described.

The approach action is an action of trying to talk to a person who comes near the telepresence robot 101 by approaching from the robot side, making a gesture, or talking with a voice.

The telepresence robot 101 calls the call center device 102 during the approach action. Depending on the timing of the call, the approach behavior is divided into four types: immediate call, automatic approach call, automatic greeting call, and automatic answering call. The robot takes an approach action using one of the four methods specified in advance by setting.

The immediate call is called immediately when it is determined by the position determination device 105 that a person is nearby. The ACD 110 assigns the operator terminal 103 to the call, and the operator starts remote operation of the robot. After the remote operation is started, the operator moves the robot to approach the customer if the distance does not reach the customer, and speaks if the distance reaches the customer (about 2 to 3 meters).

Immediate calling is a method that is suitable when the visitor of a certain business is surely known by installing a human sensor 219 at the entrance of the store, such as when the telepresence robot 101 is installed in a small store. .

In the automatic approach call, if the position determination device 105 finds that the person is nearby, if the voice does not reach the person, the robot makes a gesture of waving its hand autonomously. After drawing attention, make a call after autonomously moving to the distance that the person can reach. However, if the distance does not reach the voice even after tracking for a certain period of time, the approach action will not be called as a failure. Also, when it is known that the person is nearby, if the distance is within the range where the voice can reach the person, the call is made on the spot. The ACD 110 assigns the operator terminal 103 to the call, and the operator starts remote operation of the robot. The operator can speak to the customer since it is within reach of the customer after the remote operation starts.

The automatic approach call is different from the immediate call in that the telepresence robot 101 performs an action of approaching a person autonomously by the robot rather than an operation by an operator. When robots are installed on streets and exhibitions where people nearby are just passing by and not necessarily customers, it is possible to reduce the waste of chasing people who are not likely to be customers.

If the position determination device 105 determines that the person is nearby, if the automatic greeting call is a distance where the voice cannot reach the person, the robot makes a gesture of waving its hand autonomously. After drawing attention, it autonomously tracks and moves until it reaches a distance where the person can hear it. However, if the distance does not reach the voice even after tracking for a certain period of time, the approach action will not be called as a failure. When the distance reaches a person's voice, the robot autonomously greets the person with voice and gesture. After the greeting, the face recognition device 605 determines whether or not the person's face is facing the robot for a certain period of time, and makes a call if the face is facing the robot or if there is a voice response. If the face does not face the robot for a certain period of time and there is no answer even by voice, the approach action does not call as a failure. The ACD 110 assigns the operator terminal 103 to the call, and the operator starts remote operation of the robot. After the remote operation is started, the operator can make a conversation by bringing the robot close to an appropriate distance (about 1 to 1.5 meters) for customer interaction.

The degree to which the automatic greeting call is left to the autonomous operation of the robot is higher than that of the automatic approach call. When the degree of entrusting robots to autonomous operation increases, a flexible customer approach cannot be achieved. However, in places where there are many people, the waste of the operator chasing people who are less likely to be customers can be further reduced.

In the automatic answering call, if the position determination device 105 finds that the person is nearby, if the voice does not reach the person, the robot makes a gesture of waving its hand autonomously. After drawing attention, it autonomously tracks and moves until it reaches a distance where the person can hear it. However, if the distance does not reach the voice even after tracking for a certain period of time, the approach action will not be called as a failure. When the distance reaches a person's voice, the robot autonomously greets the person with voice and gesture. After the greeting, the face recognition device 605 determines whether or not the human face is facing the robot for a certain period of time, and if the face is facing the robot or if there is a voice response, the robot responds to the person. Autonomous approach to the right distance, make the default initial question the customer, and call if the customer answers. If there is no answer, the approach action does not call as a failure. The ACD 110 assigns the operator terminal 103 to the call, and the operator starts remote operation of the robot. After remote operation starts, the operator can continue the conversation with the customer.

In the automatic answering call, since the call is made after hearing the customer's answer to the initial question in advance, an operator having the skill according to the customer's request can be assigned by the processing of the ACD 110 according to the answer content. As a response from the customer, the result received by voice and recognized by the robot voice capture device 600 is used, or the result of the customer selecting the menu displayed on the robot side touch panel display is used.

The menu displayed on the touch panel display is, for example, “English”, “Chinese”, “Korean”, “Spanish” in the case of a guide service for foreign tourists. The operator who has is assigned.

There is an emergency call as a case where the telepresence robot makes a call other than the approach action. This is a function that automatically calls when the gyro sensor 220 detects the robot falling or when the impact sensor 221 detects a continuous attack on the robot. The ACD 110 assigns an operator terminal 103 of a specific problem handling operator for an emergency call. The problem-handling operator speaks with human voice and asks for help from the surrounding people. It is known that an autonomously operating robot may be attacked by humans with mischief, and an emergency call is a function for responding to such a situation.

Hereinafter, a method in which the ACD 110 automatically assigns a call from the telepresence robot 101 to an appropriate operator terminal 103 will be described.

A plurality of operators are registered in the ACD 110 in advance, and an ID indicating an individual operator, personal skill information acquired by each operator, and an operator terminal ID operated by each operator are registered as a set.

The above skills are expressed by keywords. For example, “sales” for sales skills, “support” for support skills, and “English” for English skills.

Skills are accompanied by grade values that quantify their goodness. The grade is from 1 to 16, and the smaller the value, the better.

In the conventional ACD for voice calls, only the skills required for voice conversation are generally registered. However, in the ACD 110 in this embodiment, in addition to the skills required for voice conversation, remote operation of the robot is performed. To do so, register the maneuvering skill “manipulation” and the acting skill “motion” indicating the level of facial expression and gesture skill.

Maneuvering skills are especially needed in road guidance operations. Also, acting skills are needed to behave in such a way, especially when a telepresence robot has a specific character.

When the telepresence robot 101 calls, it can be called by specifying a skill, a specific operator individual, or specifying nothing.

A call that specifies a skill can specify one or more skills. In the case of a plurality, the first skill and the second skill are ranked in descending order of necessity.

A call that specifies a skill can specify a customer's request input in the above automatic answering call as a skill.

A call specifying a skill can also specify a specific skill in advance to the telepresence robot 101 installed in a specific place. For example, in the above guide robot service example, the Kyoto sightseeing skill “Kyoto” is specified for the guide robot installed in Kyoto, and the Tokyo tourism skill “Tokyo” is specified for the guide robot installed in Tokyo. it can. In the guide robot service, the land skill is designated as the first skill, and the maneuvering skill “manipulation” for the route guidance is also designated as the second skill.

A call that designates a specific operator can be used when a customer's person in charge is determined in advance. For example, in the example of the bell robot service, the customer's dedicated bell staff can be called by specifying the individual customer with the face recognition device 605.

The method by which the ACD 110 selects an operator to which a call is assigned is as follows. First, a person who is currently connected to the telepresence robot is excluded from the registered operators. If multiple operators with the same skill are registered, only the operator with the same skill is registered, and if multiple operators with the same grade are registered, the previous response will be given. Operators are assigned according to the procedure of narrowing down to the operator who has a long waiting time after the completion of, and selecting one at random if there are still more than one. This procedure allows agents with appropriate skills to be assigned to calls while increasing availability.

In the method of selecting an operator for a call with a plurality of skills specified by the ACD 110, first, a person who is currently connected to the telepresence robot is excluded from registered operators, and has a skill that matches the first skill. If multiple operators with the same skill are registered as the target, only the operator with the better grade is selected, and if multiple operators with the same grade are registered, the second If the grades after the skill are compared in order, it is narrowed down to the better one, and if multiple operators of the same grade are registered for all the specified skills, the longer waiting time after the end of the previous response If there are multiple people remaining, select one at random. By assigning the operator.
[Effect of the first embodiment]

As described above, in the first embodiment, the telepresence robot 101, the call center device 102, and the operator terminal 103 are provided, and the telepresence robot 101 captures the video and audio of the customer and outputs and displays the audio from the operator. The media input / output device 104 that performs output, the position determination device 105 for determining the positional relationship between the customer and the robot, and the customer and the robot based on information obtained from the media input / output device 104 and the position determination device 105 An approach control device 106 that controls call transmission for starting a conversation, a drive unit control device 107 that receives an instruction of physical control from an operator, and a drive that performs facial expression and gesture changes and body movements by the drive unit control device 107. Unit 108 and the call center device 102 are controlled to communicate with each other A bot communication control device 109, and the call center device 102 assigns an appropriate one of a plurality of operator terminals 103 to the call transmitted from the approach control device 106, and a call It has a multimodal communication server 111 that mediates and records the communication between the audio and video between the original telepresence robot 101 and the assigned operator terminal 103 and the telepresence robot body movement. A device 112, a remote operation device 113, and an operator communication control device 114 for controlling communication with the call center device 102 are provided.

According to the first embodiment, an operator receives a remote customer's video and audio using a telepresence robot, and an operator operates the telepresence robot to output audio, facial expressions, gestures, movements, By presenting information on the display, real-time conversation between the operator and the customer via the telepresence robot can be realized.

According to the first embodiment, by using the ACD, it is possible to automatically assign a person who is free or has a necessary skill to the telepresence robot and respond to the customer. In particular, it is possible to assign an appropriate operator to the robot by enabling the ACD to process the maneuvering skill “manipulation” and the acting skill “motion” related to the remote operation of the robot.

According to the first embodiment, the approach control device brings the customer and the telepresence robot close to a distance where they can talk to start conversation, and the robot automatically calls and connects to the operator as an immediate call, Since it is possible to use four methods of automatic approach call, automatic greeting call, and automatic answering call, the conversation between the customer and the operator can be started naturally.

According to the first embodiment, when the gyro sensor detects a fall of the robot or when the impact sensor detects a continuous external attack on the telepresence robot, the robot automatically makes an emergency call. The ACD assigns an operator terminal of a specific problem handling operator to the emergency call, and the problem handling operator can speak with a human voice and ask people around the robot for help.

According to the first embodiment, instead of transmitting the operator's image and voice to the customer as they are, only the voice is output as a robot-like voice by using the robot voice output control device without using the video, so that the user feels uncomfortable. It is possible to have a conversation through a robot with less form. Further, even if the operator changes during the conversation, the same robot-like voice is output, thereby solving the problem that the consistency of the robot and the individuality of the integrated robot are impaired.

According to the first embodiment, by performing audio / video conversation and robot remote operation via a multimodal communication server, a third party connects to the multimodal communication, and in addition to the audio / video conversation, the telepresence robot Remote operation such as movement and gesture can be monitored.

According to the first embodiment, by recording data on the multimodal communication server, in addition to the audio-video conversation, the movement of the telepresence robot and the state of the remote operation such as the gesture are also recorded, which is inappropriate with the customer. You can play back and check later if there was any interaction.
[Second Embodiment]

The second embodiment has the same configuration as that of the first embodiment except that the telepresence robot 101 in the first embodiment does not have the wheels 203 shown in FIG. .

The second embodiment operates in the same procedure as that of the first embodiment except that the approach action in the first embodiment is not moved and the movement by remote operation is not performed.
[Effect of the second embodiment]

In the second embodiment, there is an advantage that the robot can be miniaturized because there is no wheel for approaching a person and having a conversation or moving with the person to guide the road. For example, in the case of the counter robot service, since the robot does not need to move much, it can be easily operated by downsizing the robot and installing it on a desk.
[Third embodiment]

The third embodiment has the same configuration as the first embodiment except that a drone (small unmanned aerial vehicle) is used instead of the telepresence robot 101 in the first embodiment.

The drone is equipped with an interpersonal camera, a microphone, and a speaker, and an operator can talk with a person in front of the drone via the drone.
[Effect of the third embodiment]

By using a call center-type telepresence robot as a drone, users can prepare a single drone and have various specialized skills such as aerial photography and high-altitude photography / confirmation work for bridge repair. You will be able to receive services from
[Fourth embodiment]

The fourth embodiment has the same configuration as that of the first embodiment except that a toy doll is used instead of the telepresence robot 101 in the first embodiment.

A toy doll is equipped with an interpersonal camera, a microphone, and a speaker, and an operator can act as a toy doll and talk with a child in front of the doll.
[Effect of the fourth embodiment]

By using a call center type telepresence robot as a toy doll, it is possible to realize a toy doll that can be connected to an operator only when a child speaks and can talk.
[Fifth Embodiment]

In the fifth embodiment, the telepresence robot with wheels in the first embodiment, the telepresence robot without wheels in the second embodiment, the drone in the third embodiment, and the toy doll in the fourth embodiment Except that one operator can connect to any kind of telepresence robot, and the configuration is the same as the embodiment for each robot.

When a telepresence robot calls a call center device, the skill representing the type of telepresence robot is always the first skill, and the other skills are set in order of priority in the second and third order. The configuration is the same as that of the embodiment.

The skill representing the type of robot with wheels is “wheel”, the skill representing the type of robot without wheels is “nowheel”, the skill representing the drone is “drone”, and the skill representing the doll is “doll”.

If the operator has the technology to operate a robot with wheels, the “wheel” skill is used. If the operator has the technology to operate a robot without wheels, the “nowheel” skill is used. "," If you have the skill to operate a doll, register "doll" in the ACD.
[Effect of Fifth Embodiment]

By setting a skill representing a specific type of robot as the first skill, a plurality of types of telepresence robots can be prepared, and a call can be assigned only to an operator who has operation skills of each robot. Since an operator having a plurality of types of robot operation techniques and an operator having a single robot operation technique can be mixed, it is possible to increase the operating rate of the operators.
[Sixth Embodiment]

In the sixth embodiment, the operator searches for another nearby telepresence robot by registering the location information of each telepresence robot in the ACD 110 in addition to operating in the same configuration and procedure as in the first embodiment. It can be so.

After the operator starts responding to the customer in step S6 in FIG. 4, an additional request can be issued to a specific operator to connect to another telepresence robot near the robot and participate in the conversation. If the request is approved, you can have a three-way conversation between the customer and two operators.
[Effect of the sixth embodiment]

When individual specialized skills are required and individual operators possess it, it is desirable to have a three-way conversation. For example, if there are separate operators who have only tourist guidance skills and operators who have only interpreter skills, an operator who has only tourist guide skills will have an operator who has interpreter skills that can handle the customer's native language. By calling one telepresence robot, customers can use both skills.

DESCRIPTION OF SYMBOLS 100 Communication robot system 101 Telepresence robot 102 Call center apparatus 103 Operator terminal 104 Media input / output apparatus 105 Position determination apparatus 106 Approach control apparatus 107 Drive part control apparatus 108 Drive part 109 Robot communication control apparatus 110 ACD
111 Multimodal Communication Server 112 Voice Input / Output Device 113 Remote Operation Device 114 Operator Communication Control Device 200 Head 201 Torso 202 Arm 203 Wheel 204 Eye 205 Eye 206 Eyebrow 207 Mouth 208 Robot Side Microphone 209 Environmental Camera / GPS
210 Wifi Communication Device 211 Interpersonal Camera 212 Robot Side Speaker 213 Robot Side Touch Panel Display 214 Neck 215 Shoulder Joint 216 Elbow Joint 217 Wrist 218 Finger 219 Human Sensor 220 Gyro Sensor 221 Shock Sensor 300 Operator 500 Human Sensor Result Receiver 501 Video Distance Recognition device 600 Robot voice capture device 601 Robot voice output control device 602 Speech synthesis device 603 Audio conversion device 604 Video capture device 605 Face recognition device 606 Robot display control device 700 Operator side microphone 701 Operator voice capture device 702 Operator side speaker 703 Operator side Voice output control device 704 Keyboard 705 Character chat input device 706 Operator side touch panel display 707 Operator-side display control device 708 Robot model display control device 709 Robot environment display control device 710 Customer image display control device 711 Document presentation control device 712 Operator-side operation pad 713 Operation signal encoding device 800 Operation signal receiving device 801 Operation signal decoding device 802 Movement control device 803 Facial expression control device 804 Gesture control device

Claims (20)

A robot remotely operated by a human operator, a call center device, and an operator terminal, wherein the robot acquires a video and audio capture device for acquiring video and audio of a user near the robot, and the user A position determination device for acquiring a relative position with respect to the robot, an approach control device for controlling call transmission to the call center device based on information obtained from the video / audio capture device and the position determination device, and the operator The robot voice output control device that outputs the input of the robot as the voice of the robot and transmits it to the user, the drive unit control device that receives a physical control instruction from the operator, and the drive unit that is controlled by the drive unit control device And a robot communication control device for controlling communication with the call center device, The center device transmits a voice between the ACD device that assigns one of the plurality of operator terminals to the call received from the approach control device, and the robot that sent the call and the assigned operator terminal. A multi-modal communication server having a video relay function and a remote function instruction relay function from the operator terminal to the robot; the operator terminal having a voice input device that captures the voice of the operator; Communication between a remote operation device for remotely operating a robot, an operator-side voice output control device that outputs the user's voice, an operator-side display control device that outputs the user's video, and the call center device Having an operator communication control device to control Communication robot system according to claim. The communication robot according to claim 1, wherein the process in which the ACD assigns the call to one of the plurality of operator terminals includes a method of determining based on a robot operation capability of the operator. system. The process of allocating the call to one of the operator terminals having a plurality of calls by the ACD has a method of determining based on a functional difference due to a difference in model or option of the robot. 2. The communication robot system according to 2. The robot has a face recognition device, and the process in which the ACD assigns the call to one of the plurality of operator terminals is the face for the user image acquired by the video / audio capture device. The communication robot system according to any one of claims 1 to 3, further comprising a method of making a determination using personal information that can be identified from a result of applying the recognition device. The said robot has a display, It has the method of displaying a character and an image on the said display according to the said operator's instruction | indication, and conveying to the said user, It is any one of Claim 1 thru | or 4 characterized by the above-mentioned. Communication robot system. The robot has a touch panel display, and the process in which the ACD assigns the call to one of the plurality of operator terminals has a method of determining by using contents input from the touch panel display by the user. The communication robot system according to any one of claims 1 to 5, wherein: 7. The robot voice output control device according to claim 1, further comprising a method of converting voice having individual voice quality inputted from the operator into voice having a constant voice quality and outputting the voice. The communication robot system according to any one of the above. The robot voice output control device has a method of converting a character input from the operator into a voice having a certain voice quality and outputting the voice. Communication robot system. 9. The method according to claim 1, wherein the approach control device includes a method of transmitting the call immediately when it is determined by the position determination device that a person is near the robot. The communication robot system described in 1. The approach control device is a gesture in which when the position determination device finds that a person is near the robot, the robot autonomously draws the person's attention if the distance does not reach the person. The method according to any one of claims 1 to 8, further comprising: a method of autonomously tracking and moving until reaching a distance where the voice can reach the person, and transmitting a call after the movement is completed. Communication robot system. The approach control device is a gesture in which when the position determination device finds that a person is near the robot, the robot autonomously draws the person's attention if the distance does not reach the person. After moving, the robot autonomously moves to the distance that the voice can reach, and after the movement is completed, the robot autonomously greets the person with voice and gesture, and then responds from the person. The communication robot system according to claim 1, further comprising a method for transmitting a call. The approach control device is a gesture in which when the position determination device finds that a person is near the robot, the robot autonomously draws the person's attention if the distance does not reach the person. After moving, the robot autonomously moves to the distance that the voice can reach, and after the movement is completed, the robot autonomously greets the person with voice and gesture, and then responds from the person. For example, the robot autonomously moves to approach the appropriate distance to approach the person, approaches the person autonomously after the approach is completed, and asks the person for the default initial question. The communication robot system according to any one of claims 1 to 8, further comprising a method for transmitting an answer and a call to the call center device. The communication robot system according to any one of claims 1 to 12, wherein the driving unit includes a moving unit that can be remotely operated by one operator automatically assigned by the ACD. 14. The communication according to claim 1, wherein the driving unit has a hand and an arm that can be gesture-operated by a remote operation from one operator automatically assigned by the ACD. Robot system. The communication robot system according to any one of claims 1 to 14, wherein the driving unit has a face whose expression can be changed by remote operation from one operator automatically assigned by the ACD. . The method according to any one of claims 1 to 15, further comprising: a method in which the robot assigned to the operator terminal by the ACD is reassigned to another operator terminal to change the operator halfway. The communication robot system according to item. In addition to assigning the operator terminal to the robot, the ACD has a conversation between the user and the two robots by assigning another operator terminal to another robot near the robot. The communication robot system according to claim 1, further comprising a method that can perform the operation. The multimodal communication server has a method by which a third party can monitor audio and video exchanged between the robot and the operator terminal, and remote operation instruction content from the operator terminal to the robot. The communication robot system according to any one of claims 1 to 17, wherein the communication robot system is characterized. The multimodal communication server records, as a log, an audio image exchanged between the robot and the operator terminal, and a remote operation instruction content from the operator terminal to the robot, and the audio recorded in the log The communication robot system according to any one of claims 1 to 18, further comprising a method of reproducing an image and the content of the remote operation instruction later. When the robot detects that the body has fallen or has received a strong physical impact on the body, the robot sends an emergency call to the ACD indicating that the body is in danger, The communication robot system according to any one of claims 1 to 19, wherein the ACD has a method of assigning an operator terminal of an operator who can respond to a problem to an emergency call.

Images