WO2022034839A1 - Robot system - Google Patents

Abstract

[Problem] Provided is a robot system capable of performing remote control even with a simple arrangement.　[Means] When control data is input, by an operation side user, from an input unit 16 of an operation side portable device 10, a control data sending means 22 sends the control data to a robot side portable device 30. A control data receiving means 42 of the robot side portable device 30 receives the control data and provides the same to a light emission control means 36. The light emission control means 36 causes a control light emission region provided in a part of a display 32 to flash on the basis of the control data to emit control light. A light emission detection means 52 of a robot base body device 50 holding the robot side portable device 30 receives the control light to obtain control light data. A robot side control means 54 controls a driving part 56 on the basis of the control light data. The driving part 56 moves a holding part holding the robot side portable device 30. The orientation of a camera 34 of the robot side portable device 30 is changed thereby, and the orientation of an image displayed on the operation side portable device 10 is also changed.

Description

Robot system

The present invention relates to a robot system that can be controlled remotely.

A system that controls the robot from a remote location has been proposed. For example, Japanese Patent Application Laid-Open No. 2017-50018 discloses a self-propelled robot that can be equipped with a tablet. The user can remove this tablet from the self-propelled robot and control the movement of the camera and each part provided in the self-propelled robot at hand. At this time, the control signal from the tablet is given to the robot via the server device of the Internet.

This allows the user to control the robot at hand, which facilitates operation.

However, in the conventional technology such as JP-A-2017-50018, in order to give a control signal to the robot from the tablet which is a mobile terminal device via the Internet, it is necessary to provide a communication device on the robot side.

Further, in Japanese Patent Laid-Open No. 2017-50018, since a tablet for operation is normally attached to the robot, it is not possible to perform full-scale remote control with a tablet owned by another user. was difficult.

It is an object of the present invention to solve the above-mentioned problems and to provide a robot system capable of remote control while having a simple configuration.

Some of the independently applicable features of this invention are listed below.

(1)-(7) The robot system according to the present invention is a robot system including an operation side portable device, a robot side portable device, and a robot substrate device.
The operation-side mobile device includes a captured image display means for displaying a robot-side captured image transmitted from the robot-side portable device on a touch display, and an operation-side captured image captured by a camera via a server device or directly. , The captured image transmitting means transmitted to the robot-side mobile device by the transmitting unit, the operation receiving means for receiving the user's operation input to the touch display, and the received operation as control data via the server device or directly. In addition, a control data transmission means for transmitting to the robot-side mobile device by the transmission unit is provided.
The robot-side portable device is a captured image display means for displaying an operation-side captured image transmitted from the operation-side portable device on a touch display, and a robot-side captured image captured by a camera via a server device or directly. A control light emitting region of at least a part of the touch display, a captured image transmitting means for transmitting to the operation-side portable device by a transmission unit, a control data receiving means for receiving control data transmitted from the operation-side portable device, and a control light emitting unit. It is provided with a light emission control means for controlling light emission in the control light emission region based on the control data and transmitting control light.
The robot substrate device is a holding unit provided so as to be movable with respect to the main body portion by a main body portion and a driving unit, and physically holds the touch display facing the touch display of the robot-side portable device. The control light in the control light emission region of the robot-side portable device held by the holding unit on the robot substrate device is detected by the holding unit having a holding wall for the robot and the light emission detecting unit, and control light data is obtained. It is provided with a light emission detecting means and, at least, a robot side control means for controlling the holding unit for holding the robot side portable device to be moved by the driving unit based on the control light data from the light emitting detecting means. It is a feature.

Therefore, although it has a simple structure, it is possible to control the orientation of the camera of the mobile device on the robot side from a remote location.

(8) The system according to the present invention is characterized in that the drive unit includes a motor for rotating the holding unit that holds the robot-side portable device.

Therefore, it is possible to drive the rotation by a motor by remote control.

(9) The system according to the present invention is characterized in that the drive unit drives at least an arm member provided on the robot substrate device.

Therefore, the arm member can be driven by remote control.

(10) The system according to the present invention is characterized in that the light emission control means of the portable device on the robot side outputs control light having a different duty ratio from the control light emission region according to the received control data.

Therefore, control can be performed with a simple configuration that is not easily affected by noise.

(11) In the system according to the present invention, the light emitting means changes the duty ratio of the control light at least depending on whether the control data is an operation content for driving the drive unit or an operation content for stopping the drive unit. It is characterized by that.

Therefore, since the control data is sent even for the stop, it is possible to reduce the misjudgment.

(12) In the system according to the present invention, the drive unit includes a motor that rotates the holding unit that holds the robot-side portable device, and the light emitting means is at least an operation content in which the control data rotates the motor clockwise. It is characterized in that the duty ratio of the control light is changed depending on whether the operation content is to rotate counterclockwise or to stop.

Therefore, it is possible to reliably control clockwise rotation, counterclockwise rotation, and stop.

(13) The system according to the present invention is characterized in that the mobile device on the operation side or the mobile device on the robot side is a smartphone or a tablet computer.

Therefore, it is possible to construct a robot system using the camera and display of a smartphone or tablet computer.

(14)-(18) The control system according to the present invention is a control system including an operation side device, an operated side display device, and an operated side main body device.
The operation-side device is an operation reception means for receiving a user's operation input to the operation input unit, and the operated-side display device by the transmission unit via the server device or directly using the received operation as control data. Equipped with a control data transmission means to transmit to
The operated side display device has a control data receiving means for receiving control data transmitted from the operation side device, and at least a part of the display as a control light emitting region, and from the control light emitting region based on the control data. Equipped with a light emission control means that outputs control light,
The operated side main body device is held by the operated side main body device by a holding portion having a holding wall for physically holding the display facing the display of the operated side display device and the holding portion. A light emission detection unit that detects the control light in the control light emission region of the operated side display device and obtains control light data, and a main body device to be operated that performs control processing based on the control light data from the light emission detection unit. It is equipped with side control means.

Therefore, although it has a simple structure, it can be controlled by the operated device from a remote location.

(19)-(23) The videophone system according to the present invention is a videophone system having an operating side videophone device and an operated side videophone device.
The operation-side videophone device is an operation reception means that receives an operation of a user input to the operation input unit, and control data transmission that the transmission unit transmits to the operated-side videophone device using the received operation as control data. Equipped with means,
The operated videophone device directs the camera or microphone of the operated videophone device based on the control data receiving means for receiving the control data transmitted from the operated videophone device and the control data. It is characterized by being equipped with a control means on the operated side to be changed.

Therefore, during a videophone call, the direction of the camera or microphone of the other party can be controlled, and an image that is easy for oneself to see and a voice that is easy to hear can be obtained.

In the present invention, in the embodiment, step S3 corresponds to the "captured image display means" of the mobile device on the operation side.

In the embodiment, step S1 corresponds to the "captured image transmitting means" of the mobile device on the operation side.

In the embodiment, steps S4 and S5 correspond to the "operation receiving means".

In the embodiment, step S7 corresponds to the "control data transmission means".

In the embodiment, step S43 corresponds to the "captured image display means" of the mobile device on the robot side.

In the embodiment, step S41 corresponds to the "captured image transmitting means" of the mobile device on the robot side.

In the embodiment, step S44 corresponds to the "control data receiving means".

In the embodiment, step S45 corresponds to the "light emission control means".

In the embodiment, step S61 corresponds to the "light emission detecting means".

In the embodiment, step S62 corresponds to the "robot side control means".

The "program" is a concept that includes not only a program that can be directly executed by the CPU, but also a source format program, a compressed program, an encrypted program, and the like.

It is a system configuration of a robot system according to an embodiment of the present invention. It is a functional block diagram of a robot system. It is a figure which shows the appearance of the smartphone 10 (30). This is the hardware configuration of the smartphone 10 (30). It is an external view of the robot substrate apparatus 50. It is sectional drawing of the holding part 50 of the robot substrate apparatus 50. It is sectional drawing of the robot substrate apparatus 50. It is a hardware configuration of the robot substrate device 50. This is the hardware configuration of the server device 70. It is a flowchart of an operation process. It is a flowchart of an operation process. This is an example of an image displayed on the operation side smartphone 10. This is an example of an image displayed on the operation side smartphone 10. It is a table for judging the control contents. It is a figure which shows the mechanism for performing a vertical drive. It is a figure which shows the case where a plurality of control light emitting regions are provided. This is an example when clock light and data light are provided. It is a figure which shows the structure of an arm. It is a functional block diagram of the control system by another embodiment. It is a configuration example according to another embodiment. It is a figure which shows the example which receives the control light by the phototransistor of a clip 33. It is a functional block diagram of the videophone system by another embodiment.

1. 1. System Configuration Figure 1 shows the overall configuration of a robot system according to an embodiment of the present invention. The robot-side portable device 30 is mounted on the robot substrate device 50. An operation-side portable device 10 is provided at a distant position so that the robot substrate device 50 and the server device 70 can communicate with each other.

FIG. 2 shows the functional configuration of the robot system. The camera 14 of the operation-side mobile device 10 captures an image of the operation-side user having the operation-side mobile device 10. The captured image transmitting means 20 transmits the captured image to the robot-side portable device 30 by the communication unit 24.

The captured image display means 38 of the robot-side portable device 30 displays the received captured image on the display 32. As a result, the robot side user can see the state of the operation side user.

The camera 34 of the robot-side mobile device 30 captures an image of a robot-side user near the robot-side mobile device 30. The captured image transmitting means 40 transmits the captured image to the operating side portable device 10 by the communication unit 44.

The captured image display means 18 of the operation-side portable device 10 displays the received captured image on the display 12. As a result, the operation side user can see the state of the robot side user.

When the control data is input from the input unit 16 of the operation side mobile device 10 by the operation side user, the control data transmission means 22 transmits this control data to the robot side mobile device 30 by the communication unit 24.

The control data receiving means 42 of the robot-side portable device 30 receives this control data and gives it to the light emission controlling means 36. Based on this control data, the light emission control means 36 blinks a control light emission region provided in a part of the display 32 to emit control light.

The light emission detecting means 32 of the robot substrate device 50 holding the robot-side portable device 30 receives this control light and obtains control light data. This control optical data is given to the robot side control means 55. The robot-side control means 55 controls the drive unit 57 based on the control optical data. The drive unit 57 moves the holding unit that holds the robot-side mobile device 30. As a result, the orientation of the camera 34 of the robot-side mobile device 30 is also changed, and the orientation of the image displayed on the operation-side portable device 10 is also changed.

As described above, the direction of the camera 34 of the robot-side mobile device 30 can be changed to control the imaging direction based on the operation command (control data) of the operation-side user. Therefore, the operation side user can perform the operation as if his / her alter ego is on the side of the robot side portable device 30.

2. 2. Configuration of Each Device FIG. 3 shows the appearance of the smartphone 10 which is an operation side terminal device. Since the appearance of the smartphone 30 which is a terminal device on the robot side is the same, the reference numerals are shown in parentheses.

A touch screen 64 (84) is provided on the front surface of the smartphone 10 (30). A speaker 66 (86) and a camera 72 (92) are provided on the right side. A microphone 70 (90) is provided on the left side.

The touch screen 84 of the smartphone 30, which is a terminal device on the robot side, is provided with a control light emitting region 31 in the lower central portion thereof. The control light emitting region 31 is used as a region for emitting control light.

FIG. 4 shows the hardware configuration of the smartphone 10 (30). The CPU 60 (80) includes a memory 62 (82), a touch display 64 (84), a speaker 66 (86), a non-volatile memory 68 (88), a microphone 70 (90), a camera 72 (92), and a communication circuit 74 ( 94) is connected. The call circuit for the telephone is omitted.

The communication circuit 74 (94) is a circuit for connecting to the Internet. The operating system 76 (96) and the operation side terminal program 78 (robot side terminal program 98) are recorded in the non-volatile memory 68 (88).

The operation side terminal program 78 (robot side terminal program 98) exerts its function in cooperation with the operating system 76 (96).

FIG. 5a shows the appearance of the robot substrate device 50. The main body 52 is provided with four legs 58 so that it can be stably placed on a desk or the like. A holding portion 54 is provided on the upper portion of the main body portion 52. As shown in FIG. 1, a recess 56 for holding the smartphone 30 is provided on the upper surface of the holding portion 54.

FIG. 5b shows a cross section of the robot substrate device 50 near the center of the smartphone 30 mounted in the recess 56. As shown in the figure, the touch display 84 of the smartphone 30 can be tilted and held so as to face upward. The holding unit 54 is provided with a phototransistor 51, which is a photodetector, so as to face the controlled light emitting region 31 of the touch display 84 of the attached smartphone 30.

FIG. 5c shows a cross-sectional view of the robot substrate device 50. The main body 52 is provided with a stepping motor 108 and a control circuit 101. The holding portion 54 is fixed to the shaft of the stepping motor 108 provided in the main body portion 52. Therefore, the orientation of the holding portion 54 can be changed by rotating the stepping motor 108. As a result, the orientation of the smartphone 30 mounted in the recess 56 of the holding portion 54 can be changed.

The phototransistor 51 shown in FIG. 5b is provided on the wall opposite to the recess 56 shown in FIG. 5c.

FIG. 6 shows the hardware configuration of the control circuit 101 of the robot substrate device 50. A memory 102, a non-volatile memory 104, a waveform shaping circuit 107, and a driver circuit 109 are connected to the CPU 100.

The signal from the phototransistor 106 is shaped by the waveform shaping circuit 107 and given to the CPU 100 as data. The driver circuit 109 is a circuit for driving the stepping motor 108 based on the command of the CPU 100.

The operating system 110 and the robot substrate program 112 are recorded in the non-volatile memory 104. The robot substrate program 112 cooperates with the operating system 110 to exert its function. It should be noted that the robot substrate program 112 may be configured to operate independently without using an operating system.

FIG. 7 shows the hardware configuration of the server device. A memory 122, a hard disk 124, and a communication circuit 126 are connected to the CPU 120. The communication circuit 126 is for connecting to the Internet.

The operating system 128 and the server program 130 are recorded on the hard disk 124. The server program 130 exerts its function in cooperation with the operating system 128.

3. 3. Robot operation FIGS. 8 and 9 show a flowchart of robot operation. The CPU 60 of the operation-side smartphone 10 (hereinafter, may be abbreviated as the operation-side smartphone 10) acquires voice data from the microphone 70 and image data from the camera 72, and transmits them to the server device 70 by the communication circuit 74. (Step S1).

The CPU 120 of the server device 70 (hereinafter, may be abbreviated as the server device 70) receives the voice data and the image data via the Internet by the communication circuit 126, and transfers the voice data and the image data to the smartphone 30 on the robot side (step S21). ..

The CPU 80 of the robot-side smartphone 30 (hereinafter, may be abbreviated as the robot-side smartphone 30) receives voice data and image data via the Internet by the communication circuit 94 (step S42). The robot-side smartphone 30 outputs the received voice data from the speaker 86, and displays the received image data on the touch display 84 (step S43).

Therefore, the robot-side user can obtain the image and voice of the operation-side user by the smartphone 30. The controlled light emitting area 31 of the touch display 84 of the robot side smartphone 30 and its vicinity are hidden by the holding unit 54, and the robot side user cannot see the display of that part. However, by making the hidden area smaller, visibility is not impaired.

The robot-side smartphone 30 acquires voice data from the microphone 90 and image data from the camera 92 and transmits them to the server device 70 by the communication circuit 94 (step S41).

The server device 70 receives the voice data and the image data via the Internet by the communication circuit 126, and transfers the voice data and the image data to the operating smartphone 10 (step S21).

The operation-side smartphone 10 receives voice data and image data via the Internet through the communication circuit 74 (step S2). The operating smartphone 10 outputs the received voice data from the speaker 66, and displays the received image data on the touch display 64 (step S3).

Therefore, the operation side user can obtain the image and voice of the robot side user by the smartphone 10. For example, as shown in FIG. 10, the face of the robot-side user and its background can be seen.

Further, the operation side smartphone 10 displays the operation button (control mark) 13 on the screen (step S3). The operation button 13 includes a right rotation button 13a and a left rotation button 13b.

When the user on the operation side taps the right rotation button 13a, the direction of the smartphone 30 on the robot side is rotated to the right for the period of the tap. Similarly, when the left rotation button 13b is tapped, the orientation of the robot-side smartphone 30 is rotated to the left for the tap period.

For example, suppose that the user on the operating side taps the right rotation button 13a in the state shown in FIG. The operation-side smartphone 10 receives the right-handed rotation control input and transmits the right-handed rotation control data to the server device 70 (steps S5 and S7).

Similarly, in the state of FIG. 10, it is assumed that the operating side user taps the left rotation button 13b. The operation side smartphone 10 receives the left rotation control input and transmits the left rotation control data to the server device 70 (steps S5 and S7).

Further, when the operating side user has not tapped any of the buttons 13a and 13b, the operating side smartphone 10 always transmits the stop control data to the server device 70 (steps S6 and S7). This is to prevent accidental rotation due to the influence of noise or the like.

The robot-side smartphone 30 receives this control data (step S44). In the following, the description will be made assuming that the right rotation control data has been received. The robot-side smartphone 30 blinks the control light emitting area 31 (see FIG. 3) of the touch display 84 with the control light corresponding to the right rotation control data (step S45).

In this embodiment, the control light that blinks at 10 Hz is used, and the lighting duty ratio (ratio of the lighting time to the time of one cycle) of the control light is changed according to the control content. For example, in this embodiment, the duty ratio is set to 15% for right rotation control data, and the duty ratio is set to 85% for left rotation control data. Further, in the case of stop control data, the duty ratio is set to 50%. Here, since it is the right rotation control data as described above, the control light having a duty ratio of 15% is emitted.

The control light emitted from the control light emitting region 31 of the touch display 84 is received by the phototransistor 51 provided in the holding portion 54 of the robot substrate device 50 (see FIG. 5b). The phototransistor 51 outputs a pulse signal (control light data) corresponding to the control light. The CPU 100 of the robot substrate device 50 (hereinafter, may be abbreviated as the robot substrate device 50) acquires this control optical data via the waveform shaping circuit 107 (see FIG. 6) (step S61).

The robot substrate device 50 detects the on-duty ratio of the control optical data, and thereby determines whether the content of the control data is clockwise rotation, counterclockwise rotation, or stop. In this embodiment, the content of the control data is determined based on the table as shown in FIG.

Here, since the control light data is based on the control light having a duty ratio of 15%, the duty ratio of this control light data is also 15%. Therefore, the robot substrate device 50 determines that the right rotation command has been given, controls the driver circuit 109, and rotates the stepping motor 108 to the right (step S62). The robot substrate device 50 continues to rotate the stepping motor 108 while the right rotation command is given.

As a result, the holding portion 54 rotates clockwise, and the robot-side smartphone 30 mounted on the holding portion 54 also rotates as a whole. Therefore, the imaging angle of the camera 92 of the robot-side smartphone 30 also changes.

In the flowcharts of FIGS. 8 and 9, it is shown that the transmission of audio / image and the transmission of control data are sequentially performed, but in reality, both processes are performed in parallel. Therefore, when the user on the operating side taps the right-handed rotation button 13a from the state of FIG. 10, the image captured by the camera 92 of the robot-side smartphone 30 rotated to the right, for example, as shown in FIG. 11, is obtained according to the tap time. Obtainable. The operating user can rotate the image to a desired position while viewing the displayed image.

As a result, the operating side user can freely adjust the direction of the camera 92 of the robot side user who is the other party of the call and obtain an image of the angle desired by himself / herself.

4. others
(1) In the above embodiment, the operation side smartphone 10 and the robot side smartphone 30 communicate with each other via the server device 70. However, direct communication may be performed by Bluetooth, Wifi, or the like without going through the server device 70. Alternatively, the communication may be performed via a device other than the server device.

(2) In the above embodiment, the phototransistor 51 is used as the light emission detection unit. However, it may be detected by using a camera or the like.

(3) In the above embodiment, the rotation is performed only while the right rotation button 13a and the left rotation button 13b are tapped. However, each time the button is tapped, the rotation may be performed by a predetermined angle (for example, 5 degrees).

(4) In the above embodiment, the content of the control data (right rotation, stop, left rotation) is shown by changing the duty ratio of the control light. However, the phase, frequency, amplitude, and the like of the control light may be changed or combined with these to indicate the content of the control data.

Also, in the above, only clockwise rotation, counterclockwise rotation, and stop are performed, but by setting the duty ratio finely (every 10%, etc.), more control contents can be shown.

For example, in the robot substrate device 50, the holding portion 54 is configured to be driven not only to the left and right but also to the top and bottom. Then, right rotation, left rotation, stop, up rotation, down rotation, and stop are set to duty ratios of 10%, 30%, 50%, 70%, and 90%, respectively. By doing so, it is possible to control not only the left and right but also the top and bottom.

FIG. 13 shows a robot substrate device 50 for rotating the smartphone 30 up / down / left / right. A holding portion 54 is provided on the main body portion 52. The holding portion 54 includes a first base member 54a, a vertical member 54b, a second base member 54c, and a groove-shaped member 54d.

The first base 54 is provided so that it can rotate in the X direction of the arrow. The rotation mechanism is the same as in FIG. 5c. A vertical member 54b extending in the vertical direction is provided at one end of the circumference of the first base member 54a, and rotates in the same manner as the first base member 54a rotates.

The vertical member 54b is provided with a second base member 54c that rotates in the Y direction of the arrow. The rotation mechanism is the same as in FIG. 5c. A groove-shaped member 54d is provided at one end of the circumference of the second base member 54a, and rotates in the same manner as the second base member 54a rotates.

The groove-shaped member 54d has a U-shaped groove, and the lower end of the smartphone 30 is held by this groove. Therefore, the camera (not shown) provided in the smartphone 30 is rotated in the vertical and horizontal directions by the rotation of the first base member 54a and the second base member 54d. The groove-shaped member 54d is provided with a phototransistor 51 as in FIG. 5b.

When controlling the robot substrate device 50 as shown in FIG. 13, it is preferable to provide not only the left and right arrow buttons 13a and 13b shown in FIG. 10 but also the up and down arrow buttons.

(5) In the above embodiment, the control light emitting region 31 is provided at only one place. However, if a plurality of locations are provided, more control contents can be realized. For example, as shown in FIG. 14, by providing two control light emitting regions 31a and 31b, two types of control signals can be transmitted at the same time. In this case, the holding unit 54 is provided with a phototransistor 51 corresponding to each of the control light emitting regions 31a and 31b.

Further, the control light emitted from the control light emission region 31a may be used as clock light, and the data light indicated by "1" and "0" may be emitted from the control light emission region 31b. "1" and "0" are transmitted depending on the value of the data light at the rising edge of the clock light. As a result, a large amount of information can be transmitted by the combination of "1" and "0" to perform complicated control.

(6) In the above embodiment, the smartphone 10 (30) is used as the mobile terminal device. However, a tablet computer, PDA, or the like may be used. Further, instead of the mobile terminal device, a stationary PC or the like may be used. In this case, on the robot side, at least the camera 92 may be held and driven by the holding unit 64.

(7) In the above embodiment, the operation side user controls to change the direction of the robot side smartphone 30. However, other drive control may be performed.

For example, the arm structure (arm mechanism) as shown in FIG. 16A may be driven and controlled. This structure can be attached to the left and right of the holding portion 54 of the robot substrate device 50. The arm fixing portion 152 is fixed to the holding portion 54. A first rotating portion 154 is attached to the arm fixing portion 152 so as to be rotatable in the A direction. A second rotating portion 156 is attached to the first rotating portion 154 so as to be rotatable in the B direction.

FIG. 16B shows the internal mechanism of the arm structure. Inside the arm fixing portion 152, an arm fixing portion base member 152a is provided. The arm fixing portion 152 is configured by covering the outside of the arm fixing portion base member 152a with a housing.

A rotation shaft 162 is rotatably provided on the arm fixing portion base member 152a. The rotary shaft 162 is configured to be rotatable in the direction of arrow A by a stepping motor (not shown). The first rotating portion base member 154a of the first rotating portion 154 is fixed to the rotating shaft 162. The first rotating portion 154 is configured by covering the outside of the first rotating portion base member 154a with a housing. Therefore, by controlling the stepping motor, the first rotating portion 154 can be rotated in the direction of arrow A with respect to the arm fixing portion 152.

The second rotating portion base member 156a is rotatably fixed to the first rotating portion base member 154a by the rotating shaft 164. The rotary shaft 164 is configured to be rotatable in the direction of arrow B by another stepping motor (not shown). A second rotating portion base member 156a is fixed to the rotating shaft 164. The second rotation portion 156 is configured by covering the outside of the second rotation base member 156a with a housing. Therefore, by controlling the stepping motor, the second rotating portion 156 can be rotated in the direction of arrow B with respect to the first rotating portion 154.

By providing the arm mechanism in this way, the expressive power of the robot can be increased. The holding portion 54 of the robot substrate device 50 may be fixed to the main body portion 52 to control only the arm mechanism.

(8) In the above embodiment, the orientation of the robot-side smartphone 30 mounted on the robot-based device 50 is controlled from the operation side. However, when the robot-side substrate device 50 is a robot that performs various tasks, the operation side can give an instruction to the robot to perform the tasks. For example, in the case of a cooking robot, while controlling the cooking robot from the operation side to cook, it is possible to teach the robot side user the cooking by using both image and voice calls.

That is, according to the present invention, by attaching a smartphone to the robot, it is possible to make an image / voice call while remotely controlling the robot. Further, since the angle of the camera and the microphone of the robot side smartphone 30 can be controlled remotely, the situation on the robot side can be accurately known.

(9) In the above embodiment, the control for driving the mechanism of the robot substrate device 50 from the operation side has been described. However, it may be controlled to drive a mechanism of a device other than the robot substrate device 50. For example, the operation of the cleaning robot configured separately from the robot substrate device 50 may be controlled.

Also, it can be used not for the control that drives the mechanism but for other controls in general. FIG. 17 shows the functional configuration.

When the operation side user inputs an input by operating the input unit 14 of the operation side device 10, the operation reception means 16 accepts the input. The control data transmission means 22 transmits the control data corresponding to the received input by the communication unit 24 to the operated side display device 30.

The control data receiving means 42 of the operated side display device 30 receives this by the communication unit 44. The light emission control means 36 emits control light for the control data from a part of the display 32.

The light emission detecting means 52 of the main body device 50 on the operated side receives the control light. The control means 54 on the operated main body side performs predetermined control based on the control light.

For example, as shown in FIG. 18, the lower part of the screen 184 of the display 180 (video conference display) on the robot side is covered with the support member 186. The screen 184 is provided with control light emitting regions 31a, 31b, 31c, 31d, 31e. A phototransistor is provided inside the support member 186 so as to face this. The state of this room is transmitted to the other party (operation side) tablet or the like by the camera 182.

The user on the operating side can operate the tablet to send control data, and the phototransistor can acquire this and change the brightness of the lighting in this room, for example. Alternatively, the magnification and focus of the camera 182 can be controlled.

As described above, in the present invention, the robot is a concept including not only a driving one but also a controlling one.

(10) In the above embodiment, blinking light is used as control light. However, instead of or in addition to this, the color of light may be used to convey information. In this case, a phototransistor capable of detecting color, a camera, or the like is used.

(11) In the above embodiment, information is transmitted by blinking light. However, a code such as a QR code (trademark) may be displayed in the control light emitting area 31 and read.

(12) In the above embodiment, the rotation command is given by tapping the buttons 13a and 13b as shown in FIG. However, you may give a command to rotate the camera in that direction by sliding it on the screen with your finger.

(13) In the above embodiment, the position of the controlled light emitting region 31 on the touch display 84 of the smartphone 30 is predetermined. However, the smartphone 30 itself detects a change in the positional capacitance due to mounting on the recess 56 (preferably a change at a level at which it is not determined that there was a touch input), and the portion is controlled by the control light emitting region 31. You may try to. By doing so, the control light can be reliably transmitted even if the mounting position is displaced.

(14) In the above embodiment, the control light emitting region 31 is provided in the lower part of the smartphone 30, and the smartphone 30 is supported by this portion. However, the control light emitting region 31 may be provided in the upper portion, the side portion, or the like, and the smartphone 30 may be supported by this portion.

Further, as shown in FIGS. 19A and 19B, the clip 33 may cover an arbitrary position of the touch display 84 of the robot-side smartphone 30. The clip 33 is provided with a photodiode, the signal of which is transmitted to the robot by a control line (not shown) or by radio. In this case, it is preferable that the control light emitting region 31 is set by the smartphone 30 itself as described above.

(15) In the above embodiment, the robot substrate device 50 and the robot-side portable device 30 are configured as separate devices. However, these may be integrated into a robot device. In this case, the control data may not be converted into the control light, and the received control data may be used as it is.

This makes it possible to control the direction of the other party's camera (or microphone), for example, in a videophone. FIG. 20 shows the functional configuration.

When the operation side user inputs an input by operating the input unit 14 of the operation side videophone device 10, the operation reception means 16 accepts the input. The control data transmission means 22 transmits the control data corresponding to the received input by the communication unit 24 to the videophone device 30 on the operated side.

The control data receiving means 42 of the videophone device 30 on the operated side receives this by the communication unit 44. The operated side control means 36 changes the direction of the camera or the microphone 34 based on the control data. Therefore, the operating side user can adjust the image and sound transmitted from the operated side videophone device 30.

The videophone system as described above can also be realized by the configuration shown in FIG.

(16) In the above embodiment, a stepping motor is used. However, instead of this, a normal motor such as a servo motor may be used.

(17) Each of the above-mentioned modifications can be implemented in combination with each other.

Claims (23)

In a robot system equipped with a mobile device on the operation side, a mobile device on the robot side, and a robot substrate device,
The operation-side portable device is
An image capture image display means for displaying the image captured on the robot side transmitted from the mobile device on the robot side on a touch display, and
An image transmitting means for transmitting an operation-side image captured by a camera to the robot-side portable device by a transmission unit via a server device or directly.
An operation reception means that accepts user operations input to the touch display,
A control data transmission means for transmitting the received operation as control data to the robot-side mobile device by the transmission unit via the server device or directly is provided.
The robot-side portable device is
An image capture image display means for displaying an operation side image transmitted from an operation side mobile device on a touch display, and
An image transmitting means for transmitting a robot-side image captured by a camera to the operation-side portable device by a transmission unit via a server device or directly.
A control data receiving means for receiving the control data transmitted from the operation-side mobile device, and
A control light emitting region is provided at least a part of the touch display, and a light emission control means for controlling light emission in the control light emission region based on the control data and transmitting the control light is provided.
The robot substrate device is
With the main body
A holding unit that is movably provided with respect to the main body by a driving unit and has a holding wall for physically holding the touch display facing the touch display of the robot-side mobile device. ,
A light emission detecting means for detecting control light in the control light emission region of the robot-side portable device held by the holding unit in the robot substrate device by the light emission detection unit and obtaining control light data.
Based on the control light data from the light emission detecting means, at least, a robot-side control means for controlling the holding portion for holding the robot-side portable device to be moved by the drive unit, and a robot-side control means.
A robot system equipped with. An image capture image display means for displaying the image captured on the robot side transmitted from the mobile device on the robot side on a touch display, and
An image transmitting means for transmitting an operation-side image captured by a camera to the robot-side portable device by a transmission unit via a server device or directly.
An operation reception means that accepts user operations input to the touch display,
In order for the robot-side mobile device to control the light emission state of the control light emitting area of the touch display based on the control data and to control the drive unit of the robot substrate device in which the robot-side portable device is held by the control light data. A control data transmission means for transmitting the received operation as control data to the robot-side mobile device by the transmission unit via the server device or directly.
Operation side mobile device equipped with. An operation-side mobile device program for realizing an operation-side mobile device used in a robot system by a computer.
An image capture image display means for displaying the image captured on the robot side transmitted from the mobile device on the robot side on a touch display, and
An image pickup transmitting means for transmitting an operation-side image captured by a camera to the operation-side mobile device via a server device or directly by a transmission unit to the robot-side mobile device.
An operation reception means that accepts user operations input to the touch display,
In order for the robot-side mobile device to control the light emission state of the control light emitting area of the touch display based on the control data and to control the drive unit of the robot substrate device in which the robot-side portable device is held by the control light data. An operation-side mobile device program for functioning as a control data transmission means for transmitting the received operation as control data to the robot-side mobile device by a transmission unit via a server device or directly. An image capture image display means for displaying an operation side image transmitted from an operation side mobile device on a touch display, and
An image transmitting means for transmitting a robot-side image captured by a camera to the operation-side portable device by a transmission unit via a server device or directly.
A control data receiving means for receiving the control data transmitted from the operation-side mobile device, and
In order to receive the control light by the light emission detection unit provided in the robot base device holding the robot-side portable device and to control the drive unit of the robot base device based on the control light data by the control light, the touch display is used. A light emission control means that controls light emission in the control light emission region based on the control data and emits control light, with at least a part thereof as a control light emission region.
Robot side mobile device equipped with. A robot-side mobile device program for realizing a robot-side mobile device by a computer.
An image capture image display means for displaying an operation side image transmitted from an operation side mobile device on a touch display, and
An image transmitting means for transmitting a robot-side image captured by a camera to the operation-side portable device by a transmission unit via a server device or directly.
A control data receiving means for receiving the control data transmitted from the operation-side mobile device, and
In order to receive the control light by the light emission detection unit provided in the robot base device holding the robot-side portable device and to control the drive unit of the robot base device based on the control light data by the control light, the touch display is used. A robot-side portable device program for controlling at least a part of the light emission in the control light emission region based on the control data and functioning as a light emission control means for transmitting the control light. With the main body
A holding unit that is movably provided with respect to the main body by a driving unit and has a holding wall for physically holding the touch display facing the touch display of the robot-side mobile device. ,
A light emission detecting means for detecting control light in the control light emission region of the robot-side portable device held by the holding unit in the robot substrate device by the light emission detection unit and obtaining control light data.
Based on the control light data from the light emission detecting means, at least, a robot-side control means for controlling the holding portion for holding the robot-side portable device to be moved by the drive unit, and a robot-side control means.
Robot base device equipped with. A robot-based device program for realizing a robot-based device by a computer.
A light emission detecting means for detecting the control light in the control light emission region of the robot-side portable device held by the holding unit in the robot substrate device by the light emission detection unit and obtaining control light data.
A robot substrate device program for functioning as a robot-side control means for controlling at least a holding portion for holding the robot-side portable device to be moved by the drive unit based on control optical data from the light emission detecting means. In any of the systems, devices or programs of claims 1-7.
The drive unit is a system, device, or program including a motor for rotating a holding unit that holds the robot-side portable device. In any of the systems, devices or programs of claims 1-8.
The drive unit is, at least, a system, device, or program that drives an arm member provided on the robot substrate device. In any of the systems, devices or programs of claims 1-9.
The light emission control means of the robot-side portable device is a system, device or program characterized in that control light having a different duty ratio is output from the control light emission region according to received control data. In the system, apparatus or program of claim 10.
The light emitting means is, at least, a system, an apparatus, or a system, an apparatus, characterized in that the duty ratio of the control light is changed depending on whether the control data is an operation content for driving the drive unit or an operation content for stopping the drive unit. program. In the system, apparatus or program of claim 11.
The drive unit includes a motor for rotating the holding unit that holds the robot-side mobile device.
The light emitting means has at least a duty ratio of the control light depending on whether the control data is an operation content for rotating the motor clockwise, an operation content for rotating the motor counterclockwise, or an operation content for stopping the motor. A system, device or program characterized by change. In any of the systems, devices or programs of claims 1-12.
An operating-side portable device or a robot-side portable device is a system, device, or program characterized by being a smartphone or tablet computer. In a control system including an operation side device, an operated side display device, and an operated side main body device,
The operation side device is
An operation receiving means that accepts the user's operation input to the operation input unit,
A control data transmission means for transmitting the received operation as control data to the operated side display device by the transmission unit via the server device or directly is provided.
The operated side display device is
A control data receiving means for receiving the control data transmitted from the operation side device, and
A control light emitting area is provided at least a part of the display, and a light emission control means for outputting control light from the control light emission area based on the control data is provided.
The operated main body device is
A holding portion having a holding wall for physically holding the display facing the display of the operated display device, and a holding portion.
A light emitting detection unit that detects the control light in the controlled light emitting region of the operated side display device held by the holding unit in the operated side main body device and obtains control light data.
A control means on the side of the main unit to be operated that performs control processing based on the control light data from the light emission detection unit.
Control system with. An operation-side captured image display means for displaying an operation-side captured image transmitted from an operation-side display device on a display, and an operation-side captured image display means.
A control data receiving means for receiving the control data transmitted from the operation side display device, and
In order to receive the control light by the light emission detection unit provided in the operated side main body device holding the operated side display device and to control the drive unit of the operated side main body device based on the control light data by the control light. A light emission control means that outputs control light from the control light emission region based on the control data, wherein at least a part of the display is a control light emission region.
Operated side display device equipped with. The operated side program for realizing the operated side display device by the computer, and the computer.
An operation-side captured image display means for displaying an operation-side captured image transmitted from an operation-side display device on a display, and an operation-side captured image display means.
A control data receiving means for receiving the control data transmitted from the operation side display device, and
In order to receive the control light by the light emission detection unit provided in the operated side main body device holding the operated side display device and to control the drive unit of the operated side main body device based on the control light data by the control light. , A program on the operated side for using at least a part of the display as a control light emission region and functioning as a light emission control means for outputting control light from the control light emission region based on the control data. A holding unit having a holding wall for physically holding the display facing the display of the display device on the operated side,
A light emitting detection unit that detects the control light in the controlled light emitting region of the operated side display device held by the holding unit in the main body device on the operated side and obtains control light data.
A control means on the side of the main unit to be operated that performs control processing based on the control light data from the light emission detection unit.
Operated side main unit equipped with. A holding portion having a holding wall for physically holding the display facing the display of the operated side display device, and the control of the operated side display device held by the holding portion on the operated side main body device. An operated main unit program for realizing a controlled main unit device having a light emitting detection unit that detects control light in a light emitting region and obtains controlled light data by a computer, and is a computer.
A program to be operated that functions as a control means on the device to be operated to perform control processing based on the control light data from the light emission detection unit. In a videophone system having a videophone device on the operating side and a videophone device on the operated side,
The operation side videophone device
An operation reception means that accepts user operations input to the operation input unit,
A control data transmission means for transmitting the received operation as control data to the operated videophone device by the transmission unit is provided.
The videophone device on the operated side is
A control data receiving means for receiving the control data transmitted from the videophone device on the operating side, and
Based on the control data, the operated side control means that changes the direction of the camera or microphone of the operated side videophone device, and
A videophone system featuring a videophone system. An operation reception means that accepts user operations input to the operation input unit,
A control data transmitting means for transmitting the received operation as control data to the operated videophone device by the transmission unit in order to change the direction of the camera or microphone of the operated videophone device.
Operation side videophone device equipped with. An operation-side videophone program for realizing an operation-side videophone device by a computer, which is a computer.
An operation reception means that accepts user operations input to the operation input unit,
The operation side for functioning as a control data transmission means for transmitting the received operation as control data to the operated videophone device by the transmitting unit in order to change the direction of the camera or microphone of the operated videophone device. Videophone program. Control data receiving means for receiving control data transmitted from the videophone device on the operating side, and
Based on the control data, the operated side control means that changes the direction of the camera or microphone of the operated side videophone device, and
The operated side videophone device equipped with. A videophone program on the operated side for realizing a videophone device on the operated side by a computer.
Control data receiving means for receiving control data transmitted from the videophone device on the operating side, and
An operated videophone program for functioning as an operated control means for changing the direction of the camera or microphone of the operated videophone device based on the control data.

Images