WO2024189740A1 - Remote control system, remote control method, rpa device, and rpa program - Google Patents

Abstract

A remote control system according to one aspect of the present invention causes a control device to control equipment installed in a test environment and comprises a storage unit which stores a control pattern for controlling the equipment to perform a predetermined action, an execution unit which causes the control device to execute the control of the equipment on the basis of the control pattern stored in the storage unit, an acquisition unit which acquires at least any piece of information in the test environment and information relating to the equipment, and a changing unit which changes the control pattern stored in the storage unit on the basis of the information acquired by the acquisition unit.

Description

Remote control system, remote control method, RPA device, and RPA program

The present invention relates to a remote control system, a remote control method, an RPA device, and an RPA program.

The Internet of Things (IoT), which aims to connect all kinds of things to the Internet, is expected to be the next generation of information and communications technology. Remote control systems that utilize these devices are attracting particular attention. Furthermore, the demand for remote control has only increased due to the COVID-19 pandemic.

For example, Non-Patent Document 1 discloses an architecture for controlling a robot via a wireless network in a situation where sensors are placed to monitor the state of the robot in the environment in which the robot operates.

In addition, a technology is known in which a control device remotely controls robots, measuring instruments, and other devices in a verification environment from a remote environment via a network, and collects information collected by each device located in the verification environment via the network.

Yuichi Ohshita and 5 others, "A study on architecture for wireless remote control of robots using environmental sensors," Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, 2018, pp.9-14

However, when trying to remotely verify a system that is in the development stage, for example when researching and developing a new system, the following issues can arise:

1) When the verification environment contains multiple pieces of hardware and software that are each to be controlled (verified), each control object needs to be controlled individually, which can complicate the configuration for remote verification.
2) Depending on the verification results, the next content to be verified may change in real time.
3) When the verification target is a prototype, for example, the verification target may not function properly due to an error or other reason.
4) When multiple people share a single verification environment to perform verification, time efficiency may decrease.

The present invention aims to provide a remote control system, a remote control method, an RPA device, and an RPA program that can change the operation to be executed by a device installed in a verified environment from the operation to be executed initially when information about the device installed in the verified environment or information within the verified environment changes.

A remote control system according to one aspect of the present invention is a remote control system that controls one or more devices installed in a verification environment using a control device, and is characterized in having a memory unit that stores a control pattern for controlling the device to perform a predetermined operation, an execution unit that causes the control device to execute control of the device based on the control pattern stored in the memory unit, an acquisition unit that acquires at least one of information within the verification environment and information about the device, and a modification unit that modifies the control pattern stored in the memory unit based on the information acquired by the acquisition unit.

Further, a remote control method according to one aspect of the present invention includes:
A remote control method for controlling one or more devices installed in a verification environment using a control device includes having an RPA device control the control device based on a control pattern stored in a memory unit to cause the device to perform a specified operation, obtaining at least one of information within the verification environment and information related to the device, and changing the control pattern stored in the memory unit based on the obtained information.

An RPA device according to one aspect of the present invention is an RPA device that causes a control device to control one or more devices installed in a verification environment, and is characterized in having a memory unit that stores a control pattern for controlling the device to perform a predetermined operation, an execution unit that causes the control device to execute control of the device based on the control pattern stored in the memory unit, an acquisition unit that acquires at least one of information within the verification environment and information related to the device, and a modification unit that modifies the control pattern stored in the memory unit based on the information acquired by the acquisition unit.

According to the present invention, when information about a device installed in a verified environment or information within the verified environment changes, the operation to be performed by the device installed in the verified environment can be changed from the operation that was originally to be performed.

FIG. 1 illustrates an example of a configuration of a remote control system according to an embodiment. A functional block diagram illustrating functions of an RPA device. FIG. 2 is a diagram illustrating a first remote control method by the remote control system. FIG. 11 is a diagram illustrating a second remote control method by the remote control system. FIG. 1 is a diagram showing a first modified example of the configuration of a remote control system. FIG. 13 is a diagram showing a second modified example of the configuration of the remote control system. FIG. 11 is a diagram illustrating a third remote control method by the remote control system. FIG. 13 is a diagram showing a third modified example of the configuration of the remote control system. FIG. 13 is a diagram showing a fourth modified example of the configuration of the remote control system. FIG. 11 is a diagram illustrating a fourth remote control method by the remote control system. FIG. 2 is a diagram illustrating an example of a hardware configuration of an RPA device. FIG. 1 is a diagram illustrating an example of a configuration of a remote control system of a comparative example.

First, the background to the invention will be explained with reference to FIG. 12. FIG. 12 is a diagram showing an example of the configuration of a remote control system 1 as a comparative example. As shown in FIG. 12, the remote control system 1 is a verification system in which a verification environment 2 is connected to a remote environment 4 via a network 3.

For example, multiple control devices 5 are placed in the remote environment 4. The control devices 5 each control multiple devices placed in the verification environment 2 via the network 3.

In addition, for example, a robot 6, measuring devices 7-1 to 7-n, and sensors 8-1 to 8-m are placed in the verification environment 2. The robot 6 is a machine that performs a predetermined operation. The measuring devices 7-1 to 7-n each measure the operation of the robot 6 or values related to the operation of the robot 6, and output the measurement results to the control device 5 via the network 3. The sensors 8-1 to 8-m each detect changes in information that occur in the verification environment 2, and output the detection results to the control device 5 via the network 3.

The robot 6, measuring devices 7-1 to 7-n, and sensors 8-1 to 8-m are devices controlled by multiple control devices 5 via the network 3. The robot 6 and measuring devices 7-1 to 7-n are verification targets installed in the verification environment 2, for example.

However, if some of these devices are prototypes, the devices may not operate normally due to an error, and the events to be verified within the verification environment 2 may not be verified.

The remote control system in one embodiment is configured to be able to change the operation to be performed by the device installed in the verified environment from the operation that was originally to be performed when information about the device installed in the verified environment or information within the verified environment changes.

Below, an example of the configuration of a remote control system 10 according to one embodiment will be described with reference to the drawings. Figure 1 is a diagram showing an example of the configuration of a remote control system 10 according to one embodiment.

As shown in FIG. 1, the remote control system 10 in one embodiment is a verification system in which a verification environment 20 is connected to a remote environment 40 via a network 30.

For example, the remote environment 40 includes a plurality of control devices 50 and an RPA (Robotic Process Automation) device 90 that controls each of the control devices 50 based on a predetermined control pattern. The control devices 50 individually control each of the plurality of devices arranged in the verification environment 20 via the network 30.

In addition, for example, a robot 60, measuring devices 70-1 to 70-n, and sensors 80-1 to 80-m are placed in the verification environment 20. The robot 60 is a machine that performs a predetermined operation. The measuring devices 70-1 to 70-n each measure the operation of the robot 60 or values related to the operation of the robot 60, and output the measurement results to the control device 50 via the network 30. Each of the sensors 80-1 to 80-m is, for example, a camera, which detects changes in information that occur in the verification environment 20 and outputs the detection results to the control device 50 via the network 30.

The robot 60, the measuring devices 70-1 to 70-n, and the sensors 80-1 to 80-m are devices controlled by multiple control devices 50 via the network 30. The robot 60 and the measuring devices 70-1 to 70-n are verification targets installed in the verification environment 2, for example.

Next, specific examples of functions possessed by the RPA device 90 will be described. FIG. 2 is a functional block diagram illustrating the functions possessed by the RPA device 90. As shown in FIG. 2, the RPA device 90 has, for example, a user interface (UI unit) 91, a memory unit 92, an execution unit 93, an acquisition unit 94, a determination unit 95, and a change unit 96.

The user interface 91 is a UI (user interface) unit that has the function of performing input and output for the user to operate the RPA device 90.

The memory unit 92 stores control patterns for controlling the devices controlled by the control device 50 to perform predetermined operations.

The execution unit 93 includes a synchronization unit 97, and causes the control device 50 to control the devices in the verification environment 2 via the network 30 based on the control pattern stored in the memory unit 92. For example, the synchronization unit 97 synchronizes the timing at which the device starts operating with the operation of the execution unit 93, for example, based on information such as a response time acquired by an acquisition unit 94 (described later) from the devices placed in the verification environment 2.

The acquisition unit 94 acquires at least one of information within the verification environment 2 and information regarding the devices within the verification environment 2 via the network 30, and outputs the acquired information to the execution unit 93, the determination unit 95, and the change unit 96. For example, the acquisition unit 94 acquires information such as the results of measurements made by each of the measuring instruments 70-1 to 70-n within the verification environment 2 and the results detected by each of the sensors 80-1 to 80-m via the network 30.

The acquisition unit 94 may also acquire the response time from when the execution unit 93 starts to cause the control device 50 to execute control of the device to when the device actually starts operating based on the control pattern stored in the memory unit 92.

When the execution unit 93 causes the control device 50 to control a device in the verification environment 2, the determination unit 95 determines whether a predetermined event has occurred in the verification environment 2 and outputs the determination result to the modification unit 96.

The change unit 96 changes the control pattern stored in the memory unit 92 based on the information acquired by the acquisition unit 94. For example, when the determination unit 95 determines that a predetermined event has not occurred in the verification environment 2, the change unit 96 changes the control pattern stored in the memory unit 92 to a new control pattern.

Next, a first remote control method using the remote control system 10 will be described. Figure 3 is a diagram illustrating an example of the first remote control method using the remote control system 10.

As shown in FIG. 3, first, the RPA device 90 sets a control scenario for each of the multiple control devices 50 based on the control pattern stored in the memory unit 92 (S100).

The multiple control devices 50 operate the robot 60 and individually control the measuring devices 70-1 to 70-n (the first device to the nth device), and the values related to the operation of the robot 60 measured by the measuring devices 70-1 to 70-n are regarded as the results of the operation check (S102 to S108).

The multiple control devices 50 also collect information from the sensors 80-1 to 80-m (various sensors) on the results of checking the operation of the measuring devices 70-1 to 70-n and on changes in information occurring within the verification environment 20 (S110).

Then, when the determination unit 95 determines that a predetermined event has not occurred in the verification environment 2 based on the information collected from the sensors 80-1 to 80-m, the control device 50 sets a new control scenario for each of the multiple control devices 50 by changing the control pattern stored in the memory unit 92 to a new control pattern (S100).

Next, a second remote control method using the remote control system 10 will be described. FIG. 4 is a diagram illustrating an example of the second remote control method using the remote control system 10. In FIG. 4 and other figures, operations that are substantially the same as those described above using FIG. 3 are denoted by the same reference numerals.

As shown in FIG. 4, first, the RPA device 90 performs settings to synchronize the measuring devices 70-1 to 70-n (the first device to the nth device) by operating, for example, the synchronization unit 97 (S200 to S202).

In other words, even if the measuring devices 70-1 to 70-n do not have a time synchronization function on their own, the RPA device 90 can synchronize the measuring devices 70-1 to 70-n. Then, the remote control system 10 continues to perform the operation shown in FIG. 3.

FIG. 5 is a diagram showing a first modified example (remote control system 10a) of the configuration of the remote control system 10. As shown in FIG. 5, the remote control system 10a according to one embodiment is a verification system in which a verification environment 20a is connected to a remote environment 40a via a network 30. Below, components that are substantially the same as those of the remote control system 10 shown in FIG. 1 are given the same reference numerals.

For example, an RPA device 90 is placed in the remote environment 40a. In addition, in the verification environment 20a, for example, multiple control devices 50, a robot 60, measuring devices 70-1 to 70-n, and sensors 80-1 to 80-m are placed.

In this way, in the remote control system 10a, the multiple control devices 50 that were placed in the remote environment 40 in the remote control system 10 are placed in the verification environment 20a. Therefore, in the remote control system 10a, the functions and configuration in the remote environment 40a are simplified, and the load on the network 30 is also reduced.

FIG. 6 is a diagram showing a second modified example (remote control system 10b) of the configuration of the remote control system 10. As shown in FIG. 6, the remote control system 10b according to one embodiment is a verification system in which a verification environment 20b is connected to a remote environment 40b via a network 30.

For example, in the remote environment 40b, a Web conference tool 100 capable of inputting and outputting images, audio, text, etc. is arranged as a user interface. In addition, in the verification environment 20b, for example, the Web conference tool 100, an RPA device 90, multiple control devices 50, a robot 60, measuring devices 70-1 to 70-n, and sensors 80-1 to 80-m are arranged.

The RPA device 90 analyzes the information received by the web conferencing tool 100 placed in the verification environment 20b (identifying the verification scenario through voice recognition, character recognition using chat on the web conference, and reaction buttons and image recognition on the web conference), and controls the robot 60 and measuring devices 70-1 to 70-n according to the verification content that has been grasped.

The Web conferencing tool 100 placed in the remote environment 40b can also control the robot 60 and measuring instruments 70-1 to 70-n based on information from sensors 80-1 to 80-m such as cameras. For example, the Web conferencing tool 100 placed in the remote environment 40b can also control the verification content in the verification environment 20b using images (gestures), voice (verification content), or text (verification content, including reactions within the Web conferencing tool 100).

For example, a worker performing operations in the remote environment 40b can obtain progress information, such as errors, sent from the verification environment 20b via the web conferencing tool 100, and can grasp events in the verification environment 20b in real time.

In other words, in the remote control system 10b, the execution unit 93 of the RPA device 90 causes multiple control devices 50 to control devices in the verification environment 20b based on at least one of images, voice, and text input via the Web conference tool 100, which serves as a user interface.

The remote control system 10b can therefore control devices in the verification environment 20b using images, sounds, or text intended to indicate the verification content.

FIG. 7 is a diagram illustrating a third remote control method using the remote control system 10b. As shown in FIG. 7, first, the worker sets a control scenario in the remote control system 10b using the Web conference tool 100 arranged in the remote environment 40b (S300).

Next, the RPA device 90 sets a control scenario for the multiple control devices 50 based on the control scenario set using the Web conference tool 100 (S302). Then, the remote control system 10b continues to perform the same operation as shown in FIG. 3.

FIG. 8 is a diagram showing a third modified example (remote control system 10c) of the configuration of the remote control system 10. As shown in FIG. 8, the remote control system 10c according to one embodiment is a verification system in which a verification environment 20b is connected to a plurality of remote environments 40b via a network 30.

For example, in each remote environment 40b, a Web conference tool 100 capable of inputting and outputting images, audio, text, etc. is arranged as a user interface. In addition, in the verification environment 20b, for example, a Web conference tool 100, an RPA device 90, multiple control devices 50, a robot 60, measuring devices 70-1 to 70-n, and sensors 80-1 to 80-m are arranged.

Therefore, in the remote control system 10c, multiple workers can verify the operation of devices placed in the verification environment 20 while monitoring events occurring in the verification environment 20b.

FIG. 9 is a diagram showing a fourth modified example (remote control system 10d) of the configuration of the remote control system 10. As shown in FIG. 9, the remote control system 10d according to one embodiment is a verification system in which multiple verification environments 20b are connected to a remote environment 40b via a network 30.

In other words, in the remote control system 10d, multiple verification environments 20b are provided, each of which has the same device, and the execution unit 93 causes the control device 50 to execute control so that the devices in each of the multiple verification environments 20b are controlled based on the same control pattern or different control patterns.

For example, when there are multiple equivalent verification environments 20b, the execution unit 93 can distribute the verification content according to the resource status of each verification environment 20b. In addition, the execution unit 93 can also verify the same verification content in each of multiple different verification environments 20b.

FIG. 10 is a diagram illustrating a fourth remote control method by the remote control system 10d. As shown in FIG. 10, the remote control system 10d, for example, has the RPA devices 90 in each verification environment 20b control the measuring devices 70-1 to 70-n (the first device to the nth device) and acquire the response times of the measuring devices 70-1 to 70-n (the first device to the nth device) (S400 to S406). The remote control system 10d then continues to perform the same operations as those shown in FIG. 7.

In this way, the remote control system 10 to 10d according to one embodiment changes the control pattern stored in the memory unit 92 based on the information acquired by the acquisition unit 94. Therefore, when the information about the devices installed in the verified environment or the information within the verified environment changes, the operation to be performed by the devices installed in the verified environment can be changed from the operation to be initially performed, and the operation of each device can be adjusted.

In addition, the remote control systems 10 to 10d in one embodiment are equipped with an RPA device 90, which makes it easy to control each device in an integrated manner and allows for centralized control of multiple verification equipment.

In addition, each function of the RPA device 90 may be configured in part or in whole by hardware such as a PLD (Programmable Logic Device) or an FPGA (Field Programmable Gate Array), or may be configured as a program executed by a processor such as a CPU.

For example, the RPA device 90 can be realized using a computer and a program, and the program can be recorded on a storage medium or provided via a network.

FIG. 11 is a diagram showing an example of the hardware configuration of an RPA device 90 according to one embodiment. As shown in FIG. 11, the RPA device 90 has an input unit 900, an output unit 901, a communication unit 902, a CPU 903, a memory 904, and a HDD 905 connected via a bus 906, and has the functions of a computer. The RPA device 90 is also capable of inputting and outputting data to and from a computer-readable storage medium 907.

The input unit 900 is, for example, a keyboard and a mouse. The output unit 901 is, for example, a display device such as a display. The communication unit 902 is, for example, a network interface.

The CPU 903 controls each component of the RPA device 90 and performs predetermined processing. The memory 904 and HDD 905 correspond to the above-mentioned storage unit 92 that stores data, etc.

The storage medium 907 is capable of storing programs and the like that cause the RPA device 90 to execute functions. Note that the architecture that constitutes the RPA device 90 is not limited to the example shown in FIG. 11.

10, 10a, 10b, 10c, 10d... Remote control system, 20, 20a, 20b... Test environment, 30... Network, 40, 40a, 40b... Remote environment, 50... Control device, 60... Robot, 70-1 to 70-n... Measuring device, 80-1 to 80-m... Sensor, 90... RPA device, 91... User interface, 92... Storage unit, 93... Execution unit, 94... Acquisition unit, 95... Determination unit, 96... Change unit, 97... Synchronization unit, 100... Web conference tool, 900... Input unit, 901... Output unit, 902... Communication unit, 903... CPU, 904... Memory, 905... HDD, 906... Bus, 907... Storage medium

Claims (8)

A remote control system for controlling one or more devices installed in a verification environment by a control device,
A storage unit that stores a control pattern for controlling the device to perform a predetermined operation;
an execution unit that causes the control device to execute control of the device based on the control pattern stored in the storage unit;
an acquisition unit that acquires at least one of information in the verification environment and information regarding the device;
a change unit that changes the control pattern stored in the storage unit based on the information acquired by the acquisition unit. a determination unit that determines whether a predetermined event has occurred in the verification environment when the execution unit causes the control device to execute control of the device,
The change unit is
2. The remote control system according to claim 1, further comprising: a control pattern stored in the memory unit that is changed when the determination unit determines that a predetermined event has not occurred in the verification environment. A user interface for inputting and outputting at least one of an image, a voice, and a character,
The execution unit is
3. The remote control system according to claim 1, wherein the control device controls the device based on at least one of an image, a voice, and a character input via the user interface. The verification environment includes:
A plurality of the devices are provided, each of which includes the same device;
The execution unit is
The remote control system according to any one of claims 1 to 3, characterized in that the control device is caused to execute control so that the devices in each of the multiple verification environments are controlled based on the same control pattern or different control patterns. The acquisition unit is
acquiring a response time from when the execution unit starts to cause the control device to execute control of the device to when the device actually starts operating based on the control pattern stored in the storage unit;
The execution unit is
5. The remote control system according to claim 1, wherein the timing at which the device starts to operate is synchronized with the execution unit based on the response time acquired by the acquisition unit. A remote control method for controlling one or more devices installed in a verification environment by a control device, comprising:
The RPA device controls the control device based on a control pattern stored in a storage unit so as to cause the device to perform a predetermined operation;
Acquire at least one of information in the verification environment and information about the device;
A remote control method comprising: changing a control pattern stored in the storage unit based on the acquired information. In an RPA device that causes a control device to control one or more devices installed in a verification environment,
A storage unit that stores a control pattern for controlling the device to perform a predetermined operation;
an execution unit that causes the control device to execute control of the device based on the control pattern stored in the storage unit;
an acquisition unit that acquires at least one of information in the verification environment and information regarding the device;
and a change unit that changes the control pattern stored in the memory unit based on the information acquired by the acquisition unit. An RPA program for causing a computer to function as each part of the RPA device described in claim 7.

Images