JP2001315080A - Remote observation equipment - Google Patents

Abstract

(57) [Summary] [PROBLEMS] A robot arranged in a remote place works in cooperation with an operator. Specifically, a mobile camera device (robot) observes a target that is difficult to observe with a fixed-point camera device. At this time, an environment model in which the robot is arranged and an operation model of the robot are created by a simple procedure. SOLUTION: When a worker designates a desired position for observation, first, a fixed-point camera device as a fixed-point observation means attempts observation, and if this is difficult, the fixed-point observation means is automatically moved by the mobile observation means. Issue a command to a robot.
At this time, an environment model and a behavior model of the robot are created in consideration of the real environment with respect to the fixed point observation result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an operation of a worker in a situation where the worker needs to confirm the situation at a remote place or perform the work at his / her own hand while operating a device at a remote place. Related to assisting technology. The present invention relates to a technique for simplifying an operation procedure of robot control.

[0002]

2. Description of the Related Art The so-called robot teleoperation technology enables remote operation of a robot installed in a remote work environment. This system is an effective method when working in a hazardous environment in a remote place.However, since teleoperation originally assumes only work in a remote place, for example, the worker While doing
For tasks such as referring to the status of remote devices and performing operations,
It is difficult to apply as it is.

[0003]

(A) It is difficult to create an environment map. As a means for operating a remote device or monitoring the status, it is conceivable to use a robot using a teleoperation technology (for example, a horizontal alley). , "Theory of master-slave control", Vol. 11, No. 6, pp. 794-802, Journal of the Robotics Society of Japan, 1993). In order to make this robot work at a desired place, a master / slave is configured, sensor information provided to the master robot is presented to a slave worker by high-sense technology, and the presentation information is used as a basis. Work by operating the master side.

In this robot operation by the master / slave, it is necessary for the worker to send an operation command to the master robot one by one, and the situation in which the worker performs the task at hand while controlling the robot has been known from the beginning. Since it is not taken into account, the work performed by the worker is impeded. Therefore, a method has been proposed in which the master robot has autonomy and performs tasks (Sato, Matsui, Hirai, "Remote work robot system featuring cooperative work with humans", Journal of the Robotics Society of Japan). , Vol.19, No.5, pp.602-613,1
991).

In this method, it is necessary to describe a model such as the shape of the environment around the robot in an environment model in advance. Creating this environment model takes a lot of time,
It is difficult to easily introduce a teleoperation system.

[0006] If camera devices are distributed in an environment, and an operator directly teaches an environment model and operation to an image of each camera device, and a worker works while providing information to the robot from the distributed camera devices. This is effective because the robot can easily work with intuitive teaching. Various methods have already been proposed for controlling the robot from the viewpoint of the camera images dispersed in the environment. They are: 1) When describing the behavior of a robot in an image (for example, Okada, Ishiguro, Ishida, "Human Robot Cooperative Navigation Using Distributed Vision", Journal of the Robotics Society of Japan, Vol.16, No.7,
pp.985-992,1998). 2) The target value in the camera image may be described (for example, Haneda and Takase, "Study on Robot Navigation Using Information Environment Information Infrastructure",
17th Annual Conference of the Robotics Society of Japan, pp.619-620,199
8).

[0007] Regarding the former, it is difficult to describe the behavior in consideration of the real environment around the robot. For the latter, it is necessary to map from the camera coordinate system to the robot coordinate system. Therefore, it is effective if a motion model can be created in consideration of a real environment and a robot can work without performing coordinate conversion.

If the present invention is limited to a remote situation monitor, it is not necessary to use a robot, but can be realized by fixing a plurality of camera devices in the environment and monitoring the remote situation. However, they are premised on operating on a desktop, and are not suitable for performing a hands-free operation while monitoring a remote situation. It would be convenient if these images could be referenced hands-free.

[0009] Another problem here is that since the installed camera device is fixed, not all images desired by the operator can be obtained depending on the degree of freedom of the PTZ of the camera device, the resolution of the camera device, and the like. There is no point. Therefore, using the teleoperation technology, instead of the fixed camera device, the slave robot equipped with the camera device is used as a function interpolation of the fixed camera device when monitoring by the fixed camera device cannot be performed. Is effective.

(B) A system model for coordinating work between a remote robot and a worker has not been studied From the viewpoint of teleoperation, it has not been studied that a remote robot works and a worker does work. . The use of a wearable personal computer can be expected to enable a worker to perform cooperative work with a remote robot while working at hand, but this has not been sufficiently studied.

As described above, according to the conventional technique, the operator cannot perform the work in various places in a hands-free manner while the remote robot operates the remote device and monitors the device.

SUMMARY OF THE INVENTION The present invention has been made in view of the above background, and an object of the present invention is to provide a remote observation apparatus in which a robot arranged in a remote place can work in cooperation with an operator. And SUMMARY OF THE INVENTION It is an object of the present invention to provide a remote location measuring device capable of creating an environment model in which a robot is arranged and an operation model of the robot by a simple procedure.

[0013]

The present invention displays fixed-point observation information obtained from a plurality of camera devices arranged in an observation target area and moving observation information obtained from a camera device provided in a moving robot. And MMI (Man Machine Int.) That controls these camera devices and robots.
erface).

According to the present invention, an environment model and an operation model of a robot are created in consideration of a real environment with respect to a fixed point observation result, and a robot can be controlled without mapping from a sensor coordinate system to a robot coordinate system. Is characterized by being able to easily monitor remote conditions and operate equipment while working hands-free in various places.

That is, the present invention relates to a remote observation apparatus, which is characterized in that the present invention is characterized in that a plurality of fixed point observing means installed in advance in an observation target area, a moving means and a mobile observing means provided with the observing means. Remote control means comprising: means for receiving observation information of the fixed point observation means and the mobile observation means; and means for transmitting control information of the fixed point observation means and the mobile observation means, wherein the control information Includes observation position designation information in the observation target area, the fixed point observation means transmits, as the observation information, an observation result of the observation position designated by the observation position designation information, Means for issuing an action command for observing the observation position to the mobile observation means when observation is difficult, the mobile observation means comprising:
There is a means for moving to the observation position in accordance with the action command.

As described above, according to the present invention, when the operator designates a position desired to be observed, first, the fixed point observation means attempts observation, and if it is difficult, the fixed point observation means automatically moves and observes. Issue an action command to the means. This allows
The command by the worker is kept to a necessary minimum, the factor that hinders the performance of the work performed by the worker is reduced, and the mobile observation means arranged at a remote place works in cooperation with the worker. it can.

The fixed point observing means is means for detecting the presence or absence of the mobile observing means in its own observing area, and according to the detection result, when the mobile observing means is present in its own observing area, its own installation position information Means for transmitting, the mobile observation means, the means for receiving the installation position information, the current position of its own according to the received installation position information and map information of the observation target area recorded in advance Recognition means.

With this, when a new observation target area is set, a fixed point observation means is arbitrarily set in the observation target area and its map information is recorded in the mobile observation means. By receiving the installation position information transmitted from the fixed point observation means, the user can recognize his / her current position. Therefore, it is possible to easily perform the pre-setting for causing the mobile observation means to recognize the current position within a short time.

The means for moving to the observation position includes: means for searching for the order of the fixed point observation means passing from the current position to the observation position in accordance with the action command with reference to the map information; Means for transmitting an identification code of the fixed point observation means that passes next,
It is preferable that means for transmitting path information including the fixed point observation means corresponding to the identification code is provided, and that the means for moving to the observation position include means for moving to the observation position in accordance with the path information.

According to this, since the mobile observation means only needs to sequentially instruct the mobile observation means of the path information of a short section in which the mobile observation means moves from one fixed-point observation means to the next fixed-point observation means, it is necessary to provide complicated and long path information. It is not necessary to create and teach. For example, if the fixed-point observation means is a camera device, the worker observes the moving observation means projected by a certain camera device and looks at the image while creating an action model using the shooting range of the camera device as an environment model. When the movement observation means arrives within the photographing range of the next camera device, the operator also teaches the behavior model using the photographing range of this camera device as an environmental model while observing the image as the route information. It is possible to do. Alternatively, such route information may be created in advance and automatically transmitted when the identification code transmitted by the mobile observation means is received.

It is preferable that the route information is created based on a wall or a passage actually existing in the observation target area. Alternatively, the route information may be created virtually based on a wall provided in the observation target area.

Thus, it is not necessary to create route information involving complicated coordinate transformation and mapping, and the route information can be easily created. For example, the worker can use the moving observation means projected on the camera device. The route information can be created in a very short time while viewing the image.

Further, the mobile observation means may include means for remotely controlling equipment in the observation target area. Thereby, the worker can not only monitor the situation in the observation target area, but also remotely control the equipment in the observation target area. The means for remote control may include, for example, a so-called robot arm.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The configuration of a remote observation apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is an overall configuration diagram of a remote observation apparatus according to an embodiment of the present invention.

The present invention relates to a remote observation device, which is characterized by camera devices 10-1 and 10 including a plurality of CCD cameras installed in advance in an observation target area.
Distributed camera unit 10 provided with 0-2 and mobile robot 11
A mobile robot unit 13 having a camera device 12 and
An MMI unit 15 including a wearable personal computer 14 for receiving observation information of the distributed camera unit 10 and the mobile robot unit 13 and transmitting control information to them;
The control information includes observation position designation information in the observation target area, and the distributed camera unit 10 transmits the observation result of the observation position designated by the observation position designation information as the observation information. However, when it is difficult to observe the observation position, the mobile robot unit 13 issues an action command to observe the observation position.
Moves to the observation position in accordance with this action command.

Each camera device 10-1 of the distributed camera unit 10
And 10-2 detect the presence or absence of the mobile robot unit 13 within its own photographing range, and when the mobile robot unit 13 is within its own photographing range, determine its own installation position information according to the detection result.・ Computer 1
6 via the wireless LAN and the mobile robot unit 1
3 receives the installation location information and recognizes its own current location according to the received installation location information and map information of the observation target area set in advance.

The camera devices 10-1 and 10-2
As a method for detecting the presence or absence of the mobile robot unit 13,
The image pattern of the mobile robot unit 13 is stored in the image processing personal computer 16 in advance,
This can be realized by detecting this image pattern from the images captured by the camera devices 10-1 and 10-2.

The mobile robot unit 13 searches for the order of the camera device 10-1 or 10-2 passing from the current position to the observation position in accordance with the action command with reference to the map information. The identification code of the next passing camera device 10-1 or 10-2 is transmitted, and the distributed camera unit 10 receives the identification code, and the mobile robot unit 13 causes the mobile robot unit 13 to correspond to the identification code of the camera device 10-1 or 10-2. The mobile robot unit 13 transmits route information necessary to reach -2, and moves to the observation position according to the route information. In the embodiment of the present invention,
The transmission of the route information is performed by the camera device 10 corresponding to the identification code.
-1 or 10-2 through the wireless LAN via the image processing personal computer 16.

The route information is created based on walls or passages actually existing in the observation target area. Alternatively, it is created based on a wall virtually provided in the observation target area.

Further, the mobile robot unit 13 may be provided with a device such as a so-called robot arm to remotely control devices in the observation target area.

Hereinafter, the remote observation apparatus according to the embodiment of the present invention will be described in more detail. An embodiment will be described for a task in which a worker checks the environmental status of a remote place while performing a task at hand. FIG. 1 shows an embodiment according to the present invention. In FIG. 1, camera devices 10-1 and 10-2 distributed in the observation target area, their personal computers 16 for image processing, camera device 12, and ultrasonic sensor 1 are shown.
The mobile robot 11 includes a mobile robot 11 and a wearable personal computer 14.

The mobile robot 11 according to the embodiment of the present invention has two wheels, each of which has an independent motor as an actuator and a pulse encoder for detecting a rotational speed. Each motor of the mobile robot 11 has a servo loop of the motor rotation speed which is used as feedback of a value from the pulse encoder. Also, TCP / IP with other devices via wireless LAN
Ultrasonic sensor 1 for measuring communication function and surrounding environment of robot
7 is provided.

The wearable personal computer 14 and the image processing personal computer 16 are connected to a wireless LAN.
Has a TCP / IP communication function with other devices.

Next, the operation of the embodiment of the present invention will be described. FIG. 2 shows an example of an individual information processing flow of the MMI unit 15, the distributed camera unit 10, and the mobile robot unit 13.
Hereinafter, individual processing flows will be described based on FIGS. 1 and 2.

1) MMI section Step 1) An operator selects a desired image from the distributed camera section 10, specifies an area in the camera image, describes an environment model, specifies an action model for the environment model,
It is transmitted to each of the camera devices 10-1 and 10-2 via TCP / IP communication (FIG. 2). 3 to 6 show examples of an environment model and an action model taught for each camera image.

(Example of Environment Model and Action Model for Camera ID = i) _Ai represents a movable area in a camera image and an action there. P _i represents a position vector of a polygon vertex, () represents a combination of polygon vertices constituting a wall, FC represents an action along a path moving along the center of an actual path, and FW represents an action along a path. Means moving along a real wall.

C: The i-th row and the j-th column are matrices expressing the connection between the areas defined as 1 if the area Ai and the area Aj are connected, and 0 if not. gate _ij : A region overlapping with another camera device (camera ID = j) is represented by two polygonal vertices.

An example in which the camera device is arranged at the position as shown in FIG. 3 will be described. FIG. 4 is an example of teaching for camera ID = 1,

[0039]

(Equation 1) Becomes FIG. 5 is an example of teaching for camera ID = 2,

[0040]

(Equation 2) Becomes FIG. 6 is an example of teaching for camera ID = 3,

[0041]

(Equation 3) Becomes

Step 2) An image of the camera device (ID = t) that the operator wants to refer to is displayed, the target point is pointed on the camera image, and the target point position vector is set to the ID.
= T to the camera device via TCP / IP communication.

Step 3) When the distributed camera unit 10 or the mobile robot unit 13 has transmitted the target image, the target image is displayed on the display (or in FIG. 2).

2) Camera device with camera ID = t Step 1) The behavior model and the environment model transmitted from the MMI unit 15 are stored (FIG. 2).

Step 2) The target point position vector transmitted from the MMI unit 15 is received, and if the task determined by this camera device can be executed, an image is transmitted to the MMI unit 15 after the task is completed. , Mobile robot part 1
3, the camera ID = t and the target value vector are transmitted.

As an example of the task, for example, when the target position vector is to be captured at the center of the image, it is determined whether the image can be positioned at the center of the image by moving the pan / tilt of the camera device. If so, pan and tilt and transmit the image to the MMI unit 15.
If it is out of the movable range, it is transmitted to the mobile robot unit 13 (FIG. 2).

Step 3) The mobile robot unit 13 is always searched for from the camera image based on the image template of the mobile robot unit 13 given in advance (FIG. 2).
If found, the coordinate value and the camera ID = t are transmitted to the mobile robot unit 13 via TCP / IP communication (FIG. 2).

Step 4) A subtask is received from the mobile robot unit 13 by TCP / IP communication. Each A _i for realizing a subtask (moving to the target value) from the subcall of the mobile robot unit 13 (moving to the target value within the camera field of view) obtained from this command and the current value of the mobile robot unit 13. Calculate the action at.

FIG. 7 shows an example of a movable area in a camera image of a camera device with a camera ID = 1 and an action there.
A ₁ represents the movable area in the camera image and the action there,

[0050]

(Equation 4) Becomes

Here, FC means action along the center of the passage. () Indicates a virtual wall, and Null indicates that there is no virtual wall. This virtual wall information is obtained by calculating a wall virtually in the camera field of view in order to realize a target subtask when there is no wall in the real environment around the mobile robot unit 13, and transmitting the wall to the mobile robot unit 13. There is a purpose to use as information.

Next, the action of the mobile robot unit 13 in the area where the mobile robot unit 13 is currently located and the position vector of the virtual wall with respect to the mobile robot unit 13 are calculated and transmitted to the mobile robot unit 13. When the mobile robot unit 13 arrives at the destination, the following action is transmitted to the mobile robot unit 13.

PutImage: action to transmit an image 3) Camera apparatus with camera ID @ t Step 1) The action model and environment model transmitted from MMI unit 15 are stored (FIG. 2).

Step 2) Based on the image template of the mobile robot unit 13, the mobile robot unit 13 is always searched from the camera image.
The umbrella coordinate value and its own camera ID are transmitted via TCP / IP communication (FIG. 2).

Step 3) A subtask is received from the mobile robot unit 13 by TCP / IP communication. The subcall of the mobile robot unit 13 obtained from this command (camera ID =
sg) and each A for realizing a subtask (moving to camera ID = sg) from the current position of the mobile robot unit 13
The action at _j is calculated.

FIG. 8 shows an example of a movable area in a camera image of a camera device having a camera ID = 3 and an action there.
Camera device with sub-goal (auxiliary destination in the process of heading for the original destination) ID = 2 of the robot (sg = 2)
Are examples of actions and virtual walls at each A _i in the case of A ₄ , A _4
2 , A ₁ respectively represent the movable area in the camera image and the action there,

[0057]

(Equation 5) Becomes

Here, FC means action along the wall, and FW
Means action along the center of the call. () Indicates a virtual wall, and Null indicates that there is no virtual wall. This virtual wall information is obtained by calculating a wall virtually in the camera field of view in order to realize a target sub-task when there is no wall in the real environment around the mobile robot unit 13.
The purpose is to use it as wall information.

Next, the action of the mobile robot unit 13 in the area where the mobile robot unit 13 is currently located and the position vector of the virtual wall with respect to the mobile robot unit 13 are calculated and transmitted to the mobile robot unit 13.

4) Mobile Robot Unit Step 1) The camera ID = t and the target value vector are received from the camera ID = t via TCP / IP communication, and the camera ID and the position vector transmitted from each camera device are received. , And store the current location (of FIG. 2).

Step 2) The order of the camera device to reach the camera with ID = t is searched based on the predefined arrangement matrix of the camera device (FIG. 2).
Regarding the search, in the present embodiment, it is possible to use another search algorithm such as the A ^* algorithm which calculates the route order of the camera device by a general horizontal search.

Step 3) A subtask (camera device ID to be reached next) is transmitted to the distributed camera unit 10, an action model and a position vector with respect to the virtual wall are received (FIG. 2), and the received action is represented by an ultrasonic wave. While the environment is being measured by the sensor 17, the wheel is driven and executed (FIG. 2). The position vector of the virtual wall obtained from the camera device is integrated with the wall information calculated by the ultrasonic sensor 17 to determine the wheel control amount.

Step 4) Step 2 to Step 3 are repeated to move to the target value of the distributed camera with camera ID = t.

Step 5) Put from camera with ID = t
When the image is received, a camera image provided in the MMI unit 15 is transmitted (FIG. 2).

In the embodiment of the present invention, an example in which two camera devices 10-1 and 10-2 are installed has been described for the sake of simplicity, but a plurality of camera devices may be further provided.

[0066]

As described above, according to the present invention,
A robot located at a remote place can work in cooperation with a worker. Further, an environment model in which the robot is arranged and an operation model of the robot can be created by a simple procedure.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a remote observation apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing an individual information processing flow of an MMI unit, a distributed camera unit, and a mobile robot unit.

FIG. 3 is a diagram showing an example of the arrangement of camera devices in an example of teaching an environment model and a behavior model.

FIG. 4 is a diagram showing an example of teaching for camera ID = 1 in an example of teaching of an environment model and an action model.

FIG. 5 is a diagram showing an example of teaching for camera ID = 2 in an example of teaching of an environment model and an action model.

FIG. 6 is a diagram showing an example of teaching for camera ID = 3 in an example of teaching of an environment model and an action model.

FIG. 7 is a diagram showing an example of a movable area in a camera image and actions in the movable area.

FIG. 8 is a diagram showing an example of a movable area in a camera image and an action there.

[Explanation of symbols]

 Reference Signs List 10 Distributed camera unit 10-1, 10-2, 12 Camera device 11 Mobile robot 13 Mobile robot unit 14 Wearable personal computer 15 MMI unit 16 Personal computer for image processing 17 Ultrasonic sensor

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (reference) // G05D 1/00 G05D 1/00 B 1/02 1/02 J (72) Inventor Yukihisa Katayama Tokyo 2-3-1, Otemachi, Chiyoda-ku Nippon Telegraph and Telephone Corporation (72) Inventor Manabu Mogi 2-3-1, Otemachi, Chiyoda-ku, Tokyo F-Term (in reference) 3F059 AA11 AA17 BB07 BC07 BC09 CA05 CA06 DA02 DA05 DA08 DB04 DB06 DB09 DC08 DD01 DD08 DD18 EA01 FA03 FA05 FA10 FB01 FB05 FB12 FB15 FC02 FC07 FC13 FC14 3F060 AA01 AA09 CA12 GA05 GA13 GD14 HA02 HA32 HA35 5H301 AA02 BB10 DD03 GG09 GG10 GG12 KK02 KK05 KK10 KK19 9A001 HH34 JJ71

Claims (6)

[Claims] A plurality of fixed-point observing means installed in advance in an observation target area; a moving observing means having moving means and observing means; a means for receiving observation information of the fixed-point observing means and the moving observing means; Remote control means comprising a fixed point observation means and a means for transmitting control information of the mobile observation means, wherein the control information includes observation position designation information in the observation target area, and the fixed point observation means Means for transmitting an observation result of the observation position designated by the observation position designation information as the observation information, and an action for observing the observation position to the mobile observation means when it is difficult to observe the observation position. Means for issuing a command, and the movement observation means includes means for moving to the observation position in accordance with the action command. apparatus. 2. The fixed point observing means includes means for detecting the presence or absence of the moving observing means in its own observing area, and installing the self observing means when the moving observing means is present in its own observing area according to the detection result. Means for transmitting position information, wherein the mobile observation means includes a means for receiving the installation position information, and the current position of the mobile observation unit according to the received installation position information and the pre-recorded map information of the observation target area. 2. The remote observation device according to claim 1, further comprising means for recognizing a position. 3. The means for moving to the observation position, the means for searching for the order of the fixed-point observation means passing from the current position to the observation position in accordance with the action command with reference to the map information; Means for transmitting an identification code of the fixed point observing means which passes next according to the following, wherein means for transmitting path information including the fixed point observing means corresponding to the identification code is provided, and means for moving to the observation position The remote observation apparatus according to claim 1 or 2, further comprising means for moving to an observation position according to the route information. 4. The remote observation device according to claim 3, wherein the route information is created based on a wall or a passage actually existing in the observation target area. 5. The remote observation apparatus according to claim 3, wherein the route information is created based on a wall virtually provided in the observation target area. 6. The remote observation apparatus according to claim 1, wherein said mobile observation means includes means for remotely controlling a device in said observation target area.