JP2022019163A - Information processing equipment, information processing methods, and computer programs - Google Patents

Abstract

[Problem] To reduce the burden on a single monitor when remotely monitoring multiple autonomously traveling mobile bodies. [Solution] An information processing device for remotely monitoring multiple autonomously traveling mobile bodies includes a route information acquisition unit, a surrounding object information acquisition unit, a conversion unit, an image generation unit, and a display control unit. The route information acquisition unit acquires route information indicating a planned travel route of each autonomously traveling mobile body. The surrounding object information acquisition unit acquires surrounding object information indicating a state including at least the position of an object surrounding each autonomously traveling mobile body. The conversion unit converts the planned travel route into a predetermined reference shape and converts the state of the surrounding objects for each autonomously traveling mobile body so that a relative relationship between the planned travel route and the state of the surrounding objects is maintained. The image generation unit generates a monitoring image indicating the planned travel route and the state of the surrounding objects after conversion for each autonomously traveling mobile body. The display control unit causes the monitoring image to be displayed on a display device. [Selected Figure] FIG. 6

Description

The techniques disclosed herein relate to information processing devices, information processing methods, and computer programs for remote monitoring of a plurality of autonomous traveling vehicles.

In recent years, autonomous traveling vehicles (also referred to as "autonomous driving vehicles") that autonomously travel without the need for human driving operations have been developed. The autonomous vehicle sets, for example, a planned travel route based on a command issued by the administrator, and is based on signals output from various sensors provided in the autonomous vehicle, information acquired by communication with the outside, and the like. Detects and recognizes the state (position, direction, speed, shape, color, etc.) of the autonomous vehicle itself and surrounding objects (other vehicles, pedestrians, traffic lights, street trees, lanes, etc.) and avoids collisions with surrounding objects. While doing so, it travels autonomously along the set planned travel route.

In addition, in order to realize smoother and safer driving of the autonomous vehicle, the remote monitoring center remotely monitors the autonomous vehicle and, if necessary, intervenes in the control of the autonomous vehicle (assists the control of the autonomous vehicle). ) Technology is known (see, for example, Patent Document 1). Examples of the mode of intervention in the control of the autonomous vehicle include assistance in detecting and recognizing the state of the autonomous vehicle itself and surrounding objects of the autonomous vehicle, and remote control of the autonomous vehicle.

Japanese Unexamined Patent Publication No. 2018-140755

When remotely monitoring an autonomous vehicle, it is desirable that one observer remotely monitor a plurality of autonomous vehicles from the viewpoint of efficiency. However, since the planned travel route and the state of surrounding objects vary from one autonomous vehicle to another, if one observer performs remote monitoring of a plurality of autonomous vehicles, the burden on the observer will be increased. There is a problem that it is big.

It should be noted that such a problem is not limited to remote monitoring of autonomous traveling vehicles, but is also a common problem for remote monitoring of moving objects other than autonomously traveling vehicles, such as autonomous traveling robots.

This specification discloses a technique capable of solving the above-mentioned problems.

The techniques disclosed herein can be realized, for example, in the following forms.

(1) The information processing device disclosed in the present specification is an information processing device for remote monitoring of a plurality of autonomous traveling mobile bodies, and includes a route information acquisition unit, a peripheral object information acquisition unit, and a conversion unit. , An image generation unit and a display control unit are provided. The route information acquisition unit acquires route information indicating the planned travel route of each autonomous traveling vehicle. The peripheral object information acquisition unit acquires peripheral object information indicating a state including at least the position of the peripheral object of each of the autonomous traveling moving bodies. The conversion unit converts the planned travel route into a predetermined reference shape and the periphery so that the relative relationship between the planned travel route and the state of the peripheral object is maintained for each autonomous traveling vehicle. Performs transformation of the state of the object. The image generation unit generates a monitoring image showing the state of the planned travel path and the peripheral objects after the conversion for each autonomous traveling vehicle. The display control unit causes the display device to display the monitoring image.

As described above, according to the present information processing apparatus, for each autonomous traveling moving body, the traveling route to the predetermined reference shape is maintained so that the relative relationship between the traveling route and the state of the surrounding object is maintained. The conversion and the conversion of the state of the peripheral object are executed, and for each autonomous traveling moving body, a monitoring image showing the planned traveling route after conversion and the state of the peripheral object is generated, and the monitoring image is displayed on the display device. Therefore, in the monitoring image, the planned travel route for each autonomous traveling vehicle is shown in a state of being aligned with a predetermined reference shape (that is, in the same shape as each other). Therefore, the observer determines the relationship between the planned travel path and the surrounding objects, as compared with the configuration in which the planned travel route for each autonomous traveling vehicle is shown as it is (that is, in a disjointed shape for each autonomous traveling vehicle). It can be easily grasped. From the above, according to this information processing apparatus, it is possible to reduce the burden on the observer when one observer performs remote monitoring of a plurality of autonomous traveling vehicles.

(2) In the information processing apparatus, the monitoring image is shown so that the converted travel schedule of each autonomous traveling vehicle overlaps with each other with the current position of each autonomous traveling vehicle as a reference point. It may be configured as an image. According to this information processing device, even when one observer performs remote monitoring of a plurality of autonomous traveling objects, the observer has a relationship between a single planned travel route and each peripheral object. You only have to monitor. Therefore, according to this information processing apparatus, it is possible to effectively reduce the burden on the observer when one observer performs remote monitoring of a plurality of autonomous traveling vehicles.

(3) The information processing apparatus further includes a vehicle information acquisition unit that acquires vehicle information indicating a state including at least the speed of each autonomous traveling vehicle, and the monitoring image is based on the speed of the autonomous traveling vehicle. The scale of the time distance derived based on the basis may be an image represented so as to be aligned for each of the autonomous traveling vehicles. According to this information processing device, even when one observer performs remote monitoring of a plurality of autonomous traveling vehicles, the observer monitors without being aware of the speed difference of each autonomous traveling vehicle. Based on the distance between the planned travel path of the autonomous traveling vehicle and each peripheral object in the image, it is possible to determine whether or not intervention in the control of the autonomous traveling vehicle is necessary. Therefore, according to this information processing apparatus, it is possible to extremely effectively reduce the burden on the observer when one observer performs remote monitoring of a plurality of autonomous traveling vehicles.

(4) In the information processing apparatus, the monitoring image may be configured as an image showing a warning area where there is a possibility of collision with the autonomous traveling moving body traveling on the planned traveling route. According to this information processing device, even when one observer remotely monitors a plurality of autonomous traveling objects, the observer is based on the positional relationship between the warning area and each peripheral object in the surveillance image. , It is possible to easily determine the necessity of intervention in the control of the autonomous traveling vehicle. Therefore, according to this information processing apparatus, it is possible to extremely effectively reduce the burden on the observer when one observer performs remote monitoring of a plurality of autonomous traveling vehicles.

(5) In the information processing apparatus, the display control unit causes the display device to display a detailed image of the selected peripheral object in response to the selection of the peripheral object included in the monitoring image. It may be configured. According to the information processing apparatus, the observer can refer to the detailed image of the peripheral object displayed on the display device to determine the necessity of intervention for the control of the autonomous traveling vehicle with respect to the peripheral object. You can perform actual interventions. Therefore, according to this information processing apparatus, appropriate remote monitoring is performed for each autonomous traveling vehicle while reducing the burden on the observer when one observer performs remote monitoring of a plurality of autonomous traveling vehicles. be able to.

(6) In the information processing apparatus, the reference shape may be a linear shape. According to this information processing device, the observer only needs to monitor the relationship between the planned traveling route having a simple shape and each peripheral object, and easily determines whether or not intervention in the control of the autonomous traveling moving body is necessary. can do. Therefore, according to this information processing apparatus, it is possible to effectively reduce the burden on the observer when one observer performs remote monitoring of a plurality of autonomous traveling vehicles.

The techniques disclosed in the present specification can be realized in various forms, for example, an information processing apparatus, an information processing method, a computer program that realizes those methods, and a temporary recording of the computer program. It can be realized in the form of an untargeted recording medium or the like.

An explanatory diagram schematically showing the configuration of the remote monitoring system 10 in the present embodiment. Explanatory diagram schematically showing the configuration of the remote monitoring management server 20 A flowchart showing a remote monitoring management process executed by the remote monitoring management server 20 of the present embodiment. Explanatory diagram which shows the outline of the remote monitoring management processing executed by the remote monitoring management server 20 of this embodiment. Explanatory diagram which shows the outline of the remote monitoring management processing executed by the remote monitoring management server 20 of this embodiment. Explanatory diagram which shows the outline of the remote monitoring management processing executed by the remote monitoring management server 20 of this embodiment.

A. Embodiment:
A-1. Configuration of remote monitoring system 10:
FIG. 1 is an explanatory diagram schematically showing the configuration of the remote monitoring system 10 in the present embodiment. The remote monitoring system 10 is a system for remotely monitoring a plurality of autonomous traveling vehicles 60 at, for example, a remote monitoring center and intervening in the control of the autonomous traveling vehicle 60 (assisting the control of the autonomous traveling vehicle 60) as necessary. ..

The autonomous traveling vehicle 60, which is the target of remote monitoring by the remote monitoring system 10, is a vehicle that autonomously travels without requiring a driving operation by a person. The autonomous traveling vehicle 60 may be a private vehicle or a commercial vehicle such as a taxi, a bus, or a truck. The autonomous traveling vehicle 60 is an example of an autonomous traveling vehicle within the scope of claims.

The autonomous traveling vehicle 60 includes a sensor 66, a communication unit 64, a driving unit 68, and an autonomous traveling control unit 62. The sensor 66 includes, for example, a camera, a millimeter-wave radar, LIDAR, an ultrasonic sensor, a GPS, a vehicle speed sensor, an acceleration sensor, and the like, and includes an autonomous traveling vehicle 60 itself and surrounding objects (other vehicles, pedestrians, traffic lights, street trees, etc.). Detects the state (position, orientation, speed, shape, color, etc.) of the lane (lane, etc.). The communication unit 64 is a communication interface that communicates with another device (for example, a remote monitoring management server 20 or a remote monitoring device 30 described later constituting the remote monitoring system 10) by a predetermined communication method. The drive unit 68 includes, for example, an engine, a motor, steering, a brake, and the like, and performs a traveling operation (acceleration, steering, braking, etc.) of the autonomous traveling vehicle 60. The autonomous travel control unit 62 is configured by, for example, a computer including a CPU and a memory, and controls autonomous travel of the autonomous travel vehicle 60. For example, the autonomous travel control unit 62 sets a planned travel route based on a command issued by an administrator (for example, the administrator of the remote monitoring system 10), and sets various signals output from the sensor 66 and the communication unit 64. The state of the autonomous traveling vehicle 60 itself and surrounding objects based on information acquired by communication with the outside via communication (for example, a high-precision three-dimensional map and information on peripheral objects detected by another autonomous traveling vehicle 60). By detecting and recognizing the above and controlling the traveling operation by the driving unit 68, the autonomous traveling vehicle 60 is autonomously driven along the set planned traveling route while avoiding a collision with a peripheral object.

However, during autonomous traveling of the autonomous traveling vehicle 60, the autonomous traveling vehicle 60 reliably detects the state of the autonomous traveling vehicle 60 itself and surrounding objects due to problems such as the detection accuracy of the sensor 66 and the traveling environment. It may not be recognized. For example, when a pedestrian is detected as a peripheral object but it cannot be determined whether or not the pedestrian crosses the road, or when a peripheral object is detected, the peripheral object interferes with the running of the autonomous traveling vehicle 60. There are no objects (for example, obviously light flying objects), there are a large number of pedestrians in the vicinity, traveling in a tunnel, traveling in heavy rain or heavy fog, and so on. In such a case, smooth autonomous traveling by the autonomous traveling vehicle 60 may be difficult, or the possibility of collision with a peripheral object may increase. In preparation for such a case, the remote monitoring system 10 remotely monitors the autonomous traveling vehicle 60, and intervenes in the control of the autonomous traveling vehicle 60 voluntarily or in response to a request from the autonomous traveling vehicle 60. (Assists the control of the autonomous traveling vehicle 60). Examples of the mode of intervention in the control of the autonomous traveling vehicle 60 by the remote monitoring system 10 include assisting in detecting and recognizing the state of the autonomous traveling vehicle 60 itself and surrounding objects of the autonomous traveling vehicle 60, and remote control of the autonomous traveling vehicle 60. Maneuvering is mentioned. As a result, smooth, comfortable, and safer running of the autonomous traveling vehicle 60 is realized.

As shown in FIG. 1, the remote monitoring system 10 includes a remote monitoring management server 20, a plurality of remote monitoring devices 30, and a network communication device 40 such as a router. Although three remote monitoring devices 30 are shown in FIG. 1, the number of remote monitoring devices 30 included in the remote monitoring system 10 may be two or less, or four or more. good. The devices constituting the remote monitoring system 10 are connected to each other so as to be able to communicate with each other via an internal network 82 such as a LAN. Further, the remote monitoring management server 20 and the plurality of remote monitoring devices 30 are communicably connected to each autonomous traveling vehicle 60 via a network communication device 40 and an external network 84 such as the Internet.

The remote monitoring device 30 is a terminal device used for remote monitoring of the autonomous traveling vehicle 60 by the observer MO, and is composed of, for example, a PC. The remote monitoring device 30 includes a display device 32 such as a liquid crystal display. When the observer MO determines the necessity of intervention in the control of the autonomous traveling vehicle 60 by referring to the monitoring image MI (described later) displayed on the display device 32 of the remote monitoring device 30, and determines that intervention is necessary. In addition, the remote monitoring device 30 is operated to intervene in the control of the autonomous traveling vehicle 60. The remote monitoring device 30 includes a user interface (for example, a keyboard, a mouse, a touch panel, a control stick, etc.) for performing remote monitoring of the autonomous traveling vehicle 60 including intervention in the control of the autonomous traveling vehicle 60. In the present embodiment, each observer MO (each remote monitoring device 30) simultaneously performs remote monitoring of a plurality of (for example, 10) autonomous traveling vehicles 60.

The remote monitoring management server 20 is a server device for managing remote monitoring using the remote monitoring device 30 by each observer MO. FIG. 2 is an explanatory diagram schematically showing the configuration of the remote monitoring management server 20. The remote monitoring management server 20 includes a control unit 210, a storage unit 220, a display unit 230, an operation input unit 240, and an interface unit 250. Each of these parts is communicably connected to each other via bus 290. The remote monitoring management server 20 is an example of an information processing device within the scope of claims.

The display unit 230 of the remote monitoring management server 20 is composed of, for example, a liquid crystal display or the like, and displays various images and information. Further, the operation input unit 240 is composed of, for example, a keyboard, a mouse, a button, a microphone, and the like, and receives operations and instructions of the administrator. The display unit 230 may be provided with a touch panel to function as an operation input unit 240. Further, the interface unit 250 is configured by, for example, a LAN interface, a USB interface, or the like, and communicates with other devices by wire or wirelessly.

The storage unit 220 of the remote monitoring management server 20 is composed of, for example, a ROM, a RAM, a hard disk drive (HDD), or the like, and stores various programs and data, and temporarily executes a work area and data when executing various programs. It is used as a typical storage area. For example, the storage unit 220 stores a remote monitoring management program CP, which is a computer program for executing a remote monitoring management process described later. The remote monitoring management program CP is provided in a state of being stored in a computer-readable recording medium (not shown) such as a CD-ROM, a DVD-ROM, or a USB memory, and is installed in the remote monitoring management server 20. It is stored in the storage unit 220.

Further, the map information 221 is stored in the storage unit 220 of the remote monitoring management server 20. The map information 221 is, for example, high-precision three-dimensional map information, and includes information on the position and shape of traffic lights, road signs, roadside trees, guardrails, buildings, and the like, in addition to information on the position and shape of roads.

The control unit 210 of the remote monitoring management server 20 is configured by, for example, a CPU or the like, and controls the operation of the remote monitoring management server 20 by executing a computer program read from the storage unit 220. For example, the control unit 210 functions as the remote monitoring management unit 211 that executes the remote monitoring management process described later by reading the remote monitoring management program CP from the storage unit 220 and executing the remote monitoring management program CP. The remote monitoring management unit 211 includes a vehicle information acquisition unit 212, a route information acquisition unit 213, a peripheral object information acquisition unit 214, a conversion unit 215, an image generation unit 216, and a display control unit 217 as functional modules. include. The functions of each of these parts will be described in accordance with the description of the remote monitoring management process described later.

A-2. Remote monitoring management process:
FIG. 3 is a flowchart showing a remote monitoring management process executed by the remote monitoring management server 20 of the present embodiment. 4 to 6 are explanatory views showing an outline of the remote monitoring management process executed by the remote monitoring management server 20 of the present embodiment.

The remote monitoring management process of the present embodiment is a process of managing the remote monitoring of the autonomous traveling vehicle 60 using the remote monitoring device 30 by each observer MO, for example, monitoring for monitoring a plurality of autonomous traveling vehicles 60. It includes a process of generating an image MI and displaying it on the display device 32 of each remote monitoring device 30. The remote monitoring management process is started in response to the administrator operating the operation input unit 240 of the remote monitoring management server 20 and inputting a start instruction. In this embodiment, the remote monitoring management process is executed for each remote monitoring device 30 (monitorer MO). Hereinafter, the remote monitoring management process executed for one remote monitoring device 30 will be described.

As described above, each autonomous vehicle 60 sets a planned travel route based on a command issued by the administrator, and detects and recognizes the state of the autonomous vehicle 60 itself and surrounding objects by using a sensor 66 or the like. It travels autonomously along the set planned travel route while avoiding collisions with surrounding objects. As an example, FIG. 4 shows a planned travel route PA and a peripheral object OB for two autonomous traveling vehicles 60 that are the targets of remote monitoring by one remote monitoring device 30. More specifically, in column A of FIG. 4, the planned travel route PA and peripheral objects OB (peripheral objects OB1 and OB2) for the autonomous traveling vehicle 60a traveling on a road having a gentle curve at a relatively high speed are shown. It is shown, and in column B of FIG. 4, the planned travel route PA and peripheral objects OB (peripheral objects OB3, OB4) for the autonomous traveling vehicle 60b trying to turn right at the intersection at a relatively low speed are shown. The arrow attached to the peripheral object OB indicates the direction (expected traveling direction) of the peripheral object OB.

First, the vehicle information acquisition unit 212 (FIG. 2) of the remote monitoring management server 20 indicates vehicle information indicating the state of each of a plurality of (for example, 10) autonomous traveling vehicles 60 to be monitored by the remote monitoring device 30. Acquire Iv (S100). In the present embodiment, the state of the autonomous traveling vehicle 60 indicated by the vehicle information Iv includes the position, direction, and speed of the autonomous traveling vehicle 60. The vehicle information acquisition unit 212 may directly acquire the vehicle information Iv indicating the state of the autonomous traveling vehicle 60 detected by the sensor 66 or the like of each autonomous traveling vehicle 60 from each autonomous traveling vehicle 60. Further, when the vehicle information Iv is stored in the storage unit of a device other than the autonomous traveling vehicle 60 (for example, the remote monitoring device 30), the vehicle information acquisition unit 212 stores the stored vehicle information Iv. You may get it.

Further, the route information acquisition unit 213 (FIG. 2) of the remote monitoring management server 20 acquires the route information Ip indicating the planned travel route PA of each of the plurality of autonomous traveling vehicles 60 (S110). The route information acquisition unit 213 may acquire the route information Ip directly from each autonomous traveling vehicle 60. Further, when the route information Ip is stored in the storage unit of a device other than the autonomous traveling vehicle 60 (for example, the remote monitoring device 30), the route information acquisition unit 213 stores the stored route information Ip. You may get it.

Further, the peripheral object information acquisition unit 214 (FIG. 2) of the remote monitoring management server 20 acquires peripheral object information Io indicating the state of each peripheral object OB of the plurality of autonomous traveling vehicles 60 (S120). In the present embodiment, the state of the peripheral object OB indicated by the peripheral object information Io includes the position, direction, and velocity of the peripheral object OB. The peripheral object information acquisition unit 214 may directly acquire peripheral object information Io indicating the state of the peripheral object OB detected by the sensor 66 or the like of each autonomous traveling vehicle 60 from each autonomous traveling vehicle 60. Further, the peripheral object information acquisition unit 214 acquires position information indicating the position of each autonomous traveling vehicle 60, refers to the map information 221 stored in the storage unit 220, and refers to the peripheral object at the position of each autonomous traveling vehicle 60. Peripheral object information Io indicating the state of OB may be acquired.

In this way, the remote monitoring of the remote monitoring management server 20 is performed by executing the vehicle information Iv acquisition process (S100), the route information Ip acquisition process (S110), and the peripheral object information Io acquisition process (S120). As shown in FIG. 4, the management unit 211 (FIG. 2) determines the state (position, direction, speed) of each autonomous traveling vehicle 60, the planned travel route PA, and the state (position, direction, speed) of the peripheral object OB. Can be recognized. The execution order of the vehicle information Iv acquisition process (S100), the route information Ip acquisition process (S110), and the peripheral object information Io acquisition process (S120) can be arbitrarily set, and two or three can be set arbitrarily. Two processes may be executed at the same time.

Next, the conversion unit 215 (FIG. 2) of the remote monitoring management server 20 performs conversion processing of the states (position, direction, speed, etc.) of the planned travel route PA and the peripheral object OB for each of the plurality of autonomous traveling vehicles 60. Do (S130). In this conversion process, the conversion of the planned travel path PA to the predetermined reference shape and the conversion of the state of the peripheral object OB are performed so that the relative relationship between the planned travel path PA and the state of the peripheral object OB is maintained. This is the process to be executed. In this embodiment, the reference shape is a linear shape. Therefore, the planned travel route PA is converted into a linear shape. In the columns A and B of FIG. 5, for the autonomous traveling vehicles 60a and 60b shown in the columns A and B of FIG. 4, the planned traveling route PA converted into a linear shape (hereinafter, "reference planned traveling route PAx"). ”) And the state (position and orientation) of the peripheral object OB after conversion are shown. In such a conversion, for example, for each point where the planned travel path PA before conversion is discretized at a predetermined pitch, a conversion matrix to one point on the reference travel route PAx having a linear shape is obtained, and the transformation matrix is used. It can be realized by converting the state (position and orientation) of the peripheral object OB by using it. Further, the state of the peripheral object OB after the conversion is shown as a heat map in which the state of the peripheral object OB after the conversion obtained for each point where the above-mentioned planned travel path PA before the conversion is discretized at a predetermined pitch is superimposed. It may be expressed as the state of the peripheral object OB after the conversion obtained for the point closest to the peripheral object OB in the planned travel route PA before the conversion.

Next, the image generation unit 216 (FIG. 2) of the remote monitoring management server 20 generates the monitoring image MI (S140). FIG. 6 shows an example of the surveillance image MI. As shown in FIG. 6, the monitoring image MI is an image showing the state of the converted travel schedule route PA (reference travel schedule route PAx) and the peripheral object OB for each autonomous traveling vehicle 60 from a bird's-eye view. That is, in the monitoring image MI, the reference travel schedule route PAx and the states of all peripheral objects OB for all the autonomous traveling vehicles 60 to be monitored by the remote monitoring device 30 (monitorer MO) are collectively shown. It is an image. For example, as shown in FIG. 6, in the surveillance image MI, the peripheral objects OB1 and OB2 existing around one autonomous traveling vehicle 60a and the peripheral objects OB3 existing around the other autonomous traveling vehicle 60b are shown. OB4 is shown.

However, in the present embodiment, the monitoring image MI is an image shown so that the reference travel schedule path PAx of each autonomous vehicle 60 overlaps with each other with the current position of each autonomous vehicle 60 as a reference point. That is, the image generation unit 216 matches the current positions of the autonomous traveling vehicles 60 with each other in the reference traveling route PAx of the autonomous traveling vehicle 60, and also extends the extension direction of the reference traveling route PAx of the autonomous traveling vehicle 60. By matching with each other, the surveillance image MI is generated. Therefore, the reference travel route PAx actually shown in the monitoring image MI is only one.

Further, in the present embodiment, the surveillance image MI is an image expressed so that the scale of the time distance derived based on the speed of the autonomous traveling vehicle 60 is aligned for each autonomous traveling vehicle 60. This point will be described below.

In the columns A and B of FIG. 5, the caution area AR is indicated by a broken line. The caution area AR is an area in which if a peripheral object OB exists in the area, there is a risk of collision between the peripheral object OB and the autonomous traveling vehicle 60 traveling on the planned travel route PA (reference travel schedule PAx). .. In the present embodiment, the warning area AR is set as a region having a predetermined width (width in a direction orthogonal to the reference travel schedule path PAx) along the reference travel schedule route PAx. The width of the warning area AR is determined based on the speed of the autonomous traveling vehicle 60. Specifically, in order to align the scale of the time distance, the faster the speed of the autonomous traveling vehicle 60, the larger the width of the warning area AR is set.

The image generation unit 216 aligns the scale of the time distance by aligning the widths of the warning areas AR set for each autonomous traveling vehicle 60 when generating the surveillance image MI. More specifically, as shown in FIG. 6, the surveillance image MI shows one warning area AR (hereinafter, referred to as “reference warning area AR0”). When generating the surveillance image MI, the image generation unit 216 converts the width of the warning area AR into the width of the reference warning area AR0 for each autonomous vehicle 60, and the state (position, etc.) of the peripheral object OB accordingly. Also convert. For example, for the autonomous traveling vehicle 60 (autonomous traveling vehicle 60b) traveling at a relatively low speed shown in column B of FIG. 5, the width of the warning area AR is relatively narrow. Assuming that the width of the warning area AR is narrower than the width of the reference warning area AR0, the peripheral object OB maintains the relative positional relationship between the warning area AR and the peripheral object OB when the surveillance image MI is generated. It is placed at a position farther from the reference travel route PAx than the actual position. On the contrary, for example, for the autonomous traveling vehicle 60 (autonomous traveling vehicle 60a) traveling at a relatively high speed shown in the column A of FIG. 5, the width of the warning area AR is relatively wide. Assuming that the width of the warning area AR is wider than the width of the reference warning area AR0, the peripheral object OB maintains the relative positional relationship between the warning area AR and the peripheral object OB when the surveillance image MI is generated. It is arranged at a position closer to the reference travel route PAx than the actual position. By generating the monitoring image MI in this way, the scales of the time distances for each autonomous traveling vehicle 60 are aligned.

The surveillance image MI generated in this way has one state of all peripheral objects OB located around all autonomous traveling vehicles 60 that are the targets of remote monitoring by the remote monitoring device 30 (monitorer MO). The image is shown with the time and distance scales aligned around the linear reference travel route PAx. That is, the surveillance image MI is an image as if one autonomous traveling vehicle 60 is being remotely monitored.

Next, the display control unit 217 (FIG. 2) of the remote monitoring management server 20 causes the generated monitoring image MI to be displayed on the display device 32 of the remote monitoring device 30 (S150). The observer MO refers to the monitoring image MI displayed on the display device 32, and simultaneously performs remote monitoring of a plurality of autonomous traveling vehicles 60 as if remote monitoring of one autonomous traveling vehicle 60 is performed.

In the remote monitoring management unit 211 (FIG. 2) of the remote monitoring management server 20, whether or not the peripheral object OB has been selected by the observer MO while the monitoring image MI is displayed on the display device 32 of the remote monitoring device 30. Is monitored (S160). The selection of the peripheral object OB is realized, for example, by designating one peripheral object OB or its surroundings on the surveillance image MI displayed on the display device 32 via the user interface of the remote monitoring device 30. .. The observer MO selects a peripheral object OB that may interfere with the autonomous traveling of the autonomous traveling vehicle 60, for example, selecting a peripheral object OB that has entered (or is likely to enter) the reference warning area AR0 in the surveillance image MI. be able to.

When it is determined in S160 that the peripheral object OB has been selected by the observer MO (S160: YES), the display control unit 217 (FIG. 2) of the remote monitoring management server 20 determines the selected peripheral object OB. The detailed image of is displayed on the display device 32 of the remote monitoring device 30 (S170). As a detailed image of the selected peripheral object OB, for example, the sensor 66 (for example, a camera) of the autonomous traveling vehicle 60 (autonomous traveling vehicle 60 approaching the peripheral object OB) corresponding to the selected peripheral object OB is used. The acquired image (video), an enlarged map of the area where the selected peripheral object OB is located, and the like are used. In the display device 32, the detailed image of the peripheral object OB may be displayed side by side with the monitoring image MI, may be displayed superimposed on the monitoring image MI, may be displayed by switching to the monitoring image MI, or may be displayed by switching to the monitoring image MI. It may be displayed as part of. The observer MO refers to the detailed image of the peripheral object OB displayed on the display device 32, determines the necessity of intervention for the control of the autonomous traveling vehicle 60 with respect to the peripheral object OB, or performs an actual intervention. You can do it.

The display control unit 217 (FIG. 2) of the remote monitoring management server 20 is in a state where the detailed image of the peripheral object OB is displayed on the display device 32 of the remote monitoring device 30, and whether or not the display end instruction of the detailed image is given. (S180), and when it is determined that the instruction to end the display of the detailed image has been given (S180: YES), the display of the detailed image is terminated (S190).

After that, the above-mentioned processes after S100 are repeatedly executed. That is, by acquiring the vehicle information Iv, the route information Ip, and the peripheral object information Io, the current position of the autonomous traveling vehicle 60, the planned travel route PA, and the state of the peripheral object OB are updated, and the monitoring image MI is updated accordingly. Will be updated. For example, when a peripheral object OB located near the traveling direction of a certain autonomous traveling vehicle 60 moves so as to approach the planned traveling route PA, the peripheral object OB is based on the reference traveling route PAx in the monitoring image MI. The surveillance image MI is updated as shown at a nearby location. Further, when a certain autonomous traveling vehicle 60 accelerates, the time distance of the autonomous traveling vehicle 60 becomes shorter, so that the peripheral object OB located near the traveling direction of the autonomous traveling vehicle 60 is used as a reference. The surveillance image MI is updated as shown at a position closer to the planned travel route PAx. On the contrary, when a certain autonomous traveling vehicle 60 decelerates, the time distance of the autonomous traveling vehicle 60 becomes long, so that the peripheral object OB located near the traveling direction of the autonomous traveling vehicle 60 is stopped. The surveillance image MI is updated so as to be shown at a position farther from the reference travel route PAx.

If it is determined in S160 that the peripheral object OB has not been selected (S160: NO), the remote monitoring management unit 211 (FIG. 2) of the remote monitoring management server 20 gives an instruction to end the remote monitoring management process. It is determined whether or not this has been done (S200). If it is determined in S200 that the end instruction of the remote monitoring management process has not been given (S200: NO), the above-mentioned processes after S100 are repeatedly executed. On the other hand, when it is determined in S200 that the instruction to end the remote monitoring management process has been given (S200: YES), the remote monitoring management unit 211 (FIG. 2) of the remote monitoring management server 20 ends the remote monitoring management process.

A-3. Effect of this embodiment:
As described above, the remote monitoring management server 20 of the present embodiment is an information processing device for remote monitoring of a plurality of autonomous traveling vehicles 60. The remote monitoring management server 20 includes a route information acquisition unit 213, a peripheral object information acquisition unit 214, a conversion unit 215, an image generation unit 216, and a display control unit 217. The route information acquisition unit 213 acquires the route information Ip indicating the planned travel route PA of each autonomous traveling vehicle 60. The peripheral object information acquisition unit 214 acquires peripheral object information Io indicating a state including at least the position of the peripheral object OB of each autonomous traveling vehicle 60. The conversion unit 215 converts the planned travel route PA into a predetermined reference shape and peripheral objects so that the relative relationship between the planned travel route PA and the state of the peripheral object OB is maintained for each autonomous traveling vehicle 60. Performs OB state conversion and. The image generation unit 216 generates a monitoring image MI showing the states of the planned travel path PA and the peripheral object OB after conversion for each autonomous traveling vehicle 60. The display control unit 217 causes the display device 32 to display the monitoring image MI.

As described above, according to the remote monitoring management server 20 of the present embodiment, a predetermined standard is set so that the relative relationship between the planned travel route PA and the state of the peripheral object OB is maintained for each autonomous traveling vehicle 60. The conversion of the planned travel route PA to the shape and the conversion of the state of the peripheral object OB are executed, and for each autonomous traveling vehicle 60, a monitoring image MI showing the status of the planned travel route PA and the peripheral object OB after conversion is generated. , The monitoring image MI is displayed on the display device 32. Therefore, in the monitoring image MI, the planned travel route PA for each autonomous traveling vehicle 60 is shown in a state of being aligned with a predetermined reference shape (in the same shape as each other). Therefore, the observer MO is compared with the configuration in which the planned travel path PA for each autonomous traveling vehicle 60 is shown as it is (that is, in a disjointed shape for each autonomous traveling vehicle 60), and the observer MO is the planned travel route PA and the peripheral object OB. The relationship with can be easily grasped. From the above, according to the remote monitoring management server 20 of the present embodiment, it is possible to reduce the burden on the observer MO when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60.

Further, in the monitoring image MI generated by the remote monitoring management server 20 of the present embodiment, the planned travel route PA (reference travel schedule PAx) after conversion of each autonomous travel vehicle 60 is the current position of each autonomous travel vehicle 60. It is an image shown so as to overlap each other with the reference point. Therefore, according to the remote monitoring management server 20 of the present embodiment, even when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60, the observer MO is a single planned travel route. Only the relationship between the PA (reference travel route PAx) and each peripheral object OB needs to be monitored. Therefore, according to the remote monitoring management server 20 of the present embodiment, it is possible to effectively reduce the burden on the observer MO when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60.

Further, the surveillance image MI generated by the remote monitoring management server 20 of the present embodiment is an image expressed so that the scale of the time distance derived based on the speed of the autonomous traveling vehicle 60 is aligned for each autonomous traveling vehicle 60. be. Therefore, according to the remote monitoring management server 20 of the present embodiment, even when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60, the observer MO is the autonomous traveling vehicle 60. Judging whether or not intervention in the control of the autonomous vehicle 60 is necessary based on the distance between the planned travel route PA of the autonomous vehicle 60 and each peripheral object OB in the monitoring image MI without being aware of the speed difference. Can be done. Therefore, according to the remote monitoring management server 20 of the present embodiment, the burden on the observer MO when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60 can be extremely effectively reduced. ..

Further, the monitoring image MI generated by the remote monitoring management server 20 of the present embodiment is a warning area which is an area where there is a possibility of collision with the autonomous traveling vehicle 60 traveling on the planned traveling route PA (reference planned traveling route PAx). It is an image showing AR. Therefore, according to the remote monitoring management server 20 of the present embodiment, even when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60, the observer MO is a warning area in the monitoring image MI. Based on the positional relationship between the AR and each peripheral object OB, it is possible to easily determine whether or not intervention in the control of the autonomous traveling vehicle 60 is necessary. Therefore, according to the remote monitoring management server 20 of the present embodiment, the burden on the observer MO when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60 can be extremely effectively reduced. ..

Further, in the remote monitoring management server 20 of the present embodiment, the display control unit 217 displays a detailed image of the selected peripheral object OB in response to the selection of the peripheral object OB included in the monitoring image MI. Display on 32. Therefore, according to the remote monitoring management server 20 of the present embodiment, the observer MO refers to the detailed image of the peripheral object OB displayed on the display device 32, and refers to the peripheral object OB with respect to the autonomous traveling vehicle 60. You can determine the need for control interventions and perform actual interventions. Therefore, according to the remote monitoring management server 20 of the present embodiment, each autonomous traveling vehicle 60 is reduced from the burden on the observer MO when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60. Appropriate remote monitoring can be performed.

Further, in the remote monitoring management server 20 of the present embodiment, the reference shape when converting the planned travel route PA is a linear shape. That is, the reference travel schedule route PAx, which is the travel schedule route PA after conversion, has a linear shape. Therefore, according to the remote monitoring management server 20 of the present embodiment, the observer MO may monitor the relationship between the planned travel route PA (reference travel schedule PAx) having a simple shape and each peripheral object OB. , The necessity of intervention in the control of the autonomous traveling vehicle 60 can be easily determined. Therefore, according to the remote monitoring management server 20 of the present embodiment, it is possible to effectively reduce the burden on the observer MO when one observer MO performs remote monitoring of a plurality of autonomous traveling vehicles 60.

B. Modification example:
The technique disclosed in the present specification is not limited to the above-described embodiment, and can be transformed into various forms without departing from the gist thereof, and for example, the following modifications are also possible.

The configuration of the remote monitoring system 10 in the above embodiment is merely an example and can be variously modified. For example, in the above embodiment, the remote monitoring management server 20 and each remote monitoring device 30 are communicably connected via the internal network 82, but the remote monitoring management server 20 and each remote monitoring device 30 are connected to the Internet or the like. It may be connected so as to be communicable via the external network of. Further, the remote monitoring management server 20 may have a function as a remote monitoring device, and the observer MO may perform remote monitoring of the autonomous traveling vehicle 60 using the remote monitoring management server 20.

In each of the above embodiments, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part of the configuration realized by the software may be replaced with hardware. ..

The content of the remote monitoring management process in the above embodiment is merely an example and can be changed in various ways. For example, in the above embodiment, the peripheral object information Io indicating the state of the peripheral object OB of each autonomous traveling vehicle 60 includes information indicating the position, direction, and speed of the peripheral object OB, but the peripheral object information Io is included. , It suffices to include information indicating at least the position of the peripheral object OB, and may not include information indicating at least one of the direction and the speed of the peripheral object OB, and indicates another state of the peripheral object OB. It may contain information.

In the above embodiment, the planned travel path PA of each autonomous vehicle 60 is converted into a linear shape, but the planned travel route PA of each autonomous vehicle 60 has a predetermined reference shape (for example, a circle) other than the linear shape. It may be converted into a shape, a spiral shape, etc.).

In the above embodiment, when the monitoring image MI is generated, the scale of the time distance derived based on the speed of the autonomous traveling vehicle 60 is aligned with each autonomous traveling vehicle 60 in the width direction (orthogonal to the reference traveling route PAx). The distance between the reference travel route PAx and the peripheral object OB in the direction) is adjusted, but the mode of aligning the scale of the time distance is not limited to this. For example, when the surveillance image MI is generated, the time distance scale may be aligned by adjusting the distance between the autonomous traveling vehicle 60 and the peripheral object OB in the direction parallel to the reference travel schedule path PAx. For example, in the surveillance image MI, the peripheral object OB located near the traveling direction of the autonomous traveling vehicle 60 having a relatively high speed is shown relatively close to the autonomous traveling vehicle 60, and the traveling of the autonomous traveling vehicle 60 having a relatively slow speed is shown. Peripheral objects OBs located near the direction may be scaled in time and distance by being shown relatively far from the autonomous vehicle 60.

In the above embodiment, the surveillance image MI is an image showing the warning area AR which is an area where there is a possibility of collision with the autonomous traveling vehicle 60 traveling on the planned travel route PA, but the monitoring image MI does not necessarily include the warning area AR. It does not need to be shown. Further, in the present embodiment, the surveillance image MI is an image expressed so that the scale of the time distance derived based on the speed of the autonomous traveling vehicle 60 is aligned for each autonomous traveling vehicle 60, but the surveillance image MI is not necessarily used. It is not necessary for the time distance scales for each autonomous vehicle 60 to be aligned. For example, in the surveillance image MI, the states of the reference travel schedule path PAx and the peripheral object OB for each autonomous vehicle 60 are merged as they are without the processing of aligning the time distance scales for each autonomous vehicle 60. It may be an image. Further, in the present embodiment, the monitoring image MI is an image showing that the converted travel schedule paths PA of each autonomous vehicle 60 overlap each other with the current position of each autonomous vehicle 60 as a reference point. , It is not always necessary to show in the monitoring image MI that the converted travel schedule paths PA of each autonomous vehicle 60 overlap each other. For example, the surveillance image MI may be an image individually showing the states of the reference travel schedule path PAx and the peripheral object OB for each autonomous traveling vehicle 60. Further, in the above embodiment, the surveillance image MI is an image showing the state of the reference travel schedule path PAx and the peripheral object OB for each autonomous traveling vehicle 60 from a bird's-eye view, but the display mode of the surveillance image MI is limited to this. I can't. For example, the surveillance image MI may be an image (for example, an image simulating the view from the driver's seat of a virtual vehicle) showing the states of the reference travel route PAx and the peripheral object OB in three dimensions. In that case, the reference travel route PAx is shown so as to appear as a reference shape (for example, a linear shape) in the image shown three-dimensionally.

Further, in the above embodiment, the remote monitoring of a plurality of autonomous traveling vehicles 60 is described as an example, but the technology disclosed in the present specification autonomously travels like, for example, an autonomous traveling robot. It can also be applied to remote monitoring of moving objects other than vehicles.

10: Remote monitoring system 20: Remote monitoring management server 30: Remote monitoring device 32: Display device 40: Network communication device 60: Autonomous traveling vehicle 62: Autonomous driving control unit 64: Communication unit 66: Sensor 68: Drive unit 82: Internal Network 84: External network 210: Control unit 211: Remote monitoring management unit 212: Vehicle information acquisition unit 213: Route information acquisition unit 214: Peripheral object information acquisition unit 215: Conversion unit 216: Image generation unit 217: Display control unit 220: Storage unit 221: Map information 230: Display unit 240: Operation input unit 250: Interface unit 290: Bus AR0: Reference alert area AR: Alert area CP: Remote monitoring management program Io: Peripheral object information Ip: Route information MI: Monitoring image MO: Observer OB: Peripheral object PA: Scheduled travel route PAx: Reference planned travel route

Claims (8)

An information processing device for remote monitoring of multiple autonomous moving objects.
A route information acquisition unit that acquires route information indicating a planned travel route of each autonomous traveling vehicle, and a route information acquisition unit.
A peripheral object information acquisition unit that acquires peripheral object information indicating a state including at least the position of the peripheral object of each autonomous traveling moving body, and a peripheral object information acquisition unit.
For each of the autonomous traveling bodies, the conversion of the planned traveling route to a predetermined reference shape and the state of the peripheral object so that the relative relationship between the planned traveling route and the state of the peripheral object is maintained. A converter that performs conversions, and
An image generation unit that generates a monitoring image showing the state of the planned travel route and the peripheral objects after the conversion for each autonomous traveling vehicle, and an image generation unit.
A display control unit that displays the monitoring image on the display device,
An information processing device equipped with. The information processing apparatus according to claim 1.
The monitoring image is an information processing device, which is an image in which the converted travel schedule of each autonomous traveling vehicle is shown to overlap each other with the current position of each autonomous traveling vehicle as a reference point. The information processing apparatus according to claim 2, further
A vehicle information acquisition unit for acquiring vehicle information indicating a state including at least a speed of each of the autonomous traveling moving bodies is provided.
The monitoring image is an information processing device, which is an image in which the scale of the time distance derived based on the speed of the autonomous traveling vehicle is expressed so as to be aligned for each of the autonomous traveling vehicles. The information processing apparatus according to claim 3.
The monitoring image is an information processing device which is an image showing a warning area where there is a possibility of collision with the autonomous traveling moving body traveling on the planned traveling route. The information processing apparatus according to any one of claims 1 to 4.
The display control unit is an information processing device that causes the display device to display a detailed image of the selected peripheral object in response to the selection of the peripheral object included in the monitoring image. The information processing apparatus according to any one of claims 1 to 5.
The information processing device whose reference shape is a linear shape. It is an information processing method for remote monitoring of multiple autonomous moving objects.
A process of acquiring route information indicating a planned travel route of each autonomous traveling vehicle, and
A step of acquiring peripheral object information indicating a state including at least a position of a peripheral object of each autonomous traveling moving body, and a step of acquiring peripheral object information.
For each of the autonomous traveling moving bodies, the conversion of the planned traveling route to a predetermined reference shape and the state of the peripheral object so that the relative relationship between the planned traveling route and the state of the peripheral object is maintained. The process of performing the conversion and
A step of generating a monitoring image showing the state of the planned travel route and the peripheral objects after the conversion for each autonomous traveling vehicle, and a step of generating a monitoring image.
The process of displaying the monitoring image on the display device and
Information processing method. A computer program for remote monitoring of multiple autonomous moving objects.
On the computer
Processing to acquire route information indicating the planned travel route of each autonomous traveling vehicle, and
The process of acquiring peripheral object information indicating a state including at least the position of the peripheral object of each autonomous traveling moving body, and
For each of the autonomous traveling bodies, the conversion of the planned traveling route to a predetermined reference shape and the state of the peripheral object so that the relative relationship between the planned traveling route and the state of the peripheral object is maintained. The process of performing the conversion and
A process of generating a monitoring image showing the state of the planned travel route and the peripheral objects after the conversion for each autonomous traveling vehicle, and a process of generating a monitoring image.
The process of displaying the surveillance image on the display device and
A computer program that runs.

Images