JP2020017240A - Monitoring support device, monitoring support program, and storage media - Google Patents

Abstract

To provide a monitoring support device that measures a traveling direction of a vehicle without being influenced by light or shadow or by rain, snow, insects or the like in outdoor use.SOLUTION: A monitoring support device MAD includes: an acquisition unit ACQ that acquires images captured by stereo cameras; a monitoring area setting unit MPS that sets a monitoring plane in the image; a direction setting unit DD that sets a reverse running direction or a forward direction in the image; a monitoring unit MON that sets an object being in contact the monitoring plane as a target object, compares a space coordinate of the target object with that of the target object for at least one frame preceding or following the image, and determines a movement direction of the target object; and an alarm unit WAN that issues an alarm when the movement direction of the target object is the reverse running direction or a direction opposite to the forward direction.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a monitoring support device, a monitoring support program, and a storage medium, and particularly to a monitoring device that measures a traveling direction of a vehicle without being affected by light or shadow, rain, snow, insects, and the like in outdoor use. The present invention relates to a support device, a monitoring support program, and a storage medium.

  In recent years, the number of elderly drivers has increased and the number of reverse running accidents on highways has increased. Regarding the monitoring of this reverse run, technical development is awaited. On the other hand, surveillance cameras have become inexpensive, many have been installed, and are used as surveillance devices.

JP-A-10-269492

  The technique disclosed in Patent Document 1 is based on “a plurality of vehicle sensors installed on a one-way road, a vehicle traveling direction determination unit that determines a traveling direction of a traveling vehicle from outputs of the plurality of vehicle sensors, And a reverse traveling vehicle determining unit that determines whether the traveling vehicle is traveling on the one-way street in the reverse direction based on the determination result of the determining unit. " This is based on the premise that a radio wave transmitting device for transmitting a radio wave such as a beacon is installed in the vehicle. Therefore, it is necessary to install a radio wave transmitting device in all vehicles entering the monitoring section, and it is difficult to introduce the radio wave transmitting device.

  In addition, there is a technique of following a specific object of interest (such as a person or a puppy) with an image of a two-dimensional camera, but the two-dimensional camera is affected by light or shadow outdoors or located between the camera and a vehicle outdoors. There are many erroneous detections due to birds, insects, rain, snow, and the like, and the technique of judging the direction of a vehicle outdoors has not been practically achieved.

  Therefore, an object of the present invention is to provide a monitoring support device, a monitoring support program, and a storage medium.

In order to solve the above-mentioned problems, a monitoring support device according to a first invention is provided.
An acquisition unit for acquiring an image captured by a stereo camera,
A monitoring surface setting unit for setting a monitoring surface in the image,
A direction setting unit for setting a reverse running direction or a forward direction in the image,
A monitoring unit that sets an object in contact with the monitoring surface as a target object, compares the spatial coordinates of the target object with the target object of at least one frame before or after the image, and obtains a moving direction of the target object When,
An alarm unit that issues an alarm when the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction;
Having.
It is characterized by having.

Also, the monitoring support device according to the second invention,
An output unit that outputs an image in which the object of interest is reflected together with the warning,
Has further.

The monitoring support device according to the third invention is
The alarm unit,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction Even if the reverse is true, if the maximum area of the object of interest on the monitoring surface is less than the first reference (excluded as a small object such as dust, a falling object, or a small volume such as a sheet) , Do not alert,
It is characterized by the following.

Further, the monitoring support device according to the fourth invention is
For each frame in which the object of interest contacts or / and passes through the monitoring surface, a three-dimensional object generation unit that collects a ring-shaped shape of the object on the monitoring surface and generates a three-dimensional object of interest from the object is provided. ,
Have more.

The monitoring support device according to a fifth aspect of the present invention
The alarm unit,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction Even if the opposite is true, if the volume of the three-dimensional object of interest is less than the first criterion (excluded as having no volume such as dust, falling objects, sheets, etc.), no alarm is issued.
It is characterized by the following.

Further, a monitoring support device according to a sixth aspect of the present invention,
The alarm unit,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction If the reverse of the above, and if the first criterion or more, or if the first criterion or more and less than the second criterion (treated as a vehicle of the first size such as a motorcycle, a car, etc.), the vehicle of the first size Give an alarm as if you have run backwards,
A monitoring support device characterized by the above-mentioned.

Further, a monitoring support device according to a seventh aspect of the present invention
An acquisition unit for acquiring an image captured by a stereo camera,
A monitoring surface setting unit configured to set a first monitoring surface and a second monitoring surface in the image;
A direction setting unit for setting a reverse running direction or a forward direction in the image,
An object that has contacted the first or second monitoring surface is set as a target object, and a time difference between when the target object first contacts the first monitoring surface and when the target object first contacts the second monitoring surface. A monitoring unit that determines the moving direction of the object of interest using
An alarm unit that issues an alarm when the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction;
Having.

Also, the monitoring support device according to the eighth invention is
An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
An alarm unit that issues an alarm when the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction;
Having.

Also, the monitoring support device according to the ninth invention is:
An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
The object of interest on the first monitoring surface of the first image and the object of interest on the second monitoring surface of the second image are merged to generate an integrated object of interest, and the integrated object of interest is virtually monitored. A projection unit that projects onto a surface,
A determining unit that determines whether the area of the integrated target object on the virtual monitoring surface exceeds a predetermined threshold,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, and the area of the integrated object of interest on the monitoring surface exceeds a predetermined threshold, an alarm is issued. An alarm unit that issues
Having.

Also, a monitoring support device according to a tenth aspect of the present invention
An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
The object of interest on the first monitoring surface of the first image and the object of interest on the second monitoring surface of the second image are merged to generate an integrated object of interest, and the integrated object of interest is virtually monitored. A projection unit that projects onto a surface,
A determination unit configured to determine whether an area (size) of the integrated target object on the virtual monitoring surface is greater than or equal to a first threshold and less than or equal to a second threshold, or whether or not exceeds a second threshold; ,
When the moving direction of the object of interest is the reverse running direction or the reverse direction of the forward direction, and the area of the integrated object of interest on the monitoring surface exceeds the first threshold value and moves to the second direction. If the value is equal to or less than the threshold, an alarm is issued together with the information that the object of interest is a vehicle of the first size (for example, a motorcycle, a small vehicle, or a passenger car), or the moving direction of the object of interest is reversed. In the case of the running direction or the reverse of the forward direction and exceeding the second threshold, the target object is a vehicle of the second size (for example, a truck, a bus, or a large vehicle). ), An alarm unit that issues an alarm,
Having.

Also, the monitoring support device according to the eleventh invention is:
An acquisition unit configured to acquire a first image captured by a first stereo camera at a first location and a second image captured by a second stereo camera at a second location;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
When the moving direction of the object of interest is a direction between the first point and the second point, the number of the object of interest is incremented by one, and the moving direction of the object of interest is changed to the first point. A measurement unit that subtracts 1 from the number of the target objects and counts the number of target objects between the first point and the second point in a case where the direction is from the second point.
Having.

  As described above, the solving means of the present invention has been described as an apparatus. However, the present invention can be realized as a method, a program, or a storage medium on which the program is substantially equivalent. Should be understood to include these. The following method and each step of the program use an arithmetic processing unit such as a CPU or DSP as necessary in data processing, and input data, processed / generated data, etc. , HDD, memory and the like.

For example, a program according to a twelfth invention that causes a computer to execute the present invention as a program is:
A monitoring support program for causing one or more arithmetic processing devices to function as the monitoring support device according to the first to eleventh aspects.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to measure the driving | running | working direction of a vehicle and to monitor a vehicle appropriately, without being affected by light and a shadow, rain, snow, an insect, etc. in outdoor use.

FIG. 1 is a block diagram showing an outline of a monitoring support device (system) according to an embodiment of the present invention. FIG. 2 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse running monitoring device. FIG. 3 is a flowchart illustrating an example of a process executed by the monitoring support apparatus (system) according to an embodiment of the present invention. FIG. 4 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse running monitoring device. FIG. 5 is a flowchart illustrating an example of a process executed by the monitoring support apparatus (system) according to an embodiment of the present invention. FIG. 6 is a schematic diagram when the monitoring support device according to an embodiment of the present invention functions as a reverse running monitoring device. FIG. 7 is a schematic diagram illustrating a technique for generating an integrated object using two stereo cameras. FIG. 8 is a schematic diagram illustrating the technique for generating an integrated object using two stereo cameras in more detail. FIG. 9 is a schematic diagram illustrating a technique for modeling an object from each cross section of the object passing through the monitoring surface. FIG. 10 is a schematic diagram illustrating a technique for classifying objects that have contacted a monitoring surface with the present device. FIG. 11 is a schematic diagram of an embodiment in which the present apparatus is realized as a car counter (vehicle number measuring apparatus for each type of vehicle). FIG. 12 is a schematic diagram of another embodiment in which the present apparatus is realized as a car counter (vehicle number measuring apparatus for each type of vehicle). FIG. 13 is a schematic diagram illustrating a state in which the vehicle tracks the trajectories PW1-PW4 of the vehicle after entering the rotary.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing an outline of a monitoring support device (system) according to an embodiment of the present invention. As shown in the figure, the monitoring support device MAD1
An acquisition unit ACQ for acquiring an image captured by a stereo camera,
A monitoring surface setting unit MPS for setting a monitoring surface in the image;
A direction setting unit DS for setting a reverse running direction or a forward direction in the image;
A monitoring unit that sets an object in contact with the monitoring surface as a target object, compares the spatial coordinates of the target object with the target object of at least one frame before or after the image, and obtains a moving direction of the target object MON,
An alarm unit WAN that issues an alarm when the moving direction of the target object is the reverse running direction or the reverse of the forward direction;
Having.

  The acquisition unit ACQ, the monitoring plane setting unit MPS, the direction setting unit DS, the monitoring unit MON, the alarm unit WAN, and the like are preferably configured by program modules loaded on the control unit CON, which is an arithmetic device such as a processor. .

  The apparatus has an output section OUT for outputting an image in which the object of interest is reflected, together with the alarm. The alarm unit WAN may double as the output unit.

  When the moving direction of the target object is the reverse running direction instead of issuing an alarm when the moving direction of the target object is the reverse running direction or the reverse of the forward direction, Or, even if the forward direction is reversed, if the maximum area of the object of interest on the monitoring surface is smaller than the first reference (a small object such as dust, a falling object, or a sheet having no volume) Does not trigger an alert. According to this configuration, it is possible to prevent a false alarm by appropriately excluding a small object such as dust, a falling object, or a sheet.

The control unit CON
For each frame in which the object of interest contacts or / and passes through the monitoring surface, a three-dimensional object generation unit GEN that collects a ring-shaped shape of the object on the monitoring surface and generates a three-dimensional object of interest from the object is provided. Have further.

In addition, the alarm unit WAN
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction Even if the opposite is true, if the volume of the three-dimensional object of interest is less than the first criterion (excluded as having no volume such as dust, falling objects, and sheets), an alarm may not be issued. It is possible. According to this configuration, it is possible to appropriately exclude a small object such as dust or a falling object, or a thin object having a small volume, such as a sheet, to prevent a false alarm.

In addition, the alarm unit WAN
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction If the reverse of the above, and if the first criterion or more, or if the first criterion or more and less than the second criterion (treated as a vehicle of the first size such as a motorcycle, a car, etc.), the vehicle of the first size A warning is issued as a reverse run. According to this configuration, it is possible to issue a warning by classifying the size of the reverse running vehicle.

  The monitoring support device MAD1 has an input unit IN, an output unit OUT, a communication unit COM, a storage unit STR, and a display unit DIS. Further, the monitoring support device MAD1 is connected to the stereo cameras MC1 and MC2 for photographing roads, routes, and the like via the communication unit COM and the network NET, and acquires images. The monitoring support device MAD1 is connected to the mobile terminal PDA1 via the communication unit COM and the network NET, and can exchange information. The output unit OUT can output information stored in the apparatus or generated information to the printer PRN or the electronic bulletin board EB1. The display unit DIS can also display information stored in the apparatus and generated information. The input unit IN receives an operation instruction or information input input via the mouse MUS or the keyboard KBD, and passes the received information or data to the acquisition unit ACQ. The storage unit STR stores reference information STD for storing a numerical value of the 1-3 reference and vehicle type information of the reference, and vehicle information CI including a vehicle object to be compared with the modeling object and the vehicle type name. .

  Each functional unit included in the control unit is preferably implemented by a program module read into the memory space of the control unit. Normally, software that causes a processor to function as each unit of the apparatus is downloaded from a storage unit or a website, installed in the PC, and activated, whereby the PC operates as the apparatus. Each of the functional units provided in the control unit is merely a single unit of a certain unit of function, and a plurality of functional units may be combined into one functional unit, or a part of the functional units may be replaced with another functional unit. It can be incorporated into a unit or divided into other independent functional units.

  As described above, the generated / extracted information, intermediate data, and acquired data are transmitted to the outside, displayed on the display unit, and the generated / extracted information, intermediate data, acquired data, and the like are stored in the storage unit. It should be noted that this is also possible in other embodiments described later. The present apparatus includes a computer such as a general-purpose computer, a special-purpose computer, a server, a PC, a mobile terminal such as a smartphone, or a program module that realizes (executes) the functions and processing procedures (methods) of the present apparatus on the computer. It is preferable to construct the present apparatus on a computer by holding it in a CPU or a storage unit of the computer or by reading it from an external server or storage, and the same applies to each of the following embodiments. In addition, each functional unit may be distributed to separate computers and devices connected by a network. In addition, a form in which a plurality of functional units are integrated into one unit, or a part of the processing steps is performed by another functional unit may be employed.

  FIG. 2 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse running monitoring device. As shown in the figure, it is assumed that there is a vehicle VH passing through a road RD. The reverse direction DIR of the vehicle is set in the image of the stereo camera CM1 that captures the vehicle VH passing through the road RD, and further, the monitoring surface VF is set in the image. Each time the vehicle touches the monitoring surface VF, the present device determines whether or not the vehicle is in the reverse running direction using the above-described technique. If it is determined that the vehicle is running in the reverse direction, an image of the vehicle and a warning indicating that the vehicle is running in the reverse direction are displayed on the electronic bulletin board EB1 installed in the reverse direction.

  FIG. 3 is a flowchart illustrating an example of a process executed by the monitoring support apparatus (system) according to an embodiment of the present invention. As shown in the figure, in step S11, a monitoring surface is set. The monitoring surface is preferably a vertical plane that crosses the road through which the vehicle passes. In step S12, a reverse running direction (or a forward direction) is set. Then, in step S13, an image is obtained. In step S14, the object touching the monitoring surface is set as the object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to move the object of interest. Find the direction. Then, in step S15, it is determined whether the moving direction of the target object is the reverse running direction or the reverse of the forward direction. If the determination condition of step S15 is satisfied, that is, if the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, the process proceeds to step S16, and an alarm is issued. At the time of the alarm, it is possible to display an image of the vehicle on a bulletin board or the like, or to play the vehicle type or number of the image by voice. If the determination is not satisfied, the process returns to step S13. Note that if the storage step is performed once, it is not necessary to re-perform until the information is changed. Alternatively, the storing step becomes unnecessary by mounting a ROM or a flash memory storing the information in an apparatus or a computer.

  FIG. 4 is a schematic diagram when the monitoring support device according to one embodiment of the present invention functions as a reverse running monitoring device. As shown in the figure, it is assumed that there is a vehicle VH passing through a road RD. The reverse running direction DIR of the vehicle is set in the image of the stereo camera CM1 that captures the vehicle VH passing through the road RD, and further, the monitoring surface VF1 and the monitoring surface VF2 are set in the image. Each time the vehicle touches the monitoring surface VF, the present device determines whether or not the vehicle is in the reverse running direction using the above-described technique. Specifically, an object that has contacted the first or second monitoring surface is set as a target object, and the time at which the target object first touched the first monitoring surface and the time at which the target object first touched the second monitoring surface. The moving direction of the object of interest is determined using the time difference at the time of contact. If it is determined that the vehicle is running in the reverse direction, an image of the vehicle and a warning indicating that the vehicle is running in the reverse direction are displayed on the electronic bulletin board EB1 installed in the reverse direction. According to this configuration, since there are two monitoring surfaces, it is easy to determine the direction, and the two monitoring surfaces can eliminate contact of flying objects such as sheets and cardboard. That is, false alarms can be suppressed. The flying object has a remarkable change in the shape at the time of contact at each contact surface, so that such a thing can be eliminated.

  FIG. 5 is a flowchart illustrating an example of a process executed by the monitoring support apparatus (system) according to an embodiment of the present invention. As shown in the figure, in step S21, first and second monitoring surfaces are set. The monitoring surface is preferably a vertical plane that crosses the road through which the vehicle passes. In step S22, a reverse running direction (or a forward direction) is set. Then, in step S23, an image is obtained. In step S24, an object that has contacted the first or second monitoring surface is set as a target object, and the time at which the target object first contacted the first monitoring surface, and the time that the target object first contacted the second monitoring surface. The moving direction of the object of interest is obtained using the time difference at the time of the movement. Then, in a step S25, it is determined whether or not the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction. If the determination condition of step S25 is satisfied, the process proceeds to step S26, and an alarm is issued. At the time of the warning, it is possible to display an image of the vehicle on a bulletin board or the like, or to play the vehicle type or number of the image by voice. If the determination condition is not satisfied, the process returns to step S23.

  FIG. 6 is a schematic diagram when the monitoring support device according to an embodiment of the present invention functions as a reverse running monitoring device. As shown in the figure, it is assumed that there is a vehicle VH passing through a road RD. The reverse running direction DIR of the vehicle is set in the image of the stereo camera CM1 that captures the vehicle VH passing through the road RD, and the monitoring surface VF1 is further set in the image. In the present configuration, the reverse running direction DIR of the vehicle is similarly set in the image of the stereo camera CM2 placed at a position facing the stereo camera CM1 with the road RD interposed therebetween, and further monitoring is performed in the image. The surface VF2 is set. The monitoring surface VF1 and the monitoring surface VF2 are set at the same position as spatial coordinates. That is, one of the features of the present configuration is to monitor the same monitoring surface from different positions. Each time the vehicle touches the monitoring surfaces VF1 and VF2, the device determines whether or not the vehicle is in the reverse running direction using the above-described technique. If it is determined that the vehicle is running in the reverse direction, an image of the vehicle and a warning indicating that the vehicle is running in the reverse direction are displayed on the electronic bulletin board EB1 installed in the reverse direction. One of the advantages of this configuration is that, with one camera, the field of view is blocked by a bird or insect passing between the camera and the subject (reverse vehicle), thereby eliminating the risk of omission of detection. . Even when the field of view of the stereo camera CM1 is blocked by a bird or the like, it is very unlikely that the field of view of the stereo camera CM2 will be blocked at the same time, so it is possible to reliably prevent omission of detection. .

  FIG. 7 is a schematic diagram illustrating a technique for generating an integrated object using two stereo cameras. In this case, a monitoring support device having the following configuration is used.

The monitoring support device is
An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared. A monitoring unit for determining a moving direction;
The object of interest on the first monitoring surface of the first image and the object of interest on the second monitoring surface of the second image are merged to generate an integrated object of interest, and the integrated object of interest is virtually monitored. A projection unit that projects onto a surface,
A determining unit that determines whether the area of the integrated target object on the virtual monitoring surface exceeds a predetermined threshold,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, and the area of the integrated object of interest on the monitoring surface exceeds a predetermined threshold, an alarm is issued. An alarm unit that issues
Having.

  As shown in the figure, when the monitoring surface is touched by a small vehicle, the stereo camera CM1 recognizes the object of interest AO1 on the first monitoring surface VF1 of the first image. Similarly, it is recognized that the small vehicle touches the surveillance surface even on the surveillance surface VF2 installed at the same spatial coordinates as the surveillance surface VF1. That is, the stereo camera CM2 recognizes the target object AO2 on the second monitoring surface VF2 of the second image. Then, the projection unit merges the target object on the first monitoring surface of the first image with the target object on the second monitoring surface of the second image to generate an integrated target object, and The target object IAO1 is projected on the virtual monitoring plane MVFI. The advantage of this technique is that two stereo cameras are monitoring the same plane surveillance surface from different angles, so that one stereo camera can capture the contours of objects that cannot be captured, the whole object, etc. , And by merging the objects captured by the respective cameras, it becomes possible to recognize a substantially complete outline of the object.

  FIG. 8 is a schematic diagram illustrating the technique for generating an integrated object using two stereo cameras in more detail. As shown in the figure, on the monitoring surface VF1, it is possible to recognize only the upper side SDT1 and the right side SDR1 of the object. On the other hand, on the monitoring surface VF2, it is possible to recognize only the upper side SDT2 and the right side SDL1 of the object. Then, the present apparatus merges these sides to generate an integrated object of interest, and projects the integrated object of interest IAO2 on the virtual monitoring plane MVFI. According to this technique, the size of the contact surface in the middle can be accurately grasped from the time when the contact with the monitor surface is started, and the object that contacts the monitor surface can be estimated. That is, the object passing through the surveillance plane can recognize only two sides (an upper side and a side on one side) close to the surveillance camera. Therefore, by merging the monitoring plane images of the cameras from two different directions as in the present technique, it is possible to estimate and grasp an accurate cross section of the monitoring plane of the object passing through the monitoring plane.

  FIG. 9 is a schematic diagram illustrating a technique for modeling an object from each cross section of the object passing through the monitoring surface. That is, the present apparatus collects, for each frame in which the object of interest contacts or / and passes through the monitoring surface, a ring-shaped shape of the object on the monitoring surface, and generates a three-dimensional object of interest from the object. This can be realized by providing a three-dimensional object generation unit. As shown in the figure, the present apparatus recognizes that the nose (tip portion) of the vehicle VH1 is in contact with the monitoring surface VF1 in the scene SN1. This scene SN1 is converted into a format in which the target object AO1 is projected on the virtual monitoring plane VFI1. Similarly, in the scene SN2, the present apparatus recognizes that the body (intermediate portion) of the vehicle VH1 is in contact with the monitoring surface VF1. This scene SN2 is converted into a format in which the target object AO2 is projected on the virtual monitoring plane VFI2. Then, in the scene SN3, the present device recognizes that the tail (rear end portion) of the vehicle VH1 is in contact with the monitoring surface VF1. This scene SN3 is converted into a format in which the target object AO3 is projected on the virtual monitoring plane VFI3. For convenience of drawing and explanation, there is only one intermediate scene. For example, when shooting at 60 frames / sec, depending on the speed of the vehicle, a scene in which one vehicle passes through the monitoring surface Is over ten and several dozen sections of the vehicle (that is, the object of interest) are projected. Finally, a modeling object MDO is generated based on these attention objects AO1-3 and the estimated vehicle speed (for example, the speed is obtained based on the time between frames and the moving distance of the attention object between frames). This modeling object MDO specifies the vehicle type by comparing and referring to the vehicle type database stored in the storage unit. In this case, it is estimated that the model is a model xx1 of the company A with a small vehicle. The more frames per second, the more accurate the modeling object can be generated. Further, in this example, the image is shot with one stereo camera, but it is of course possible to apply the present technique to images shot with two stereo cameras.

FIG. 10 is a schematic diagram illustrating a technique for classifying objects that have contacted a monitoring surface with the present device. As shown in the figure, in a process IST01, a contact area of a thing on a monitoring surface is obtained. Next, in processing IST02, the area of the object (or the modeling object) on the monitoring surface is obtained, or the size of the object is estimated from the area. Next, in processing IST03, the size (or area or volume) of the object is compared with a reference. The reference is a numerical value stored in the storage unit and includes a first reference, a second reference, and a third reference, and is set as follows.
Less than the first standard: trash, falling objects, birds, small animals, etc. More than the first standard, less than the second standard: motorcycles and bicycles More than the second standard, less than the third standard: small vehicles {passenger cars, etc.}
3rd criterion and above: large vehicles (truck, bus, etc.)
The numerical value of each reference may be an area or a volume.

  In this example, if it is less than the first criterion (such as "garbage" as shown), the process proceeds to processing IST04, and no alarm is issued. If it is equal to or more than the first criterion and less than the second criterion (such as "bike" as shown), the process proceeds to processing IST05, and a warning is issued as a motorcycle. Those that are equal to or more than the second criterion and less than the third criterion (such as small vehicles as shown) proceed to processing IST06, and are reported as small vehicles (such as passenger cars). Then, a vehicle having a third criterion or more (such as a large vehicle) proceeds to the process IST07 and issues a report as a large vehicle (such as a truck or a bus). In this way, the present apparatus can classify the reversely running object into a rough vehicle type.

  FIG. 11 is a schematic diagram of an embodiment in which the present apparatus is realized as a car counter (vehicle number measuring apparatus for each type of vehicle). This can be realized by the following device configuration.

The monitoring support device is
An acquisition unit configured to acquire a first image captured by a first stereo camera at a first location and a second image captured by a second stereo camera at a second location;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
When the moving direction of the object of interest is a direction between the first point and the second point, the number of the object of interest is incremented by one, and the moving direction of the object of interest is changed to the first point. A measurement unit that subtracts 1 from the number of the target objects and counts the number of target objects between the first point and the second point in a case where the direction is from the second point.
Having.

As shown in the figure, a stereo camera CM1 is installed on the approach path of the parking area PA on the highway, a monitoring surface VF1 is set there, and the number of vehicles entering the PA is counted (the numerical value is CT1). Similarly, the stereo camera CM2 is installed on the exit route of the parking area PA, the monitoring surface VF2 is set there, and the number of vehicles exiting from the PA is counted (the numerical value is CT2). Then, the number of vehicles staying in the PA is measured by the following equation.
Number of vehicles staying in PA = CT1-CT2
The present device can also measure the size of a vehicle, that is, the type of vehicle, and therefore can measure the number of stays for each type of vehicle. In this case, it is possible to estimate the usage rate of the parking space of each PA type from the number of stays of each type of vehicle, and this information can be displayed on a bulletin board on the main line before the PA. As a result, it is possible to prevent a situation in which there is no parking space for the type of vehicle that the driver drives in the PA that has been very close. In addition, this technique can be applied to a rotary at a station.

In this example, vehicles VH2, VH3, and VH4 have entered through monitoring surface VF1.
CTT1 = 3
Then, VH4 passes through monitoring surface VF2 and exits from PA.
CT2 = 1
Therefore, the number of PA staying vehicles = CT1-CT2
= 3-1
= 2.

  As shown in the figure, the “current number of vehicles”, the “total number of vehicles today”, and the like for each vehicle type are recorded in the car counter CCNT1 and displayed on the display unit or stored in the storage unit. It is possible.

  FIG. 12 is a schematic diagram of another embodiment in which the present apparatus is realized as a car counter (vehicle number measuring apparatus for each type of vehicle). The device configuration is almost the same as that described in FIG. 11 unless otherwise specified. The additional device is an overhead camera BCM that can overlook the rotary ROT. This will be described later with reference to FIG. Also in FIG. 12, it is possible to record the “current number of vehicles”, the “total number of vehicles today”, and the like of the vehicle type in the car counter CCNT2 and display them on the display unit or store them in the storage unit. is there.

  FIG. 13 is a schematic diagram illustrating a state in which the vehicle tracks the trajectories PW1-PW4 of the vehicle after entering the rotary. The overhead camera BCM is for tracking vehicles that have entered the rotary, one by one, from entry to exit. As shown in FIG. 13, the vehicle VH3-1 moves to a position PT1 along a locus PW1. Then, it is possible to record a state in which the vehicle VH3-1 is moving from the position PT1 to the position PT2 through the locus PW2. Then, the vehicle VH3-1 moves from the position PT2 through the locus PW3 to the position PT3, further passes through the locus PW4, passes through the position PT4, and retreats from the rotary. In this way, the present apparatus can record and analyze the state of the movement and parking of each vehicle during the rotary stay in real time or after the fact. In this embodiment, one vehicle is tracked. However, it is also possible to simultaneously track a plurality of vehicles and different types of vehicles (for example, a motorcycle and a truck). As shown in the figure, the “current number of vehicles”, the “total number of vehicles today”, and the like for each vehicle type are recorded in the car counter CCNT3 and displayed on the display unit or stored in the storage unit. It is possible. Further, it is possible to record “current average staying time”, “average staying time today” and the like for each vehicle type, display them on the display unit, and store them in the storage unit.

  Although the present invention has been described with reference to the drawings and embodiments, it should be noted that those skilled in the art can easily make various changes and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, processes and functions included in each unit, each step, and the like can be rearranged so as not to be logically inconsistent, and a plurality of means / units, steps, and the like can be combined into one or divided. It is possible. Alternatively, it should be noted that some components, functions, processes, steps, and the like of the apparatus, the method, the program, and the like according to the present invention can be arranged in a remote server or the like.

ACQ Acquisition unit AO1 Attention object AO2 Attention object AO3 Attention object CI Vehicle information CM1 Stereo camera CM2 Stereo camera COM Communication unit CON Control unit DIR Reverse running direction DIS Display unit DS Direction setting unit EB1 Electronic bulletin board GEN Dimensional object generation unit IAO1 Integrated object of interest IAO2 Integrated object of interest IN Input unit IST01 Process IST02 Process IST03 Process IST04 Process IST05 Process IST06 Process IST07 Process KBD Keyboard MAD1 monitoring support device MC1 stereo camera MDO modeling object MON monitoring unit MPS monitoring surface setting unit MUS mouse MVFI virtual monitoring surface NET network OUT output unit PA parking unit A PDA1... Portable terminal PRN... Printer RD... Road S11... Step S12... Step S13. SDR1 Right side SDT1 Top side SDT2 Top side SN1 Scene SN2 Scene SN3 Scene STD Reference information STR Storage unit VF Monitoring surface VF1 Monitoring surface VF2 Monitoring surface VFI1 Virtual monitoring surface VFI2 Virtual monitoring surface VFI3 Virtual monitoring plane VH Vehicle VH1 Vehicle VH2 Vehicle WAN Alarm section

Claims (13)

An acquisition unit for acquiring an image captured by a stereo camera,
A monitoring surface setting unit for setting a monitoring surface in the image,
A direction setting unit for setting a reverse running direction or a forward direction in the image,
A monitoring unit that sets an object in contact with the monitoring surface as a target object, compares the spatial coordinates of the target object with the target object of at least one frame before or after the image, and obtains a moving direction of the target object When,
An alarm unit that issues an alarm when the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction;
A monitoring support device having: The monitoring support device according to claim 1,
An output unit that outputs an image in which the object of interest is reflected together with the warning,
A monitoring support device further comprising: The monitoring support device according to claim 1 or 2,
The alarm unit,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction Even if the reverse is true, if the maximum area of the object of interest on the monitoring surface is less than the first reference (excluded as a small object such as dust, a falling object, or a small volume such as a sheet) , Do not alert,
A monitoring support device characterized by the above-mentioned. The monitoring support device according to claim 1 or 2,
For each frame in which the object of interest contacts or / and passes through the monitoring surface, a three-dimensional object generation unit that collects a ring-shaped shape of the object on the monitoring surface and generates a three-dimensional object of interest from the object is provided. ,
A monitoring support device further provided. The monitoring support device according to claim 4,
The alarm unit,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction Even if the opposite is true, if the volume of the three-dimensional object of interest is less than the first criterion (excluded as having no volume such as dust, falling objects, sheets, etc.), no alarm is issued.
A monitoring support device characterized by the above-mentioned. The monitoring support device according to claim 4,
The alarm unit,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, instead of issuing an alarm, if the moving direction of the object of interest is the reverse running direction, or the forward direction If the reverse of the above, and if the first criterion or more, or if the first criterion or more and less than the second criterion (handled as a first size vehicle such as a motorcycle, a car, etc.), the vehicle of the first size Give an alarm as if you have run backwards,
A monitoring support device characterized by the above-mentioned. An acquisition unit for acquiring an image captured by a stereo camera,
A monitoring surface setting unit configured to set a first monitoring surface and a second monitoring surface in the image;
A direction setting unit for setting a reverse running direction or a forward direction in the image,
An object that has contacted the first or second monitoring surface is set as a target object, and a time difference between when the target object first contacts the first monitoring surface and when the target object first contacts the second monitoring surface. A monitoring unit that determines the moving direction of the object of interest using
An alarm unit that issues an alarm when the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction;
A monitoring support device having: An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
An alarm unit that issues an alarm when the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction;
A monitoring support device having: An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
The object of interest on the first monitoring surface of the first image and the object of interest on the second monitoring surface of the second image are merged to generate an integrated object of interest, and the integrated object of interest is virtually monitored. A projection unit that projects onto a surface,
A determining unit that determines whether the area of the integrated target object on the virtual monitoring surface exceeds a predetermined threshold,
If the moving direction of the object of interest is the reverse running direction or the reverse of the forward direction, and the area of the integrated object of interest on the monitoring surface exceeds a predetermined threshold, an alarm is issued. An alarm unit that issues
A monitoring support device having: An acquisition unit configured to acquire a first image photographed by the first stereo camera and a second image photographed by the second stereo camera;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image at the same spatial coordinates as the first monitoring surface;
A direction setting unit that sets a reverse running direction or a forward direction in the first and second images;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
The object of interest on the first monitoring surface of the first image and the object of interest on the second monitoring surface of the second image are merged to generate an integrated object of interest, and the integrated object of interest is virtually monitored. A projection unit that projects onto a surface,
A determination unit configured to determine whether an area (size) of the integrated target object on the virtual monitoring surface is greater than or equal to a first threshold and less than or equal to a second threshold, or whether or not exceeds a second threshold; ,
When the moving direction of the object of interest is the reverse running direction or the reverse direction of the forward direction, and the area of the integrated object of interest on the monitoring surface exceeds the first threshold value and moves to the second direction. If the value is equal to or less than the threshold, an alarm is issued together with the information that the object of interest is a vehicle of the first size (for example, a motorcycle, a small vehicle, or a passenger car), or the moving direction of the object of interest is reversed. In the case of the running direction or the reverse of the forward direction and exceeding the second threshold, the target object is a vehicle of the second size (for example, a truck, a bus, or a large vehicle). ), An alarm unit that issues an alarm,
A monitoring support device having: An acquisition unit configured to acquire a first image captured by a first stereo camera at a first location and a second image captured by a second stereo camera at a second location;
A monitoring surface setting unit configured to set a first monitoring surface in the first image and a second monitoring surface in the second image;
An object that has contacted the first or second monitoring surface is set as an object of interest, and the spatial coordinates of the object of interest and the object of interest in at least one frame before or after the image are compared to determine the object of interest. A monitoring unit for determining a moving direction;
When the moving direction of the object of interest is a direction between the first point and the second point, the number of the object of interest is incremented by one, and the moving direction of the object of interest is changed to the first point. A measurement unit that subtracts 1 from the number of the target objects and counts the number of target objects between the first point and the second point in a case where the direction is from the second point.
A monitoring support device having:   A monitoring support program that causes one or a plurality of arithmetic processing devices to function as the monitoring support device according to claim 1.   A computer-readable storage medium storing the monitoring support program according to claim 12.

Images