KR101970442B1 - Illegal parking enforcement system Using Fast R-CNN based on Vehicle detection - Google Patents

Abstract

The present invention relates to an illegal parking intervention system using Fast R-CNN based vehicle detection. According to an aspect of the present invention, there is provided an illegal driving method of an illegal driving vehicle, comprising: a first step of acquiring an image through a camera; A second step of dividing the entire region of the image into a near region and a far region on the basis of a predetermined reference line in the image; A third step of dividing the image into a plurality of predetermined blocks; A fourth step of deriving a feature map to which the plurality of block units are applied by using at least one of a plurality of convolution layers of a Convolution Neural Network (CNN); A fifth step of determining whether the object included in the image is a vehicle using the feature map; And if the object is a vehicle, recognizing the vehicle number from the vehicle number area of the vehicle. If the object is included in the remote area, And if the object is included in the near region, deriving a feature map for the region containing the object using only a predetermined number of layers among the plurality of convolution layers can do.

Description

[0001] The present invention relates to an illegal parking enforcement system using Fast R-CNN based vehicle detection,

The present invention relates to an illegal parking intervention system using Fast R-CNN based vehicle detection.

In spite of the rapid growth of demand for automobiles due to economic growth and income increase, the manpower to manage the road situation and traffic situation is insufficient. Therefore, efforts are being made to overcome the existing poor traffic management system with limited personnel.

As a part of this effort, the development of a system for automatic recognition (including number recognition) of vehicles is actively under way. Through the recognition of the vehicle or the recognition of the number (character) of the vehicle, it can be used effectively in various fields such as traffic control, traffic volume investigation, arresting of the stolen vehicle, control of access vehicles, parking facility management, and so on.

In particular, the processes for recognizing characters on the license plate differ from those of other characters, and therefore, robust processing methods must be considered because distortion occurs due to camera noise, illumination change, weather, and the like due to environmental influences. However, in the case of a license plate area of a vehicle, its contents are limited due to its inherent characteristics, and its structure is simpler than general character recognition (pattern recognition). For this reason, License Plate Recognition (LPR) is the most common system for efficiently managing environmental characteristics, increasing demand for vehicles, manpower supply and management of parking space resources.

The LPR system or number recognition technology was first developed in the UK in 1976. Over the following decades, as technology evolved and market demand grew, LPR systems grew steadily and expanded steadily in Southeast Asia and other European countries. As a result, the LPR system market is growing significantly in North America. This led to strong motivation for effective crime prevention and prevention technologies, which enabled them to become more active in the wider market.

Previous Vehicle Number Identification (LPR) or Automated LPR (ALPR) systems use the Surveillance Method using Optical Character Recognition (OCR) in images (images) obtained from cameras to read vehicle license plates. . In recent years, the name of the parking management system has been working efficiently for parking spaces. Currently, LPR system is solving manpower supply, labor cost burden, and charge leakage problem by adjusting the parking charge in relation to the parking environment. As the demand of LPR system is constantly increasing, technological change and development are continuing.

In addition, CCTV (closed circuit television) has been installed in various places in the world because of public safety and personal security, and data currently being recorded through CCTV is used as evidence for crime prevention and crime have.

Korean Intellectual Property Office Registration No. 10-1565978

SUMMARY OF THE INVENTION The present invention provides a user with an illegal parking regulation system using Fast R-CNN based vehicle detection.

The present invention relates to an effective method for detecting a target vehicle in an illegal parking intermittent system, and relates to a problem of calculation of a bounding box in Max-Pooling according to size, position and distance difference of object detection in existing CNN (Convolution Neural Network) environment .

Specifically, in the case of most prior arts, in the case of an illegally parked vehicle, a target object is detected in a patterned form in order to analyze a feature component, whereas in the present invention, the same object change By setting the feature vector distribution to be different according to the distance, it can be processed effectively compared with the existing method.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are not intended to limit the invention to the precise form disclosed. It can be understood.

According to an aspect of the present invention, there is provided an illegal driving method for an illegal driving vehicle, comprising: a first step of acquiring an image through a camera; A second step of dividing the entire region of the image into a near region and a far region on the basis of a predetermined reference line in the image; A third step of dividing the image into a plurality of predetermined blocks; A fourth step of deriving a feature map to which the plurality of block units are applied by using at least one of a plurality of convolution layers of a Convolution Neural Network (CNN); A fifth step of determining whether the object included in the image is a vehicle using the feature map; And if the object is a vehicle, recognizing the vehicle number from the vehicle number area of the vehicle. If the object is included in the remote area, And if the object is included in the near region, deriving a feature map for the region containing the object using only a predetermined number of layers among the plurality of convolution layers can do.

The method may further include a second step of, when the object is included in the remote area in the second step, converting the scale of the remote area between the second step and the third step .

In addition, the fifth step may include a step 5-1 of comparing the feature maps related to the object for each of the plurality of block units. A 5-2 step of determining a max pooling area in which the object is most frequently present among the plurality of block units; And a fifth step of determining whether the object is a vehicle using the feature value of the vehicle based on the maximum pulling area.

In addition, the step 5-3 may further include: 5-4 setting a region of interest (ROI) based on the max pooling region; A fifth step of setting a bounding box based on the set ROI; And comparing the feature value of the object in the bounding box with the feature value of the vehicle to determine whether the object is the vehicle.

According to another aspect of the present invention, there is provided an illegal parking regulation system using Fast R-CNN based vehicle detection, comprising: a camera for acquiring an image; And a control unit for dividing the entire region of the image into a near region and a far region on the basis of a predetermined reference line in the image and dividing the image into a plurality of predetermined block units, The method comprising: deriving a feature map to which the plurality of block units are applied by using at least one of a plurality of convolution layers of a neural network (CNN) The method comprising the steps of: determining whether the object is a vehicle, recognizing a vehicle number from the vehicle number area of the vehicle when the object is a vehicle, and, if the object is included in the remote area, Deriving a feature map for an area containing an object, and when the object is included in the close range, Using only a layer of a predetermined number of layers we can obtain the characteristic map for the area including the object.

Also, when the object is included in the remote area, the control unit may convert the scale of the remote area.

Also, the controller compares the feature map associated with the object for each of the plurality of block units, determines a max pooling area in which the object is most frequently present among the plurality of block units, It is possible to determine whether the object is a vehicle by using the feature value of the vehicle based on the area.

The controller may set a region of interest (ROI) based on the max pooling region, set a bounding box based on the set ROI, It is possible to determine whether the object is the vehicle by comparing feature values of the vehicle.

The present invention can provide the user with an illegal parking regulation system using Fast R-CNN based vehicle detection.

The present invention relates to an effective method for detecting a target vehicle in an illegal parking intermittent system, and relates to a problem of calculation of a bounding box in Max-Pooling according to size, position and distance difference of object detection in existing CNN (Convolution Neural Network) environment Can be solved.

Specifically, in the case of most prior arts, in the case of an illegally parked vehicle, a target object is detected in a patterned form in order to analyze a feature component, whereas in the present invention, the same object change By setting the feature vector distribution to be different according to the distance, it can be processed effectively compared with the existing method.

It should be understood, however, that the effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those skilled in the art to which the present invention belongs It will be possible.

FIG. 1 shows an example of a block diagram of an illegal parking intermittent system that can be applied to the present invention.
Fig. 2 shows an example of a block diagram of a control section that can be applied to the present invention.
FIGS. 3A and 3B are a plan view and a side view of a structure in which an illegal parking intermittent system related to an example of the present invention is installed.
4 shows an example of a neural network structure related to the present invention.
Fig. 5 shows an example of a convolutional neural network related to the present invention.
Figure 6 illustrates a system flow diagram based on Improved F-CNNs proposed by the present invention.
FIG. 7 shows an example of object detection based on Fast R-CNN in the context of the present invention.
FIG. 8 shows an example of setting a block unit in an image according to the present invention.
Fig. 9 shows an example of the AlexNet structure applied to the present invention.
10 shows an example of a structure for area size fixing in a Faster R-CNN structure in connection with the present invention.
11 shows an example of the fixed area output value in the context of the present invention.
Fig. 12 shows an example of a convolution layer structure for spatial location area detection in an image, in the context of the present invention.
Fig. 13 shows an example of characteristic vector components in a space for detecting an object region, in the context of the present invention.
FIG. 14 shows a flowchart of an object detection step applied to the present invention.
FIG. 15 shows experimental results, FIG. 16 shows experimental results according to object loss, and FIG. 17 shows experimental results showing improved processing results.

Illegal parking regulation system

Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In addition, the embodiment described below does not unduly limit the contents of the present invention described in the claims, and the entire configuration described in this embodiment is not necessarily essential as the solution means of the present invention.

Conventionally, in order to prevent traffic congestion and congestion caused by illegal parking vehicles, a surveillance camera was installed in a prohibited area to control illegal parking vehicles.

However, there has been a problem that an object other than a vehicle existing in the vicinity of an emergency stop zone is included in an image photographed by a detection camera, and thus, a problem of erroneous detection of an object other than the vehicle frequently occurs. In addition, there is a problem that it is difficult to control the vehicle that is parked far away from the detection camera.

Hereinafter, the illegal parking regulation system proposed by the present invention will be described in detail.

FIG. 1 shows an example of a block diagram of an illegal parking intermittent system that can be applied to the present invention.

The illegal driving control system 10 includes a camera module 100, an output unit 200, a communication unit 300, a memory 400, an interface unit 500, a power supply unit 600, a control unit 700, can do.

However, the components shown in FIG. 1 are not essential, and an illegal preliminary intermittent system having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The camera module 100 processes an image frame such as a still image or a moving image obtained by photographing a parking space prohibited area. The image frame processed by the camera module 100 may be stored in the memory 400 or may be transmitted to the outside through the communication unit 300. The processed image frame may be displayed on the display unit 210. [

At this time, two or more cameras used in the camera module 100 may be provided, depending on the use environment.

For example, the camera module 100 may include a detection camera 110, an intermittent camera 120, and the like. However, the present invention is not limited to such a configuration, and may include a configuration including other shooting means.

The detection camera 110 photographs the prohibited area and can detect whether or not the vehicle enters the prohibited area using the sensed image.

The detection camera 110 may be implemented using a camera equipped with a fisheye lens. In the case of using a fisheye lens having a wide angle of view, it is possible to take an image of an omnidirectional (360) region around the detection camera 110 and utilize it as a detection image.

However, even if the detection camera 110 is installed, it is not possible to enlarge and view a specific point of the pseudo range-prohibited area, and therefore, an intermittent camera 120 capable of performing a pan, tilt, and zoom operation is needed.

The intermittent camera 120 may perform at least one of a pan operation, a tilt operation, and a zoom operation to photograph a license plate of a vehicle illegally parked in the prohibited area. The intermittent camera 120 may preferably use an LPR (License Plate Recognition) camera.

The LPR camera is provided with a tilting device capable of tilting in the x-axis, the y-axis and the z-axis direction, respectively. Such a configuration is easy for those having ordinary skill in the art to which the present invention belongs. . Such an LPR camera has a configuration capable of photographing a zoom-in image or a zoom-out image with respect to the vehicle by rotating the camera at designated x, y coordinates.

Meanwhile, the output unit 200 generates an output related to a visual or auditory sense, and may include a display unit 210, a warning unit 220, and the like.

The display unit 210 can display (output) the information processed in the illegal parking intermittent system 10. For example, a UI (User Interface) or a GUI (Graphic User Interface) associated with the illegal parking intermittent system 10 is displayed. A still image or moving image photographed by the camera module 100, a video image transmitted / received through the communication unit 300, a UI, a GUI, and the like.

In addition, the display unit 210 may be used as an illumination light for interrupting the illegally parked vehicle at night or in a dark space, or for controlling the crime prevention area when the illegal parking intermittent system 10 is used for crime prevention.

The display unit 210 may be a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) A flexible display, and a three-dimensional display (3D display).

The warning unit 220 outputs a signal for notifying the occurrence of the event of the illegal parking system control system 10. Examples of events that occur in the illegal parking control system (10) include illegal parking of vehicles in the prohibited parking area.

The warning unit 220 may output a form other than an audio signal or a video signal, for example, a warning notice document. In this case, a printer for outputting a warning notice document may be connected.

The video signal may be output through the display unit 210. In this case, the display unit 210 may be classified as a kind of the warning unit 220. [

The communication unit 300 may include one or more modules that enable communication between the illegal parking intermittent system 10 and the wire / wireless communication system or between the illegal parking intermittent system 10 and the network in which the illegal parking intermittent system 10 is located. . ≪ / RTI > For example, the communication unit 300 may include a wired / wireless Internet module 310, a short distance communication module 320, a location information module 330, and the like.

The wired / wireless Internet module 310 refers to a module for wired / wireless Internet access, and may be built in or enclosed in an illegal parking system.

WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), WCDMA (Wideband Code Division Multiple Access), HSDPA (High Speed Downlink Packet Access) , LTE (Long Term Evolution), or the like can be used.

The short-range communication module 320 is a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as the technique of the short range communication.

The position information module 330 is a module for obtaining the position of the illegal parking system 10, and a representative example thereof is a global positioning system (GPS) module. According to the present technology, the GPS module calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain three-dimensional current position information according to latitude, longitude, and altitude It can be calculated accurately. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module can calculate speed information by continuously calculating the current position in real time.

Meanwhile, the memory 400 may store a program for processing and controlling the control unit 700, and may store a program for temporarily storing input / output data (e.g., feature vector information such as still images and moving images) Function may be performed. The memory 400 may also store the frequency of use of each of the data (for example, frequency of use for each feature vector information).

The memory 400 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.) ), A random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- A magnetic disk, an optical disk, a memory, a magnetic disk, or an optical disk. The illegal parking intermittent system 10 may operate in connection with a web storage that performs the storage function of the memory 400 on the Internet.

On the other hand, the interface unit 500 serves as a pathway to all the external devices connected to the illegal parking intermittent system 10. The interface unit 500 receives data from an external device or receives power from the external device and transmits the data to each component in the illegal parking intermittent system 10 or transmits data in the illegal intermittent intermittent intermittent system 10 to an external device do.

For example, a port for connecting an image or an image photographed by the camera module 100 to an external device, an external charger port, a wired / wireless data port, a memory card port, An audio I / O port, a video I / O port, and the like may be included in the interface unit 500.

Meanwhile, the power supply unit 600 receives external power and internal power under the control of the controller 700, and supplies power required for operation of the respective components.

On the other hand, the controller (700) typically controls the overall operation of the illegal parking system (10). For example, related control and processing are performed for video shooting, detection of illegally parked vehicle, and creation of intervention data.

For example, the control unit 700 receives the image photographed by the detection camera 110, and can determine whether to intercept the vehicle when the vehicle is parked in the prohibited area. In addition, when the illuminance of the diaphragm-prohibited area is not uniform, the control unit 700 detects the illuminance and adjusts the light intensity of the display unit 210 toward the bright area to be weak and adjusts the light intensity of the display unit 210 Can be strongly controlled.

The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of a processor, controllers, micro-controllers, microprocessors, and other electronic units for performing other functions. In some cases, The embodiments described may be implemented by the control unit 700 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in a software application written in a suitable programming language. The software code is stored in the memory 400 and can be executed by the control unit 700. [

Meanwhile, FIG. 2 shows an example of a block diagram of a control unit that can be applied to the present invention.

2, the control unit 700 which can be applied to an example of the invention includes a vehicle recognition unit 710, an illegal parking distance determination unit 720, a license plate recognition unit 730, a filtering unit 740, Means 750, memory control means 760, and the like. However, the configuration of the controller 700 is not limited thereto, and it is possible to implement the configuration with more or fewer components.

First, the vehicle recognition unit 710 receives the detection image for the prohibited area for the emergency stop camera 110, and can analyze the feature vector information from the detection image using the feature vector.

The characteristic vector is a method of extracting and analyzing a feature vector for recognizing a vehicle to be interrupted by the illegal parking regulation system 10. The feature vector is classified into a Scale-Invariant Feature Transform (SIFT), a Speed Up Robust Feature (SURF) Block Matching Algorithm, Histogram of Oriented Gradients, Haar-like features, and Gabor Wavelet (filter).

Here, the SIFT algorithm is a typical algorithm for extracting a feature vector of an image, and has characteristics that are invariant to image rotation, scaling, movement, partial illumination change, and projective transform. The SIFT algorithm can extract attributes such as position, scale, and direction of a feature in consideration of the characteristics of the local image.

That is, the first step is to select the sample points (or candidate pixels) by searching the maximum and minimum (extrema) in the scale space generated by the DOG (Difference-Of-Gaussian) function (Scale-space extreme detection).

In the second step, keypoints are selected based on the stability value (Keypoint localization).

In the third step, one or more directions are assigned to each keypoint (Orientation assignment).

As a final step, a keypoint descriptor is created using local image gradients.

A detailed description of the SIFT algorithm is given in detail in the article " Distinctive image features from scale-invariant keypoints ".

In addition, the SURF algorithm is an algorithm that finds characteristics that are invariant to environmental changes by taking into account environment changes such as size, illumination, and viewpoint from multiple images. It is similar to the SIFT algorithm, which is generally known to have excellent performance, It is a greatly improved algorithm.

A detailed description of the SURF algorithm is given in detail in the " Robust Interest Point Detection and Descriptor " paper.

In addition, the HOG algorithm is based on the fact that the appearance and shape of a specific object in an image can be determined by the gradient magnitude or the distribution in the corner direction.

The HOG algorithm divides an image into a plurality of regions called cells and can recognize a specific object by creating a histogram of a slope or a corner direction for each cell.

A detailed description of the HOG algorithm is given in detail in the article " Histograms of oriented gradients for human detection ".

In addition, the Haar-like features algorithm can recognize a specific object by using a prototype representing the characteristic information of the edge and a prototype representing the characteristic information of the line. A detailed description is given in the article "An Extended Set of Haar-like Features for Rapid Object Detection".

A detailed description of the Gabor Wavelet (filter) algorithm is given in detail in "Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters."

Further, a detailed description of the BMA algorithm is given in detail in " Combined shape and feature-based video analysis and its application to non-regid object tracking ".

The feature vector information extracted by the feature vector is compared with the learning data stored in the memory 400.

Since the learning data includes information of vector components of a feature vector of a general vehicle image, the vehicle recognizing unit 710 can determine whether there is a vehicle among the objects photographed in the detected image, have.

Also, in the process of recognizing the vehicle from the detected image, the linear nearest point search can be used, which is a method of determining the number of feature vectors with high similarity among the compared feature vectors. However, since the linear nearest point search must be compared with all the learning data stored in the memory 400, the calculation amount is too large and the calculation speed may be slow. Therefore, a method of creating an index related to the feature vector information in the learning data to speed up the search may be used.

Further, the vehicle recognizing means 710 can determine whether the vehicle enters the prohibited area and whether the vehicle has stopped in the prohibited area.

For this purpose, a motion detection algorithm for detecting the motion of the vehicle or a video image detection algorithm for comparing the background image of the prohibited area with the parking time image can be used. Preferably, both algorithms can be used for more accurate vehicle detection have.

On the other hand, the illegal parking discrimination unit 720 can judge whether or not the vehicle is parked on the parking prohibited area.

That is, when the vehicle recognizing unit 710 receives the detection image photographed by the detection camera 110 and recognizes the vehicle in the prohibited area, the illegal parking distance judging unit 720 judges whether or not the vehicle is illegally parked It is possible to decide whether or not to perform intermittent processing.

It can not be concluded that the vehicle was illegally parked only on the basis of the recognition of the vehicle on the detection image, and it can be said that the vehicle was illegally parked only when the vehicle stopped in the prohibited parking area for a predetermined period of time.

Therefore, the illegal parking distance determining means 720 can determine whether the vehicle is illegally parked by using the sensing images photographed at predetermined intervals of time.

That is, when the illegal parking distance determination unit 720 recognizes the presence of the vehicle in the parking space prohibited area at the beginning, the detection camera 110 controls to shoot the parking space prohibited area again after a predetermined time elapses. For this, the camera control unit 750 can control the operation of the detection camera 110 at the same time.

The vehicle recognizing means 710 receives the sensed image photographed after the lapse of the predetermined time, analyzes the feature vector again, and extracts the feature vector information from the sensed image.

The extracted feature vector is compared with the learning data stored in the memory 400. The vehicle recognition unit 710 can determine whether there is a vehicle among the objects photographed in the detection image photographed after a predetermined time elapses, Can be recognized.

When the illegal parking distance determining means 720 determines that the vehicle recognized in the detection image after the lapse of the preset time is the same as the recognized vehicle in the initial detection image, the vehicle is regarded as having made illegal parking.

On the other hand, when it is judged that the vehicle recognized in the detection image after the lapse of the predetermined time and the vehicle recognized in the initial detection image are not the same, the vehicle is recognized as having not illegally parked the vehicle, Photograph the prohibited area and search for entry of the vehicle.

On the other hand, the license plate recognizing means 730 can recognize the characters of the license plate of the illegally parked vehicle by receiving the image photographed by the intermittent camera 120.

For character recognition, one of the three detection methods of the license plate position can be executed first.

First, the feature region of the license plate is detected using the vertical and horizontal edge information from the photographed image.

The second is to detect the position of the license plate by the scan data analysis.

The third is to detect the exact license plates by directly searching for numbers and letters.

When the position of the license plate is detected, the recognition algorithm uses the numbers, letters (consonants, vowels, and vowels) to recognize the characters by template matching (Hangul consonant + number) ) Are classified in detail and the recognized characters are re-confirmed, thereby minimizing the error in decoding the characters.

On the other hand, the filtering means 740 can decide whether or not to exclude from the object of interception in consideration of the size of the vehicle captured on the detection image.

In the case of photographing the detection image for the prohibited area using the detection camera 110, the size of the vehicle photographed on the photographed detection image varies depending on the distance from the detection camera 110 to the actual vehicle position.

That is, since the lower end portion of the detection image is a space near the support where the illegal parking regulation system 10 is installed, if the actual vehicle is located close to the detection camera 110, the vehicle captured on the detection image is positioned at the lower end portion , And the size of the vehicle on the detection image will be large.

On the other hand, if the actual vehicle is located far from the detection camera 110, the vehicle captured on the detection image will be positioned at the upper part of the detection image, and the size of the vehicle captured on the detection image will be smaller.

In this way, the filtering means 740 can calculate the distance from the detection camera 110 to the position of the actual vehicle using the position of the detected image of the vehicle recognized by the vehicle recognition means 710. [

Further, the filtering means 740 can calculate the expected vehicle image size in the detected image using the calculated distance, and the size of the vehicle image on the detected image is much different from the expected size of the vehicle image It is possible to prevent misjudgment by excluding it from the object of interception.

That is, it is calculated that the vehicle recognized by the vehicle recognition means 710 is located at the lower end portion of the detection image and is located close to the detection camera 110. If the size of the vehicle image on the detection image is small, The vehicle is not actually a vehicle and is likely to be an error, so it can be excluded from the control.

Conversely, if it is calculated that the vehicle recognized by the vehicle recognition means 710 is located at the upper portion of the detection image and is located far from the detection camera 110, if the size of the vehicle image on the detection image is large, The vehicle is not actually a vehicle but is likely to be an error, so it can also be excluded from the control.

Meanwhile, the camera control unit 750 can control the overall operation of the detection camera 110 and the intermittent camera 120 belonging to the camera module 100.

The camera control unit 750 can control the detection camera 110 to photograph the detection image of the illegal parking prohibited area so that the illegal parking intermittent system 10 interrupts the vehicle stopping in the illegal parking prohibited area.

In addition, the camera control means 750 can control to shoot a detection image for the forbidden zone again after a predetermined time so as to judge whether or not the vehicle stopped in the prohibited area is on the prohibited area have. This can be controlled together with the illegal parking discrimination means 720.

The camera control unit 750 receives the positional information of the stopped vehicle determined by the vehicle recognition unit 710 and can photograph the license plate of the vehicle stopped by the intermittent camera 120 in the prohibited area Tilt, and zoom operations of the intermittent camera 120 so that the intermittent camera 120 may be controlled by the intermittent operation.

On the other hand, the memory control means 760 can control the overall operation of various data input / output to / from the memory 400.

The vehicle recognition unit 710 of the control unit 700 extracts a feature vector from the detected image and compares the extracted feature vector with feature vector information of a general vehicle image of the memory 400. [

For this, the memory control means 760 can control the memory 400 to output the learning data including the feature vector information about the general vehicle image to the vehicle recognition means 710. [

If the feature vector information for the vehicle recognized by the vehicle recognition means 710 and / or the filtering means 750 does not exist in the learning data of the memory 400, it can be used for comparison with the next learning data The feature vector information may be stored in the memory 400.

To this end, the memory control means 760 may control the feature vector information for the vehicle recognized by the vehicle recognition means 710 and / or the filtering means 750 to be input to the memory 400. [

Hereinafter, an installation structure of an illegal parking regulation system having the above-described configuration will be described with reference to the drawings.

FIGS. 3A and 3B are a plan view and a side view of a structure in which an illegal parking intermittent system related to an example of the present invention is installed.

3A and 3B, the illegal parking system system 10 of the present invention may include a support 12, a crossbar 14, a detection camera 110, an intermittent camera 120, and the like.

However, the constituent elements shown in Figs. 3A and 3B are not essential, and an illegal preliminary intermittent interrupting system having more or fewer components may be constructed. Hereinafter, the installation structure of FIGS. 3A and 3B will be described.

It is preferable that the column 12 is fixedly installed on the ground within the prohibited area or near the ground, and is fixed to the ground by a bolt or inserted and fixed in the ground. The post 12 can be installed at a sufficient height to allow the intermittent camera 120 to take a wide area.

The support 12 may be formed hollow so that electric wires connected to the detector camera 110, the intermittent camera 120, the output unit 200, the communication unit 300, And the material thereof is preferably made of a metal material which is not corroded by rainwater or the like.

At least one cross bar 14 may be connected to the upper part of the column 12 and the inside of the column 12 may be hollow as in the case of the column 12, .

The crossbar 14 may be positioned horizontally with the ground, preferably above the anti-vehicular zone.

The detection camera 110 and the intermittent camera 120 can process an image such as a still image or a moving image obtained by photographing the prohibited area. The detection camera 110 and the intermittent camera 120 may be installed at the end of the crossbar 14, and preferably disposed on a prohibited area for intermission.

As shown in FIG. 3A, in the illegal parking intermittent system 10 in which the illegal parking intermittent system 10 is installed, a car 21 parked near the detection camera 110 and a vehicle 22 parked away from the detection camera 110, This can be.

As described above, when the detection camera 110 is used to photograph a detection image for a pseudo range-prohibited area, the vehicle 21 that has been parked near the detection camera 110 is positioned at the lower end of the detection image.

Therefore, the filtering unit 740 can calculate the distance from the detection camera 110 to the vehicle 21 using the position of the vehicle on the detection image.

The distance between the detection camera 110 and the vehicle 22 can be calculated from the top of the detection image.

Illegal parking control system using Fast R-CNN based vehicle detection

CCTV (closed circuit television) has been installed and is still in progress in modern times as a reason to be threatened by public safety, facility and personal safety, and convenience of living.

In this background, CCTV intelligent video security plays a role of video surveillance, disaster prevention and disaster prevention through object detection and behavior analysis of images.

Existing object detection methods and methods are based on background modeling, and they are detected by differences of foreground and background. Extracting features using HoG, LBP, and Color Histogram are detected through machine learning such as Adaboost, SVM and ASEF Filter. .

Then object tracking uses several machine learning techniques such as Basian tracking, Particle filtering, Mean-shift, and Cam-Shift.

Fig. 4 shows an example of a neural network structure.

As shown in FIG. 4, the recent Deep Learning image recognition technology has been developed on the basis of CNN and has been developed to a fast and high detection rate such as YOLO through RCNN, Fast R-CNN, and Faster R-CNN .

Such object detection and tracking (analysis) algorithms have been steadily developed and are being developed to replace human cognitive abilities, and are being utilized and applied to the entire industry including unmanned vehicles.

It is developed and applied to enhance the recognition rate and improve the completeness of image security by applying the CNN based Deep Learning algorithm out of the existing method in the CCTV image.

With the advances of these technologies, there is a tendency to be introduced into application services for vehicles, and it is used to recognize contents of video, natural language, voice, control, and communication in various systems ranging from mobile apps to autonomous vehicles , Complex data and various data types.

Especially, it is a tendency to increase the accuracy of vehicle detection in various industrial fields such as illegal parking control, bus control, parking control, and autonomous driving.

Detectors corresponding to the vehicle detection unit in the existing illegal parking intermittent system extract feature information and select a method of detecting the vehicle according to a threshold value set as a reference, or use a difference between a background component and a component containing vehicle information And a method of detecting a vehicle.

Such a method may be an erroneous detection in the form of a target other than a vehicle depending on whether the feature information is lost or not (a person or a similar object) It is difficult to detect vehicle information of a desired object because it is vulnerable to change of environment by weather and time frame.

Therefore, unlike existing methods for vehicle detection, in this development, we apply detection method to detect features of vehicle using method based on machine learning learning such as deep learning and apply various learning patterns to improve accuracy. It detects complex and various types of vehicles in road environment.

Deeplearning methods defined in most general terms are applied differently according to the application technique in the basic structure and the accuracy of detection is different according to the size of the learning data structure (layout) There are drawbacks that can appear effectively. Even though the content of the technique according to the application method is different, it is commonly referred to as deep run due to the configuration in the same system structure.

Deep learning is a kind of machine learning. Deep learning is used in various fields such as image recognition, speech recognition, and natural language processing. Unlike the method of extracting the existing vehicle feature information, in the deep learning, "feature extraction" is used. In the existing method, a person has to designate each feature in the image, but in the case of the deep learning, Learning is a big difference.

This feature can lead to learning of feature extraction since it has the meaning of the network structure of the past neural network (NN: Neural network), which means that there are many neurons (neurons) in the human brain and one neuron It is connected to the flow of signals that receive or transmit signals to the neurons, allowing them to create a variety of information.

The neural network (NN: Neural network) is a part that is reminiscent of this structure.

The structure shown in FIG. 4 is referred to as a network because a plurality of neurons of a neural network are connected to each other. When input data is input to the input layer, various inputs are received from the structure of the hidden layer, (Unit, connection strength) according to the number of nodes and output from the output layer.

When three or more neural networks are superimposed, they are called Deep Neural Network (DNN), and machine learning using them is called deep learning.

In the structure that learns by inputting a large amount of data, it can be designed by maximizing the efficiency and effect of learning because it adjusts the weight of the neuron and is related to the signal intensity of the output according to the operation such as weight sum or product of connection strength .

Such a structure requires modification to process vast amounts of data, and even when the amount of learning data is enormous, a more complex structure is possible. Convolutional Neural Network (CNN), which is one of the features of this structure, is an omnidirectional neural network that constitutes a connection pattern of neurons and is designed to perform processing between layers and is composed of several convolution layers do.

The structure of CNN is suitable for 2D input data and can be applied and applied to video field such as CCTV camera.

Fig. 5 shows an example of a convolutional neural network applied to the present invention.

FIG. 5 is a CNN structure similar to the conventional neural network structure. However, since it extracts meaningful features through convolution and many layers are stacked, feature extraction, dimensionality reduction through sub-sampling, And feature classification from the lower layer.

In this complex structure, each layer has a learning process, and it has a process of obtaining an answer to a new input based on the learned contents, and provides various learning models by setting learning parameters (weights) Can be applied.

For example, there are models such as a Feedforward Multilayer Perceptron (MLP) and a Convolutional Deep Belief Network (CDBN).

These models are used instead of detecting objects in images (images), and the similarity between learned data and input data is calculated, analyzed and classified.

In this development, we need to use a learning model that targets detection or detection methods rather than classification methods for vehicle information extraction. Therefore, we use the RCNN (Region-based Convolution Networks) approach Go ahead.

In this case, in the case of R-CNN, an area is designated based on the object to be searched in advance, and the object is detected based on the area.

However, when the number of target objects is plural, it is difficult to guarantee the real-time property because the cost is high according to the processing time.

In addition, a plurality of target vehicles exist in an image (image) for extracting information of a vehicle, and in some cases, there are objects (package box, two-wheeled vehicle, bicycle, cart, etc.) Although accuracy within the area is important, there are some systems that need to reflect processing speed such as Fast RCNN (Fast RCNN) in order to secure real-time performance.

In general, the R-CNN and Fast RCNN approach can improve the accuracy and processing speed to detect (detect) the objects in the image (image). On the other hand, in multi-channel streaming processing Especially, since there are many changes in the accuracy performance depending on the distance between the object and the object in the camera position and the processing speed according to the image (image) size is measured differently, need.

Therefore, in this development, it is necessary to secure the system configuration based on the multi-channel camera based on the real-time processing in order to request the high accuracy along with the processing speed. Respectively. Especially, for vehicle detection, selective decision for a specific layer layer, accuracy and identification in a position interval according to distance, and vehicle recognition through number recognition are performed.

Figure 6 illustrates a system flow diagram based on Improved F-CNNs proposed by the present invention.

Referring to FIG. 6, since convolutional-based features in CNN are generally more complicated and sophisticated and abstract information than Haar-like-based features that only analyze the difference between horizontal and vertical blocks, It is useful at processing speeds than Haar-like features obtained with a small number of search kernels based on one location.

However, the obtained features contain only information.

CNN, on the other hand, can use various search kernels and get features, and have features that are complex and elaborate information, though the processing speed is slower than Haar-like features.

This feature makes the accuracy of the Haar-like feature high by using multiple layers of convolutional layers.

However, in order to solve this problem, the CNN requires a modification to the convolutional layer, and the ratio (size) or distance (the distance from the camera to the target object Distance). Therefore, only the layer layer which can be divided into the size of the vehicle can be selected without using the entire layer layer and developed as a processing speed and an effective detection method.

Each step in Fig. 6 will be described in detail below.

First, the input and image calibration steps S10 and S11 will be described.

The input of the image is digitized data acquired from the camera. A moving image is a target, and each frame (image) is composed of two-dimensional data.

In addition, the image correction specifies the detection area between the input coordinates to detect the object (vehicle) of interest in advance.

At this time, the region within the region is redefined to be divided into two regions, and the object (vehicle) detection region is performed for each region.

The two areas are set to distinguish between distant and near.

The reason for this is that even if the same object (vehicle) exists in the image, the size of the object (vehicle) is different from each other, so that the texture of the object (vehicle) Information is displayed so that the processing result input data is reliable.

In addition, when the first area is detected from two areas, that is, a near object (vehicle) is detected, and the object (vehicle) has an object area having a large size because the distance from the camera is short.

Also, since the object (vehicle) is distant from the camera and there are small object areas, the object (vehicle) of interest is detected because the object is located in the second area among the two areas, Since the size difference (distribution) between the unspecified objects becomes similar to each other, it is set as an area by zooming in / out of the camera or conversion of the scale in the camera setting (function) so that the feature information in the area can be easily identified.

Next, step S12 will be described.

S12 ESKRP is an optional layer within F-CNNs. It applies deep-learning technology to detect desired object (vehicle) as a step after input data and image correction setting.

In most cases, a range that utilizes deep learning sets categories within the scope of sensing and recognizing objects and defines them to calculate values and distributions for the conditions.

In this system, a CNN-based detector is used to process a target object (vehicle) and a non-target object (human).

At this time, the learning data for the objects are used for learning with VOC Pascal 2007, VOC Pascal 2012, MS COCO, Caltech pedestrian dataset, ImageNet 2012, 2014, MIOTCD dataset.

The general case of Fast-CNN for improving processing speed based on CNN applied to this system is shown in FIG.

FIG. 7 shows an example of object detection based on Fast R-CNN in the context of the present invention.

Referring to FIG. 7, in order to detect an object (vehicle), a result is output in an area form from each layer based on CNN, and after reconstructing the values in a form of a fully-connected network, (Repositioning) based on the size and position of the target object (vehicle) in the area result for the layer layer.

In addition, the Fully-Connected layer is composed of a convolution layer and a pooling layer, and has a connect or weight structure according to the parameters. The Fully-Connected layer extracts features from the convolution and pooling layers, It is the process of judging which class (category) belongs.

At this time, the image of the input data is divided into small blocks and feature information is extracted from the blocks through each network in the CNN structure.

FIG. 8 shows an example of setting a block unit in an image according to the present invention.

As shown in FIG. 8, the reason for processing in the form of a block unit is to process the number of outputs in a fixed form due to its structural characteristics, and to output the position and size information of the object (feature information, object vehicle) This is because the position setting is easy.

7, a selective search method is used as a method of extracting a region from a feature map, and AlexNet is used as a feature extraction method.

In this case, it is used as feature vector for n outputs in fully connected layer. Classification is done using SVM (Support Vector Machine).

Meanwhile, FIG. 9 shows an example of the AlexNet structure applied to the present invention.

Referring to FIG. 9, the AlexNet structure is composed of a total of eight layers. The first to fifth layers are Convolutional layers and the next three layers are Fully connected layers.

Each parallel layer extracts and learns each independent feature for the image.

As a result, in order to extract the region, the size of the feature map is converted into the input data according to the sequence of the AlexNet feature vector and the classifier.

As a result, there is distortion due to size conversion in the form of block unit, but there is also loss in feature region due to input size and processing speed problem.

In addition, when the size of the input data is large or small, the position and size of the result area due to information lost in the process of extracting features for each layer in the feature map are affected, Feature information), it is important to select the size information.

In this step, the position and size information output according to the location information of the near object information or the remote object information may be determined differently.

In the present invention, the input data is largely divided into two areas in the process of being processed in the object feature map and the respective layers, and the input data are processed as inputs.

The reason why the input data is different is that it is possible to strictly distinguish between the accuracy and the processing speed in the case where the object features existing in the near and far ranges are differently processed and the form lost in the link structure between the layers in the AlexNet structure Minimize.

In addition, in the object feature extraction, in order to set the output position in accordance with the fixing of the position and size information of the detection area to the output number, the shape of the area and the classifier detected in each layer layer is changed into a single structure, The number is defined and displayed as the object detection area for the final output area.

10 shows an example of a structure for area size fixing in a Faster R-CNN structure in connection with the present invention.

FIG. 10 illustrates a method of improving the accuracy by calculating the output values of the respective size positions when the size values of the different positions exist in the object detection result.

The distribution of the output values for each of the regions and the output values represented by the regions in the object information are as shown in FIG. 11 below.

11 shows an example of the fixed area output value in the context of the present invention.

In Figures 10 and 11, in order to set the maximum number of output objects that can be represented at the corresponding positions in the n region values, the output signals are rearranged to have the set values for the center positions in the respective regions, The position of the final object area is selected.

For the same result as in FIG. 11, when an image is input as a pixel in a convolution layer, it is generally composed of a matrix NxNx3, a horizontal and a vertical, and a channel (dimension), and each channel is composed of R, G,

Also, when calculating the feature pattern for each channel in the convolution layer, it is calculated as a determinant expression, which is independent of the size.

FIG. 12 shows an object location and size including all object feature information as reconstructed regions for region extraction in the convolution layer.

The feature map for detecting the feature region is shown in FIG.

The detailed steps of the object (vehicle characteristic) detection are as shown in Fig. 14 by using the method of calculating the feature vector and improving the reconstruction structure of the position output according to the fixed size between the convolution layers in CNN.

FIG. 14 shows a flowchart of an object detection step applied to the present invention.

14, an input image is acquired (S21), a block unit is generated (S22), a Faster T-CNN is applied (S23), an ROI feature is calculated (S24), a bounding box is set S25), and performs object recognition (S26).

On the other hand, regarding the object detection in the illegal parking control, calculation of the pooling layer in the Fast R-CNN structure in the object detection step is determined according to the following equations (1) and (2).

Equation 1

Equation 2

로 정의한다.Equation 1 is an area of an object (object vehicle) for pooling calculation

.

In Equation (1), k-th is used to compare the learning data to the area of the block unit in the image.

At this time, the pooling result of the object (object vehicle) region is output to the pooling layer of j-th.

는 필터계수이고 이미지내의 블록단위에서 필터계수와의 컨볼루션을 의미하며 영역내의 후보영역을 찾고, 해당 높은 값을 계산하여 Max-pooling 영역을 계산한다.

Is a filter coefficient, which means convolution with a filter coefficient in a block unit in an image, finds a candidate region within the image, and calculates a corresponding high value to calculate a Max-pooling region.

In this case, although it is determined that the object of interest is included at least in the problem, it can be selected as an inaccurate region of the feature vector in the case of the problem of the background component region rather than the object region and the size of the interest object.

In order to correct such distorted (lost) information, the following equation (3) is used.

Equation 3

에 대해서 손실정보를

로 나타내며, 수학식 1에 의해

는 max-pooling 인텍스를 나타낸다. In Equation (3)

Loss information for

, And is expressed by Equation 1

Represents the max-pooling index.

This shows the position information found in each area after the convolution calculation.

는 미분에 의해서 계산되고 손실값을 나타낸다.

Is calculated by the derivative and represents the loss value.

에 누적된다.At this time, the region extraction result for all ROIs is

.

Through this iterative process, RoI (Region of Interest) pooling is performed on the RoI (Region of Interest) generated by the RPN (Region proposal network), and all feature maps for each Region are generated in the same fixed size . Through this, objects (objects) within each Region of Interest (RoI) are classified. Object classification is based on SVM (Support Vector Machine) and it is determined whether the object is a simple object or not.

Finally, for the object (object), calculate the bounding box for the final output and simply use linear regression.

The calculation of the bounding box is shown in Equation (4) and Equation (5).

In Equation (4), BBO represents the most ideal bounding box.

Equation 4

Equation 5

는 Max pooling에서 인덱스된 영역을 말한다. In Equation (4), G is a ground truth box,

Refers to the area indexed by Max pooling.

Equation (5) produces the ideal bounding box by calculating the range by overlapping the corresponding regions.

The system divides the process into two areas at the input stage and improves the accuracy of the object at a distance and near by improving the pooling problem.

6, the step S13 of extracting the number plate area is performed after the step S12. If the number is succeeded, the number is recognized (S14). If the number is succeeded, the detected frame is stored (S15).

Hereinafter, experimental results and results of applying the present invention to illegal parking vehicles will be described.

FIG. 15 shows experimental results, FIG. 16 shows experimental results according to object loss, and FIG. 17 shows experimental results showing improved processing results.

In this paper, we propose a method to detect objects in the fast R-CNN based on the method of detecting vehicles in illegal parking regulations. The performance of the system is deteriorated.

In order to solve this problem, it is possible to improve the performance of objects detection in the circle and near by dividing the region in the input step.

Figure 15 shows the result of using the proposed method (based on Fast R-CNN) to detect illegally parked vehicles in real environment.

In this case, although the input data was used without dividing into two regions in the image, there was no problem in detecting the target vehicle in the near distance, but in the case of the distance, the characteristic due to the ambiguous boundaries or the ROI As the result of Max Pooling is different from the result of pooling, it is analyzed as a factor that has inaccurate data.

In the case of FIG. 16 as well, as shown in FIG. 15, the characteristic distribution of some regions among the small objects is similar to that of the surrounding region, or the characteristic vector components are distributed differently depending on the size.

In the case of FIG. 17, in order to improve the results of FIGS. 15 and 16, the two regions of the input data are divided and the same improved method is performed. By distinguishing the difference between the object size and the POI pooling, This is a result of improving the accuracy in detection.

As described above, the present invention can provide a user with an illegal parking intervention system using Fast R-CNN based vehicle detection.

In addition, the present invention relates to an effective method for detecting a target vehicle in an illegal parking control system. In the conventional CNN (Convolution Neural Network) environment, Bounding Box calculation in Max-Pooling according to size, I can solve the problem.

Specifically, in the case of most prior arts, in the case of an illegally parked vehicle, a target object is detected in a patterned form in order to analyze a feature component, whereas in the present invention, the same object change By setting the feature vector distribution to be different according to the distance, it can be processed effectively compared with the existing method.

The above-described embodiments of the present invention can be implemented by various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of hardware implementation, the method according to embodiments of the present invention may be implemented in one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs) , FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of an implementation by firmware or software, the method according to embodiments of the present invention may be implemented in the form of a module, a procedure or a function for performing the functions or operations described above. The software code can be stored in a memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may exchange data with the processor by various well-known means.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The foregoing description of the preferred embodiments of the invention disclosed herein has been presented to enable any person skilled in the art to make and use the present invention. While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. For example, those skilled in the art can utilize each of the configurations described in the above-described embodiments in a manner of mutually combining them. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention. The present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, claims that do not have an explicit citation in the claims may be combined to form an embodiment or be included in a new claim by amendment after the filing.

Claims (8)

A first step of acquiring an image through a camera;
A second step of dividing the entire region of the image into a near region and a far region on the basis of a predetermined reference line in the image;
A third step of dividing the image into a plurality of predetermined blocks;
A fourth step of deriving a feature map to which the plurality of block units are applied by using at least one of a plurality of convolution layers of a Convolution Neural Network (CNN);
A fifth step of determining whether the object included in the image is a vehicle using the feature map; And
And if the object is a vehicle, recognizing the vehicle number from the vehicle number area of the vehicle,
Deriving a feature map for an area including the object using the plurality of convolution layers when the object is included in the remote area,
Wherein when the object is included in the near region, a feature map for an area including the object is derived using only a predetermined number of layers among the plurality of convolution layers. An illegal parking control method using.
The method according to claim 1,
If the object is included in the remote area in the second step,
And a second step of converting the scale of the long distance area between the second step and the third step.
The method according to claim 1,
In the fifth step,
Comparing the feature maps associated with the object for each of the plurality of block units;
A 5-2 step of determining a max pooling area in which the object is most frequently present among the plurality of block units; And
And determining whether the object is a vehicle based on the feature value of the vehicle on the basis of the maximum pulling area. The fast R-CNN based vehicle detection method .
The method of claim 3,
In the step 5-3,
Setting a region of interest (ROI) based on the maximum pulling region;
A fifth step of setting a bounding box based on the set ROI; And
And comparing the feature value of the object in the bounding box with the feature value of the vehicle to determine whether the object is the vehicle or not. Illegal parking regulations.
A camera for acquiring an image;
And a controller for dividing the entire region of the image into a near region and a far region on the basis of a predetermined reference line in the image and dividing the image into a plurality of predetermined blocks,
Wherein,
A feature map to which the plurality of block units are applied is derived using at least one of a plurality of convolution layers of a Convolution Neural Network (CNN)
Determines whether the object included in the image is a vehicle using the feature map,
When the object is a vehicle, recognizing the vehicle number from the vehicle number area of the vehicle,
Deriving a feature map for an area including the object using the plurality of convolution layers when the object is included in the remote area,
Wherein when the object is included in the near region, a feature map for an area including the object is derived using only a predetermined number of layers among the plurality of convolution layers. Illegal parking regulation system using.
6. The method of claim 5,
When the object is included in the remote area,
Wherein the control unit converts the scale of the long distance region.
6. The method of claim 5,
Wherein,
Comparing feature maps associated with the object for each of the plurality of block units,
Determining a max pooling region in which the object most frequently exists among the plurality of block units,
And determining whether the object is a vehicle based on the feature value of the vehicle based on the maximum pulling area.
8. The method of claim 7,
Wherein,
Setting a region of interest (ROI) based on the max pooling region,
Sets a bounding box based on the set ROI,
And comparing the feature value of the object in the bounding box with the feature value of the vehicle to determine whether the object is the vehicle, based on Fast R-CNN based vehicle detection.

Images