KR102636107B1 - Method and system for providing parking convenience service using deep learning video recognition technology - Google Patents

Abstract

The present invention relates to a method and system for providing parking convenience services using deep learning image recognition technology. The system for providing parking convenience services using deep learning image recognition technology according to the present invention is a CCTV camera installed in a parking lot using a deep learning model. A video analysis server that extracts the vehicle's license plate information and empty parking space number from the CCTV image captured in and analyzes the extracted information to generate parking location information of the user's vehicle and information on an available parking space where the vehicle can be parked; And a service providing server that receives the parking location information and the free parking surface information from the video analysis server and provides the information to the user terminal.

Description

Method and system for providing parking convenience services using deep learning video recognition technology {METHOD AND SYSTEM FOR PROVIDING PARKING CONVENIENCE SERVICE USING DEEP LEARNING VIDEO RECOGNITION TECHNOLOGY}

An embodiment of the present invention relates to a method and system for providing parking convenience services using deep learning image recognition technology that can provide information about free parking spaces and automatic notifications about parking locations when entering a parking lot.

With the rapid increase in the number of cars, parking difficulties are becoming more serious, and this parking problem is becoming a social issue.

In addition, in order to park a vehicle, the driver has no choice but to access parking lot information near the area where he or she is located or the parking lot information of the destination he or she wants to go to by visiting the parking lot in person or using means such as a phone call.

Therefore, there is an urgent need to develop a device or system that can more efficiently and quickly provide parking information desired by drivers.

In addition, the number of parking spaces provided in a conventional parking lot is managed by using sensors installed at the entrance of the parking lot, installing sensors on each parking surface, or photographing the parking lot and analyzing the captured images. did.

However, this conventional method has the problem of being expensive and unable to quickly and accurately convey various information about the parking lot to the user.

In addition, when using a large-scale indoor parking lot, it is difficult to find a free parking space, and there are frequent cases of forgetting the parking location after a certain period of time after parking, and parking spaces such as large discount stores, residential and commercial complexes, and duplex parking lots are frequent. The more crowded and complex these places are, the more the demand for guidance services before and after parking is increasing.

The present invention was developed to solve the above-described problem. Compared to the prior art, which requires expensive parking control equipment for parking services, the present invention seeks to provide a more useful service with only video shooting equipment.

To explain in more detail, parking control methods and systems according to the prior art provide parking services through the installation of expensive infrastructure equipment (sensors, control centers, etc.), and provide somewhat inaccurate services such as guidance on available parking spaces for each floor. On the other hand, the present invention seeks to provide a higher quality service in which related services can be provided only with video surveillance devices (CCTV cameras and transmission devices) installed in the parking lot, and even parking surface services are possible.

In addition, the present invention seeks to solve the problem of user demand for a parking guidance function that informs the available parking space and parking location when parking indoors through AI-based image recognition technology and mobile application (App) technology.

In addition, the present invention solves the parking guidance service of a large parking lot through a method and system using AI (deep learning) image reading technology and app service technology, and provides video shooting, automatic image recognition deep learning technology, DB, and application (APP). ) We aim to provide convenient parking services by converging IT technologies such as development.

To solve the above-described problem, the parking convenience service providing system using deep learning image recognition technology according to an embodiment of the present invention uses a deep learning model to obtain vehicle license plate information and empty space from CCTV images captured by CCTV cameras installed in the parking lot. An image analysis server that extracts a parking surface number, analyzes the extracted information, and generates information on the parking location of the user's vehicle and information on a free parking space where the vehicle can be parked; And a service providing server that receives the parking location information and the free parking surface information from the video analysis server and provides the information to the user terminal.

According to another embodiment of the present invention, the video analysis server can extract license plate characters, parking surface markings, or parking area marking characters of a plurality of vehicles from the CCTV image.

According to another embodiment of the present invention, the video analysis server constructs a deep learning model by learning using a data set, and classifies and labels the CCTV images to learn the deep learning model. can be performed.

According to another embodiment of the present invention, the video analysis server configures a data set consisting of learning data and test data to be used for classification, evaluates the performance of learning using the data set, and You can configure a deep learning model by deciding which deep learning model to use, modifying the determined deep learning model, or adding a new deep learning algorithm.

According to another embodiment of the present invention, the video analysis server can configure the deep learning model by repeating learning, evaluation, and environment settings using the deep learning model.

According to another embodiment of the present invention, the service providing server provides downloading and installation of an application to the user terminal, and provides the parking location information and the free parking space through a parking lot map displayed through the application. Information can be provided.

A method of providing parking convenience services using deep learning image recognition technology according to an embodiment of the present invention is a video analysis server that uses the deep learning model configured above to collect vehicle license plate information and empty space from CCTV images captured by CCTV cameras installed in the parking lot. An information extraction step of extracting a parking lot number; An information generation step in which the image analysis server analyzes the extracted information to generate parking location information of the user's vehicle and information on a free parking space where the vehicle can be parked; And an information provision step in which the service providing server receives the parking location information and the spare parking surface information from the video analysis server and provides the information to the user terminal.

According to another embodiment of the present invention, in the information extraction step, the video analysis server may extract license plate characters, parking surface marking characters, or parking area marking characters of a plurality of vehicles from the CCTV image.

According to another embodiment of the present invention, before the information extraction step, a deep learning learning step in which the image analysis server learns using a data set to construct a deep learning model may be further included.

According to another embodiment of the present invention, in the deep learning learning step, the video analysis server may perform classification and labeling of the CCTV image to learn the deep learning model.

According to another embodiment of the present invention, the deep learning learning step includes configuring a data set consisting of learning data and test data to be used by the image analysis server for classification; The video analysis server evaluating the performance of learning using the data set and determining a deep learning model to be used for the learning; and @ where the video analysis server configures a deep learning model by modifying the deep learning model or adding a new deep learning algorithm.

According to another embodiment of the present invention, in the deep learning learning step, the video analysis server may configure the deep learning model by repeating learning, evaluation, and environment settings using the deep learning model.

According to another embodiment of the present invention, the information providing step includes providing the service providing server to download and install an application to the user terminal; and providing, by the service providing server, the parking location information and the free parking space information through a parking lot map displayed through the application.

According to the present invention, a more useful service can be provided with only video recording equipment compared to existing services that require expensive parking control equipment for parking service.

To explain in more detail, parking control methods and systems according to the prior art provide parking services through the installation of expensive infrastructure equipment (sensors, control centers, etc.), and provide somewhat inaccurate services such as guidance on available parking spaces for each floor. On the other hand, the present invention can provide related services only with video surveillance devices (CCTV cameras and transmission devices) installed in the parking lot, and can provide higher quality services including parking surface services.

In addition, according to the present invention, the user's demand for a parking guidance function that informs the available parking space and parking location when parking indoors can be solved through AI-based image recognition technology and mobile application (App) technology.

In addition, according to the present invention, parking guidance services in large parking lots are solved through a method and system using AI (deep learning) image reading technology and app service technology, and video shooting, automatic image recognition deep learning technology, DB, and application ( Convenient parking services can be provided by converging IT technologies such as APP (App) development.

In addition, if CCTV is already installed in the parking space, it is possible to expand to additional services such as receiving related CCTV images and providing parking convenience services.

In addition, through the present invention, users are provided with guidance on empty parking spaces upon entering a complex parking lot on their smartphone, and at the same time, parking lot owners can provide differentiated services for parking convenience and are able to provide a differentiated service for parking convenience. It serves as a foundation for expansion into additional services such as product promotion and exhibitions.

Figure 1 is a conceptual diagram of a parking convenience service provision system using deep learning image recognition technology according to an embodiment of the present invention.
FIGS. 2A and 2B are diagrams for explaining an image analysis method using a deep learning model according to an embodiment of the present invention, and FIGS. 2C to 2E are diagrams showing labeling of images such as license plates for learning vehicles according to an embodiment of the present invention. This is a diagram to explain how to create a dataset through labeling using a tool.
Figure 3 is a flowchart illustrating a method of providing parking convenience services using deep learning image recognition technology according to an embodiment of the present invention.
Figures 4 to 9 are diagrams for explaining a method of providing information on the parking position of a vehicle and information on a free parking space where a vehicle can be parked through an application of a user terminal according to an embodiment of the present invention.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings. However, in describing the embodiments, if it is determined that specific descriptions of related known functions or configurations may unnecessarily obscure the gist of the present invention, detailed descriptions thereof will be omitted. Additionally, the size of each component in the drawings may be exaggerated for explanation and does not mean the actual size.

Figure 1 is a conceptual diagram of a parking convenience service provision system using deep learning image recognition technology according to an embodiment of the present invention, and Figure 2 illustrates an image analysis method using a deep learning model according to an embodiment of the present invention. This is a drawing for

From now on, the configuration of a parking convenience service providing system using deep learning image recognition technology according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

Referring to FIG. 1, a parking convenience service provision system using deep learning image recognition technology according to an embodiment of the present invention is comprised of an image analysis server 110 and a service provision server 120.

When a user uses a parking service by car, a CCTV camera 101 installed in the parking lot captures CCTV images of the parking lot. That is, the CCTV camera 101 collects and transmits video information of the parking facility, and the video analysis server 110 receives the CCTV video.

The video analysis server 110 uses a deep learning model to extract vehicle license plate information and empty parking lot numbers from CCTV images captured by the CCTV camera 101 installed in the parking lot.

At this time, the image analysis server 110 may extract license plate characters, parking surface markings, or parking area marking characters of a plurality of vehicles from the CCTV image.

In order to analyze such images, the video analysis server 110 can construct a deep learning model by learning using a data set. More specifically, in order to learn the deep learning model, the video analysis server 110 can construct a deep learning model for each vehicle type. Classification and labeling of CCTV images can be performed.

When training the video analysis server 110, a data set consisting of learning data and test data to be used for classification is formed, the performance of learning using the data set is evaluated, and the data set to be used for learning is evaluated. A deep learning model can be determined, and the deep learning model can be configured by modifying the determined deep learning model or adding a new deep learning algorithm.

To explain in more detail, the video analysis server 110 configures the original data set to be used for the classification problem into a learning data set and a test data set, and evaluates the performance through performance evaluation measures such as accuracy for performance evaluation of the recognition problem. can be evaluated.

Also, referring to FIGS. 2A to 2D, as a process of building a deep learning model to extract vehicle information (vehicle license plate) and parking area information through analysis of collected video data, the video analysis server 110 performs image classification (Image Classification) ) As models, you can use Lenet, AlexNet, ZFNet, GooleNet, VGNet, or ResNet models, and as object detection models, you can use FCN, R-CNN, Fast RCNN, or Faster RCNN models, and license plate and parking surface By setting up and implementing a deep learning learning model to extract character information of numbers, the final reviewed deep learning model is set up on the video analysis server 110 to perform training and evaluation based on the labeled image dataset. Build a learning model by repeatedly setting the optimal environment (configuration).

In this way, the image analysis server 110 configures the deep learning model by repeating learning, evaluation, and environment settings using the deep learning model configured as above, thereby enabling extraction and analysis of information with higher accuracy.

The image analysis server 110 analyzes the information extracted in this way and generates information on the parking location of the user's vehicle and information on a free parking space where the vehicle can be parked.

In addition, according to an embodiment of the present invention, the video analysis server 110 performs tests and performance evaluations on CCTV collected images, and extracts related information from CCTV collected images of multiple vehicles that are periodically transmitted. Through evaluation of performance and accuracy, performance can be verified according to the transmission, collection, analysis, and number extraction stages of parked vehicle images.

The service providing server 120 receives the parking location information and the free parking space information from the video analysis server and provides them to the user terminal 130.

More specifically, the service providing server 120 may provide downloading and installation of an application to the user terminal configured as a smart terminal, and may provide the parking location information and the parking lot map displayed through the application. Information on the available parking space can be provided.

In this way, according to one embodiment of the present invention, the available parking space and the parking location of the user's vehicle can be provided to the user through a smart terminal application. More specifically, the user's vehicle image recognition and information about the parking location will be transmitted to the registered smart terminal, and the parking space number marked on the empty parking space will be recognized and the free parking space information will be transmitted to the smart terminal. You can.

In addition, according to an embodiment of the present invention, related databases (125 ) can be built, and based on analysis of the intelligent parking-related market, more convenient parking-related services can be provided between users with smart devices and parking lots with CCTV installed.

In this way, according to the present invention, the user's demand for a parking guidance function that informs the available parking space and parking location when parking indoors can be solved through AI-based image recognition technology and mobile application (App) technology.

In addition, according to the present invention, parking guidance services in large parking lots are solved through a method and system using AI (deep learning) image reading technology and app service technology, and video shooting, automatic image recognition deep learning technology, DB, and application ( Convenient parking services can be provided by converging IT technologies such as APP (App) development.

Figures 2c to 2e show labeling of the collected image dataset using a labeling tool to enable the deep learning model to recognize target objects (vehicle license plate, parking zone number) in the image. It is a drawing. The labeling task is set by setting a Bounding Box for the extraction target area of the image and specifying the category of the related object. The labeling results are stored in a file type called “XML” or “Json” depending on the type of labeling tool.

Figure 3 is a flowchart for explaining a method of providing parking convenience services using deep learning image recognition technology according to an embodiment of the present invention, and Figures 4 to 9 are diagrams showing an application (application of a user terminal) according to an embodiment of the present invention. This is a drawing to explain how to provide information on the parking location of a vehicle and information on available parking spaces where a vehicle can be parked through an application.

From now on, a method of providing a parking convenience service using deep learning image recognition technology according to an embodiment of the present invention will be described with reference to FIGS. 3 to 9.

First, the video analysis server learns using the data set and constructs a deep learning model (S310).

At this time, the video analysis server constructs a learning model by repeatedly learning the image dataset and labeling information that are subject to learning.

More specifically, the video analysis server configures a data set consisting of learning data and test data to be used by the video analysis server for classification, and the video analysis server evaluates the performance of learning using the data set. After determining the deep learning model to be used for learning, the video analysis server may configure the deep learning model by modifying the deep learning model or adding a new deep learning algorithm.

In addition, the video analysis server may configure the deep learning model by repeating learning, evaluation, and environment settings using the deep learning model.

Afterwards, the video analysis server receives CCTV images captured from CCTV cameras installed in the parking lot. In the case of images extracted from regular resolution CCTV rather than high-resolution CCTV, the resolution is converted to a structure that is easy to recognize in the deep learning model. To do this, the image is converted into a high-resolution image using an image conversion program (S320).

Thereafter, the video analysis server extracts the vehicle's license plate information and the empty parking lot number from the CCTV image captured by the CCTV camera installed in the parking lot using the configured deep learning model (S330).

At this time, the video analysis server can extract license plate characters, parking surface markings, or parking area marking characters of a plurality of vehicles from the CCTV image.

Afterwards, the image analysis server analyzes the extracted information and generates information on the parking location of the user's vehicle and information on a free parking space where the vehicle can park (S340).

In addition, the video analysis server provides the parking location information and the free parking space information to the user terminal (S350).

At this time, more specifically, as shown in FIG. 4, when the service providing server provides download and installation of an application to the user terminal and the user logs in, the service providing server is displayed through the application. The parking location information and the available parking space information can be provided through a map.

Referring to Figures 5 and 6, the service providing server provides floor and area information as parking location information of the user's vehicle to the user terminal through an application, and provides information on the parking location of the vehicle more conveniently and intuitively through a parking lot map. Information can be provided.

In addition, referring to FIGS. 7 and 8, the service providing server provides information on the number of available parking spaces, floors, and areas as information on available parking spaces through the application, and at this time, as shown in FIG. 9, the information on the parking lot map is provided. Information on available parking spaces can be provided conveniently and intuitively.

In this way, according to one embodiment of the present invention, the available parking space and the parking location of the user's vehicle can be provided to the user through a smart terminal application. More specifically, the user's vehicle image recognition and information about the parking location will be transmitted to the registered smart terminal, and the parking space number marked on the empty parking space will be recognized and the free parking space information will be transmitted to the smart terminal. You can.

In this way, according to the present invention, the user's demand for a parking guidance function that informs the available parking space and parking location when parking indoors can be solved through AI-based image recognition technology and mobile application (App) technology.

In the detailed description of the present invention as described above, specific embodiments have been described. However, various modifications are possible without departing from the scope of the present invention. The technical idea of the present invention should not be limited to the above-described embodiments of the present invention, but should be determined not only by the claims but also by equivalents to the claims.

101: CCTV camera
110: Video analysis server
120: Service provision server
125: database
130: user terminal

Claims (13)

Using a deep learning model, the vehicle's license plate information and the parking space number marked on the empty parking space are extracted from CCTV images captured by multiple CCTV cameras installed in the parking lot, and the extracted information is analyzed to park the user's vehicle. A video analysis server that generates location information and information on available parking spaces where vehicles can park; and
It includes a service providing server that receives the parking location information and the free parking surface information from the video analysis server and provides them to the user terminal.
The service providing server provides downloading and installation of an application to the user terminal, and provides parking lot map information using a database built with parking lot information including parking lot interior map information through the downloaded and installed application. And, through the application, floor and area information is provided as parking location information of the user's vehicle to the user terminal, and the parking location information of the vehicle is displayed through the parking lot map information to determine the parking location information and the free parking. Provides visual information,
The video analysis server builds a database consisting of vehicle information including license plate number and vehicle type, member information, user information including smartphone number, and parking lot information including parking lot name and inside map of the parking lot,
The video analysis server extracts license plate characters, parking surface marking characters, or parking area marking characters of a plurality of vehicles from the CCTV image, learns them using a data set, and constructs a deep learning model, and the deep In order to learn the learning model, classification and labeling of the CCTV images are performed, a data set consisting of learning data and test data to be used for classification is formed, and the performance of learning using the data set is evaluated. Evaluate and determine the deep learning model to be used for learning, configure the deep learning model by modifying the determined deep learning model or adding a new deep learning algorithm, and repeat learning, evaluation, and environment settings using the deep learning model. Constructing the deep learning model,
The video analysis server uses Lenet, AlexNet, ZFNet, GooleNet, VGNet, or ResNet models as image classification models, and FCN, R-CNN, Fast RCNN, or Faster RCNN models as object detection models. A parking convenience service provision system using deep learning image recognition technology.
delete delete delete delete delete An information extraction step in which the video analysis server extracts the license plate information of the vehicle and the parking surface number marked on the empty parking surface from CCTV images captured from a plurality of CCTV cameras installed in the parking lot using a deep learning model;
An information generation step in which the image analysis server analyzes the extracted information to generate parking location information of the user's vehicle and information on a free parking space where the vehicle can be parked; and
It includes an information provision step in which the service providing server receives the parking location information and the free parking surface information from the video analysis server and provides the information to the user terminal,
The information providing step includes the service providing server providing download and installation of an application to the user terminal; The service providing server providing parking lot map information using a database built with parking lot information including parking lot interior map information through the downloaded and installed application; And the service providing server provides floor and area information as parking location information of the user's vehicle to the user terminal through the application, and displays the parking location information of the vehicle through the parking lot map information to provide the parking location information. and visually providing the free parking space information.
The information extraction step extracts license plate characters, parking surface symbols, or parking zone symbols of a plurality of vehicles from the CCTV image,
The information generation step includes building a database consisting of vehicle information including license plate number and vehicle type, member information, user information including smartphone number, parking lot name, and parking lot information including an internal map of the parking lot. ,
In the information generation step, a deep learning model is constructed by learning using a data set, classification and labeling of the CCTV images are performed to learn the deep learning model, and learning to be used for the classification is performed. Construct a data set consisting of data and test data, evaluate the performance of learning using the data set, determine a deep learning model to be used for learning, and modify the determined deep learning model or create a new deep learning algorithm. It includes the step of configuring the deep learning model by adding and repeating learning, evaluation, and environment settings using the deep learning model,
At this time, Lenet, AlexNet, ZFNet, GooleNet, VGNet, or ResNet models are used as image classification models, and FCN, R-CNN, Fast RCNN, or Faster RCNN models are used as object detection models. A method of providing parking convenience services using deep learning image recognition technology.
delete delete delete delete delete delete