KR102482427B1 - system and method for recognizing and extracting traffic safety signs for generating precision road maps - Google Patents

Abstract

A traffic safety sign recognition/extraction system for generating a high-precision map according to an embodiment of the present invention includes an input unit for receiving a road driving image; a lane detection unit that detects a lane within an image frame of the road driving image using a deep learning algorithm, and then analyzes and classifies information of the detected lane; After creating an integrated object detection area by merging and expanding the detection areas of at least two or more deep learning algorithms that detect image objects with different detection methods, the image frame in which the lane is detected according to the preset detection area condition value a road safety sign object detection unit for detecting a road safety sign object located in the integrated object detection area within the vehicle; and an object analyzer for analyzing detailed shapes based on feature points extracted according to color, saturation, and brightness of the detected road safety sign object.

Description

Traffic safety sign recognition/extraction method and system for generating precision road maps {system and method for recognizing and extracting traffic safety signs for generating precision road maps}

The present invention relates to a method and system for recognizing/extracting traffic safety signs for generating a high-precision map.

Recently, with the rapid development of information processing and communication technology, a method of constructing geographic information as map information data and using it for transportation has been widely used. Such map information data is produced based on physically photographed geographic information and is used for wayfinding through navigation, safe driving, and automatic driving.

A navigation system that utilizes such map data is an automatic vehicle navigation system, and is a device that provides route guidance to a destination through voice and monitor screens using GPS that determines the current location using satellites. These navigation devices include a GPS receiver receiving current location information of the vehicle from GPS satellites, a map DB in which map information such as road safety signs is registered, and map information according to the current location identified through the GPS receiver. An operating program displayed on the screen is provided.

Map information constructed in the map DB of the navigation system is generated through measurement of actual geographic information. As such map information changes according to new construction of roads or changes in the transportation system, regular or irregular update of map information shall. Updating map information of such navigation is usually done manually. In particular, since traffic signs constantly change, the map manager frequently moves to the location on the map to check the road safety signs one by one, and then the road safety included in the map information. There was a cumbersome problem of updating the information on the sign.

Registered Patent Publication No. 10-1199959

An object of the present invention is to provide a method and system for recognizing/extracting traffic safety signs for generating a high-precision map that can solve the conventional problems.

A traffic safety sign recognition/extraction system for generating a high-precision map according to an embodiment of the present invention for solving the above problems includes an input unit for receiving a road driving image; a lane detection unit that detects a lane within an image frame of the road driving image using a deep learning algorithm, and then analyzes and classifies information of the detected lane; After generating an integrated object detection area by merging and expanding the detection areas of each of at least two or more deep learning object detection algorithms that detect image objects with different detection methods, the lane is detected according to the preset detection area condition value a road safety sign object detector detecting a road safety sign object located in the integrated object detection area within an image frame; an object analyzer that analyzes detailed shapes based on feature points extracted according to color, saturation, and brightness of the detected road safety sign object; and an object tracking unit that compares and determines whether road safety sign objects detected in the front/back image frame match, and if matched, provides an ID assigned to the road safety sign object and detailed lane information in the form of a queue. The object tracking unit checks whether or not a road sign object is detected in the n+1 th image frame and whether an ID is allocated from the object analysis unit, and then calculates an optimal value of an offset vector between the center of the n th image frame and the center of the n+1 th image frame. After calculation, it is determined whether the road sign objects are matched based on the optimal value of the offset vector, and after stacking image frames in which the same road safety sign object is detected, if the stacking value exceeds the reference value, the stacked image frames It is characterized by storing and transmitting road safety sign object ID, segment, object detection area and lane information in the form of a queue.

A traffic safety sign recognition/extraction method for generating a road map with precision according to an embodiment of the present invention to solve the above problems is to detect a lane in an image frame of a road driving image using a deep learning algorithm, and then the detected lane Analyzing and classifying information of; After creating an integrated object detection area by merging and extending the object detection areas of at least two or more deep learning object detection algorithms, the integrated object detection area is detected in the image frame in which the lane is detected according to the preset detection area condition value. Detecting a road safety sign object located thereon; Analyzing an image of a road safety sign object based on feature points extracted according to color, saturation, and brightness of the detected road safety sign object; After allocating a detection ID for identifying the road safety sign object detected in the image frame to the image frame and comparing and determining whether or not the road safety sign object detected in the previous/post image frame matches, if they are matched, road safety A step of providing an ID assigned to a sign object and detailed lane information in the form of a queue, wherein the step of providing in the form of a queue stacks image frames in which the same road safety sign object is detected, and then the stacked value exceeds the reference value. If it exceeds, it is characterized in that it is a step of storing and transmitting the road safety sign object ID, segment, object detection area, and lane information of the stacked image frames in the form of a queue.

Therefore, using the traffic safety sign recognition/extraction system and method for generating a precision road map according to an embodiment of the present invention, the detection accuracy of road safety signs is increased by expanding the detection area using two deep learning object detection algorithms. , subdividing the filtering process for the detection area of the road safety sign, and tracking objects in successive frames, thereby providing an advantage of improving the accuracy of the detection position of the road safety sign in the road driving image.

1 is a block diagram of a traffic safety sign recognition/extraction system for generating a precision map according to an embodiment of the present invention.
FIG. 2 is an exemplary diagram explaining the operation of the lane detecting unit shown in FIG. 1 .
FIG. 3 is a detailed block diagram of an object detection unit shown in FIG. 1 .
4 is a flow chart explaining a process of analyzing an object located within a road marking area by an object analysis unit, and FIG. 5 is a flowchart explaining a process of analyzing an image of an object located in an area other than a road marking area by an object analysis unit. .
FIG. 6 is an exemplary view illustrating an operation of an object tracking unit shown in FIG. 1 .
7 is a flowchart illustrating a method for recognizing/extracting traffic safety signs for generating a precision road map according to an embodiment of the present invention.
FIG. 8 is a detailed flowchart of the process S720 shown in FIG. 7 .
FIG. 9 is a detailed flowchart of the process S730 shown in FIG. 7 .
10 and 11 are detailed flowcharts of the process S740 shown in FIG. 7 .
FIG. 12 is a detailed flowchart of the process S760 shown in FIG. 7 .
13 is an illustration of an example computing environment in which one or more embodiments set forth herein may be implemented.

Specific structural or functional descriptions of the embodiments of the present invention disclosed in this specification or application are merely exemplified for the purpose of explaining the embodiments according to the present invention, and the embodiments according to the present invention may be implemented in various forms. and should not be construed as being limited to the embodiments described in this specification or application.

Embodiments according to the present invention can apply various changes and can have various forms, so specific embodiments are illustrated in the drawings and described in detail in this specification or application. However, this is not intended to limit the embodiments according to the concept of the present invention to a specific disclosed form, and should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

Terms such as first and/or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component, e.g., without departing from the scope of rights according to the concept of the present invention, a first component may be termed a second component, and similarly The second component may also be referred to as the first component.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle. Other expressions describing the relationship between elements, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", etc., should be interpreted similarly.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate that a specified feature, number, step, operation, component, part, or combination thereof exists, but that one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, a traffic safety sign recognition/extraction method and system for generating a precision map according to an embodiment of the present invention will be described in more detail based on the accompanying drawings.

1 is a block diagram of a traffic safety sign recognition/extraction system for generating a high-precision map according to an embodiment of the present invention, FIG. 2 is an exemplary diagram explaining the operation of a lane detection unit shown in FIG. 1, and FIG. A detailed block diagram of the object detection unit shown in FIG. 1, FIG. 4 is a flow chart explaining a process of analyzing an object located within the road marking area in the object analysis unit, and FIG. 5 is an image of an object located in an area other than the road marking area. It is a flow chart explaining the process of analyzing in the object analyzing unit, and FIG. 6 is an exemplary diagram explaining the operation of the object tracking unit shown in FIG. 1 .

First, as shown in FIG. 1, the traffic safety sign recognition/extraction system 100 for generating a precision road map according to an embodiment of the present invention includes an input unit 110, a lane detection unit 120, and a road safety sign object. It includes a detection unit 130, an object analysis unit 140, an ID allocation unit 150 and an object tracking unit 160.

On the other hand, the present invention may include a preprocessing component for performing a preprocessing process on an image frame of a road driving image, and the preprocessing component is histogram smoothing to uniformize the distribution of contrast values of each image frame of the input road driving image, For example, it may be configured to calculate the frequency of lightness values, calculate the accumulated histogram value using the frequency, normalize it, and apply a gray scale mapping function to the normalized accumulated histogram to map gray level values.

The preprocessing process mentioned in the present invention is only a representative example of a method of preprocessing an image frame, and other preprocessing methods can also be applied in the present invention.

The input unit 110 is a component that receives a road driving image.

Next, referring to FIG. 2 , the lane detection unit 120 may be configured to detect lanes in each preprocessed image frame using a deep learning algorithm, and then analyze and classify information of the detected lanes.

More specifically, the lane detection unit 120 detects lanes using a deep learning algorithm, corrects the road markings and average width between the detected lanes through a morphology filtering (calculation) process, and then corrects the After removing areas other than the lane area and redundant road markings, the corrected lane may be classified into one of a dotted line, a solid line, an exclusive lane, and a center line. As a deep learning algorithm, convolutional LSTM (Long Short-Term Memory) can be used, and various other deep learning algorithms can be used, of course.

For reference, Morphology is a set of extensive image processing operations that process images according to their shapes. In Morphology operations, each pixel of an image is adjusted according to the values of other neighboring pixels, and the size and shape of its neighbors are selected. This means that the target image is calculated and processed in a specific form.

Basically, morphology calculation is an erode and dilate calculation process, which applies a filter based on the current pixel, checks the value within the filter area, and modifies the value of the current pixel.

Here, erosion means substituting the minimum pixel value among pixels within the filter area to the current pixel value, dilation means substituting the maximum pixel value among pixels within the filter area to the current pixel value, and morphology operation is a known technique. Optionally, a more detailed description will be omitted.

Next, referring to FIG. 3, the road safety sign object detection unit 130 generates an integrated object detection area by merging and expanding the detection bounding boxes of two different deep learning object detection algorithms, It may be configured to detect a road safety sign object located in the integrated object detection area within the image frame in which the lane is detected according to a set detection area condition value.

More specifically, the road safety sign object detection unit 130 includes an object detection area adjusting unit 131 and an object detection area filtering unit 132 .

The object detection area adjusting unit 131 overlaps the object detection area (bounding box) of each of the first deep learning object detection algorithm and the second deep learning object detection algorithm to exceed a preset overlap value, preferably 60%. may be a configuration for generating the integrated object detection area.

For reference, the first deep learning object detection algorithm has a fast object detection speed but low recognition accuracy, and the second deep learning object detection algorithm has a slow object detection speed but high recognition accuracy. Gaussian YOLO v3 may be used as the first deep learning object detection algorithm, and MASK R-CNN may be used as the second deep learning object detection algorithm. The Gaussian YOLO v3 and MASK R-CNN algorithms are examples of the first and second deep learning object algorithms, but are not limited thereto, and other appropriate deep learning object algorithms suitable for the characteristics of the algorithm may be used.

Therefore, in the present invention, an integrated object detection area is created and used so that the advantages of the two deep learning object detection algorithms can be derived by integrating and extending the detection bounding boxes of the two deep learning object detection algorithms.

Meanwhile, the object detection area adjusting unit 131 may detect a road safety sign object using a second deep learning object detection algorithm when the overlapped integrated object detection area is smaller than a preset overlap value (60%).

The object detection area filtering unit 132 is configured to filter the location of the road safety sign object detected in the integrated object detection area to a preset detection area condition value, wherein the preset detection area condition value is the road safety sign. An area value for a detection position of a marker object, and the area value includes a road surface area, a left area of the center line, and a right curb area.

More specifically, the object detection area filtering unit 132 first determines whether the location of the integrated object detection area is a road sign area, and if it is not a road sign area, secondly determines whether the location of the integrated object detection area is to the left of the center line, and if not to the left of the center line. In this case, it is possible to reduce an error in the detection rate of the road safety sign object and improve the accuracy by specifying the location area where the integrated object detection area is generated by thirdly determining whether it is on the right side of the curb.

Next, referring to FIGS. 4 and 5, the object analysis unit 140 analyzes (determines) the detailed shape based on the feature points extracted according to the color, saturation, and brightness of the road safety sign object in the filtered integrated object detection area. can be a composition.

More specifically, referring to FIG. 5 , the object analysis unit 140 determines the location of the integrated object detection area (bounding box) where the road safety sign object is detected when the location of the bounding box is located in the road surface area. After converting the pixel RGB values into HSV (hue, saturation, brightness), generating a threshold of the road safety sign object through a preset threshold algorithm, and then determining the object and the road sign area based on the generated threshold After dividing into white and black, noise is removed through morphology filtering, contours of the detected road safety sign object are extracted and fitted, and the road marking area is based on the feature points of the fitted contour. Determine the shape of the road sign object located at . Of course, the Otsu Algorithm may be used as the above-described threshold value algorithm, and other appropriate algorithms may be used.

In addition, referring to FIG. 4, the object analysis unit 140, when the location of the object detection area (bounding box) where the road safety sign object is detected is located in an area other than the road surface area, the color histogram for the object detection area Calculate (contrast value distribution for pixels), convert RGB values of pixels in the object detection area into HSV (hue, saturation, brightness), and convert the object detection area into CIE (Commission Internationale d'Eclairage) color space coordinates After conversion, the red, blue, and green color bands of the object detection area are counted using the color histogram and the HSV value, and L (brightness) and A on the CIE (Commission Internationale d'Eclairage) color space coordinates (Complementary color from green to red) and B (complementary color from yellow to blue) are extracted, and the object detection area including the LAB value of the color space coordinates and the count value of the RGB band is converted into a gray scale image. After conversion, a threshold value of the road safety sign object is generated through a preset threshold value algorithm (Otsu Algorithm), and after dividing the object and foreground area based on the generated threshold value, morphology filtering is performed. After noise is removed and contours of the detected road safety sign object are extracted, the shape (including color) of the road safety sign object is determined based on the location of feature points of the contour.

Next, the ID allocator 160 may be a component that allocates a detection ID for identifying the road safety sign object detected in the image frame analyzed by the object analyzer 150 to the image frame.

Next, referring to FIG. 6, the object tracking unit 170 compares and determines whether the road safety sign objects detected in the before and after image frames of the road driving video match, and if matched, the ID assigned to the road safety sign object. , It may be configured to provide detailed information and lane information in the form of a queue.

More specifically, the object tracking unit 170 checks whether or not a road sign object is detected and an ID is allocated in the n+1 th image frame from the object analyzer 150, and then the center of the n th image frame and the n+1 th image frame. After calculating the optimum value of the offset vector at the center of the frame, whether or not the road sign object is matched is determined based on the optimum value of the offset vector.

In addition, the object tracking unit 170 stacks image frames in which the same road sign object is detected, and if the stacking value exceeds a reference value, the road safety sign object ID, segment, and object detection area (bounding box) of the stacked image frames ) and lane information are stored in the form of a queue and provided to an external server.

7 is a flowchart illustrating a method for recognizing/extracting traffic safety signs for generating a precision road map according to an embodiment of the present invention, FIG. 8 is a detailed flowchart of process S720 shown in FIG. 7, and FIG. 10 and 11 are detailed flowcharts of the process S740 shown in FIG. 7 , and FIG. 12 is a detailed flow chart of the process S760 shown in FIG. 7 .

First, referring to FIG. 7 , in the traffic safety sign recognition/extraction method (S700) for generating a precision road map according to an embodiment of the present invention, first, an image frame of a road driving image input from an input unit is converted into a pre-processing unit 120 ) in preprocessing (S710).

Here, in the step S710, histogram equalization is performed to uniformize the distribution of brightness values of each image frame of the input road driving image, for example, calculating the frequency of brightness values, calculating the accumulated histogram value using the frequency, and then normalizing, and mapping gray level values by applying a gray scale mapping function to the normalized accumulation histogram.

When the process S710 is completed, lanes in each preprocessed image frame are detected using a deep learning algorithm, and then types of detected lanes are analyzed and classified (S720).

More specifically, in the step S720, lanes are detected using a deep learning algorithm, road markings and average widths between the detected lanes are corrected through a morphology filtering (calculation) process, and then the corrected lane area is corrected. It may be a process of classifying the corrected lane into one of a dotted line, a solid line, an exclusive lane, and a center line after removing other areas and redundant road markings. As described above, convolutional long short-term memory (LSTM) can be used as a deep learning algorithm, and various other deep learning algorithms can be used, of course.

For reference, Morphology is a set of extensive image processing operations that process images according to their shapes. In Morphology operations, each pixel of an image is adjusted according to the values of other neighboring pixels, and the size and shape of its neighbors are selected. This means that the target image is calculated and processed in a specific form.

Basically, morphology calculation is an erode and dilate calculation process, which applies a filter based on the current pixel, checks the value within the filter area, and modifies the value of the current pixel.

Here, erosion means substituting the minimum pixel value among pixels within the filter area to the current pixel value, dilation means substituting the maximum pixel value among pixels within the filter area to the current pixel value, and morphology operation is a known technique. Optionally, a more detailed description will be omitted.

When the S720 process is completed, an integrated object detection area is created by merging and expanding the object detection areas of the two different deep learning object detection algorithms, and then within the image frame in which the lane is detected according to the preset detection area condition value. Detects a road safety sign object located in the integrated object detection area (S730).

More specifically, the step S730 overlaps the bounding boxes of each of the first deep learning object detection algorithm and the second deep learning object detection algorithm to exceed a predetermined overlap value, preferably 60%. and generating an integrated object detection area (S731).

For reference, the first deep learning object detection algorithm has a fast object detection speed but low recognition accuracy, and the second deep learning object detection algorithm has a slow object detection speed but high recognition accuracy. Gaussian YOLO v3 may be used as the first deep learning object detection algorithm, and MASK R-CNN may be used as the second deep learning object detection algorithm. The Gaussian YOLO v3 and MASK R-CNN algorithms are examples of the first and second deep learning object algorithms, but are not limited thereto, and other appropriate deep learning object algorithms suitable for the characteristics of the algorithm may be used.

Accordingly, in the present invention, an integrated object detection area is created and used so that the advantages of the two deep learning object detection algorithms can be derived by integrating and extending the detection bounding boxes of the two deep learning object detection algorithms, and overlapping When the integrated object detection area is smaller than the preset overlap value (60%), a process of detecting a road safety sign object is performed with the second deep learning object detection algorithm.

Meanwhile, the process S730 includes a step of filtering the object detection area with a predetermined detection area condition value based on the position of the road safety sign object detected in the integrated object detection area (S732).

The S732 process first determines whether the integrated object detection area is a road sign area, secondly determines whether it is on the left side of the center line if it is not a road sign area, and thirdly determines whether it is on the right side of the curb if it is not on the left side of the center line. Including the process of doing, it is possible to reduce errors in the detection rate of road safety sign objects and improve accuracy by specifying the location area where the integrated object detection area (bounding box) is generated through the above-described area filtering process.

On the other hand, when the S730 process is completed, the object analysis unit 150 determines the detailed shape based on the feature points extracted according to the color, saturation, and brightness of the road safety sign object in the integrated object detection area filtered by the object analysis unit 150. Analysis (judgment) (S740).

Here, in the step S740, when the location of the object detection area (bounding box) where the road safety sign object is detected is located in the road marking area, the pixel RGB values of the object detection area (bounding box) are converted into HSV (hue, saturation, brightness) ), after generating the threshold value of the road safety sign object through a preset threshold algorithm, and then classifying the object and the road sign area into white and black based on the generated threshold value, through morphology filtering After removing noise, extracting and fitting the contours of the detected road safety sign object, and then determining the shape of the road sign object located in the road marking area based on the feature points of the fitted contour. do. As described above, the Otsu Algorithm can be used as the threshold algorithm, and other suitable algorithms can be used as well.

In addition, in the S740 process, when the location of the object detection area (bounding box) where the road safety sign object is detected is located in an area other than the road marking area, the color histogram (contrast value distribution chart for pixels) for the object detection area ) Calculation, conversion of pixel RGB values of the object detection area into HSV (hue, saturation, brightness) and conversion of the object detection area into CIE (Commission Internationale d'Eclairage) color space coordinates, the color histogram and the HSV Red, blue, and green color bands of the object detection area are counted using the values, and L (brightness), A (complementary color from green to red), and B ( Complementary color of yellow to blue) is extracted, and the object detection area including the LAB value of the color space coordinates and the count value of the RGB band is converted into a gray scale image, and then a preset threshold algorithm ( Otsu Algorithm), after generating the threshold value of the road safety sign object, dividing the object and the foreground area based on the generated threshold value, removing noise through morphological filtering, and After extracting the contours, the process of determining the shape (including color) of the road safety sign object based on the location of feature points of the contours is included.

Thereafter, when the process of S740 is completed, the ID allocation unit 150 allocates a detection ID for identifying the road safety sign object detected in the image frame analyzed in the process of S740 to the image frame (S750).

Next, when the S750 process is completed, the object tracking unit 160 compares and determines whether or not the road safety sign object detected in the before and after image frames of the road driving image is matched, and if it is matched, it is assigned to the road safety sign object. Provided ID, detailed information and lane information in the form of a queue (S760).

More specifically, in the step S760, after confirming whether or not a road sign object has been detected and ID assigned in the n+1 th image frame from the object analyzer 140, the center of the n th image frame and the center of the n+1 th image frame After calculating the optimal value of the offset vector of , determining whether or not the road sign object is matched based on the optimal value of the offset vector, stacking image frames in which the same road sign object is detected, and then stacking the stacked value exceeds the reference value. The bottom includes a series of processes of storing the road safety sign object ID, segment, object detection area (bounding box) and lane information of the stacked image frames in the form of a queue and providing them to an external server.

Therefore, using the traffic safety sign recognition/extraction method and system for generating a high-precision road map according to an embodiment of the present invention, by minimizing detection errors by subdividing the detection area of a road safety sign object detected in a road driving image, However, it can improve the discrimination of the photographed traffic safety signs, and provides the advantage of early detection of detection error information.

13 is a diagram illustrating an example computing environment in which one or more embodiments set forth herein may be implemented, an illustration of a system 1000 that includes a computing device 1100 configured to implement one or more embodiments described above. shows For example, computing device 1100 may be vehicle-mounted, a personal computer, server computer, handheld or laptop device, mobile device (mobile phone, PDA, media player, etc.), multiprocessor system, consumer electronic device, minicomputers, mainframe computers, distributed computing environments that include any of the foregoing systems or devices, and the like.

Computing device 1100 may include at least one processing unit 1110 and memory 1120 . Here, the processing unit 1110 may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Arrays (FPGA), and the like. and may have a plurality of cores. The memory 1120 may be volatile memory (eg, RAM, etc.), non-volatile memory (eg, ROM, flash memory, etc.), or a combination thereof. Additionally, computing device 1100 may include additional storage 1130 . Storage 1130 includes, but is not limited to, magnetic storage, optical storage, and the like. The storage 1130 may store computer readable instructions for implementing one or more embodiments disclosed herein, and may also store other computer readable instructions for implementing an operating system, application programs, and the like. Computer readable instructions stored in storage 1130 may be loaded into memory 1120 for execution by processing unit 1110 . Computing device 1100 can also include input device(s) 1140 and output device(s) 1150 .

Here, input device(s) 1140 may include, for example, a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, or any other input device. Output device(s) 1150 may also include, for example, one or more displays, speakers, printers, or any other output devices, or the like. Additionally, computing device 1100 may use an input device or output device included in another computing device as input device(s) 1140 or output device(s) 1150 . Computing device 1100 may also include communication connection(s) 1160 that allow computing device 1100 to communicate with other devices (eg, computing device 1300).

Here, communication connection(s) 1160 may be a modem, network interface card (NIC), integrated network interface, radio frequency transmitter/receiver, infrared port, USB connection, or other device for connecting computing device 1100 to other computing devices. May contain interfaces. Further, communication connection(s) 1160 may include a wired connection or a wireless connection. Each component of the aforementioned computing device 1100 may be connected by various interconnections such as a bus (eg, peripheral component interconnection (PCI), USB, firmware (IEEE 1394), optical bus structure, etc.) and may be interconnected by the network 1200. Terms such as "component" and "system" as used herein generally refer to a computer-related entity that is hardware, a combination of hardware and software, software, or software in execution.

For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. For example, both the application running on the controller and the controller may be components. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer or distributed between two or more computers.

The present invention is not limited by the foregoing embodiments and accompanying drawings. It will be clear to those skilled in the art that the components according to the present invention can be substituted, modified, and changed without departing from the technical spirit of the present invention.

100: Traffic safety sign recognition/extraction system for generating maps with precision
110: input unit
120: lane detection unit
130: road safety sign object detection unit
140: object analysis unit
150: ID allocation unit
160: object tracking unit

Claims (20)

an input unit for receiving a road driving image;
a lane detection unit that detects a lane within an image frame of the road driving image using a deep learning algorithm, and then analyzes and classifies information of the detected lane;
After generating an integrated object detection area by merging and expanding the detection areas of each of at least two or more deep learning object detection algorithms that detect image objects with different detection methods, the lane is detected according to the preset detection area condition value a road safety sign object detector detecting a road safety sign object located in the integrated object detection area within an image frame;
an object analyzer that analyzes detailed shapes based on feature points extracted according to color, saturation, and brightness of the detected road safety sign object; and
An object tracking unit that compares and determines whether the road safety sign objects detected in the front and rear image frames are matched, and if matched, provides the ID assigned to the road safety sign object and detailed lane information in the form of a queue,
The object tracking unit
After confirming whether or not a road sign object has been detected and assigned an ID in the n+1 th image frame from the object analyzer, an optimal value of an offset vector between the center of the n th image frame and the center of the n+1 th image frame is calculated. , determining whether or not the road sign object is matched based on the optimum value of the offset vector,
Accuracy of storing and transmitting the road safety sign object ID, segment, object detection area, and lane information of the stacked image frames in a queue form after stacking image frames in which the same road safety sign object is detected and the stacking value exceeds the reference value Traffic safety sign recognition/extraction system for log map creation. According to claim 1,
The traffic safety sign recognition/extraction system for generating a map with precision, further comprising a detection ID allocator for allocating a detection ID for identifying the road safety sign object detected in the image frame to the image frame. delete According to claim 1,
The lane detection unit
After detecting lanes using a deep learning algorithm, traffic safety sign recognition/recognition for generating maps with precision classifying the lanes into one of dotted lines, solid lines, exclusive lanes, and center lines from the road surface and average width between the detected lanes extraction system. According to claim 1,
The object detection unit
Traffic safety sign recognition for precision map creation including an object detection area adjustment unit that overlaps the object detection area of each of the at least two deep learning object detection algorithms to generate the integrated object detection area exceeding a preset overlap value /extract system. According to claim 5,
The object detection unit
A traffic safety sign recognition/extraction system for generating a high-precision map including an object detection area filtering unit for filtering the presence or absence of identification of road safety sign objects detected in the integrated object detection area with a preset detection area condition value. According to claim 6,
The preset detection area condition value is
An area value for a detection position of the road safety sign object, wherein the area value includes a road surface area, a left area of a center line, and a right curb area. According to claim 5,
The object detection unit
When the value obtained by overlapping the object detection area of each of the at least two or more deep learning object detection algorithms is smaller than the preset overlap value,
A traffic safety sign recognition/extraction system for generating maps with precision for detecting the road safety sign object with a deep learning object detection algorithm having high detection accuracy among the at least two or more deep learning object detection algorithms. According to claim 1,
The object analysis unit
After extracting the outline of the road safety sign object detected by the object detection unit, based on the location of feature points of the outline, at least one of the shape and color of the road safety sign object is analyzed to create a traffic safety sign for map generation with precision. Recognition/extraction system. delete delete After detecting lanes within the image frame of the road driving image using a deep learning algorithm, analyzing and classifying information on the detected lanes;
After creating an integrated object detection area by merging and extending the object detection areas of at least two or more deep learning object detection algorithms, the integrated object detection area is detected in the image frame in which the lane is detected according to the preset detection area condition value. Detecting a road safety sign object located thereon;
Analyzing an image of a road safety sign object based on feature points extracted according to color, saturation, and brightness of the detected road safety sign object;
Allocating a detection ID for identifying a road safety sign object detected in the image frame to an image frame; and
Comparing and judging the matching of the road safety sign objects detected in the front and back image frames, and then providing the ID assigned to the road safety sign object and detailed lane information in the form of a queue when matching,
The step of providing in the form of a queue
After stacking the image frames in which the same road safety sign object is detected, storing and transmitting the road safety sign object ID, segment, object detection area, and lane information of the stacked image frames in a queue form when the stacking value exceeds the reference value Traffic safety sign recognition/extraction method for map generation with phosphorus precision. According to claim 12,
Analyzing and classifying the information of the detected lane
After detecting the lane using the deep learning algorithm, traffic for generating a precision road map is a step of classifying the lane into one of a dotted line, a solid line, an exclusive lane, and a center line from the road surface and average width between the detected lanes. Safety mark recognition/extraction method. According to claim 12,
Detecting the road safety sign object
A method of recognizing/extracting traffic safety signs for generating maps with precision, comprising overlapping the object detection areas of each of the at least two or more object detection algorithms to generate the integrated object detection area exceeding a preset overlap value. According to claim 14,
Detecting the road safety sign object
A method for recognizing/extracting traffic safety signs for generating maps with precision, comprising the step of filtering the identification of road safety sign objects detected in the integrated object detection area with a preset detection area condition value. According to claim 15,
The preset detection area condition value is
An area value for a detection position of the road safety sign object, wherein the area value includes a road surface area, a left area of a center line, and a right curb area. According to claim 12,
Detecting the road safety sign object
When the value obtained by overlapping the object detection area of each of the at least two or more deep learning object detection algorithms is smaller than the preset overlap value,
Detecting the road safety sign object with a deep learning object detection algorithm having high detection accuracy among the at least two deep learning object detection algorithms. According to claim 12,
Analyzing the image of the road safety sign object
After extracting the contour of the detected road safety mark object, based on the position of feature points of the contour, at least one of the shape and color of the road safety mark object is analyzed for a traffic safety sign for generating a map with precision. Recognition/extraction method. According to claim 12,
The step of comparing and determining the matching of the road safety sign object
After confirming whether or not a road sign object is detected and assigned an ID in the n+1 th image frame from the object analyzer, an optimal value of an offset vector between the center of the n th image frame and the center of the n+1 th image frame is calculated, A traffic safety sign recognition/extraction method for generating a map with precision, which is a step of determining whether the road sign object matches on the basis of an optimal value of an offset vector. delete

Images