WO2018101526A1 - Method for detecting road region and lane by using lidar data, and system therefor - Google Patents

Abstract

A method for detecting a road region and a lane by using lidar data, and a system therefor are disclosed. Provided is a method implemented by a computer, comprising the steps of: obtaining point type lidar data from a multi-channel lidar sensor mounted on a vehicle; and detecting a road region in which the vehicle can travel, by using a gradient of a lidar point between neighboring channels in a lidar point obtained from the multi-channel lidar sensor.

Description

Method and system for detecting road area and lane using lidar data

The following description relates to a technology for detecting a road area and a lane in which a vehicle can travel.

Recently, vehicle driving assistance systems such as free running and unmanned driving have been researched and developed. Therefore, there is a need to detect a lane drawn on a road for a vehicle driving assistance system, and various methods have been studied together.

In the conventional case, a lane detection method may be classified into a model-based, feature point-based, and area-based detection method. As an example of lane detection technology, Korean Patent Laid-Open Publication No. 10-2011-0046607 (published May 06, 2011) uses a HSV color model and an edge model to extract a lane feature from a camera image and then extracts a bay from the lane feature. A technique for detecting a lane through a decision rule is disclosed.

Conventionally, in order to detect a lane in a camera image, a complicated image processing of several steps must be performed, and computational time is required according to a relatively large number of processing procedures. In addition, since the camera cannot acquire any data such as the presence and type of obstacles in both a dark environment where light does not exist and a very strong light environment, lane detection is difficult according to external environmental conditions.

Recently, research on lane detection technology based on lidar (laser radar) data has been conducted.

Provided are methods and systems that can accurately detect a substantial driveable area of a roadway using lidar data.

Provided is a method and system that can accurately detect lanes on a driveable area of a road using lidar data.

A computer-implemented method comprising the steps of: acquiring point-type lidar data from a multichannel lidar sensor mounted on a vehicle; And detecting a road area in which the vehicle may travel using the inclination of the lidar point between neighboring channels at the lidar point acquired from the multi-channel lidar sensor.

According to an aspect, the acquiring may include acquiring a lidar point of a radial form according to an omnidirectional scan through the multichannel lidar sensor.

According to another aspect, the detecting of the road area may include: calculating an inclination of a lidar point between neighboring channels for each scan angle of the multichannel lidar sensor; And detecting the road area by classifying a rider point having the slope less than or equal to a predetermined threshold value.

According to another aspect, the step of calculating the inclination, it is possible to calculate the inclination of the lidar point between the neighboring channels in a direction away from the vehicle from the lidar point scanning the closest to the vehicle.

According to another aspect, the threshold value may be set according to a road surface requirement including a shape of a road or irregularities.

According to another aspect, the method may further include detecting a lane in the road area by using the reflection intensity of a lidar point included in the road area.

According to another aspect, the detecting of the lane may detect the lane by classifying a lidar point having a reflection intensity of a predetermined value or more among the lidar points included in the road area.

According to another aspect, the detecting of the lane may include: classifying a lidar point having a reflection intensity of a predetermined value or more among lidar points included in the road area; Setting a straight line parameter based on the position and driving direction of the vehicle; And classifying a lidar point included in a line of the parameter or located within a set range among the classified lidar points into a lane area.

According to another aspect, the setting of the straight line parameter may include: setting a straight line parameter corresponding to a driving direction of the vehicle as an initial value on both sides of a lane width based on the position of the vehicle; And updating the straight line parameter by using a Lidar point classified into the lane area.

According to another aspect, the method may further include calculating and providing a Lidar point detected as the lane as a coordinate value to be displayed on a map.

In a computer program coupled to a computer system and recorded on a computer-readable recording medium for executing a road area and lane detection method, the road area and lane detection method is obtained from a multichannel lidar sensor mounted on a vehicle. Acquiring LiDAR data in the form of points; Detecting a road area in which the vehicle can travel using the inclination of the rider points between neighboring channels at the rider points acquired from the multichannel rider sensors; And detecting a lane on the corresponding road area by classifying a rider point corresponding to a linear parameter set based on the position and driving direction of the vehicle among the rider points included in the road area. do.

A computer-implemented system, comprising: at least one processor configured to execute computer-readable instructions, the at least one processor being configured to perform an omni-directional scan from a multichannel lidar sensor mounted on a vehicle. A rider data acquisition unit for obtaining rider data in the form of points; And a road area detector configured to detect a road area in which the vehicle may travel by using an inclination of a rider point between neighboring channels at a rider point obtained from the multichannel rider sensor.

According to the embodiments of the present invention, the slope or partial loss of the road itself is detected by detecting the road region based on the inclination between points using LiDAR data, instead of simply detecting the road region satisfying the plane condition. In spite of various road surface requirements such as bumps or speed bumps caused by renovation work, the driving area on the road can be more robustly detected.

According to embodiments of the present invention, by using the parameters of a straight line on the basis of the position and the driving direction of the vehicle, the lane is classified in the lane capable of driving on the road by classifying the lidar data on the road included in the straight line into lanes. It can be accurately distinguished from other road surface markings such as arrows, crosswalks, crosswalk notices, stop lines, and the like.

1 is a block diagram illustrating an example of an internal configuration of a computer system according to an embodiment of the present invention.

2 illustrates an example of components that may be included in a processor of a computer system according to an exemplary embodiment of the present invention.

3 is a flowchart illustrating an example of a road area and a lane detection method which may be performed by a computer system according to an exemplary embodiment of the present invention.

4 illustrates an example of a multichannel lidar sensor in one embodiment of the present invention.

FIG. 5 is an exemplary diagram for describing a process of classifying a road area that may be driven according to an embodiment of the present invention.

6 is an exemplary diagram for describing a process of classifying road markings according to an exemplary embodiment of the present invention.

FIG. 7 is an exemplary diagram for describing a process of classifying a lane among road markings according to an exemplary embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to a technique for detecting a road area and a lane capable of driving a vehicle using lidar data.

Embodiments, including those specifically disclosed herein, can achieve accurate road area and lane detection using lidar data, thereby achieving significant advantages in terms of accuracy, efficiency, speed, cost savings, and the like. .

1 is a block diagram illustrating an example of an internal configuration of a computer system according to an embodiment of the present invention. For example, a road area and lane detection system according to embodiments of the present invention may be implemented through the computer system 100 of FIG. 1. As shown in FIG. 1, the computer system 100 is a processor 110, a memory 120, a permanent storage device 130, a bus 140, and input / output as components for executing a road area and lane detection method. It may include an interface 150 and a network interface 160.

The processor 110 may include or be part of any device capable of processing a sequence of instructions as a component for detecting a road area and lane capable of driving using lidar data. Processor 110 may include, for example, a processor within a computer processor, mobile device or other electronic device, and / or a digital processor. The processor 110 may be included in, for example, a server computing device, a server computer, a series of server computers, a server farm, a cloud computer, a content platform, and the like. The processor 110 may be connected to the memory 120 through the bus 140.

Memory 120 may include volatile memory, permanent, virtual, or other memory for storing information used by or output by computer system 100. The memory 120 may include, for example, random access memory (RAM) and / or dynamic RAM (DRAM). Memory 120 may be used to store any information, such as status information of computer system 100. The memory 120 may also be used to store instructions of the computer system 100, including, for example, instructions for detecting a driveable area of a road and a lane. Computer system 100 may include one or more processors 110 as needed or where appropriate.

Bus 140 may include a communication infrastructure that enables interaction between various components of computer system 100. Bus 140 may carry data, for example, between components of computer system 100, for example, between processor 110 and memory 120. Bus 140 may include wireless and / or wired communication media between components of computer system 100 and may include parallel, serial, or other topology arrangements.

Persistent storage 130 is a component, such as a memory or other persistent storage, such as used by computer system 100 to store data for some extended period of time (eg, relative to memory 120). Can include them. Persistent storage 130 may include non-volatile main memory as used by processor 110 in computer system 100. Persistent storage 130 may include, for example, flash memory, hard disk, optical disk, or other computer readable medium.

The input / output interface 150 may include interfaces for a keyboard, mouse, voice command input, display, or other input or output device. Configuration commands and / or inputs for road area and lane detection may be received via the input / output interface 150.

Network interface 160 may include one or more interfaces to networks such as a local area network or the Internet. Network interface 160 may include interfaces for wired or wireless connections. Configuration commands and / or input for road area and lane detection may be received via network interface 160.

In addition, in other embodiments, computer system 100 may include more components than the components of FIG. 1. However, it is not necessary to clearly show most of the prior art components. For example, the computer system 100 may be implemented to include at least some of the input / output devices connected to the input / output interface 150 described above, or may include a transceiver, a global positioning system (GPS) module, a camera, various sensors, It may further include other components such as a database.

Embodiments of the present invention relate to a technology for detecting a road area and a lane that can be driven using LiDAR data, which may be applied to a vehicle driving assistance system such as free driving or unmanned driving, but is not limited thereto.

2 is a diagram illustrating an example of components that may be included in a processor of a computer system according to an embodiment of the present invention, and FIG. 3 is a road area that may be performed by the computer system according to an embodiment of the present invention. And an example of a lane detection method.

As illustrated in FIG. 2, the processor 110 may include a LiDAR data acquisition unit 210, a road area detection unit 220, a lane detection unit 230, and a lane coordinate providing unit 240. The components of such a processor 110 may be representations of different functions performed by the processor 110 in accordance with a control instruction provided by at least one program code. For example, the lidar data acquisition unit 210 can be used as a functional representation that the processor 110 operates to control the computer system 100 to acquire lidar data. The processor 110 and the components of the processor 110 may perform steps S310 to S350 included in the road area and lane detection method of FIG. 3. For example, the processor 110 and the components of the processor 110 may be implemented to execute instructions according to the code of the operating system included in the memory 120 and the at least one program code described above. Here, the at least one program code may correspond to a code of a program implemented to process the road area and the lane detection method.

The road area and lane detection methods may not occur in the order shown, and some of the steps may be omitted or additional processes may be included.

In operation S310, the processor 110 may load program code stored in a program file for a road area and a lane detection method, into the memory 120. For example, the program file for the road area and the lane detection method may be stored in the persistent storage 130 described with reference to FIG. 1, and the processor 110 may store the program stored in the persistent storage 130 through a bus. Computer system 110 may be controlled such that program code from a file is loaded into memory 120. At this time, each of the processor 110 and the lidar data acquisition unit 210, the road area detection unit 220, the lane detection unit 230, and the lane coordinate providing unit 240 included in the memory 120 is included in the memory 120. It may be different functional representations of the processor 110 for executing instructions of a corresponding portion of the program code loaded in to execute subsequent steps S320 to S350. In order to execute the steps S320 to S350, the processor 110 and the components of the processor 110 may directly process an operation according to a control command or control the computer system 100.

In step S320, the lidar data acquisition unit 210 performs a point-type lidar data according to an omnidirectional scan around the vehicle from the multichannel lidar sensor through a wired or wireless connection with the multichannel lidar sensor mounted in the vehicle. (Hereinafter referred to as 'lidar point') can be obtained. The lidar data acquisition unit 210 may control a multi-channel lidar sensor having a scan angle of 360 degrees to acquire a lidar point according to the omnidirectional scan. The lidar point may include a value indicating a distance from the lidar sensor and a value indicating the reflection intensity.

In the multi-channel lidar sensor used in the present invention, laser scanners as many as the number of channels are arranged in a line in the up and down direction to shoot light and return the light to measure distance and reflection intensity for each channel. In this case, the multi-channel lidar sensor has a mirror that reflects light in the sensor for omnidirectional scanning, so that the mirror rotates 360 degrees to measure the distance and reflection intensity of the omnidirectional surrounding the vehicle. When the vehicle equipped with the multi-channel lidar sensor scans around the vehicle while driving, the lidar data acquisition unit 210 may acquire lidar points having the same scan angle at the same time as the number of channels. As the mirror rotates within the sensor, ie the scanning angle changes, the LiDa points accumulate at a high rate.

The degree of denseness of the scan angle is different for each lidar sensor. For example, as shown in FIG. 4, when using the multi-channel lidar sensor 40 in which the number of channels is 32 and the scan angle is changed by 1.33 degrees, the lidar data is used. The acquisition unit 210 may acquire 32 LiDa points in units of 1.33 degrees through the multi-channel LiDAR sensor 40. The lidar points acquired through 32 channels while the mirror rotates 360 degrees in the multi-channel lidar sensor 40 form a radial shape according to the omnidirectional scan around the sensor position.

In FIG. 3, in step S330, the road area detection unit 220 has a rider point having an inclination less than or equal to a threshold value based on the inclination of the rider points between neighboring channels at the rider point acquired from the multichannel rider sensor. By classifying, it is possible to detect a road area that can travel. As an example of a method of classifying a road area on which a vehicle may travel using lidar data, the road area detection unit 220 may radiate in a direction away from the lidar points scanned by the lidar sensor nearest to the vehicle. The inclination between the lidar points can be calculated sequentially. The road area detector 220 may calculate the inclination between the lidar points scanned in the neighboring channels for the same point in time and the same scan angle among the lidar points in order to calculate the inclination between the lidar points in a radial form. The road area detection unit 220 calculates the inclination of the rider points between neighboring channels from the rider point nearest to the position of the rider sensor, and the rider points between neighboring channels from the position of the rider sensor. The road region may be classified to a point before the slope of the crossover exceeds a predetermined threshold value. The road area detection unit 220 may classify the road area for 360 degrees by repeating the above method for each scan angle.

5 is an exemplary diagram for describing a process of classifying a road area according to an embodiment of the present invention. Assuming that the vehicle 50 equipped with the multi-channel lidar sensor is currently located on the road, the lidar points in the radial form according to the 360-degree omnidirectional scan about the vehicle 50 through the multi-channel lidar sensor 501 can be obtained.

When an obstacle such as another vehicle or curb exists near the vehicle 50, the inclination of the rider points between neighboring channels increases rapidly due to the obstacle. Most roads have convex shapes with high central parts and relatively low edges for drainage, and are not detected as road areas despite the fact that there are some irregularities due to partial damage or repair work. There can be. In order to solve this problem, a threshold value, which is a criterion for classifying road areas, may be set in consideration of road requirements such as the shape of a road and general unevenness.

Referring to FIG. 5, the road area detector 220 calculates the inclination of the lidar point X between neighboring channels at the same scan angle 51, and when the inclination exceeds a specific threshold, the scan angle 51. ) Can be classified into the road area 503 from the position of the vehicle 50 to the lidar point just before the threshold is exceeded, and this process is repeated for each scan angle to form the road area for 360 degrees. Can be classified.

Therefore, in the present invention, when the road area is classified, the inclination between the points is calculated in a radial form around the sensor position (that is, the vehicle), and when the distribution of the points is maintained below a specific inclination, it may be classified as a road area that can be driven. This method still classifies some of the irregularities caused by the convex shape of a typical road or partial breakage or renovation into the road area, but it can be classified as a low but abrupt slope change point such as a road curb, a road divider, a sight bar, or a parked area. After the point where obstacles such as other vehicles are located, it is possible to effectively classify the non-driving area.

3, the lane detection unit 230 classifies the rider points corresponding to the linear parameters set based on the position and the driving direction of the vehicle among the rider points classified as the road areas in step S340. Lanes can be detected. The lidar point may include a value indicating a reflection intensity, together with a value indicating a distance to the multichannel lidar sensor. Various road markings such as direction arrows, crosswalks, pedestrian crossing notices, stop lines, etc., including lanes in the road area are made by painting with paint, and the paint on the road markings has a certain intensity of reflection over the lidar. do.

6 to 7 are exemplary diagrams for describing a process of detecting a lane according to an embodiment of the present invention.

Referring to FIG. 6, the lane detection unit 230 includes a rider point corresponding to a road surface display of rider points 605 having a reflection intensity of a predetermined value or more, among the rider points classified as the road area 503. Can be classified as

Subsequently, the lane detecting unit 230 first classifies the position of the current lidar sensor, that is, the vehicle 50, in order to classify the lidar point corresponding to the lane among the lidar points 605 corresponding to the road surface display, as shown in FIG. 7. The straight line parameter 707 extending in the driving direction of the vehicle 50 may be set as an initial value at both sides of the standard space to the lane width of the standard. In addition, the lane detector 230 may classify the lidar points included in the line of the linear parameter 707 among the lidar points 605 corresponding to the road surface display or located within a predetermined range as the lane area.

The lane detection unit 230 may update the straight line parameter 707 by using a Lidar point classified into the lane area. The lane detection unit 230 may repeatedly apply an prediction-maximization algorithm that updates the linear parameter 707 with a newly found lidar point in the lane area.

The surrounding environment of the vehicle 50 changes as the vehicle 50 travels until the omnidirectional scan of the next cycle is made, and since the position of the lane is likely to be similar to the previous position, the lidar data included in the previous straight line parameter is included. Repeat the classification and update of straight line parameters. Using this method, even if the vehicle 50 changes lanes while driving, the lanes may be continuously detected based on the updated straight line parameters while continuously updating the straight line parameters as the reference for lane detection. According to the performance of the lidar sensor, it is possible to stably classify four lanes corresponding to three lanes at a time, and since the parameter update occurs frequently in the scan area close to the lidar sensor, processing for the curved lane is not necessary. .

Referring back to FIG. 3, in operation S350, the lane coordinate providing unit 240 may calculate coordinates corresponding to Lidar points classified as lanes and provide them as coordinate values for lane marking on a map. In this case, the lane coordinate providing unit 240 may store LiDAR points classified into lane areas based on a straight line parameter and calculate points connecting them at regular intervals, and calculate GPS coordinates corresponding to the calculated points. Can be specified as a coordinate value for lane marking on the map. In other words, the lane coordinate providing unit 240 may calculate and provide a point divided into lanes as a set of GPS coordinate points to be expressed as a lane on the map.

As described above, according to the embodiments of the present invention, the road area may be detected by detecting the road area based on the inclination between points using lidar data, instead of simply detecting the road area satisfying the plane condition. In spite of various road requirements such as bumps or speed bumps caused by losses or repairs, the roadside area can be detected more robustly. Furthermore, according to embodiments of the present invention, by using the parameters of the straight line based on the position and the driving direction of the vehicle, the lidar data on the road included in the straight line is classified as a lane, and the lane in the available area on the road. Can be accurately distinguished from other road surface markings such as directional arrows, crosswalks, crosswalk notices, stop lines, and the like.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable PLU (programmable). It can be implemented using one or more general purpose or special purpose computers, such as logic units, microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. It can be embodied in. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. In this case, the medium may be to continuously store a program executable by the computer, or to temporarily store for execution or download. In addition, the medium may be a variety of recording means or storage means in the form of a single or several hardware combined, not limited to a medium directly connected to any computer system, it may be distributed on the network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions. In addition, examples of another medium may include a recording medium or a storage medium managed by an app store that distributes an application, a site that supplies or distributes various software, a server, or the like.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (11)

In a computer implemented method, Obtaining lidar data in the form of points from a multichannel lidar sensor mounted on a vehicle; And Detecting a road area on which the vehicle can travel using the inclination of the rider points between neighboring channels at the rider points acquired from the multi-channel rider sensors How to include. The method of claim 1, The acquiring step is Acquiring a radial lidar point according to an omnidirectional scan through the multichannel lidar sensor Characterized by the above. The method of claim 1, Detecting the road area, Calculating the inclination of the lidar points between neighboring channels for each scan angle of the multichannel lidar sensor; And Detecting the road area by classifying a rider point whose slope is less than or equal to a predetermined threshold value How to include. The method of claim 3, The step of calculating the inclination, Calculating a slope of a rider point between neighboring channels in a direction away from the vehicle from a rider point scanning the closest point to the vehicle Characterized by the above. The method of claim 3, The threshold value is set according to the road surface requirements including the shape of the road or the unevenness. Characterized by the above. The method of claim 1, Detecting a lane in the road area by using a reflection intensity of a lidar point included in the road area How to include more. The method of claim 6, Detecting the lane, Detecting the lane by classifying a lidar point having a reflection intensity of a predetermined value or more among lidar points included in the road area; Characterized by the above. The method of claim 6, Detecting the lane, Classifying a lidar point having a reflection intensity of a predetermined value or more among lidar points included in the road area; Setting a straight line parameter based on the position and driving direction of the vehicle; And Classifying a lidar point included in a line of the parameter or located within a set range among the classified lidar points into a lane area; How to include. The method of claim 8, Setting the parameters of the straight line, Setting a straight line parameter corresponding to a driving direction of the vehicle as an initial value on both sides of a lane width based on the position of the vehicle; And Updating the straight line parameters using lidar points classified into the lane area. How to include. The method of claim 6, Calculating and providing a lidar point detected as the lane as a coordinate value to be displayed on a map How to include more. In a computer implemented system, At least one processor implemented to execute computer-readable instructions Including, The at least one processor, A lidar data acquisition unit for acquiring point-based lidar data according to an omnidirectional scan from a multichannel lidar sensor mounted in a vehicle; And A road area detection unit detects a road area in which the vehicle can travel using the inclination of the rider points between neighboring channels at the rider points acquired from the multichannel rider sensors. System comprising a.

Images