TWI842321B - Lane detection method and lane detection device - Google Patents

Abstract

The present invention provides a lane detection method and a lane detection device. The lane detection method comprises 1) acquiring a first image, which comprises an image of at least one vehicle; 2)obtaining contour and orientation of at least one vehicle according to the first image, and the orientation is the direction of the front of at least one vehicle; 3)obtaining track of at least one vehicle according to the contour and orientation of at least one vehicle, and the track is moving track of at least one vehicle; 4)obtaining lane information according to the track of at least one vehicle. The lane detection method provided in this invention can obtain lane information according to the acquired vehicle image, thus assisting in lane line detection.

Description

Lane detection method and lane detection device

The present invention relates to the field of computer technology, and specifically to a lane detection method and a lane detection device.

Lane detection is an important perception task in the autonomous driving system. Lane detection can help complete many upper-level applications. For example, it can help systems such as Lane Departure Warning (LDW) and Lane Keep Assist (LKA). It can also assist the Forward Collision Warning (FCW) to determine the nearest vehicle on the path to explain the vehicle path planning. However, due to wind and sand erosion, rain erosion, road construction or aging of the road, the lane lines on the road are easily worn or even disappear. In this way, it will become quite difficult to directly detect the lane lines on the road by relying solely on image detection technology.

In response to the above problems, this application provides a lane detection method and a lane detection device, which can obtain lane information through collected vehicle information to assist in lane detection.

The first aspect of the present application provides a lane detection method, which is applied to a vehicle, comprising: obtaining a first image, the first image including an image of at least one vehicle; obtaining the outline and orientation of at least one vehicle based on the first image, the orientation being the front direction of at least one vehicle; obtaining the track of at least one vehicle based on the outline and orientation of at least one vehicle, the track being the moving track direction of at least one vehicle; obtaining lane information based on the track of at least one vehicle.

In some embodiments, when the first image is a single-frame image and the first image includes images of at least two vehicles, obtaining the trajectory of at least one vehicle according to the outline and orientation of at least one vehicle includes: obtaining the center points of at least two vehicles according to the outlines of at least two vehicles; establishing at least one trajectory grouping according to the center points and orientations of at least two vehicles; obtaining the trajectory of at least two vehicles according to at least one trajectory grouping.

In some embodiments, at least one track group is established based on the center points and orientations of at least two vehicles, including: selecting one of the at least two vehicles as a calibration vehicle, the calibration vehicle being the vehicle with the smallest y-axis coordinate in the first image among the at least two vehicles and not assigned to the track group; taking the center of the calibration vehicle as the origin and the orientation of the calibration vehicle as the direction vector, obtaining the track extension line of the calibration vehicle; determining whether the track extension line of the calibration vehicle intersects with the rear image of the other vehicle; if the track extension line of the calibration vehicle intersects with the rear image of the other vehicle, If the images intersect, the calibration vehicle and the vehicle that intersects with the calibration vehicle's extended track are assigned to a track group, and the vehicle that intersects with the calibration vehicle's extended track is used as the new calibration vehicle, and the calibration vehicle's center is used as the origin and the calibration vehicle's orientation is used as the direction vector to obtain the calibration vehicle's extended track; if the calibration vehicle's extended track does not intersect with the rear image of other vehicles, return to select one of at least two vehicles as the calibration vehicle; repeat the above steps until all vehicles are assigned to at least one track group.

In some embodiments, the method further includes: determining whether the trajectory group is an isolated group according to the number of vehicles in the trajectory group, wherein the isolated group includes a vehicle; if the trajectory group is an isolated group, determining whether the trajectory of the vehicle in the isolated group intersects with the trajectory of other vehicles or whether the distance between the trajectory of the vehicle in the isolated group and the trajectory of other vehicles is less than a preset distance value; If the trajectory of the vehicle in the isolated group intersects with the trajectory of other vehicles or the distance between the trajectory of the vehicle in the isolated group and the trajectory of other vehicles is less than a preset distance value, deleting the isolated group.

In some embodiments, when the first image includes at least two frames and the first image includes images of at least two vehicles, the method further includes: selecting a preset number of frames in the first image; selecting a calibration image frame, the calibration image frame being the earliest frame in the preset number of frames; obtaining the outlines of at least two vehicles and the center points of at least two vehicles according to the calibration image frame; At least one track group is established based on the center point and orientation of the vehicle, and the tracks of at least two vehicles are obtained based on at least one track group; after the tracks of all vehicles in the calibration image frame are obtained, return to select the calibration image frame; repeat the above steps until the tracks of all vehicles in the image of the preset number of frames are obtained; and the lane information is obtained based on the tracks of all vehicles in the image of the preset number of frames.

In some embodiments, when the first image is a single-frame image and the first image includes a vehicle, obtaining the trajectory of at least one vehicle based on the outline and orientation of the at least one vehicle includes: obtaining the center point of the vehicle based on the outline of the vehicle; obtaining the trajectory of the vehicle based on the center point of the vehicle and the orientation of the vehicle.

In some embodiments, when the first image includes at least two frames of images and the first image includes a vehicle, obtaining the trajectory of at least one vehicle according to the outline and orientation of the at least one vehicle includes: obtaining timing information of at least two frames of images; obtaining the center point of the vehicle according to the outline of the vehicle; obtaining the trajectory of the vehicle according to the timing information, the center point of the vehicle and the orientation of the vehicle.

In some embodiments, the trajectory of the vehicle is obtained according to the timing information, the center point of the vehicle and the orientation of the vehicle, including: obtaining the orientation of the vehicle in at least two frames of images according to the timing information; determining whether the angle between the orientations of the vehicle in adjacent frames of images is greater than a preset angle value, wherein the timing of the adjacent frames of images is adjacent; if the angle is greater than the preset angle value, the trajectory of the vehicle is obtained according to a curve regression algorithm, the center point of the vehicle and the orientation of the vehicle; if the angle is less than or equal to the preset angle value, the trajectory of the vehicle is obtained according to a linear regression algorithm, the center point of the vehicle and the orientation of the vehicle.

In some embodiments, the method further includes: obtaining a second image, the second image including a lane line; obtaining first lane information based on the second image; and obtaining second lane information based on the first lane information and the lane information.

The second aspect of the present application also provides a lane detection device, which is applied to a vehicle, comprising: an image acquisition module, the image acquisition module is used to acquire a first image, the first image includes an image of at least one vehicle; a contour acquisition module, connected to the image acquisition module, the contour acquisition module is used to acquire the contour and orientation of at least one vehicle according to the first image, the orientation being at least The front direction of a vehicle; a track acquisition module, connected to the contour acquisition module, the track acquisition module is used to acquire the track of at least one vehicle according to the contour and direction of at least one vehicle, and the track is the moving track direction of at least one vehicle; a lane acquisition module, connected to the track acquisition module, the lane acquisition module is used to acquire lane information according to the track of at least one vehicle.

This application obtains the outline and orientation of at least one vehicle on the first image, and then determines the trajectory of at least one vehicle, thereby preliminarily obtaining lane information based on the trajectory of at least one vehicle, thereby assisting in lane detection.

S1-S4, S401-S403, S501-S506, S801-S803, S101-S106, S121-S122, S141-S143: Steps

101: Vehicles

102: Terminal equipment

103: Image acquisition equipment

104: Radar equipment

100: Lane detection device

10: Image acquisition module

20: Outline acquisition module

30:Trajectory acquisition module

40: Lane acquisition module

200: Electronic equipment

201: Processor

202: Memory

203: Computer Programs

Figure 1 is a system architecture diagram of the lane detection method provided in this application.

Figure 2 is a schematic diagram of the process of the lane detection method provided in this application.

Figure 3 is a schematic diagram of the overall detection frame provided in an embodiment of this application.

Figure 4 is a schematic diagram of the process of the sub-steps of step S2 in Figure 2 under the first scenario.

Figure 5 is a schematic diagram of the process of the sub-steps of step S402 in Figure 4.

Figures 6A-6C are schematic diagrams of the process of track grouping in an embodiment of the present application.

Figure 7 is a schematic diagram of the track grouping of an embodiment of this application.

Figure 8 is a schematic diagram of the process of determining isolated groups in an embodiment of this application.

Figures 9A-9C are schematic diagrams of three isolated groups in an embodiment of this application.

Figure 10 is a schematic diagram of the process of the sub-steps of step S2 in Figure 2 in the second scenario.

Figure 11 is a schematic diagram of the overlay of image frames at different time sequences in an embodiment of the present application.

Figure 12 is a schematic diagram of the process of the sub-steps of step S2 in Figure 2 under the third scenario.

Figure 13 is a schematic diagram of a vehicle track in an embodiment of the present application.

Figure 14 is a schematic diagram of the process of the sub-steps of step S2 in Figure 2 under the fourth scenario.

FIG15A is a schematic diagram showing the merging of the first images at different time sequences of an embodiment of the present application.

FIG. 15B is a schematic diagram showing the merging of first images at different time sequences in another embodiment of the present application.

Figure 16 is a schematic diagram of the module of the lane detection device provided in an embodiment of this application.

Figure 17 is a schematic diagram of the structure of the electronic device that uses the present invention to implement the lane detection method.

The following will combine the attached figures in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary skilled persons in this field without creative labor are within the scope of protection of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art in the technical field of the present invention. The terms used herein in the specification of the present invention are only for describing specific embodiments and are not intended to limit the present invention. The term "and/or" used herein includes any and all combinations of one or more related listed items.

The following is a detailed description of some implementation methods of the present invention in conjunction with the attached figures. The following embodiments and features in the embodiments can be combined with each other without conflict.

In current high-speed serial bus protocols, the clock signal is usually included in the data, and there is no need for an independent clock signal data line to be transmitted in parallel with the data data line. In this way, in order to correctly sample the data signal at the receiving end, the clock signal in the data signal needs to be extracted at the receiving end, which is the clock recovery process.

Lane detection is an important perception task in the autonomous driving system. Lane detection can help complete many upper-level applications. For example, it can help systems such as Lane Departure Warning (LDW) and Lane Keep Assist (LKA). It can also assist the Forward Collision Warning (FCW) to determine the nearest vehicle on the path to explain the vehicle path planning. However, due to wind and sand erosion, rain erosion, road construction or aging of the road, the lane lines on the road are easily worn or even disappear. In this way, it will become quite difficult to directly detect the lanes on the road by relying solely on image detection technology.

To this end, this application provides a lane detection method for obtaining lane information to assist in lane detection.

First, the system architecture involved in this application embodiment is introduced.

FIG1 is a system architecture diagram of a lane detection method provided in an embodiment of the present application. The system includes a vehicle 101, a terminal device 102, an image acquisition device 103, and a radar device 104. The terminal device 102 is connected to the image acquisition device 103 and the radar device 104 for communication, so that the image acquisition device 103 and the radar device 104 can send the data collected by each to the terminal device 102. The vehicle 101 is also connected to the terminal device 102 for communication, so that the terminal device 102 can send control instructions to the vehicle 101.

The specific terminal device 102 can be installed inside the vehicle 101, or it can be a terminal device currently carried by the driver. The terminal device 102 includes a processor for data information processing. Through the processor, the terminal device 102 can process the data collected by the image acquisition device 103 or the radar device 104, thereby determining the lane information corresponding to the vehicle 101. In addition, the terminal device 102 can also provide an interface for human-computer interaction, through which the current road conditions and route planning map and other information can be displayed to the user.

The image acquisition device 103 is installed on the outside of the vehicle 101. Specifically, multiple image acquisition devices 103 can be installed around the body of the vehicle 101. For example, four image acquisition devices 103 can be installed around the body of the vehicle 101, and the four image acquisition devices can be used as a front view image acquisition device, a rear view image acquisition device, a left view image acquisition device, and a right view image acquisition device. Among them, the front view image acquisition device is installed at the center of the front of the vehicle, the rear view image acquisition device is installed at the center of the rear of the vehicle, the left view image acquisition device is installed at the midpoint of the left side of the vehicle along the length direction, and the right view image acquisition device is installed at the midpoint of the right side of the vehicle along the length direction. It is worth noting that the above is only an example of four image acquisition devices. In actual application, more or fewer image acquisition devices 103 may be installed around the body of the vehicle 101. The image acquisition device 103 can collect images of the road conditions around the vehicle 101 while the vehicle 101 is driving. By processing and analyzing the collected road condition images, obstacles, lane lines, signal lights, and signal light stop lines around the vehicle 101 can be detected. That is, by processing and analyzing the collected road condition images, preliminary obstacle detection results, lane line detection results, signal light detection results, and stop line detection results can be obtained.

The radar device 104 is installed on the outside of the vehicle 101. Specifically, at least one radar device 104 can be installed at the center of the top of the vehicle 101. Alternatively, multiple radar devices 104 can be installed at different positions on the top of the vehicle 101, for example, four radar devices 104 can be installed at the four corners of the top of the vehicle 101. The radar device 104 can detect obstacles, especially the position and movement parameters of moving obstacles can be accurately detected. It can be understood that obstacles can be other vehicles or mobile devices traveling around the vehicle 101.

It should be noted that the terminal device 102 may be a vehicle-mounted terminal device or other mobile terminal device currently inside the vehicle 101. For example, the terminal device 102 may be a terminal such as an industrial computer, a portable computer, a smart phone, a tablet computer, etc. The image acquisition device 103 may be a camera capable of image acquisition or a camera, etc. For example, the image acquisition device 103 may be a fisheye surround camera. The radar device 104 may be a laser radar and/or a millimeter wave radar.

Optionally, the terminal device 102 in the above system architecture may be replaced by a vehicle control unit (VCU). In this case, the method steps provided in the following embodiments may be applied to the vehicle 101 and executed by the vehicle control unit of the vehicle 101.

Please continue to refer to Figure 2, which is a schematic diagram of the process of the lane detection method provided in an embodiment of the present application. The following embodiment is explained by taking the lane detection method applied to vehicle 101 and executed by the vehicle control unit of vehicle 101 as an example.

In this application, the lane detection method includes the following steps:

Step S1: Obtain a first image, the first image includes an image of at least one vehicle.

In the embodiment of the present application, multiple initial images are obtained by the image acquisition device 103 and/or the radar device 104, and the multiple initial images are processed to filter out the first image. The initial image is an image of the surrounding area or the road section of the vehicle 101 acquired by the image acquisition device 103 and/or the radar device 104. The initial image can be a picture acquired by the image acquisition device 103 or video data including multiple frames of pictures. The initial image can also be a picture generated based on the point cloud data acquired by the radar device 104.

In some embodiments, based on image detection technology or visual recognition technology, an initial image including at least one vehicle can be selected from the multiple initial images collected by the image acquisition device 103 as the first image. This application does not limit the specific image detection technology.

In other embodiments, based on target detection and other technologies, the point cloud data collected by the radar device 104, i.e., the initial image, can be used to filter out an image including at least one vehicle as the first image.

Specifically, the vehicles mentioned in step S1 are vehicles located around vehicle 101, for example, other vehicles located on the same road as vehicle 101.

Step S2: Obtain the outline and orientation of at least one vehicle based on the first image, where the orientation is the front direction of at least one vehicle.

Please refer to Figure 3. Specifically, the detection model constructed by training the known deep learning network technology in this application processes each vehicle displayed on the first image to generate the overall detection frame and the front detection frame (or rear detection frame) of the corresponding vehicle on each first image. It can be understood that the front detection frame is opposite to the side where the overall detection frame is located, that is, the direction of the front of the vehicle is the direction of the corresponding vehicle. The rear detection frame is opposite to the side where the overall detection frame is located, which is the direction of the corresponding vehicle.

In this application, the overall detection frame and the front detection frame (or rear detection frame) are three-dimensional (3D) detection frames, for example, they can be approximately rectangular detection frames. In this way, the position of the 3D detection frame can be represented by the coordinates of 8 corner points (C1-C8 in Figure 3), or by 4 corner points, the height value of the bottom of the vehicle to the ground, and the height value between the bottom of the vehicle and the top of the vehicle (not shown).

It can be understood that compared with the two-dimensional detection frame, the 3D detection frame focuses on the positioning and identification of the target object in the 3D coordinate system in the real world. The geometric information on the 3D detection frame can be used to measure the distance between the vehicle 101 and other key targets (such as other vehicles on the same lane). In this way, the present application is conducive to establishing a connection between the data obtained based on the image and the real environment data by obtaining the 3D detection frame of the vehicle as the vehicle outline.

Step S3: Obtain the trajectory of at least one vehicle based on the contour and orientation of at least one vehicle, where the trajectory is the moving trajectory direction of at least one vehicle.

Specifically, in this step, the coordinates of the center point (X0, Y0) of each vehicle are calculated based on the obtained outline of each vehicle, that is, the overall detection frame. Then, the trajectory of the vehicle is formed based on the center point and orientation of the vehicle. The trajectory represents the moving trajectory direction of at least one vehicle on the first image.

In this application, the vertex of the lower left corner of the first image is taken as the origin, the axis extending rightward from the origin of the first image is taken as the X axis of the first image, and the axis extending upward from the origin of the first image is taken as the Y axis.

In some embodiments, the trajectory of at least one vehicle may be represented by a curve formed by connecting the center point coordinates of multiple vehicles.

Furthermore, in order to improve the accuracy of lane detection, in some embodiments, a preset number of first images within a continuous period of time may be obtained to form a longer moving trajectory of at least one vehicle within a period of time; in other embodiments, the moving trajectories of multiple vehicles in the same first image may be obtained to form a longer moving trajectory formed by multiple vehicles at the same time point.

Step S4: Obtain lane information based on the trajectory of at least one vehicle.

Specifically, the corresponding curve equation can be obtained based on the center point coordinates of the vehicle that forms the track in step S3.

It can be understood that lanes usually restrict the driving trajectory and driving direction of vehicles. In other words, the driving trajectory and driving direction of vehicles on the same lane are usually roughly the same. In this way, a curve analysis can be performed on the trajectory of at least one vehicle to obtain the curve equation of the trajectory as lane information such as the lane trajectory.

Furthermore, the direction of the lane can be confirmed based on the direction of the vehicle track; the number of lanes can also be determined based on the number of vehicle tracks detected in the first image.

It can be understood that the lane detection method provided by this application obtains the outline and orientation of at least one vehicle on the first image, and then determines the track of at least one vehicle, thereby preliminarily obtaining lane information based on the track of at least one vehicle, thereby assisting in realizing lane detection.

Furthermore, in some embodiments, the lane detection method also includes:

Step S5: Obtain a second image, and the second image includes a lane line; obtain first lane information based on the second image; obtain second lane information based on the first lane information and the lane information.

In step S5, the second image may be an image including lane lines that is selected from a plurality of initial images. The first information may include partial lane line information, etc. The second lane information may be more complete lane information output after the lane information is combined with the first lane information, for example, including at least one of lane line markings, arrows, or lane quantity information.

In this way, the lane detection method provided by this application can initially obtain lane information based on the obtained vehicle track, and then combine and output the second lane information based on the obtained first lane information, so as to enrich the lane information to a greater extent.

Furthermore, the following content of this application further explains the detailed processing process of the above step S2 based on the different first images obtained.

Please refer to FIG. 4. In some embodiments, when the first image is a single-frame image and the first image includes images of at least two vehicles, such as FIG. 6A, step S2 further includes:

Step S401: Obtain the center points of at least two vehicles based on the outlines of at least two vehicles.

Step S402: Establish at least one track grouping according to the center points and directions of at least two vehicles; please refer to Figure 5 for details, step S402 further includes:

Step S501: Select one of at least two vehicles as a calibration vehicle, such as vehicle a (see FIG6B ). The calibration vehicle is the vehicle with the smallest y-axis coordinate in the first image and not assigned to a track group among the at least two vehicles.

Step S502: Take the center of the calibration vehicle as the origin and the orientation of the calibration vehicle as the direction vector to obtain the trajectory extension line of the calibration vehicle.

In some embodiments, the center point coordinates of the front detection frame can be obtained first, and then the center point coordinates of the whole detection frame and the center point coordinates of the front detection frame are connected as a direction vector, and the center point coordinates of the whole detection frame are the origin of the direction vector, and the direction of the direction vector is the same as the direction of the side where the front detection frame is located. In other embodiments, the center point coordinates of the rear detection frame can also be obtained first, and then the center point coordinates of the whole detection frame and the center point coordinates of the rear detection frame are connected as a direction vector, and the center point coordinates of the whole detection frame are the origin of the direction vector, and the direction of the direction vector is opposite to the direction of the side where the rear detection frame is located.

Step S503: Determine whether the extended track line of the calibrated vehicle intersects with the rear image of other vehicles;

Step S504: If the extended track of the calibration vehicle (e.g., vehicle a in FIG. 6B ) intersects with the rear image of another vehicle (e.g., vehicle b in FIG. 6B ), the calibration vehicle and the vehicle intersecting with the extended track of the calibration vehicle are assigned to a track group, and the vehicle intersecting with the extended track of the calibration vehicle is assigned to a track group. As a new calibration vehicle, re-calibrate the center of the calibration vehicle as the origin and the orientation of the calibration vehicle as the direction vector to obtain the trajectory extension line of the calibration vehicle; for example, please refer to Figure 6C, when the trajectory extension line of vehicle b intersects with the rear of vehicle c, then vehicles a, b and c can be regarded as a trajectory group.

Step S505: If the extended track line of the calibration vehicle does not intersect with the rear image of other vehicles, then return to select one of at least two vehicles as the calibration vehicle;

Step S506: Repeat the above steps until all vehicles are assigned to at least one track group.

For example, referring to FIG. 7, according to the above steps S501-S506, the vehicle in the first image shown in FIG. 6A can be assigned to three track groups.

Step S403: Obtain the trajectories of at least two vehicles according to at least one trajectory grouping.

In some embodiments, step S403 also includes:

Step S801: Determine whether the track group is an isolated group based on the number of vehicles in the track group, wherein an isolated group includes a vehicle.

For example, referring to Figures 9A to 9C, the track groups where vehicles d, e, and f are located form isolated groups respectively.

Step S802: If the track group is an isolated group, determine whether the track of the vehicle in the isolated group intersects with the track of other vehicles or whether the distance between the track of the vehicle in the isolated group and the track of other vehicles is less than a preset distance value.

Step S803: If the track of the vehicle in the isolated group intersects with the track of other vehicles or the distance between the track of the vehicle in the isolated group and the track of other vehicles is less than a preset distance value, the isolated group is deleted.

It can be understood that when the track of the vehicles in the isolated group intersects with the track of the vehicles in other groups, or the distance between the track of the vehicles in the isolated group and the track of other vehicles is lower than a preset distance value, it means that the vehicles in the isolated group may move from one lane to another. Therefore, the vehicles in the current isolated group are not in a normal lane driving state, and the isolated group can be deleted.

When the tracks of the vehicles in an isolated group do not meet the above conditions, it means that the vehicles in the isolated group are most likely driving on a normal lane.

Thus, through the above steps, the first image of a single frame including at least two vehicles can be processed to obtain a track grouping similar to that shown in FIG. 7 or FIG. 9C. Thus, by respectively obtaining the curve equation for the center point coordinates of each vehicle in the above track grouping, the curve of the lane on the first image, the number of lanes and the direction of the lane can be preliminarily obtained.

Taking Figure 9C as an example, when the three curves obtained on the three track groups in Figure 9C are k1, k2 and k3, it means that the current road may have three lanes, and the directions of the three lanes are roughly the same, and the curves of the three lanes may be k1, k2 and k3. It can be understood that in some embodiments, the above-obtained curves k1, k2 and k3 may also be transformed accordingly for the need of a unified coordinate system, and this application is not limited to this.

Further, please continue to refer to FIG. 10 , when the first image includes at least two frames of images, and the first image includes images of at least two vehicles, the method further includes:

Step S101: Select an image with a preset number of frames in the first image.

Step S102: Select the calibration image frame, which is the earliest frame in the image sequence among the preset number of frames.

For example, please refer to Figure 11, in step S101, the first frame image and the second frame image in the first image are selected.

It can be understood that when the image acquisition device 103 and the radar device 104 acquire the initial image, the initial image is marked with a corresponding time stamp. In this way, the timing information can be obtained by respectively obtaining the time stamps of the initial image corresponding to the first frame image and the second frame image.

Step S103: Obtain the outlines of at least two vehicles and the center points of at least two vehicles according to the calibration image frame, establish at least one track grouping according to the center points and directions of at least two vehicles, and obtain the tracks of at least two vehicles according to at least one track grouping.

It can be understood that the process of establishing track groups in step S103 is the same as the process of establishing track groups in step S402, and will not be repeated here.

Step S104: After obtaining the tracks of all vehicles in the calibration image frame, return to select another image frame as the calibration image frame;

Step S105: Repeat the above steps until the trajectories of all vehicles in the image of the preset number of frames are obtained.

Step S106: Obtain lane information based on the tracks of all vehicles in the image of a preset number of frames.

In step S106, two tracks whose preset distance is less than the preset value are merged. In this way, by merging the tracks in multiple frames of images, the distance of the farthest lane in multiple frames of images can be comprehensively calculated.

Further, please continue to refer to FIG. 12. When the first image is a single-frame image and the first image includes a vehicle (see FIG. 13), the trajectory of at least one vehicle is obtained according to the outline and orientation of at least one vehicle, including:

Step S121: Obtain the center point of the vehicle based on the outline of the vehicle.

Step S122: Obtain the trajectory of the vehicle based on the center point of the vehicle and the direction of the vehicle.

It can be understood that the curve equation of the vehicle's trajectory obtained in step S122 is a straight line equation (Y=mN+n). At this time, step S5 can be combined to further improve the accuracy of lane detection.

Further, please continue to refer to Figure 14. When the first image includes at least two frames of images and the first image includes a vehicle, the trajectory of at least one vehicle is obtained according to the outline and orientation of at least one vehicle, including:

Step S141: Obtain timing information of at least two frames of images.

Step S142: Obtain the center point of the vehicle based on the outline of the vehicle.

Step S143: Obtain the trajectory of the vehicle based on the timing information, the center point of the vehicle and the direction of the vehicle.

The specific step S143 also includes the following steps: Acquire the orientation of the vehicle in at least two frames of images according to the timing information; determine whether the angle between the orientations of the vehicle in adjacent frames of images is greater than a preset angle value, wherein the timing of the adjacent frames of images is adjacent; if the angle is greater than the preset angle value, obtain the trajectory of the vehicle according to the curve regression algorithm, the center point of the vehicle and the orientation of the vehicle; if the angle is less than or equal to the preset angle value, obtain the trajectory of the vehicle according to the linear regression algorithm, the center point of the vehicle and the orientation of the vehicle.

For example, please refer to Figure 15A. Vehicle g, vehicle h, and vehicle i in Figure 15A are images of the same vehicle at three consecutive time points.

In Figure 15A, the angle between the orientation of vehicle g and vehicle h is greater than the preset angle value, and the angle between vehicle h and vehicle i is greater than the preset angle value. Thus, in Figure 15A, the lane may have curved, and the orientation of the vehicle has changed significantly, resulting in a larger angle between the orientations of the vehicle in different time series images. Therefore, the trajectory of the vehicle is obtained by using the curve regression algorithm, the center point of the vehicle, and the orientation of the vehicle.

In Figure 15B, the angle between the orientations of vehicle j and vehicle k is less than or equal to the preset angle value. Therefore, in Figure 15B, the lane may be approximately a straight line, and the orientation of the vehicle has not changed significantly, making the angle between the orientations of the vehicles in different time series images smaller. Therefore, the trajectory of the vehicle is obtained by using the curve regression algorithm, the center point of the vehicle and the orientation of the vehicle.

Further referring to FIG. 16 , the second embodiment of the present application also provides a lane detection device 100 that can be applied to a mobile vehicle. The lane detection device 100 includes an image acquisition module 10, a contour acquisition module 20, a track acquisition module 30, and a lane acquisition module 40.

The image acquisition module 10 is used to acquire a first image. The first image includes an image of at least one vehicle.

The contour acquisition module 20 is connected to the image acquisition module 10. The contour acquisition module 20 is used to acquire the contour and orientation of at least one vehicle according to the first image, and the orientation is the front direction of at least one vehicle.

The trajectory acquisition module 30 is connected to the contour acquisition module 20. The trajectory acquisition module 30 is used to acquire the trajectory of at least one vehicle according to the contour and orientation of at least one vehicle, and the trajectory is the moving trajectory direction of at least one vehicle.

The lane acquisition module 40 is connected to the track acquisition module 30. The lane acquisition module 40 is used to acquire lane information based on the track of at least one vehicle.

It can be understood that the image acquisition module 10, the contour acquisition module 20, the track acquisition module 30 and the lane acquisition module 40 are used to execute the steps in the embodiments corresponding to Figures 1 to 16. Please refer to the relevant description of the previous embodiment for details, which will not be repeated here.

Please refer to Figure 17. The third embodiment of the present application also provides an electronic device 200. The electronic device 200 includes a processor 201 and a memory 202. The memory 202 stores a computer program 203. The computer program 203 includes at least one instruction. The processor 201 is used to execute at least one instruction stored in the memory 202 to implement steps S1 to S5 in the above-mentioned bit pose adjustment method embodiment.

Furthermore, the computer program 203 can be divided into one or more units, one or more units are stored in the memory 202, and executed by the processor 201 to complete the present invention. One or more units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 203 in the electronic device. For example, the computer program 203 can be divided into an image acquisition unit, a contour acquisition unit, a track acquisition unit, and a lane acquisition unit, and the specific functions of each unit are as follows: The image acquisition unit is used to acquire a first image. The first image includes an image of at least one vehicle.

The contour acquisition unit is connected to the image acquisition unit. The contour acquisition unit is used to acquire the contour and orientation of at least one vehicle according to the first image, wherein the orientation is the front direction of at least one vehicle.

The trajectory acquisition unit is connected to the contour acquisition unit. The trajectory acquisition unit is used to acquire the trajectory of at least one vehicle according to the contour and orientation of at least one vehicle, and the trajectory is the moving trajectory direction of at least one vehicle.

The lane acquisition unit is connected to the track acquisition unit. The lane acquisition unit is used to acquire lane information based on the track of at least one vehicle.

It is understandable that the electronic device 200 may include, but is not limited to, a processor 201 and a memory 202. Those skilled in the art may understand that FIG. 3 is only an example of the electronic device 200 and does not constitute a limitation on the electronic device 200. The electronic device 200 may include more or fewer components than shown, or may combine certain components, or different components. For example, the electronic device 200 may also include input and output devices, network access devices, buses, etc.

It can be understood that the processor 201 can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or processor 201 can also be any conventional processor, etc. Processor 201 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

The memory 202 can be used to store computer programs and/or modules/units. The processor 201 realizes various functions of the electronic device by running or executing the computer programs and/or modules/units stored in the memory 202 and calling the data stored in the memory 202. The memory 202 can mainly include a program storage area and a data storage area. Among them, the program storage area can store the operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created according to the use of the electronic device (such as video data, audio data, phone book, etc.). In addition, the memory 202 may include high-speed random access memory and non-volatile memory, such as a hard drive, memory, a plug-in hard drive, a smart media card (SMC), a secure digital (SD) card, a flash memory card (Flash Card), at least one disk memory device, a flash memory device, or other volatile solid-state memory devices.

Another embodiment of the present application also provides a computer-readable storage medium, which stores a computer program including at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above posture adjustment method.

The exemplary computer program may be divided into one or more modules/units, one or more modules/units are stored in the memory 202 and executed by the processor 201 to complete the present invention. One or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

The present invention can implement all or part of the processes in the above-mentioned embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical signal and software distribution medium, etc. It should be noted that the content of computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

In the several embodiments provided by the present invention, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the electronic device embodiments described above are only schematic. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

In addition, each functional module in each embodiment of the present invention can be integrated into the same processing module, each module can be physically stored separately, or two or more modules can be integrated into the same module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of hardware plus software functional modules.

It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments and can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. In addition, it is obvious that the word "including" does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or electronic devices described in the electronic device can also be implemented by the same module or electronic device through software or hardware. The words first, second, etc. are used to indicate names and do not indicate any specific order.

The above implementations are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention is described in detail with reference to the above preferred implementations, those with ordinary skills in this field should understand that the technical solution of the present invention can be modified or replaced equivalently without departing from the spirit and scope of the technical solution of the present invention. Those with ordinary skills in this field can also make other changes within the spirit of the present invention for the design of the present invention, as long as they do not deviate from the technical effects of the present invention. Such changes made in accordance with the spirit of the present invention should be included in the scope of protection required by the present invention.

S1-S4: Steps

Claims (9)

A lane detection method is applied to a vehicle, wherein the method comprises: obtaining a first image, wherein the first image comprises an image of at least one vehicle; obtaining the outline and orientation of the at least one vehicle according to the first image, wherein the orientation is the direction of the front of the at least one vehicle; obtaining the track of the at least one vehicle according to the outline and orientation of the at least one vehicle, wherein the track is the direction of the moving track of the at least one vehicle; obtaining a lane detection method according to the .... Obtaining lane information; wherein, when the first image is a single-frame image and the first image includes images of at least two vehicles, obtaining the trajectory of the at least one vehicle according to the outline and orientation of the at least one vehicle includes: obtaining the center point of the at least two vehicles according to the outline of the at least two vehicles; establishing at least one trajectory grouping according to the center point and orientation of the at least two vehicles; obtaining the trajectory of the at least two vehicles according to the at least one trajectory grouping. A lane detection method as described in claim 1, wherein the step of establishing at least one track grouping based on the center points and orientations of the at least two vehicles comprises: a. selecting one of the at least two vehicles as a calibration vehicle, wherein the calibration vehicle is the vehicle with the smallest y-axis coordinate in the first image among the at least two vehicles and which is not assigned to the track grouping; b. obtaining the track extension line of the calibration vehicle with the center of the calibration vehicle as the origin and the orientation of the calibration vehicle as the direction vector; c. determining whether the track extension line of the calibration vehicle intersects with the rear image of other vehicles; d. determining whether the track extension line of the calibration vehicle intersects with the rear image of other vehicles if the track extension line of the calibration vehicle intersects with the rear image of other vehicles; and If the rear image of the calibration vehicle intersects with the rear image of the other vehicle, the calibration vehicle and the vehicle intersecting with the trajectory extension line of the calibration vehicle are assigned to one of the trajectory groups, the vehicle intersecting with the trajectory extension line of the calibration vehicle is used as the new calibration vehicle, and the calibration vehicle is re-performed with the center of the calibration vehicle as the origin and the orientation of the calibration vehicle as the direction vector to obtain the trajectory extension line of the calibration vehicle; e. If the trajectory extension line of the calibration vehicle does not intersect with the rear image of other vehicles, the trajectory extension line of the calibration vehicle is used as a trajectory group; repeat the above steps a to e until all the vehicles are assigned to at least one of the trajectory groups. The lane detection method as described in claim 2, wherein the method further comprises: judging whether the track group is an isolated group according to the number of the vehicles in the track group, wherein the isolated group includes a vehicle; if the track group is the isolated group, judging whether the track of the vehicle in the isolated group intersects with the track of other vehicles or whether the distance between the track of the vehicle in the isolated group and the track of other vehicles is less than a preset distance value; if the track of the vehicle in the isolated group intersects with the track of other vehicles or the distance between the track of the vehicle in the isolated group and the track of other vehicles is less than a preset distance value, deleting the isolated group. The lane detection method as described in claim 1, wherein when the first image includes at least two frames and the first image includes images of at least two vehicles, the method further includes: f. selecting a preset number of frames in the first image; g. selecting a calibration image frame, the calibration image frame being the earliest frame in the preset number of frames; h. obtaining the outlines of the at least two vehicles and the center points of the at least two vehicles according to the calibration image frame; Establish at least one track group based on the center points and orientations of the at least two vehicles, and obtain the tracks of the at least two vehicles based on the at least one track group; i. After obtaining the tracks of all the vehicles in the calibration image frame, return to the selected calibration image frame; repeat the above steps f to i until the tracks of all the vehicles in the image of the preset number of frames are obtained; obtain the lane information based on the tracks of all the vehicles in the image of the preset number of frames. The lane detection method as described in claim 1, wherein when the first image is a single-frame image and the first image includes a vehicle, the step of obtaining the trajectory of the at least one vehicle based on the outline and orientation of the at least one vehicle includes: obtaining the center point of the vehicle based on the outline of the vehicle; obtaining the trajectory of the vehicle based on the center point of the vehicle and the orientation of the vehicle. The lane detection method as described in claim 1, wherein when the first image includes at least two frames of images and the first image includes a vehicle, the step of obtaining the trajectory of the at least one vehicle based on the outline and orientation of the at least one vehicle includes: obtaining timing information of the at least two frames of images; obtaining the center point of the vehicle based on the outline of the vehicle; obtaining the trajectory of the vehicle based on the timing information, the center point of the vehicle and the orientation of the vehicle. The lane detection method as described in claim 6, wherein the obtaining of the trajectory of the vehicle based on the timing information, the center point of the vehicle and the orientation of the vehicle comprises: obtaining the orientation of the vehicle in the at least two frames of images based on the timing information; determining whether the angle between the orientations of the vehicles in adjacent frames of images is greater than a preset angle value, wherein The time sequence of the adjacent frame images is adjacent; if the angle is greater than the preset angle value, the trajectory of the vehicle is obtained according to the curve regression algorithm, the center point of the vehicle and the direction of the vehicle; if the angle is less than or equal to the preset angle value, the trajectory of the vehicle is obtained according to the linear regression algorithm, the center point of the vehicle and the direction of the vehicle. The lane detection method as described in claim 1, wherein the method further comprises: obtaining a second image, wherein the second image comprises a lane line; obtaining first information of the lane according to the second image; and obtaining second lane information according to the first information of the lane and the lane information. A lane detection device is applied to a vehicle, and the improvement is that it includes: an image acquisition module, the image acquisition module is used to acquire a first image, the first image includes an image of at least one vehicle; a contour acquisition module, connected to the image acquisition module, the contour acquisition module is used to acquire the contour and orientation of the at least one vehicle according to the first image, the orientation is the front direction of the at least one vehicle; a track acquisition module, connected to the contour acquisition module, the track acquisition module is used to acquire the track of the at least one vehicle according to the contour and orientation of the at least one vehicle, the track is the movement of the at least one vehicle Dynamic trajectory direction; a lane acquisition module connected to the trajectory acquisition module, the lane acquisition module is used to acquire lane information according to the trajectory of the at least one vehicle; wherein, when the first image is a single-frame image and the first image includes images of at least two vehicles, the acquisition of the trajectory of the at least one vehicle according to the outline and orientation of the at least one vehicle includes: acquiring the center point of the at least two vehicles according to the outline of the at least two vehicles; establishing at least one trajectory grouping according to the center point and orientation of the at least two vehicles; and acquiring the trajectory of the at least two vehicles according to the at least one trajectory grouping.