CN113353067A - Multi-environment detection and multi-mode matching parallel parking path planning system based on panoramic camera - Google Patents
Multi-environment detection and multi-mode matching parallel parking path planning system based on panoramic camera Download PDFInfo
- Publication number
- CN113353067A CN113353067A CN202110795200.0A CN202110795200A CN113353067A CN 113353067 A CN113353067 A CN 113353067A CN 202110795200 A CN202110795200 A CN 202110795200A CN 113353067 A CN113353067 A CN 113353067A
- Authority
- CN
- China
- Prior art keywords
- parking
- pose
- vehicle
- path
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/06—Automatic manoeuvring for parking
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2552/00—Input parameters relating to infrastructure
- B60W2552/50—Barriers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2555/00—Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
- B60W2555/20—Ambient conditions, e.g. wind or rain
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application discloses a multi-environment detection and multi-mode matching parallel parking path planning system based on a panoramic camera. The method comprises the following steps: the method comprises the steps of acquiring image information by adopting a liftable image acquisition device installed on a roof, analyzing parameters of a vehicle body and a parking environment through the image information, classifying environment information such as identified road information and parking spaces, carrying out modal matching according to the environment type, and planning a parking path. According to the embodiment of the invention, the lifting catadioptric camera is used for realizing information acquisition of the surrounding environment, the lifting height can be adjusted according to different vehicle types, the camera splicing problem is solved, and the cost is reduced; obtaining vehicle body parameters in a self-adaptive mode; in addition, the path is planned by using a multi-environment and multi-mode matching mode, different path planning modes are selected according to different modes, and the application range of parallel parking is enlarged.
Description
Technical Field
The invention relates to the technical field of automatic parking, in particular to a parallel parking path planning method and system.
Background
The automatic parallel parking technology is an important component of the automatic parking technology and is also a technology which is difficult to park. The parking assisting device can assist a driver to realize rapid parking, reduce accidents caused by thought factors such as vehicle collision and the like, and reduce the skill requirement of the driver. Nowadays, many manufacturers at home and abroad are equipped with automatic parking systems in mass-produced vehicle models.
The existing automatic parallel parking technology has the following defects:
1. at present, most automatic parallel parking systems use multiple cameras to splice and acquire surrounding environment information, and have the disadvantages of high cost, complex structure, high difficulty in multi-camera correction and low imaging quality;
2. the parking space and the road width are strictly required, and normal parking can not be realized under the conditions of narrow parking spaces and narrow roads;
3. in the face of a scene of parking lot beauty, if a vehicle to be parked exists behind the vehicle, the vehicle can be driven into the parking lot in advance easily by adopting a normal parking technology, and the parking efficiency is greatly reduced.
Disclosure of Invention
In view of the above, the present invention is directed to a system for planning parallel parking paths based on multi-environment detection and multi-mode matching of a panoramic camera to overcome the above problems or to solve the above problems in part
The purpose of the invention can be realized by the following technical scheme:
a multi-environment detection and multi-mode matching parallel parking path planning system based on a panoramic camera comprises: an image acquisition device; an image processing module; an environment detection module; a data module; a modality classification and matching module; a path planning module;
the method comprises the following steps: when a user executes a parking command, the liftable image acquisition device installed on the roof ascends to acquire image information, vehicle body parameters and a parking environment are analyzed through the image information, environment information such as identified road information and parking spaces is classified, modal matching is carried out according to the environment type, and a parking path is planned. The lifting catadioptric camera is used for acquiring information of surrounding environment, the lifting height can be adjusted according to different vehicle types, the camera splicing problem is solved, and the cost is reduced; obtaining vehicle body parameters in a self-adaptive mode; in addition, the path is planned by using a multi-environment and multi-mode matching mode, different path planning modes are selected according to different modes, and the application range of parallel parking is enlarged.
Optionally, the catadioptric panoramic camera is configured to acquire vehicle and environment image information, where the vehicle information image includes vehicle shape, size, color, and the like, and the environment information includes an environment information image within an effective viewing angle range around the vehicle; an image processing unit for developing the image information into a distortion-free top view.
Optionally, the described fixing means comprise a lifting device and a bottom fixing base for being placed on the roof, in a fixing position and in the lifting device; the lifting device is used for realizing the vertical lifting of the panoramic camera on the roof of the vehicle.
The image processing unit comprises an image unfolding unit, and the image processing unit is used for unfolding the image into a top view; and the distortion correction unit is used for calibrating the camera based on the top view, correcting the top view by using the correction parameters, and correcting the top view by using a bilinear interpolation method according to a homography matrix to obtain a distortion-free top view.
Further, the expanding step S5 includes:
s51, according to the optical reflection and principle and the geometric principle, setting the generated target top view as IMG _ VERT, the size of the generated target top view as (H, W), and acquiring the panorama as IMG _ SRC, the size of which is (H-SRC)0,W0) Panoramic camera mirror surface parameters
The focal length of the camera is f;
s52, establishing a side coordinate system XOY on a two-dimensional plane by taking the focus of the reflector of the panoramic camera as a center;
s53. order
Setting the coordinates of any point pixel point on the top view of the target as (x, y), calculating the interpolation point (x, y) of the panoramic image corresponding to the top view pixel point (x, y)0,y0)，
S54, the expansion result is IMG _ VERT (x, y) ═ IMG _ SRC (x)0,y0)。
Optionally, the distortion correction unit corrects the distortion of the top view. Further, the step S6 includes:
s61, mounting the panoramic camera and the fixing device on the top end of the vehicle body, enabling the camera to be perpendicular to the ground as far as possible, placing camera calibration devices in front of, behind, on the left and on the right of the vehicle body, respectively moving the four calibration devices and ensuring that the four calibration devices are within a camera shooting range, and storing image data;
s62, according to the top view expanding method of claim 5, expanding a top view IMG _ VERT of the stored image data;
s63, acquiring distortion parameters and relative external parameters according to the top view IMG _ VERT image;
and S64, carrying out distortion parameter correction and rotation transformation on the subsequent image to enable the image to be vertical to the ground.
And a target classifier is arranged in the detection module and used for classifying the image sample into information of parking spaces, road lines and other obstacles of rear vehicles. The detection step S8 includes:
s81, obtaining the surrounding environment information of the vehicle, and obtaining parking space information and road information according to the surrounding environment information of the vehicle. The parking space information comprises parking space parameters of each parking space and barrier information in the parking space, and the road information comprises road width parameters and whether rear vehicle coming parameters exist or not;
s82, storing the parking space information and the width parameters of the roads adjacent to the parking space information according to the information of the obstacles in the parking space;
s83, updating the stored parking space information of each parking space and the width parameters of the adjacent roads in real time according to the motion condition of the vehicle;
s84, determining whether parking can be performed and the distance between the parking vehicle and the parking space according to the parking space parameters of each parking space and the obstacle information in the parking space which are updated in real time;
and S85, classifying the detected parking environment according to the real-time environment information.
Optionally, the data module is configured to estimate vehicle data and includes: vehicle length, vehicle width, wheel base, front suspension, rear suspension, maximum equivalent front wheel rotation angle and maximum equivalent front wheel rotation angle rotation speed. The data module calculates the length and width of the vehicle by using the image information; the vehicle attitude change is calculated using the adjacent image information, and then vehicle data is calculated.
Optionally, the modality classification and matching module includes, but is not limited to, a single-step parking modality M1, a multi-step parking modality M2, and a parking space preemption modality M3. And simultaneously matching the parking environment with the parking mode according to the set environment classification.
Optionally, the parking path is a path between an initial parking pose and a target parking pose according to the initial pose of the vehicle, the target parking pose and the intermediate pose information, and the path is planned in a segmented manner according to the intermediate pose, so as to obtain a target driving path. Wherein the target travel path is for the vehicle to travel from the start pose to the target parking pose.
The intermediate posture refers to a vehicle posture that the vehicle can run out of the garage without collision at the maximum steering wheel angle, and at least part of the vehicle is located in the garage where the target parking space is located when the vehicle is located in the intermediate posture;
the target parking attitude refers to the vehicle attitude that the vehicle center coincides with the target parking space center, and the vehicle body is parallel to the parking space.
The path segment between the initial parking pose and the middle pose is composed of two one-way arc segments, and the condition that the vehicle does not collide with the two ends of the parking space, the parking space line on the inner side and the road boundary in the path process between the initial parking pose and the middle pose is met.
And the path segment between the intermediate pose and the target parking pose consists of a plurality of circular arcs and a plurality of straight track segments, and the condition that the vehicle does not collide with the two ends of the parking space and the inner parking space line in the process of the path between the initial parking pose and the intermediate pose is met.
Optionally, the path target parking pose generated by the single-step parking mode coincides with the intermediate pose, that is, the path planning on the target pose is not required to be performed by the intermediate pose.
Optionally, the path target parking pose generated by the single-step parking mode is not coincident with the intermediate pose, that is, the intermediate pose is required to perform path planning on the target pose.
Optionally, the initial parking pose of the parking space preemption mode path is located behind the target parking space, the path segment planning between the initial parking poses by the intermediate pose is performed according to the initial parking pose, the intermediate pose and the preemption pose information, and the path segment between the initial parking pose and the intermediate pose by the intermediate pose is obtained by performing segment division according to the initial parking pose, the intermediate pose and the preemption pose information.
And when the vehicle is positioned in the preemption pose, at least part of the vehicle is positioned in a garage where the target parking space is positioned.
And according to the initial parking pose, the preempted pose and the steering pose, carrying out segmentation division according to the initial parking pose, the steering pose and the preempted pose to obtain a path segment between the initial parking pose and the preempted pose.
Wherein the steering pose is an intermediate vehicle pose at which the vehicle can travel from the initial parking pose without collision to the preemption pose state, and at least a portion of the vehicle is located within a garage in which the target parking space is located when the vehicle is in the steering pose.
And the path between the initial parking pose and the steering pose and the preempting pose are segmented, and the path tracks of the vehicles are all unidirectional arc tracks.
The invention discloses the following technical effects: the invention uses a catadioptric panoramic camera vertically placed on the roof to obtain the images of the vehicle body and the environment, realizes self-adaption to obtain vehicle data according to vehicle control and image information, divides the images into four different parking environments according to the environment information, sequentially matches three parking modes, plans a parking path according to the selected mode, and solves the parking modes of standard parking spaces, narrow parking spaces and roads and the parking mode of responding to the rear vehicle to seize the parking spaces.
Drawings
For the purposes of promoting an understanding of the principles of the invention, its technical solutions and advantages, reference will now be made in detail to the present invention as illustrated in the accompanying drawings, in which:
FIG. 1 is a flow chart of a parallel parking path planning of the present invention;
FIG. 2 is a schematic view of the panoramic camera installation of the present invention;
FIG. 3 is an expanded top view of the present invention;
FIG. 4 is a schematic illustration of a modal 1 parking path in an embodiment of the present invention;
FIG. 5 is a schematic illustration of a modal 2 parking path in an embodiment of the present invention;
FIG. 6 is a schematic illustration of modal 3 parking paths in an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the intended purpose, the method of the present invention will be further described in detail with reference to the accompanying drawings, and obviously, the embodiments are only a part of the embodiments of the present invention. Based on the embodiments of the present invention, other embodiments obtained by persons of ordinary skill in the art without any creative effort belong to the protection scope of the present invention.
The invention provides a multi-environment detection and multi-mode matching parallel parking path planning system based on a panoramic camera, which is characterized by comprising the following components: an image acquisition device; an image processing module; an environment detection module; a data module; a modality classification and matching module; and a path planning module.
When a user executes a parking command, the image acquisition device arranged on the roof ascends to acquire image information, analyzes vehicle body parameters and surrounding environment information, identifies and classifies parallel parking spaces meeting parking requirements, performs modal matching according to the environment type, plans a parking path and performs tracking control to complete a parking task.
The image acquisition device comprises a catadioptric panoramic camera and a fixing device, and vehicle and environment image information is acquired through the catadioptric panoramic camera, wherein the vehicle information image comprises the shape, size, color and the like of a vehicle, and the environment information comprises an environment information image in an effective visual angle range around the vehicle;
the described fixing device comprises a mobile fixed slide block, a guide rail and a bottom fixed base, wherein the bottom fixed base is used for being vertically arranged on a vehicle roof, a fixed position and a guide rail; the movable fixed sliding block is used for fixing the panoramic camera and moving on the guide rail, so that the camera is lifted, and the range of the acquired image is adjusted.
After acquiring the image information, the image processing unit develops it into a top view without distortion. The image processing unit comprises an image unfolding unit used for unfolding the image into a top view; and the distortion correction unit is used for calibrating the camera based on the top view, correcting the top view by using the correction parameters, and correcting the top view by using a bilinear interpolation method according to the homography matrix to obtain a distortion-free top view. Wherein the image expanding unit, specifically the expanding top view method step S5, comprises:
s51, mounting the panoramic camera and the fixing device on the top end of the vehicle body, and enabling the camera to be perpendicular to the ground as much as possible;
s52, according to the optical reflection and principle and the geometric principle, setting the generated target top view as IMG _ VERT, the size of the generated target top view as (H, W), and acquiring the panorama as IMG _ SRC, the size of which is (H-SRC)0,W0) Panoramic camera mirror surface parameters
The focal length of the camera is f;
s53, establishing a side coordinate system XOY on a two-dimensional plane by taking the focus of the reflector of the panoramic camera as a center;
s54. order
If the coordinates of any point pixel point on the top view of the target are (x, y), then there are
a)
b)
S55, the expansion result is IMG _ VERT (x, y) ═ IMG _ SRC (x)0,y0)。
Wherein the distortion correction unit, the specific distortion correction method step S6 includes:
s61, mounting the panoramic camera and the fixing device on the top end of the vehicle body, enabling the camera to be perpendicular to the ground as far as possible, placing camera calibration devices in front of, behind, on the left and on the right of the vehicle body, respectively moving the four calibration devices and ensuring that the four calibration devices are within a camera shooting range, and storing image data;
s62, according to the top view expanding method of claim 5, expanding a top view IMG _ VERT of the stored image data;
s63, acquiring distortion parameters and relative external parameters according to the top view IMG _ VERT image;
and S64, carrying out distortion parameter correction and rotation transformation on the subsequent image to enable the image to be vertical to the ground.
And the environment detection module is used for classifying the image samples into parking space, road lines and other obstacle information of rear vehicles. The specific classification step S8 includes:
s81, obtaining the surrounding environment information of the vehicle, and obtaining parking space information and road information according to the surrounding environment information of the vehicle. The parking space information comprises parking space parameters of each parking space and barrier information in the parking space, and the road information comprises road width parameters and whether rear vehicle coming parameters exist or not;
s82, storing the parking space information and the width parameters of the roads adjacent to the parking space information according to the information of the obstacles in the parking space;
s83, updating the stored parking space information of each parking space and the width parameters of the adjacent roads in real time according to the motion condition of the vehicle;
s84, determining whether parking can be performed and the distance between the parking vehicle and the parking space according to the parking space parameters of each parking space and the obstacle information in the parking space which are updated in real time;
s85, classifying the detected parking spaces into four types according to the real-time environment information, wherein the four types include a standard parking environment E1, a narrow parking space environment E2,
narrow road environment E3, rear coming vehicle environment E4. The specific classification method step S9 includes:
s91, inputting vehicle body parameters and parking space size (L)p,Wp) Width of road WrAnd whether there is a vehicle behind;
s92, according to the Ackerman steering principle, judging that the steering wheel is full, wherein the minimum parking space which can be driven out in one step is as follows:
s93, calculating the parking space as (L) according to the geometric principlep1，Wp1) The minimum steering angle that can be stepped out, the minimum road width is:
s94, calculating the curve transition distance n according to the convolution curve calculation method0；
S95, classifying conditions of the standard parking space environment E1 are as follows: the size of the parking space is not less than (L)p1，Wp1) The width of the lane is not less than Wr；
The narrow parking space environment E2 is classified into the following conditions: the size of the parking space is larger than (L, W) and smaller than (L)p1，Wp1) The width of the lane is not less than Wr；
The narrow road environment E3 is classified into: the size of the parking space is larger than (L, W), and the width of the lane is smaller than Wr；
The classification conditions of the parking space preemption environment E4 are as follows: the coming car exists at the rear part, and the size of the parking space is larger than (L, W).
The data module for estimating vehicle data includes: vehicle length, vehicle width, wheel base, front suspension, rear suspension, maximum equivalent front wheel rotation angle and maximum equivalent front wheel rotation angle rotation speed. The data module calculates the length and width of the vehicle by using the image information; the vehicle attitude change is calculated using the adjacent image information, and then vehicle data is calculated.
The modality classification and matching module includes: a single-step parking mode M1, a multi-step parking mode M2 and a parking space preemption mode M3, and simultaneously classifying according to the environment: and matching the standard parking space environment E1, the narrow parking space environment E2, the narrow road environment E3 and the space occupation environment E4. The specific matching method comprises the following steps: m1 matched E1, M2 matched E2-E3, M3 matched E4.
After vehicle body parameters and surrounding environment information are obtained and environment classification and mode matching are carried out, a mode type is selected for parking path planning. The specific method comprises the following steps: and according to the initial attitude, the target parking pose and the intermediate pose information of the vehicle, performing segmented planning on the path between the initial parking pose and the target parking pose according to the intermediate pose to obtain a target driving path. Wherein the target travel path is for the vehicle to travel from the start pose to the target parking pose.
The intermediate pose refers to a vehicle pose in which the vehicle can run out of the garage without collision at the maximum steering wheel angle, and at least part of the vehicle is located in the garage where the target parking space is located when the vehicle is located in the intermediate pose;
the target parking gesture refers to a vehicle gesture that the vehicle center coincides with the target parking space center, and the vehicle body is parallel to the parking space.
The path segment between the initial parking pose and the middle pose is composed of two one-way arc segments, and the condition that the vehicle does not collide with the two ends of the parking space, the parking space line on the inner side and the road boundary in the path process between the initial parking pose and the middle pose is met.
And the path segment between the intermediate pose and the target parking pose consists of a plurality of circular arcs and a plurality of straight track segments, and the condition that the vehicle does not collide with the two ends of the parking space and the inner parking space line in the process of the path between the initial parking pose and the intermediate pose is met.
The method for planning different paths for different modes comprises the following specific steps:
1. in the single-step parking mode, the path target parking pose generated in the single-step parking mode is coincident with the intermediate pose, namely, the path planning is carried out on the target pose without the intermediate pose.
2. In the multi-step parking mode, the path target parking pose generated by the multi-step parking mode is not coincident with the intermediate pose, namely, the intermediate pose is required to perform path planning on the target pose.
3. In the parking space occupying mode, the intermediate parking position needs to be further planned by the automobile parking position, and the requirements are as follows: and the intermediate pose segments the path between the initial parking poses according to the initial parking pose, the intermediate pose and the preemption pose information, and the intermediate pose segments the path between the initial parking poses according to the intermediate pose and the preemption pose information to obtain the intermediate pose segment.
And when the vehicle is positioned in the preemption pose, at least part of the vehicle is positioned in a garage where the target parking space is positioned.
And according to the initial parking pose, the preempted pose and the steering pose, carrying out segmentation division according to the initial parking pose, the steering pose and the preempted pose to obtain a path segment between the initial parking pose and the preempted pose.
Wherein the steering pose is an intermediate vehicle pose at which the vehicle can travel from the initial parking pose without collision to the preemption pose state, and at least a portion of the vehicle is located within a garage in which the target parking space is located when the vehicle is in the steering pose.
And the path between the initial parking pose and the steering pose and the preempting pose are segmented, and the path tracks of the vehicles are all unidirectional arc tracks.
The invention discloses the following technical effects: the invention uses a catadioptric panoramic camera vertically placed on the roof to obtain the images of the vehicle body and the environment, realizes self-adaption to obtain vehicle data according to vehicle control and image information, divides the images into four different parking environments according to the environment information, sequentially matches three parking modes, plans a parking path according to the selected mode, and solves the parking modes of standard parking spaces, narrow parking spaces and roads and the parking mode of responding to the rear vehicle to seize the parking spaces.
In the description of the present invention, it is to be understood that the term four parking environments: standard parking environment E1, narrow parking space environment E2, narrow road environment E3, rear coming vehicle environment E4 and three types of parking modes: the single-step parking mode M1, the multi-step parking mode M2, the parking space preemption mode M3, and the corresponding environment classification method and mode matching method are preferred modes selected based on this example for ease of description of the invention, and are not intended to indicate or imply that the device or element in question must have a particular construction and operation, and therefore should not be construed as limiting the invention.
The above-described embodiments are merely illustrative of the preferred embodiments of the present invention, and do not limit the scope of the present invention, and various modifications and improvements of the technical solutions of the present invention may be made by those skilled in the art without departing from the spirit of the present invention, which is defined by the claims.
Claims (10)
1. The utility model provides a multi-environment detects parallel parking route planning system with multimode matching based on panoramic camera which characterized in that includes:
an image acquisition device; an image processing module; an environment detection module; a data module; a modality classification and matching module; a path planning module;
when a user executes a parking command, the parallel parking path planning system ascends the image acquisition device arranged on the roof and acquires image information, analyzes vehicle body parameters and surrounding environment information, identifies and classifies parallel parking spaces meeting parking requirements, performs modal matching according to the environment type, and plans a parking path.
2. The image capturing device according to claim 1, comprising:
the device comprises a catadioptric panoramic camera and a fixing device, wherein the catadioptric panoramic camera is used for acquiring vehicle and environment image information, the vehicle information image comprises vehicle appearance, size, color and the like, and the environment information comprises an environment information image in an effective visual angle range around the vehicle; an image processing unit for developing the image information into a distortion-free top view.
The described fixing device comprises a lifting device and a bottom fixing base, wherein the bottom fixing base is used for being arranged on a roof, a fixing position and the lifting device; the lifting device is used for realizing the vertical lifting of the panoramic camera on the roof of the vehicle.
3. The panoramic camera device of claims 1 and 2, characterized in that the image processing unit comprises an image unfolding unit for unfolding the image into a top view; and the distortion correction unit is used for calibrating the camera based on the top view, correcting the top view by using the correction parameters, and correcting the top view by using a bilinear interpolation method according to a homography matrix to obtain a distortion-free top view.
The method for unfolding a top view is characterized in that the step S3 includes:
s31, mounting the panoramic camera and the fixing device on the top end of the vehicle body, and enabling the camera to be perpendicular to the ground as much as possible;
s32, according to the optical reflection and principle and the geometric principle, setting the generated target top view as IMG _ VERT, the size of the generated target top view as (H, W), and acquiring the panorama as IMG _ SRC, the size of which is (H-SRC)0,W0) Panoramic camera mirror surface parameters
The focal length of the camera is f;
s33, establishing a side coordinate system XOY on a two-dimensional plane by taking the focus of the reflector of the panoramic camera as a center;
s34, according to the geometric principle and the specular reflection principle, coordinates of pixel points of any point on the target top view are (x, y), and parameters are set
Then there are:
s35, the expansion result is IMG _ VERT (x, y) ═ IMG _ SRC (x)0,y0)。
4. The distortion correction unit according to claim 3, the distortion correction method, wherein the step S4 includes:
s41, mounting the panoramic camera and the fixing device on the top end of the vehicle body, enabling the camera to be perpendicular to the ground as far as possible, placing camera calibration devices in front of, behind, on the left and on the right of the vehicle body, respectively moving the four calibration devices and ensuring that the four calibration devices are within a camera shooting range, and storing image data;
s42, according to the top view expanding method of claim 5, expanding a top view IMG _ VERT of the stored image data;
s43, according to the top view IMG _ VERT image, distortion parameters and relative external parameters are obtained;
and S44, carrying out distortion parameter correction and rotation transformation on the subsequent image to enable the image to be vertical to the ground.
5. The system for planning parallel parking paths according to claim 1, wherein a target classifier is provided in the environment detection module, and the target classifier is used for classifying the image samples into information of parking spaces, road lines and other obstacles of vehicles behind.
The method comprises the following steps:
s51, obtaining vehicle surrounding environment information, and obtaining parking space information and road information according to the vehicle surrounding environment information. The parking space information comprises parking space parameters of each parking space and barrier information in the parking space, and the road information comprises road width parameters and whether rear vehicle coming parameters exist or not;
s52, storing the parking space information and the width parameters of the roads adjacent to the parking space information according to the information of the obstacles in the parking space;
s53, updating the stored parking space information of each parking space and the width parameters of the adjacent roads in real time according to the motion condition of the vehicle;
s54, determining whether parking can be performed and the distance between the parking vehicle and the parking space according to the parking space parameters of each parking space and the obstacle information in the parking space which are updated in real time;
and S55, classifying the detected parking environment according to the real-time environment information.
6. The parallel parking path planning system of claim 1 wherein the data module for estimating vehicle data comprises: vehicle length, vehicle width, wheel base, front suspension, rear suspension, maximum equivalent front wheel rotation angle and maximum equivalent front wheel rotation angle rotation speed.
The data module calculates the length and width of the vehicle by using the image information; the vehicle attitude change is calculated using the adjacent image information, and then vehicle data is calculated.
7. The system for parallel parking path planning according to claim 1, wherein the modal classification classifies parking modalities according to actual parking manners, including but not limited to a single-step parking modality M1, a multi-step parking modality M2, a preemption parking space modality M3, etc., classifies parking environments according to claim 5, and matches parking environments with parking modalities.
8. The parallel parking path planning system according to claim 1, wherein the parking path refers to a target driving path obtained by performing a segmented planning on the intermediate pose and a path between the intermediate pose and the target parking pose according to an initial pose of a vehicle, a target parking pose and intermediate pose information and according to the initial parking pose and the target parking pose. Wherein the target travel path is for the vehicle to travel from the start pose to the target parking pose; the intermediate pose refers to a vehicle pose in which the vehicle can run out of the garage without collision in a steering wheel corner, and at least part of the vehicle is located in the garage where the target parking space is located when the vehicle is located in the intermediate pose; the target parking gesture refers to a vehicle gesture that the vehicle center coincides with the target parking space center, and the vehicle body is parallel to the parking space.
And the path segment between the initial parking pose and the middle pose consists of a plurality of one-way arcs, and the condition that the vehicle does not collide with the two ends of the parking space, the parking space line at the inner side and the road boundary in the path process between the initial parking pose and the middle pose is met. The path segment between the intermediate pose and the target parking pose consists of a plurality of arc segments and a plurality of straight line track segments, and the condition that the vehicle does not collide with the two ends of the parking space and the parking space line on the inner side in the process of the path between the initial parking pose and the intermediate pose is met.
9. A parking modality according to claim 7, characterized by comprising:
and in the single-step parking mode, the generated path target parking pose is coincident with the intermediate pose. And path planning is divided into paths of the initial parking pose to the target parking pose. Namely, a parking mode of path planning on a target pose without an intermediate pose is required;
and in the multi-step parking mode, the generated path target parking pose is not coincident with the intermediate pose. The path planning is divided into an initial parking pose to intermediate pose path and an intermediate pose to target parking pose path. Namely a parking mode in which path planning is required for the intermediate pose to the target pose.
The parking space preemption mode refers to an environment that a vehicle comes from behind, a parking space preemption path needs to be generated, and the initial parking pose of the parking space preemption path is located behind the target parking space. The path planning comprises a path of the initial parking pose to the middle pose, the path segmentation planning between the initial parking pose and the middle pose is segmented according to the initial parking pose and the middle pose, and the path segmentation method comprises the following steps: and starting a parking pose to seize a pose path, and seizing a pose to an intermediate pose path.
10. The parking stall preemption modality of claim 9, wherein the preemption pose is a vehicle pose in which the vehicle is able to travel from an initial parking pose to the preemption pose and the preemption pose is from the intermediate pose state, respectively, without colliding with a parking stall corner, and wherein at least a portion of the vehicle is located within a parking stall in which the target parking stall is located when the vehicle is in the preemption pose; at least part of paths of the initial parking pose to the path of the preemptive pose meet the condition that the existing vehicles are partially positioned in the parking spaces where the target parking spaces are located.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795200.0A CN113353067A (en) | 2021-07-14 | 2021-07-14 | Multi-environment detection and multi-mode matching parallel parking path planning system based on panoramic camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795200.0A CN113353067A (en) | 2021-07-14 | 2021-07-14 | Multi-environment detection and multi-mode matching parallel parking path planning system based on panoramic camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113353067A true CN113353067A (en) | 2021-09-07 |
Family
ID=77539412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110795200.0A Pending CN113353067A (en) | 2021-07-14 | 2021-07-14 | Multi-environment detection and multi-mode matching parallel parking path planning system based on panoramic camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113353067A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115520178A (en) * | 2022-09-27 | 2022-12-27 | 文远苏行(江苏)科技有限公司 | Automatic parking method, device, storage medium, computer equipment |
| JP2023080414A (en) * | 2021-11-30 | 2023-06-09 | スズキ株式会社 | Vehicle sensor mounting structure |
| CN119840607A (en) * | 2025-02-05 | 2025-04-18 | 奇瑞汽车股份有限公司 | Method, device and equipment for generating parking track and computer readable storage medium |
Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6665003B1 (en) * | 1998-09-17 | 2003-12-16 | Issum Research Development Company Of The Hebrew University Of Jerusalem | System and method for generating and displaying panoramic images and movies |
| US20100208032A1 (en) * | 2007-07-29 | 2010-08-19 | Nanophotonics Co., Ltd. | Method and apparatus for obtaining panoramic and rectilinear images using rotationally symmetric wide-angle lens |
| JP2010183265A (en) * | 2009-02-04 | 2010-08-19 | Alpine Electronics Inc | Image display device and method for calculating camera-mounting angle of the same |
| WO2011087354A2 (en) * | 2010-01-15 | 2011-07-21 | Mimos Berhad | Vehicle parking and obstacle avoidance system using a single panoramic camera |
| CN102774324A (en) * | 2012-07-06 | 2012-11-14 | 广东好帮手电子科技股份有限公司 | Panoramic parking assist system and implementation method thereof |
| CN103065318A (en) * | 2012-12-30 | 2013-04-24 | 深圳普捷利科技有限公司 | Curved surface projection method and device of multi-camera panorama system |
| CN104691415A (en) * | 2013-12-07 | 2015-06-10 | 惠州市德赛西威汽车电子有限公司 | Panoramic auxiliary parking device |
| CN106560364A (en) * | 2015-10-05 | 2017-04-12 | 现代自动车株式会社 | Parking Guidance Apparatus And Method For Vehicle |
| CN107249934A (en) * | 2015-02-17 | 2017-10-13 | 康蒂-特米克微电子有限公司 | Method and device for displaying surrounding environment of vehicle without distortion |
| CN107618452A (en) * | 2017-09-30 | 2018-01-23 | 百度在线网络技术（北京）有限公司 | Panorama gathers vehicle roof platform and panorama collecting vehicle |
| CN107792179A (en) * | 2017-09-27 | 2018-03-13 | 浙江零跑科技有限公司 | A kind of parking guidance method based on vehicle-mounted viewing system |
| CN108320575A (en) * | 2018-03-16 | 2018-07-24 | 上海芭比信息技术服务有限公司 | A kind of guiding of parking stall and reverse vehicle searching system and its parking space-detecting device, method |
| CN108791278A (en) * | 2018-06-21 | 2018-11-13 | 重庆大学 | Side coil is parked control system and its control method |
| CN109131082A (en) * | 2018-08-31 | 2019-01-04 | 深圳以恒科技有限公司 | A kind of monocular panorama based entirely on vision is parked image system and its method of parking |
| CN110475107A (en) * | 2018-05-11 | 2019-11-19 | 福特全球技术公司 | The distortion correction of vehicle panoramic visual camera projection |
| CN110781883A (en) * | 2019-09-18 | 2020-02-11 | 江苏大学 | A parking space size recognition system and method based on a 360-degree surround view camera |
| CN110843694A (en) * | 2019-10-30 | 2020-02-28 | 俞天力 | 360-degree panoramic camera device for unmanned automobile and vision sensing equipment |
| CN111369439A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Panoramic view image real-time splicing method for automatic parking stall identification based on panoramic view |
| KR102132578B1 (en) * | 2020-03-31 | 2020-07-09 | 조명원 | Monitoring system and monitoring method using raceway lighting and camera |
| CN111862673A (en) * | 2020-06-24 | 2020-10-30 | 北京易航远智科技有限公司 | Parking lot vehicle self-positioning and map construction method based on top view |
| CN211844288U (en) * | 2019-01-25 | 2020-11-03 | 北京电影学院 | Liftable panoramic image motion acquisition system conforming to human eye vision height |
| CN112124201A (en) * | 2020-10-10 | 2020-12-25 | 深圳道可视科技有限公司 | Panoramic parking system with visible picture blind areas and method thereof |
-
2021
- 2021-07-14 CN CN202110795200.0A patent/CN113353067A/en active Pending
Patent Citations (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6665003B1 (en) * | 1998-09-17 | 2003-12-16 | Issum Research Development Company Of The Hebrew University Of Jerusalem | System and method for generating and displaying panoramic images and movies |
| US20100208032A1 (en) * | 2007-07-29 | 2010-08-19 | Nanophotonics Co., Ltd. | Method and apparatus for obtaining panoramic and rectilinear images using rotationally symmetric wide-angle lens |
| JP2010183265A (en) * | 2009-02-04 | 2010-08-19 | Alpine Electronics Inc | Image display device and method for calculating camera-mounting angle of the same |
| WO2011087354A2 (en) * | 2010-01-15 | 2011-07-21 | Mimos Berhad | Vehicle parking and obstacle avoidance system using a single panoramic camera |
| CN102774324A (en) * | 2012-07-06 | 2012-11-14 | 广东好帮手电子科技股份有限公司 | Panoramic parking assist system and implementation method thereof |
| CN103065318A (en) * | 2012-12-30 | 2013-04-24 | 深圳普捷利科技有限公司 | Curved surface projection method and device of multi-camera panorama system |
| CN104691415A (en) * | 2013-12-07 | 2015-06-10 | 惠州市德赛西威汽车电子有限公司 | Panoramic auxiliary parking device |
| CN107249934A (en) * | 2015-02-17 | 2017-10-13 | 康蒂-特米克微电子有限公司 | Method and device for displaying surrounding environment of vehicle without distortion |
| CN106560364A (en) * | 2015-10-05 | 2017-04-12 | 现代自动车株式会社 | Parking Guidance Apparatus And Method For Vehicle |
| CN107792179A (en) * | 2017-09-27 | 2018-03-13 | 浙江零跑科技有限公司 | A kind of parking guidance method based on vehicle-mounted viewing system |
| CN107618452A (en) * | 2017-09-30 | 2018-01-23 | 百度在线网络技术（北京）有限公司 | Panorama gathers vehicle roof platform and panorama collecting vehicle |
| CN108320575A (en) * | 2018-03-16 | 2018-07-24 | 上海芭比信息技术服务有限公司 | A kind of guiding of parking stall and reverse vehicle searching system and its parking space-detecting device, method |
| CN110475107A (en) * | 2018-05-11 | 2019-11-19 | 福特全球技术公司 | The distortion correction of vehicle panoramic visual camera projection |
| CN108791278A (en) * | 2018-06-21 | 2018-11-13 | 重庆大学 | Side coil is parked control system and its control method |
| CN109131082A (en) * | 2018-08-31 | 2019-01-04 | 深圳以恒科技有限公司 | A kind of monocular panorama based entirely on vision is parked image system and its method of parking |
| CN211844288U (en) * | 2019-01-25 | 2020-11-03 | 北京电影学院 | Liftable panoramic image motion acquisition system conforming to human eye vision height |
| CN110781883A (en) * | 2019-09-18 | 2020-02-11 | 江苏大学 | A parking space size recognition system and method based on a 360-degree surround view camera |
| CN110843694A (en) * | 2019-10-30 | 2020-02-28 | 俞天力 | 360-degree panoramic camera device for unmanned automobile and vision sensing equipment |
| CN111369439A (en) * | 2020-02-29 | 2020-07-03 | 华南理工大学 | Panoramic view image real-time splicing method for automatic parking stall identification based on panoramic view |
| KR102132578B1 (en) * | 2020-03-31 | 2020-07-09 | 조명원 | Monitoring system and monitoring method using raceway lighting and camera |
| CN111862673A (en) * | 2020-06-24 | 2020-10-30 | 北京易航远智科技有限公司 | Parking lot vehicle self-positioning and map construction method based on top view |
| CN112124201A (en) * | 2020-10-10 | 2020-12-25 | 深圳道可视科技有限公司 | Panoramic parking system with visible picture blind areas and method thereof |
Non-Patent Citations (6)
| Title |
|---|
| DAVID LI等: "A Log-Rectilinear Transformation for Foveated 360-degree Video Streaming", 《 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS》 * |
| RUZHONG CHENG等: "Panorama parking assistant system with improved particle swarm optimization method", 《JOURNAL OF ELECTRONIC IMAGING》 * |
| XUQIN YAN等: "A Calibration Method for the Surround-View Parking Assistant System", 《2020 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS)》 * |
| ZHONGLONG XIONG等: "Research of bird"s-eye panoramic view for vehicle parking", 《2011 INTERNATIONAL CONFERENCE ON MULTIMEDIA TECHNOLOGY》 * |
| 杨刚等: "基于参考点配准法的全景泊车系统研究", 《传感器与微系统》 * |
| 杨少平等: "采用全景相机的移动机器人视觉伺服", 《计算机工程与设计》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2023080414A (en) * | 2021-11-30 | 2023-06-09 | スズキ株式会社 | Vehicle sensor mounting structure |
| CN115520178A (en) * | 2022-09-27 | 2022-12-27 | 文远苏行(江苏)科技有限公司 | Automatic parking method, device, storage medium, computer equipment |
| CN115520178B (en) * | 2022-09-27 | 2025-12-05 | 文远苏行(江苏)科技有限公司 | Automatic parking methods, devices, storage media, and computer equipment |
| CN119840607A (en) * | 2025-02-05 | 2025-04-18 | 奇瑞汽车股份有限公司 | Method, device and equipment for generating parking track and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9311711B2 (en) | Image processing apparatus and image processing method | |
| CN108128245B (en) | Vehicle environment imaging system and method | |
| US7684590B2 (en) | Method of recognizing and/or tracking objects | |
| US8089512B2 (en) | Driving support device, driving support method and computer program | |
| US9573524B2 (en) | Inspection device and method of head up display for vehicle | |
| JP4695167B2 (en) | Method and apparatus for correcting distortion and enhancing an image in a vehicle rear view system | |
| EP3859593B1 (en) | Vehicle feature acquisition method and device | |
| CN112184818A (en) | Vision-based vehicle positioning method and parking lot management system applying same | |
| JP4958279B2 (en) | Object detection device | |
| CN110203210A (en) | A kind of lane departure warning method, terminal device and storage medium | |
| US8477191B2 (en) | On-vehicle image pickup apparatus | |
| CN113353067A (en) | Multi-environment detection and multi-mode matching parallel parking path planning system based on panoramic camera | |
| JP2007300181A (en) | Peripheral recognition device, peripheral recognition method, and program | |
| KR20200000953A (en) | Around view monitoring system and calibration method for around view cameras | |
| US10964059B2 (en) | Static camera calibration using motion of vehicle portion | |
| CN113320474A (en) | Automatic parking method and device based on panoramic image and human-computer interaction | |
| KR102031635B1 (en) | Collision warning device and method using heterogeneous cameras having overlapped capture area | |
| CN109703465B (en) | Control method and device for vehicle-mounted image sensor | |
| CN110780287A (en) | Distance measurement method and distance measurement system based on monocular camera | |
| Krasner et al. | Automatic parking identification and vehicle guidance with road awareness | |
| US20240034305A1 (en) | Parking Assistance Method and Parking Assistance Device | |
| CN113459951A (en) | Vehicle exterior environment display method and device, vehicle, equipment and storage medium | |
| EP3163533A1 (en) | Adaptive view projection for a vehicle camera system | |
| US20240103525A1 (en) | Vehicle and control method thereof | |
| CN112389419A (en) | Method for identifying parking space and parking assistance system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210907 |
|
| WD01 | Invention patent application deemed withdrawn after publication |