JP2024064989A - Parking assistance device and parking assistance method - Google Patents

Abstract

To provide a parking support apparatus and a parking support method that enable smooth automatic parking in a short period of time.SOLUTION: A parking support apparatus includes: a parking frame detection unit that detects a parking frame on the basis of a peripheral image showing the periphery of a vehicle; a storage unit that stores the history of any operation of an occupant of the vehicle or of any movement of the vehicle; an instruction detection unit that detects a parking instruction of the occupant on the basis of the history; and a vehicle control unit that automatically parks the vehicle after the instruction detection unit detects the parking instruction. The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after turning by steering from straight traveling, and the occupant has performed a prescribed parking instruction operation.SELECTED DRAWING: Figure 4

Description

This disclosure relates to a parking assistance device and a parking assistance method.

Conventionally, parking assistance devices are known that can automatically park a vehicle in a parking space by steering the vehicle at a predetermined position around the parking space and driving backward from the position where the vehicle has been turned at a predetermined angle. For example, Patent Document 1 discloses a configuration in which a parking assistance device is activated when a vehicle is parked next to a parking space, and the parking position of the vehicle in the parking space is uniquely determined.

JP 2022-69161 A

However, in the configuration of the conventional technology, after activating the parking assistance device, it is necessary to detect a parking space and to confirm the safety of the area through which the vehicle will pass when parking, which lengthens the time the vehicle spends in the passage. As a result, the overall time required for parking increases, and the vehicle behind may honk its horn while the parking assistance device is being operated.

The objective of this disclosure is to provide a parking assistance device and parking assistance method that can perform automatic parking smoothly in a short amount of time.

The parking assistance device according to the present disclosure is
A parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle;
A storage unit that stores a history of either an operation by a passenger of the vehicle or a movement of the vehicle;
an instruction detection unit that detects a parking instruction from the occupant based on the history;
a vehicle control unit that performs automatic parking of the vehicle after the instruction detection unit detects the parking instruction;
Equipped with
The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has steered from a straight line to a turn and then stopped, and when the occupant has performed a specified parking instruction operation.

The parking assistance method according to the present disclosure includes:
Detecting a parking space based on a surrounding image showing the surroundings of the vehicle;
storing a history of either the operation of an occupant of the vehicle or the motion of the vehicle;
detecting a parking instruction from the occupant based on the history;
after detecting the parking instruction, automatically parking the vehicle;
having
In the step of detecting the parking instruction, if the history indicates that the vehicle has stopped after steering from a straight line and turning, and if the occupant has performed a specified parking instruction operation, it is determined that the parking instruction has been detected.

This disclosure allows automatic parking to be performed smoothly and in a short amount of time.

FIG. 1 is a diagram for explaining automatic parking by a vehicle to which a parking assistance device according to a first embodiment of the present disclosure is applied. 1 is a diagram showing a vehicle to which a parking assistance device according to an embodiment of the present invention can be applied. FIG. 2 is a diagram showing a network configuration of a system to which a parking assistance device is applied. FIG. 2 is a block diagram showing hardware that implements the functions of the parking assistance device. FIG. 2 is a block diagram showing a parking assistance device. 10A and 10B are diagrams for explaining setting of a parking frame detection area. 1A and 1B are diagrams illustrating an example of parking space detection using a rear camera. FIG. 13 is a diagram showing an example in which the camera cannot capture the entire parking space; FIG. 11 is a diagram showing an example in which the entire parking space is captured by the front camera. FIG. 11 is a diagram illustrating an example in which a plurality of parking spaces are detected. 4 is a flowchart showing an example of an operation of parking assistance control in the parking assistance device according to the first embodiment. 4 is a flowchart showing an example of the operation of parking assistance control in the parking assistance device according to the first embodiment. 4 is a flowchart showing an example of an operation of parking assistance control in the parking assistance device according to the first embodiment. FIG. 11 is a diagram for explaining an example of parking a vehicle in the second embodiment. 10 is a flowchart showing an example of an operation of parking assistance control in the parking assistance device according to the second embodiment. 10 is a flowchart showing an example of an operation of parking assistance control in the parking assistance device according to the second embodiment. 10 is a flowchart showing an example of an operation of parking assist control in the parking assist device according to the second embodiment. 13A to 13C are diagrams for explaining an example of a steering action of a vehicle in the third embodiment. 13A to 13C are diagrams for explaining an example of a steering action of a vehicle in the third embodiment. 13A to 13C are diagrams for explaining an example of a steering action of a vehicle in the third embodiment. 13A to 13C are diagrams for explaining an example of a steering action of a vehicle in the third embodiment. 13A to 13C are diagrams for explaining an example of a steering action of a vehicle in the third embodiment. 13A to 13C are diagrams for explaining an example of a steering action of a vehicle in the third embodiment.

(First embodiment)
Hereinafter, the first embodiment of the present disclosure will be described in detail with reference to the drawings. FIG. 1 is a diagram for explaining automatic parking by a vehicle 1 to which a parking assistance device 100 according to the first embodiment of the present disclosure is applied. FIG. 2 is a diagram showing a vehicle 1 to which the parking assistance device 100 according to the present embodiment can be applied. In the description of FIG. 1 and the like, a Cartesian coordinate system (X, Y) is used. The Cartesian coordinate system (X, Y) is also used in the drawings to be described later. For example, the X direction indicates the left-right direction relative to the parking position (the position of the vehicle 1 in the parking frame), and the Y direction indicates the front-rear direction relative to the parking position. In addition, when expressing the position of the vehicle 1, it is expressed as the position of a representative point of the vehicle 1 in principle. The representative point of the vehicle 1 is the midpoint of the ground contact points of the two rear wheels. In addition, when expressing the trajectory of the movement of the vehicle 1, it is expressed as the trajectory of the representative point of the vehicle 1.

As shown in FIG. 1, the vehicle 1 is equipped with a camera 2 (see FIG. 2, etc.) that monitors the periphery of the vehicle, and is configured to be able to perform automatic parking in a parking lot with one or more parking spaces by using the parking assistance device 100 shown in FIG. 3. A parking space is an area enclosed by two approximately parallel parking frame lines. Each parking frame line is spaced apart at a distance greater than the width of the vehicle 1.

In the example shown in FIG. 1, a parking lot is shown in which three parking spaces S1, S2, and S3 are arranged in order from the negative side in the X direction. The parking space S1 is a parking space sandwiched between the parking space lines L1 and L2, the parking space S2 is a parking space sandwiched between the parking space lines L2 and L3, and the parking space S3 is a parking space sandwiched between the parking space lines L3 and L4. The left side is the negative side in the X direction and the right side is the positive side in the X direction, with a line parallel to the parking space lines L2 and L3 and equidistant from both the parking space lines L2 and L3 as the boundary. The boundary line between the positive and negative sides in the Y direction is the short side of the parking space that is closer to the vehicle 1.

Here, an example of the case where the vehicle 1 automatically parks in the parking space S2 will be described. First, the vehicle 1 moves straight in the direction from the negative side to the positive side in the X direction in the portion on the positive side in the Y direction of the three parking spaces S1, S2, and S3. The vehicle 1 in the straight-moving state turns at position P1 in front of the parking space line L2 located on the negative side in the X direction among the parking space lines L2 and L3 of the parking space S2, turns while moving forward to the positive side in the Y direction, and stops at position P2. Position P2 is a position on the positive side in the X direction and on the positive side in the Y direction of the parking space line L3, and is a position where the vehicle 1 can park between the parking space lines L2 and L3 by turning while retreating from position P2. The vehicle 1 travels to position P2 by manual operation of the occupant.

When the occupant of the vehicle 1 performs a parking instruction operation such as putting the gear in R (reverse), the vehicle 1 performs automatic parking. The vehicle 1 turns while backing up from position P2, enters parking space S2, and stops at position P3 within parking space S2 while driving backward. The hazard lights may be turned on during automatic parking. In addition, since it is possible to determine which parking space the vehicle will be parked in based on the steering position and the steering direction, the occupant does not need to specify a parking position during automatic parking. In the following explanation, it is assumed that the vehicle 1 performs the above-mentioned behavior during automatic parking.

As shown in FIG. 2, cameras 2 are provided at four locations on the front, rear, left and right sides of the vehicle 1. Each camera 2 is equipped with a fisheye lens and has a horizontal viewing range of 180 degrees or more (see dashed lines). Each camera 2 is mounted at a depression angle to capture the road surface, so when the range of the road surface captured is converted into a horizontal viewing range, a single camera 2 captures a range of approximately 240 degrees of the road surface. For example, the front and rear wheels and the sides of the vehicle are captured in the image captured by the side cameras 2A provided on the left and right sides of the vehicle body.

As shown in FIG. 3, the vehicle 1 includes, in addition to the four cameras 2, an operation device 10, an HMI (human machine interface) device 20, a vehicle control device 30, and a parking assistance device 100. The operation device 10 is for the driver (passenger) to manually operate, and includes a physical switch on a panel at the driver's seat, software switches displayed on a touch panel, and other devices for driving operations such as a steering wheel, pedals, and gears. The HMI device 20 is used as an HMI for the passenger to input operations to the parking assistance device 100, such as a touch panel attached to a navigation device 40 provided in the vehicle 1. The touch panel may be included in the operation device 10, and various switches grounded to the driver's seat may also be included in the operation device 10, so the operation device 10 and the HMI device 20 may be overlapping.

The vehicle control device 30 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an input/output circuit (not shown). When the occupant operates the operation device 10, the vehicle control device 30 accepts the operation. In normal driving mode, the vehicle control device 30 drives a motor (not shown) in response to operation information to control the steering angle and vehicle speed, and simultaneously outputs operation information and vehicle information such as steering angle and vehicle speed to a LAN (Local Area Network: in-vehicle LAN, etc.). In parking assistance mode, the vehicle control device 30 receives speed and steering angle commands from the parking assistance device 100 via the LAN, and controls the speed and steering angle in response to the commands.

The parking assistance device 100 obtains operation information of the operation device 10 via the LAN because the vehicle control device 30 monitors the operation of the operation device 10. The occupant can operate the parking assistance device 100 via the HMI device 20. The parking assistance device 100 may receive the position information of the vehicle 1 output by the navigation device 40 via the LAN, or may obtain it directly from the navigation device 40. The camera 2 constantly outputs the captured image to the parking assistance device 100, and even when parking assistance is not performed, the parking assistance device 100 generates and outputs a display image showing the surroundings of the vehicle from the captured image. In addition, the above-mentioned touch panel also functions as a display that outputs the display image generated by the parking assistance device 100, so the parking assistance device 100 and the HMI device 20 are directly connected. The parking assistance device 100 may directly receive information on the operation performed on the touch panel, or may receive it via the LAN. In addition, the parking assistance device 100 can receive the position information of the vehicle 1 output by the navigation device 40 via the LAN.

The functions of the parking assistance device 100 may be implemented in the hardware shown in FIG. 4. The parking assistance device 100 includes a CPU 101, a ROM 102, a RAM 103, an I/O (input/output interface) 104, and an IMP (Image Processor) 105. The parking assistance device 100 may be a computer in which each element is connected by a bus. In addition, multiple elements may be housed in one chip, or one element may be composed of multiple chips. The bus may not be a single bus, but may be a combination of multiple types of buses. For example, the CPU 101, ROM 102, RAM 103, and IMP 105 housed in one chip may be connected by a parallel bus, and the I/O 104, which is composed of multiple chips, may be connected to the chip housing the CPU 101, etc. by a serial bus.

The CPU 101 controls the entire parking assistance device 100. The ROM 102 is an electrically rewritable memory that stores the programs executed by the CPU 101 and also functions as a non-volatile data storage area, retaining data and the like even when the parking assistance device 100 is turned off. The RAM 103 is used for temporary storage as a working area for the CPU 101. For example, data that only needs to be stored temporarily, such as the latest surrounding image, is stored in the RAM 103. The RAM 103 has a capacity that allows multiple surrounding images to be stored. The IMP 105 is a processor with improved processing performance specialized for image processing, and executes the processing of the image acquisition unit 130, parking space detection unit 140, and display image output unit 180 shown in FIG. 5.

The camera 2 constantly outputs the captured camera images to the parking assistance device 100. Even when parking assistance is not being performed, the parking assistance device 100 generates and outputs a display image showing the periphery of the vehicle 1 from the camera images. As shown in FIG. 5, the parking assistance device 100 has an operation reception unit 110, a state management unit 120, an image acquisition unit 130, a parking space detection unit 140, a route calculation unit 150, a driving control unit 160, a storage unit 170, and a display image output unit 180.

The operation reception unit 110 receives operation information indicating the operation of the occupant, including driving operations.

The state management unit 120 manages the parking assistance state (state) according to the operation information and the state of the vehicle. There are multiple parking assistance states, for example, state 0 to state 8.

State 0 is the initial state, and is the state when the parking assistance function is not activated. State 1 is the idle state, and is the state when vehicle speed information and position information are acquired, and when the vehicle speed drops below a threshold, it is determined whether the vehicle is on a road. A road is a road on a map, such as a public road that is not a parking lot with parking spaces.

State 2 is a monitoring state in which white line detection is performed at a low frequency. State 3 is a preparation state in which captured images are linked to time information and recorded. State 4 is a detection state in which parking space detection is performed speculatively. State 5 is a calculation state in which, once the vehicle has turned 30 degrees or more and stopped, a parking path calculation is performed speculatively and the occupant is asked whether they intend to park.

State 6 is a query state, in which the parking route is displayed and the vehicle waits for a parking instruction operation. State 7 is a self-driving state, in which the vehicle performs automatic parking according to the calculated route. State 8 is an end state, in which the end of automatic parking is notified.

The image acquisition unit 130 acquires camera images and generates a surrounding image showing the surroundings of the vehicle 1. The surrounding image may be an overhead image in which the camera image is projected onto the road surface, an image in another format, or the camera image itself. In this embodiment, the surrounding image is assumed to be an overhead image.

The parking space detection unit 140 sets a parking space detection area in the peripheral image for detecting a parking space, and extracts a white line from the peripheral image within the parking space detection area. The parking space detection area may be the entire peripheral image, or may be an area in the peripheral image where a parking space is expected. By limiting the parking space detection area to an area where a parking space is expected, the processing time required for detection can be shortened. In addition, by limiting the size of the parking space detection area to a size that can accommodate one parking space but not two parking spaces, the processing when two parking spaces are detected simultaneously can be omitted. In general, detection is performed for the purpose of obtaining information about a specific object, and if a specific object is detected as a result of the detection, it is determined that "detection has been made" and information about the detected object is output. The results of detection are divided into cases where detection has been made and cases where detection has not been made, and if detection has not been made, the purpose of detection is not achieved. If detection has been made, the purpose of detection has been achieved, so the detection can be rephrased as successful detection.

The parking space detection unit 140 further generates pairs of the extracted white lines and searches for pairs of white lines that meet the conditions of a parking space. For example, when the parking space detection unit 140 detects a pair of white lines from the surrounding image, if the following three conditions are met: the two white lines are parallel on the road surface, the two white lines are longer than the vehicle length, and the distance between the two white lines is greater than the vehicle width, the parking space detection unit 140 determines that the area between the two white lines is an area where the vehicle 1 can be parked (i.e., a parking space). When a parking space is detected, the white lines that make up the parking space can be said to be parking space lines. The parking space detection unit 140 outputs the endpoints of the two parking space lines, that is, the coordinates of the four corners of the parking space. These coordinates are relative coordinates based on the vehicle itself.

The route calculation unit 150 sets the target parking position so that the center of the vehicle body is located in the center of the coordinates of the four corners of the parking space output by the parking space detection unit 140, and sets the target parking angle (the orientation of the vehicle body when parked at the target parking position) so that it is parallel to the long side of the parking space, and calculates the parking route to the target parking position. A proposed publicly known method may be applied to calculate the parking route.

The driving control unit 160 controls the steering angle and vehicle speed so that the vehicle 1 drives along the calculated parking path. More specifically, the driving control unit 160 outputs to the vehicle control device 30 the steering angle and vehicle speed command values according to the steering angle and vehicle speed target values. The driving control unit 160 calculates the actual steering angle and vehicle speed based on the data output by the vehicle control device 30. For example, the actual vehicle speed is calculated from the wheel rotation speed data and the wheel outer circumference data. The actual steering angle may be data obtained from the steering device, or may be calculated from the inner wheel difference calculated from the difference in the wheel rotation speed. When there is a difference between the target value and the actual value, the driving control unit 160 corrects the steering angle and vehicle speed command values so that the actual steering angle and vehicle speed values become the same as the steering angle and vehicle speed target values.

The driving control unit 160 tracks the position and orientation of the vehicle 1 while it is traveling, based on the calculated steering angle and actual measured vehicle speed values. In other words, the driving control unit 160 calculates the driving trajectory of the vehicle 1 during automatic driving or manual driving. The driving trajectory may be calculated in the form of movement information in which the travel distance, travel direction, and vehicle body angle are added to the time for each unit time. The form of the movement information is not limited to this, and may be, for example, a form using coordinate information.

During automatic driving, the driving control unit 160 compares the parking path calculated by the route calculation unit 150 with the driving trajectory represented by the movement information, and corrects the steering angle so that the driving trajectory follows the parking path. For example, when the vehicle 1 reverses while turning with a steering angle, if the driving trajectory passes outside the parking path as viewed from the center point of the turn, the driving control unit 160 increases the steering angle to reduce the turning radius. This causes the driving trajectory of the vehicle 1 to be pulled back toward the parking path.

The storage unit 170 stores the operation information acquired by the operation reception unit 110, the movement information calculated by the driving control unit 160, the surrounding image generated by the image acquisition unit 130, the parking space line and parking space position information detected by the parking space detection unit 140, etc. in a history information area in chronological order, linking them to time. Since the vehicle 1 moves in response to the operation of the occupant, it is not necessary for either the movement information or the operation information to be stored in the storage unit 170, but in this embodiment, both the movement information and the operation information are stored in the storage unit 170.

For example, the storage unit 170 may add information every unit time and fix the size of the storage area allocated for each addition (memory block size). In this case, the history information area is divided into memory blocks and the starting address of the information increases monotonically every unit time, so that the address at which the information is stored is linked to the time. Alternatively, the storage unit 170 may not fix the memory size, but may provide a separate starting address table that links the starting address to the time, and update the starting address table each time information is added.

By identifying the time using any method, it is possible to access the most recent information at that time. Also, by accumulating movement information up to that time, it is possible to identify the relative positional relationship of the position at that time to the current position. Conversely, positional information can be added in chronological order as history information, and the relative positional relationship to the current position can be identified from the difference between the positional information at that time and the current position. In other words, positional information can be calculated from history information.

In addition, for large data such as peripheral images, only the first address needs to be stored in the history information area.

Since it is sufficient for history information to be able to be referenced going back a specified period of time, old history information that is no longer necessary may be erased from memory unit 170. For example, a history information area may be set in memory unit 170 as a fixed area sandwiched between a first address and a last address, and when the address storing information reaches the last address, it may be made into a loop memory that continues storing information in the first address. In this way, past history information going back one loop is overwritten and erased, eliminating the need for an erase process.

Storing in the history information area does not need to be done all the time, but can be done in states 2 to 7 above.

The memory unit 170 also stores the parking route calculated by the route calculation unit 150, allowing the driving control unit 160 to refer to the parking route.

In addition, since parking space detection takes time to process, the parking space information, which is the detection result, may be obtained at a time significantly delayed from the time the camera image was captured. Even in this case, the parking space information is stored in the memory unit 170 in association with the capture time. For example, the memory unit 170 adds information every unit time, and the memory block is assigned fixedly regardless of the amount of data written in one addition by fixing the capacity of the memory area (memory block size) assigned to one addition. When the information is added to the memory unit 170, the leading address of the surrounding image is obtained, but the result of detecting the parking space in the surrounding image (parking space information) has not been obtained at that time. In this case, the parking assistance device 100 writes only the obtained information into the memory block without waiting for the parking space information, and when the parking space information is obtained, the parking space information is written into the memory block in which the leading address of the surrounding image that was the subject of the detection is written.

For example, if the vehicle 1 turns in state 3 and speculatively executes parking space detection when it proceeds to state 4, whether parking space information has been obtained or not when it proceeds to state 5 will change depending on the relationship between the time from when the vehicle 1 turns to when it stops and the time required for parking space detection. For example, if parking space information is not written in the corresponding memory block when it proceeds to state 5, the process of starting the route calculation may be a waiting loop that polls the address where the parking space information is written, or the processor (e.g., IMP) that performs the parking space detection may be set to generate an interrupt when the parking space detection is completed. In this way, the route calculation can be started immediately when the parking space detection is completed. In this way, by storing parking space information, etc. in the storage unit 170 in association with the shooting time, the automatic parking process can be efficiently controlled.

The display image output unit 180 generates a display image based on a camera image or a surrounding image. The display image may be an overhead image of the surroundings of the vehicle 1 viewed from above. In other words, the display image output unit 180 may output an overhead image as the display image.

The display image output unit 180 displays a message or a figure superimposed on the display image in response to a request from the state management unit 120. For example, when automatic parking is started, the display image output unit 180 may display a semi-transparent rectangle indicating the position of the parking space detected by the parking space detection unit 140 superimposed on the overhead image, and may display the route from the vehicle to the target parking position calculated by the route calculation unit 150 with a dotted line.

The message may not only be displayed on an image, but may also be read aloud. Alternatively, the message may be output only as audio. Specifically, the state management unit 120 issues a command to specify one of the preset texts and output it, and in response, the display image output unit 180 generates an image of a character string corresponding to the specified text, superimposes this on the display image, and outputs it to the HMI device 20. At the same time, the display image output unit 180 outputs the audio data that was saved together with the text to the HMI device 20 together with the display image. This allows the occupant to receive the message even if they are not looking at the HMI device 20.

Next, the states managed by the state management unit 120 will be described in detail. As described above, the state management unit 120 accepts operations by the occupant and manages the parking assistance state. The operations by the occupant are not limited to operations of the parking assistance device 100, but include operations for driving. For example, when the occupant starts the vehicle 1 by turning on the IG (Ignition)-ON, the parking assistance device 100 also starts up during the startup sequence of the vehicle 1, and the state progresses from state 0 (initial state) to state 1 (pause state).

In state 1 (pause state), only the state management unit 120 and the driving control unit 160 operate. The state management unit 120 acquires vehicle speed information via the driving control unit 160, and when the vehicle speed drops below a threshold (e.g., 10 km/h), queries the navigation device 40 as to whether the vehicle 1 is located on a road. If the vehicle 1 is off the road (e.g., on a store premises), the state management unit 120 advances the state to 2. State 1 may be called a road driving scene. Since a state corresponds to a scene, the state management unit 120 may be said to be an example of a scene determination unit, which will be described later. In addition, the state management unit 120 may be said to be an example of a function restriction unit, since it partially or fully restricts functions such as the parking space detection unit depending on the state (scene).

In state 2 (monitoring state), part of the parking space detection unit 140 also operates. In other words, in state 1 (road driving scene), the execution of all functions of the parking space detection unit 140 is restricted. The parking space detection unit 140 periodically (for example, at one-second intervals) performs white line detection. White line detection is performed by releasing some of the functions of the parking space detection unit 140 and not performing other functions such as evaluating whether a white line is a parking space line. The parking space detection unit 140 detects white lines that are at angles within a predetermined range (60 degrees to 120 degrees) with respect to the axis of symmetry of the vehicle 1 (a line passing through the center of the front end of the vehicle and the center of the rear end of the vehicle 1) from an image taken around the vehicle 1. Since white line detection is performed for the purpose of determining whether the vehicle 1 is in a parking lot, the interval or length of the white lines is not evaluated. Therefore, a limited range in which only the part in front of the parking space line is captured in the surrounding image may be set as the detection range. If the parking space detection unit 140 detects multiple white lines that meet the conditions, it determines that the vehicle 1 is in a parking lot, and the state management unit 120 advances the state.

In the example shown in FIG. 1, state 2 is the state when the vehicle 1 is moving straight ahead in the +Y direction from the parking spaces S1, S2, and S3, and before the white line is detected.

In state 3 (preparation state), the peripheral images are linked to time information and stored in the memory unit 170. Of the images used for white line detection, only images in which a white line (a white line approximately perpendicular to the axis of symmetry of the vehicle 1) is detected may be stored as peripheral images, or images taken periodically (e.g., every second) may be stored as peripheral images without being selected.

The state management unit 120 stores in the memory unit 170 the time when a steering force equal to or greater than a threshold value (e.g., 1 kg) is applied to the steering wheel from a straight-ahead state with a steering angle of approximately 0 as the steering time, and advances the state.

In the example shown in FIG. 1, state 3 is when the vehicle 1 is moving straight ahead in the +Y direction from the parking spaces S1, S2, and S3, from when the white line is detected until when the vehicle 1 turns at position P1.

In state 4 (detection state), the parking space detection unit 140 detects a parking space from an image captured just before the steering wheel is turned. Alternatively, the parking space detection unit 140 may detect a parking space from an image captured at the time of steering wheel turning.

At this point, it has not been decided whether or not to perform automatic parking, so this parking space detection can be said to be speculative. If there is a parked vehicle in the parking space, part of the parking space line will be blocked by the parked vehicle, so the length of the detected white line will not meet the conditions, and the parking space will not be detected. Therefore, when a parking space is detected, it can be said that an empty parking space is detected.

The state management unit 120 advances the state when the vehicle body turns 30 to 60 degrees from the steering point and the vehicle 1 stops.

In the example shown in FIG. 1, state 4 is the state in which vehicle 1 steers at position P1 and stops at position P2.

In state 5 (calculation state), the route calculation unit 150 accumulates movement information from the time when the image in which the parking space was detected was captured to the time when the vehicle 1 stopped, and calculates the vehicle position (coordinate offset) at the time of capture based on the current vehicle position. After parking space detection is completed, the route calculation unit 150 adds the coordinate offset to the parking space position information, and calculates the parking space position information based on the current vehicle position. The route calculation unit 150 calculates a parking route for parking in the parking space from the current vehicle position.

At this point, it has not been decided whether or not to perform automatic parking, so it can be said that this is a speculative route calculation. Once the parking route has been calculated, the display image output unit 180 outputs an image showing the parking space and the parking route, and outputs a message asking the occupant if they wish to park. After outputting the image and message, the state management unit 120 advances the state.

In the example shown in FIG. 1, state 5 is the state from when the vehicle 1 stops at position P2 and the parking assistance device 100 calculates the parking path until it outputs the above message.

In state 6 (inquiry state), the operation reception unit 110 receives a parking instruction operation. If a parking instruction operation is performed, the state management unit 120 advances the state. In the example shown in FIG. 1, state 6 is the state after the message is output and before the occupant performs a parking instruction operation.

In state 7 (self-driving state), the driving control unit 160 performs automatic parking according to the calculated route. When automatic parking is completed, the state management unit 120 advances the state. In the example shown in FIG. 1, state 7 is the state in which the vehicle 1 performs automatic parking from position P2 to position P3.

In state 8 (end state), the parking assistance device 100 notifies the end of automatic parking. After the notification is completed, the state management unit 120 returns the state to state 2. In the example shown in FIG. 1, state 8 is the state from when the vehicle 1 stops at position P3 until the above notification is made.

When the vehicle 1 drives to a vacant parking space where it is to be parked, the occupant turns the steering wheel when he or she visually confirms that the parking space is available. Therefore, the parking space detection unit 140 may set a parking space detection area according to the position of the occupant when turning the steering wheel. The position of the occupant may be identified as the position of the occupant's head.

The occupant must also visually confirm the direction in which the vehicle 1 is moving, so he or she visually confirms the parking space located diagonally forward. However, there is a time lag between seeing the vacant parking space and turning the steering wheel, and the vehicle 1 may be moving to the side of the vacant parking space by the time the steering wheel is turned. Therefore, it is preferable for the parking space detection unit 140 to set the parking space detection area diagonally forward from the side of the occupant.

Figure 6 shows parking spaces S1, S2, and S3 similar to those shown in Figure 1, and an example is shown in which a vehicle 1 traveling from the + side to the - side in the X direction is attempting to park in parking space S2 in a portion on the - side of parking spaces S1, S2, and S3 in the Y direction. In this example, when the vehicle 1 turns at a position P4 in front of the parking space line L3 in order to park in the parking space S2, the portion from the side of the occupant (the + side in the Y direction) to the diagonally forward (the - side in the X direction and the + side in the Y direction) is set as the parking space detection area D1.

Because the occupant turns the steering wheel to park backward, the parking space detection area is located on the opposite side of the vehicle 1 across the line along which the vehicle 1 has traveled. Therefore, the parking space detection area is preferably set on the opposite side of the direction in which the vehicle 1 has turned across the line along which the vehicle 1 has traveled, and includes the side of the occupant or diagonally forward, based on the position of the occupant when the vehicle 1 turns. In other words, the line along which the vehicle 1 has traveled straight can be said to be a line extending the straight part of the route along which the vehicle 1 has traveled. When turning the steering wheel, as shown in FIG. 6, the camera 2A (side camera) provided on the side of the vehicle 1 is often located to the side of the vacant parking space, so the parking space detection unit 140 may set the side of the side camera as the parking detection area.

In addition, if the vehicle 1 is steered when an occupant is positioned in front of a vacant parking space and turns approximately 45 degrees before stopping, the camera 2 (rear camera) installed at the rear of the vehicle 1 often captures the entire vacant parking space, as shown in Figure 7.

As with FIG. 6, FIG. 7 shows an example in which vehicle 1 is attempting to park in parking space S2. After vehicle 1 steers to park in parking space S2, it moves forward while turning and stops, and the camera 2 installed at the rear of vehicle 1 is able to detect the four corners of parking space S2.

Therefore, for example, the parking space detection unit 140 may not perform speculative parking space detection in state 4, but may first perform parking space detection from the image of the rear camera in state 5, and if no parking space is detected, may perform parking space detection using the image at the time of turning.

In this way, the effect of concealing the processing time is smaller than when parking space detection is started speculatively in state 4, but since parking space detection is started speculatively as soon as vehicle 1 is stopped without waiting for an automatic parking command, the time required for parking is shorter than with the conventional method.

For example, if a driver passes in front of a vacant parking space and then steering is performed, or if the vehicle 1 turns with a small steering angle and a large turning radius, the entire vehicle body may pass in the direction of the entrance of the vacant parking space. In this case, if there is a parked vehicle in a parking space adjacent to the vacant parking space, the vacant parking space (long side) may be blocked by the parked vehicle, and the rear camera may not capture the entire vacant parking space (for example, see Figure 8). If speculative parking space detection is not performed in State 4, parking space detection is first performed in State 5 using the image from the rear camera, and after the parking space detection fails, parking space detection is performed using the image at the time of steering.

As in FIG. 6, FIG. 8 shows an example in which vehicle 1 is attempting to park in parking space S2. When vehicle 1 turns while moving forward after steering to park in parking space S2, a parked vehicle 1A in parking space S1 on the negative side of parking space S2 in the X direction blocks parking space line L2, so that the camera 2 installed at the rear of vehicle 1 cannot capture the entire parking space S2.

In this way, parking space detection is performed twice, so the time required for parking will be longer than with conventional methods. However, if the driver drives appropriately, this state will not occur, so it can be said that the probability of this state occurring is low. Therefore, the expected time required for parking will be smaller than with conventional methods.

In addition, for example, speculative parking space detection may be performed in state 3 rather than starting in state 4 after steering. In this way, it is expected that parking space detection will be completed when the vehicle 1 turns and stops and enters state 5. In this case, since only route calculation needs to be performed in state 5, the parking position and parking route can be presented at an earlier timing than when parking space detection is performed in state 4.

At this time, parking space detection may be performed using images from a camera 2 (front camera) installed at the front of the vehicle 1. The front camera approaches the entrance to the parking space before the occupant, so it can detect the parking space at an earlier timing.

For example, FIG. 9 shows an example in which the front camera 2 of a vehicle 1 moving straight from the + side in the X direction to the - side in the X direction from parking space S3 captures the entire parking space S1, which is located on the most - side (furthest forward) in the X direction among parking spaces S1, S2, and S3.

However, if the front camera passes over the line that is an extension of the long side of the parking frame line, the long side will be outside the camera's field of view. However, since parking frame lines are drawn periodically, this can be addressed by predicting the best position for parking space detection. Since the front camera is positioned lower than the side cameras, it can often capture the parking frame line without being blocked by parked vehicles, even if it is just before the line that is an extension of the long side.

When speculative parking space detection is performed in state 3, it may happen that multiple parking spaces are detected at the time of state 5, such as parking space S1 and parking space S2 in FIG. 10. In such a case, information on a parking space whose image was captured just before position P5 at the time of steering (for example, parking space S1) is selected from the history information (not the most recent parking space detection result obtained at the time of steering). Since the purpose is to reduce the time required for parking, it is more effective to not allow the occupant to make a selection.

If the time required for the vehicle 1 to move forward by one parking space is shorter than the time required for parking space detection, the parking space detection in the area of the parking space S1 is not completed at the time of generating the peripheral image of the parking space S1. In such a case, the parking space detection unit 140 may store the peripheral image of the parking space S1 in the storage unit 170, skip the parking space detection, and perform the parking space detection when the information of the parking space S1 becomes necessary. If the parking space detection in the area of the parking space S2 is terminated and the parking space detection in the area of the parking space S1 is started, the parking space detection of the parking space S2 that was terminated will be re-executed when the information of the parking space S2 becomes necessary. Therefore, the expected value of the time required for parking is larger when the parking space detection is terminated. Alternatively, the parking assistance device 100 may request the occupant to decelerate the vehicle 1 so that both the parking space S1 and the parking space S2 can be detected so that the parking space detection is not required to be skipped.

The standard interval between parking frame lines (longer sides of parking frame lines) is 2.5 m. When vehicle 1 travels in front of a parking frame at 2.5 m per second, vehicle 1 moves forward one frame per second. 2.5 m per second is 9 km/h, which is slightly faster than the 8 km/h speed posted in many parking lots. When vehicle 1 is travelling while searching for an empty parking frame, it is travelled at a speed slower than 8 km/h. For example, if vehicle 1 is travelled while storing surrounding images in storage unit 170 at intervals of one frame per second, there is no risk of missing an empty parking frame.

For example, when a parking space is detected from an image captured by the front camera, it is preferable to capture the image with the front camera positioned slightly in front of the extension of the parking space line, so that the interval between capture positions is fixed (for example, 2.5 m intervals) rather than making the time interval between captures constant. By repeating parking space line detection from state 2 onwards, the distance between the parking space lines can be known, so the position of the next parking space line can be predicted, and if capture is performed when the vehicle 1 advances to that position, a front camera image captured at a position suitable for parking space detection can be obtained. When traveling at a speed of 8 km/h, the time interval for storing the front camera image is 1 second or more. Also, since the speed is not necessarily constant and the time interval between captures is irregular, it is preferable to store the capture time as well.

For example, if movement information is stored in a memory block at one-second intervals, it is advisable to store the first address of the surrounding image in the latest memory block, as well as the fractional part of the image capture time that is less than one second. This makes it possible to use an interpolation method to calculate the distance traveled by vehicle 1 between the time the movement information was stored and the time the image was captured. This makes it possible to later correctly calculate the distance from the surrounding image to the detected parking space.

The storage interval for movement information may also be shortened (for example, 0.1 second intervals) from the time of steering. Storing movement information in detail allows for more accurate estimation of the movement of vehicle 1 from the steering point to the parking position. The storage interval may be coarse, since the movement information before steering is not directly related to the estimation accuracy of the parking space position. If the time when automatic parking begins differs from the time when the image was taken, the amount of movement between them must be compensated for, but this can be compensated for by the above method.

Next, an example of the operation of the parking assistance device 100 according to the first embodiment will be described. Figures 11, 12, and 13 are flowcharts showing an example of the operation of the parking assistance control in the parking assistance device 100 according to the first embodiment. Note that the following flowchart starts when the power of the vehicle 1 is turned on.

As shown in FIG. 11, the parking assistance device 100 (state management unit 120) sets the parking assistance state to state 0 (step S101). In step S101, the state management unit 120 initializes the internal state of the parking assistance device 100.

Next, the state management unit 120 sets the parking assistance state to State 1 (step S102). In step S102, the state management unit 120 acquires vehicle speed information from the driving control unit 160.

Then, the state management unit 120 determines whether the vehicle speed is less than a speed threshold (e.g., 10 km/h) (step S103). If the result of the determination is that the vehicle speed is equal to or greater than the speed threshold (step S103, YES), the process returns to step S102.

On the other hand, if the vehicle speed is less than the speed threshold (step S103, NO), the parking assistance device 100 (state management unit 120) determines whether the vehicle 1 is on the road (step S104). The determination as to whether the vehicle 1 is on the road is made based on information from the navigation device 40. If the result of the determination is that the vehicle 1 is on the road (step S104, YES), the process returns to step S102.

On the other hand, if the vehicle 1 is not on the road (step S104, NO), the state management unit 120 sets the parking assistance state to state 2 (step S105). Note that even if information from the navigation device 40 cannot be obtained, the system may proceed to state 2. This may be said to skip step S104. In state 2 (step S105), the parking assistance device 100 (parking space detection unit 140) performs white line detection at a low frequency (1 second intervals).

In state 2, the parking assistance device 100 (state management unit 120) determines whether the conditions for judging a parking scene are met (step S106). The conditions for judging a parking scene are that there are multiple white lines that are approximately perpendicular to the axis of symmetry of the vehicle 1, and that these white lines are parallel to each other. If the result of the determination is that the conditions for judging a parking scene are not met (step S106, NO), the process returns to step S105.

On the other hand, if the parking scene determination condition is met (step S106, YES), the state management unit 120 sets the parking assistance state to State 3 (step S107). In step S107, the parking assistance device 100 (storage unit 170) stores the surrounding image in association with the time information.

Next, the state management unit 120 acquires steering angle data from the driving control unit 160 and determines whether the absolute value of the steering angle is greater than an angle threshold value (e.g., 5 degrees) (step S108). If the result of the determination is that the absolute value of the steering angle is greater than the angle threshold value (step S108, YES), the process of step S108 is repeated.

On the other hand, if the absolute value of the steering angle is less than the angle threshold value (step S108, NO), as shown in FIG. 12, the state management unit 120 obtains data on the steering force applied to the steering wheel from the driving control unit 160 and determines whether the absolute value of the steering force is greater than a threshold value (e.g., 1 kg) (step S109). In other words, when turning, that is, when the steering angle is large, it is not determined that steering has occurred, but it is determined that steering has occurred when the vehicle is traveling straight, that is, when the steering angle is small and a steering force is applied. In addition, the conditions for determining straight traveling may include moving forward a predetermined distance (e.g., 1 m) with a small steering angle.

If the result of the determination is that the absolute value of the steering force is less than the threshold value (step S109, NO), the process returns to step S108 (see also FIG. 11). On the other hand, if the absolute value of the steering force is greater than the threshold value (step S109, YES), the state management unit 120 sets the parking assistance state to state 4 (step S110).

In step S110, the parking assistance device 100 (storage unit 170) stores the steering time, and the parking space detection unit 140 starts parking space detection using the image captured just before the steering time. The parking assistance device 100 (route calculation unit 150) also determines the straight-ahead direction based on history information before the steering time. Specifically, at the time of steering, the surrounding image generated from the image captured just before the steering remains in the memory, so the storage unit 170 prohibits overwriting of the area containing the surrounding image and writes the head address and the shooting time to the memory block. The status management unit 120 then passes the head address to the parking space detection unit 140 and instructs it to detect a parking space. The state management unit 120 also passes the start address to the route calculation unit 150, and the route calculation unit 150 reads the traveling direction (yaw angle) of the vehicle 1 from 3 seconds before the steering time to the steering time from the history information (memory block up to 3 seconds ago) and determines the average value as the straight heading direction. Note that the above process and values are just an example.

Next, the parking assistance device 100 (route calculation unit 150) calculates a turning angle based on the straight heading direction (step S111), and the state management unit 120 determines whether the turning angle is greater than a first angle (e.g., 60 degrees) (step S112). The turning angle is calculated, for example, by subtracting the straight heading direction from the yaw angle and finding the absolute value.

If the result of the determination is that the turning angle is greater than the first angle (step S112, YES), the parking assistance device 100 terminates the parking space detection and cancels the steering time (step S113). After that, the process returns to step S108 (see also FIG. 11). In step S113, if the turning angle is greater than the first angle, the parking assistance device 100 (state management unit 120) determines that the vehicle 1 has made a right or left turn and cancels the process associated with the steering.

On the other hand, if the turning angle is equal to or less than the first angle (step S112, NO), the parking assistance device 100 determines whether the vehicle speed is 0 (step S114). If the result of the determination is that the vehicle speed is not 0 (step S114, NO), the process returns to step S111. In other words, the turning angle monitoring process is in a loop state until the vehicle 1 stops.

On the other hand, if the vehicle speed is 0 (step S114, YES), the parking assistance device 100 (state management unit 120) determines whether the turning angle is greater than a second angle (e.g., 30 degrees) (step S115). If the result of the determination is that the turning angle is equal to or less than the second angle (step S115, NO), the process returns to step S111. In other words, if the turning angle is small, the turning angle monitoring process enters a loop state.

On the other hand, if the turning angle is greater than the second angle (step S115, YES), the state management unit 120 sets the parking assistance state to state 5 (step S116). This can also be said to proceed to state 5 (step S116) when the vehicle 1 turns 45 degrees ± 15 degrees from a straight-ahead state and stops. In step S116, the parking assistance device 100 determines the amount of movement of the vehicle 1 since the image capture time. For example, the route calculation unit 150 adds up the amount of movement since the image capture time of the image that is the subject of parking space detection, which is recorded in the history information, and calculates the current position (including relative coordinates and yaw angle) based on the position of the vehicle 1 at the image capture time.

In state 5, the parking assistance device 100 (state management unit 120) determines whether or not parking space detection has been completed (step S117). If the result of the determination is that parking space detection has not been completed (step S117, NO), the processing of step S117 is repeated. On the other hand, if parking space detection has been completed (step S117, YES), the parking assistance device 100 determines whether or not parking space detection has been successful (step S118).

If the result of the determination is that the parking space detection was not successful (step S118, NO), the parking assistance device 100 (display image output unit 180) outputs a message to the effect that the parking space was not detected (step S119), and this control ends (see also FIG. 13). If the parking space detection was not successful, this can be rephrased as a case in which the parking space detection unit 140 did not detect a parking space, and if the parking space detection was successful, this can be rephrased as a case in which the parking space detection unit 140 detected a parking space.

On the other hand, if the parking space detection is successful (step S118, YES), as shown in FIG. 13, the parking assistance device 100 calculates the position of the parking space (step S120). In step S116, the current position is calculated based on the position of the vehicle 1 at the time the image was captured, and the result of the parking space detection indicates the position of the parking space based on the position of the vehicle 1 at the time the image was captured. Therefore, by adding up the two, the position of the parking space based on the current position can be calculated. This calculation may be performed by the route calculation unit 150. Next, the route calculation unit 150 executes the parking route calculation based on the position of the parking space based on the current position (step S121).

Next, the parking assistance device 100 (state management unit 120) determines whether the parking route calculation is complete (step S122). If the result of the determination is that the parking route calculation is not complete (step S122, NO), the processing of step S122 is repeated. On the other hand, if the parking route calculation is complete (step S122, YES), the parking assistance device 100 determines whether the parking route calculation is successful (step S123).

If the result of the determination is that the parking route calculation was not successful (step S123, NO), the parking assistance device 100 (display image output unit 180) outputs a message to the effect that the parking route cannot be set (step S124), and this control ends.

On the other hand, if the parking path calculation is successful (step S123, YES), the state management unit 120 sets the parking assistance state to state 6 (step S125). In step S125, the display image output unit 180 outputs a message asking the occupant whether or not to select automatic parking, and the state management unit 120 sets the timer to 0. The timer is for measuring the waiting time for the occupant's response to the above message, i.e., the inquiry about automatic parking.

In state 6, the state management unit 120 determines whether the timer is less than 3 seconds (step S126). If the timer is 3 seconds or more (step S126, NO), the parking assistance device 100 outputs a message to the effect that automatic parking is to be canceled (step S127), and this control ends. In other words, if the occupant does not respond within 3 seconds, it is assumed that the occupant did not select automatic parking.

On the other hand, if the timer is less than three seconds (step S126, YES), the parking assistance device 100 determines whether or not a parking instruction operation has been performed (step S128). A parking instruction operation includes at least one of the following, or a combination of two or more of: changing the gear position, releasing the steering wheel, releasing the brake, turning on the hazard lights, and operating a button corresponding to a parking instruction. For example, if the gear is in R and the steering wheel and brake are released, the parking assistance device 100 determines that a parking instruction operation has been performed.

If the determination result is that no parking instruction operation has been made (step S128, NO), the process returns to step S126. On the other hand, if a parking instruction operation has been made (step S128, YES), the state management unit 120 sets the parking assistance state to state 7 (step S129). In step S129, the parking assistance device 100 (driving control unit 160) controls the vehicle 1 to start automatic driving. This can also be rephrased as saying that if the occupant makes a parking instruction operation within three seconds, the occupant is deemed to have selected automatic parking, and automatic parking is started.

Next, the parking assistance device 100 determines whether the vehicle position is a parking position (step S130). If the result of the determination is that the vehicle position is not a parking position (step S130, NO), the processing of step S130 is repeated. This can also be said to continue automatic driving until the vehicle 1 reaches the parking position.

On the other hand, if the vehicle position is a parking position (step S130, YES), the parking assistance device 100 outputs a message indicating that automatic parking has ended (step S131). After that, the process returns to step S105.

Note that in the above steps S104 to S124, if the vehicle speed becomes greater than the speed threshold, the process may immediately transition to step S102. Also, in the above steps S105 to S124, if the position of the vehicle 1 is on the road, the process may immediately transition to step S102. In other words, even if the vehicle 1 is traveling at low speed off the road, a white line is detected, and storage of the movement information of the vehicle 1 is started, when the vehicle 1 appears on the road or the vehicle speed increases, the parking assistance device 100 may determine that the scene is not a parking scene and return to step S102, where it is determined whether or not the scene is a parking scene.

In addition, in the judgment process from step S125 onwards, the occupant may give an instruction not to perform automatic parking. For example, when waiting for a parking instruction operation in steps 126 and 128, the HMI device 20 may receive an instruction not to perform automatic parking. If an instruction is given by the occupant, the instruction takes priority, so the judgment may not be based on the position or speed of the vehicle 1. In addition, in any step, if the IG is turned off, this control ends.

The message asking the occupant whether or not to select automatic parking may be output at a timing earlier than step S125. In other words, the occupant is speculatively queried without waiting for the completion of route calculation. For example, the above message may be output at the point in time when the vehicle 1 turns 45 degrees ±15 degrees and stops in step S116. In this way, route calculation is performed in parallel with the occupant issuing the parking instruction operation, so the overall time required for parking can be shortened.

In addition, since there are multiple parking instruction operations and each of the multiple parking instruction operations has its own order, the step of determining the parking instruction operation may be divided into multiple steps. For example, when the gear position is in R after step S116, a message may be output asking the occupant whether or not to select automatic parking, and when the steering wheel and brake are subsequently released, acceptance of the parking instruction operation may be completed and automatic driving may begin.

In this way, when the timing of the inquiry to the occupant is advanced, the parking assistance device 100 may lengthen the inquiry waiting time in step S126 (the time until it is assumed that the occupant has not selected automatic parking) taking into consideration the time required for the occupant to operate the vehicle and the time required for the calculation processing performed in parallel. For example, when the parking assistance device 100 makes an inquiry when the gear position is in R, the above waiting time may be increased by 1 second from 3 seconds in step S126 to 4 seconds, and when the inquiry is made at the time of step S116 (when the vehicle is stopped), the waiting time may be increased by 3 seconds from 3 seconds to 6 seconds.

According to the present embodiment configured as described above, the state management unit 120 detects a parking instruction from the occupant based on the history of operation information or movement information of the vehicle 1 stored in the memory unit 170. The state management unit 120 corresponds to the "instruction detection unit" of the present disclosure.

Specifically, the state management unit 120 determines that a parking instruction has been detected when the history of the operation information or movement information of the vehicle 1 indicates that the vehicle 1 has stopped after steering and turning from a straight line and the occupant has performed a predetermined parking instruction operation. When the state management unit 120 detects a parking instruction, the driving control unit 160 outputs a command to the vehicle control device 30 to automatically park the vehicle 1 based on the parking space information. The driving control unit 160 corresponds to the "vehicle control unit" of this disclosure.

As a result, when vehicle 1 has turned and stopped, automatic parking can be initiated in response to a parking instruction. The route for automatic parking generally overlaps with the area vehicle 1 has passed, so there is no need to request safety confirmation again. In addition, since where to park can be determined from the position and direction of turning, there is no need to specify a parking position.

In this way, in this embodiment, no confirmation or instruction is required when starting automatic parking, so automatic parking can begin in a short time.

In addition, in this embodiment, the vehicle 1 turning from a straight-ahead state is used as a trigger to perform speculative parking space detection, so parking space detection can be performed using the time between turning the steering wheel and stopping the vehicle. As a result, automatic parking can be started in a shorter time than in a configuration in which parking space detection is performed after the vehicle stops. Therefore, automatic parking can be performed more smoothly in this embodiment.

The state management unit 120 also determines whether the scene is a parking scene in which the vehicle 1 is parked, and when the scene is a parking scene, the state of the parking assistance is set to state 3, the state of the parking assistance in a scene where it is determined whether the scene is a parking scene is set to state 2, and the state of the parking assistance in a scene where it is not determined whether the scene is a parking scene (a scene in which the vehicle is driving on a road) is set to state 1. In other words, when the scene is not a parking scene, the state management unit 120 sets the state of the parking assistance to state 2, which restricts the functions of the parking assistance device 100 more than state 3 and after, and when it is not necessary to determine whether the scene is a parking scene, the state management unit 120 sets the state of the parking assistance to state 2, which restricts the functions of the parking assistance device 100 even more than state 3 and after, and when it is not necessary to determine whether the scene is a parking scene, the state management unit 120 sets the state of the parking assistance to state 1, which further restricts the functions of the parking assistance device 100. Thus, the state management unit 120 corresponds to the "scene determination unit" and "function restriction unit" of the present disclosure.

In other words, when the vehicle is traveling on a road and it is clearly not a parking scene (State 1), the state management unit 120 does not allow the parking space detection unit 140 to function at all. Also, when determining whether or not the scene is a parking scene (State 2), the state management unit 120 only allows the parking space detection unit 140 to detect white lines at a low frequency, but does not allow the parking space detection unit 140 to detect parking spaces or the storage of history by the storage unit 170. In other words, the function restrictions imposed by the state management unit 120 limit the parking space detection by the parking space detection unit 140 and the storage of history by the storage unit 170 depending on the scene.

As a result, parking space detection and history storage are not performed all the time, which reduces the operational load of the parking assistance device 100. Note that it is also possible to perform either the parking space detection restriction or the history storage restriction described above.

In addition, when the vehicle 1 is off the road and the vehicle speed is equal to or lower than a specified value (the above-mentioned speed threshold), the state management unit 120 sets the parking assistance state to State 2, and otherwise sets the state to State 1. In other words, when the vehicle 1 is off the road and the vehicle speed is equal to or lower than a specified value (the above-mentioned speed threshold), the state management unit 120 does not restrict the execution of white line detection by the parking space detection unit 140.

As a result, when the vehicle 1 is traveling at a relatively slow speed off the road, i.e. in a location that can be assumed to be a parking lot, detection of white lines is performed without any restrictions, so that the operational load of determining that it is a parking scene is not unnecessarily incurred.

In addition, the state management unit 120 determines that the scene is a parking scene when the vehicle 1 is off the road, the vehicle speed is below a specified value, and multiple white lines that form approximately right angles with the traveling direction of the vehicle 1 are detected.

In this way, when determining whether a parking scene exists, the white lines indicating an aisle are distinguished from parking frame lines, so that it is possible to avoid erroneously determining that a scene without a parking frame exists as a parking scene. The state management unit 120 may determine that a parking scene exists when either the vehicle 1 is located off the road, the vehicle speed is equal to or lower than a specified value, or multiple white lines that form an angle of approximately right angles with the traveling direction of the vehicle 1 are detected. For example, the fact that the vehicle 1 is located off the road may be excluded from the determination conditions.

The parking space detection unit 140 may be located on the opposite side of the line along which the vehicle 1 travels to the direction in which the vehicle 1 turns, and the parking space detection area may be set to include the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 turns.

As a result, the parking space detection area is limited to an area where a parking space is expected, thereby shortening the processing time required for detection. Also, by limiting the size of the parking space detection area to a size that can accommodate one parking space but not two, it is possible to omit processing when two parking spaces are detected simultaneously.

The parking space detection unit 140 may also select a peripheral image from the peripheral image that is on the opposite side of the direction in which the vehicle 1 turned, across the line along which the vehicle 1 traveled straight, and that includes an area that is located to the side or diagonally forward of the occupant based on the position of the occupant when the vehicle 1 turned, and perform parking space detection based on the selected peripheral image.

This also limits the parking space detection area to the area where a parking space is expected, thereby shortening the processing time required for detection.

In addition, the route calculation unit 150 calculates the parking route based on information about the parking space selected from the surrounding image, i.e., information about the parking space detected by the parking space detection unit 140.

This allows route calculations to be targeted, shortening processing time compared to a configuration that calculates parking routes for all parking spaces.

In addition, if the parking space detection unit 140 detects a parking space at the stopping position, the driving control unit 160 performs automatic parking toward the detected parking space, and if a parking space is not detected at the stopping position, the parking space detection unit 140 may detect a parking space based on a surrounding image captured when turning or before turning.

In other words, the parking assistance device 100 may first detect a parking space from the image captured by the rear camera at the vehicle's stopped position, and if no parking space is detected, may use the image captured at the time of steering to detect a parking space.

In this way, parking space detection is not started speculatively before the vehicle 1 is stopped, so there is no unnecessary operational load for parking space detection. Furthermore, since the probability of not being able to detect a parking space from the image of the rear camera at the stopped position is low, the expected value of the time required for parking space detection is not significantly greater than the time required for one parking space detection, and there is an effect of time reduction by detecting parking instructions based on the operation information or movement information history of the vehicle 1. As a result, the time required for parking can be shortened compared to conventional methods.

The hazard lights may also be turned on during automatic parking. In other words, the driving control unit 160 may output a command to turn on the hazard lights when the vehicle 1 turns from a straight-ahead state, stops at a position where it has turned a predetermined angle or more, and performs automatic driving.

This makes it easier for people around to know that the vehicle is in autonomous driving mode.

(Modification of the first embodiment)
In the following, a modified example of the first embodiment of the present disclosure will be described. In the first embodiment, a scene determination is performed, and when the vehicle 1 moves out of the road, white line detection is started according to the result of the scene determination, and when a periodically drawn white line is detected, parking space detection is started according to the result of the scene determination.

Specifically, the state management unit 120 functions as a scene determination unit and a function restriction unit, and determines whether or not it is a parking scene, and restricts parking space detection and storage of the movement history by the storage unit based on the determination result, but this determination may be left to the occupant. For example, a start button for starting the parking assistance device 100 may be provided, and the parking space detection unit 140 may start parking space detection when the start button is pressed. In other words, the occupant's operation may be substituted for the scene determination.

In the first embodiment, an example was shown in which a surrounding image was stored while the vehicle 1 was traveling, and the parking space detection unit 140 detected a parking space when the vehicle was turned. However, the parking space detection may be performed at all times, rather than only at specific times. In addition, the storage unit 170 may store information on the detected parking space, and the route calculation unit 150 may calculate a route based on the parking space information stored in the storage unit 170.

In the first embodiment, an example is shown in which a parking frame that matches the history is selected based on the position of the occupant at the time of transition from straight driving to turning. However, a parking frame that matches the history may be selected based on the position of the vehicle or the occupant at the time of transition from straight driving to turning. For example, when the occupant is looking at the image of the camera 2 (front camera) installed at the front of the vehicle 1, the front camera approaches the entrance of the parking frame before the occupant, so it is possible to know that the parking frame is available at an earlier timing, and if the parking frame is detected in the front camera image, it may be notified that the parking frame has been detected before the occupant can see it. In other words, regardless of the position of the occupant, it may be determined that there is an available parking frame and transition to turning, so a parking frame that matches the history may be selected based on the position of the vehicle. The entrance of the parking frame is a convenient name given to the area on the aisle in front of the parking frame, and may be called the area adjacent to the short side of the entrance of the parking frame, or may be called the area extending from the parking frame to the aisle side.

Second Embodiment
Hereinafter, a second embodiment of the present disclosure will be described. In the first embodiment, the operation or route until the occupant stops the vehicle in the parking lot is stored as a history, and the parking instruction is regarded as including an instruction of a target parking frame for parking the vehicle. An example was shown in which a parking frame detected by the parking frame detection unit is selected as a target parking frame, which is located on the opposite side of the vehicle's stop position across the path when the vehicle moves straight ahead and matches the history. In other words, the parking position of the subsequent automatic parking is determined by driving to the stop position. When a parking frame is detected, information on the positional relationship of the vehicle 1 with respect to the parking frame at that time is obtained, and by adding information on the vehicle speed and path, the transition of the positional relationship when parking is intended can be predicted. Therefore, at the time when the positional relationship between the vehicle 1 and the parking frame or the positional relationship between the occupant and the parking frame becomes a specific positional relationship, an appropriate notification based on the information on the parking frame detected by the parking frame detection unit is performed, thereby enabling driving to the stop position to be supported. In the second embodiment, driving support to the stop position is disclosed.

In the first embodiment, an example was shown in which the parking space detection area was limited to a size that would not accommodate two parking spaces, but the parking space detection area may be set so that multiple parking spaces can be detected simultaneously. When multiple adjacent parking spaces are detected, driving assistance may be provided up to the stopping position so that the occupant can park in the parking space they intend to park in.

When multiple consecutive parking frames are detected, such as parking frame S1 and parking frame S2 in FIG. 10, the parking frame intended by the occupant may not match the target parking frame (the parking frame for which the target parking position is set). The parking assistance device 100 determines the target parking frame based on the position of the vehicle or the occupant at the time when the vehicle transitions from moving straight to turning when steering. In other words, since there is a range of positions where the target parking frame becomes parking frame S1 and a range of positions where the target parking frame becomes parking frame S2, when the occupant is located near the boundary line separating the two ranges, a mismatch between the parking frame intended by the occupant and the target parking frame is likely to occur.

For example, suppose that the occupant sees parking space S1 diagonally ahead and turns the vehicle 1 with the intention of parking in parking space S1, and starts turning the steering wheel at point P5. If the parking assistance device 100 selects parking space S2 as the target parking space based on the fact that point P5 where the vehicle 1 turns is located in front of parking space S2 (frontage), the target parking space S2 will not match the parking space S1 intended by the occupant.

In the second embodiment, the parking assistance device 100 provides assistance by notifying the driver, thereby guiding the driver to turn the steering wheel while avoiding timing when erroneous estimation is likely to occur. The second embodiment will be described below.

The example shown in FIG. 14 shows a parking lot in which three parking spaces S4, S5, and S6 are arranged in order from the negative side in the X direction. Parking space S4 is a parking space sandwiched between parking space lines L5 and L6, parking space S5 is a parking space sandwiched between parking space lines L6 and L7, and parking space S6 is a parking space sandwiched between parking space lines L7 and L8. In the example shown in FIG. 14, the vacant parking spaces are S5 and S6, and the vehicle 1 is traveling along a passage facing the three parking spaces S4, S5, and S6 from the negative side to the positive side in the X direction.

Figure 14 shows Q1 to Q5, which are the positions of the occupant (driver) at each time when the vehicle 1 passes in front of a parking space. Note that the position of the occupant and the vehicle 1 is shown at position Q1, but the vehicle 1 is omitted and only the occupant is shown at positions Q2 to Q5. Q1 is the position of the occupant at the time when the detection of the parking space S5 is notified, for example. It is advisable to notify the occupant in advance that there will be a countdown and that the driver should turn the steering wheel during the countdown, for example, when the parking assistance device 100 starts detecting the parking space. In this way, the occupant can turn the steering wheel at a timing that suits his/her parking intention and the notification.

For example, while the occupant's position is in the range from Q1 to Q2, the parking assistance device 100 announces "Would you like to park in the parking space in front?" Then, since parking space S6 is farther from the occupant than parking space S5, which is closer, it can be determined that the occupant is being asked whether to park the vehicle 1 in parking space S5. Next, while the occupant's position moves from Q2 to Q3, the parking assistance device 100 counts down "3, 2, 1." Since the occupant has heard in advance that "Please turn the steering wheel during the countdown," if the occupant wants to park in parking space S5, he or she can simply turn the steering wheel accordingly. Then, the target parking position is set to parking space S5. If the occupant does not want to park in parking space S5, he or she does not need to turn the steering wheel.

When the occupant's position reaches Q3, the parking assistance device 100 announces, "Would you like to enter the next space?" The occupant knows from the positional relationship between the parking space and the vehicle 1 that the next space is parking space S6. Next, while the occupant's position moves from Q4 to Q5, the parking assistance device 100 counts down, "3, 2, 1." If the occupant turns the steering wheel during this time, the target parking position is set to parking space S6.

In this way, by receiving a notification at a timing that corresponds to the relative position with respect to the parking space and turning the steering wheel in response to the notification, the occupant can reliably set the target parking position to the parking space where the occupant intends to park.

Next, an example of the operation of the parking assistance device 100 according to the second embodiment will be described. Figures 15 and 16 are flowcharts showing an example of the operation of parking assistance control in the parking assistance device 100 according to the second embodiment. The flow in Figure 16 starts when the button for starting the parking assistance device 100 is pressed. It is assumed that at the start, the vehicle 1 is traveling at low speed through the aisles of the parking lot, and its path is approximately perpendicular to the detected white lines. The flowcharts in Figures 15 and 16 shown below show the parts related to notifications, and omit the parts not related to notifications. Therefore, the processes shown in Figures 11 to 13 in the first embodiment may be executed in parallel with the processes in the following flowcharts.

When the button to start the parking assistance device 100 is pressed, the parking assistance device 100 starts speed control (step S201) as shown in FIG. 15. This speed control is a control to suppress the speed of the vehicle 1 to increase the time for the occupant to judge the parking space. For example, when the vehicle speed is controlled to 3 km/h, the vehicle 1 passes in front of a parking space 2.5 m apart in 3 seconds.

The parking assistance device 100 further starts parking space detection (step S202). The parking space detection is performed based on the image from the front camera. The speed control and parking space detection are not limited to the above steps, but are continuously performed in the processing from step S202 onwards.

After step S202, the parking assistance device 100 determines whether or not a parking space has been detected (step S203). If the result of the determination is that a parking space has not been detected (step S203, NO), the processing of step S203 is repeated.

On the other hand, if a parking space is detected (step S203, YES), the parking assistance device 100 calculates the time margin based on the distance to the parking space and the vehicle speed (step S204). The time margin is, for example, the time until the vehicle 1 reaches the guidance end point. The guidance end point is a position corresponding to Q3 or Q5 in FIG. 14, and is the position where the countdown by the parking assistance device 100 ends.

After step S204, the parking assistance device 100 determines whether the time margin is 5 seconds or less (step S205). If the time margin exceeds 5 seconds (step S205, NO), the process returns to step S204.

If the time remaining is 5 seconds or less (step S205, YES), the parking assistance device 100 starts outputting message A (step S206). That is, it waits until the time remaining is 5 seconds before outputting message A. Message A may be a message including a countdown, such as "Would you like to enter this parking space? 3, 2, 1." In this way, the parking assistance device 100 adjusts the time so that the vehicle 1 will reach the guidance end point when the output of message A ends.

Next, the parking assistance device 100 determines whether the absolute value of the steering angle is greater than 5 degrees (step S207). Note that the processes of steps S207 and S208 are executed while message A is being output in step S206. In other words, it is determined whether the steering wheel is turned during the countdown.

If the result of the determination is that the absolute value of the steering angle is greater than 5 degrees (step S207, YES), the process proceeds to step S218. On the other hand, if the absolute value of the steering angle is less than 5 degrees (step S207, NO), the parking assistance device 100 determines whether the time margin is 0 seconds or less (step S208). This time margin is the same as the number notified as message A. In other words, it is determined whether the steering wheel is turned before the countdown is notified as zero. If the result of the determination is that the time margin is greater than 0 seconds (step S208, NO), the process returns to step S207. On the other hand, if the time margin is less than 0 seconds (step S208, YES), that is, if zero has been notified, the parking assistance device 100 sets the timer to 0 seconds and starts timing (step S209). The timer counts the elapsed time since starting timing.

After step S209, the parking assistance device 100 outputs a beep sound such as "beep" (step S210). The beep sound is a sound to notify that the parking space detected in step S203 is not the target parking space because no steering operation was detected during the guidance period, and continues only while the processing of this step S210 is being executed.

As shown in FIG. 16, after step S210, the parking assistance device 100 determines whether a parking space has been detected (step S211). The parking space to be determined in step S211 is not the first parking space detected in step S203, but the next parking space. After detecting a parking space in step S203, parking space detection continues, and if the next parking space is detected, step S211 immediately determines YES. If the determination result is that a parking space has not been detected (step S211, NO), the parking assistance device 100 determines whether the timer is less than 1 second (step S212).

If the timer is greater than one second (step S212, NO), the process returns to step S203. Since step S203 is a step for detecting the first parking space, if the next parking space is not detected within one second, it means that it is determined that the situation is not one in which parking spaces should be detected continuously. In other words, if the next parking space is not detected within one second, the next detected parking space is treated as the first parking space. On the other hand, if the timer is less than one second (step S212, YES), the process returns to step S210. Since step S210 is a beep sound step, a beep sound is output for an upper limit of one second.

Returning to the judgment of step S211, if the next parking space is detected (step S211, YES), the parking assistance device 100 calculates the time margin (step S213). The time margin is, for example, the time until the vehicle 1 reaches the guidance end point.

After step S213, the parking assistance device 100 determines whether the time margin is 4 seconds or less (step S214). If the time margin exceeds 4 seconds (step S214, NO), the process returns to step S213.

On the other hand, if the time remaining is 4 seconds or less (step S214, YES), the parking assistance device 100 starts outputting message B (step S215). That is, it waits until the time remaining is 4 seconds before outputting message B. The time threshold being shorter than step S205 corresponds to message B being shorter than message A, since message B is guidance for the second parking space. Message B may be, for example, a countdown message such as "Shall I enter next? 3, 2, 1." Furthermore, the parking assistance device 100 adjusts the time so that the vehicle 1 reaches the guidance end point when the output of message B ends.

Next, the parking assistance device 100 determines whether the absolute value of the steering angle is greater than 5 degrees (step S216). Note that the processes of steps S216 and S217 are executed while message B is being output. In other words, it is determined whether the steering wheel is turned during the countdown.

If the result of the determination is that the absolute value of the steering angle is 5 degrees or less (step S216, NO), the parking assistance device 100 determines whether the time margin is 0 seconds or less (step S217). This time margin is the same as the number notified as message A. In other words, it is determined whether the steering wheel is turned before the countdown is notified as zero. If the result of the determination is that the time margin is greater than 0 seconds (step S217, NO), the process returns to step S216.

On the other hand, if the time remaining is 0 seconds or less (step S217, YES), the process returns to step S209. In other words, if the time remaining is reported as zero, a beep sound such as "beep" is emitted to notify the driver that the detected parking space is not to be used as the target parking space.

Returning to the judgment in step S216, if the absolute value of the steering angle is greater than 5 degrees (step S216, YES), the process transitions to step S218. The same applies if the result is YES in step S207. In step S218, the parking assistance device 100 starts outputting message C (step S218). Message C is a notification that the target parking frame has been determined, and may be, for example, a message such as "Automatic parking will be performed in this frame. Please park with your back to the frame." After step S218, this control ends.

According to the second embodiment configured as described above, even if there are consecutive vacant parking spaces, it is possible to accept an action to specify a parking space so that the space in which the vehicle 1 will be parked is not unclear.

The flowcharts shown in Figures 15 and 16 are an example of control that guides the driver to turn the steering wheel when the target parking space can be identified, and an example of control for notification when parking in the identified parking space is shown below. The process of the flowchart shown in Figure 17 is an example of control for notification when parking, and this may be performed after step S218 in Figure 16.

The flowchart shown in FIG. 17 starts with the premise that the occupant has turned the steering wheel and is moving forward toward the reverse start position (turning position). To detect that the reverse start position has been reached, the parking assistance device 100 determines whether the vehicle speed is 0 (step S219). If the result of the determination is that the vehicle speed is not 0 (step S219, NO), the processing of step S219 is repeated.

On the other hand, if the vehicle speed is 0 (step S219, YES), the vehicle has reached the reverse start position, so the parking assistance device 100 superimposes a frame image on the image of the surroundings of the vehicle 1, calculates the parking path, and outputs message D (step S220). Message D is a request to confirm the target parking frame, and may be, for example, a message such as "If this parking frame is acceptable, put the gear in reverse and release the brake."

Next, the parking assistance device 100 determines whether or not a state in which the gear is in R and the brake is released has been detected (step S221). If the result of the determination is that a state in which the gear is in R and the brake is released has not been detected (step S221, NO), the process of step S221 is repeated. In other words, the device waits for the occupant to confirm the target parking frame.

On the other hand, if the gear is in R and the brake is released (step S221, YES), the parking assistance device 100 outputs a message E notifying the start of automatic parking (step S222). The message E may be, for example, a message such as "Automatic parking will start. Please be careful of the area around the vehicle."

After step S222, the parking assistance device 100 automatically drives the vehicle 1 along the parking path (step S223).

After step S223, the parking assistance device 100 determines whether the vehicle 1 has reached the end point (step S224). If the result of the determination is that the vehicle 1 has not reached the end point (step S224, NO), the process returns to step S223.

On the other hand, if the vehicle 1 reaches the end point (step S224, YES), the parking assistance device 100 automatically stops the vehicle 1 and outputs a message F notifying the end of automatic parking (step S225). The message F may be, for example, a message such as "Automatic parking has ended. Please put the gear in park."

After step S225, the parking assistance device 100 determines whether or not it has detected that the gear has been put in P (step S226). If the result of the determination is that it has not detected that the gear has been put in P (step S226, NO), the processing of step S226 is repeated. On the other hand, if it has detected that the gear has been put in P (step S226, YES), this control ends.

In order to avoid complicating the flowchart, illustrations and explanations of exception handling, such as when the occupant does not follow the instructions in the message, are omitted. This exception handling can be implemented with appropriate supplementation. For example, there may be cases where the occupant turns the steering wheel while a parking space is being detected, but has no intention of parking in the detected parking space and simply turns down an aisle. As an exception handling for such a case, in step S219, if the vehicle speed does not become 0 even after a predetermined time has elapsed, it may be determined that the steering operation is not an operation for indicating a parking space, and the process may transition to step S203.

In this way, when multiple parking frames are detected, such as parking frame S1 and parking frame S2 in FIG. 10, support by notification can prevent erroneous setting of the target parking frame. However, when only one parking frame is detected, as in FIG. 6, no erroneous setting of the target parking frame occurs, so support by notification is ineffective and may be annoying to the occupant.

Therefore, when only one parking space is detected, it is advisable to make the number of notifications smaller than when multiple parking spaces are detected. For example, when multiple parking spaces are detected, a notification is given of the inquiry into the intention to park and the time remaining, such as "Would you like to park in this parking space? 3, 2, 1," but when only one parking space is detected, it is also acceptable to only be notified of the inquiry into the intention to park, such as "Would you like to park in this parking space?" Furthermore, when only one parking space is detected, it is also acceptable to make it so that nothing is notified.

Third Embodiment
Hereinafter, a third embodiment of the present disclosure will be described. If the parking frame intended for parking is reliably set as the target parking frame, the driver can save the trouble of checking the target parking frame and correcting the target parking frame. Therefore, the parking assistance device 100 may allow the driver to steer the vehicle so as to identify the parking frame intended for parking, and set the target parking frame based on the observation and analysis of the steering behavior.

Figures 18 to 23 are diagrams for explaining examples of steering behavior of the vehicle 1 in the third embodiment.

In the example shown in FIG. 18, etc., a parking lot is shown in which four parking spaces S7, S8, S9, and S10 are arranged in order from the negative side in the X direction. The parking space S7 is a parking space sandwiched between parking space lines L9 and L10, the parking space S8 is a parking space sandwiched between parking space lines L10 and L11, the parking space S9 is a parking space sandwiched between parking space lines L11 and L12, and the parking space S10 is a parking space sandwiched between parking space lines L12 and L13. In the example shown in FIG. 18, etc., the vacant parking spaces are S8 and S9, and the vehicle 1 proceeds along a passage facing the four parking spaces S7, S8, S9, and S10 from the positive side to the negative side in the X direction, and a steering action is shown to identify the parking space in which the vehicle 1 intends to park.

The parking assistance device 100 detects a turn in the opposite direction to the turning direction at the time the vehicle stopped, as indicated by the history, and selects a parking frame that matches the history based on either the position of the vehicle when the reverse turn started, or the position of the vehicle when the reverse turn ended, or the direction of the vehicle body. A reverse turn is a steering that swings the front end of the vehicle body toward the opposite side of the stopping position, that is, toward the parking frame, and may be called a reverse steering. For example, as shown in FIG. 18, when the occupant steers the vehicle body so that the corner of the front end of the vehicle body approaches the parking frame in which the vehicle is intended to park, the parking frame to which the corner of the vehicle body is closest may be determined and the target parking frame may be set. In the example of FIG. 18, the parking frame S9 corresponding to the position when the vehicle 1 moves the furthest in the +Y direction is set as the target parking frame.

In addition, as shown in FIG. 19, for example, when the occupant turns the steering wheel toward the parking position, the parking assistance device 100 may identify a parking space indicated by a line W extending forward from the right side of the vehicle body, i.e., the side facing the parking space, and set the target parking space. In the example of FIG. 19, the parking space S8 overlapping with the line W is set as the target parking space.

In addition, the parking assistance device 100 may, for example, identify the moving direction M of the right front corner immediately before the vehicle 1 turns left, as shown in FIG. 20, and set a parking space in the moving direction M (the direction of the arrow M) as the target parking space. In the example of FIG. 20, a parking space S9 that overlaps with the moving direction M is set as the target parking space.

Also, as shown in FIG. 21, for example, the parking assistance device 100 may specify the direction E (the direction of the arrow E) of the right front corner as seen by the occupant just before the vehicle 1 turns left, and set the parking frame in that direction E as the target parking frame. In the example of FIG. 21, the parking frame S8 overlapping with the direction E is set as the target parking frame. In other words, the steering to specify the intended parking frame may be any of steering to the opposite side of the intended parking frame at the entrance of the intended parking frame, steering to temporarily turn to the side of the intended parking frame at the entrance of the intended parking frame, steering to direct the vehicle body to the intended parking frame, and steering to bring the vehicle body closer to the intended parking frame. When specifying a parking frame based on the orientation (direction) of the vehicle body, it may be any of the direction pointed by the side on the opposite side to the parking position, the direction of travel of the corner of the tip of the vehicle body on the opposite side to the parking position, or the direction in which the occupant looks at the corner of the tip of the side on the opposite side to the parking position.

When identifying a parking space based on the direction, the parking assistance device 100 may determine the area from the entrance on the aisle side of the parking space to the center of the parking space as the determination area that represents the parking space. This determination area is the range that the occupant looks at when selecting a parking space, and when the occupant operates the vehicle 1 toward this determination area, the occupant operates the vehicle 1 toward this determination area. For example, the parking assistance device 100 may set the center of each determination area, evaluate the distance between a line extending in the direction and each center, and set the parking space that is the smallest distance from the line extending in the direction among the centers of the multiple determination areas as the target parking space.

The steering to identify the parking space where the occupant intends to park may be steering based on a reverse steering operation as shown in Figures 22 and 23. A reverse steering operation is, for example, a steering action in which the driver makes a small turn to the right before making a large turn to the left, causing the trajectory to expand to the right. Reverse steering can be a habitual habit, and there are many people who perform reverse steering unconsciously. Therefore, even if the occupant performs a reverse steering operation when steering the vehicle 1 for parking, there is no risk that the behavior will be seen as unusual by those around.

The position where the reverse steering operation was performed may be specified as the position where the reverse steering operation started. For example, as shown in FIG. 22, if the position of the front end of the vehicle body at the time when the reverse steering operation started is closest to the center line C1 of the parking space S9 among the multiple parking spaces, the parking assistance device 100 sets the target parking space to the parking space S9. Also, as shown in FIG. 23, if the position of the front end of the vehicle body at the time when the reverse steering operation started is closest to the center line C2 of the parking space S8 among the multiple parking spaces, the parking assistance device 100 sets the target parking space to the parking space S8. In other words, a parking space that matches the history may be selected based on either the position of the vehicle when the reverse turning started, or the position of the vehicle when the reverse turning ended, or the direction of the vehicle body.

When multiple parking spaces are detected, such as parking space S1 and parking space S2 in FIG. 10, steering to specify the parking space in which the vehicle is intended to park is effective. However, when only one parking space is detected, as in FIG. 6, no error in setting the target parking space occurs, and therefore no special steering is required.

Therefore, the parking assistance device 100 may request steering to specify the parking space in which the vehicle is intended to park only when multiple parking spaces are detected, and may not notify the driver of steering or may only notify the driver briefly when only one parking space is detected.

For example, if there are multiple vacant parking spaces on the right side of the vehicle 1, the parking assistance device 100 may announce a predetermined message. The predetermined message may be, for example, any of the following: "Turn the steering wheel in front of the parking space where you want to park," "Lightly turn the front of the vehicle body toward the parking space where you want to park," "Bring the corner of the vehicle body closer to the parking space where you want to park," or "Before turning the front of the vehicle body to the left, point the front of the vehicle body toward the parking space where you want to park."

Following the predetermined message, the parking assistance device 100 may announce, for example, "Automatic parking will begin if you park with your back to the parking space." Also, if only one parking space is detected, the parking assistance device 100 may not request steering to specify the parking space in which the driver wants to park, but may simply announce, "Automatic parking will begin if you park with your back to the parking space."

In addition, the parking assistance device 100 does not need to repeat the notification for each parking space even when there are multiple vacant parking spaces. For example, if there is no steering to specify the intended parking space in front of the parking space S1, the target parking space candidates are narrowed down to one, so there is no need to repeat the announcement for the parking space S2. For example, when the parking assistance device 100 detects the second parking space, it may announce only once, "Please gently move the front end of the vehicle toward the parking space you want to park in." And, if there is no corresponding action in front of the first parking space S1, the parking assistance device 100 can determine the intention to park even if there is no corresponding action in front of the next parking space S2, so it may set the parking space S2 as the target parking space.

In addition, the above embodiments are merely examples of concrete ways of implementing the present disclosure, and the technical scope of the present disclosure should not be interpreted in a limiting manner based on them. In other words, the present disclosure can be implemented in various forms without departing from its gist or main features.

The device disclosed herein is useful as a parking assistance device and parking assistance method that can perform automatic parking smoothly in a short time.

REFERENCE SIGNS LIST 1 vehicle 2 camera 10 operation device 20 HMI device 30 vehicle control device 40 navigation device 100 parking assistance device 101 CPU
102 ROM
103 RAM
104 I/O
105 IMP
REFERENCE SIGNS LIST 110 Operation reception unit 120 Status management unit 130 Image acquisition unit 140 Parking space detection unit 150 Route calculation unit 160 Travel control unit 170 Storage unit 180 Display image output unit

Claims (20)

A parking space detection unit that detects a parking space based on a surrounding image showing the surroundings of the vehicle;
A storage unit that stores a history of either an operation by a passenger of the vehicle or a movement of the vehicle;
an instruction detection unit that detects a parking instruction from the occupant based on the history;
a vehicle control unit that performs automatic parking of the vehicle after the instruction detection unit detects the parking instruction;
Equipped with
The instruction detection unit determines that the parking instruction has been detected when the history indicates that the vehicle has stopped after steering from a straight line to a turn and the occupant has performed a predetermined parking instruction operation.
Parking assistance device. The predetermined parking instruction operation includes at least one of changing a gear position, releasing a steering wheel, releasing a brake, turning on a hazard lamp, and operating a button corresponding to a parking instruction.
2. The parking assistance device according to claim 1. a scene determination unit that determines whether or not a scene is a parking scene in which the vehicle is parked;
A function limiting unit that limits a function of the parking assistance device when the scene is not a parking scene;
Further equipped with
The function restriction performed by the function restriction unit is
When the scene is not a parking scene, the parking space detection unit is restricted from detecting a parking space;
If the scene is not a parking scene, limiting the storage of the history by the storage unit;
At least one of
2. The parking assistance device according to claim 1. A route calculation unit that calculates a route for the automatic parking is further provided.
The parking instruction includes an instruction for a target parking space in which the vehicle is to be parked.
The instruction detection unit selects, as a target parking frame, a parking frame that is located on the opposite side of the vehicle stop position across the path of the vehicle when traveling straight ahead and that matches the history, among the parking frames detected by the parking frame detection unit;
The route calculation unit calculates the route based on information of the target parking frame.
2. The parking assistance device according to claim 1. The instruction detection unit selects a parking frame that matches the history based on a position of the vehicle or an occupant at a time when the vehicle transitions from moving straight to turning, as indicated by the history.
5. A parking assistance device according to claim 4. The instruction detection unit is
Detecting a turn in a direction opposite to a turning direction at the time when the vehicle stopped, which is indicated by the history;
the position of the vehicle when the reverse turn began; or
selecting a parking frame that matches the history based on either the position of the vehicle or the orientation of the vehicle body when the reverse turn is completed;
5. A parking assistance device according to claim 4. The storage unit stores either the surrounding image captured while the vehicle is traveling or information on a parking space detected while the vehicle is traveling,
The instruction detection unit selects the target parking frame from the information of the parking frame detected by the parking frame detection unit or the information of the parking frame detected while the vehicle is traveling and stored in the storage unit based on the surrounding image captured while the vehicle is traveling and stored in the storage unit.
5. A parking assistance device according to claim 4. The parking space detection unit is
A parking space detection is performed based on the surrounding image captured when the vehicle is stopped.
When a parking frame is detected based on the peripheral image captured when the vehicle is stopped, the target parking frame is selected from information on the parking frame based on the peripheral image captured when the vehicle is stopped.
8. A parking assistance device according to claim 7. The position of the occupant is the position of the head of the occupant.
7. A parking assistance device according to claim 5 or 6. The orientation of the vehicle body is
The direction in which the side surface of the vehicle body facing the opposite direction is pointing, or
The direction of travel of the tip corner of the vehicle body on the opposite side, or
The direction in which the occupant looks at the tip corner of the vehicle body on the opposite side,
Either
7. A parking assistance device according to claim 6. When it is determined that the parking instruction has been detected, the vehicle control unit outputs a command to turn on a hazard lamp.
2. The parking assistance device according to claim 1. A notification unit that notifies an occupant of the vehicle is further provided.
The notification includes a notification based on information of the parking space detected by the parking space detection unit.
2. The parking assistance device according to claim 1. The notification includes an inquiry about an intention to park in the parking space, and is notified when the parking space and the vehicle body are in a predetermined positional relationship.
13. A parking aid according to claim 12. The notification includes a notification of a timing to steer the vehicle.
13. A parking aid according to claim 12. The notification includes a notification requesting steering to specify a parking frame in which parking is intended.
13. A parking aid according to claim 12. The steering to designate the intended parking frame is
Steering to the opposite side of the intended parking space at the entrance of the intended parking space,
Steering that temporarily turns toward the intended parking space at the entrance of the intended parking space;
Steering to point the vehicle towards the intended parking space,
Steering to bring the vehicle closer to the intended parking space,
Contains any of the following:
16. A parking aid according to claim 15. When the parking frame detection unit detects one parking frame, the notification unit reduces the notification compared to when the parking frame detection unit detects a plurality of parking frames.
A parking assistance device according to any one of claims 12 to 16. The function restriction unit does not restrict the execution of detection of a white line by the parking space detection unit when the vehicle is located outside a road and the vehicle speed is equal to or lower than a specified value.
4. The parking assistance device according to claim 3. The scene determination unit
The vehicle is off the road and the vehicle speed is equal to or lower than a specified value; and
a plurality of white lines are detected, the angles of which with respect to the traveling direction of the vehicle are within a predetermined range;
If either or both of the above conditions are met, the scene is determined to be a parking scene.
4. The parking assistance device according to claim 3. Detecting a parking space based on a surrounding image showing the surroundings of the vehicle;
storing a history of either the operation of an occupant of the vehicle or the motion of the vehicle;
detecting a parking instruction from the occupant based on the history;
after detecting the parking instruction, automatically parking the vehicle;
having
In the step of detecting the parking instruction, when the history indicates that the vehicle has been steered from a straight-ahead direction to a turn and then stopped, and when the occupant has performed a predetermined parking instruction operation, it is determined that the parking instruction has been detected.
Parking assistance methods.

Images