JP2019077271A - Electronic device and parking assistance method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic device capable of drawing a traveling prediction line even when the vehicle is stopped while facilitating drawing of a traveling prediction line linked to a steering angle of the steering wheel. An in-vehicle device 100 of the present invention includes an image pickup unit 130 that captures an image of the periphery of a vehicle, an estimation means that estimates a turning radius of the vehicle based on a moving image captured by the image pickup unit 130, and a steering angle of a vehicle handle. A display unit that displays an image captured by the image pickup unit 130, an acquisition means for acquiring steering angle information regarding the steering wheel, a storage means for storing the relationship between the estimated turning radius and the steering wheel steering angle at the time of the estimation, and a display unit. It has 160 and a drawing means for drawing a traveling prediction line linked to the steering angle of the steering wheel on the image displayed on the display unit 160. When the vehicle is stopped, the drawing means draws a travel prediction line by reading the turning radius corresponding to the steering angle of the steering wheel from the storage means. [Selection diagram] Fig. 2

Description

  The present invention relates to an electronic device provided with a parking assistance function, and to drawing of a predicted travel line displayed on a captured image when parking.

  A driving support system using an image of an imaging camera mounted on a vehicle has been put to practical use. For example, a side view camera is used to warn of obstacles on the side of the vehicle, a rear view camera is used to support parking behind the vehicle, a top view system is used to display an overhead image of the vehicle and obstacles around the vehicle We warn about things.

  In parking assistance using a vehicle-mounted camera, a rear image is displayed on the display in conjunction with the back gear of the vehicle, and a predicted travel line linked to the steering angle is drawn on the rear image. For example, as shown in FIG. 1, a boundary line (hatching) 20 indicating the boundary of the parking space is displayed in the rear image 10, and further, on the rear image 10, a travel prediction line 30 interlocking with the steering angle is detected. It is drawn. Reference numeral 32 denotes a predicted travel line when the vehicle travels backward while maintaining the current steering angle. The driver can accurately park the vehicle in the parking space by moving backward while adjusting the steering angle so as to make the predicted travel lines 30, 32 coincide with the boundary line 20.

  Generally, in order to draw a travel prediction line, the correspondence between the steering angle of the steering wheel of the vehicle and the turning radius needs to be known. However, since it is troublesome to measure such correspondence in advance, in Patent Document 1, the turning center and turning radius of the vehicle are obtained from the moving image obtained by imaging the rear of the vehicle, and the travel prediction line is drawn using this.

Patent No. 6020736

  However, the conventional parking assistance method has the following problems. In order to draw a travel prediction line linked to the steering angle of the steering wheel, measuring in advance the correspondence relationship between the steering wheel steering angle and the turning radius in various types of vehicles is a labor-intensive and time-consuming task. . Therefore, as a result, the drawing of the travel prediction line can be applied only to some types of vehicles. Further, as in Patent Document 1, in a system in which the turning center and turning radius are estimated from the moving image of the on-vehicle camera, since the moving image can not be obtained while the vehicle is stopped, it is possible to estimate the turning radius. Can not. Therefore, there is the inconvenience that, even when the steering wheel is rotated while the vehicle is in the stopped state, it is not possible to draw a predicted travel line linked thereto.

  According to the present invention, it is possible to solve the above-described conventional problems and to draw a travel prediction line even while the vehicle is stopped while facilitating drawing of a travel prediction line linked to the steering angle of the steering wheel as compared with the conventional. An electronic device having a parking assistance function and a parking assistance method are provided.

  An electronic device provided with a parking assistance function according to the present invention comprises: an imaging means for imaging the periphery of the vehicle; an estimation means for estimating a turning radius of the vehicle based on a moving image imaged by the imaging means; An acquisition means for acquiring steering wheel steering angle information related to a corner, a storage means for storing a relationship between the estimated turning radius and the steering wheel steering angle at the time of the estimation, and an image captured by the imaging means A display means and a drawing means for drawing a travel prediction line linked to a steering angle on the image displayed on the display means, wherein the drawing means is used when the vehicle is stopped or a moving image is displayed. When it can not be obtained, the travel prediction line is drawn by reading the turning radius corresponding to the steering angle from the storage means.

  In one embodiment, the pre-rendering means draws the predicted travel line according to the turning radius estimated from the moving image when the vehicle is moving or when a moving image can be obtained. In one embodiment, the display means displays the predicted travel line on a rear image captured by the imaging means when the vehicle reverses. The display means displays the predicted travel line on the overhead image of the vehicle when the vehicle moves forward. In one embodiment, the storage means updates the relationship between the turning radius and the steering angle when the turning radius is estimated by the estimation means.

  A parking assistance method in an electronic device having a parking assistance function according to the present invention determines whether or not the own vehicle is stopped, and when it is determined that the own vehicle is not stopped, an image captured by the imaging camera The turning radius of the vehicle is estimated based on the moving image of the data, the travel prediction line is drawn on the captured image based on the estimated turning radius, and the relationship between the estimated turning radius and the steering angle is stored in storage means If it is determined that the vehicle is at a standstill, the steering wheel steering angle is acquired, the turning radius corresponding to the acquired steering wheel steering angle is acquired from the storage means, and the travel prediction line is calculated based on the acquired turning radius. Draw on the captured image.

  According to the present invention, since the relationship between the estimated turning radius and the steering angle at the time of estimation is stored, while the vehicle is at a stop, the stored turning radius is read to draw a travel prediction line While driving, it is possible to draw a travel prediction line based on the turning radius estimated from the moving image. Furthermore, it becomes possible to develop the drawing of the predicted travel line to other vehicle types.

It is a figure which shows the example of the driving | running | working prediction line displayed at the time of parking. It is a block diagram showing composition of an in-vehicle device concerning an example of the present invention. It is a block diagram showing the functional composition of the parking assistance program concerning the example of the present invention. It is a figure explaining the method to calculate turning radius from a moving image. It is an example of the table which memorize | stores the correspondence of the steering wheel angle | corner and turning radius which concern on the Example of this invention. It is a flow chart explaining parking support operation concerning an example of the present invention. It is a figure showing an example of drawing of a run prediction line interlocked with a steering wheel steering angle concerning an example of the present invention. It is a figure showing an example of drawing of a run prediction line interlocked with a steering wheel steering angle concerning an example of the present invention.

  Next, embodiments of the present invention will be described in detail. The electronic device according to the present invention can be a computer device mounted on a vehicle, a portable information terminal device (for example, a smartphone, a laptop personal computer device). The electronic device according to the present invention has at least a parking assist function using an on-vehicle camera, and can be equipped with various functions other than that. For example, even if it has a navigation function, a communication function, etc. Good.

  Next, embodiments of the present invention will be described with reference to the drawings. FIG. 2 is a block diagram showing the configuration of the in-vehicle apparatus according to the embodiment of the present invention. The on-vehicle apparatus 100 according to the present embodiment includes an input unit 110, a navigation unit 120, an imaging unit 130, a travel information acquisition unit 140, an audio output unit 150, a display unit 160, a communication unit 170, a storage unit 180, and a control unit 190. It consists of

  The input unit 110 receives an instruction from the user and provides the instruction to the control unit 190. The navigation unit 120 calculates the position of the vehicle based on GPS signals and a self-contained navigation sensor (for example, a gyro sensor, an acceleration sensor, etc.), and guides a road map or the like around the vehicle position. Further, the navigation unit 120 searches for a route from the current location to the destination based on the map data, and guides the searched route.

  The imaging unit 130 includes one or more imaging cameras that image at least the rear of the vehicle. Furthermore, the imaging unit 130 can include a plurality of imaging cameras for imaging the front and the side of the vehicle. The image data captured by the imaging unit 130 is provided to the control unit 190, and the control unit 190 displays the captured image data on the display unit 160 in the parking assistance mode to perform parking assistance.

  The travel information acquisition unit 140 acquires information on the travel of the vehicle. In one embodiment, the travel information acquisition unit 140 acquires information on the steering angle of the vehicle, information on gear position, vehicle speed information, and the like via the in-vehicle bus. These pieces of information are used to control parking assistance.

  The voice output unit 150 voice-outputs the guidance of the navigation unit 120, and outputs a voice for parking assistance in the parking assistance mode. The display unit 160 displays a road map by the navigation unit 120, and displays a rear image and a top view image of the vehicle in the parking assistance mode.

  The communication unit 170 enables wired or wireless data communication with an external device, an external network, or the like. The storage unit 180 can store map data used by the navigation unit 120, and can store application software for executing functions that can be implemented in the in-vehicle apparatus 100.

  The control unit 190 controls the overall operation of the in-vehicle apparatus 100, and in a preferred embodiment, the control unit 190 performs parking assistance when the vehicle is parked. For example, the control unit 190 includes a microcontroller or processor including a ROM / RAM or the like, and controls parking assistance by executing a parking assistance program stored therein.

  FIG. 3 is a block diagram showing a functional configuration of the parking assistance program of the present embodiment. The parking support program 200 of this embodiment includes a parking mode determination unit 210, a parking support image generation unit 220, a steering wheel steering angle acquisition unit 230, a turning radius estimation unit 240, a travel prediction line information storage unit 250, and a travel prediction line drawing unit 260. including.

  The parking mode determination unit 210 determines whether the vehicle has transitioned to a parking operation. Specifically, when the gear position information acquired from the traveling information acquisition unit 140 is a back gear, the parking mode determination unit 210 determines that the vehicle is in the reverse parking mode. In addition, when the gear position information acquired from the traveling information acquisition unit 140 is a drive gear and the vehicle speed is equal to or less than a predetermined speed, the parking mode determination unit 210 determines that the forward parking mode is selected. However, the forward parking mode is not necessarily essential but may be optional.

  When it is determined by the parking mode determination unit 210 that the parking mode is reverse or forward, the parking assistance image generation unit 220 generates a parking assistance image based on the image data captured by the imaging unit 130. In the forward parking mode, the parking assistance image is a bird's-eye view image looking down on the vehicle, and in the reverse parking mode, the parking assistance image is a rear image obtained by imaging the rear of the vehicle. The generated parking assistance image is displayed on the display unit 160.

  The steering wheel steering angle acquisition unit 230 acquires information on the steering wheel steering angle from among the traveling information acquired via the traveling information acquisition unit 140. The steering angle represents the steering angle of the steering wheel. The handle is usually rotatable about 400 degrees in one direction, and therefore about 800 degrees in both directions. When parking is performed, the driver appropriately changes the angle of the steering wheel, but information on the steering angle according to the change is acquired.

  The turning radius estimation unit 240 estimates the turning radius of the vehicle based on the moving image captured by the imaging unit 130 when the vehicle is moving. As shown in FIG. 4, the end P1 of the line segment L in the vehicle length direction on the image data captured from the vehicle M is determined, and the movement distance is determined from the difference between the ends P1 and P2 on the continuous image data The movement direction Δθ formed by the line segment L along the vehicle length direction and the straight line passing through the end portions P1 and P2 corresponds to the rotation center angle Δθ, and ΔL = R · Δθ. The turning radius R is calculated from the relationship. Then, the turning center Q is calculated from the turning radius R and the position of the end P1. Alternatively, the turning center Q is calculated on the line WR connecting the rear wheels from the positional relationship with the host vehicle M. The details of the calculation method are also described in Patent Document 1.

  The travel prediction line information storage unit 260 stores the relationship between the turning radius estimated by the turning radius estimation unit 240 and the steering angle when the steering wheel is estimated. For example, as shown in the table of FIG. 5, the relationship between the steering angle [°] and the turning radius [m] is stored. For example, the minus sign indicates that the handle rotation direction is counterclockwise, and the plus sign indicates that the handle rotation direction is clockwise. There is no initial value in the table, and whenever the turning radius is estimated from the moving image, the relationship between the steering angle and the turning radius is stored in the correspondence table. In addition, if the steering wheel angles overlap, the latest correspondence relationship is updated. For example, when the air pressure of the tire changes depending on the number of people in the vehicle, the turning radius changes, so it is desirable to update to the latest correspondence.

  The traveling prediction line drawing unit 260 generates a parking assistance image based on the traveling prediction line based on the turning radius estimated by the turning radius estimation unit 240 when the vehicle is moving, that is, when a moving image can be obtained. It draws on the parking assistance image generated by the part 220. In this case, the turning center may be calculated by the turning radius estimation unit 240 or may be obtained from the positional relationship with the host vehicle. On the other hand, when the vehicle is at a standstill, that is, when it is not possible to obtain a moving image, the predicted travel line information storage unit 260 is operated using the current steering wheel angle acquired by the steering wheel angle acquisition unit 230 as a key. The turning radius is read from the created table (FIG. 5), and a travel prediction line based on the read turning radius is drawn on the parking assistance image. The turning center in this case can be obtained from the positional relationship with the vehicle.

  Next, the parking assistance operation of the present embodiment will be described with reference to the flowchart of FIG. First, it is determined by the parking mode determination unit 210 whether or not it is the parking assistance mode (S100). For example, if the gear position is the back gear, it is determined that the reverse parking mode is on the other hand, and if the gear position is the drive gear and the vehicle speed is low, it is determined that the forward parking mode is. In the determination of the forward parking mode, the fact that a parking frame or the like is captured in the image data of the front on-vehicle camera may be added to the determination condition.

  If it is determined that the parking mode is set, the parking assistance image generation unit 220 generates a parking assistance image based on the image data provided from the imaging unit 130 and causes the display unit 160 to display the parking assistance image (S102). In the reverse parking mode, the display unit 160 displays an image obtained by capturing the rear of the vehicle, and in the forward parking mode, the display unit 160 displays an overhead image around the vehicle.

  Next, the travel prediction line drawing unit 260 determines whether the vehicle is at a stop (S104). For example, when the vehicle speed information acquired from the traveling information acquisition unit 140 is zero, when the gear position is the parking gear, and when the side brake is on, it is determined that the vehicle is stopped. When it is determined that the vehicle is not stopped, that is, when it is determined that the vehicle is moving, the travel prediction line drawing unit 260 causes the turning radius estimation unit 240 to estimate the turning radius from the moving image, and the estimated turning radius The travel prediction line is drawn on the parking support image based on the (S106).

  FIG. 7 shows an example of a predicted travel line in the reverse parking mode. As shown in the figure, the display unit 160 displays, for example, a parking assistance image 300 obtained by imaging the rear of the vehicle, and the parking assistance image 300 displays a white line 310 indicating a parking section. The predicted travel line 320 drawn by the predicted travel line drawing unit 260 is displayed in the parking assistance image 300, and the predicted travel line 320 changes in accordance with the change in the steering angle of the steering wheel. The travel prediction line 320 does not necessarily have to include the line 322 that represents the rear end of the vehicle, and may be only two prediction lines that represent travel of the rear wheels. FIG. 8 shows an example of a travel prediction line in the forward parking mode. A parking support image including a bird's-eye view image 400 and an image 410 obtained by capturing the front of the vehicle is displayed on the display unit 160, and a white line 420 indicating an annotation position is displayed on the parking support image. The predicted travel line 430 drawn by the display 260 is displayed.

  Next, the travel prediction line information storage unit 250 stores or updates the relationship between the estimated turning radius and the steering angle of the steering wheel in a table (S108).

  On the other hand, when it is determined that the own vehicle is at a stop, the predicted travel line drawing unit 260 acquires the current steering angle through the steering angle acquisition unit 230 (S120), and acquires the acquired steering angle. The turning radius corresponding to the steering angle is read out from the table as a key (S122), and a predicted travel line is drawn on the parking assistance image based on the read turning radius (S124). If the turning radius corresponding to the steering angle is not stored, the travel prediction line drawing unit 260 does not draw the travel prediction line. However, if the difference between the acquired steering angle and the steering angle stored in the table is within the allowable range, in other words, the difference between the predicted travel lines drawn according to the difference in turning radius is within the allowable range In this case, a predicted travel line may be drawn according to the turning radius. Furthermore, even if there is no information in the table, if there is data before and after (not necessarily adjacent), it is possible to generate data by interpolation and draw a predicted travel line. For example, if there is no data of steering angle at 300 ° (turning radius) and there is data of steering angle at 290 ° and 310 ° (turning radius), the steering radius of 300 ° is complemented from both of them. It is possible to draw a travel prediction line of a turning radius of 300 °.

  As described above, according to this embodiment, when the vehicle is moving, a predicted travel line is drawn based on the turning radius estimated from the moving image, and at this time, the relationship between the detected steering angle and turning radius Is stored in the table, it is possible to draw a travel prediction line by reading from the table the turning radius corresponding to the detected steering angle while the vehicle is stopped.

  Although the preferred embodiments of the present invention have been described above in detail, the present invention is not limited to the specific embodiments, and various modifications may be made within the scope of the present invention as set forth in the claims. Modifications and changes are possible.

100: in-vehicle device 110: input unit 120: navigation unit 130: imaging unit 140: travel information acquisition unit 150: voice output unit 160: display unit 170: communication unit 180: storage unit 190: control unit 200: parking support program 210: Parking mode determination unit 220: parking support image generation unit 230: steering wheel steering angle acquisition unit 230
240: turning radius estimation unit 250: predicted travel line information storage unit 260: predicted travel line drawing unit

Claims (6)

An electronic device with a parking assistance function,
Imaging means for imaging the surroundings of the vehicle;
Estimating means for estimating a turning radius of a vehicle based on a moving image captured by the imaging means;
Acquisition means for acquiring steering wheel steering angle information related to steering wheel steering angle of the vehicle;
Storage means for storing the relationship between the estimated turning radius and the steering angle at the time of the estimation;
Display means for displaying an image captured by the imaging means;
And a drawing unit that draws a predicted travel line linked to the steering angle of the steering wheel on the image displayed on the display unit.
The drawing device is an electronic device that draws a traveling prediction line by reading a turning radius corresponding to a steering wheel steering angle from the storage device when the vehicle is stopped or when a moving image can not be obtained. The electronic device according to claim 1, wherein the pre-rendering means draws the predicted travel line according to the turning radius estimated from the moving image when the vehicle is moving or when a moving image can be obtained. The electronic device according to claim 1, wherein the display unit displays the predicted travel line on a rear image captured by the imaging unit when the vehicle reverses. The electronic device according to claim 1, wherein the display means displays the predicted travel line on a bird's-eye image of a vehicle when the vehicle moves forward. The electronic device according to claim 1, wherein the storage unit updates the relationship between the turning radius and the steering angle when the turning radius is estimated by the estimation unit. A parking assistance method for an electronic device having a parking assistance function, comprising:
Determine if your car is stopped,
If it is determined that the host vehicle is not at a stop, the turning radius of the host vehicle is estimated based on the moving image of the image data captured by the imaging camera, and a travel prediction line is drawn on the captured image based on the estimated radius. And storing the relationship between the estimated turning radius and the steering angle in the storage means,
When it is determined that the host vehicle is at a stop, the steering wheel steering angle is acquired, the turning radius corresponding to the acquired steering wheel steering angle is acquired from the storage unit, and the travel prediction line is captured based on the acquired turning radius Parking assistance method to draw on.

Images