JP2012004693A - Driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driving support device allowing easier recognition of a distance from a vehicle to obstacles.SOLUTION: The driving support device comprises: an extraction unit 5 for extracting pixel columns corresponding to positions, at which distance markers L1 and L2 are overwritten, from images taken by on-board cameras 1a and 1b; a creation unit 6 that determines whether or not any obstacle is imaged at the pixel columns or not, and creates a third image data indicating a portion X1' of an obstacle X1 when the obstacle X1 is imaged; a composition unit 7 for creating composite image data, indicating a composite image in which the portion X1' of the obstacle X1 is additionally written over the distance markers that are written over the images taken, when the creation unit 6 creates the third image data; and a display 3 for displaying the created composite image.

Description

  The present invention relates to a driving support device that displays a captured image of a periphery of a vehicle captured by an in-vehicle camera on a display screen provided in a vehicle interior.

  Conventionally, as an apparatus for assisting driving such as parallel parking and garage parking, a vehicle periphery including a road surface is imaged by an in-vehicle camera, and a plurality of distance lines (distance markers) indicating the distance from the vehicle are displayed on the captured image There has been known a driving support device that displays a superimposed display so as to extend in the vehicle width direction and displays it on a display screen in a vehicle interior (for example, Patent Document 1).

JP-A-11-334470

  However, when the vehicle-mounted camera images the vehicle periphery so as to include the road surface, the obstacle (three-dimensional object) appears to overlap the road surface in the depth direction of the captured image (the direction away from the vehicle). According to the driving assistance apparatus being used, for example, even when there is an obstacle at a distance of 0.4 m from the vehicle, the distance from the vehicle is 0.5 m in the captured image. A distance marker may be superimposed on the obstacle.

  And the distance marker which shows the distance longer than the distance from a vehicle to an obstacle in this way is superimposed on this obstacle, and a driver | operator is deeper than the distance (0.5m) which a distance marker shows. There is a risk of misperception (illusion) that an obstacle exists at a far position in the direction.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a driving support device capable of easily grasping a sense of distance from a vehicle to an obstacle.

  The driving support device according to the present invention determines whether or not an obstacle appears in the pixel row corresponding to the position where the distance marker is overwritten in the captured image. An image representing a part of an object is overwritten on the distance marker, and a composite image in which the distance marker is erased is displayed.

  That is, the driving support apparatus according to the present invention overwrites the captured image with the in-vehicle camera that captures the periphery of the vehicle, the first storage unit that stores the first image data representing the captured image captured by the in-vehicle camera. A second storage unit configured to store second image data representing a distance marker indicating a distance from the vehicle, and a pixel row corresponding to a position where the distance marker is overwritten in the captured image. An extraction unit for extracting and determining whether or not an obstacle is reflected in the pixel row, and when it is determined that the obstacle is reflected, third image data representing a portion of the obstacle reflected in the pixel row And a generation unit that overwrites the distance marker on the captured image based on the first image data and the second image data, and the generation unit generates the third image data when the generation unit generates the third image data. image And having a combining unit for generating a composite image data representing a composite image portion was further overwriting of the obstacle on the distance marker based on chromatography data, and a display unit for displaying the composite image.

  According to the driving support apparatus according to the present invention configured as described above, the extraction unit extracts a pixel column corresponding to a position where the distance marker is overwritten in the captured image, and the generation unit includes an obstacle in the pixel column. Is determined, and if it is determined that the obstacle is captured, third image data representing the portion of the obstacle captured in the pixel row is generated, and the combining unit detects the distance to the captured image. When the marker is overwritten and the generation unit generates the third image data, the composite image data representing the composite image in which the obstacle portion is further overwritten on the distance marker is generated, so that the distance marker is erased. Since the composite image is displayed and the obstacle positioned in front of the distance marker is not hidden by the distance marker, the sense of distance from the vehicle to the obstacle can be easily grasped.

  According to the driving support device of the present invention, it is possible to easily grasp the sense of distance from the vehicle to the obstacle.

It is a block diagram which shows the structure of the driving assistance device of this embodiment. It is an example which showed the imaging condition with the vehicle-mounted camera (rear camera) of the conventional driving assistance apparatus. (A) is a figure which shows the picked-up image by the vehicle-mounted camera of FIG. 2, (b) is a composite image after the image processing by the conventional driving assistance apparatus is performed with respect to the picked-up image of (a). FIG. It is an example which showed the imaging condition with the vehicle-mounted camera of the driving assistance device of FIG. It is a figure explaining the flow of the image processing by the driving assistance device of FIG. FIG. 6 is an enlarged view of waveforms B and C in FIG. 5. It is a figure which shows the synthetic | combination process by the synthetic | combination part of FIG.

  Hereinafter, an embodiment for realizing a driving support device of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of the driving support apparatus 100 of the present embodiment. As shown in FIG. 1, the driving support apparatus 100 according to the present embodiment includes in-vehicle cameras 1a and 1b, image processing means 2 that performs image processing on captured images captured by the in-vehicle cameras 1a and 1b, and image processing. And a monitor 3 (display unit) for displaying an image (composite image) after the image processing is performed by the means 2.

  The in-vehicle cameras 1a and 1b are stereo cameras composed of the in-vehicle camera 1a and the in-vehicle camera 1b. These vehicle-mounted cameras 1a and 1b are both provided with the optical axis directed in the same direction, and image the vehicle periphery including the road surface.

  As shown in FIG. 1, the image processing means 2 includes a decoder 4, an extraction unit 5, a generation unit 6, a synthesis unit 7, an encoder 8, a first memory 9 (first storage unit), and a second memory 10 ( A second storage unit) and a third memory 11 are provided.

  The decoder 4 decodes the image data of the captured image input by the in-vehicle cameras 1a and 1b. Moreover, the extraction part 5 extracts the image data of the pixel row | line corresponding to the position where a distance marker is overwritten about each picked-up image by vehicle-mounted camera 1a, 1b.

  Here, the distance marker means a plurality of distance lines that are virtually set to be overwritten on the captured image and that extend in the vehicle width direction indicating the distance D from the vehicle (FIG. 3B, FIG. 3). 7 distance markers L1 and L2).

  As illustrated in FIG. 1, the generation unit 6 includes a detection unit 12 that detects a boundary position (edge position) between a road surface and a surface other than the road surface that are reflected in the pixel row based on the image data of the pixel row extracted by the extraction unit 5. A distance measuring unit 13 is provided for measuring a distance Dx (see FIG. 5) from the vehicle to the subject shown in the pixel row extracted by the extracting unit 5 based on the stereo parallax by the in-vehicle cameras 1a and 1b.

  Further, the generation unit 6 determines whether an obstacle is reflected in the pixel row based on the edge position detected by the detection unit 12 and the distance Dx from the vehicle to the subject measured by the distance measurement unit 13. In the case where an obstacle is shown, third image data (see FIG. 7) representing the part of the obstacle shown in the extracted pixel row is generated.

  The first memory 9 outputs image data of a captured image from either the in-vehicle camera 1a or 1b from the decoder 4 (in this embodiment, output of image data of an image captured by the in-vehicle camera 1a). The output image data (first image data) is stored.

  Then, the second memory 10 includes a plurality of distance markers that are virtually set to be overwritten on the captured image (in this embodiment, a distance marker L1 indicating a position 0.5 m away from the vehicle and 1 from the vehicle). The second image data representing the distance marker L2) indicating a position separated by .0 m is stored, and the third memory 11 stores the third image data generated by the generating unit 6.

  The synthesizing unit 7 is connected to the first memory 9, the second memory 10, and the third memory 11. The synthesizing unit 7 overwrites the captured image with a distance marker based on the first image data and the second image data, and when the generating unit 6 generates the third image data, the synthesizing unit 7 is based on the third image data. The composite image data representing the composite image in which the obstacle marker is further overwritten on the distance marker is generated.

  Furthermore, an encoder 8 that encodes the composite image data generated by the combining unit 7 is provided. The encoder 8 is provided in the vehicle interior and transmits to the monitor 3, and the monitor 3 displays a composite image on the display screen.

  Next, regarding image processing by the driving support device 100, [image processing by a conventional driving support device], [extraction processing], [edge position detection processing], [distance measurement processing], [third image data generation processing] and This will be described separately in [Image composition processing].

[Image processing by conventional driving support devices]
FIG. 2 is an example showing a state of imaging by a vehicle-mounted camera of a conventional driving support device, and FIG. 3A is a diagram showing a captured image by the vehicle-mounted camera (rear camera) of FIG. ) Is a diagram showing a composite image after image processing by the conventional driving support device is performed on the captured image of FIG.

  For example, when the vehicle-mounted camera (rear camera) of the conventional driving support apparatus captures the vehicle periphery including the road surface in the situation as shown in FIG. 2, a captured image as shown in FIG. 3A is obtained.

  Here, the obstacles X1 and X2 shown in FIGS. 2 and 3 are both solid objects, and the obstacle X1 is present at a position 0.4 m away from the vehicle, and the obstacle X2 is 0.9 m away from the vehicle. Exists in position.

  In addition, a pair of white lines are drawn on the road surface with an interval slightly wider than the vehicle width. The broken lines shown in FIGS. 2 and 3 (a) are imaginary lines for representing positions 0.5 m and 1.0 m away from the vehicle.

  Then, when image processing by the conventional driving support device is performed on the captured image shown in FIG. 3A, a distance marker L1 (D = 0.5 m) is displayed on the captured image as shown in FIG. 3B. And a composite image on which the distance marker L2 (D = 1.0 m) is superimposed and displayed.

  However, in this composite image, the obstacle X1 is hidden by the distance marker L1, and the obstacle X2 is hidden by the distance marker L2.

  For this reason, the driver has an obstacle at a position farther in the depth direction than the distance D (= 0.5 m) indicated by the distance marker L1. In addition, the obstacle X2 present at a position 0.9 m away from the vehicle seems to be present at a position farther in the depth direction than the distance D (= 1.0 m) indicated by the distance marker L2. There is a risk of misunderstanding (illusion).

  On the other hand, in the driving assistance device 100 of the present embodiment, it is possible to easily grasp the sense of distance from the vehicle to the obstacle by performing the following image processing.

[Extraction process]
FIG. 4 is an example showing an imaging situation by the in-vehicle cameras 1a and 1b of the driving support apparatus 100 in FIG. 1, and FIG. 5 is a diagram for explaining the flow of image processing by the driving support apparatus 100 in FIG.

  As shown in FIG. 4, a case will be described in which the in-vehicle cameras 1a and 1b of the driving assistance apparatus 100 image the vehicle periphery including the road surface in a situation similar to the situation shown in FIG.

  First, the in-vehicle cameras 1 a and 1 b output image data of captured images to the decoder 4. Then, the decoder 4 decodes these image data and outputs them to the extraction unit 5.

  The extraction unit 5 stores in advance a distance D (= 0.5 m, 1.0 m) from the vehicle indicated by the distance markers L1 and L2, and the extraction unit 5 stores the distance D (= 0.5 m). , 1.0 m), pixel image data (in this embodiment, a luminance signal) corresponding to the position where the distance markers L1, L2 are overwritten for the respective captured images by the in-vehicle cameras 1a, 1b. Extract.

  For example, the luminance signal of the pixel column corresponding to the position where the distance marker L1 is overwritten is like the waveforms A1 and A2 in FIG.

  That is, when a position 0.5 m away from the vehicle is indicated by a broken line (virtual line) in FIG. 5, a road surface along the virtual line appears in the pixel row corresponding to the position where the distance marker L1 is overwritten. Become.

  However, since there is an obstacle X1 (distance Dx = 0.4 m from the vehicle) located in front of the imaginary line, this pixel is arranged such that a part of the obstacle X1 overlaps the road surface along the imaginary line. It appears in the column. Here, a part of the obstacle X1 reflected in the pixel row is referred to as “part X1 ′ of the obstacle X1”.

  And since the road surface is usually low in luminance and the portion X1 ′ of the obstacle X1 is higher in luminance than the road surface, some of the luminance is high as shown by the waveforms A1 and A2 in FIG. A luminance signal is extracted.

  The extraction process by the extraction unit 5 is intended to speed up the process, and only the distance marker L1 in the foreground is extracted, and the other distance markers (such as the distance marker L2) that are separated in the depth direction are extracted. It is also possible to configure so that there is no.

[Edge position detection processing]
FIG. 6 is an enlarged view of waveforms B and C in FIG. The extraction unit 5 outputs the extracted luminance signal (for example, the waveforms A1 and A2 in FIG. 5) of the pixel column to the detection unit 12 of the generation unit 6. And the detection part 12 detects the edge position of the road surface reflected in a pixel row | line | column, and other than a road surface.

  Specifically, first, the detection unit 12 shifts the luminance signal of the pixel column (waveform B1 in FIG. 6 = waveform A1 shown in FIG. 5) to the left (waveform B2 in FIG. 6), and obtains the difference (FIG. 6 waveform B3).

  Next, the detection unit 12 shifts the luminance signal of the pixel column (waveform B1 in FIG. 6) to the right (waveform B4 in FIG. 6), and obtains the difference (waveform B5 in FIG. 6).

  Then, the detection unit 12 combines these differences (waveform B3 and waveform B5) to emphasize the luminance difference (waveform B6 in FIG. 6), and detects the position where the luminance difference is emphasized as an edge position.

  The detection of the edge position by the detection unit 12 is performed for each captured image by the in-vehicle cameras 1a and 1b.

  Further, the waveform C shown in FIGS. 5 and 6 can be smoothed like the waveforms D1 and D2 shown in FIG. 5, and only a portion having a particularly large difference can be detected as an edge position. .

[Distance measurement processing]
After the edge position is detected by the detection unit 12, the distance measurement unit 13 specifies a subject (candidate for obstacle) other than the road surface based on the detected edge position, and the distance from the specified subject to the vehicle. Measure Dx.

  That is, for example, as shown in FIG. 5, the line-of-sight angles θ1 and θ2 (or the distances d1 and d2 from the center position of the image sensor to the specified pixel) with respect to the specified subject and the optical axes of the in-vehicle cameras 1a and 1b. The distance Dx from the subject to the vehicle is measured based on the focal lengths f and f of the in-vehicle cameras 1a and 1b and the distance L between the in-vehicle cameras 1a and 1b.

(Distinction between flat objects and solid objects)
Here, in the extracted pixel row, not only a three-dimensional object such as the obstacles X1 and X2, but also a road sign (planar object), such as a white line drawn on the road surface, may appear.

  Further, since the brightness of the image of such a planar object also increases in the extracted luminance signal of the pixel row, the edge position is detected by the detection unit 12 and the distance measurement process by the distance measurement unit 13 is performed.

  However, if this flat object image is generated as third image data in the same manner as the three-dimensional object image and the distance marker L1, L2 is overwritten with the flat object image so as not to be hidden by the distance markers L1, L2, There is a possibility that the driver may have the illusion that the planar object is a three-dimensional object positioned in front of the distance markers L1 and L2.

  Therefore, when generating the third image data, the generation unit 6 distinguishes whether the subject other than the road surface shown in the pixel row is a three-dimensional object or a planar object.

  Specifically, when the white line drawn on the road surface is reflected in the pixel row corresponding to the position where the distance marker L1 is overwritten, the distance Dx from the white line measured by the distance measuring unit 13 to the vehicle is the distance marker. It becomes equal to the distance D indicated by L1 (Dx = D = 0.5 m).

  Therefore, when the distance D measured by the distance measuring unit 13 is equal to the distance Dx indicated by the distance marker L1, the generating unit 6 is a road sign (planar object) such as a white line drawn on the road surface. A plane object and a three-dimensional object are distinguished as being.

  As described above, the generation unit 6 distinguishes between a planar object and a three-dimensional object, and determines whether a three-dimensional object (obstacle) is reflected in the extracted pixel row.

  In the present embodiment, the obstacles X1 and X2 have the distance Dx (= 0.4 m, 0.9 m) measured by the distance measuring unit 13 and the distance D (= 0.5 m, 1.. Since it is shorter than 0 m), it is distinguished as a three-dimensional object, and the generation unit 6 determines that these obstacles X1 and X2 are reflected in the extracted pixel row.

[Third image data generation processing]
FIG. 7 is a diagram showing a composition process by the composition unit 7 in FIG.

  As shown in FIG. 7, the image represented by the first image data (that is, the captured image) is the first layer, and the image represented by the second image data (that is, the images of the distance markers L1 and L2) is the second layer and the third layer. The image represented by the image data (that is, the image of the part X1 ′ of the obstacle X1) is set as the third layer.

  As described above, the generation unit 6 distinguishes between a planar object and a three-dimensional object, and when it is determined that the obstacle X1 that is a three-dimensional object is reflected in the pixel row, the portion X1 ′ of the obstacle X1 The third image data representing the image is generated.

  When generating the third image data, the generation unit 6 obtains a waveform E as shown in FIG. 5 based on the detected edge position. This waveform E represents the timing for drawing the image of the part X1 ′ of the obstacle X1 as the third image layer shown in FIG. 7 (the position in the captured image of the pixel in which the part X1 ′ of the obstacle X1 appears). ing.

  Then, the generation unit 6 draws the image of the part X1 ′ of the obstacle X1 as the third image layer based on the waveform E, and does not draw the captured image at other timings, so that the third image layer ( That is, the third image data) is generated.

  Further, the generation unit 6 outputs the generated third image data to the third memory 11, and the third memory 11 stores the third image data.

[Image composition processing]
As illustrated in FIG. 7, the combining unit 7 overwrites the captured image with a distance marker based on the first image layer represented by the first image data and the second image layer represented by the second image data, as illustrated in FIG. 7. The synthesized image data representing the synthesized image is generated.

  In addition, when the generation unit 6 generates the third image data, the combining unit 7 displays the obstacle marker on the distance marker based on the third image layer represented by the third image data, as shown in FIG. Composite image data representing a composite image in which the portion is further overwritten is generated.

  As a result, a composite image in which the obstacle X1 positioned in front of the distance marker L1 and the obstacle X2 positioned in front of the distance marker L2 are not hidden by the distance markers L1 and L2 via the encoder 8 is obtained. Displayed by the monitor 3.

  According to the driving support apparatus 100 according to the present embodiment configured as described above, the extraction unit 5 extracts a pixel row corresponding to a position where the distance markers L1 and L2 are overwritten in the captured image, and the generation unit 6 Determines whether or not an obstacle is reflected in the pixel row, and if it is determined that the obstacle is reflected, generates third image data representing the portion X1 ′ of the obstacle X1 reflected in the pixel row. Then, when the synthesizing unit 7 overwrites the distance marker on the captured image and the generating unit 6 generates the third image data, the synthesizing image data representing the synthesized image in which the obstacle marker is further overwritten on the distance marker is generated. Therefore, for the pixel in which the obstacle X1 is shown, a composite image as if the distance marker L1 is erased is displayed on the monitor 3, and the obstacles X1 and X2 positioned in front of the distance markers L1 and L2 are distanced. Markers L1, L In concealed it eliminates, it is possible to easily grasp the sense of distance from the vehicle to the obstacle X1, X2.

  Furthermore, the generation unit 6 includes a distance measurement unit 13 that measures a distance Dx from the subject to the vehicle shown in the extracted pixel row, and the distance Dx measured by the distance measurement unit 13 is the distance indicated by the distance marker L1. If it is shorter than D, it is determined that the subject is an obstacle. For example, a road sign (planar object) such as a white line drawn on the road surface is not generated as the third image data, and the road sign is solid. It can be prevented from being recognized as an object.

  Then, the generation unit 6 includes a detection unit 12 that detects a boundary position (edge position) between the road surface and other than the road surface among the subjects captured in the pixel row based on the extracted image data of the pixel row, and measures the distance. Since the unit 13 identifies the subject whose distance is to be measured based on the edge position detected by the detection unit 12, it is possible to reduce the number of subjects whose distance is to be measured, and to perform the calculation process quickly.

  Further, since the detection unit 12 detects the edge position based on the luminance signal of the pixel column, the timing (waveform E) for rendering the image of the part X1 ′ of the obstacle X1 as the third image layer with a small amount of calculation processing. ).

  Moreover, since the synthesis unit 7 generates a synthesized image (synthesized image data) using the first image layer, the second image layer, and the third image layer, a direct signal is output with respect to the first image data representing the captured image. The amount of processing can be reduced compared with the case of processing.

(Modification 1)
In the above-described embodiment, the case where the detection unit 12 detects the edge position based on the luminance signal of the pixel row has been described. However, the driving support device according to the present invention is not limited to such a form, for example, The detection unit 12 may be configured to detect the edge position based on the color signal of the pixel column.

  According to the driving support apparatus configured as described above, the edge position can be detected even for an obstacle with a small difference in luminance from the road surface.

(Modification 2)
In the above-described embodiment, the case where the synthesis unit 7 generates a synthesized image (synthesized image data) using the first image layer, the second image layer, and the third image layer has been described. The support device only needs to display a composite image on the monitor so that an obstacle positioned in front of the distance marker is not hidden by the distance marker. For example, the support device has a configuration in which signal processing is performed directly on the captured image. It is also possible to adopt.

1a, 1b Car-mounted camera 2 Image processing means 3 Monitor (display unit)
5 Extracting Unit 6 Generating Unit 7 Combining Unit 9 First Memory (First Storage Unit)
10 Second memory (second storage unit)
11 Third memory 100 Driving support device

Claims (5)

  An in-vehicle camera that images the periphery of the vehicle, a first storage unit that stores first image data representing a captured image captured by the in-vehicle camera, and the vehicle that is set to be overwritten on the captured image. A second storage unit that stores second image data representing a distance marker indicating a distance; an extraction unit that extracts a pixel column corresponding to a position where the distance marker is overwritten in the captured image; and a failure in the pixel column A generation unit that determines whether or not an object is captured, and that determines that the obstacle is captured; generates a third image data representing a portion of the obstacle captured in the pixel row; and the first image The distance marker is overwritten on the captured image based on the data and the second image data, and when the generation unit generates the third image data, the obstacle marker is added to the distance marker based on the third image data. Driving support apparatus characterized by comprising: a synthesizing unit for generating synthetic image data representing a composite image is overwritten with the portion of the object Additionally, a display unit displaying the composite image.   The generation unit includes a distance measurement unit that measures a distance from a subject captured in the pixel row to a vehicle, and when the distance measured by the distance measurement unit is shorter than the distance indicated by the distance marker, The driving support device according to claim 1, wherein the subject is determined to be the obstacle. The generation unit includes a detection unit that detects a boundary position between a road surface and a non-road surface among subjects captured in the pixel column based on image data of the pixel column,
The driving assistance device according to claim 2, wherein the distance measurement unit specifies a subject whose distance is to be measured based on the boundary position detected by the detection unit.   The driving support device according to claim 3, wherein the detection unit detects the boundary position based on a luminance signal of the pixel row.   The driving support device according to claim 3, wherein the detection unit detects the boundary position based on a color signal of the pixel column.

Images