JP2010286995A - Image processing system for vehicle - Google Patents

Abstract

A vehicle image processing system capable of improving the recognition accuracy of a lane mark in front of a vehicle even in a backlit state.
[Solution]
A vehicle image processing system 100 that executes image processing based on a front image and a rear image that are captured images of a front camera 12 that images a road surface in front of the vehicle 10 and a rear camera 13 that images a road surface in the back is determined by backlight determination. When it is determined by the means 110 and the backlight determination means 100 that the backlight is in the backlight state, a lane mark candidate is detected from either the front image or the rear image, and the detected lane mark candidate is extracted from the remaining image. And a lane mark detection unit 120 that detects a true lane mark present on the road surface ahead of the vehicle 10 by matching with the image feature amount.
[Selection] Figure 1

Description

  The present invention relates to an image processing system for a vehicle that detects a lane mark of a road from a road surface image acquired via an imaging unit such as a camera.

  Conventionally, in this type of vehicle image processing system, as shown in Patent Document 1 below, a front camera and a rear camera are mounted on the front and rear of the vehicle, respectively, and an image with high illuminance is selected from these images. A technique for detecting a road lane mark based on the image is known.

  In the vehicle image processing system shown in Patent Document 2 below, a front camera and a rear camera are mounted in front and rear of the vehicle, respectively, and images whose maximum and minimum luminance values are within a predetermined range are provided. A technique for determining a good image and detecting a road lane mark based on the image is known.

JP 2008-225822 A Japanese Patent Laid-Open No. 2002-99999

  However, in a backlit state where light such as sunlight is irradiated to the vehicle from a direction opposite to the traveling direction of the vehicle, the road lane is based only on the selected image of either the front camera or the rear camera. When the mark is detected, the recognition accuracy of the lane mark may be lowered.

  That is, when an image of the front camera is selected in the backlight state, there is a high possibility that a road surface portion having no lane mark such as an eaves or an artificial groove provided on the road is erroneously recognized as a lane mark. Further, it is difficult to extract the luminance contrast for the road surface portion where the color information of the yellow lane mark or the lane mark of the white lane mark is not provided.

  On the other hand, when an image of the rear camera is selected in the backlight state, if the arrangement pattern of the lane mark extracted from the image is different between the rear and front of the vehicle, the actual lane mark present in front of the vehicle and the rear lane The lane mark inferred from the mark is different.

  Therefore, an object of the present invention is to provide a vehicle image processing system that can improve the recognition accuracy of a lane mark in front of the vehicle even in a backlight state.

  An image processing system for a vehicle according to a first aspect of the present invention is based on image processing performed by a front imaging unit that is mounted on a vehicle and images a road surface in front of the vehicle, and a rear imaging unit that images a road surface behind the vehicle. The image processing system for vehicles that performs the above-mentioned, wherein light such as sunlight is generated based on one or both of the front image captured by the front imaging unit and the rear image captured by the rear imaging unit. When it is determined that the backlight is illuminated by the backlight from the direction opposite to the traveling direction of the vehicle, and the backlight determining means determines that the vehicle is in the backlight state. A lane mark candidate is detected from either the front image or the rear image, and the detected lane mark candidate is matched with an image feature amount extracted from the remaining image. And by, characterized in that it comprises a lane mark detection means for detecting a true lane marks present in the vehicle front road.

  According to the vehicle image processing system of the first invention, when it is determined that the backlight is in the backlight state, the lane mark candidate is detected from one of the front image and the rear image, and the lane mark candidate is By matching with the image feature amount extracted from the other image, the lane mark present on the road surface ahead of the vehicle is detected.

  That is, in backlit conditions, useful information for recognizing not only lane mark candidates or image features that are behind the vehicle, but also lane marks that are located in front of the vehicle, although the reliability is low. The front image containing is used. As a result, the recognition accuracy of the lane mark existing in the front image is improved as compared with the case where either one of the front image and the rear image is selectively used.

  The vehicular image processing system according to a second aspect is the vehicular image processing system according to the first aspect, wherein the lane mark detection means detects a lane mark candidate based on the front image, and the detected lane mark candidate vehicle. When calculating the position in the width direction and determining that the backlight is in the backlight state by the backlight determination unit, either the luminance or the color component of the position corresponding to the position in the rear image as the image feature amount Based on either or both, a true lane mark is detected from the lane mark candidate.

  According to the vehicle image processing system of the second aspect of the invention, in the backlit state, not only the rear image that is highly reliable from the viewpoint of extracting one or both of the luminance and color components as the image feature amount behind the vehicle, but also the reliability Although useful, information useful for recognizing a lane mark present in front of the vehicle, that is, a front image including a lane mark candidate for specifying a position from which the image feature amount is to be extracted in the rear image is used. As a result, the recognition accuracy of the lane mark existing in the front image is improved as compared with the case where either one of the front image and the rear image is selectively used.

  A vehicle image processing system according to a third aspect of the present invention is the vehicle image processing system according to the first aspect, wherein the lane mark detection means is based on the rear image when the backlight determination means determines that the backlight is in a backlight state. Detecting a lane mark candidate, calculating a position in the vehicle width direction of the detected lane mark candidate, and detecting a true lane mark based on a luminance of a position corresponding to the calculated position in the front image. Features.

  According to the vehicle image processing system of the third aspect of the invention, in the backlit state, not only the rear image having high reliability from the viewpoint of extracting the lane mark candidate in the rear of the vehicle, but also the lane existing in front of the vehicle although the reliability is low. Information useful for the recognition of the mark, that is, a front image including luminance is used. As a result, the recognition accuracy of the lane mark existing in the front image is improved as compared with the case where either one of the front image and the rear image is selectively used.

  The vehicle image processing system according to a fourth aspect of the present invention is the vehicle image processing system according to any one of the first to third aspects, wherein the backlight determining means has a low luminance region surrounding the lower end center of the rear image. In addition, it is characterized in that it is determined to be in a backlight state.

  According to the vehicle image processing system of the fourth aspect of the invention, in view of the high probability that the shadow of the vehicle appears on the rear road surface of the vehicle in the backlight state, it is determined simply and accurately whether or not it is in the backlight state. Can be done.

1 is a configuration diagram of a vehicle image processing system according to an embodiment of the present invention. 5 is a flowchart showing lane mark detection processing according to the first embodiment. The flowchart which shows a backlight determination process. Explanatory drawing which shows the example of a front image, and its image processing. Explanatory drawing which shows the example of a back image, and its image processing. Explanatory drawing which shows the processing content by a backlight determination means. Explanatory drawing which shows the example of a process of the back image by a backlight determination means. The flowchart which shows the detection process of the lane mark in 2nd Embodiment. Explanatory drawing which shows the content of the image process in 2nd Embodiment.

  As shown in FIG. 1, the vehicle image processing system 100 includes an electronic control unit (computer) 11 as hardware mounted on the vehicle 10. The vehicle image processing system 100 includes a front camera (front imaging unit) 12 that captures a color road surface image (front image) in front of the vehicle 10 and a rear camera that captures a color road surface image (rear image) behind the vehicle 10. (Backward imaging means) 13 is connected to perform image processing based on the road surface images that are these captured images.

  The vehicle image processing system 100 includes a storage unit 101, a backlight determination unit 110, and a lane mark detection unit 120.

  The storage unit 101 stores a plurality of lane mark standards (lane mark color, standard width, etc.).

  Based on the rear image acquired from the rear camera 13, the backlight determination unit 110 is in a backlight state in which light such as sunlight is applied to the vehicle 10 from a direction opposite to the traveling direction of the vehicle 10. Determine whether.

  The lane mark detection unit 120 detects a lane mark candidate from either the front image or the rear image, and detects a true lane mark present on the road surface in front of the vehicle 10 based on the detected lane mark candidate.

[First Embodiment]
Next, functions (vehicle image processing method) of the vehicle image processing system configured as described above will be described with reference to FIGS.

  First, the lane mark detection unit 120 acquires a front image from the front camera 12 (FIG. 2 / STEP 1).

  Next, the lane mark detection means 120 executes a lane mark candidate detection process for detecting a lane mark candidate from the acquired front image (FIG. 2 / STEP 2).

  Regarding the details of the lane mark candidate detection process, a technique according to Japanese Patent Application Laid-Open No. 11-85999 by the applicant of the present application may be adopted. The mark detection means 120 recognizes an edge point that changes from high luminance (bright) to low luminance (dark) with the screen scanning direction to the right as a “negative edge point”, and changes from low luminance (dark) to high luminance (bright). The turning edge point is recognized as a “normal edge point”. Note that the luminance value of each pixel of the image is calculated from, for example, the R value, G value, and B value of each pixel of the acquired color image I0.

  For example, as shown in FIG. 4A, the front image has white lines (lane marks) M on the left and right sides of the lane, and the repair trace R extends from the white line M toward the center of the lane along the road extending direction. It shall be. At this time, it is assumed that the repaired trace R becomes a high-luminance portion in the front image due to the irradiation of sunlight that is the cause of the backlight state, and the white line on the left side of the lane has a reduced luminance contrast with respect to the road surface portion other than the white line.

  In this case, as shown in FIG. 4B, a point on the left edge of the white line M or repair mark R is recognized as a positive edge point (represented by “+”), and a point on the right edge. Are recognized as negative edge points (represented by “−”).

  Thereby, as shown in FIG. 4B, the left white line M is a positive edge point group having a linear component, as shown in FIG. 4B, between the positive edge point group and the negative edge point group having a linear component. L1a and the negative edge point group L1b are recognized as lane mark candidates M1, and the repair trace R is recognized as a lane mark candidate M2 having a straight edge component L2a and a negative edge point group L2b, and the right white line M is Are recognized as lane mark candidates M3 by the positive edge point group L3a and the negative edge point group L3b having a straight line component.

  Next, the lane mark detection means 120 calculates the position in the vehicle width direction from the center of the detected vehicle 10 of the lane mark candidate (FIG. 2 / STEP 3).

  Specifically, the lane mark detection unit 120 converts the linear component data (L1a, L1b,..., L3a, L3b) of each lane mark candidate M1, M2, M3 from the image space to the real space (fixed to the vehicle 10). Projective transformation to the left and right X coordinates in the real space of each lane mark candidate M1, M2, M3 is calculated as the position in the vehicle width direction.

  Next, the backlight determination unit 110 determines whether or not the vehicle 10 is in the backlight state based on the rear image acquired from the rear camera 13 (FIG. 2 / STEP 4). Details of the backlight determination process by the backlight determination unit 110 will be described later with reference to FIG.

  When the backlight determination unit 110 determines that the backlight is not in the backlight state (FIG. 2 / STEP4... NO), the lane mark detection unit 120 determines whether each lane mark candidate is a true lane mark. The lane mark determination process is executed (FIG. 2 / STEP 10).

  Specifically, in the lane mark determination process, it is determined whether or not the lane mark candidate is a true lane mark based on some or all of the plurality of determination elements. As the determination element, for example, whether or not the width of the lane mark candidate calculated from the position in the vehicle width direction matches the standard width of the lane mark stored in the storage unit 101, and whether the edge of the lane mark candidate is continuous The linearity component of the lane mark candidate is the image component of the image, and whether or not the total error with the linear approximation equation or the curve approximation equation representing the arrangement of the edge row is a certain value or less. Examples of the determination element include whether or not the left region or the right region is within a reasonable range as a lane mark.

  When it is determined that the lane mark candidate is a true lane mark (FIG. 2 / STEP 10... YES), the lane mark candidate is detected as a true lane mark (FIG. 2 / STEP 11).

  On the other hand, when it is determined that the lane mark is not a true lane mark (FIG. 2 / STEP 10... NO), the lane mark candidate is excluded as a false lane mark (FIG. 2 / STEP 12).

  Next, a case where it is determined in STEP 4 that the backlight is in the backlight state (FIG. 2 / STEP 4... YES) will be described.

  In this case, the lane mark detection unit 120 acquires a rear image from the rear camera 13 (FIG. 2 / STEP 5).

  Next, the lane mark detection unit 120 acquires an image feature amount at a position corresponding to the position in the vehicle width direction calculated at STEP 3 in the acquired rear image (FIG. 2 / STEP 6).

  The image feature amount acquired here is obtained by projective transformation of the X coordinates of the left end and right end in the real space of each lane mark candidate M1, M2, M3 calculated in STEP 3 to the image space in the rear image. Get the color components and brightness of the image area.

  For example, as shown in FIG. 5A, the rear image has white lines (lane marks) M on the left and right sides of the lane, and the shadow area S of the host vehicle 10 on the lower side of the image. In this case, when the X coordinate of the left end and the right end in the real space of each lane mark candidate M1, M2, M3 calculated in STEP 3 is projective transformed to the image space in the rear image, each lane mark as shown in FIG. Regions M1 ′, M2 ′, and M3 ′ specified by the linear components (L1a ′, L1b ′,..., L3a ′, L3b ′) corresponding to the candidates M1, M2, and M3 are obtained. Then, the lane mark detection means 120 is a color component that is characteristic in the regions M1 ′, M2 ′, and M3 ′ (high R value position color components (R value, G value, B value)) and luminance value (maximum luminance value or average luminance value).

  Instead of acquiring the color component and the luminance value as an average value of an arbitrary point or a plurality of points in the regions M1 ′, M2 ′, and M3 ′, the lane mark detection unit 120 uses the lane mark candidate (first number) from the region. 2 lane mark candidates) may be detected, and the color components and luminance values of the detected second lane mark candidates may be acquired.

  Next, the lane mark detection unit 120 determines whether or not the color components acquired from the respective regions M1 ′, M2 ′, and M3 ′ match the color components of the lane marks stored in the storage unit 101 (FIG. 2 / STEP7).

  If it is determined that there is a color that matches the color component of each of the areas M1 ′, M2 ′, and M3 ′ (FIG. 2 / STEP7... YES), the process proceeds to STEP10 to execute the lane mark determination process. .

  On the other hand, when it is determined that there is no color that matches the color component of each of the areas M1 ′, M2 ′, and M3 ′ (FIG. 2 / STEP7... NO), the lane mark detection unit 120 determines that each of the areas M1 ′, M2 It is determined whether or not the luminance value acquired from ', M3' is higher than a preset luminance threshold (FIG. 2 / STEP8). Here, the luminance threshold value is set to a value that can be determined when the luminance value as the image feature amount is a white line in consideration of various conditions.

  When it is determined that the luminance value acquired from each of the areas M1 ′, M2 ′, and M3 ′ is high luminance that is equal to or higher than the luminance threshold value (FIG. 2 / STEP8... YES), the process proceeds to STEP10, and the lane mark Execute the judgment process.

  On the other hand, when it is determined that the luminance value acquired from each of the areas M1 ′, M2 ′, and M3 ′ is not high luminance equal to or higher than the luminance threshold (FIG. 2 / STEP8... NO), the lane mark candidate is a false lane mark. (FIG. 2 / STEP 12).

  In the present embodiment, when either one of STEP 7 and STEP 8 is satisfied, the lane mark candidate detected from the front image can be a true lane mark. Instead, STEP 7 and STEP 8 are used. If both of the above are satisfied, the lane mark candidate detected from the front image may be a true lane mark.

  The above is the description of the overall function (vehicle image processing method) of the vehicle image processing system in the first embodiment.

  According to the vehicle image processing system of the first embodiment, the repair trace R detected as a high-intensity portion in the front image due to the reflection of sunlight in the backlight state is in the region M2 ′ of the corresponding rear image. Since a color component or luminance value that is characteristic in the case of a lane mark is not extracted, it can be reliably excluded. In the front image, the white line on the left side of the lane in which the lane mark is difficult to detect due to a decrease in luminance contrast due to the backlight condition is also a lane mark based on the corresponding region M1 ′ of the rear image. In this case, since characteristic color components and luminance values are extracted, this can be reliably detected as a true lane mark.

  Next, the details of the backlight determination process (FIG. 2 / STEP 4) will be described with reference to FIG.

  First, the backlight determination unit 110 acquires a rear image from the rear camera 13 (FIG. 3 / STEP 40).

  Next, the backlight determination unit 110 performs a horizontal edge extraction process on the acquired rear image (FIG. 3 / STEP 41). Specifically, the filter value obtained sequentially by sequentially scanning the horizontal edge square filter shown in FIG. 6A in the horizontal direction is stored and held as the value of the center coordinate of the horizontal edge square filter in the rear image coordinate system. . The filter value of the horizontal edge square filter is calculated as the sum of products of the values of the elements of the filter and the luminance values of the pixels in the image.

  For example, as shown in FIG. 6B, a state where the horizontal edge square filter is located at a location where a low luminance pixel (shaded portion) and a high luminance pixel (non-hatched portion) coexist. Think. For simplicity, it is assumed that the luminance values of all the low luminance pixels are “−1”, while the luminance values of the high luminance pixels are all “+1”. Then, the filter value in this state is calculated as “+1” × (1 + 1 + 1 + 0) + “− 1” × (0 + 0-1-1-1) = “+ 6”.

  Next, the backlight determination unit 110 performs vertical edge extraction processing on the acquired rear image (FIG. 3 / STEP 42). Specifically, the filter value sequentially obtained by sequentially scanning the vertical edge square filter shown in FIG. 6C in the vertical direction is used as the value of the center coordinate of the vertical edge square filter in the rear image coordinate system. Keep in memory. The filter value of the vertical edge square filter is calculated as the sum of products of the values of the elements of the filter and the luminance values of the pixels in the image.

  For example, as shown in FIG. 6C, a state where the vertical edge square filter is located at a location where a low luminance pixel (hatched portion) and a high luminance pixel (portion not shaded) coexist. Think. For simplicity, it is assumed that the brightness values of the low brightness pixels are all “−1”, while the brightness values of the high brightness pixels are all “+1”. Then, the filter value in this state is calculated as “+1” × (1 + 1 + 0−1) + “− 1” × (1 + 0−1−1) = “+ 2”.

  Next, the backlight determination unit 110 calculates an edge vector having the horizontal edge filter value and the vertical edge filter value stored as the central coordinate value of the square filter as components (FIG. 3 / STEP 43). Specifically, in the edge vector at the center coordinate position shown in FIGS. 6B and 6D, the vertical edge filter value is the X component and the horizontal edge filter value is the Y component, as shown in FIG. 6E. (2, 6).

Next, the backlight determination unit 110 determines whether there is a shadow region that satisfies the following condition (FIG. 3 / STEP 45).
(Condition 1)
A lower end of the rear image is defined by a center position of the lower end part or a region including the center position. For example, the extracted region is defined by the one or more line segments, and the Y component of the left edge vector is positive, while the Y component of the right edge vector is negative.

(Condition 2)
The position in the image coordinate system is determined by the coordinate value of the pixel, and a predetermined number or more of minute line segments in which the posture (tilt) in the image coordinate system is determined by the posture of the edge vector in the pixel are continuously in the direction of the edge vector. It is defined by one or more line segments obtained by being connected.

(Condition 3)
The sum of Y components of edge vectors constituting the line segment defining the left side is not less than a positive threshold, and the sum of Y components of edge vectors constituting the line segment defining the right side is not more than a negative threshold (condition) 3 indicates that the area defined by the line segment is composed of low-luminance pixels).

(Condition 4)
A predetermined area in which the area of the extraction area is equal to or larger than a predetermined area corresponding to the shadow of the host vehicle 10 and the width of the extraction area (the length in the Y direction in the screen coordinate system) is set based on the vehicle width. It is equivalent to the width.

  Then, when there is a shadow area (FIG. 3 / STEP 45... YES), the backlight determination unit 110 determines that the backlight is in the backlight state (FIG. 3 / STEP 46), and ends this process.

  On the other hand, if there is no shadow area (FIG. 3 / STEP 45... NO), the backlight determining unit 110 determines that the backlight is not in the backlight state (FIG. 3 / STEP 47), and ends this process.

  Specifically, three regions T1, T2, and T3 shown in FIG. 7 are extracted from the edge vector calculated in STEP 42 for the rear image shown in FIG. 5A. Does not satisfy the conditions 1, 3 and 4, but the area T1 satisfies all of the conditions 1 to 4 above. Therefore, for the rear image shown in FIG. Is present (FIG. 3 / STEP 45... YES) and is determined to be in the backlight state (FIG. 3 / STEP 46).

  The above is the details of the backlight determination process (FIG. 2 / STEP 4).

[Second Embodiment]
Next, a modification of the first embodiment will be described with reference to FIG.

  First, the backlight determination unit 110 determines whether or not it is in the backlight state based on the rear image acquired from the rear camera 13 (FIG. 8 / STEP 4). Note that this process is the same as the backlight determination process of FIG.

  Next, when the backlight determination unit 110 determines that the backlight is not in the backlight state (FIG. 8 / STEP 4... NO), the lane mark detection unit 120 acquires a front image from the front camera 12 (FIG. 8 / STEP 51). A lane mark candidate detection process for detecting a lane mark candidate from the acquired forward image is executed (FIG. 8 / STEP 52). These processes are the same as those in FIG. 2 / STEP1 and STEP2.

  Next, the lane mark detection unit 120 executes a lane mark determination process for determining whether or not each detected lane mark candidate is a true lane mark (FIG. 8 / STEP 70). If it is determined (FIG. 8 / STEP 70... YES), the lane mark candidate is detected as a true lane mark (FIG. 8 / STEP 71), and if it is determined that it is not a true lane mark (FIG. 8). 8 / STEP 70 ... NO), the lane mark candidate is excluded as a false lane mark (FIG. 8 / STEP 72). These processes are the same as those in FIG. 2 / STEP10 to STEP12.

  Next, a case where it is determined in STEP 4 that the backlight is in the backlight state (FIG. 8 / STEP 4... YES) will be described.

  In this case, the lane mark detection means 120 acquires a rear image from the rear camera 13 (FIG. 2 / STEP 61).

  Next, the lane mark detection unit 120 executes a lane mark candidate detection process for detecting a lane mark candidate from the acquired rear image (FIG. 8 / STEP 62). Note that this processing method is the same as FIG. 2 / STEP 2 except that the image processing target is different between the front image and the rear image.

  For example, when the rear image is as shown in FIG. 5A, as shown in FIG. 9A, as a region sandwiched between a positive edge point group and a negative edge point group having a linear component, The left white line M is recognized as a lane mark candidate M6 by a positive edge point group L6a and a negative edge point group L6b having a straight line component, and the right white line M is a positive edge point group L7a and a negative edge point group L7b having a straight line component. Is recognized as a lane mark candidate M7.

  Next, the lane mark detection means 120 calculates the position in the vehicle width direction from the center of the detected vehicle 10 of the lane mark candidate (FIG. 8 / STEP 63). This processing method is the same as that in FIG. 2 / STEP 3 except that the image processing target is different between the front image and the rear image. Accordingly, the linear components (L6a, L6b, L7a, L7b) is subjected to projective transformation from image space to real space (coordinate space fixed to the vehicle 10), so that the X coordinates of the left end and right end of each lane mark candidate M6, M7 in the real space can be converted into the vehicle width. Calculate as the position in the direction.

  Next, the lane mark detection unit 120 acquires a front image from the front camera 12 (FIG. 8 / STEP 64), and detects a lane mark candidate from a position corresponding to the position in the vehicle width direction calculated in STEP 63 in the acquired front image. (FIG. 8 / STEP 65).

  For example, when the left and right X coordinates of the lane mark candidates M6 and M7 shown in FIG. 9A in the real space are projectively transformed to the image space in the forward image, the forward image in FIG. Regions M6 ′ and M7 ′ specified by the linear components (L6a ′, L6b ′, L7a ′, and L7b ′) corresponding to the lane mark candidates M6 and M7 as shown in FIG. Then, the lane mark detection unit 120 detects lane mark candidates from the areas M6 ′ and M7 ′. Here, the detection method of the lane mark may be the same as in FIG. 2 / STEP 2, but the luminance value as the image feature amount is the lane mark (white line) in consideration that the front image is in the backlight state. It is preferable to detect this.

  Thereby, as shown in FIG. 9B, the lane mark candidate M2 is detected from the region M6 ′, and the lane mark candidate M1 is detected from the region M7 ′.

  Next, the lane mark detection unit 120 determines whether or not a lane mark candidate is detected from each of the regions M6 ′ and M7 ′ (FIG. 8 / STEP 66).

  If a lane mark candidate is detected from each of the areas M6 ′ and M7 ′ (FIG. 8 / STEP 66... YES), the process proceeds to STEP 70 and the lane mark determination process is executed. In this case, the content of the lane mark determination process in STEP 70 is executed in advance for the lane mark candidate detected from the rear image in STEP 62, so that the process in STEP 70 is omitted and the lane mark determination process is detected from the corresponding position. The lane mark candidate may be detected as a new lane mark as it is.

  On the other hand, if no lane mark candidate is detected from each of the regions M6 ′ and M7 ′ (FIG. 8 / STEP 66... NO), the lane mark candidate is excluded as a false lane mark (STEP 72).

  The above is the description of the overall function (vehicle image processing method) of the vehicle image processing system of the second embodiment.

  According to the vehicular image processing system of the second embodiment, the area M6 ′ of the front image in which a true lane mark can exist based on the rear image that is an image reflecting the original road surface state even under backlight conditions. , M7 ′ can be limited. Therefore, a true lane mark that can exist in front of the vehicle can be accurately detected from a front image in which the detection accuracy of the lane mark candidate can be reduced due to reflection on the road surface or the like.

  Thus, according to the vehicle image processing system of the first embodiment and the second embodiment, even under a backlight condition, the corresponding lane mark candidate corresponds to a true lane mark existing on the road surface ahead of the vehicle 10. Therefore, the lateral position control of the vehicle 10 based on the recognition result can be made appropriate.

  In the first and second embodiments, as shown in FIG. 3, it is determined whether or not the backlight is in the backlight state based on the rear image acquired from the rear camera 13. However, the present invention is not limited to this. Instead, various existing methods can be used. For example, the backlight determination may be performed based on whether or not the illuminance of the front image acquired from the front camera 12 exceeds a threshold when the backlight is backlit, and this may be combined with the backlight determination in the above embodiment.

DESCRIPTION OF SYMBOLS 10 ... Vehicle, 11 ... Electronic control unit, 12 ... Front camera (front imaging means), 13 ... Rear camera (rear imaging means), 100 ... Image processing system, 101 ... Memory means, 110 ... Backlight judging means, 120 Lane mark detection means.

Claims (4)

An image processing system for a vehicle that executes image processing based on captured images of a front imaging unit that is mounted on a vehicle and images a road surface in front of the vehicle and a rear imaging unit that images a road surface behind the vehicle. ,
Based on one or both of the front image captured by the front imaging unit and the rear image captured by the rear imaging unit, light such as sunlight from the direction opposite to the traveling direction of the vehicle A backlight determining means for determining whether or not the backlight is irradiated with respect to,
When it is determined that the backlight is in the backlight state by the backlight determination unit, lane mark candidates are detected from either the front image or the rear image, and the detected lane mark candidates are extracted from the remaining image. A vehicle image processing system comprising: lane mark detection means for detecting a true lane mark present on the road surface ahead of the vehicle by matching with an image feature amount. The vehicle image processing system according to claim 1,
The lane mark detection unit detects a lane mark candidate based on the front image, calculates a position of the detected lane mark candidate in the vehicle width direction, and is determined to be in a backlight state by the backlight determination unit. A true lane mark is detected from the lane mark candidate based on one or both of luminance and color components at a position corresponding to the position in the rear image as the image feature amount. A vehicle image processing system. The vehicle image processing system according to claim 1,
The lane mark detection unit detects a lane mark candidate based on the rear image when the backlight determination unit determines that the backlight is in the backlight state, and calculates the position of the detected lane mark candidate in the vehicle width direction. In addition, a true lane mark is detected based on the luminance at a position corresponding to the calculated position in the front image. The vehicle image processing system according to any one of claims 1 to 3,
The vehicle image processing system according to claim 1, wherein the backlight determination unit determines that the backlight is in a backlight state when there is a low-luminance region surrounding the lower center of the rear image.