JP2014191684A - Image processing apparatus and image processing method - Google Patents

Abstract

An image processing apparatus capable of appropriately detecting a detection target region is provided.
A plurality of reference images in which a detection target region is identified are divided into a plurality of reference regions, color information is extracted from a reference region with coordinate positions in a two-dimensional image coordinate system, and a reference having a common coordinate position By defining a criterion for determining whether an image corresponding to the coordinate position is a detection target region from the characteristics of the color information of the region, the criterion is acquired for each coordinate position of the two-dimensional coordinate system. Determination reference acquisition unit 20, captured image acquisition unit 20 that acquires a captured image, and extraction that divides the captured image into a plurality of determination regions and extracts color information from a reference region with a coordinate position in a two-dimensional image coordinate system. A reference region having a coordinate position that is the same as that of the determination region and the determination region is specified, and using the determination criterion of the reference region, it is determined whether the determination region is the detection target region from the color information of the determination region. Judgment means The image processing apparatus characterized in that it comprises 0, the.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus and an image processing method.

  2. Description of the Related Art Conventionally, a technique for detecting a detection target region such as a road on which a host vehicle travels is known based on color information of a captured image captured by a camera (see, for example, Patent Document 1).

Japanese Patent No. 2963138

  However, since the conventional technique detects a target detection region based on color information of a captured image, for example, when a road on which the host vehicle is traveling is detected as a detection target region, a building similar in color to the road There are cases in which the target detection area cannot be detected properly, for example, erroneously detecting (for example, a building) as the detection target area.

  An object of the present invention is to provide an image processing apparatus capable of appropriately detecting a detection target region.

  The present invention divides a reference image in which a detection target region is identified into a plurality of regions, extracts color information from a reference region with a coordinate position in a two-dimensional image coordinate system, and extracts the color of a reference region having a common coordinate position. By defining a criterion for determining whether or not an image corresponding to the coordinate position is a detection target region from the feature of the information, the criterion is acquired for each coordinate position of the two-dimensional coordinate system. Then, the captured image is divided into a plurality of areas, color information is extracted from each of the determination areas to which the coordinate positions in the two-dimensional image coordinate system are attached, and the determination areas are shared by using a determination criterion that is common to the determination areas The above problem is solved by determining whether or not the determination area is a detection target area from the characteristics of the color information.

  According to the present invention, by obtaining a determination criterion for each coordinate position in the two-dimensional image coordinate system, using the determination criterion corresponding to each coordinate position, this coordinate position can be obtained from the characteristics of the color information at the coordinate position. It can be determined whether or not the image corresponding to is a detection target region. Thus, for example, it is effective to erroneously detect such an image as a detection target region such as a road even when the color of the image is similar to a road at a coordinate position where a building such as a building is easily detected. As a result, the detection accuracy of the detection target region can be improved.

1 is a schematic configuration diagram of an image processing system according to an embodiment. It is a figure which shows an example of the labeled sample image. It is a figure which shows an example of a reference | standard area | region. It is a figure which shows an example of the histogram of the color information of a reference | standard area | region. 4 is a diagram illustrating an example of reference area data stored in a storage device 30. FIG. It is a flowchart which shows the criteria production | generation process which concerns on this embodiment. It is a flowchart which shows the object area | region detection process which concerns on this embodiment. It is a figure for demonstrating the effect of the image processing system which concerns on this embodiment. It is a figure for demonstrating the other form of the detection method of a detection target area | region.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, the present invention will be described by exemplifying an image processing system that is mounted on a vehicle and detects a running path of the host vehicle that is a detection target region from a captured image obtained by capturing the periphery of the vehicle.

  FIG. 1 is a schematic configuration diagram of an image processing system according to the present embodiment. As illustrated in FIG. 1, the image processing system 1 according to the present embodiment includes a camera 10, an image processing device 20, and a storage device 30.

  The camera 10 includes an image sensor in which a plurality of photoelectric conversion elements such as a CCD image sensor, a MOS sensor, or a CID are two-dimensionally arranged. Color filters corresponding to red (R), green (G), and blue (B) are arranged on the imaging surface of the imaging element of the camera 10 by, for example, a Bayer array, and correspond to red (R). R component image signal from R pixel where color filter is arranged, G component image signal from G pixel where color filter corresponding to green (G) is arranged, color filter corresponding to blue (B) is arranged A B component image signal is output from each of the B pixels. The camera 10 is installed so as to image a road ahead of the traveling direction of the host vehicle, for example, and thereby an imaging range including a road surface (hereinafter also referred to as a traveling path) of the road on which the host vehicle travels. Take an image. The captured image captured by the camera 10 is transmitted to the image processing device 20.

  The storage device 30 stores a plurality of sample images labeled with the detection target area (running path). FIG. 2 is a diagram illustrating an example of a sample image stored in the storage device 30. The sample image stored in the storage device 30 is, for example, a color captured image captured in advance by the camera 10 and is a detection target region by manual operation of a designer or a developer as shown in FIG. The road surface (running road) is labeled (identified). In FIG. 2, the image area of the labeled sample image is shown in gray.

  As shown in FIG. 1, the image processing apparatus 20 includes a ROM (Read Only Memory) in which various programs are stored, a CPU (Central Processing Unit) as an operation circuit for executing the programs stored in the ROM, and an access. RAM (Random Access Memory) functioning as a possible storage device.

  The image processing apparatus 20 according to the present embodiment uses a so-called random forest method to calculate the characteristics of color information of a detection target area or a non-detection target area from a plurality of sample images stored in the storage device 30 in a two-dimensional manner. By learning for each coordinate position in the image coordinate system, it is determined whether or not the captured image captured by the camera 10 is a detection target region (running path) for each coordinate position, thereby detecting the captured image. The target area is detected. In order to realize the above-described functions, the image processing apparatus 20 realizes each function of a determination reference generation function, a captured image acquisition function, a color information extraction function, and a target area determination function. Hereinafter, each function realized by the image processing apparatus 20 will be described.

  First, the determination criterion generation function of the image processing apparatus 20 will be described. The determination criterion generation function generates a determination criterion for determining a detection target region (running path) in the captured image captured by the camera 10 based on the sample image stored in the storage device 30. Specifically, the determination criterion generation function first divides each of the plurality of sample images stored in the storage device 30 into a plurality of regions, and specifies the plurality of divided regions as reference regions. Here, FIG. 3 is a diagram for explaining the reference region. In the example illustrated in FIG. 3, the determination criterion generation function divides the sample image into 11 × 15 = 165 reference regions. In FIG. 3, for the sake of convenience of explanation, the configuration in which the sample image is divided into 165 reference regions is illustrated. However, the number of reference regions is not limited to this example, and may be smaller than 165. It can also be enlarged.

  Next, the determination criterion generation function extracts color information from each reference region constituting the sample image. Specifically, the determination criterion generation function extracts an image signal of R component, G component, or B component from each pixel corresponding to the reference region, and converts the R component, G component, and B component into each reference region. Three corresponding histograms are generated. Here, FIG. 4 is a diagram showing an example of the histogram of the color information (RGB component) of the reference region, (A) is the R component histogram, (B) is the G component histogram, and (C) is the C component histogram. An example of a histogram of a B component is shown.

  For example, as shown in FIG. 4A, the determination criterion generation function divides the magnitude of the component value of the R component extracted from the R pixel in the reference region into, for example, four ranges and is included in each range. A histogram of the R component is generated by counting the frequency based on the number of R pixels. Similarly, as shown in FIG. 4B, the determination reference generation function generates a histogram of the G component based on the G component extracted from the G pixel in the reference area. As shown, a B component histogram is generated based on the B component extracted from the B pixels in the reference region.

  The determination criterion generation function generates three histograms corresponding to the R component, the G component, and the B component in all the reference regions constituting the sample image, and further stores all the histograms stored in the storage device 30. Similar processing is performed on the sample images, and three histograms corresponding to the R component, the G component, and the B component are generated from all the reference regions of all the sample images.

  Further, the determination reference generation function may be configured to extract saturation S, lightness V, and hue H (hereinafter also referred to as HSV component) from the reference area as color information of the reference area. Specifically, the determination criterion generation function can first convert the RGB components extracted from each pixel in the reference region to calculate the saturation S, lightness V, and hue H of the reference region.

  Then, the determination reference generation function divides the magnitude of the saturation S of each pixel area in the calculated reference area into four ranges, and counts the frequency based on the number of pixels included in each range. Thus, a histogram of the saturation S in the reference area can be generated. Similarly, the determination reference generation function may generate a histogram of lightness V in the reference area and a histogram of hue H in the reference area based on the calculated brightness V and hue H in the reference area. it can. In this case as well, the determination reference generation function generates three histograms corresponding to the saturation S, the lightness V, and the hue H in all reference regions constituting the sample image. Three histograms of saturation S, brightness V, and hue H are generated. In addition, the determination criterion generation function performs the same processing for all the sample images stored in the storage device 30, and generates three histograms corresponding to the HSV components from all the reference regions of all the sample images.

  Then, as shown in FIG. 5, the determination reference generation function stores three histograms of the R component, G component, and B component of the reference region together with the coordinate position of the reference region. Here, in the example shown in FIG. 5, in the two-dimensional image coordinate system in which the coordinate position at the upper left of the screen is the origin (0, 0), the X axis extends from the left to the right of the screen and from the upper to the lower of the screen. The Y axis is taken in the direction. The determination reference generation function acquires the coordinate position of each reference region in such a two-dimensional image coordinate system, and stores the coordinate position of each reference region together with three histograms generated based on the color information of the reference region. To do. For example, in the example shown in FIG. 5, in the two-dimensional image coordinate system in which the upper left coordinate position of the screen is the origin (0, 0), a reference region having an X coordinate of 12 and a Y coordinate of 6 is illustrated. In this case, as shown in FIG. 5, the determination criterion generation function stores the three histograms of the reference region and the coordinate position (12, 6) of the reference region in the storage device 30. The origin coordinates are not limited to the upper left corner of the screen, and may be, for example, the lower left corner of the screen.

  The determination reference generation function may be configured to specify the coordinate position of each reference region with the coordinate position where the vanishing point is present as the origin O (0, 0). In this way, in the two-dimensional image coordinate system having the origin at the coordinate position where the vanishing point exists, by obtaining the coordinate position of the reference area, for example, the image area above the vanishing point is the detection target area (running path). Therefore, it is possible to exclude the detection target region from the detection target, and thereby the detection target region can be calculated with higher accuracy. The vanishing point may be obtained by a known method.

  Then, the determination reference generation function is an image corresponding to the coordinate position based on the color information of the reference area having the same coordinate position in the above-described two-dimensional image coordinate system among the plurality of reference areas of the plurality of sample images. A criterion for determining (classifying) whether or not is a detection target region is generated. For example, the determination criterion generation function extracts a plurality of reference regions having a common coordinate position from a plurality of reference regions of a plurality of sample images, and each histogram of R component, G component, and B component of the extracted plurality of reference regions ( Alternatively, the relationship between the saturation S, lightness V, and hue H histograms) and the presence / absence of labeling of the reference region (that is, whether or not the region is a detection target region) is associated based on a predetermined algorithm. Thus, a classification tree for determining (classifying) whether or not the reference area is a detection target area from the color information of the reference area can be generated as a determination reference.

  For example, the determination reference generation function determines (classifies) whether or not an image corresponding to the coordinate position is a detection target area based on histograms of R components of a plurality of reference areas having a common coordinate position. A classification tree is generated, and the generated classification tree is stored in the storage device 30 as a determination criterion corresponding to the R component of the coordinate position. Similarly, the determination criterion generation function stores a determination criterion corresponding to the G component at the coordinate position and a determination criterion corresponding to the G component at the coordinate position in the storage device 30.

  As described above, the determination criterion generation function performs R component, G component for each coordinate position based on the histograms of R component, G component, and B component (or histograms of saturation S, lightness V, and hue H). Three classification trees corresponding to the component, B component (or saturation S, lightness V, hue H) correspond to the R component, G component, B component (or saturation S, lightness V, hue H). It is generated as a determination criterion, and the determination criterion generated for each coordinate position is stored in the storage device 30.

  Next, the captured image acquisition function of the image processing apparatus 20 will be described. The captured image acquisition function acquires a captured image captured by the camera 10 from the camera 10.

  The color information extraction function of the image processing apparatus 20 extracts color information from the captured image acquired by the captured image acquisition function. Specifically, the color information extraction function first divides a captured image captured by the camera 10 into a plurality of areas, and specifies the plurality of divided areas as determination areas. For example, as shown in FIG. 3, the color information extraction function divides the captured image into 11 × 15 = 165 areas, similar to the number of divisions of the reference area obtained by dividing the sample image by the determination reference function. An area is specified as a determination area.

  Then, the color information extraction function extracts color information from each identified determination area. For example, the color information extraction function extracts an R component image signal from an R pixel, a G component image signal from a G pixel, and a B component image signal from a B pixel, as shown in FIG. Thus, as shown in FIG. 4, an R component histogram, a G component histogram, and a B component histogram in the determination region are generated. Alternatively, the color information extraction function converts the RGB component of the determination region into an HSV component, thereby causing an H component (hue) histogram, an S component (saturation) histogram, and a V component (lightness) in the determination region. And generate a histogram.

  The target area determination function of the image processing apparatus 20 determines whether each determination area constituting the captured image is a detection target area (running road) using the determination reference generated by the determination reference generation function. As described above, the determination criterion generated by the determination criterion generation function is based on the relationship between the characteristics of the color information of the image and the detection target region or the non-detection target region from a plurality of sample images labeled with the detection target region. This is a classification tree learned for each coordinate position of the two-dimensional image coordinate system. Therefore, the target area determination function determines (classifies) whether or not the determination area is a detection target area from the characteristics of the color information of the determination area by using a determination criterion corresponding to the coordinate position of the determination area. be able to.

  Specifically, the target area determination function first reads from the storage device 30 a determination criterion that is common to the determination area and the coordinate position in order to determine whether or not each determination area is a detection target area. Then, the target area determination function determines whether the determination area is a detection target area from the color information of the determination area extracted by the color information extraction function, using the read determination criterion. For example, the target area determination function uses an R component determination reference that has the same coordinate position as the determination area, and determines whether the determination area is a detection target area or a non-detection target from the characteristics of the R component histogram of the determination area. It is determined (classified) whether it is one of the areas. Similarly, the target region determination function uses the G component determination reference that has the same coordinate position as the determination region, and this determination region is a detection target region based on the characteristics of the G component histogram of the determination region. Whether or not this determination region is a detection target region based on the characteristics of the B component histogram of this determination region, using a determination criterion for the B component that has the same coordinate position as that of the determination region. judge. Thereby, the target area determination function can obtain three determination results as to whether or not this determination area is a detection target area.

  In addition, when the determination criterion is generated based on the color information of the saturation S, the lightness V, and the hue H by the determination criterion generation function, the target region determination function is the determination region extracted by the color information extraction function. Whether or not the determination region is a detection target region is determined from the histogram of the saturation S, lightness V, and hue H. For example, the target area determination function uses the determination criterion of the saturation S that has the same coordinate position as that of the determination area, and determines whether or not the determination area is a detection target area from the characteristics of the histogram of the saturation S of the determination area. Determine whether. Similarly, the target area determination function uses the lightness V determination criterion that has the same coordinate position as the determination area, and determines whether the determination area is a detection target area from the characteristics of the brightness V histogram of the determination area. Whether or not this determination region is a detection target region is determined based on the characteristics of the hue H histogram of this determination region using the determination criterion of hue H that has the same coordinate position as the determination region. judge. Also in this case, the target area determination function can obtain three determination results for each coordinate position of the two-dimensional image coordinate system.

  Then, the target area determination function determines whether or not the determination area at each coordinate position is a detection target area based on the three determination results corresponding to each coordinate position. In other words, the target area determination function determines that if there are many determination results that the determination area corresponding to the coordinate position is the detection target area among the three determination results corresponding to each coordinate position, When it is determined that the region is a region and there are many determination results that the determination region corresponding to the coordinate position is not the detection target region, it is determined that the determination region is not the detection target region.

  Further, the target area determination function performs the same processing for all the determination areas constituting the captured image, and determines whether each determination area is a detection target area. Thereby, the target area determination function can detect the detection target area from the captured image.

  Subsequently, with reference to FIG. 6, the determination criterion generation processing according to the present embodiment will be described. FIG. 6 is a flowchart showing determination criterion generation processing according to the present embodiment. Note that the determination criterion generation process described below is executed by the determination criterion generation function of the image processing apparatus 20.

  First, in step S <b> 101, a sample image for which a criterion is not generated is acquired from among a plurality of sample images stored in the storage device 30. In step S102, as shown in FIG. 2, the sample image acquired in step S101 is divided into a plurality of reference regions. For example, in the example shown in FIG. 2, it is divided into 11 × 15 = 165 reference areas.

  In steps S103 to S106, the processes in steps S103 to S106 are performed for each of the plurality of reference regions divided in step S102. First, in step S103, a reference area where the processes of steps S103 to S106 are not performed is acquired.

  In step S104, color information is extracted from the reference area acquired in step S103. Specifically, as shown in FIG. 4, the determination criterion generation function extracts the R component, the G component, and the B component from the R pixel, the G pixel, and the B pixel in the reference region, and extracts the extracted pixels. The distribution of the R component, G component, and B component in the region is generated as a histogram. Further, by converting the RGB components of the reference region into HSV components, a histogram of saturation S, brightness V, and hue H may be generated instead of the histograms of R component, G component, and B component.

  In step S105, the coordinate position of the reference area is acquired for the reference area from which the color information has been extracted in step S104. For example, in the example illustrated in FIG. 5, the determination reference generation function acquires the coordinate position of the reference region as (12, 6) in the two-dimensional image coordinate system with the coordinate position at the upper left of the screen as the origin (0, 0). . The determination reference generation function may be configured to detect the vanishing point and acquire the coordinate position of the reference region in the two-dimensional image coordinate system with the origin at the coordinate position where the detected vanishing point exists.

  In step S106, color information of the reference area extracted in step S103, that is, a histogram of R, G, and B components (or a histogram of saturation S, lightness V, and hue H) in the reference area, The coordinate position of the reference area acquired in S104 is stored in the storage device 30.

  In step S107, it is determined whether or not the processing in steps S103 to S106 has been performed for all the reference regions divided in step S102. If the processing of steps S103 to S106 has been performed for all the reference regions divided in step S102, the process proceeds to step S108, while the processing of steps S103 to S106 is performed for all the reference regions divided in step S102. When the process is not performed, the process returns to step S103, and the process of steps S103 to S106 is performed for the reference region where the process of steps S103 to S106 is not performed. As a result, R, G, and B component histograms (or histograms of saturation S, brightness V, and hue H) and coordinate positions are obtained for all reference regions obtained by dividing the sample image. .

  In step S108, it is determined whether or not the processing in steps S102 to S107 has been performed for all the sample images acquired in step S101. If the processing in steps S102 to S107 has been performed for all the sample images acquired in step S101, the process proceeds to step S109. On the other hand, if the processing of steps S102 to S107 has not been performed for all the sample images acquired in step S101, the process returns to step S101, and the steps of steps S102 to S107 are performed for the sample images that have not been processed. The processes of S102 to S107 are performed. As a result, for all reference regions of all sample images stored in the storage device 30, histograms of R, G, and B components (or histograms of saturation S, lightness V, hue H) and coordinates Position.

  In step S109, R, G, and B component histograms (or histograms of saturation S, brightness V, and hue H) and coordinate positions for all reference regions of all sample images and the coordinate position are stored in the storage device 30. Read from. In order to determine (classify) whether or not the coordinate position is the detection target area based on the color information of the reference area having the same coordinate position in the two-dimensional image coordinate system among the plurality of read reference areas. A classification tree is generated.

  Specifically, the determination reference generation function is a histogram (or saturation S) of the R component, the G component, and the B component of the reference region having a common coordinate position in the two-dimensional image coordinate system among the plurality of read reference regions. , Lightness V, hue H histogram), classification trees corresponding to the R component, G component, and B component (or saturation S, lightness V, hue H) are generated. That is, the determination reference generation function is based on a predetermined algorithm, and the distribution of each component for each of the R component, G component, and B component (or saturation S, brightness V, hue H) of the reference region. In some cases, the detection target region can be determined for each coordinate position by learning whether it can be determined (classified) as a detection target region or can be determined (classified) as not being a detection target region. A classification tree for determining (classifying) whether or not is generated. As described above, in this embodiment, three classification trees based on histograms of R, G, and B components (or histograms of saturation S, lightness V, and hue H) are generated for each coordinate position.

  Then, the determination criterion generation function stores the three classification trees generated for each coordinate position in the storage device 30 as determination criteria corresponding to each coordinate position. As described above, in this embodiment, the generation of the determination criterion for determining the detection target region is performed.

  Subsequently, a target area detection process according to the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing target area detection processing according to the present embodiment. Note that the target region detection process described below is executed by the image processing apparatus 20.

  First, in step S <b> 201, the captured image captured by the camera 10 is acquired by the captured image acquisition function of the image processing apparatus 20. In step S202, the captured image acquired in step S201 is divided into a plurality of determination areas by the color information extraction function of the image processing apparatus 20. For example, in this embodiment, since the sample image is divided into 11 × 15 = 165 reference regions, the color information extraction function divides the captured image into 11 × 15 = 165 determination regions.

  Next, the processes in steps S203 to S206 are performed for each of the plurality of determination areas divided in step S202. First, in step S203, color information is extracted from the determination area divided in step S202 by the color information extraction function. Specifically, the color information extraction function extracts the R component, the G component, and the B component from the determination region, respectively, and based on the extracted R component, G component, and B component, the R component, the G component, and Three histograms corresponding to each of the B components are generated. In the determination reference generation process shown in FIG. 6, when the determination reference is generated based on the saturation S, lightness V, and hue H, the R component, G component, and B component of the extracted determination region Saturation S, lightness V, and hue H in the determination area are calculated from the image, and based on the calculated saturation S, lightness V, and hue H, each of the saturation S, lightness V, and hue H is handled. Three histograms are generated.

  In step S204, the coordinate position of the determination area from which the color information is extracted in step S203 is acquired by the color information extraction function of the image processing apparatus 20. Note that the color information extraction function acquires the coordinate position of the determination region in a coordinate system common to the two-dimensional image coordinate system in the determination reference generation process. That is, in the determination reference generation process, when the coordinate position of the reference area is specified by the two-dimensional image coordinate system with the coordinate position at the upper left of the screen as the origin, the coordinate position of the determination area is also the upper left coordinate position of the screen. Is specified by a two-dimensional image coordinate system with the origin as the origin. Alternatively, in the determination reference generation process, when the coordinate position of the reference region is specified by the two-dimensional image coordinate system with the coordinate position where the vanishing point exists as the origin, the vanishing point is also detected for the coordinate position of the determination region. It is specified by a two-dimensional image coordinate system with the existing coordinate position as the origin.

  In step S <b> 205, the target area determination function of the image processing apparatus 20 reads out a determination standard having the same coordinate position as that of the determination area from among the determination standards stored in the storage device 30. In step S206, the target region determination function determines whether or not this determination region is a detection target region using the determination criterion read in step S205.

  Here, the determination criterion according to the present embodiment is a detection target region when there is a feature of any color information using a plurality of sample images labeled with the detection target region, or is not a detection target region. Is learned for each coordinate position. Therefore, the target area determination function determines whether the determination area is a detection target area from the characteristics of the color information of the determination area extracted by the color information extraction function using a determination criterion that has the same coordinate position as the determination area. Can be determined.

  Next, in step S207, it is determined whether or not the processing in steps S203 to S206 has been performed for all the determination areas divided in step S202. When the processes of steps S203 to S206 have been performed for all the determination areas divided in step S202, the target area detection process is terminated, and the process returns to step S201 to detect the target area for the captured image of the next frame. Execute the process. On the other hand, if the processes of steps S203 to S206 are not performed for all the determination areas divided in step S202, the process returns to step S203, and the determination areas for which the processes of steps S203 to S206 are not performed are performed. The processes of S203 to S206 are performed. Thereby, it is determined whether or not all the determination areas constituting the captured image are detection target areas, and as a result, the detection target areas can be detected from the captured image.

  As described above, in the present embodiment, a sample image labeled with a detection target region such as a runway is divided into a plurality of reference regions, and color information is extracted from each of the divided reference regions. Acquire color information of a plurality of reference areas. Then, from the color information of a plurality of reference areas having the same coordinate position in the two-dimensional image coordinate system, it is possible to determine (classify) that the color information of the reference area is a detection target area, or In order to determine whether it can be determined (classified) that it is not a detection target region for each coordinate position, and to determine whether or not the image corresponding to this coordinate position is a detection target region It is generated as a judgment criterion. Then, the captured image captured by the camera 10 is divided into a plurality of determination areas, color information is extracted from the divided determination areas, and the determination information having the same coordinate position as the determination area is used to determine the color information of the determination area. From the feature, it is determined whether or not this determination area is a detection target area. As described above, in the present embodiment, by generating a determination reference for each coordinate position from the color information of the reference area at each coordinate position, for example, a building (building) whose color is similar to the runway that is the detection target area. For example, for a coordinate position where a large number of subjects erroneously detected as a runway are captured only with color information, a determination criterion is generated that determines that the color information is similar to the runway but is not a detection target region. Therefore, in this embodiment, even if a building is imaged in the captured image, whether the determination area where the building is imaged is a target area using a determination criterion that has the same coordinate position as the determination area where the building is imaged. When it is determined whether or not, it is possible to effectively prevent such a determination region from being erroneously detected as a detection target region, and as a result, the detection accuracy of the detection target region can be improved.

  Here, FIG. 8 is a diagram for explaining the effect of the present embodiment, and FIG. 8A is a detection accuracy when the detection target region is determined based on only the color information of the RGB components as in the prior art. (B) shows a case where the detection target region is determined based on the color information of the RGB components and the coordinate position of the two-dimensional image coordinate system with the coordinate position at the upper left of the screen as the origin, as in this embodiment (C) shows the detection accuracy when the detection target region is determined based only on the color information of the HSV component as in the conventional case, and (D) shows the HSV component according to this embodiment. The detection accuracy in the case where the detection target region is determined based on the color information and the coordinate position of the two-dimensional image coordinate system with the vanishing point as the origin is shown.

  When determining the target detection area based on only the RGB components as in the past, for example, when a building such as a building that is not the detection target area is similar to the color of the road surface that is the target detection area, In some cases, the detection target area is erroneously detected, and as a result, as shown in FIG. On the other hand, in this embodiment, as shown in FIG. 8B, the determination region is detected from the characteristics of the RGB components of the determination region using a determination criterion having the same coordinate position as the determination region. By determining whether or not it is an area, even if the color of the building such as a building is similar to the runway, it is determined that the building such as a building is determined to be a runway at such a coordinate position. Since it can prevent effectively, compared with the example shown to FIG. 8 (A), the detection precision of a detection object area | region can be improved.

  Similarly, when the target detection area is determined based only on the color information of the HSV component as in the past, a building such as a building that is not the detection target area is similar to the color of the road surface that is the target detection area. In some cases, it may be erroneously detected as a detection target region, and as a result, as shown in FIG. On the other hand, in this embodiment, as shown in FIG. 8D, the determination region is detected from the feature of the HSV component of the determination region using a determination criterion having the same coordinate position as the determination region. By determining whether or not it is an area, even if the color of the building such as a building is similar to the runway, it is determined that the building such as a building is determined to be a runway at such a coordinate position. Since this can be effectively prevented, the detection accuracy of the detection target region can be improved as compared with the example shown in FIG.

  In the present embodiment, it is also possible to determine whether or not the detection target area is based on the color information of the RGB component, or whether or not the detection target area is based on the color information of the HSV component. Can also be determined. As shown in FIG. 8, when determining whether or not it is a detection target region based on the color information of the HSV component, determining whether or not it is a detection target region based on the color information of the RGB component The detection accuracy of the detection target region is higher than that. On the other hand, since the color information of the HSV component is obtained from the color information of the RGB component, it is based on the color information of the HSV component that it is determined whether the region is a detection target region based on the color information of the RGB component. Rather than determining the detection target area, the detection target area determination process can be performed quickly. Therefore, depending on the usage environment and purpose of the image processing system 1, whether to determine the detection target region based on the RGB component color information or whether to detect the detection target region based on the HSV component color information is appropriately determined. It is preferable to select.

  Furthermore, in the present embodiment, as shown in FIG. 8, the coordinate position of the reference area and the determination area is specified by a two-dimensional image coordinate system with a predetermined coordinate position such as a coordinate position at the upper left of the screen as the origin. The detection target area can be determined for each coordinate position, or by detecting the coordinate position of the reference area and the determination area in the two-dimensional image coordinate system with the coordinate position where the vanishing point exists as the origin. The target area can also be determined. When determining the detection target area using the two-dimensional image coordinate system with the origin at the coordinate position where the vanishing point exists, for example, it is determined that the image area above the vanishing point is not the detection target area (running path). By excluding the detection target area from the detection target, the detection accuracy of the detection target area is thereby improved compared to the case where the detection target area is determined using a two-dimensional image coordinate system with a predetermined coordinate position as the origin. Can be improved. On the other hand, when the detection target region is determined using a two-dimensional image coordinate system with a predetermined coordinate position as the origin, since the calculation of the vanishing point is unnecessary, the coordinate position where the vanishing point exists is set as the origin. Compared with the case where the detection target region is determined using the two-dimensional image coordinate system, the detection target region determination processing can be performed quickly. Therefore, according to the use environment and purpose of the image processing system 1, the detection target region is determined using a two-dimensional image coordinate system with the coordinate position where the vanishing point exists as the origin, or the predetermined coordinate position is set as the origin. It is preferable to appropriately select whether to determine the detection target region using the two-dimensional image coordinate system.

  In the present embodiment, for each coordinate position, three classification trees corresponding to the R component, the G component, and the B component (or saturation S, lightness V, and hue H) are generated as determination criteria. By using these three classification trees, it is determined whether or not the determination area is a detection target area, so that three determination results are output in each determination area. And the determination result with many numbers is selected as a determination result in this determination area | region among these determination results. In this way, by determining whether or not the determination region is a detection target region using a plurality of classification trees, the determination accuracy of the detection target region is improved as compared with the case where determination is performed with one classification tree. Can do.

  Further, in this embodiment, it is possible to detect the detection target region (road surface) simply by preparing the determination criterion in advance and applying the determination criterion prepared to the latest captured image captured by the camera 10. The detection result can be obtained in real time.

  The embodiment described above is described for facilitating understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  For example, in the above-described embodiment, the configuration in which the RGB component or the HSV component is extracted as the color information of the reference region and the determination region is exemplified. However, the configuration is not limited to this configuration. Further, a gradation, that is, a luminance gradient may be extracted.

  For example, as shown in FIG. 9A, each reference area is divided into nine fan-shaped areas having a central angle of 20 °, and the luminance of the divided nine fan-shaped areas is shown as shown in FIG. 9B. The histogram can be generated as a histogram indicating the gradation of the reference area, and a determination reference can be generated based on the generated gradation histogram. In this case, a histogram showing the gradation of the determination area is generated as color information of the determination area constituting the captured image by the same method as in FIGS. 9A and 9B, and the coordinate position is common to the determination area. Whether or not the determination area is a detection target area is determined from the characteristics of the gradation histogram of the determination area using the determination criterion. Also in this case, it is possible to appropriately detect whether or not the determination area is a detection target area. In addition, FIG. 9 is a figure for demonstrating the other form of the detection method of a detection target area | region.

  In the above-described embodiment, the configuration in which the RGB component or the HSV component is extracted as the color information of the reference region and the determination region is exemplified. However, the present invention is not limited to this configuration, and for example, both the RGB component and the HSV component are extracted. Then, it may be configured to generate a determination criterion corresponding to the extracted RGB component and HSV component.

  As described above, the determination accuracy of the detection target area using the HSV component color information is higher than the determination accuracy, but the detection speed of the detection target area determination using the RGB component color information is faster. Therefore, for example, when the captured image is captured at a short frame rate, such as when the host vehicle is traveling at high speed, the detection target area is determined using the color information of the RGB components. When the captured image can be captured at a long frame rate, the detection target region is determined using the HSV component color information, such as determining the detection target region using the HSV component color information, and the RGB component. It is preferable to switch the determination of the detection target region using the color information.

  Furthermore, in the above-described embodiment, three histograms corresponding to the R component, the G component, and the B component (or three histograms corresponding to the saturation S, the lightness V, and the hue H) are respectively obtained in the reference region and the determination region. Although the configuration to be generated has been illustrated, the present invention is not limited to this configuration. For example, as described below, the number of histograms generated in the reference region and the determination region may be increased. That is, the reference region and the determination region are further divided into four regions, and for each of these regions, three histograms corresponding to the R component, G component, and B component (or saturation S, lightness V, and hue H are supported). 3 histograms for the R component, the G component, and the B component (or the histogram for saturation S, lightness V, and hue H) from the reference region and the determination region. = 12 can be generated. That is, by dividing the reference region and the determination region into four regions, four histograms corresponding to the R component, four histograms corresponding to the G component, and four histograms corresponding to the B component are generated. Accordingly, 12 histograms can be generated in the reference region and the determination region (the same applies to the histograms for saturation S, lightness V, and hue H). In this case, since 12 classification trees can be obtained as determination criteria for each coordinate position, by determining whether or not the determination region is a detection target region using such a determination criterion, The determination accuracy can be improved.

  In the above-described embodiment, three classification trees are determined by extracting RGB components or HSV components as color information of the reference region and the determination region, and generating three histograms based on the RGB components or HSV components. Although the configuration generated as the reference is illustrated, the present invention is not limited to this configuration. For example, 6 classification trees are extracted by extracting RGB components and HSV components and generating 6 histograms based on the RGB components and HSV components. May be generated as a criterion. Further, as described above, a gradation may be extracted as the color information of the reference area and the determination area and combined with the RGB component and / or HSV component.

  Further, in the above-described embodiment, when the captured image is divided into a plurality of determination areas, the number of determination areas is set to be the same as the number of reference areas into which the sample image is divided. However, the number of determination areas may be different from the number of reference areas. Even in such a case, for example, when the sample image and the captured image are superimposed, the coordinate position of the determination region that overlaps the reference region is associated with the coordinate position of the reference region, so that the coordinates of the determination region It is possible to determine whether or not this determination region is a detection target region by using the determination criterion of the reference region corresponding to the position. As described above, the determination reference having the same coordinate position as the determination region in the present invention is generated based on the color information of the reference region overlapping with the determination region when the sample image and the captured image are superimposed. It is a criterion.

  The captured image acquisition function of the image processing apparatus 20 of the above-described embodiment is the captured image acquisition unit of the present invention, the color information extraction function is the extraction unit of the present invention, and the target area determination function is the determination criterion acquisition unit of the present invention. And determination means respectively.

DESCRIPTION OF SYMBOLS 1 ... Image processing system 10 ... Camera 20 ... Image processing apparatus 30 ... Memory | storage device

Claims (8)

A plurality of reference images in which detection target areas are identified are divided into a plurality of reference areas, color information is extracted from the reference areas with coordinate positions in a two-dimensional image coordinate system, and the reference areas have a common coordinate position. By defining a criterion for determining whether an image corresponding to the coordinate position is the detection target region from the characteristics of the color information, the criterion for each coordinate position of the two-dimensional coordinate system A criterion acquisition means for acquiring
Captured image acquisition means for acquiring a captured image;
An extraction unit that divides the captured image into a plurality of determination areas and extracts color information from the determination areas with coordinate positions in the two-dimensional image coordinate system;
Among the plurality of determination criteria acquired for each coordinate position, the determination region is the detection target region based on the characteristics of the color information of the determination region using a determination criterion that has the same coordinate position as the determination region. An image processing apparatus comprising: determination means for determining whether or not. The image processing apparatus according to claim 1,
The determination criterion acquisition unit obtains the color information distribution of the reference region having the same coordinate position as a feature of the color information of the reference region having the same coordinate position, so that the color of the reference region having the same coordinate position is obtained. An image processing apparatus, wherein the determination criterion corresponding to the coordinate position is defined from an information distribution. The image processing apparatus according to claim 2,
The determination reference acquisition means obtains the distribution of the RGB image signal in the reference region having the same coordinate position as a feature of the color information having the same coordinate position, so that the RGB image signal of the reference region having the same coordinate position is obtained. Defining the criterion corresponding to the coordinate position from the distribution of
The determination unit determines whether the determination region is the detection target region from the distribution of the RGB image signals in the determination region, using the determination criterion defined based on the distribution of the RGB image signals. An image processing apparatus characterized by determining. The image processing apparatus according to claim 2,
The determination reference acquisition unit obtains the distribution of the HSV image signal in the reference region having the same coordinate position as a feature of color information having the same coordinate position, so that the HSV image signal of the reference region having the same coordinate position is obtained. Defining the criterion corresponding to the coordinate position from the distribution of
The determination unit determines whether the determination region is the detection target region based on the distribution of the HSV image signal in the determination region using the determination criterion defined based on the distribution of the HSV image signal. An image processing apparatus characterized by determining. The image processing apparatus according to any one of claims 1 to 4,
The determination criterion acquisition unit obtains a luminance gradient in the reference region having a common coordinate position as a characteristic of color information having the common coordinate position, thereby obtaining the luminance gradient in the reference region having the common coordinate position from the luminance gradient in the reference region. Define the criterion corresponding to the coordinate position,
The determination unit determines whether the determination region is the detection target region from the luminance gradient in the determination region, using the determination criterion defined based on the luminance gradient. An image processing apparatus. An image processing apparatus according to any one of claims 1 to 5,
The determination reference acquisition means acquires the determination reference for each coordinate position of the two-dimensional image coordinate system in the two-dimensional image coordinate system having a predetermined coordinate position as an origin,
In the two-dimensional image coordinate system in which the predetermined coordinate position is the origin, the determination unit uses the determination criterion that is common to the determination area and the coordinate position, and determines the determination area from the color information of the determination area. An image processing apparatus for determining whether or not a detection target area. An image processing apparatus according to any one of claims 1 to 5,
Further comprising vanishing point detecting means for detecting the vanishing point,
The determination criterion acquisition means acquires the determination criterion for each coordinate position of the two-dimensional image coordinate system in a two-dimensional image coordinate system with the coordinate position of the vanishing point as an origin,
In the two-dimensional image coordinate system in which the coordinate position of the vanishing point is the origin, the determination unit uses the determination criterion in which the coordinate position is the same as the determination area, and determines the determination area from the color information of the determination area. An image processing apparatus that determines whether or not the detection target region is present. A plurality of reference images in which detection target areas are identified are divided into a plurality of reference areas, color information is extracted from the reference areas with coordinate positions in the two-dimensional image coordinate system, and the colors of the reference areas having the same coordinate positions By defining a criterion for determining whether or not an image corresponding to the coordinate position is the detection target region from information features, the determination criterion is obtained for each coordinate position of the two-dimensional coordinate system. A criteria acquisition step to perform,
A captured image is acquired, the acquired captured image is divided into a plurality of determination regions, color information is extracted from the determination region with the coordinate position of the two-dimensional image coordinate system, and the determination region and the coordinate position are common A target region detection step of determining whether the determination region is the detection target region from the characteristics of the color information of the determination region using the determination criterion of the reference region. And an image processing method.

Images