JP2016197388A - Recognition device, recognition method, and program - Google Patents

Abstract

An object of the present invention is to provide a recognition device, a recognition method, and a program capable of more accurately recognizing a signal area of a traffic light from image information. A recognition apparatus according to an embodiment includes an image acquisition unit that acquires image information around a vehicle, a position acquisition unit that acquires position information indicating the position of the vehicle, and a traffic signal in an area specified by the position information. A selection unit for selecting recognition information for recognition, a recognition unit for recognizing a signal shape region indicating the shape of a signal of a traffic light from image information based on the recognition information, and a signal shape region based on the recognition information. And a determination unit that determines whether or not the signal region indicates. [Selection] Figure 2

Description

  The present invention relates to a recognition apparatus, a recognition method, and a program.

  2. Description of the Related Art Conventionally, a technique for assisting a driver or memorizing a video at the time of an accident using an in-vehicle camera has been known. For example, driver assistance technology using an in-vehicle camera includes an automatic brake function for avoiding an obstacle or reducing an impact at the time of a collision, and an alarm function for warning the maintenance of a distance from a preceding vehicle.

  Further, for example, Patent Document 1 discloses an invention for accurately recognizing a signal of a traffic signal from an image by recognizing separately the color of the signal of the traffic signal and the shape of the signal from two images taken of the same subject. Is disclosed.

  However, in the prior art, there is a possibility that an object similar to the color and shape of the signal area of the traffic signal may be erroneously recognized as the signal area of the traffic signal.

  The present invention has been made in view of the above, and an object of the present invention is to provide a recognition device, a recognition method, and a program capable of more accurately recognizing a signal area of a traffic light from image information.

  In order to solve the above-described problems and achieve the object, the present invention provides an image acquisition unit that acquires image information around a vehicle, a position acquisition unit that acquires position information indicating the position of the vehicle, and the position A selection unit for selecting recognition information for recognizing a traffic signal in a region identified from information, a recognition unit for recognizing a signal shape region indicating a signal shape of the traffic signal from the image information based on the recognition information, A determination unit that determines whether the signal shape area is a signal area indicating a signal of the traffic light based on the recognition information.

  According to the present invention, there is an effect that the signal area of the traffic light can be more accurately recognized from the image information.

FIG. 1 is a diagram illustrating an example of a vehicle on which the recognition apparatus according to the first embodiment is mounted. FIG. 2 is a diagram illustrating an example of the configuration of the recognition apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example 1 of image information including a traffic light. FIG. 4 is a diagram illustrating an example 2 of image information including a traffic light. FIG. 5 is a diagram illustrating an example of the (U, V) distribution of red signal pixels. FIG. 6 is a diagram illustrating an example of the (U, V) distribution of the green signal pixels. FIG. 7 is a diagram illustrating an example of the (U, V) distribution of yellow signal pixels. FIG. 8 is a diagram illustrating an example of a green signal pixel region according to the first embodiment. FIG. 9 is a diagram illustrating an example of the expansion region of the green signal pixel according to the first embodiment. FIG. 10 is a diagram illustrating an example of the green signal shape region of the first embodiment. FIG. 11 is a diagram illustrating an example of a rectangular area according to the first embodiment. FIG. 12 is a diagram illustrating an example of a peripheral region according to the first embodiment. FIG. 13 is a diagram illustrating an example of a recognition result of a green signal according to the first embodiment. FIG. 14 is a flowchart illustrating an example of a recognition method according to the first embodiment. FIG. 15 is a diagram illustrating an example of a hardware configuration of the image acquisition unit according to the first embodiment. FIG. 16 is a diagram illustrating an example of the peripheral region of the green signal shape region of the second embodiment. FIG. 17 is a diagram illustrating an example of the peripheral region of the red signal shape region of the second embodiment. FIG. 18 is a diagram illustrating an example of a peripheral region of the green signal shape region of the second embodiment. FIG. 19 is a diagram illustrating an example of a peripheral region of the red signal shape region of the second embodiment. FIG. 20 is a diagram for explaining an example of a method for determining the position and size of the peripheral region of the green signal shape region of the second embodiment.

  Hereinafter, embodiments of a recognition device, a recognition method, and a program will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram illustrating an example of a vehicle 200 on which the recognition device 100 according to the first embodiment is mounted. The recognition device 100 according to the first embodiment is installed in the vicinity of the rearview mirror of the windshield of the vehicle 200. The recognition apparatus 100 according to the first embodiment recognizes a signal area indicating a signal of the traffic light 300 from image information through image recognition processing.

  FIG. 2 is a diagram illustrating an example of the configuration of the recognition apparatus 100 according to the first embodiment. The recognition apparatus 100 according to the first embodiment includes an image acquisition unit 10, a position acquisition unit 11, and a signal processing unit 20. The signal processing unit 20 includes an interface unit 21, a selection unit 22, a storage unit 23, a recognition unit 24, a determination unit 25, and an output unit 26.

  The image acquisition unit 10 acquires color image information expressed in the (R, G, B) color space. The image acquisition unit 10 inputs the image information to the interface unit 21. The image acquisition unit 10 is a camera that captures the surroundings of the vehicle 200, for example.

  The position acquisition unit 11 acquires position information indicating the position of the vehicle 200 and inputs the position information to the interface unit 21. The position information is, for example, GPS (Global Positioning System) information, map coordinate information indicating the position of the vehicle 200, and the like.

  When receiving image information from the image acquisition unit 10, the interface unit 21 converts the image information into an information format that can be received by the recognition unit 24 as a time-series continuous image frame. The interface unit 21 inputs the image information whose information format has been converted to the recognition unit 24.

  Further, when the interface unit 21 receives the position information from the position acquisition unit 11, the interface unit 21 inputs the position information to the selection unit 22.

  When receiving the location information from the interface unit 21, the selection unit 22 identifies the area from the location information. Then, the selection unit 22 selects recognition information for recognizing the traffic signal 300 in the specified area from one or more pieces of recognition information stored in the storage unit 23. The selection unit 22 inputs the recognition information to the recognition unit 24 and the determination unit 25.

  The storage unit 23 stores information. The memory | storage part 23 memorize | stores recognition information for every area, for example. The signal shape and signal color of the traffic light 300 vary depending on the region. For example, the shape of the traffic light 300 may be a horizontal type or a vertical type depending on the region.

  FIG. 3 is a diagram illustrating an example 1 of image information including the traffic light 300a. The example of FIG. 3 shows the traffic light 300a when the signal arrangement is a horizontal type. The green signal area 101 is an area showing a green signal. For example, in Japan, the horizontal traffic light 300a is common.

  FIG. 4 is a diagram illustrating an example 2 of image information including the traffic light 300b. The example of FIG. 4 shows the traffic light 300b when the signal arrangement is vertical. The green signal area 401 is an area showing a green signal. For example, in the United States, a vertical traffic light 300b is common. In Japan, vertical traffic lights 300b are installed in areas with a large amount of snow.

  The recognition information is information for recognizing the traffic signal 300 in the area specified from the position information. The recognition information includes color information for determining the signal pixel and feature information of the peripheral area of the signal shape area. The signal shape area is an area indicating the signal shape of the traffic light 300. The feature information of the peripheral area of the signal shape area will be described later with reference to Table 1 and FIG.

  The color information for determining a signal pixel includes, for example, a threshold value for determining a red signal pixel, a threshold value for determining a blue signal pixel, and a threshold value for determining a yellow signal pixel.

  The red signal pixel is a pixel indicating the color of the red signal. The threshold values for determining the red signal pixel are the U threshold value (Ur-min and Ur-max) and the V threshold value (Vr-min and Vr-max). This threshold is determined based on the (U, V) distribution of red signal pixels.

  FIG. 5 is a diagram illustrating an example of the (U, V) distribution of red signal pixels. The example of FIG. 5 shows the (U, V) distribution of red signal pixels obtained from pixel information in the (R, G, B) color space of a plurality of image samples in which the red signal is captured. The specific values of the U threshold (Ur-min and Ur-max) and the V threshold (Vr-min and Vr-max) may be arbitrarily determined. However, if the range of the threshold is too wide, there is a high possibility that a pixel that is not a red signal color is detected. Therefore, the threshold is set so as not to overlap with the (U, V) value of a pixel that is not a red signal.

  The blue signal pixel is a pixel indicating the color of the blue signal. The threshold values for determining the green light pixel are the U threshold values (Ub-min and Ub-max) and the V threshold values (Vb-min and Vb-max). This threshold is determined based on the (U, V) distribution of the green signal pixels.

  FIG. 6 is a diagram illustrating an example of the (U, V) distribution of the green signal pixels. The example of FIG. 6 shows a (U, V) distribution of blue signal pixels obtained from pixel information in the (R, G, B) color space of a plurality of image samples in which a blue signal is captured. Note that specific values of the U threshold (Ub-min and Ub-max) and the V threshold (Vb-min and Vb-max) may be arbitrarily determined. However, if the range of the threshold is too wide, the possibility of detecting a pixel that is not the color of the blue signal increases, so the threshold is set so as not to overlap the (U, V) value of the pixel that is not the blue signal.

  The yellow signal pixel is a pixel indicating the color of the yellow signal. The threshold values for determining the yellow signal pixel are the U threshold values (Uy-min and Uy-max) and the V threshold values (Vy-min and Vy-max). This threshold is determined based on the (U, V) distribution of yellow signal pixels.

  FIG. 7 is a diagram illustrating an example of the (U, V) distribution of yellow signal pixels. The example of FIG. 7 shows the (U, V) distribution of yellow signal pixels obtained from pixel information in the (R, G, B) color space of a plurality of image samples in which the yellow signal is captured. The specific values of the U threshold (Uy-min and Uy-max) and the V threshold (Vy-min and Vy-max) may be arbitrarily determined. However, if the range of the threshold is too wide, the possibility of detecting a pixel that is not the color of the yellow signal increases, so the threshold is set so as not to overlap with the (U, V) value of the pixel that is not the yellow signal.

  In the following description, a specific example will be described in which the selection unit 22 selects recognition information for recognizing the signal of the traffic light 300a (see FIG. 3).

  Returning to FIG. 2, the recognition unit 24 receives image information from the interface unit 21 and receives recognition information from the selection unit 22. The recognition unit 24 recognizes the signal shape area based on the recognition information. Specifically, the recognition unit 24 first converts image information in the (R, G, B) color space into image information in the (Y, U, V) color space using the following equation (1). .

  Next, when the (U, V) value of the image information in the (Y, U, V) color space is included in the threshold range for determining the red signal pixel included in the recognition information, the recognition unit 24 V) A pixel having a value is recognized as a red signal pixel. Similarly, when the (U, V) value of the image information in the (Y, U, V) color space is included in the threshold range for determining the green signal pixel included in the recognition information, the recognition unit 24 (U, V) A pixel having a value is recognized as a green signal pixel. When the (U, V) value of the image information in the (Y, U, V) color space is included in the threshold range for determining the yellow signal pixel included in the recognition information, the recognizing unit 24 (U, V ) Recognize a pixel having a value as a yellow signal pixel.

  Hereinafter, a specific example will be described in which the recognition unit 24 recognizes a green signal pixel. The case of the red signal pixel and the yellow signal pixel is the same as that of the blue signal pixel.

  FIG. 8 is a diagram illustrating an example of a green light pixel region 102 according to the first embodiment. The size of the green signal pixel region 102 in FIG. 8 is smaller than the actual size of the green signal region 101 (see FIG. 3). That is, the example of FIG. 8 shows a case where an area that should originally be recognized as a blue signal area is not recognized as a blue signal pixel due to the influence of a noise pixel. The noise pixel is a noise pixel caused by surrounding conditions at the time of shooting, a noise pixel caused by characteristics of the image sensor, a noise pixel caused by dust attached to the surface of the image sensor, or the like. The noise pixels caused by the surrounding state at the time of shooting are pixels in a blue signal region that is reflected by light such as sunlight when the traffic light 300 is shot by the image acquisition unit 10. The noise pixel resulting from the characteristics of the image sensor is, for example, a pixel affected by random noise.

  Returning to FIG. 2, the recognition unit 24 performs an expansion process for expanding the green signal pixel region 102. Specifically, for each pixel in the green signal pixel region 102, the recognition unit 24 covers the pixel with a plurality of pixels, thereby expanding the green signal pixel region 102 into the green signal pixel expansion region. The recognition unit 24 covers each pixel with, for example, n × n (n is an integer of 1 or more) block pixels. For example, when n = 7, each pixel of the green signal pixel region 102 is expanded into an expansion region of the green signal pixel further including 48 (7 × 7-1) pixels around the pixel.

  FIG. 9 is a diagram illustrating an example of the expansion region 103 of the green signal pixel according to the first embodiment. In the example of FIG. 9, the recognition unit 24 performs expansion processing on the green signal pixel region 102 recognized by the recognition unit 24, thereby expanding the green signal pixel including the green signal shape region 104 indicating the original green signal shape. A case where the region 103 is obtained is shown.

  Returning to FIG. 2, the recognition unit 24 performs shape recognition processing for recognizing the shape of the green signal shape region 104 indicating the shape of the green signal. Specifically, the recognition unit 24 determines whether or not a circular pixel region can be recognized in the expansion region 103 of the green signal pixel by performing a Hough conversion on the region 102 of the green signal pixel. When the circular pixel region can be recognized, the recognition unit 24 recognizes that the circular pixel region is the green signal shape region 104 indicating the shape of the green signal of the traffic light 300.

  FIG. 10 is a diagram illustrating an example of the green signal shape region 104 of the first embodiment. The example of FIG. 10 shows an example in which the green signal shape region 104 is recognized as a circular pixel region by performing the Hough transform on the green signal pixel region 102 (see FIG. 8).

  Returning to FIG. 2, the recognition unit 24 next recognizes a rectangular area circumscribing the green signal shape area 104.

  FIG. 11 is a diagram illustrating an example of the rectangular area 105 according to the first embodiment. The example of FIG. 11 illustrates a case where the recognition unit 24 recognizes a rectangular area 105 that circumscribes the green signal shape area 104 recognized as a circular pixel area.

  Returning to FIG. 2, the recognition unit 24 then inputs information indicating the rectangular area 105 to the determination unit 25.

  The determination unit 25 receives recognition information from the selection unit 22 and receives information indicating the rectangular area 105 from the recognition unit 24. The determination unit 25 determines whether or not the green signal shape area 104 inscribed in the rectangular area 105 is the green signal area 101 of the traffic light 300a based on the recognition information. Specifically, when the feature information of the peripheral region of the green signal shape region 104 and the feature information of the peripheral region of the green signal region 101 of the traffic light 300a included in the recognition information are similar, the determination unit 25 determines the green signal shape. It is determined that the area 104 is the green signal area 101 of the traffic light 300a.

  Here, an example of feature information indicating features of the peripheral region of the signal region will be described.

  Table 1 shows an example of characteristic information of the traffic light 300a.

  The feature information includes a position, a size, and an average luminance range for each signal type. For example, the position of the peripheral area of the green signal area 101 is (X-x1, Y-y1). The size of the peripheral area of the green signal area 101 is (W1, H1). Here, the position and size of the peripheral area will be described.

  FIG. 12 is a diagram illustrating an example of a peripheral region according to the first embodiment. FIG. 12 shows a case where the feature information of the peripheral region 106 of the green signal shape region 104 is compared with the feature information of the peripheral region of the green signal region 101 of the traffic light 300a. The coordinates of the point O are (X, Y). A point O indicates the center of a circular pixel region (green signal shape region 104) inscribed in the rectangular region 105. The coordinates of the point P are (X-x1, Y-y1). The peripheral region 106 is determined by the point P, the width W1, and the height H1.

  For example, the average luminance range in the peripheral region of the green signal region 101 is (Imin1, Imax1). The average luminance range is determined based on the color of the casing of the traffic light 300a, for example. The determination unit 25 calculates the average luminance of the region 107 included in the peripheral region 106, and determines that the green signal shape region 104 is the green signal region 101 of the traffic light 300a when the average luminance is not less than Imin1 and not more than Imax1. .

  Thus, for example, an image area in which a part of a red sign that is the same as the color of the red signal is hidden behind a tree branch and the part where the red sign is visible through the branch is circular is a red signal area. Can be prevented from being erroneously recognized.

  FIG. 13 is a diagram illustrating an example of a recognition result of a green signal according to the first embodiment. The example of FIG. 13 illustrates a case where the green signal of the traffic light 300 a included in the image information illustrated in FIG. 3 is recognized as a rectangular area 105 including the green signal area 101.

  Returning to FIG. 2, the determination unit 25 inputs recognition result information indicating the rectangular region 105 including the green signal region 101 to the output unit 26. Upon receiving the recognition result information from the determination unit 25, the output unit 26 outputs the recognition result information. The recognition result information is used for processing such as detection of dangerous driving such as signal ignorance and sudden start by the driver, for example.

  Next, the recognition method of the first embodiment will be described.

  FIG. 14 is a flowchart illustrating an example of a recognition method according to the first embodiment. Since the description is the same when the signal is a blue signal, a yellow signal, and a red signal, in the description of the recognition method of the first embodiment, a case where the signal is a blue signal will be described as an example. A specific example will be described in which a green signal (see FIG. 3) of the traffic light 300a is recognized.

  First, the image acquisition unit 10 captures the surroundings of the vehicle 200 and acquires color image information expressed in the (R, G, B) color space (step S1). Next, the recognizing unit 24 uses the above equation (1) to convert the color image information expressed in the (R, G, B) color space into the image information in the (Y, U, V) color space. (Step S2).

  Moreover, the position acquisition part 11 acquires the positional information which shows the position of the vehicle 200 (step S3). Next, the selection unit 22 selects recognition information for recognizing the traffic signal 300a in the area specified by the position information from one or more pieces of recognition information stored in the storage unit 23 (step S4).

  Next, when the recognition unit 24 includes the (U, V) value of the image information in the (Y, U, V) color space within the threshold range for determining the green signal pixel included in the recognition information, the (U, V) , V) a pixel having a value is recognized as a green signal pixel (step S5). Next, the recognizing unit 24 expands the green signal pixel region 102 recognized in step S5 (step S6).

  Next, the recognizing unit 24 determines whether or not the green signal shape region 104 indicating the shape of the green signal can be recognized from the expansion region 103 of the green signal pixel obtained by the process of step S6 (step S7). Specifically, the recognition unit 24 performs Hough transform on the green signal pixel region 102 to determine whether or not a circular pixel region (the green signal shape region 104) can be recognized in the green signal pixel expansion region 103.

  If the green signal shape area 104 can be recognized (step S7, Yes), the recognition process proceeds to step S8. If the green signal shape area 104 cannot be recognized (No at Step S7), the recognition process is terminated.

  Next, the determination unit 25 determines whether or not the feature information of the peripheral region of the green signal shape region 104 is similar to the feature information of the peripheral region of the green signal region 101 of the traffic light 300a included in the recognition information selected in step S4. Is determined (step S8).

  When it is similar (step S8, Yes), the determination unit 25 determines that the green signal shape area 104 recognized in step S7 is the green signal area 101 (step S9). Next, the output unit 26 outputs recognition result information indicating the rectangular area 105 including the green signal area 101 (step S10). If they are not similar (step S8, No), the recognition process is terminated.

  Next, an example of the hardware configuration of the recognition apparatus 100 according to the first embodiment will be described.

  First, an example of the hardware configuration of the image acquisition unit 10 will be described.

  FIG. 15 is a diagram illustrating an example of a hardware configuration of the image acquisition unit 10 according to the first embodiment. The image acquisition unit 10 according to the first embodiment includes a photographing optical system 201, a mechanical shutter 202, a motor driver 203, a CCD (Charge Coupled Device) 204, a CDS (Correlated Double Sampling) circuit 205, an A / D converter. 206, timing signal generator 207, image processing circuit 208, LCD (Liquid Crystal Display) 209, CPU (Central Processing Memory) 210, RAM (Random Access Memory) 211, ROM (Read Only Memory) 212, SDRAM (synchronous SDRAM) Access Memory) 213, compression / decompression circuit 214, memo 215, an operation unit 216 and an output I / F 217.

  The image processing circuit 208, CPU 210, RAM 211, ROM 212, SDRAM 213, compression / decompression circuit 214, memory 215, operation unit 216, and output I / F 217 are connected via a bus 220.

  The photographing optical system 201 collects the light reflected by the subject. The mechanical shutter 202 is opened for a predetermined time so that the light condensed by the photographing optical system 201 is incident on the CCD 204. A motor driver 203 drives the photographing optical system 201 and the mechanical shutter 202.

  The CCD 204 forms light incident through the mechanical shutter 202 as a subject image, and inputs analog image information indicating the subject image to the CDS circuit 205. When the analog image information is received from the CCD 204, the CDS circuit 205 removes the noise component of the image information and inputs the analog image information from which the noise component has been removed to the A / D converter 206. When receiving analog image information from the CDS circuit 205, the A / D converter 206 converts the analog image information into digital image information. The A / D converter 206 inputs digital image information to the image processing circuit 208. The timing signal generator 207 transmits a timing signal to the CCD 204, the CDS circuit 205, and the A / D converter 206 in accordance with a control signal from the CPU 210, so that the CCD 204, the CDS circuit 205, and the A / D converter 206 are transmitted. Control the timing of operation.

  Upon receiving digital image information from the A / D converter 206, the image processing circuit 208 uses the SDRAM 213 to perform image processing on the digital image information. Image processing includes, for example, CrCb conversion processing, white balance control processing, contrast correction processing, edge enhancement processing, and color conversion processing. The white balance control process is an image process for adjusting the color density of image information. The contrast correction process is an image process for adjusting the contrast of image information. The edge enhancement process is a process for adjusting the sharpness of image information. The color conversion process is an image process for adjusting the hue of image information.

  The image processing circuit 208 inputs the image information subjected to the above-described image processing to the LCD 209 or the compression / decompression circuit 214. The LCD 209 is a liquid crystal display that displays image information received from the image processing circuit 208.

  The CPU 210 controls the operation of the image acquisition unit 10 by executing a program. The RAM 211 is a work area when the CPU 210 executes a program, and a storage area that can be read and written and is used for storing various types of information. The ROM 212 is a read-only storage area that stores programs executed by the CPU 210.

  The SDRAM 213 is a storage area for temporarily storing image information to be processed when the image processing circuit 208 performs image processing.

  When receiving the image information from the image processing circuit 208, the compression / decompression circuit 214 compresses the image information. The compression / decompression circuit 214 stores the compressed image information in the memory 215. When the compression / decompression circuit 214 receives image information from the memory 215, the compression / decompression circuit 214 decompresses the image information. The compression / decompression circuit 214 temporarily stores the decompressed image information in the SDRAM 213. The memory 215 stores the compressed image information.

  The operation unit 216 receives an operation from the user of the image acquisition unit 10. The operation unit 216 receives an operation for storing, for example, image information displayed on the LCD 209 in the memory 215. The output I / F 217 is an interface for transmitting image information from the image acquisition unit 10 to the signal processing unit 20.

  Note that the interface unit 21, the selection unit 22, the recognition unit 24, the determination unit 25, and the output unit 26 of the signal processing unit 20 described with reference to FIG. 2 may be implemented as hardware as a signal processing board (signal processing circuit). Alternatively, it may be realized by software (program) executed by the CPU 210 of the image acquisition unit 10 and the CPU of another device.

  The position acquisition unit 11 is realized by a GPS device, for example. The storage unit 23 is realized by, for example, a memory on the signal processing board and the memory 215 of the image acquisition unit 10.

  A program executed by the recognition apparatus 100 (CPU 210) of the first embodiment is an installable or executable file, and is a computer such as a CD-ROM, a memory card, a CD-R, and a DVD (Digital Versatile Disk). Recorded on a readable storage medium and provided as a computer program product.

  Note that the program executed by the recognition apparatus 100 according to the first embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Moreover, you may comprise so that the program performed with the recognition apparatus 100 of 1st Embodiment may be provided via networks, such as the internet, without downloading.

  Further, the program of the recognition apparatus 100 of the first embodiment may be configured to be provided by being incorporated in advance in the ROM 212 or the like.

  When the selection unit 22, the recognition unit 24, the determination unit 25, the output unit 26, and the like are realized by a program executed by the recognition apparatus 100 of the first embodiment, the selection unit 22, the recognition unit 24, the determination unit 25, and the output The unit 26 and the like are realized in the RAM 211 when the CPU 210 reads out and executes a program from the ROM 212 or the storage medium.

  As described above, in the recognition apparatus 100 according to the first embodiment, the selection unit 22 selects recognition information for recognizing the traffic signal 300 in the area specified from the position information. The recognition unit 24 recognizes a signal shape region indicating the shape of the signal of the traffic light 300 from the image information based on the recognition information. And the determination part 25 determines whether a signal shape area | region is a signal area | region which shows the signal of the traffic light 300 based on recognition information. As a result, the signal area of the traffic light 300 can be more accurately recognized from the image information.

(Second Embodiment)
Next, a second embodiment will be described. In the recognition apparatus 100 of the second embodiment, the peripheral area used for the determination is different from that of the first embodiment in order to prevent erroneous recognition of the area indicating the lighted signal. The determination unit 25 of the second embodiment uses a peripheral region including a region indicating a signal that is not lit for determination. In the description of the second embodiment, the description similar to the description of the first embodiment is omitted, and different portions from the first embodiment will be described.

  The configuration of the recognition apparatus 100 according to the second embodiment is the same as the configuration example (see FIG. 2) of the recognition apparatus 100 according to the first embodiment. That is, the recognition apparatus 100 according to the second embodiment includes an image acquisition unit 10, a position acquisition unit 11, and a signal processing unit 20. The signal processing unit 20 includes an interface unit 21, a selection unit 22, a storage unit 23, a recognition unit 24, a determination unit 25, and an output unit 26.

  The description of the image acquisition unit 10, the position acquisition unit 11, the interface unit 21, the selection unit 22, the storage unit 23, and the output unit 26 is the same as the description of the first embodiment, and will be omitted.

  The recognition unit 24 receives image information from the interface unit 21 and receives recognition information from the selection unit 22. The recognition information includes color information for determining a signal pixel indicating a lighted signal, and feature information of the surrounding area. The peripheral area of the second embodiment includes an area indicating a signal that is not lit.

  FIG. 16 is a diagram illustrating an example of the peripheral regions 502 and 504 of the green signal shape region 104 according to the second embodiment. The example of FIG. 16 shows a case where the signal arrangement is a horizontal traffic light 300a. The green signal shape area 104 indicates a green signal that is lit. The peripheral area 502 includes an area 501 showing a yellow signal that is not lit. The peripheral area 504 includes an area 503 showing a red signal that is not lit.

  FIG. 17 is a diagram illustrating an example of the peripheral regions 502 and 516 of the red signal shape region 511 according to the second embodiment. The example of FIG. 17 shows the case where the signal arrangement is a horizontal traffic light 300a. A red signal shape region 511 indicates a red signal that is lit. The peripheral area 502 includes an area 501 showing a yellow signal that is not lit. The peripheral area 516 includes an area 515 indicating a green light that is not lit.

  FIG. 18 is a diagram illustrating an example of the peripheral regions 524 and 526 of the green signal shape region 521 of the second embodiment. The example of FIG. 18 shows a case where the signal arrangement is a vertical traffic light 300b. A green signal shape area 521 indicates a green signal that is lit. The peripheral area 524 includes an area 523 showing a yellow signal that is not lit. The peripheral region 526 includes a region 525 that shows a red signal that is not lit.

  FIG. 19 is a diagram illustrating an example of the peripheral regions 524 and 536 of the red signal shape region 531 according to the second embodiment. The example of FIG. 19 shows a case where the signal arrangement is a vertical traffic light 300b. A red signal shape area 531 indicates a red signal that is lit. The peripheral area 524 includes an area 523 showing a yellow signal that is not lit. The peripheral area 536 includes an area 535 showing a green light that is not lit.

  Hereinafter, the operation of the recognition unit 24 and the determination unit 25 of the second embodiment will be described by taking the case of FIG. 16 (when the green light of the traffic light 300a is recognized) as an example.

  The recognition unit 24 recognizes the green signal shape region 104 and the peripheral regions 502 and 504 based on the recognition information. Since the method for recognizing the green signal shape region 104 is the same as that in the first embodiment, the description thereof is omitted. After recognizing the green signal shape region 104, the recognition unit 24 determines the positions of the peripheral regions 502 and 504 based on the relative position with respect to the position of the green signal shape region 104. The relative position varies depending on the type of the traffic light 300 and the like, and thus is determined based on the feature information included in the recognition information selected by the selection unit 22.

  Here, an example of feature information indicating the features of the peripheral regions 502 and 504 of the green signal shape region 104 will be described.

  Table 2 shows an example of the feature information of the peripheral areas 502 and 504 of the green signal shape area 104 of the traffic light 300a.

  The feature information includes a position, a size, and an average luminance range for each peripheral region. For example, the position of the peripheral region 502 is (X + x4, Y + y4). Here, (X, Y) is a coordinate indicating the center of the green signal shape region 104. The size of the peripheral area 502 is (W4, H4). The average luminance range associated with the peripheral area 502 is (Imin4, Imax4). This average luminance range indicates the average luminance range of the region 501 indicating the non-lighted yellow signal included in the peripheral region 502.

  FIG. 20 is a diagram for explaining an example of a method for determining the position and size of the peripheral region 502 of the green signal shape region 104 according to the second embodiment. First, the recognition unit 24 recognizes the green signal shape region 104 by the method described in the first embodiment. Next, the recognizing unit 24 determines the coordinates (X + x4, Y + y4) of the point P, which is relatively separated from the center coordinates O (X, Y) of the green signal shape region 104 by x4 in the x direction and y4 in the y direction. The position of the area 502 is determined. Next, the recognizing unit 24 determines the size of the peripheral region 502 based on the width W4 and the height H4.

  The recognition unit 24 determines the position and size of the peripheral area 504 based on the green signal shape area 104 and the recognition information in the same manner as the peripheral area 502.

  Based on the average luminance range indicated by the feature information included in the recognition information, the determination unit 25 determines whether or not the green signal shape region 104 is a signal region indicating the green signal of the traffic light 300a. Specifically, first, the determination unit 25 extracts a circular area 501 by Hough transformation in the peripheral area 502 and extracts a circular area 503 by Hough transformation in the peripheral area 504. Next, the determination unit 25 calculates the average brightness of the circular area 501 and the average brightness of the circular area 503. Next, when the average luminance of the circular region 501 is not less than Imin4 and not more than Imax4 and the average luminance of the circular region 503 is not less than Imin5 and not more than Imax5, the determination unit 25 determines that the green signal shape region 104 is the traffic light 300a. The green signal area 101 is determined.

  Note that the determination unit 25 may determine whether the green signal shape region 104 is the green signal region 101 of the traffic light 300a by determining one of the peripheral region 502 and the peripheral region 504.

  As described above, in the recognition apparatus 100 according to the second embodiment, the determination unit 25 determines whether the signal shape region is a signal region by using a peripheral region including a region indicating a signal that is not lit. judge.

  Thereby, according to the recognition apparatus 100 of 2nd Embodiment, the effect similar to the recognition apparatus 100 of 1st Embodiment is acquired. For example, an image area in which a part of a red sign with the same color as the red signal is hidden behind a tree branch and the part where the red sign is visible through the branch is circular is mistaken as a red signal area. It can be prevented from being recognized.

DESCRIPTION OF SYMBOLS 10 Image acquisition part 11 Position acquisition part 20 Signal processing part 21 Interface part 22 Selection part 23 Storage part 24 Recognition part 25 Judgment part 26 Output part 100 Recognition apparatus 101 Green signal area 102 Green signal pixel area 103 Blue signal pixel expansion area 104 Blue signal shape Area 105 Rectangular area 106 Peripheral area 107 Area included in the peripheral area 200 Vehicle 201 Imaging optical system 202 Mechanical shutter 203 Motor driver 204 CCD
205 CDS circuit 206 A / D converter 207 Timing signal generator 208 Image processing circuit 209 LCD
210 CPU
211 RAM
212 ROM
213 SDRAM
214 Compression / Expansion Circuit 215 Memory 216 Operation Unit 217 Output I / F
220 Bus 300 Traffic light 401 Blue signal area 501 Area indicating a yellow signal that is not lit 502 Peripheral area 503 Area indicating a red signal that is not lit 504 Peripheral area 511 Area indicating a red signal that is lit 515 Blue signal that is not lit Area 516 Peripheral area 521 Area indicating a green signal that is lit 523 Area indicating a yellow signal that is not lit 524 Area 525 Area indicating a red signal that is not lit 526 Area surrounding the red signal 526 Area that indicates a red signal that is lit 535 Area showing green light that is not lit 536 Peripheral area

JP 2009-244946 A

Claims (8)

An image acquisition unit for acquiring image information around the vehicle;
A position acquisition unit for acquiring position information indicating the position of the vehicle;
A selection unit for selecting recognition information for recognizing a traffic signal in a region identified from the position information;
A recognition unit for recognizing a signal shape region indicating the shape of the signal of the traffic light from the image information based on the recognition information;
A determination unit that determines whether or not the signal shape region is a signal region indicating a signal of the traffic light based on the recognition information;
A recognition device comprising: The recognition information includes color information indicating the color of the signal of the traffic light,
The recognizing unit recognizes a signal pixel indicating the color of the signal of the traffic light from the image information based on the color information, and covers the signal pixel with a plurality of pixels, thereby defining the signal pixel region as a signal pixel. A pixel region obtained by expanding the expansion region of the signal pixel and performing a Hough transform on the expansion region of the signal pixel is recognized as the signal shape region.
The recognition device according to claim 1. The color information is color information in a (Y, U, V) color space,
The image acquisition unit acquires the image information of (R, G, B) color space,
The recognition unit converts the image information of the (R, G, B) color space into image information of the (Y, U, V) color space, and converts the image information to the color information of the (Y, U, V) color space. Recognizing the signal pixel included in the image information of the (Y, U, V) color space based on
The recognition apparatus according to claim 2. The recognition information includes feature information indicating features of a peripheral region of the signal region,
The determination unit determines that the signal shape region is the signal region when the feature information is similar to the feature information of the peripheral region of the signal shape region.
The recognition device according to claim 1. The feature information includes information indicating a range of average luminance in a peripheral region of the signal region,
The determination unit determines that the signal shape region is the signal region when the average luminance of the peripheral region of the signal shape region is included in the average luminance range of the peripheral region of the signal region;
The recognition apparatus according to claim 4. The peripheral area includes an area indicating a signal that is not lit.
The recognition apparatus according to claim 4 or 5. A recognition method of a recognition device comprising: an image acquisition unit that acquires image information around a vehicle; and a position acquisition unit that acquires position information indicating the position of the vehicle,
A recognition device selecting recognition information for recognizing a traffic signal in a region identified from the position information;
A step of recognizing a signal shape region indicating a signal shape of the traffic light from the image information based on the recognition information;
A recognition device determining whether the signal shape region is a signal region indicating a signal of the traffic light based on the recognition information;
A recognition method including: A recognition apparatus comprising: an image acquisition unit that acquires image information around a vehicle; and a position acquisition unit that acquires position information indicating the position of the vehicle.
A selection unit for selecting recognition information for recognizing a traffic signal in a region identified from the position information;
A recognition unit for recognizing a signal shape region indicating the shape of the signal of the traffic light from the image information based on the recognition information;
A determination unit that determines whether the signal shape region is a signal region indicating a signal of the traffic light based on the recognition information;
Program to function as.