JP2020064681A - Recognition device, recognition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a recognition device, a recognition method and a program capable of more accurately recognizing a signal area of a traffic light from image information.
A recognition device according to an embodiment includes an image acquisition unit that acquires image information around a vehicle, a position acquisition unit that acquires position information indicating a position of the vehicle, and a traffic light in an area specified by the position information. A selection unit that selects recognition information for recognition, a recognition unit that recognizes a signal shape region indicating the shape of a signal of a traffic signal from image information based on the recognition information, and a signal shape region is a signal of the traffic signal based on the recognition information. And a determination unit that determines whether or not the signal area is
[Selection diagram] Figure 2

Description

  The present invention relates to a recognition device, a recognition method and a program.

  2. Description of the Related Art Techniques for assisting a driver and storing an image at the time of an accident using an in-vehicle camera have been known. For example, driver assist technology using an in-vehicle camera includes an automatic braking function for avoiding an obstacle or reducing an impact at the time of a collision, and an alarm function for warning a distance between a preceding vehicle and the like.

  Further, for example, in Patent Document 1, an invention in which a signal color of a traffic signal and a signal shape are separately recognized from two images in which the same subject is photographed, so that the signal of the traffic signal is accurately recognized from the image. Is disclosed.

  However, in the related art, there is a possibility that an object having a color and shape similar to the signal area of the traffic signal may be erroneously recognized as the signal area of the traffic signal.

  The present invention has been made in view of the above, and an object of the present invention is to provide a recognition device, a recognition method, and a program capable of more accurately recognizing a signal area of a traffic light from image information.

  In order to solve the above-mentioned problems and achieve the object, the present invention provides an image acquisition unit that acquires image information of the surroundings of a vehicle, a position acquisition unit that acquires position information indicating the position of the vehicle, and the position. A selection unit for selecting recognition information for recognizing a traffic light in an area specified from the information; a recognition unit for recognizing a signal shape region indicating the shape of the signal of the traffic light from the image information based on the recognition information; A determination unit that determines whether the signal shape region is a signal region indicating a signal of the traffic light based on the recognition information.

  According to the present invention, it is possible to more accurately recognize the signal area of the traffic signal from the image information.

FIG. 1 is a diagram showing an example of a vehicle equipped with the recognition device of the first embodiment. FIG. 2 is a diagram showing an example of the configuration of the recognition device according to the first embodiment. FIG. 3 is a diagram showing Example 1 of image information including a traffic light. FIG. 4 is a diagram illustrating a second example of image information including a traffic light. FIG. 5 is a diagram showing an example of the (U, V) distribution of red signal pixels. FIG. 6 is a diagram showing an example of the (U, V) distribution of blue signal pixels. FIG. 7 is a diagram showing an example of the (U, V) distribution of yellow signal pixels. FIG. 8 is a diagram showing an example of a blue signal pixel region according to the first embodiment. FIG. 9 is a diagram showing an example of the expansion region of the green signal pixel according to the first embodiment. FIG. 10 is a diagram showing an example of the green signal shape region of the first embodiment. FIG. 11 is a diagram showing an example of the rectangular area of the first embodiment. FIG. 12 is a diagram showing an example of the peripheral region of the first embodiment. FIG. 13 is a diagram showing an example of a green signal recognition result of the first embodiment. FIG. 14 is a flowchart showing an example of the recognition method of the first embodiment. FIG. 15 is a diagram illustrating an example of the hardware configuration of the image acquisition unit of the first embodiment. FIG. 16 is a diagram showing an example of a peripheral region of the green signal shape region of the second embodiment. FIG. 17 is a diagram showing an example of a peripheral area of the red traffic light shape area of the second embodiment. FIG. 18 is a diagram showing an example of a peripheral region of the green signal shape region of the second embodiment. FIG. 19 is a diagram showing an example of a peripheral area of the red traffic light shape area of the second embodiment. FIG. 20 is a diagram for explaining an example of the method of determining the position and size of the peripheral area of the green signal shape area according to the second embodiment.

  Hereinafter, embodiments of a recognition device, a recognition method, and a program will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram showing an example of a vehicle 200 equipped with the recognition device 100 of the first embodiment. The recognition device 100 of the first embodiment is installed near the rearview mirror of the windshield of the vehicle 200. The recognition apparatus 100 according to the first embodiment recognizes a signal area indicating a signal of the traffic signal 300 from image information by image recognition processing.

  FIG. 2 is a diagram showing an example of the configuration of the recognition device 100 according to the first embodiment. The recognition device 100 according to the first embodiment includes an image acquisition unit 10, a position acquisition unit 11, and a signal processing unit 20. The signal processing unit 20 includes an interface unit 21, a selection unit 22, a storage unit 23, a recognition unit 24, a determination unit 25, and an output unit 26.

  The image acquisition unit 10 acquires image information of a color represented by the (R, G, B) color space. The image acquisition unit 10 inputs the image information to the interface unit 21. The image acquisition unit 10 is, for example, a camera that captures an image around the vehicle 200.

  The position acquisition unit 11 acquires position information indicating the position of the vehicle 200 and inputs the position information into the interface unit 21. The position information is, for example, GPS (Global Positioning System) information, coordinate information of a map indicating the position of the vehicle 200, and the like.

  When the interface unit 21 receives the image information from the image acquisition unit 10, the interface unit 21 converts the image information into an information format that the recognition unit 24 can accept as a time-series continuous image frame. The interface unit 21 inputs the image information whose information format has been converted to the recognition unit 24.

  When the interface unit 21 receives the position information from the position acquisition unit 11, the interface unit 21 inputs the position information to the selection unit 22.

  Upon receiving the position information from the interface unit 21, the selection unit 22 specifies the area from the position information. Then, the selection unit 22 selects recognition information for recognizing the traffic signal 300 in the specified area from one or more pieces of recognition information stored in the storage unit 23. The selection unit 22 inputs the recognition information to the recognition unit 24 and the determination unit 25.

  The storage unit 23 stores information. The storage unit 23 stores, for example, the recognition information for each area. The shape of the signal and the color of the signal of the traffic signal 300 vary depending on the region. For example, the shape of the traffic light 300 may be horizontal or vertical depending on the area.

  FIG. 3 is a diagram showing Example 1 of image information including the traffic light 300a. The example of FIG. 3 shows a traffic light 300a when the signal array is horizontal. The green signal area 101 is an area showing a green signal. For example, in Japan, a horizontal traffic light 300a is common.

  FIG. 4 is a diagram illustrating a second example of image information including the traffic light 300b. The example of FIG. 4 shows the traffic signal 300b in the case where the signal arrangement is vertical. The green signal area 401 is an area showing a green signal. For example, in the United States, a vertical traffic light 300b is common. Even in Japan, a vertical traffic light 300b is installed in an area with a large amount of snow.

  The recognition information is information for recognizing the traffic signal 300 in the area specified by the position information. The recognition information includes color information for determining the signal pixel and characteristic information of the peripheral area of the signal shape area. The signal shape area is an area indicating the shape of the signal of the traffic signal 300. A description of the characteristic information of the peripheral area of the signal shape area will be given later with reference to Table 1 and FIG.

  The color information for determining a signal pixel includes, for example, a threshold for determining a red signal pixel, a threshold for determining a blue signal pixel, and a threshold for determining a yellow signal pixel.

  The red signal pixel is a pixel indicating the color of the red signal. The threshold values for determining the red signal pixels are the U threshold value (Ur-min and Ur-max) and the V threshold value (Vr-min and Vr-max). This threshold is determined based on the (U, V) distribution of red signal pixels.

  FIG. 5 is a diagram showing an example of the (U, V) distribution of red signal pixels. The example of FIG. 5 shows the (U, V) distribution of red signal pixels obtained from the pixel information of the (R, G, B) color space of a plurality of image samples in which the red signal is captured. The specific values of the U threshold (Ur-min and Ur-max) and the V threshold (Vr-min and Vr-max) may be arbitrarily determined. However, if the range of the threshold value is too wide, the possibility that a pixel that is not a red signal color is detected increases, so the threshold value is set so as not to overlap the (U, V) value of the pixel that is not a red signal.

  The blue signal pixel is a pixel indicating the color of the blue signal. The threshold value for determining the green signal pixel is the U threshold value (Ub-min and Ub-max) and the V threshold value (Vb-min and Vb-max). This threshold is determined based on the (U, V) distribution of the green signal pixels.

  FIG. 6 is a diagram showing an example of the (U, V) distribution of blue signal pixels. The example of FIG. 6 shows the (U, V) distribution of the blue signal pixels obtained from the pixel information of the (R, G, B) color space of the plurality of image samples in which the blue signal is captured. Note that specific values of the U threshold (Ub-min and Ub-max) and the V threshold (Vb-min and Vb-max) may be arbitrarily determined. However, if the range of the threshold value is too wide, the possibility that a pixel that is not a blue signal color is detected increases, so the threshold value is set so as not to overlap the (U, V) value of the pixel that is not a blue signal.

  The yellow signal pixel is a pixel indicating the color of the yellow signal. The threshold values for determining the yellow signal pixel are the U threshold value (Uy-min and Uy-max) and the V threshold value (Vy-min and Vy-max). This threshold is determined based on the (U, V) distribution of the yellow signal pixels.

  FIG. 7 is a diagram showing an example of the (U, V) distribution of yellow signal pixels. The example of FIG. 7 shows the (U, V) distribution of the yellow signal pixels obtained from the pixel information of the (R, G, B) color space of the plurality of image samples in which the yellow signal is captured. The specific values of the U threshold (Uy-min and Uy-max) and the V threshold (Vy-min and Vy-max) may be arbitrarily determined. However, if the range of the threshold value is too wide, there is a high possibility that a pixel that is not the color of the yellow signal is detected. Therefore, the threshold value is set so that it does not overlap the (U, V) value of the pixel that is not the yellow signal.

  In the following description, a case will be specifically described where the selection unit 22 selects the recognition information for recognizing the signal of the traffic signal 300a (see FIG. 3).

  Returning to FIG. 2, the recognition unit 24 receives the image information from the interface unit 21 and the recognition information from the selection unit 22. The recognition unit 24 recognizes the signal shape region based on the recognition information. Specifically, first, the recognition unit 24 converts the image information in the (R, G, B) color space into the image information in the (Y, U, V) color space using the following expression (1). .

  Next, when the (U, V) value of the image information of the (Y, U, V) color space is included in the range of the threshold value for determining the red signal pixel included in the recognition information, the recognition unit 24 applies the (U, V) value. V) Pixels having a value are recognized as red signal pixels. Similarly, when the (U, V) value of the image information in the (Y, U, V) color space is included in the range of the threshold value for determining the green signal pixel included in the recognition information, the recognition unit 24 applies the (U, V) value. V) Pixels having a value are recognized as green signal pixels. Further, when the (U, V) value of the image information in the (Y, U, V) color space is included in the range of the threshold value for determining the yellow signal pixel included in the recognition information, the recognition unit 24 applies the (U, V) value. ) A pixel having a value is recognized as a yellow signal pixel.

  Hereinafter, a case where the green signal pixel is recognized by the recognition unit 24 will be specifically described as an example. The case of the red signal pixel and the yellow signal pixel is similar to that of the blue signal pixel.

  FIG. 8 is a diagram showing an example of the area 102 of the green signal pixel of the first embodiment. The size of the green signal area 102 of FIG. 8 is smaller than the actual size of the green signal area 101 (see FIG. 3). That is, the example of FIG. 8 shows a case where a region that should originally be recognized as a blue signal region is not recognized as a blue signal pixel due to the influence of noise pixels. The noise pixels are noise pixels due to the surrounding conditions at the time of shooting, noise pixels due to the characteristics of the image sensor, noise pixels due to dust adhering to the surface of the image sensor, and the like. The noise pixel due to the surrounding state at the time of shooting is a pixel in a blue signal region that was reflected by light such as sunlight when the traffic light 300 was shot by the image acquisition unit 10. The noise pixel caused by the characteristics of the image sensor is, for example, a pixel affected by random noise.

  Returning to FIG. 2, the recognition unit 24 then performs expansion processing for expanding the area 102 of the blue signal pixel. Specifically, the recognition unit 24 expands the area 102 of the blue signal pixel into the expanded area of the blue signal pixel by covering each pixel of the area 102 of the blue signal pixel with a plurality of pixels. The recognition unit 24 covers each pixel with, for example, n × n (n is an integer of 1 or more) block pixels. For example, in the case of n = 7, each pixel in the area 102 of the green signal pixel is expanded to the expanded area of the green signal pixel which further includes 48 (7 × 7-1) pixels around the pixel.

  FIG. 9 is a diagram showing an example of the expanded area 103 of the green signal pixel of the first embodiment. In the example of FIG. 9, the recognition unit 24 performs expansion processing on the area 102 of the blue signal pixels recognized by the recognition unit 24, so that the expansion of the blue signal pixels including the blue signal shape area 104 showing the original shape of the blue signal. The case where the area 103 is obtained is shown.

  Returning to FIG. 2, next, the recognition unit 24 performs shape recognition processing for recognizing the shape of the green signal shape region 104 indicating the shape of the green signal. Specifically, the recognizing unit 24 determines whether or not a circular pixel area can be recognized in the expanded area 103 of the blue signal pixel by Hough transforming the area 102 of the blue signal pixel. When the circular pixel area can be recognized, the recognition unit 24 recognizes that the circular pixel area is the blue signal shape area 104 indicating the shape of the green signal of the traffic light 300.

  FIG. 10 is a diagram showing an example of the green signal shape region 104 of the first embodiment. The example in FIG. 10 shows an example in which the blue signal shape region 104 is recognized as a circular pixel region by Hough transforming the region 102 (see FIG. 8) of the blue signal pixel.

  Returning to FIG. 2, next, the recognition unit 24 recognizes a rectangular area circumscribing the green signal shape area 104.

  FIG. 11 is a diagram showing an example of the rectangular area 105 of the first embodiment. The example of FIG. 11 shows a case where the recognition unit 24 recognizes a rectangular area 105 circumscribing the blue signal shape area 104 recognized as a circular pixel area.

  Returning to FIG. 2, the recognition unit 24 then inputs information indicating the rectangular area 105 to the determination unit 25.

  The determination unit 25 receives the recognition information from the selection unit 22 and the information indicating the rectangular area 105 from the recognition unit 24. The determination unit 25 determines whether or not the green signal shape area 104 inscribed in the rectangular area 105 is the green signal area 101 of the traffic light 300a based on the recognition information. Specifically, when the characteristic information of the peripheral area of the green signal shape area 104 and the characteristic information of the peripheral area of the green signal area 101 of the traffic signal 300a included in the recognition information are similar, the determination unit 25 determines the green signal shape. The area 104 is determined to be the green signal area 101 of the traffic light 300a.

  Here, an example of the characteristic information indicating the characteristics of the peripheral area of the signal area will be described.

  Table 1 shows an example of the characteristic information of the traffic light 300a.

  The characteristic information includes the position, size, and average brightness range for each signal type. For example, the position of the peripheral area of the green signal area 101 is (X-x1, Y-y1). The size of the peripheral area of the green signal area 101 is (W1, H1). Here, the position and size of the peripheral area will be described.

  FIG. 12 is a diagram showing an example of the peripheral region of the first embodiment. FIG. 12 shows a case where the characteristic information of the peripheral area 106 of the green signal shape area 104 is compared with the characteristic information of the peripheral area of the green signal area 101 of the traffic light 300a. The coordinates of the point O are (X, Y). A point O indicates the center of a circular pixel area (blue signal shape area 104) inscribed in the rectangular area 105. The coordinates of the point P are (X-x1, Y-y1). The peripheral region 106 is determined by the point P, the width W1 and the height H1.

  Further, for example, the average luminance range of the peripheral area of the green signal area 101 is (Imin1, Imax1). The average brightness range is determined, for example, based on the color of the housing of the traffic light 300a. The determination unit 25 calculates the average luminance of the region 107 included in the peripheral region 106, and when the average luminance is Imin1 or more and Imax1 or less, determines that the green signal shape region 104 is the green signal region 101 of the traffic signal 300a. .

  As a result, for example, an image area where a part of the red sign that is the same as the red traffic light is hidden by the branches and leaves of the tree and the part where the red sign is visible through the branches is circular is the red traffic light area. Can be prevented from being erroneously recognized as.

  FIG. 13 is a diagram showing an example of a green signal recognition result of the first embodiment. The example of FIG. 13 shows a case where the green signal of the traffic light 300 a included in the image information shown in FIG. 3 is recognized as the rectangular area 105 including the green signal area 101.

  Returning to FIG. 2, the determination unit 25 inputs the recognition result information indicating the rectangular area 105 including the green signal area 101 to the output unit 26. When receiving the recognition result information from the determination unit 25, the output unit 26 outputs the recognition result information. The recognition result information is used, for example, for processing such as detection of dangerous driving such as ignoring signals by the driver and sudden start.

  Next, the recognition method of the first embodiment will be described.

  FIG. 14 is a flowchart showing an example of the recognition method of the first embodiment. Since the description is the same when the signal is a blue signal, a yellow signal, and a red signal, in the description of the recognition method of the first embodiment, the case where the signal is a blue signal will be described as an example. A case where the green signal (see FIG. 3) of the traffic light 300a is recognized will be specifically described as an example.

  First, the image acquisition unit 10 captures an image of the surroundings of the vehicle 200 and acquires color image information represented by the (R, G, B) color space (step S1). Next, the recognition unit 24 uses the image information of the color expressed in the (R, G, B) color space to calculate the image information of the (Y, U, V) color space by using the above equation (1). (Step S2).

  The position acquisition unit 11 also acquires position information indicating the position of the vehicle 200 (step S3). Next, the selection unit 22 selects recognition information for recognizing the traffic signal 300a in the area specified by the position information from one or more pieces of recognition information stored in the storage unit 23 (step S4).

  Next, when the (U, V) value of the image information of the (Y, U, V) color space is included in the range of the threshold value for determining the green signal pixel included in the recognition information, the recognition unit 24 determines that (U , V) values are recognized as blue signal pixels (step S5). Next, the recognition unit 24 expands the area 102 of the green signal pixel recognized in step S5 (step S6).

  Next, the recognizing unit 24 determines whether or not the blue signal shape area 104 indicating the shape of the blue signal can be recognized from the expanded area 103 of the blue signal pixel obtained by the processing of step S6 (step S7). Specifically, the recognition unit 24 determines whether or not a circular pixel area (blue signal shape area 104) can be recognized in the expanded area 103 of the blue signal pixel by Hough transforming the area 102 of the blue signal pixel.

  If the green signal shape region 104 can be recognized (Yes in step S7), the recognition process proceeds to step S8. If the green signal shape region 104 cannot be recognized (No in step S7), the recognition process ends.

  Next, the determination unit 25 determines whether the characteristic information of the peripheral area of the green signal shape area 104 is similar to the characteristic information of the peripheral area of the green signal area 101 of the traffic signal 300a included in the recognition information selected in step S4. It is determined (step S8).

  When they are similar (Yes in step S8), the determination unit 25 determines that the green signal shape region 104 recognized in step S7 is the green signal region 101 (step S9). Next, the output unit 26 outputs the recognition result information indicating the rectangular area 105 including the green signal area 101 (step S10). If they are not similar (step S8, No), the recognition process ends.

  Next, an example of the hardware configuration of the recognition device 100 of the first embodiment will be described.

  First, an example of the hardware configuration of the image acquisition unit 10 will be described.

  FIG. 15 is a diagram showing an example of the hardware configuration of the image acquisition unit 10 of the first embodiment. The image acquisition unit 10 of the first embodiment includes a photographing optical system 201, a mechanical shutter 202, a motor driver 203, a CCD (Charge Coupled Device) 204, a CDS (Correlated Double Sampling) circuit 205, and an A / D converter. 206, a timing signal generator 207, an image processing circuit 208, an LCD (Liquid Crystal Display) 209, a CPU (Central Processing Unit) 210, a RAM (Random Access Memory) 211, a ROM (Read Only Memory) 212, SDRAM (Synchronous SDRAM). Access Memory) 213, compression / expansion circuit 214, memo 215, an operation unit 216 and an output I / F 217.

  The image processing circuit 208, the CPU 210, the RAM 211, the ROM 212, the SDRAM 213, the compression / expansion circuit 214, the memory 215, the operation unit 216, and the output I / F 217 are connected via the bus 220.

  The photographing optical system 201 collects the light reflected by the subject. The mechanical shutter 202 is opened for a predetermined time to make the light condensed by the photographing optical system 201 incident on the CCD 204. The motor driver 203 drives the photographing optical system 201 and the mechanical shutter 202.

  The CCD 204 forms the light incident through the mechanical shutter 202 as an image of a subject and inputs analog image information indicating the image of the subject to the CDS circuit 205. Upon receiving the analog image information from the CCD 204, the CDS circuit 205 removes the noise component of the image information and inputs the analog image information from which the noise component is removed to the A / D converter 206. Upon receiving analog image information from the CDS circuit 205, the A / D converter 206 converts the analog image information into digital image information. The A / D converter 206 inputs digital image information to the image processing circuit 208. The timing signal generator 207 transmits the timing signal to the CCD 204, the CDS circuit 205, and the A / D converter 206 in response to the control signal from the CPU 210, so that the CCD 204, the CDS circuit 205, and the A / D converter 206 receive the timing signal. Control when it works.

  Upon receiving the digital image information from the A / D converter 206, the image processing circuit 208 uses the SDRAM 213 to perform image processing of the digital image information. The image processing is, for example, CrCb conversion processing, white balance control processing, contrast correction processing, edge enhancement processing, and color conversion processing. The white balance control process is an image process for adjusting the color density of image information. The contrast correction process is an image process for adjusting the contrast of image information. The edge enhancement process is a process of adjusting the sharpness of image information. The color conversion process is an image process for adjusting the hue of image information.

  The image processing circuit 208 inputs the image information subjected to the above-mentioned image processing to the LCD 209 or the compression / expansion circuit 214. The LCD 209 is a liquid crystal display that displays the image information received from the image processing circuit 208.

  The CPU 210 controls the operation of the image acquisition unit 10 by executing the program. The RAM 211 is a work area when the CPU 210 executes a program and a readable / writable storage area used for storing various information. The ROM 212 is a read-only storage area that stores programs executed by the CPU 210.

  The SDRAM 213 is a storage area that temporarily stores image information to be image-processed when the image processing circuit 208 performs image processing.

  Upon receiving the image information from the image processing circuit 208, the compression / expansion circuit 214 compresses the image information. The compression / expansion circuit 214 stores the compressed image information in the memory 215. Further, when receiving the image information from the memory 215, the compression / expansion circuit 214 expands the image information. The compression / expansion circuit 214 temporarily stores the expanded image information in the SDRAM 213. The memory 215 stores the compressed image information.

  The operation unit 216 receives an operation from the user of the image acquisition unit 10. The operation unit 216 receives an operation of storing the image information displayed on the LCD 209 in the memory 215, for example. The output I / F 217 is an interface for transmitting image information from the image acquisition unit 10 to the signal processing unit 20.

  The interface unit 21, the selection unit 22, the recognition unit 24, the determination unit 25, and the output unit 26 of the signal processing unit 20 described above with reference to FIG. 2 may be realized by hardware as a signal processing board (signal processing circuit). It may be realized by software (program) executed by the CPU 210 of the image acquisition unit 10, the CPU of another device, or the like.

  The position acquisition unit 11 is realized by, for example, a GPS device. The storage unit 23 is realized by, for example, a memory on the signal processing board, the memory 215 of the image acquisition unit 10, and the like.

  The program executed by the recognition device 100 (CPU 210) of the first embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a memory card, a CD-R, and a DVD (Digital Versatile Disk). It is recorded on a computer-readable storage medium and provided as a computer program product.

  The program executed by the recognition device 100 of the first embodiment may be stored in a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the program executed by the recognition apparatus 100 of the first embodiment may be provided via a network such as the Internet without being downloaded.

  Further, the program of the recognition device 100 of the first embodiment may be incorporated in the ROM 212 or the like in advance and provided.

  When the selection unit 22, the recognition unit 24, the determination unit 25, the output unit 26, and the like are realized by the program executed by the recognition device 100 according to the first embodiment, the selection unit 22, the recognition unit 24, the determination unit 25, and the output. The unit 26 and the like are realized in the RAM 211 when the CPU 210 reads out and executes the program from the ROM 212 or the storage medium or the like.

  As described above, in the recognition device 100 of the first embodiment, the selection unit 22 selects the recognition information for recognizing the traffic signal 300 in the area specified by the position information. The recognition unit 24 recognizes a signal shape region indicating the shape of the signal of the traffic signal 300 from the image information based on the recognition information. Then, the determination unit 25 determines whether or not the signal shape region is a signal region indicating the signal of the traffic signal 300, based on the recognition information. Accordingly, the signal area of the traffic signal 300 can be recognized more accurately from the image information.

(Second embodiment)
Next, a second embodiment will be described. In the recognition device 100 of the second embodiment, the peripheral area used for determination in order to prevent erroneous recognition of an area indicating a lit signal is different from that of the first embodiment. The determination unit 25 of the second embodiment uses the peripheral area including the area indicating the signal that is not lit for the determination. In the description of the second embodiment, the same description as that of the first embodiment will be omitted, and only different points from the first embodiment will be described.

  The configuration of the recognition device 100 according to the second embodiment is the same as the example (see FIG. 2) of the configuration of the recognition device 100 according to the first embodiment. That is, the recognition device 100 of the second embodiment includes the image acquisition unit 10, the position acquisition unit 11, and the signal processing unit 20. The signal processing unit 20 includes an interface unit 21, a selection unit 22, a storage unit 23, a recognition unit 24, a determination unit 25, and an output unit 26.

  The description of the image acquisition unit 10, the position acquisition unit 11, the interface unit 21, the selection unit 22, the storage unit 23, and the output unit 26 is the same as the description of the first embodiment, and will be omitted.

  The recognition unit 24 receives the image information from the interface unit 21 and the recognition information from the selection unit 22. The recognition information includes color information for determining a signal pixel indicating a lighted signal and characteristic information of the peripheral area. The peripheral area of the second embodiment includes an area showing a signal that is not lit.

  FIG. 16 is a diagram showing an example of peripheral regions 502 and 504 of the green signal shape region 104 of the second embodiment. The example of FIG. 16 shows a case where the signal arrangement is a horizontal traffic light 300a. The green signal shape area 104 indicates a green signal that is lit. The peripheral area 502 includes an area 501 showing a yellow signal that is not lit. The peripheral area 504 includes an area 503 that indicates a red signal that is not lit.

  FIG. 17 is a diagram showing an example of the peripheral areas 502 and 516 of the red traffic light shape area 511 of the second embodiment. The example of FIG. 17 shows a case where the signal arrangement is a horizontal traffic light 300a. The red traffic light shape area 511 indicates a red traffic light that is lit. The peripheral area 502 includes an area 501 showing a yellow signal that is not lit. The peripheral area 516 includes an area 515 that indicates a green signal that is not lit.

  FIG. 18 is a diagram showing an example of peripheral areas 524 and 526 of the green signal shape area 521 of the second embodiment. The example of FIG. 18 shows a case where the signal arrangement is a vertical traffic light 300b. The green signal shape area 521 indicates a green signal that is lit. The peripheral area 524 includes an area 523 indicating a yellow signal that is not lit. The peripheral area 526 includes an area 525 indicating a red signal that is not illuminated.

  FIG. 19 is a diagram showing an example of peripheral areas 524 and 536 of the red traffic light shape area 531 of the second embodiment. The example of FIG. 19 shows the case where the signal arrangement is a vertical traffic light 300b. The red traffic light shape area 531 shows a red traffic light that is lit. The peripheral area 524 includes an area 523 indicating a yellow signal that is not lit. The peripheral area 536 includes an area 535 that indicates an unlit green signal.

  Hereinafter, the operation of the recognition unit 24 and the determination unit 25 of the second embodiment will be described by taking the case of FIG. 16 (case of recognizing the green signal of the traffic light 300a) as an example.

  The recognition unit 24 recognizes the green signal shape region 104 and the peripheral regions 502 and 504 based on the recognition information. The method of recognizing the green signal shape region 104 is the same as that in the first embodiment, and thus the description thereof is omitted. After recognizing the green signal shape region 104, the recognition unit 24 determines the positions of the peripheral regions 502 and 504 based on the relative position to the position of the green signal shape region 104. Note that the relative position changes depending on the type of the traffic signal 300, and thus is determined based on the feature information included in the recognition information selected by the selection unit 22.

  Here, an example of the characteristic information indicating the characteristics of the peripheral areas 502 and 504 of the green signal shape area 104 will be described.

  Table 2 shows an example of the characteristic information of the peripheral areas 502 and 504 of the green signal shape area 104 of the traffic light 300a.

  The characteristic information includes the position, size, and average brightness range for each peripheral area. For example, the position of the peripheral area 502 is (X + x4, Y + y4). Here, (X, Y) are coordinates indicating the center of the green signal shape region 104. The size of the peripheral area 502 is (W4, H4). The average luminance range associated with the peripheral area 502 is (Imin4, Imax4). This average brightness range indicates the range of the average brightness of the area 501 that includes the yellow signal that is not illuminated and that is included in the peripheral area 502.

  FIG. 20 is a diagram for explaining an example of a method of determining the position and size of the peripheral area 502 of the green signal shape area 104 of the second embodiment. First, the recognition unit 24 recognizes the green signal shape region 104 by the method described in the first embodiment. Next, the recognition unit 24 determines the coordinates (X + x4, Y + y4) of the point P, which is relatively distant by x4 in the x direction and y4 in the y direction from the center coordinate O (X, Y) of the green signal shape region 104, in the vicinity. The position of the area 502 is determined. Next, the recognition unit 24 determines the size of the peripheral area 502 based on the width W4 and the height H4.

  The recognition unit 24 determines the position and size of the peripheral area 504 based on the green signal shape area 104 and the recognition information, similarly to the case of the peripheral area 502.

  The determination unit 25 determines whether or not the green signal shape region 104 is the signal region indicating the green signal of the traffic light 300a based on the average luminance range indicated by the feature information included in the recognition information. Specifically, first, the determination unit 25 extracts the circular area 501 by the Hough transform in the peripheral area 502, and extracts the circular area 503 by the Hough transform in the peripheral area 504. Next, the determination unit 25 calculates the average brightness of the circular area 501 and the average brightness of the circular area 503. Next, when the average luminance of the circular area 501 is Imin4 or more and Imax4 or less and the average luminance of the circular area 503 is Imin5 or more and Imax5 or less, the determination unit 25 sets the green signal shape area 104 to the traffic light 300a. It is determined to be the green signal area 101.

  Note that the determination unit 25 may determine whether the green signal shape area 104 is the green signal area 101 of the traffic signal 300a by determining one of the peripheral area 502 and the peripheral area 504.

  As described above, in the recognition device 100 of the second embodiment, the determination unit 25 uses the peripheral area including the area indicating the signal that is not lit to determine whether the signal shape area is the signal area. judge.

  As a result, according to the recognition device 100 of the second embodiment, the same effect as that of the recognition device 100 of the first embodiment can be obtained. For example, an image area in which a part of the red sign that is the same as the red traffic light is hidden by the branches and leaves of the tree and the part where the red sign is visible through the branches is a circle is mistaken as the red traffic light area. You can prevent it from being recognized.

10 image acquisition unit 11 position acquisition unit 20 signal processing unit 21 interface unit 22 selection unit 23 storage unit 24 recognition unit 25 determination unit 26 output unit 100 recognition device 101 green signal region 102 blue signal pixel region 103 blue signal pixel expansion region 104 blue signal shape Area 105 Rectangular Area 106 Peripheral Area 107 Area Included in Peripheral Area 200 Vehicle 201 Photographing Optical System 202 Mechanical Shutter 203 Motor Driver 204 CCD
205 CDS circuit 206 A / D converter 207 Timing signal generator 208 Image processing circuit 209 LCD
210 CPU
211 RAM
212 ROM
213 SDRAM
214 compression / expansion circuit 215 memory 216 operation unit 217 output I / F
220 Bus 300 Traffic light 401 Green signal area 501 Area where yellow signal is not lit 502 Peripheral area 503 Area where red signal is not lit 504 Peripheral area 511 Area where red signal is lit 515 Green signal is not lit Shown area 516 Peripheral area 521 Area where blue signal is lit 523 Area where yellow signal is not lit 524 Peripheral area 525 Area where red signal is not lit 526 Peripheral area 531 Area where red signal is lit 535 Area that shows unlit green signal 536 Peripheral area

JP, 2009-244946, A

Claims (8)

An image acquisition unit that acquires image information around the vehicle,
A position acquisition unit that acquires position information indicating the position of the vehicle,
A selection unit for selecting recognition information for recognizing a traffic signal in the area specified from the position information;
A recognition unit that recognizes a signal shape region indicating the shape of the signal of the traffic light from the image information based on the recognition information;
A determination unit that determines whether the signal shape region is a signal region indicating a signal of the traffic light based on the recognition information,
A recognition device including. The recognition information includes color information indicating the color of the signal of the traffic light,
The recognition unit recognizes a signal pixel indicating the color of the signal of the traffic light from the image information based on the color information, and covers the signal pixel with a plurality of pixels, thereby defining the area of the signal pixel as the signal pixel. A pixel area obtained by Huff transforming the expanded area of the signal pixel is recognized as the signal shape area.
The recognition device according to claim 1. The color information is color information in a (Y, U, V) color space,
The image acquisition unit acquires the image information in the (R, G, B) color space,
The recognition unit converts the image information of the (R, G, B) color space into the image information of the (Y, U, V) color space, and converts the image information into the color information of the (Y, U, V) color space. Recognizing the signal pixel included in the image information of the (Y, U, V) color space based on the
The recognition device according to claim 2. The recognition information includes characteristic information indicating characteristics of a peripheral region of the signal region,
The determination unit determines that the signal shape region is the signal region when the feature information and the feature information of the peripheral region of the signal shape region are similar to each other,
The recognition device according to claim 1. The characteristic information includes information indicating a range of average luminance of a peripheral region of the signal region,
The determination unit determines that the signal shape region is the signal region when the average brightness of the peripheral region of the signal shape region is included in the range of the average brightness of the peripheral region of the signal region,
The recognition device according to claim 4. The peripheral region includes a region showing a signal that is not lit,
The recognition device according to claim 4 or 5. A recognition method for a recognition device, comprising: an image acquisition unit that acquires image information around a vehicle; and a position acquisition unit that acquires position information indicating the position of the vehicle,
A step of selecting recognition information for the recognition device to recognize a traffic signal in the area specified from the position information,
A step of recognizing a signal shape region indicating a shape of the signal of the traffic light from the image information based on the recognition information;
A step of recognizing, based on the recognition information, determining whether the signal shape area is a signal area indicating a signal of the traffic light,
Recognition method including. A recognition device comprising: an image acquisition unit that acquires image information around the vehicle; and a position acquisition unit that acquires position information indicating the position of the vehicle,
A selection unit for selecting recognition information for recognizing a traffic signal in the area specified from the position information;
A recognition unit that recognizes a signal shape region indicating the shape of the signal of the traffic light from the image information based on the recognition information;
A determination unit that determines whether the signal shape region is a signal region indicating a signal of the traffic light based on the recognition information,
Program to function as.

Images