JP2010264777A - Parking assistance device for the visually impaired, parking assistance method for the visually impaired, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a parking support device for color weak persons, a parking support method for color weak persons, and a program in which even color weak persons can drive. <P>SOLUTION: The parking support device for supporting a driver to park a vehicle using images by an in-vehicle camera includes: an image display means for displaying images by switching images for persons with regular color vision, and images for the color weak persons; and a color combination means for showing the images by combining compensating colors set to be identified to the color weak persons. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a parking assistance device for a visually impaired person, a parking assistance method for a visually impaired person, and a program.

  In recent years, with the progress of image processing technology, a car navigation device or a digital television device has been mounted on a vehicle. Some devices of this type display a captured image behind the vehicle when parking in a parking lot (see, for example, Patent Documents 1 to 4).

  The invention described in Patent Document 1 is a device that displays an image obtained by photographing the rear of a vehicle with a camera mounted on the rear of the vehicle, converts the rear image of the vehicle into a bird's-eye view from a vertical direction, and displays the image on a monitor. It is.

  The invention described in Patent Document 2 extracts a region close to the vehicle away from the field of view of the in-vehicle camera from the past image, matches the image outside the field of view with the image in the region to the bird's eye image, and displays the combined image on the monitor It is something to be made.

  The invention described in Patent Document 3 uses the recognized parking frame line to calculate an angle formed by the parking frame line and the front-rear direction of the vehicle, and parking assistance information for parking the vehicle in the parking frame from this angle. The direction in which the steering wheel operation should be rotated is determined, and this direction is displayed on the screen.

  In the invention described in Patent Document 4, a signal light recognizer is mounted on a vehicle so that the color of the signal light is correctly recognized.

  However, the above-described image display device may not perform display settings according to vision for a driver with low color, and it is difficult for both the general color vision person and the color weak person to see the display content on the screen due to environmental changes. There was a point.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a parking assistance device for a visually impaired person, a parking assistance method for a visually impaired person, and a program that can be driven even by a visually impaired person.

  In order to solve the above-mentioned problem, the invention according to claim 1 switches between an image for a general color vision person and an image for a color weak person in a parking assistance device that performs parking assistance to a driver using an image from a vehicle-mounted camera. It comprises screen display means for freely displaying, and color combination means for presenting a combination of correction colors set so as to enable identification of color weak persons.

  The invention according to claim 2 is the invention according to claim 1, wherein the color combination means is a selection means capable of selecting a color combination that can be easily viewed by the color weak person among images taken by the in-vehicle camera. It is characterized by having.

  According to a third aspect of the present invention, in the first aspect of the present invention, the signal lamp detecting means for detecting the signal lamp using the contour of the signal lamp and the chromaticity information, and the color information of the signal lamp detected by the signal lamp detecting section is the color. A signal lamp presenting means for presenting to the weak is provided.

  According to a fourth aspect of the present invention, in the first aspect of the present invention, in the first aspect of the invention, the handle direction determining means for determining a direction of turning the handle by an angle formed by a parking frame and a vehicle on the screen from the in-vehicle camera, An operation error determination unit that determines an operation error of a steering wheel and a feedback display unit that feeds back the operation error to the driver.

  According to a fifth aspect of the present invention, in the fourth aspect of the invention, the feedback display means includes an operation hint guide display means for displaying on the screen with an arrow and a blinking arrow.

  According to a sixth aspect of the present invention, in the parking assistance method for parking assistance to the driver using the image from the in-vehicle camera, the image for the general color vision person and the image for the color weak person are displayed in a switchable manner. Is presented in combination with correction colors set to be identifiable.

  According to a seventh aspect of the present invention, there is provided a program for controlling a parking assistance device that provides parking assistance to a driver using an image from an in-vehicle camera. And a color combination means for executing a procedure for combining and presenting correction colors set so as to be identifiable by the color weak.

  According to the present invention, an image for a general color vision person and an image for a color weak person are displayed in a switchable manner, and the color weak person is presented with a combination of correction colors set so that the color weak person can be identified. Since it becomes possible to identify, it is possible to provide a parking assistance device for the visually impaired person, a parking assistance method for the visually impaired person, and a program that can be driven even by the visually impaired person.

(A) is an example of the conceptual diagram of the passenger car to which the parking assistance apparatus for the visually impaired according to the present invention is applied, and (b) is an example of the block diagram of the parking assistance apparatus shown in (a). It is an example of the flowchart for demonstrating operation | movement of the parking assistance apparatus shown to Fig.1 (a), (b). It is an example of the flowchart for demonstrating operation | movement of the parking assistance apparatus shown to Fig.1 (a), (b). It is an example of the flowchart which shows the switching operation | movement of the parking assistance apparatus shown in FIG. It is an example of the image displayed on the display 2 shown to Fig.1 (a), (b) in the case of garage entry. (A) is the screen of the display 2 shown in FIG. 1 (a), (b) is an example of an enlarged view of the area L1 of the color combination guide of the screen shown in (a), (c) , (B) is another example. It is an example of the combination of the color of the background displayed on the display 2 shown to Fig.1 (a), (b), and the color of an arrow and a handle. It is an example of the flowchart at the time of recognizing a signal lamp. It is an example of a signal lamp. (A) is a screen displayed on the display 2 shown to Fig.1 (a), (b) is an enlarged view of the area | region L1 of (a).

<Embodiment 1>
(Constitution)
Fig.1 (a) is an example of the conceptual diagram of the passenger car which applied the parking assistance apparatus for the visually impaired according to this invention, FIG.1 (b) is a block diagram of the parking assistance apparatus shown to Fig.1 (a). It is an example.
In FIG. 1 (a), an in-vehicle camera (for example, a CCD (charge-coupled device) camera) 1 serving as an imaging means is arranged behind the passenger car (on the right side of the figure) so as to take an image of an obliquely downward view. The image taken by the in-vehicle camera 1 is image-processed by the image processing device 3 built in the passenger car and displayed on the display 2 installed on the front side of the driver's seat. These in-vehicle camera 1, display 2, and image processing device 3 constitute a parking assistance device for the visually impaired.

In FIG. 1B, the image processing apparatus 3 includes a central processing unit (CPU) 4, a read only memory (ROM) 5, a random access memory (RAM) 6, an interface (I / F) 7, and a bus line 9. Have
The display 2 as the screen display means has a function of displaying an image taken by the in-vehicle camera 1 and necessary information, and examples thereof include a liquid crystal panel (or a plasma panel or an organic EL (electroluminescence) panel).
The CPU 4 has a function of comprehensively controlling the parking assistance device for the visually impaired, and for example, a microprocessor can be mentioned.
The ROM 5 is a storage medium in which an operation program is stored, for example, a mask ROM.
The RAM 6 is a storage medium for temporarily storing images taken by the in-vehicle camera 1, and examples thereof include a flash memory.
The I / F 7 has a function of mediating data between the in-vehicle camera 1 and the image processing device 3.
The operation unit 8 may be a remote control switch. Here, the operation unit 8 is described as a touch panel provided on the display 2.
The bus line 9 has a function of exchanging data and control signals between the in-vehicle camera 1, the display 2, the CPU 4, the ROM 5, the RAM 6, and the operation unit 8.

(Operation)
FIG. 3 is an example of a flowchart for explaining the operation of the parking assistance apparatus shown in FIG.
When photographing is started when the passenger car is parked, the CPU 4 obtains an image luminance average value based on the luminance of each pixel of an image obtained by photographing the rear of the passenger car with the in-vehicle camera 1 (step S300).
The CPU 4 determines whether or not the average luminance value is within the threshold range (step S310). When the CPU 4 determines that the average luminance value is within the threshold range (step S310 / YES), the CPU 4 displays it on the display 2 ( If it is determined that the average luminance value is not within the threshold range (step S310 / NO), the average luminance value is adjusted (step S330), and the process returns to step S310. That is, the average luminance value is compared with an optimal threshold range set in advance, and the contrast of the displayed image is adjusted within the threshold range.

FIG. 4 is an example of a flowchart showing the switching operation of the parking assistance apparatus shown in FIG.
The CPU 4 displays a necessary message (for example, “Switch to color support mode? Yes No” or “Switch to color mode? Yes No”) on the color screen of the display 2 (step S400). A button for switching to an easily recognizable screen for the visually impaired is displayed (step S410).
The CPU 4 determines whether or not the switching button on the touch panel has been pressed (step S410), and determines that the switching button (“Yes” in the case of “switch to color support mode”) has been pressed (step S410 / YES), display in the color support mode for the weak (Step S420), and if it is determined that the switch button (“No” for “Switch to color support mode”) is pressed (Step S410 / NO), The display is performed in the color mode for general color vision (step S430).

Here, the present system can be displayed by switching between two screens: a color screen for a general color vision person and a color screen that can be easily identified by a color weak person.
If the driver is a general color senser, the driver will guide the user with the screen background and the default color handle.
When the driver is a color weak person, a color combination of a guide that can be easily recognized by the color weak person can be selected according to the color of the image taken from the in-vehicle camera 1.

Here, color weakness and the like will be described.
The color weakness is a light color vision abnormality, and it is difficult to identify a specific color.
There are an estimated 3 million people with discoloration in Japan, of which 1 in 20 men and 1 in 500 women. It is said that over 200 million people worldwide. Depending on the type of genetic mutation, the color looks different (most people are not completely black and white, and it is difficult to identify a specific color), and no cure has been found. There are also people whose color vision changes due to diseases such as cataracts.
People with color deficiencies often cannot distinguish red and black characters, especially purple and blue, red and green, etc., dark red is easily confused with black, and peach and light blue are also confused. Has been.

  For this reason, contrasting lightness and warm and cold colors (for example, blue and yellow) and contrasting clear and pastel colors (light orange and green) Pink). Examples of colors that can be easily distinguished by the color weak are yellow and black, yellow and blue, white and black, white and blue, white and green, white and red, and a combination of red and blue.

6A is a screen of the display 2 shown in FIG. 1A, and FIG. 6B is an example of an enlarged view of the color combination guide region L1 of the screen shown in FIG. 6A. Yes, FIG. 6C is another example of FIG.
FIG. 6B shows a handle (steering) 10a, an arrow 11a indicating the direction of cutting the handle 10a, and guide areas L2 to L6. The driver can select one of the guide areas L2 to L6 by operating the operation unit 8.
The arrow 11a and the handle 11a have the same color and are different from the background color in the region L1.

FIG. 7 is an example of a combination of a background color, an arrow, and a handle color displayed on the display 2 shown in FIGS.
For example, when the driver operates the operation unit 8 to select the guide region L2 in the region L1 of the display 2, the color (background color) of the region L1 is yellow, and the color of the arrow 11a and the handle 10a is black ( FIG. 6 (a)). When the driver operates the operation unit 8 to select the guide region L3 in the region L1 of the display 2, the color of the region L1 is black, and the color of the arrow 11b and the handle 10b is yellow (FIG. 6B). .
In this way, the background color, the arrow and the color of the handle are displayed in a color that is easy to identify for the weak, so you can instantly grasp the direction to turn the handle even if the driver is weak, Easy garage entry into the parking lot. The display in the area L1 is an operation hint guide display means. The guide areas L2 to L6 are color combination means.

  Here, the bird's-eye view image of the area within the field of view of the in-vehicle camera 1 is obtained by the coordinate conversion processing similar to the device disclosed in Patent Document 1. In addition, a composite image of a region outside the field of view of the in-vehicle camera 1 is obtained by a process similar to the process disclosed in Patent Document 2. Furthermore, a parking frame line can be detected by the process disclosed in Patent Document 3.

Next, the operation of the parking assistance device for the visually impaired according to the present invention will be described with reference to FIGS. 2, 3, and 5.
FIGS. 2 and 3 are examples of a flowchart for explaining the operation of the parking assistance apparatus shown in FIGS. 1A and 1B. FIG. 5 is a diagram of FIG. It is an example of the image displayed on the display 2 shown to (b).
The image processing apparatus 3 stores image data captured by the in-vehicle camera 1 in the RAM 6, and the image data is output from the RAM 6 to the in-vehicle display 2 in accordance with image rendering.
The CPU 4 determines whether or not the shift position is back (step S110). When the CPU 4 determines that the shift position is not back (step S110 / NO), the CPU 4 stands by and determines that the shift position is back (step S110). / YES), the optimum place for the contrast of the background image is determined and adjusted (step S120).
The CPU 4 selects a switching button (step S130), creates a bird's eye view image (step S140), synthesizes a camera out-of-view area (step S150), and performs a parking frame line detection process (step S160).
The CPU 4 calculates an angle θ between the detected parking frame L10 and the front-rear direction of the passenger car as shown in FIG. 5 (step S170).

Next, the CPU 4 determines the direction in which the handle should be turned so that the angle θ calculated in step S170 approaches 0 (step S180).
The CPU 4 guides the direction to turn the handle on the screen with an arrow (step S190). The angle θ formed at this time is stored in the RAM 6. The angle θ made according to the movement of the passenger car is compared with the angle θ made in the RAM 6 and updated (step S200).
However, if the angle θ formed increases, an operation error has occurred. At that time, the arrow (arrow 11a in FIG. 6A or arrow 11b in FIG. 6B) displayed on the area L1 of the screen of the display 2 is changed to a blinking state (step S210).
Next, the CPU 4 determines whether or not the formed angle θ is within ± 1 ° (step S220). If the CPU 4 determines that the formed angle θ is not within ± 1 ° (step S220 / NO), the CPU 4 proceeds to step S110. Returning, when it is determined that the angle is within ± 1 ° (step S220 / YES), parking is determined (step S230), and the process returns to step S110.

  The color display of the arrow and the blinking arrow is performed according to the above-described procedure (step S400 to step S430). When the switching button is selected, the guide shown in FIG. 6B or 6C is displayed on the monitor screen.

  As described above, according to the present invention, according to the change of the angle θ formed by the parking frame and the vehicle longitudinal direction, the operation operation of the steering wheel is notified from the operation hint guide display means to both the color-blind person and the general color-sensitive person. Yes, the vehicle can be parked in the parking frame.

  In addition, since it is possible to select a guide color combination that is easy to see for users with color weakness due to the background color of the image, both the captured image of the in-vehicle camera and the operation hint guide can be displayed on the monitor screen in an easy-to-view manner.

<Embodiment 2>
FIG. 8 is an example of a flowchart for recognizing a signal lamp.
You may process using the image image | photographed for every time (refer patent document 4). As shown in the flowchart of FIG. 8, the signal light search flow may be affected by the climate, the position of the sun, street lights, etc., and color saturation may occur.
Specifically, when the image is input (step 800), the CPU 4 eliminates irrelevant colors (step S810), removes pixels that are equal to or higher than the saturated color threshold (step S820), differentiates the image (step S830), The contour is detected by the Hough transform (step S840), the signal lamp is recognized (step S850), the signal lamp color is recognized (step S860), and the signal lamp image is determined (step S870).

Here, in order to correctly recognize the color, pixels having two values of RGB (Red Green Blue) of 250 or more and the remaining one value of 180 or more are removed from the image as color saturation pixels. Further, in order to lighten the processing, the color of the photographed image is removed other than red, blue and yellow. The contour of a signal lamp (horizontal traffic signal) can be recognized by image processing such as image differentiation and Hough processing. The contour of the signal light sample is shown in FIG.
FIG. 9 is an example of a signal lamp.

  In order to avoid errors due to changes in climate and environment, yellow, blue, and red histogram tables are created and stored using signal light images taken in various environments in advance. The stored histograms for each color can be compared and determined to determine the color of the extracted signal light.

FIG. 10A is a screen displayed on the display 2 shown in FIG. 1A, and FIG. 10B is an enlarged view of a region L1 in FIG. 10A.
By operating the switching button of the operation unit 8, as shown in FIG. 10B, it is possible to switch from the screen showing the direction of turning the handle to the signal light detection screen.
However, in the case of a configuration in which a signal lamp is displayed on the screen, the in-vehicle camera 1 is mounted not only in the rear part but also in front of the driver's seat or at the front end of the vehicle.
In FIG. 10B, areas L2 to L6 display color combinations that are easy for color-blind people to identify, as in the areas shown in FIGS. 6B and 6C.
In Japan, where traffic lights are arranged on the left, red, yellow, and blue are determined from the right.
For example, in FIG. 10B, the right side is lit, which means “red”. In this case, the outline of the lighting parts 20 to 22 of the color combination guide signal lamp, the outline of the support part 23, and the inside of the lighting part 22 are displayed in black, and the inside of the support part 23 and the lighting parts 20 and 21 that are not lit. Is displayed in yellow.

  As mentioned above, according to this Embodiment, even the color weak person can grasp | ascertain the lighting state of a signal lamp easily.

<Program and storage medium>
The parking assistance device for the visually impaired person according to the present invention described above is realized by a program that causes a computer to execute processing. Examples of the computer include general-purpose computers such as a microprocessor, but the present invention is not limited to this. Therefore, as an example, a case where the present invention is realized by a program will be described below.

In a program for controlling a parking assistance device that provides parking assistance to a driver using an image from an in-vehicle camera,
On the computer,
(1) A procedure for the screen display means to display an image for a general color vision person and an image for a color weak person in a switchable manner,
(2) A program that causes the color combination means to execute a procedure of presenting a combination of correction colors that are set so that the color weak can be identified.

Thereby, as long as there is a computer environment in which the program can be executed, the parking assistance device for the visually impaired person according to the present invention can be realized anywhere.
Such a program may be stored in a computer-readable storage medium.

  Here, examples of the storage medium include computer-readable storage media such as a CD-ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (CD Recordable), and a DVD (Digital Versatile Disk). , Flash memory, RAM (Random Access Memory), ROM (Read Only Memory), semiconductor memory such as FeRAM (ferroelectric memory), and HDD (Hard Disc Drive).

  The above-described embodiment shows an example of a preferred embodiment of the present invention, and the present invention is not limited thereto, and various modifications can be made without departing from the scope of the invention. is there.

1 In-vehicle camera 2 Display 3 Image processing unit 4 CPU
5 ROM
6 RAM
7 I / F
8 Operation section 9 Bus line 10a, 10b Handle (steering)
11a, 11b Arrow 12 Supporting part 20-22 Lighting part

Japanese Patent No. 3663801 Japanese Patent No. 3677458 JP 2007-161193 A JP 2009-15759 A

Claims (7)

In the parking assistance device that provides parking assistance to the driver using the image from the in-vehicle camera,
Screen display means for switching between an image for a general color vision person and an image for a color blind person,
And a color combination means for presenting a combination of correction colors set so that the color weak person can be identified.   2. The parking assistance for color-impaired persons according to claim 1, wherein the color combination means includes a selection means capable of selecting a color combination that is easy for the color-weak person to view from images taken by the in-vehicle camera. apparatus. Signal light detection means for detecting the signal light using the contour and chromaticity information of the signal light;
2. The parking assistance device for a weak colorant according to claim 1, further comprising a signal light presenting unit for presenting the color information of the signal light detected by the signal light detection unit to the weak colorant. A handle direction determining means for determining a direction of turning a handle by an angle formed by a parking frame and a vehicle on the screen from the in-vehicle camera;
An operation error determination means for determining an operation error of the handle;
The parking assist device for the visually impaired according to claim 1, further comprising feedback display means for feeding back the operation error to the driver.   5. The parking assistance device for color-impaired persons according to claim 4, wherein the feedback display means includes operation hint guide display means for displaying on the screen with an arrow and a blinking arrow. In the parking support method for parking support to the driver using the image from the in-vehicle camera,
Display images for general color vision and images for color blind people in a switchable manner,
A parking assistance method for color-blind persons, characterized by presenting a combination of correction colors set so that the color-blind persons can be identified. In a program for controlling a parking assistance device that provides parking assistance to a driver using an image from an in-vehicle camera,
On the computer,
A procedure for the screen display means to switch between an image for a general color vision person and an image for a color weak person in a switchable manner;
A program for causing a color combination means to execute a procedure of presenting a combination of correction colors that are set so that a color weak person can be identified.

Images