JP2006350844A - Command input device - Google Patents

Abstract

A hand command is accurately extracted and determined from imaging data of an infrared camera.
An infrared camera 10 for imaging a predetermined detection area, an extraction means 21 for extracting the shape and / or movement of a passenger's hand based on imaging data, and the extracted hand shape and / or hand. A determination unit 22 for determining a command corresponding to the movement of the vehicle, an execution unit 23 for executing the determined command, and a background surface of a detection area imaged by the infrared camera 10 to the temperature of the occupant, the minimum temperature resolution of the infrared camera 10 The background warming means 50 for warming so as to be higher than the first predetermined temperature to which is added is provided.
[Selection] Figure 1

Description

  The present invention relates to a command input device that receives a command input to a vehicle-mounted computer and expressed by an occupant, and more particularly to a command input device using an infrared camera.

  With the diversification of in-vehicle devices mounted on vehicles, input operations performed by drivers have become complicated. In order to reduce the burden on the driver while driving, an input device has been proposed that recognizes the shape and movement of the finger that the driver has pushed out while grasping the steering wheel with an image sensor and receives commands according to the shape and movement of the finger. . In order to be able to recognize the shape of a finger at night or in dark places where it is difficult to capture images with a normal camera, some devices of this type employ an infrared image sensor that detects infrared rays emitted from an object. (See Patent Document 1 and Patent Document 2).

However, depending on the season, temperature, and occupant temperature (body temperature, hand temperature, etc.), the temperature difference between the hand to be detected and the background (difference in the amount of infrared rays) is small, so the shape and movement of the hand are extracted from the background. There is a problem that the command presented by the driver cannot be recognized or is erroneously recognized.
JP 2001-216069 A JP 2003-131785 A

  An object of the present invention is to improve the recognition rate of a command based on the shape of a hand indicated by an occupant.

  According to the present invention, an occupant's hand shape and / or hand movement is extracted based on imaging data obtained by an infrared camera imaging a predetermined detection area, and the extracted hand shape and / or hand motion is extracted. A command input device for determining a command corresponding to a motion and executing the determined command, wherein a background surface of a detection area captured by an infrared camera is added to a passenger's temperature by adding the minimum temperature resolution of the infrared camera. (1) A command input device further provided with background heating means for heating to a temperature higher than a predetermined temperature is provided.

  As a result, a temperature difference can be generated between the hand to be recognized and the background surface, so that the shape of the hand shown by the occupant is not affected by the season, temperature, temperature of the driver's hand, etc. Etc. can be recognized accurately.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The command input device 100 according to the present embodiment is mounted on a vehicle, and as shown in FIG. 1, the occupant grips the steering of the vehicle and forms a hand or finger (such as the number of fingers and fingers that are extended) or the hand or fingers. Accepts command input expressed by actions. The command of this embodiment is a control command for in-vehicle devices such as an on-vehicle audio device, an air conditioner, and a navigation device.

  FIG. 2 shows a block configuration of the command input device 100 of the present embodiment. As shown in FIG. 2, the command input device 100 includes an infrared camera 10, an input control device 20, and a background heating unit 50. The command input device 100 is connected to an external in-vehicle device 200 so as to be able to exchange information via an in-vehicle LAN. The command input device 100 sends a command received via the in-vehicle LAN toward the in-vehicle device 200.

  The command input device 100 of the present embodiment captures a command indicated by an occupant with the infrared camera 10 and extracts and determines a command based on the captured image data. FIG. 3 shows the arrangement of the infrared camera 10. FIG. 3 shows the detection region 6 when the occupant's hand 7 holds the steering wheel as viewed from above. As shown in FIG. 3, the infrared camera 10 of the present embodiment is attached to the steering column 2 and includes a predetermined detection including an occupant's hand 7 holding the steering 1 and an inner wall surface of the front door 4 serving as a background surface. The area is imaged, and infrared data of the background surface and the occupant's hand in the detection area is acquired. The reason why the detection area is set in this manner is to prevent the imaging data from including an occupant's hand and an object other than the background, for example, a part of the occupant's body. Accordingly, since only the background surface and the hand of the occupant to be detected can be imaged, it is possible to accurately extract the shape of the hand indicated by the occupant. Although FIG. 3 shows the arrangement of each component in the right steering vehicle, the left and right steering vehicles have the opposite positional relationship.

  As shown in the figure, in the present embodiment, the background heating means 50 for heating the inner wall surface of the front door 4 serving as the background surface of the detection region imaged by the infrared camera 10 is provided. The background heating means 50 of the present embodiment is an electric heating element 51 (51A to 51F) and is embedded inside the inner wall surface of the front door 4 in the area imaged by the infrared camera 10.

  Further, the background heating means 50 of the present embodiment includes a heating controller 52 that controls heating of the heating element 51, and the background surface of the detection region imaged by the infrared camera 10 is converted into the minimum temperature resolution of the infrared camera 10 to the occupant temperature. It heats so that it may become temperature higher than the 1st predetermined temperature which added (NETD: Noise Equivalent Temp. Difference). Specifically, the heating controller 52 includes a background surface temperature sensor 521 that detects the temperature of the background surface, and when the temperature of the background surface measured by the background surface temperature sensor 521 is lower than the first predetermined temperature, the first is detected. The background surface is heated until a predetermined temperature is reached.

  The first predetermined temperature in this example is a value obtained by adding a minimum temperature resolution to the “occupant temperature”. The “occupant temperature” can be arbitrarily defined. For example, the preset “occupant temperature” may be defined as “occupant temperature”, or the preset “occupant hand temperature” may be defined. It may be defined as “occupant temperature”. The body temperature of the occupant and the temperature of the occupant's hands are preferably calculated in advance based on the actual measured values of the temperature of a sufficient number of monitor hands, but based on the actual measured values of the occupant using the command input device 100. May be defined. That is, the occupant's body temperature and hand temperature may be measured based on the imaging data of the infrared camera 10, and the “occupant temperature” may be set based on the measured value. The temperature value of the minimum temperature resolution is set in advance based on the temperature value of the minimum temperature resolution of each infrared camera 10 to be mounted.

  Although not particularly limited, the background heating means 50 includes an ambient temperature monitor 53 that detects the temperature around the detection area, and the heating controller 52 detects the background surface when the detected temperature around the detection area is lower than the second predetermined temperature. It is preferable to control so as to reduce the amount of heat for heating. When the temperature around the detection area is low, it can be inferred that the temperature of the occupant to be detected (including the occupant's body temperature and the temperature of the occupant's hand) also decreases. For this reason, in this example, the amount of heat applied to the background surface is reduced in accordance with a decrease in the temperature of the occupant, and the amount of heat is controlled so as not to overheat the background surface. Thereby, energy consumption required for heating the background surface can be saved.

  Although the background heating means 50 of this embodiment will not be specifically limited if it is a heater which can heat the background surface of a detection area | region, the specific aspect was shown in FIGS.

  FIG. 4A shows the background heating means 50 having a plurality of heating elements 51A interspersed. The background heating means 50 is built inside the door inner wall surface 4. FIG. 4B shows an enlarged part P of the background heating means 50 shown in FIG. Each heating element 51A is electrically connected to uniformly heat the background surface. Thereby, energy consumption can be saved compared with heating the entire surface.

  FIG. 5A shows the background heating means 50 having linear heating elements 51B arranged radially from a predetermined point in the detection region. The position of the predetermined point is not particularly limited, but is preferably set at the center of the occupant's palm imaged in the detection area. The background heating means 50 is built inside the door inner wall surface 4. FIG. 5B shows a state in which a hand image is superimposed on the background heating unit 50 shown in FIG. Since the fingers of the hand extend radially, the heating elements 51B provided radially can efficiently heat the portions corresponding to the fingers in the background surface. Thereby, a temperature difference between the finger and the background can be efficiently generated, and energy consumption can be saved as compared with heating the entire surface.

  FIG. 6A shows the background heating means 50 having linear heating elements 51C arranged in a mesh pattern. The background heating means 50 is built inside the door inner wall surface 4. FIG. 6B shows a state in which a hand image is superimposed on the background heating means 50 shown in FIG. Each heating element 51C is electrically connected to uniformly heat the background surface. Thereby, energy consumption can be saved compared with heating the entire surface. In addition, since the fingers of the hand extend radially and the fingers of the hand intersect with the mesh-like heating element 51C, the size of the hand is determined with reference to the position of the heating element (the position of the portion that is particularly heated by the heating element). The position of the sheath finger can be determined.

  FIG. 7A shows the background heating means 50 having a linear heating element 51D arranged in a fan shape. The background heating means 50 is built inside the door inner wall surface 4. FIG. 7B shows a state in which a hand image is superimposed on the background heating unit 50 shown in FIG. Since the fingers of the hand extend radially, the heating elements 51D provided radially can efficiently heat the portions corresponding to the fingers in the background surface. Thereby, the temperature difference between the finger to be detected and the background surface can be efficiently generated, and energy consumption can be saved as compared with heating the entire surface. Further, although not particularly limited, the heating element 51D is preferably arranged so as to pass through the feature point Q when extracting the shape of the hand or the like. By doing in this way, a temperature difference can be efficiently generated between the feature point of the hand and the corresponding background surface, and the shape of the hand and the like can be recognized accurately.

  FIG. 8A shows a background heating means 50 having linear heating elements 51E arranged concentrically. The background heating means 50 is built inside the door inner wall surface 4. FIG. 8B shows a state in which a hand image is superimposed on the background heating means 50 shown in FIG. Each heating element 51E is electrically connected to uniformly heat the background surface. Thereby, energy consumption can be saved compared with heating the entire surface. In general, the fingers of the hand extend radially, and the fingers of the hand intersect with the concentric heating element 51E. Therefore, the position of the heating element (the position of the part heated particularly by the heating element) is used as a reference. The size and finger position can be determined.

  FIG. 9A shows the background heating means 50 having a linear heating element 51F arranged in a spiral shape. The background heating means 50 is built inside the door inner wall surface 4. FIG. 9B shows a state in which a hand image is superimposed on the background heating unit 50 shown in FIG. Each heating element 51F is electrically connected to uniformly heat the background surface. Thereby, energy consumption can be saved compared with heating the entire surface. In general, the fingers of the hand extend radially, and the fingers of the hand intersect with the concentric heating element 51E. Therefore, the position of the heating element (the position of the part heated particularly by the heating element) is used as a reference. The size and finger position can be determined. Moreover, since the background heating means 50 of this example can be comprised with the one linear heating element 51F, it can prevent that an electrical connection becomes complicated. Although not particularly limited, it is preferable to further include a rotation mechanism that rotates a linear heating element arranged in a spiral shape. By rotating the spiral heating element 51F, the entire background surface can be heated without spots.

  The temperature adjustment operation of the background heating means 50 configured as described above will be described based on the flowchart of FIG.

  As shown in FIG. 10, after the operation is started (S1), the background surface temperature sensor 521 detects the temperature T1 of the inner wall surface of the vehicle front door 4 serving as the background surface of the detection region (S2). It is determined whether or not the detected background temperature T1 is lower than a preset first predetermined temperature P1 (S3). Although the setting method of the first predetermined temperature P1 is not particularly limited, as shown in FIG. 11A, in this example, α larger than the minimum temperature resolution value is added to the occupant's body temperature (corresponding to the occupant's temperature) 37 ° C. Was defined as the first predetermined temperature. When the temperature T1 of the background surface is lower than the first predetermined temperature P1 (Yes in S3), the heating controller 53 starts heating the background surface (S4). After the start of heating, the background surface temperature sensor 521 monitors the temperature of the background surface. The heating controller 53 heats the background surface until the temperature T1 of the background surface becomes equal to or higher than the first predetermined temperature P1 (No in S5). When the temperature T1 of the background surface becomes equal to or higher than the first predetermined temperature P1 (Yes in S5), the heating controller 53 stops the heating of the background surface (S6). The background surface temperature sensor 521 continues monitoring the background surface temperature until an end command is input (S7 to S9).

  Although not particularly limited, in the present embodiment, the processes of S10 to S12 are performed in parallel with the processes of S2 to S9. This process is a process of reducing the amount of heat for heating the background according to the temperature (air temperature) around the detection region. In this example, in order to save the heating amount of the background surface, when the temperature (air temperature) around the detection area is equal to or lower than T2, the first predetermined temperature P1 is corrected to a first predetermined temperature P2 lower than this. . As shown in FIG. 11B, when the air temperature decreases, the body temperature and the hand surface temperature also decrease. That is, when the air temperature is low, a difference greater than the minimum temperature resolution can be generated between the temperature of the background surface and the body temperature of the occupant or the surface temperature of the hand without heating the background surface to P1. When the ambient air temperature is lower than the predetermined temperature T2, the heating controller 52 of the present embodiment corrects the heating setting temperature P1 of the background surface to be low (P2), and performs heating control so as to reduce the amount of heat for heating the background surface. I do. The method for saving the heating amount is not limited to this method, and the output amount of the background heating unit 50 may be reduced by a predetermined amount, or the output time of the background heating unit 50 may be reduced by a predetermined time.

  According to this embodiment, by heating the background surface to the first predetermined temperature, it is possible to generate a temperature difference that is equal to or higher than the minimum temperature resolution between the hand to be detected and the background surface. Therefore, regardless of the season, temperature, and driver's body temperature (hand temperature), it is possible to distinguish the shape and movement of the occupant's hand from the background and accurately extract them based on the imaging data of the infrared camera. it can. Although not particularly limited, the background heating means 50 operates prior to the operation of the input control device 20, and creates a predetermined temperature difference according to the minimum temperature resolution between the hand or finger of the occupant to be detected and the background surface. It is preferable to make it occur. By operating the input control device 20 with the background surface heated, the input control device 20 can accurately extract and determine commands based on the temperature difference generated between the background surface and the occupant's hand. it can.

  Next, the input control device 20 will be described. The “input control device 20” of the present embodiment includes an extraction unit 21, a determination unit 22, and an execution unit 23, and executes a command indicated by an occupant based on an image captured by the infrared camera 10.

  The “extracting means 21” extracts the shape of the hand and / or the movement of the hand from the image data captured by the infrared camera 10. Although the extraction method is not particularly limited, the distribution of the hot spots included in the temperature band based on the epidermis temperature (24 to 34 ° C.) of the human hand is extracted, and the shape of the human hand and / or the human hand is extracted from this distribution. Extract hand movements.

  The “determination unit 22” determines the content of the command based on the hand shape and / or hand movement extracted by the extraction unit 21. The determination method is not particularly limited, and the extracted hand shape and / or hand movement may be recognized based on the feature points (such as fingertips) or may be determined by a pattern matching method. In the present embodiment, the extracted hand shape and / or hand movement is compared with the hand shape and / or hand movement previously assigned to the command, and Determine hand movement commands. The determination unit 22 acquires correspondence information in which the content of the command is associated with the hand shape and / or hand movement in advance from a readable storage unit (RAM or ROM). In the correspondence information, a predetermined command associated in advance from the hand shape and / or hand movement is specified. A predetermined hand shape (for example, the number of fingers) or the like may be associated with a predetermined command (for example, device ON / OFF), or a predetermined hand shape (for example, a finger A combination of the number (number), etc., and a predetermined command (for example, volume UP / DOWN after audio selection) may be associated.

  The “execution means 23” causes the command determined by the determination means 22 to be executed. The execution means 23 sends the command to the device controlled by the determined command. Each in-vehicle device 200 that has acquired the command operates a function corresponding to the acquired command.

  The control procedure of the input control device 20 is shown in FIG. As shown in FIG. 12, after activation (S101), the infrared camera 10 images a predetermined detection area including the occupant's hand holding the steering wheel 1 (S102). The extraction means 21 extracts the shape of the occupant's hand and / or the movement of the occupant's hand based on the imaging data acquired by the infrared camera 10 (S103). The determination unit 22 determines a command corresponding to the shape and / or movement of the occupant's hand extracted by the extraction unit 21 (S104). The execution unit 23 outputs the command determined by the determination unit 22 to the control unit of the in-vehicle device 200 (S105). The in-vehicle device 200 controls the in-vehicle device 200 based on the received command.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

It is a figure which shows the usage example of the command input device of this embodiment. It is a block block diagram of the command input device of this embodiment. It is a figure which shows an example of the detection area | region of this embodiment. (A) (B) is a figure which shows the 1st example of the background heating means of this embodiment. (A) (B) is a figure which shows the 2nd example of the background heating means of this embodiment. (A) (B) is a figure which shows the 3rd example of the background heating means of this embodiment. (A) and (B) are figures which show the 4th example of the background heating means of this embodiment. (A) and (B) are figures which show the 5th example of the background heating means of this embodiment. (A) (B) is a figure which shows the 6th example of the background heating means of this embodiment. It is a flowchart figure which shows the control procedure of the background heating means of this embodiment. It is a figure for demonstrating the control method of the background heating means of this embodiment. It is a flowchart figure which shows the control procedure of the input control apparatus of this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Command input device 10 ... Infrared camera 20 ... Input control device 21 ... Extraction means 22 ... Determination means 23 ... Execution means 50 ... Background heating means 51A-51F ... Heating element 52 ... Heating controller 53 ... Ambient temperature monitor 1 ... Steering 2 ... Steering column 4 ... Front door 7 ... Passenger hand 71 ... Passenger finger 200 ... In-vehicle device

Claims (10)

An infrared camera for imaging a predetermined detection area;
Extraction means for extracting the shape and / or movement of the occupant's hand based on the imaging data of the infrared camera;
Determining means for determining a command corresponding to the hand shape and / or hand movement extracted by the extracting means;
A command input device having execution means for executing the command determined by the determination means,
A command comprising background heating means for heating the background surface of the detection area captured by the infrared camera to a temperature higher than a first predetermined temperature obtained by adding the minimum temperature resolution of the infrared camera to the occupant temperature. Input device.   The command input device according to claim 1, wherein the background heating means has a plurality of interspersed heating elements.   The command input device according to claim 1, wherein the background heating means includes linear heating elements arranged radially from a predetermined point in the detection region.   The command input device according to claim 1, wherein the background heating means includes linear heating elements arranged in a mesh pattern.   The command input device according to claim 1, wherein the background heating means includes a linear heating element arranged in a fan shape.   The command input device according to claim 5, wherein the heating element is disposed so as to pass through a feature point when the extraction unit extracts a hand shape.   The command input device according to claim 1, wherein the background heating means has linear heating elements arranged concentrically.   The command input device according to claim 1, wherein the background heating means includes a linear heating element arranged in a spiral shape.   The command input device according to claim 8, wherein the background heating means further includes a rotation mechanism that rotates a linear heating element arranged in a spiral shape. The background heating means detects the temperature around the detection area, and reduces the amount of heat for heating the background surface when the detected temperature around the detection area is lower than a second predetermined temperature. The command input device according to any one of 9.

Images