JP2018038553A - In-vehicle display device, control method for in-vehicle display device, and program - Google Patents

Abstract

An in-vehicle display device capable of realizing further improvement is provided. An in-vehicle display device (10) includes a display screen (104), a visible light camera (106) for receiving visible light emitted from a person who receives light emitted from the display screen (104), an infrared LED (107), and an infrared LED (107). Comprises an infrared camera 108 for receiving infrared light emitted from a person who has received the emitted infrared light, and a CPU 101. The CPU 101 is extracted from the intensity of the visible light received by the visible light camera 106. Based on the waveform and the waveform extracted from the received light intensity of the infrared light received by the infrared light camera 106, the emission of the infrared light by the infrared light LED 107 is controlled. Calculates pulse wave information on a pulse wave of a person from a waveform extracted from the received light intensity of the received infrared light, and outputs the calculated pulse wave information. [Selection diagram] Fig. 1

Description

  The present invention relates to an in-vehicle display device, a control method for the in-vehicle display device, and a program.

  Patent Document 1 discloses a control means technique for controlling a vehicle awakening illumination device based on route search information. Patent Document 2 discloses a technique for reducing the luminance of an image to be displayed, determining the amount of backlight in the display device based on the luminance of the image, and controlling the backlight from the determined amount of light. .

JP 2006-27534 A JP 2011-248325 A

  However, the techniques disclosed in Patent Document 1 and Patent Document 2 require further improvement.

  An in-vehicle display device according to an aspect of the present disclosure includes a display screen, a visible light receiving unit that receives visible light emitted from a person who has received light emitted from the display screen, an infrared light source, and the infrared light. An infrared light receiving unit that receives infrared light emitted from a person who has received infrared light emitted from a light source, and a processor, wherein the processor is extracted from visible light received by the visible light receiving unit. The infrared light is emitted from the infrared light source based on the waveform extracted from the infrared light received by the infrared light receiving unit, and the infrared light is controlled after the control. Pulse wave information related to the person's pulse wave is calculated from the waveform extracted from the infrared light received by the light receiving unit, and the calculated pulse wave information is output.

  These general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. The system, method, integrated circuit, computer program And any combination of recording media.

  According to the said aspect, the further improvement is realizable.

FIG. 1 is a block diagram illustrating a hardware configuration of the in-vehicle display device according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the in-vehicle display device according to the first embodiment. FIG. 3 is an external view of the in-vehicle display device according to the first embodiment. FIG. 4 is a diagram illustrating an example of an actual usage scene of the in-vehicle display device according to the first embodiment. FIG. 5 is an explanatory diagram of light irradiation in the visible light emitting unit in the first embodiment. FIG. 6 is a diagram illustrating an example of a user position adjustment method of the in-vehicle display device according to the first embodiment. FIG. 7 is a diagram illustrating an example of a user position adjustment method of the in-vehicle display device according to the first embodiment. FIG. 8 is a diagram for explaining the light emission timing of the visible light emitting unit in the first embodiment. FIG. 9 is a diagram illustrating a method for irradiating a user with the visible light emitting unit in the first embodiment. FIG. 10 is a graph illustrating an example of a luminance change of the visible light image and the infrared light image in the first embodiment. FIG. 11 is a graph illustrating an example of calculation of pulse wave timing. FIG. 12 is a graph showing an example of the heartbeat interval time acquired in time series. FIG. 13 is a graph for explaining a method of extracting an inflection point from a pulse wave. FIG. 14 is a graph showing a visible light waveform for explaining a method of calculating the slope from the top to the bottom of the visible light waveform. FIG. 15 is a graph showing an infrared light waveform when a human skin image is acquired by an infrared light camera for each level of the light amount of the infrared light source. FIG. 16 is a graph showing the first heartbeat interval time and the second heartbeat interval time plotted with data in time series order. FIG. 17 is a diagram for describing a specific example of determining whether or not the heartbeat interval time is appropriate. FIG. 18 is a diagram for explaining an example in which excessive acquisition of peak points is performed in a visible light waveform and excessive acquisition of peak points is not performed in a corresponding infrared light waveform. FIG. 19 is a diagram for explaining a case where the degree of correlation is calculated using inflection points. FIG. 20 is a diagram for explaining an example in which the condition that the number of peak points in the first predetermined period exceeds the first threshold value although the number of peak points is excessive is not applicable. FIG. 21 is a diagram illustrating an example for explaining that the peak point acquired during the adjustment of the light amount of the light source is not used for calculating the degree of correlation between the visible light waveform and the infrared light waveform. FIG. 22 is a diagram showing an example of the simplest steps for decreasing the light amount of the visible light source to zero and increasing the light amount of the infrared light source to an appropriate light amount using the pulse wave measuring device. is there. FIG. 23 illustrates waiting for light source control until two or more predetermined feature points continuous from the waveform are extracted within the second predetermined period in each of the visible light waveform and the infrared light waveform. FIG. FIG. 24 is a diagram illustrating a determination method in the operation determination unit in the first embodiment. FIG. 25 is a diagram illustrating an information display method using a display screen according to Embodiment 1. FIG. 26 is a flowchart illustrating a determination process for controlling the light source in the operation determination unit according to the first embodiment. FIG. 27 is a flowchart showing a flow of processing of the light amount adjustment method in the light source control unit in the first embodiment. FIG. 28 is a flowchart showing a flow of processing of the in-vehicle display device in the first embodiment. FIG. 29 is a flowchart showing a flow of processing for calculating the excess of the number of pulse wave peaks in the first embodiment. FIG. 30 is a flowchart showing a flow of processing for calculating the degree of coincidence of pulse wave feature values in the first embodiment. FIG. 31 is a flowchart showing details of light amount adjustment processing in the first embodiment. FIG. 32 is a block diagram illustrating a configuration of the in-vehicle display device according to the second embodiment. FIG. 33 is an external view of the in-vehicle display device according to the second embodiment. FIG. 34 is a diagram illustrating a question to the user in the initial state of the handle position determination unit in the second embodiment. FIG. 35 is a diagram for explaining a difference in light emitting means depending on a user's selection of the visible light emitting unit and the infrared light emitting unit in the second embodiment. FIG. 36 is a diagram illustrating a light emitting method of a visible light light emitting unit in the second embodiment. FIG. 37 is a diagram illustrating a driving method of the driving unit in the second embodiment. FIG. 38 is a flowchart showing a flow of determination processing by the driver determination unit in the second embodiment. FIG. 39 is a block diagram illustrating a configuration of the in-vehicle display device according to the third embodiment.

(Knowledge that became the basis of the present invention)
The inventor has found that the following problems occur with respect to the technique described in the “Background Art” column.

  The technique disclosed in Patent Document 1 controls the amount of light and time in order to wake up the user by applying light toward the user with a lighting device in the vehicle. There is a problem that the amount of light of the illumination device used in this technology is not set to the amount of light that can acquire a pulse wave for obtaining biological information such as a user's heart rate.

  Further, the technique disclosed in Patent Document 2 controls the amount of light of a backlight in a display device in correspondence with an image to be displayed. With this technique, there is a problem that it is impossible to irradiate the user with a sufficient amount of light to acquire a pulse wave.

  On the other hand, light noise due to the surrounding environment is large during driving of the vehicle. In addition, when visible light is used to acquire the user's biological information during driving of the vehicle at night, it may hinder driving. Therefore, it is useful to acquire pulse waves using infrared light.

  However, since it is difficult to acquire pulse waves using infrared light, the amount of light from the infrared light source must be determined in correspondence with the pulse wave in the visible light region, because there is only a subtle change in the size of infrared light. There is.

  An in-vehicle display device according to an aspect of the present disclosure includes a display screen, a visible light receiving unit that receives visible light emitted from a person who has received light emitted from the display screen, an infrared light source, and the infrared light. An infrared light receiving unit that receives infrared light emitted from a person who has received infrared light emitted from a light source, and a processor, wherein the processor is extracted from visible light received by the visible light receiving unit. The infrared light is emitted from the infrared light source based on the waveform extracted from the infrared light received by the infrared light receiving unit, and the infrared light is controlled after the control. Pulse wave information related to the person's pulse wave is calculated from the waveform extracted from the infrared light received by the light receiving unit, and the calculated pulse wave information is output.

  According to this, the vehicle-mounted display device calculates the light amount of an infrared light source suitable for measuring a person's pulse wave, and acquires the person's pulse wave by irradiating the person with the calculated amount of infrared light. . In this way, the in-vehicle display device can acquire a user's pulse wave in a non-contact manner in the vehicle.

  In addition, for example, in the control of the emission of infrared light, the processor includes a feature amount of a pulse wave component included in a waveform extracted from visible light received by the visible light receiving unit, and the infrared light receiving unit. Calculates the degree of correlation with the feature quantity of the pulse wave component included in the waveform extracted from the received infrared light, and based on the calculated degree of correlation, the infrared light emitted from the infrared light source Take control.

  According to this, the vehicle-mounted display device controls the amount of infrared light irradiated to a person based on the magnitude of the correlation between the visible light pulse wave and the infrared light pulse wave. This makes it possible to change the amount of infrared light while maintaining a state in which there is a correlation between the visible light pulse wave and the infrared light pulse wave, and as a result, a light amount appropriate for acquiring a human pulse wave. It is possible to irradiate a person with infrared rays.

  Further, for example, the processor further performs the control of the emission of infrared light by the infrared light source based on the driving state of a vehicle on which the in-vehicle display device is mounted.

  According to this, the vehicle-mounted display device is suitable for measuring pulse waves by infrared light by irradiating a person with visible light and infrared light when appropriate according to the state of whether the vehicle is driving or the like. The appropriate amount of light can be determined. Specifically, it is possible to perform processing for determining the amount of infrared light to be irradiated to a person at the time of starting the engine.

  Further, for example, the processor further controls the display screen so that visible light emitted from the person is reduced, and a pulse wave component from a waveform extracted from infrared light received by the infrared light receiving unit. The control of the display screen is stopped in a state where the feature amount is obtained.

  According to this, the in-vehicle display device measures the correlation between the visible light pulse wave and the infrared light pulse wave while reducing the amount of visible light irradiated to the person, and is suitable for measuring the pulse wave by infrared light. The appropriate amount of light can be determined.

  Further, for example, the processor reduces the amount of light emitted from the display screen so that the amount of light irradiated to the human eye is less than or equal to a predetermined amount.

  According to this, when determining the appropriate amount of infrared light, the in-vehicle display device suppresses the human being feeling dazzled by irradiating visible light to the human eye by controlling the light amount. Can do.

  Further, for example, the processor changes the posture of the display screen so that the amount of light applied to the human eye is less than or equal to a predetermined amount.

  According to this, when determining the appropriate amount of infrared light, the in-vehicle display device can detect that a person feels dazzling by irradiating human eyes with visible light by controlling the attitude of the display screen. Can be suppressed.

  Further, for example, the processor performs the control of the emission of infrared light by the infrared light source when the acquired driving state is a driving state indicating a state where the vehicle is stopped.

  According to this, the vehicle-mounted display device can determine the amount of light appropriate for measuring the pulse wave by infrared light while the vehicle is stopped.

  In addition, for example, the processor further receives handle position information indicating whether a vehicle on which the in-vehicle display device is disposed is a right handle vehicle or a left handle vehicle, and based on the received handle position information, The display screen is tilted so that the light emission direction by the display screen approaches a direction facing a seat having a handle.

  According to this, the in-vehicle display device can appropriately acquire the pulse wave of the user who is in the driver's seat by changing the posture of the display screen according to the steering wheel position of the vehicle.

  Further, for example, the in-vehicle display device includes the two infrared light sources, and the two infrared light sources are arranged at positions sandwiching the display screen in a horizontal direction in a front view of the display screen. .

  According to this, the vehicle-mounted display device appropriately detects the pulse wave of the user riding in the driver seat of the left-hand drive vehicle or right-hand drive vehicle by using one of the two infrared light sources arranged on the left and right. Can be obtained.

  Further, for example, when the received handle position information indicates a right-hand drive vehicle, the processor further causes only the right half of the display screen to emit light.

  According to this, the vehicle-mounted display device can acquire a user's pulse wave in a non-contact manner in a right-hand drive vehicle.

  Further, for example, when the received handle position information indicates a left-hand drive vehicle, the processor further causes only the left half of the display screen to emit light.

  According to this, the vehicle-mounted display device can acquire a user's pulse wave in a non-contact manner in a left-hand drive vehicle.

  Moreover, about the control method of the vehicle-mounted display apparatus which concerns on 1 aspect of this indication, the said vehicle-mounted display apparatus receives the visible light received from the person who received the display screen and the light which the said display screen emitted. Part, an infrared light source, an infrared light receiving unit that receives infrared light emitted from a person who has received infrared light emitted from the infrared light source, and a processor, Based on the waveform extracted from the visible light received by the visible light receiving unit and the waveform extracted from the infrared light received by the infrared light receiving unit by the processor, the red light by the infrared light source The emission of external light is controlled, pulse wave information related to the person's pulse wave is calculated from the waveform extracted from the infrared light received by the infrared light receiving unit after the control, and the calculated pulse wave information Is output.

  According to this, there exists an effect similar to the said vehicle-mounted display apparatus.

  Moreover, the program which concerns on 1 aspect of this indication is a program for making a computer perform the control method of said vehicle-mounted display apparatus.

  According to this, there exists an effect similar to the said vehicle-mounted display apparatus.

(Embodiment 1)
In the present embodiment, an in-vehicle display device that acquires a user's pulse wave in a vehicle without contact will be described. This in-vehicle display device measures a user's pulse wave in the visible light region in a non-contact manner in the vehicle, and at the same time, measures the user's pulse wave in the infrared light region, and measures the waveform of these measured pulse waves. Each of the visible light source and the infrared light source is controlled based on the correlation.

  The in-vehicle display device 10 according to the present embodiment will be described.

  FIG. 1 is a block diagram showing a hardware configuration of the in-vehicle display device 10 according to the present embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the in-vehicle display device 10 according to the present embodiment.

  As shown in FIG. 1, an in-vehicle display device 10 includes a CPU (Central Processing Unit) 101, a main memory 102, a storage 103, a display screen 104, a visible light camera 106, and an infrared light LED (Light Emitting). (Diode) 107 and an infrared light camera 108.

  The CPU 101 is a processor that executes a control program stored in the storage 103 or the like.

  The main memory 102 is a volatile storage area (main storage device) used as a work area used when the CPU 101 executes a control program.

  The storage 103 is a non-volatile storage area (auxiliary storage device) that holds control programs, various data, and the like.

  The display screen 104 is a display screen that displays an image by light emission of a plurality of pixels. The display screen 104 includes a visible light LED 105 for causing the pixel to emit light with visible light, and expresses pixels of various colors by adjusting the intensity of light emitted from the visible light LED 105. The adjustment of the intensity of the light emitted from the visible light LED 105 may be performed by changing the intensity of the light emitted from the visible light LED 105. The visible light LED 105 emits light having a certain intensity and receives the light. It may be performed by adjusting the degree of light transmission by an element (for example, a liquid crystal element). The display screen 104 is, for example, a liquid crystal display. In this case, the visible light LED 105 corresponds to a backlight of the liquid crystal display. Note that other light emitting elements that emit visible light may be used instead of the visible light LED 105.

  The visible light camera 106 is an imaging device that captures light in the visible light region.

  The infrared light LED 107 is an LED that emits light in the infrared region. In addition, it is also possible to use other light emitting elements or the like that emit infrared rays instead of the infrared LED 107.

  The infrared camera 108 is an imaging device that captures light in the infrared light region.

  Each component to be described later of the in-vehicle display device 10 can be realized by the CPU 101 executing a program stored in the storage 103 using the main memory 102 or the like.

  As shown in FIG. 2, the in-vehicle display device 10 includes a visible light source 121, a visible light imaging unit 122, an infrared light source 123, an infrared light imaging unit 124, a visible light waveform calculation unit 111, and red. An external light waveform calculation unit 112, a correlation degree calculation unit 113, a light source control unit 114, a biological information calculation unit 115, and a driving determination unit 116 are provided. The visible light source 121 is provided in the display screen 104.

  The visible light source 121 irradiates the user's skin with visible light using the visible light LED 105.

  The visible light imaging unit 122 images the user's skin in the visible light region using the visible light camera 106. The visible light imaging unit 122 corresponds to a visible light receiving unit.

  The infrared light source 123 irradiates the user's skin with infrared light using the infrared light LED 107.

  The infrared light imaging unit 124 images the user's skin in the infrared light region using the infrared light camera 108. The infrared light imaging unit 124 corresponds to an infrared light receiving unit.

  The visible light waveform calculation unit 111 extracts a pulse wave waveform from an image captured by the visible light imaging unit 122 with visible light.

  The infrared light waveform calculation unit 112 extracts a pulse wave waveform from an image captured by the infrared light imaging unit 124 with infrared light.

  The correlation calculation unit 113 includes a pulse wave obtained with visible light (hereinafter also referred to as visible light pulse wave) and a pulse wave obtained with infrared light (hereinafter also referred to as infrared light pulse wave). The waveform is compared and the degree of correlation is calculated.

  The light source control unit 114 is based on the waveform extracted from the visible light received by the visible light imaging unit 122 and the waveform extracted from the infrared light received by the infrared light imaging unit 124. Controls the emission of infrared light by. Specifically, the light source control unit 114 uses the feature amount of the pulse wave component included in the waveform extracted from the visible light received by the visible light imaging unit 122 and the infrared light received by the infrared light imaging unit 124. The degree of correlation with the feature quantity of the pulse wave component included in the extracted waveform is calculated, and the emission of infrared light by the infrared light source 123 is controlled based on the calculated degree of correlation.

  The biological information calculation unit 115 calculates pulse wave information indicating the user's pulse wave from the visible light pulse wave and the infrared light pulse wave, and outputs the calculated pulse wave information.

  The driving determination unit 116 determines the driving state of the vehicle (for example, whether the vehicle is driving or stopped).

  Each of the above functional blocks will be described later in detail.

  FIG. 3 is an external view of the in-vehicle display device 10 according to the present embodiment. FIG. 3 shows an example of the case where the in-vehicle display device 10 is realized as a car navigation device. However, the in-vehicle display device 10 is a vehicle rear seat monitor device, an electronic mirror device, a playback device for music, and the like. There may be.

  As shown in FIG. 3, the in-vehicle display device 10 has a casing, and each component shown in FIG. 1 is arranged in the casing.

  A visible light camera 106, an infrared light LED 107, and an infrared light camera 108 are arranged side by side on the top of the housing. A display screen 104 is provided on the front surface of the in-vehicle display device 10.

  In general, the position of the face of the driver who is the user is often higher than the main body of the car navigation device in use. Therefore, by arranging the visible light camera 106, the infrared light LED 107, and the infrared light camera 108 on the upper part of the car navigation apparatus, it is possible to efficiently irradiate the user's face with infrared light, and to realize visible light. There is an advantage that the resolution of an image acquired by the camera 106 and the infrared light camera 108 can be increased.

(Display screen 104 and visible light source 121)
The visible light source 121 is a light source that irradiates the user with light in the visible light region. The light irradiation amount by the visible light source 121 is adjusted by the light source control unit 114. Specifically, the visible light source 121 emits light having a wavelength range of 400 to 800 nm. The visible light source 121 is realized by, for example, a display screen 104 of a car navigation device in a vehicle as shown in FIG.

  Moreover, the vehicle-mounted display apparatus 10 is installed near the center of the dashboard in a vehicle similarly to the conventional car navigation apparatus. At this time, as shown in FIG. 5, when the in-vehicle display device 10 installed near the center of the dashboard irradiates light with the visible light source 121, the pulse wave is easily obtained on the face of the user around the cheek. It has the feature that it hits light. Specifically, when the left and right halves of the user's face, for example, when the vehicle is a right-hand drive vehicle and the user is a driver, the light emitted by the visible light source 121 on the left half of the user's face Hits. On the other hand, when the vehicle is a left-hand drive vehicle and the user is a driver, the light emitted from the visible light source 121 hits the right half of the user's face. When the visible light imaging unit 122 captures a user's face, it is easier to acquire a pulse wave because there are no characteristic parts such as eyes and nose when the user's face is captured from the side rather than from the front. There is.

  In addition, although it demonstrated that the irradiation amount control of the visible light source 121 was performed by the light source control part 114, it is not restricted to this. For example, the user himself / herself may manually control the light irradiation amount using a controller. The user may adjust the direction of light emitted from the visible light source 121.

  For example, as shown in FIG. 6, the rear surface of the display screen 104 of the in-vehicle display device 10 is adjusted so that the user can be exposed to light by adjusting the posture of the display screen 104 of the in-vehicle display device 10 with his / her own hand. A universal joint mechanism or the like may be provided on the side. Thereby, even when the position of the face of the user (driver) is different for each user, it is possible to shine light on the position of the user's face. In particular, a male user and a female user often have different face positions during driving. Therefore, by allowing the driver to freely adjust the direction of the display screen 104, the user's pulse wave can be detected more accurately.

  In the present embodiment, the display screen 104 of the car navigation device is used as a visible light source. However, a visible light source may be additionally provided next to the display screen 104 of the car navigation device. In general, the display screen 104 of the car navigation device is often used to check a map or a current location. On the other hand, it is desirable that the light emitted from the visible light source hits the cheek area of the user. Therefore, if the intensity of illumination on the display screen of the car navigation device is insufficient, a visible light source may be newly installed. Thus, by allowing the user to adjust only the direction of illumination of visible light, there is an advantage that both ease of confirmation of a map or the like using the car navigation device and improvement of the adjustment accuracy of the illumination intensity can be achieved. .

  Furthermore, when it is not clear how the user himself / herself sets the posture of the in-vehicle display device 10 (display screen 104), instruction information for adjusting the posture may be presented to the user. Specifically, the posture of the display screen 104 is adjusted when a predetermined time (for example, 10 seconds) elapses after the signal that enables control of the visible light source is sent from the driving determination unit 116 to the light source control unit 114. The instruction information may be displayed on the display screen 104 (see FIG. 7). In FIG. 7, the target position of the user's face (the broken-line circle in the drawing) is shown at the center of the display screen 104, and the user may adjust the position or posture of the in-vehicle display device 10 in accordance with the target position.

  The in-vehicle display device 10 is installed near the center of the dashboard of the vehicle, but is not limited to this. For example, the in-vehicle display device 10 may be installed in front of the user. At this time, the visible light source 121 may irradiate the entire face, not the half of the face, from the front of the user. Thereby, the range which can acquire a pulse wave among user's faces becomes wide, and a visible light pulse wave and an infrared light pulse wave can be acquired more correctly. For example, when the outside of the vehicle is bright in the daytime and the in-vehicle display device 10 is installed near the center of the dashboard, it is mainly half of the user's face, in other words, the window in the user's face. Although only the opposite site is illuminated, there may be cases where a visible light pulse wave and an infrared light pulse wave cannot be acquired due to insufficient light irradiation. On the other hand, if the in-vehicle display device 10 is installed in front of the user, the window side portion of the user's face can be irradiated with light, and the visible light pulse wave and the infrared light pulse wave can be acquired. There is an advantage that the measured pulse wave becomes more accurate.

  The amount of light emitted from the visible light source 121 will be described. The visible light source 121 is used when the surroundings are so bright that it is difficult to obtain a pulse wave from the user's face with visible light (for example, at night or when the vehicle is located in a tunnel). When the engine is started, or when the surroundings become bright enough that it is difficult to obtain a pulse wave from the user's face by visible light during driving. Details of the activation means will be described in the operation determination unit 116 described later.

  For example, as shown in FIG. 8A, when the visible light source 121 is activated when the engine is started, the amount of light emitted from the visible light source 121 is zero before the activation. The visible light source 121 increases the amount of light simultaneously with the start of the engine until the illuminance in the vehicle reaches, for example, 1000 lux, and the visible light pulse wave acquired by the visible light waveform calculator 111 and the infrared light waveform calculator. The light quantity of each light source is controlled so that the infrared light pulse wave acquired by 112 matches. Note that the calculation method of the degree of correlation between the visible light pulse wave and the infrared light pulse wave and a specific method of light source control will be described in the correlation degree calculation unit 113 and the light source control unit 114 described later.

  Further, for example, as shown in FIG. 8B, when the surroundings become dark during driving and the visible light pulse wave cannot be acquired, the illuminance in the vehicle before the surroundings are dark is about 50 lux. This is an example of illuminance when the user uses the navigation function of the car navigation device. In this state, when the visible light waveform calculation unit 111 cannot acquire the visible light pulse wave, the driving determination unit 116 starts driving determination. When the vehicle stops traveling, the driving determination unit 116 transmits an adjustable signal to the light source control unit 114, and the visible light source 121 determines the amount of emitted light, and the illuminance in the vehicle is, for example, 1000 lux. Raise until After that, similarly to when the engine is started, the correlation calculation unit 113 compares the waveform of the visible light pulse wave with the infrared light pulse wave, and the visible light source 121 and the infrared light are acquired so that the infrared light pulse wave can be acquired. The light source 123 is controlled. Thereby, the vehicle-mounted display apparatus 10 can detect a pulse wave also in the tunnel where an outdoor light does not enter into a vehicle, or an indoor parking lot. In particular, when the vehicle is located in the tunnel for a relatively long time (specifically, when traveling in a tunnel with a relatively long distance, or in a tunnel due to traffic jams regardless of the length of the tunnel) The user's heart rate information can be detected when the vehicle is running at a low speed or stopped.

  Although the visible light source 121 is controlled so that the illuminance in the vehicle becomes 1000 lux when the ambient environment becomes dark when the engine is started, the present invention is not limited to this. Since the illuminance can acquire pulse waves from the user's face, a smaller value (for example, 500 lux) may be used. On the other hand, if the illuminance becomes too high, the interior of the vehicle becomes too bright, and the user may feel glare and cause an accident. Considering this, it is preferable that the brightness in the vehicle by the light emitted from the visible light source 121 falls within a range of about 500 lux to 2500 lux. In the case of a user who has acquired a pulse wave once, the illuminance when the pulse wave is acquired under the control of the visible light waveform calculation unit 111 and the light source control unit 114 when the pulse wave is acquired is stored. In addition, the light emission amount of the visible light source 121 may be set so as to have the stored illuminance. As a result, there are advantages of reducing the time required for acquiring the pulse wave and reducing the effort of adjusting the amount of light every time. If it is the same user, there is a high possibility that the color characteristics of the face surface are the same every time, so it is highly possible that the pulse wave can be acquired just by illuminating the user's face with the illuminance once stored Is.

  In addition, the visible light source 121 can acquire a visible light pulse wave by the visible light waveform calculation unit 111 and the peak-to-bottom inclination of the pulse wave at the time of acquisition is the largest. May be recorded, and whenever the light amount of the visible light source 121 is increased to acquire a pulse wave, the light amount may be recorded.

  The visible light source 121 adjusts the posture of the in-vehicle display device 10 (display screen 104) by the user himself so that the light hits the user's face, but the present invention is not limited to this. As shown in FIG. 9A, when the light emitted from the visible light source 121 is directed to the user's eyes when the user is driving, the user may feel dazzling and cause an accident. Therefore, for example, when the user has an opportunity to adjust the posture of the in-vehicle display device 10 in advance, as shown in FIG. 9B, the user's cheek has a relatively strong illuminance, and the user's eyes The vicinity may be set to have a relatively weak illuminance. Moreover, you may set so that the light quantity irradiated to a user's eyes may be below predetermined. This control can be realized, for example, by reducing the amount of light emitted from the display screen 104 or by changing the attitude of the display screen 104 (so-called tilt control). Specifically, it can be realized by setting the center of the light beam emitted from the visible light source 121 so as to be positioned below the central portion of the cheek. This makes it possible to adjust the illuminance without disturbing the user's driving. In addition, glare can also be avoided by a user wearing sunglasses. If it does in this way, in a face recognition process, there exists an advantage which can detect a user's cheek easily based on the position of sunglasses.

  When the visible light imaging unit 122 performs face recognition in advance and increases the intensity of light emitted by the visible light source 121, first, light is irradiated from the cheek to the periphery of the jaw, and the irradiation range is gradually increased. You may make it stop before irradiating eyes. This can be realized by preventing the brightness of the eye position obtained as a result of the face recognition from increasing, that is, by stopping the light irradiation when a sign of an increase in the brightness of the eye position is observed.

(Visible light imaging unit 122)
The visible light imaging unit 122 images an irradiation target irradiated with visible light from the visible light source 121 in the visible light region. Specifically, the visible light imaging unit 122 captures a visible light image obtained by imaging a user's skin as an irradiation target in a visible light region (for example, color), and a visible light waveform calculation unit 111 of the in-vehicle display device 10. Output to. The visible light imaging unit 122 outputs, for example, a skin image obtained by imaging skin including a human face or hand as a visible light image. The skin image is an image obtained by capturing the same portion of the skin including a human face or hand at a plurality of timings that are temporally continuous, and includes, for example, a moving image or a plurality of still images. The visible light imaging unit 122 is realized by the visible light camera 106, for example.

(Infrared light source 123)
The infrared light source 123 irradiates the user with infrared light, and the amount of light to be irradiated is adjusted by the light source control unit 114. The infrared light source 123 is realized by the infrared LED 107, for example.

(Infrared light imaging unit 124)
The infrared light imaging unit 124 images an irradiation target irradiated with infrared light from the infrared light source 123 in the infrared light region. Specifically, the infrared light imaging unit 124 captures an infrared light image obtained by imaging a user's skin as an irradiation target in an infrared light region (for example, monochrome), and the infrared light of the in-vehicle display device 10. The result is output to the waveform calculator 112. The infrared light imaging unit 124 images the same part as the part imaged by the visible light imaging unit 122. For example, the infrared light imaging unit 124 outputs a skin image obtained by imaging skin including a human face or hand as an infrared light image. This is because the infrared light imaging unit 124 can acquire the same pulse wave in the visible light region and the infrared light region by capturing the same region as the region captured by the visible light imaging unit 122. This is because it is easy to compare feature quantities.

  Note that, as an imaging method of the same part, a region of interest (ROI) having the same size is set in the visible light imaging unit 122 and the infrared light imaging unit 124. And it is judged whether the same site | part is imaged by comparing using the pattern recognition about the image in the said ROI imaged with the visible light imaging part 122 and the infrared light imaging part 124, for example. May be. Further, face recognition is performed on each of the visible light image obtained by the visible light imaging unit 122 and the infrared light image obtained by the infrared light imaging unit 124, and the coordinates of the feature points in the eyes, nose, mouth, and the like are performed. The same part is identified by calculating the coordinates (relative position) from the feature points of the eyes, nose, mouth, etc., considering the size ratio of the eyes, nose, mouth, etc. May be.

  Similar to the skin image obtained by the visible light imaging unit 122, the skin image obtained by the infrared light imaging unit 124 has a plurality of timings at which the same portion of the skin including a human face or hand is temporally continuous. For example, it is composed of a moving image or a plurality of still images. The infrared light imaging unit 124 is realized by, for example, the infrared light camera 108.

(Visible light waveform calculation unit 111)
The visible light waveform calculation unit 111 acquires a visible light image from the visible light imaging unit 122 and extracts a visible light waveform that is a waveform indicating a user's visible light pulse wave from the acquired visible light image. The visible light waveform calculation unit 111 may extract a plurality of first feature points that are predetermined feature points in the extracted visible light waveform. The predetermined feature point is, for example, a peak point that is a vertex or a bottom point in a waveform of one cycle of the user's pulse wave.

  The visible light waveform calculation unit 111 acquires a pulse wave timing as a feature point of the visible light waveform, and calculates a heartbeat interval time from the adjacent pulse wave timings. That is, the visible light waveform calculation unit 111 calculates the time between each of the extracted first feature points and another first feature point adjacent to the first feature point as the first heartbeat interval time. .

  Specifically, the visible light waveform calculation unit 111 extracts a visible light waveform based on temporal changes in luminance extracted from a plurality of visible light images each associated with an imaged timing. In other words, each of the plurality of visible light images acquired from the visible light imaging unit 122 is associated with the time (time point) when the visible light image is captured by the visible light imaging unit 122. The visible light waveform calculation unit 111 acquires the pulse wave timing of the user (hereinafter also referred to as pulse wave timing) by acquiring the interval between predetermined feature points of the visible light waveform. Then, the visible light waveform calculation unit 111 calculates, for each of the obtained plurality of pulse wave timings, an interval between the pulse wave timing and the next pulse wave timing as a heartbeat interval time.

  For example, the visible light waveform calculation unit 111 uses the extracted visible light waveform to specify the timing with the largest luminance change, and specifies the specified timing as the pulse wave timing. Alternatively, the visible light waveform calculation unit 111 identifies the positions of the faces or hands in the plurality of visible light images using the face or hand pattern held in advance, and changes the luminance with time at the identified positions. To identify the visible light waveform. The visible light waveform calculation unit 111 calculates the pulse wave timing using the identified visible light waveform. Here, the pulse wave timing is a time at a predetermined feature point in the time waveform of the luminance, that is, the time waveform of the pulse wave. The predetermined feature point is, for example, a peak position (vertex time) in the luminance time waveform. The peak position can be specified by using a known local search method including, for example, a hill-climbing method, an autocorrelation method, and a method using a differential function. The visible light waveform calculation unit 111 is realized by, for example, the CPU 101, the main memory 102, the storage 103, and the like.

  In general, a pulse wave is a change in blood pressure or volume in the peripheral vasculature as the heart beats. That is, the pulse wave is a change in the volume of the blood vessel when blood is pumped out from the heart due to contraction of the heart and reaches the face or hand. Thus, when the volume of the blood vessel in the face or hand changes, the amount of blood passing through the blood vessel changes, and the skin color changes depending on the amount of components in the blood such as hemoglobin. For this reason, the brightness of the face or hand in the captured image changes according to the pulse wave. That is, information on blood movement can be acquired by using temporal changes in luminance of the face or hand obtained from images obtained by capturing the face or hand at a plurality of timings. As described above, the visible light waveform calculation unit 111 acquires pulse wave timing by calculating information related to blood movement from a plurality of images captured in time series.

  In obtaining the pulse wave timing in the visible light region, it is desirable to use an image in which the luminance in the green wavelength region in the visible light image is captured. This is because, in an image captured in the visible light region, a change due to the pulse wave appears greatly in the luminance in the wavelength region near the green. In a visible light image including a plurality of pixels, the luminance in the green wavelength region of a pixel corresponding to a face or hand in a state where a lot of blood is flowing is equivalent to a face or hand in a state where a small amount of blood is flowing It is smaller than the luminance in the green wavelength region of the pixel to be used.

  FIG. 10A is a graph showing an example of a luminance change of a visible light image, particularly a green luminance change in the present embodiment. Specifically, FIG. 10A shows a change in luminance of the green component (G) in the cheek region of the user in the visible light image captured by the visible light imaging unit 122. FIG. In the graph of FIG. 10A, the horizontal axis indicates time, and the vertical axis indicates the luminance of the green component (G). It can be seen that the luminance change shown in FIG. 10A periodically changes in luminance due to the pulse wave.

  When skin is imaged in a daily environment, that is, in a visible light region, a visible light image includes noise due to scattered light from lighting or various factors. Therefore, it is desirable that the visible light waveform calculation unit 111 performs signal processing using a filter or the like on the visible light image acquired from the visible light imaging unit 122, and obtains a visible light image including many skin brightness changes caused by pulse waves. . An example of a filter used for signal processing is a low-pass filter. That is, in this embodiment, the visible light waveform calculation unit 111 performs a visible light waveform extraction process using the luminance change of the green component (G) that has passed through the low-pass filter.

  (A) of FIG. 10 is a graph which shows an example of calculation of the pulse wave timing in this Embodiment. In the graph of FIG. 10A, the horizontal axis indicates time, and the vertical axis indicates luminance. In the time waveform of the graph of FIG. 10A, each point at times t1 to t5 is an inflection point or a vertex. Each point in the time waveform of the graph includes an inflection point and a peak point (vertex and bottom point) as feature points. The vertex is a point at the maximum value that is convex upward in the time waveform, and the base point is the point at the minimum value that is convex downward in the time waveform. At each of the above points included in the time waveform, the time at the point (vertex) where the luminance is higher than any of the previous or subsequent time points, or the point (bottom point) where the luminance is lower than any of the previous or subsequent time points. Time is pulse wave timing.

  A method for specifying the position of the vertex, that is, a peak search method will be described using the luminance time waveform of the graph shown in FIG. The visible light waveform calculation unit 111 sets the current reference point as the point at time t2 in the luminance time waveform. The visible light waveform calculation unit 111 compares the point at time t2 with the point at the previous time t1, and compares the point at time t2 with the point at the next time t3. The visible light waveform calculation unit 111 determines that the reference point is positive when the luminance of the reference point is higher than the luminance of the point at the previous time and the point at the next time. That is, in this case, the visible light waveform calculation unit 111 determines that the reference point is a peak point (vertex) and the time at the reference point is the pulse wave timing.

  On the other hand, the visible light waveform calculation unit 111 determines NO when the luminance of the reference point is smaller than the luminance of at least one of the point at the previous time and the point at the next time. That is, in this case, the visible light waveform calculation unit 111 determines that the reference point is not a peak point (vertex) and that the time at the reference point is not a pulse wave timing.

  In FIG. 11A, the luminance at the point of time t2 is larger than the luminance at the point of time t1, but the luminance at the point of time t2 is smaller than the luminance at the point of time t3. The point at time t2 is determined as NO. Next, the visible light waveform calculation unit 111 increments the reference point by one and sets the next point at time t3 as the reference point. Since the luminance at the point at time t3 is higher than the luminance at the point at time t2 immediately before time t3 and the point at time t4 immediately after time t3, the visible light waveform calculation unit 111 performs the calculation at time t3. The point is determined to be positive. The visible light waveform calculator 111 outputs the time at the point determined to be positive to the correlation calculator 113 as the pulse wave timing. Thereby, as shown in (b) of Drawing 11, the time of a white circle mark is specified as pulse wave timing.

  In addition, the visible light waveform calculation unit 111 considers that the heartbeat interval time is between 333 ms and 1000 ms, for example, based on knowledge of a general heart rate (eg, 60 bpm to 180 bpm) in specifying the pulse wave timing. Thus, the pulse wave timing may be specified. The visible light waveform calculation unit 111 does not need to perform the above-described luminance comparison at all points by considering a general heartbeat interval time, and is appropriate if the luminance comparison is performed only at some points. The pulse wave timing can be specified. That is, the above-described luminance comparison may be performed using each point in the range from 333 ms to 1000 ms from the recently acquired pulse wave timing as a reference point. In this case, the next pulse wave timing can be specified without performing luminance comparison using points before that range as reference points. Therefore, it is possible to acquire a pulse wave timing that is robust in a daily environment.

  The visible light waveform calculation unit 111 further calculates a heartbeat interval time by calculating a time difference between the obtained adjacent pulse wave timings. The heartbeat interval time varies in time series. For this reason, it can be used to calculate the degree of correlation at a predetermined feature point between the visible light waveform and the infrared light waveform by comparing with the heartbeat interval time of the pulse wave specified from the infrared light waveform acquired in the same period. Can do.

  FIG. 12 is a graph showing an example of the heartbeat interval time acquired in time series. In the graph of FIG. 12, the horizontal axis indicates the data number associated with the heartbeat interval time acquired in time series, and the vertical axis indicates the heartbeat interval time. As shown in FIG. 12, it can be seen that the heartbeat interval time varies with time. The data number indicates the order in which data (here, heartbeat interval time) is stored in the memory. That is, the data number corresponding to the heartbeat interval time recorded in the nth (n is a natural number) is “n”.

  The visible light waveform calculation unit 111 may further extract the time of the inflection point immediately after the pulse wave timing in the visible light waveform. Specifically, the visible light waveform calculation unit 111 obtains the minimum point of the visible light differential luminance by calculating the first derivative of the luminance value of the visible light waveform, and sets the time that becomes the minimum point as the time of the inflection point. (Hereinafter referred to as inflection point timing) is calculated. That is, the visible light waveform calculation unit 111 may extract a plurality of inflection points from the top to the bottom as predetermined feature points.

  The visible light waveform calculation unit 111 also calculates the inflection point timing in consideration of the fact that the heartbeat interval time is between 333 ms and 1000 ms, for example, based on general heart rate knowledge. Point timing may be calculated. As a result, even if the visible light waveform includes an inflection point that is completely unrelated to the heartbeat, the inflection point is not specified, so that the inflection point timing can be calculated more accurately. .

  FIG. 13 is a graph for explaining a method of extracting an inflection point from a pulse wave. Specifically, (a) of FIG. 13 is a graph showing a visible light waveform obtained from a visible light image, and (b) of FIG. 13 is plotted with the first derivative values of (a) of FIG. It is a graph. In FIG. 13A, a circle represents a vertex among peak points, and an X represents an inflection point. In FIG. 13B, a circle indicates a point corresponding to the vertex in FIG. 13A, and an X indicates a point corresponding to the inflection point in FIG. In the graph of FIG. 13A, the horizontal axis indicates time, and the vertical axis indicates the luminance value. In the graph of FIG. 13B, the horizontal axis represents time, and the vertical axis represents the derivative of the luminance value.

  In the extraction of the visible light waveform, a visible light image in which green light is captured as described above is used. The principle of this visible light waveform extraction will be described. When the blood volume in blood vessels such as the face or hand increases or decreases according to the pulse wave, the amount of hemoglobin in the blood increases or decreases according to the blood volume. That is, the amount of hemoglobin that absorbs light in the green wavelength region increases or decreases according to the increase or decrease of the blood volume in the blood vessel. For this reason, in the visible light image captured by the visible light imaging unit 122, the color of the skin near the blood vessel changes according to the increase or decrease of the blood volume, and the luminance value of the green component in particular of the visible light varies. . Specifically, since hemoglobin absorbs green light, the luminance value in the visible light image decreases by the amount absorbed by hemoglobin.

  Further, the visible light waveform has a characteristic that the gradient from the top point to the next bottom point is steeper than the gradient from the bottom point to the top point. Therefore, it is relatively susceptible to noise between the bottom point and the apex. On the other hand, since the slope is steep from the top to the next bottom point, it is not easily affected by noise. For this reason, the inflection point timing existing between the vertex and the bottom point is also less susceptible to noise and has a feature that it can be obtained relatively stably. From the above, the visible light waveform calculation unit 111 may calculate the time difference between the inflection points existing from the top to the bottom as the heartbeat interval time.

  Moreover, the peak point of the visible light waveform described above is a portion where the differential coefficient becomes 0 immediately before the inflection point. Specifically, as shown in FIG. 13B, the time at the point where the differential coefficient immediately before the mark X, which is the inflection point, becomes 0, is the time of the circle indicating the vertex in FIG. You can see that Using this feature, the visible light waveform calculation unit 111 may limit the vertex acquired from the visible light waveform to only the vertex immediately before the inflection point.

  The visible light waveform calculation unit 111 further calculates the inclination from the top to the bottom of the visible light waveform. That is, the visible light waveform calculation unit 111 calculates the slope from the top to the bottom of the waveform of one cycle of the user's pulse wave defined by the heartbeat interval time in the visible light waveform. The tilt should be as large as possible. This is because the greater the inclination, the greater the kurtosis of the vertex in the visible light waveform, and the smaller the time lag of the pulse wave timing due to filter processing or the like.

  FIG. 14 is a graph showing a visible light waveform for explaining a method of calculating the slope from the top to the bottom of the visible light waveform. In the graph of FIG. 14, the horizontal axis indicates time, the vertical axis indicates the luminance value, the circle indicates a vertex, and the triangle indicates a bottom point. The visible light waveform calculation unit 111 connects the vertex (circle) and the next base point (triangle) with a straight line, and calculates the slope of the straight line. The inclination calculated here differs depending on the amount of light emitted by the light source in the visible light source 121, the skin area of the user acquired by the visible light imaging unit 122, and the like. Therefore, the pulse wave can be acquired clearly, for example, the ROI corresponding to the light amount of the visible light source 121 and the user's site in the visible light imaging unit 122 so that the heartbeat interval time can be continuously acquired from 333 ms to 1000 ms. Can be set, inclination information can be recorded, and compared with inclination information in the pulse wave of infrared light. Further, the visible light waveform calculation unit 111 is in the initial state, that is, after the visible light source 121 is turned on, the light source control unit 114 causes the visible light amount of the visible light source 121 or the infrared light source 123 to be infrared. The inclination from the top to the bottom of the visible light waveform in the state until the amount of light is changed is recorded in the memory (for example, the storage 103) as the first inclination A. The in-vehicle display device 10 gradually reduces the light amount of the visible light source 121 and increases the light amount of the infrared light source 123 while comparing the feature points between the visible light waveform and the infrared light waveform. It is characterized by going. In this way, in order to gradually reduce the amount of visible light, it is in the initial state that the gradient from the top to the bottom of the visible light waveform becomes the largest.

(Infrared light waveform calculation unit 112)
The infrared light waveform calculation unit 112 acquires an infrared light image from the infrared light imaging unit 124, and extracts an infrared light waveform that is a waveform indicating a user's pulse wave from the acquired infrared light image. The infrared light waveform calculation unit 112 may extract a plurality of second feature points that are predetermined feature points in the extracted infrared light waveform. The predetermined feature point is, for example, a peak point that is a vertex or a bottom point in a waveform of one cycle of the user's pulse wave.

  Similar to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 acquires the pulse wave timing as a feature point of the infrared light waveform, and calculates the heartbeat interval time from the adjacent pulse wave timing. In other words, the infrared light waveform calculation unit 112 calculates the time between each of the extracted second feature points and the other second feature points adjacent to the second feature point as the second heartbeat interval time. To do. Specifically, the infrared light waveform calculation unit 112 extracts an infrared light waveform based on a temporal change in luminance extracted from a plurality of infrared light images. That is, each of the plurality of infrared light images acquired from the infrared light imaging unit 124 is associated with the time (time point) when the infrared light image is captured by the infrared light imaging unit 124. The infrared light waveform calculation unit 112 acquires the pulse wave timing of the user by acquiring an interval between predetermined feature points of the infrared light waveform. Then, for each of the obtained plurality of pulse wave timings, the infrared light waveform calculation unit 112 calculates an interval between the pulse wave timing and the next pulse wave timing as a heartbeat interval time.

  Here, similarly to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 calculates a peak position as a predetermined feature point of the infrared light waveform, for example, a hill climbing method, an autocorrelation method, and a differential function. It can be identified using a known local search method including the method used. The infrared light waveform calculation unit 112 is realized by, for example, the CPU 101, the main memory 102, the storage 103, and the like, similarly to the visible light waveform calculation unit 111.

  In general, in an infrared light image, as in a visible light image, the luminance of a skin region, for example, a face or a hand in the image changes depending on the amount of a component in blood such as hemoglobin. That is, information on blood movement can be acquired by using temporal changes in luminance of the face or hand obtained from images obtained by capturing the face or hand at a plurality of timings. In this manner, the infrared light waveform calculation unit 112 acquires pulse wave timing by calculating information related to blood movement from a plurality of images taken in time series.

  In acquiring the pulse wave timing in the infrared light region, it is desirable to use an image in which the luminance in the wavelength region of 800 nm or more in the infrared light image is captured. This is because, in an image captured in the infrared light region, a change due to the pulse wave appears greatly in the luminance in the wavelength region near 800 to 950 nm.

  FIG. 10B is a graph showing an example of the luminance change of the infrared light image in the present embodiment. Specifically, FIG. 10B shows the luminance change of the cheek region of the user in the infrared light image captured by the infrared light imaging unit 124. In the graph of FIG. 10B, the horizontal axis indicates time, and the vertical axis indicates luminance. It can be seen that the luminance change shown in (b) of FIG. 10 periodically changes in luminance due to the pulse wave.

  However, when skin is imaged in the infrared light region, the amount of absorption of infrared light by hemoglobin is less than when skin is imaged in the visible light region. That is, the infrared light image captured in the infrared light region is likely to contain noise due to various factors such as body movement. Therefore, by applying signal processing to the captured infrared light image using a filter or the like and irradiating the user's skin area with an appropriate amount of infrared light, the infrared light contains many skin luminance changes caused by pulse waves. It is desirable to obtain a light image. An example of a filter used for signal processing is a low-pass filter. In other words, in this embodiment, the infrared light waveform calculation unit 112 performs infrared light waveform extraction processing using the luminance change of the infrared light that has passed through the ρ bus filter. The method for determining the amount of infrared light by the infrared light source 123 is described in the correlation calculation unit 113 or the light source control unit 114.

  Next, a peak search method in the infrared light waveform calculation unit 112 will be described. The peak search in the infrared light waveform can use the same method as the peak search in the visible light waveform.

  As with the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 specifies the pulse wave timing based on knowledge of a general heart rate (for example, 60 bpm to 180 bpm), for example, 333 ms to 1000 ms. The pulse wave timing may be specified in consideration of the period up to. The infrared light waveform calculation unit 112 does not need to perform the above-described luminance comparison at all points by considering a general heartbeat interval time, and is appropriate if the luminance comparison is performed only at some points. It is possible to specify the correct pulse wave timing. That is, the above-described luminance comparison may be performed using each point in the range from 333 ms to 1000 ms from the recently acquired pulse wave timing as a reference point. In this case, the next pulse wave timing can be specified without performing luminance comparison using points before that range as reference points.

  Similar to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 calculates a heartbeat interval time by calculating a time difference between the obtained adjacent pulse wave timings. Further, the infrared light waveform calculation unit 112 may further extract the time of the inflection point immediately after the pulse wave timing in the infrared light waveform. Specifically, the infrared light waveform calculation unit 112 obtains the minimum point of the infrared light differential luminance by calculating the first derivative of the luminance value of the infrared light waveform, and changes the time at which the minimum point is inflected. The point time (inflection point timing) is calculated. That is, the infrared light waveform calculation unit 112 may extract a plurality of inflection points from the top to the bottom as predetermined feature points.

  Similarly to the visible light waveform calculation unit 111, the infrared light waveform calculation unit 112 calculates a tilt from the top to the bottom of the infrared light waveform.

  As described above, the infrared light waveform calculation unit 112 performs a process similar to that of the visible light waveform calculation unit 111 to extract a plurality of predetermined feature points as second feature points. However, the infrared light waveform varies greatly depending on the amount of infrared light emitted from the light source as compared with the visible light waveform. That is, the infrared light waveform is more susceptible to the light amount of the light source than the visible light waveform.

  FIG. 15 is a graph showing an infrared light waveform when a human skin image is acquired by an infrared light camera for each level of the light amount of the infrared light source. In FIG. 15, the light amount level in the infrared light source is increased in order from (a) to (d). That is, the light source level indicates that the light source level 1 has the smallest light amount, the light amount increases as the light source level increases, and the light source level 4 has the largest light amount. The light source level indicates that the control voltage of the light source increases by about 0.5 V every time the level increases by 1. Further, a circle in each graph of FIG. 15 indicates the peak position (vertex) of the pulse wave. As shown in FIG. 15A, when the amount of light in the light source is small, the noise becomes larger than the infrared light from the infrared light source, and it is difficult to specify the pulse wave timing. On the other hand, as shown in (c) and (d) of FIG. 15, when the amount of light in the light source is large, the change in skin luminance corresponding to the pulse wave is buried in the light amount of the light source, and the shape of the pulse wave is reduced. It is difficult to specify the pulse wave timing.

  By the way, when acquiring a pulse wave using an image that is irradiated with visible light and captured in the visible light region, even if the visible light is irradiated with a light amount that is not too strong for the user's eyes, the pulse wave is sufficiently emitted with the irradiation amount. You can get it. However, when acquiring a pulse wave using an image that is irradiated with infrared light and captured in the infrared light region, even if the light amount of the infrared light is controlled, as described above, it contains noise or the infrared light There is too much light. For this reason, it is difficult to acquire a pulse wave only within a considerably narrow light amount range. In addition, even if the amount of light from the infrared light source is determined in advance to a predetermined value, it varies depending on the area of the skin to be acquired, the skin quality of the user, the color of the skin, and the like. Is difficult. Therefore, it is necessary to control the amount of infrared light to an appropriate value while reducing the amount of visible light so that the visible light waveform matches the infrared light waveform by the correlation degree calculation unit 113 described below. There is.

(Correlation degree calculation unit 113)
The correlation degree calculation unit 113 calculates the degree of correlation between the visible light waveform obtained from the visible light waveform calculation unit 111 and the infrared light waveform obtained from the infrared light waveform calculation unit 112. Correlation degree calculation unit 113 then determines a command to adjust each light quantity in visible light source 121 and infrared light source 123 according to the calculated correlation level, and sends the determined command to light source control unit 114.

  Correlation degree calculation unit 113 uses a plurality of first heart beat interval times calculated from a visible light waveform and a plurality of second heart beat interval times calculated from an infrared light waveform as feature amounts. Obtained from the infrared light waveform calculation unit 112, respectively. Then, the correlation degree calculation unit 113 calculates the degree of correlation between the plurality of first heart beat interval times and the plurality of second heart beat interval times corresponding to each other in time series.

  FIG. 16 is a graph showing the first heartbeat interval time and the second heartbeat interval time plotted with data in time series order. In the graph of FIG. 16, the horizontal axis indicates the data numbers in time series, and the vertical axis indicates the heartbeat interval time corresponding to each data number. Here, the data number indicates the order in which data of each heartbeat interval time is stored in the memory. That is, in the first heartbeat interval time, the data number corresponding to the heartbeat interval time recorded nth (n is a natural number) is “n”. In the second heartbeat interval time, the data number corresponding to the heartbeat interval time recorded nth (n is a natural number) is “n”. Furthermore, since the first heartbeat interval time and the second heartbeat interval time are the results of measuring the pulse wave at the same timing, as a general rule, as long as there is no measurement error, if the data numbers are the same, the timing is almost the same. It can be said that it is the result of measuring the pulse wave. That is, the plurality of first heart beat interval times and the plurality of second heart beat interval times include a pair of first heart beat interval times and second heart beat interval times corresponding to each other in time series.

  The correlation degree calculation unit 113 calculates the degree of correlation between the plurality of first heartbeat intervals and the plurality of second heartbeat intervals using the correlation method. For example, if the correlation coefficient as the degree of correlation is a second threshold value, for example, 0.8 or more, the correlation degree calculation unit 113 has a plurality of first heart beat interval times and a plurality of second heart beat interval times. For example, a signal of “TRUE” is transmitted to the light source control unit 114 as a signal indicating that they substantially match. On the other hand, if the correlation coefficient is a value smaller than a second threshold, for example, 0.8, the correlation degree calculation unit 113 has a plurality of first heartbeat intervals and a plurality of second heartbeat intervals. For example, a signal “FALSE” is transmitted to the light source control unit 114 as a signal indicating that they do not match.

  Further, the correlation degree calculation unit 113 determines whether each heartbeat interval time is appropriate as well as the correlation between the first heartbeat interval time and the second heartbeat interval time, and transmits the determination result to the light source control unit 114. May be. Specifically, the correlation calculation unit 113 calculates the time interval between the first heartbeat interval time and the second heartbeat interval time that correspond to each other in time series among the plurality of first heartbeat interval times and the plurality of second heartbeat interval times. It is determined whether or not the absolute error between the two exceeds a third threshold (for example, 200 ms). For example, the correlation calculation unit 113 calculates the absolute error of the first heartbeat interval time and the second heartbeat interval time having the same data number, and determines whether or not the absolute error exceeds the third threshold value. Then, for example, when it is determined that the absolute error exceeds the third threshold value, the correlation calculation unit 113 determines that the number of peak points of either the visible light waveform or the infrared light waveform is excessive. To do. Correlation degree calculation unit 113 then transmits the waveform (visible light waveform or infrared light waveform) with the excessive number of peak points to light source control unit 114. The absolute error calculation is obtained by the following equation 1.

e = RRI _RGB -RRI _IR (Formula 1)

In Equation 1, e indicates the absolute error between the corresponding first heartbeat interval time and the second heartbeat interval time, RRI _RGB indicates the first heartbeat interval time, and RRI _IR indicates the second heartbeat interval time.

  Further, if e is smaller than (−1) × third threshold (for example, −200 ms), correlation degree calculation unit 113 determines that the number of peak points in visible light is excessive, and e is the third If it is larger than the threshold (for example, 200 ms), it is determined that the number of peak points in the infrared light is excessive. Then, correlation degree calculation section 113 transmits information indicating whether the waveform having the excessive number of peak points is the visible light waveform or the infrared light waveform to light source control section 114 as the determination result. In this way, it is possible to specify that the peak point is excessively acquired in either waveform or the acquisition of the peak point is unsuccessful from the shift in the corresponding heartbeat interval time between the two waveforms.

  Correlation degree calculation unit 113, for example, has a corresponding absolute error of the first heartbeat interval time and the second heartbeat interval time that exceeds the third threshold, and the peak point is excessively acquired in the visible light waveform. If it is determined that the signal is present, a “False, RGB” signal indicating the result of the determination is transmitted to the light source control unit 114. If the correlation degree calculation unit 113 determines that the absolute error exceeds the third threshold and that the peak point is excessively acquired in the infrared light waveform, the correlation calculation unit 113 determines that the light source control unit 114 performs the determination. A “False, IR” signal indicating the result is transmitted.

  FIG. 17 is a diagram for describing a specific example of determining whether or not the heartbeat interval time is appropriate. FIG. 17A is a graph showing a case where a plurality of acquired heartbeat intervals are not appropriate. FIG. 17B is a graph showing an example of a visible light waveform or an infrared light waveform corresponding to FIG. In the graph of FIG. 17A, the horizontal axis indicates the data numbers in time series, and the vertical axis indicates the heartbeat interval time corresponding to each data number. In the graph of FIG. 17B, the horizontal axis indicates time, and the vertical axis indicates luminance in the image.

  In FIG. 17A, the two heartbeat intervals surrounded by a dotted line are not appropriate. The heartbeat interval time generally fluctuates and fluctuates, but the value hardly fluctuates abruptly. For example, as shown in FIG. 17A, in the region other than the portion surrounded by the dotted line, the average value is about 950 ms, and the standard deviation is about 50 ms. However, the value of the heartbeat interval time at the two points surrounded by the dotted line changes abruptly to about 600 to 700 ms. This occurs because the portion with the broken line in FIG. 17B is acquired as the peak point. That is, it occurs when the peak points are acquired excessively in the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112.

  When only the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112 obtains a result as shown in FIG. 17, a plurality of first heart beat interval times and a plurality of second heart beat interval times are obtained. When the number of data is compared, the number of data does not match.

  This is shown in FIG. FIG. 18 is a diagram for explaining an example in which excessive acquisition of peak points is performed in a visible light waveform and excessive acquisition of peak points is not performed in a corresponding infrared light waveform.

  A plurality of data of the first or second heart beat interval time is stored in the storage 103 in the form of (data No, heart beat interval time), for example. Data indicating a plurality of first heartbeat intervals acquired in the visible light waveform is, for example, (x, t20-t11), (x + 1, t12-t20), (x + 2, t13-t12). Moreover, the data which shows the some 2nd heartbeat interval time acquired in an infrared-light waveform will be (x, t12-t11), (x + 1, t13-t12), for example. Thereby, when the data acquired in each of the visible light waveform and the infrared light waveform are compared, the number of data is shifted although it is the data acquired in the same time interval t11 to t13. As a result, the data correspondence between the subsequent first heartbeat interval time and the second heartbeat interval time is all shifted, and the degree of correlation of the time fluctuation of the heartbeat interval time is shifted.

  Therefore, the correlation degree calculation unit 113 has an absolute error of the heartbeat interval time in each data number of the first or second heartbeat interval time obtained by the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112. When the third threshold value is, for example, 200 ms or more, one pulse wave peak having the larger number of peak points is deleted. Then, the degree-of-correlation calculation unit 113 performs a process of reducing each subsequent data number by one from the data number corresponding to the deleted peak.

In other words, as described above, when the degree-of-correlation calculating unit 113 determines that the peak points (that is, the predetermined feature points) are acquired excessively, the waveform with the larger number of the predetermined feature points (visible light waveform or A predetermined feature point that is a reference for calculating the heartbeat interval time in the infrared light waveform) may be excluded from the calculation target of the heartbeat interval time. In other words, if e is smaller than (−1) × the third threshold, the correlation degree calculation unit 113 sets the peak point that is the basis of the RRI _RGB calculation used to calculate the e as the first heartbeat interval time. Excluded from the operation target. If e is larger than the third threshold value, correlation degree calculation unit 113 excludes the peak point that is the basis for the RRI _IR calculation used to calculate e from the calculation target of the second heartbeat interval time.

  Further, excessive acquisition of peak points occurs due to a lot of noise in the acquired waveform (visible light waveform or infrared light waveform). For this reason, it is ascertained whether the excessively acquired waveform is a visible light waveform or an infrared light waveform, and for example, a signal such as “FALSE, RGB” is generated as described above, and the generated signal Is transmitted to the light source control unit 114. That is, if the light source control unit 114 receives the “FALSE, RGB” signal, the heartbeat interval times between the visible light waveform and the infrared light waveform are not matched, and the cause is not matched. It can be grasped that it is a visible light waveform. As described above, since the data shift in the acquisition of the peak point between the visible light waveform and the infrared light waveform can be grasped and information indicating the grasped result can be transmitted to the light source control unit 114, the visible light waveform and the infrared light waveform can be transmitted. It becomes possible to acquire a user's pulse wave more accurately.

  In the correlation degree calculation unit 113, the determination of the degree of correlation between the first heartbeat interval time and the second heartbeat interval time is made with the second threshold being 0.8, but the present invention is not limited to this. Specifically, the second threshold value may be changed according to the accuracy of the biological information that the user wants to measure. For example, if the user wants to more accurately acquire biological information during sleep, for example, information such as heartbeat and blood pressure, by accurately extracting pulse waves with infrared light during sleep, the second criterion is used. May be increased to a value such as 0.9, for example.

  Further, when the second threshold value of the correlation coefficient used as a reference is adjusted, the reliability of the acquired data may be displayed on the display screen 104 according to the adjusted second threshold value. For example, if the feature amount between the visible light waveform and the infrared light waveform does not match easily and the amount of light from the visible light source cannot be reduced during sleep or the like, the second correlation coefficient as the reference is used. The threshold value may be changed to a value smaller than 0.8, such as 0.6. At this time, since the accuracy related to the degree of correlation becomes small, it may be displayed on the presentation device 40 that the degree of reliability has become small.

  Correlation degree calculation unit 113, when the correlation coefficient of the first and second heartbeat interval times acquired in time series from the visible light waveform and the infrared light waveform is smaller than the second threshold, or visible light waveform calculation unit 111 When the peak point of the first predetermined period is excessively acquired in the infrared light waveform calculation unit 112, the visible light waveform and the infrared light waveform are obtained using the inflection points of the visible light waveform and the infrared light waveform, respectively. The degree of correlation may be determined. Specifically, as described above, when the correlation coefficient of the first and second heartbeat interval times in the visible light waveform and the infrared light waveform is smaller than a second threshold, for example, 0.8, or visible The number of peak points acquired by the optical waveform calculation unit 111 and the infrared light waveform calculation unit 112 do not match in the first predetermined section (for example, 5 seconds), and the number of peak points in at least one waveform is the first. When the threshold value (for example, 10) is exceeded, the inflection points in both the visible light waveform and the infrared light waveform are used, and the correlation of the time interval information between the inflection points is determined in each waveform. May be.

  That is, the correlation calculation unit 113 determines whether or not the number of peak points in the visible light waveform or the infrared light waveform exceeds the first threshold value in the first predetermined period. When the degree of correlation calculation unit 113 determines that the number of peak points exceeds the first threshold during the first predetermined period, the correlation degree calculation unit 113 may perform the following process. That is, the correlation degree calculation unit 113 causes the visible light waveform calculation unit 111 to extract a plurality of inflection points from the top to the bottom of the visible light waveform as the first feature points. In addition, the correlation calculation unit 113 causes the infrared light waveform calculation unit 112 to extract a plurality of inflection points from the top to the bottom of the infrared light waveform as second feature points. Further, the correlation degree calculation unit 113 sets the time between the first feature point adjacent to the first feature point for each of the extracted first feature points to the visible light waveform calculation unit 111. One heartbeat interval time is calculated. In addition, the correlation degree calculation unit 113 gives the infrared light waveform calculation unit 112 the time between each of the extracted second feature points and another second feature point adjacent to the second feature point. It is calculated as the second heartbeat interval time. Then, the correlation degree calculation unit 113 calculates a correlation degree between the plurality of first heartbeat interval times and the plurality of second heartbeat interval times, which correspond to each other in time series, as a correlation degree.

  FIG. 19 is a diagram for explaining a case where the degree of correlation is calculated using inflection points. 19A is a graph showing the peak points (vertices) acquired in the visible light waveform, and FIG. 19B is a graph showing the peak points (vertices) acquired in the infrared light waveform. It is. 19A and 19B, both the horizontal axis indicates time, the vertical axis indicates luminance, the black circle indicates the acquired vertex, and the white circle indicates the acquired inflection point.

  In (a) of FIG. 19, the peak point is excessively acquired in the visible light waveform, and the peak point is equal to or higher than the first threshold value or exceeds the first threshold value in the first predetermined period (5 seconds). It can be seen that there are 10 or 11. On the other hand, in FIG. 19B, the peak point is acquired at a constant heartbeat interval time in the infrared light waveform, and the standard deviation is 100 ms or less. At this time, the time-series data numbers indicating the first and second heartbeat interval times in the visible light waveform and the infrared light waveform are shifted.

  Therefore, the correlation degree calculation unit 113 uses the inflection points existing between the top and bottom points of each pulse wave, which are obtained from the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112, to display visible light. The degree of correlation between the waveform and the infrared light waveform may be calculated. For example, the correlation calculation unit 113 causes the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112 to calculate the first heartbeat interval time and the second heartbeat interval time calculated using the inflection point, and The degree of correlation between the first and second heartbeat interval times is calculated. As a specific calculation method, the evaluation is performed based on the correlation or absolute error of the heartbeat interval time between the inflection points of the visible light waveform and the infrared light waveform.

  Note that in the correlation calculation unit 113, when the correlation coefficient of the heartbeat interval time in the visible light waveform or the infrared light waveform is smaller than the second threshold, or the number of peak points in the visible light waveform or the infrared light waveform is In the first predetermined period, when the number of peak points in at least one waveform is greater than the first threshold, the correlation degree between the visible light waveform and the infrared light waveform using the heartbeat interval time between the inflection points. However, the present invention is not limited to this. For example, the correlation degree calculation unit 113 may calculate the degree of correlation between the visible light waveform and the infrared light waveform using the heartbeat interval time between the inflection points from the beginning without using the peak point. As a result, even if peak points cannot be accurately obtained from the visible light waveform or infrared light waveform, the time similar to the heartbeat interval time is calculated by calculating the heartbeat interval time between the inflection points. it can. However, the heartbeat interval time between the inflection points is less susceptible to noise than the heartbeat interval time that can be acquired from the peak point, but the inflection point is likely to fluctuate between the vertex and the bottom point. That is, the peak-to-vertex heartbeat interval time is stable, for example, the standard deviation tends to be within 100 ms, and the time error tends to be smaller than the heartbeat interval time between the inflection points. Therefore, in the present disclosure, the heartbeat interval time calculated from the peak point is preferentially used unless otherwise specified.

  In addition to the above, when the following condition is satisfied, the correlation degree calculation unit 113 is used for calculating the correlation degree instead of the heartbeat interval time for calculating the heartbeat interval time between the inflection points from the peak point. Also good. The condition is, for example, the standard deviation of the heartbeat interval time corresponding to the waveform having the smaller number of peak points of the visible light waveform and the infrared light waveform among the plurality of heartbeat intervals and the plurality of heartbeat intervals. Is less than or equal to a fourth threshold value (for example, 100 ms). This is because when determining whether or not an excessive number of peak points has been acquired based on only the number of peak points in the first predetermined period, the peak points in the first predetermined period are actually excessive. This does not apply to the condition that the number of s exceeds the first threshold, and there is a possibility that an excessively acquired peak point may be overlooked.

  For example, FIG. 20 is a diagram for explaining an example in which the condition that the number of peak points in the first predetermined period exceeds the first threshold value although the number of peak points is excessive is not satisfied. 20A and 20B, both the horizontal axis indicates time, the vertical axis indicates luminance, the black circle indicates the acquired vertex, and the white circle indicates the acquired inflection point.

  As shown in FIG. 20A, when the number of peak points acquired in 5 seconds in the visible light waveform is 8, the number of peak points in the first predetermined period exceeds the first threshold. However, the number of peak points different from the number of peak points acquired in the infrared light waveform shown in FIG. 20B is acquired. At this time, as described above, if even one peak point is acquired excessively, there is a problem that the data numbers in the first heartbeat interval time and the second heartbeat interval time are shifted one by one. Alternatively, if it can be shown that the heartbeat interval time of either one of the infrared light waveforms is substantially constant, it can be adjusted (deleted) according to the number of peak points of the waveform. Details of the adjustment of the peak point are as described with reference to FIG.

  It should be noted that when the standard deviation of the heartbeat interval time in the first predetermined period exceeds the fourth threshold value in both the visible light waveform and the infrared light waveform, the correlation degree calculation unit 113 selects an appropriate value from both waveforms. It is determined that the pulse wave timing cannot be acquired, and a “False, Both” signal indicating that appropriate pulse wave timing cannot be acquired from both waveforms is transmitted to the light source control unit 114.

  Correlation degree calculation unit 113, when the in-vehicle display device 10 is started to be used and when the peak point can be appropriately acquired in the first predetermined period by visible light waveform calculation unit 111 (that is, the standard deviation of the heartbeat interval time) Is smaller than the fourth threshold value), the visible light waveform computing unit 111 stores the result of the visible light waveform computation unit 111 as the first slope A as the slope between the vertex and the bottom of the visible light waveform. Then, every time the light amount in the visible light source 121 or the infrared light source 123 is changed by the light source control unit 114, the correlation degree calculation unit 113 changes the second slope between the vertex and the bottom of the infrared light waveform. A command is sent to the light source control unit 114 so that the inclination A is 1. Further, the correlation calculation unit 113 does not need to use the peak point acquired during the adjustment of the light amount of the light source in the light source control unit 114 for calculation of the correlation between the visible light waveform and the infrared light waveform. Good.

  FIG. 21 is a diagram illustrating an example for explaining that the peak point acquired during the adjustment of the light amount of the light source is not used for calculating the degree of correlation between the visible light waveform and the infrared light waveform. In the graph of FIG. 21, the horizontal axis indicates time, the vertical axis indicates luminance, and the state in which the light amount of the light source is adjusted in the shaded area. White circles and black circles indicate the acquired peak points.

  As shown in FIG. 21, by adjusting the light amount of the light source, the luminance gain of the visible light waveform or the infrared light waveform changes, and the kurtosis of the peak point also changes accordingly. When the peak point after the kurtosis is changed is filtered by the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112, the peak point is determined by the kurtosis of the peak of the raw waveform before the filter is applied. The position changes back and forth on the time axis. This error is not a problem as long as the heart rate is calculated as biometric information, but in the case of calculating blood pressure from the pulse wave propagation time, the influence of this error is large. Therefore, in the in-vehicle display device 10 of the present disclosure, a predetermined feature point (from the visible light waveform or the infrared light waveform acquired while the light amount of the visible light source 121 or the infrared light source 123 is controlled by the control signal ( That is, the peak point) need not be extracted.

  It should be noted that when the correlation coefficient of the heartbeat interval time in the visible light waveform and the infrared light waveform is smaller than the second threshold, the correlation degree calculation unit 113 has an excessive number of peak points in one or both waveforms. Assuming that the heartbeat interval time error and the standard deviation of each heartbeat interval time are calculated and the predetermined condition is satisfied, the heartbeat interval time between the inflection points between the top and bottom of the waveform is used. However, it is not limited to this. For example, even if the correlation coefficient between the first heartbeat interval time and the second heartbeat interval time is smaller than the second threshold, the correlation degree calculation unit 113 can appropriately acquire the peak points in both waveforms (for example, both If the standard deviations in the heartbeat interval time of the waveform are both equal to or smaller than the fourth threshold value), only the “False” signal is transmitted to the light source control unit 114.

  In this way, the correlation degree calculation unit 113 outputs signals (for example, “True”, “False”, and the like corresponding to the calculated correlation degree and the extraction result of the predetermined feature points from the visible light waveform and the infrared light waveform. "False, RGB", "False, IR", or "False, Both") is transmitted to the light source control unit 114.

(Light source controller 114)
The light source control unit 114 controls the light amount of the light source in the visible light source 121 from the vehicle driving state determined and transmitted by the driving determination unit 116. Further, the amount of visible light emitted from the visible light source 121 and the amount of infrared light emitted from the infrared light source 123 are controlled in accordance with the signal received from the correlation calculation unit 113. Specifically, the light source control unit 114 controls the display screen 104 so that the visible light emitted from the user is reduced, and the pulse wave component from the waveform extracted from the infrared light received by the infrared light imaging unit 124. Control of the display screen 104 may be stopped in a state where the feature amount is obtained.

  Specifically, the driving determination unit 116 sets “Flag = ON” when the user cannot start acquiring the visible light pulse wave due to the engine being started or the surrounding ambient light becoming dark. By receiving the signal, the amount of light emitted from the visible light source 121 is increased, and the pulse wave feature amounts of the visible light pulse wave and the infrared light pulse wave obtained from the correlation calculation unit 113 are matched. The light quantity of the light source in the infrared light source 123 and the visible light source 121 is controlled from the degree.

  For example, the light source control unit 114 is visible when the engine is not running, the user is driving (that is, the vehicle is moving), or the illuminance in the vehicle is greater than a predetermined threshold (for example, 50 lux). When the optical waveform calculation unit 111 can acquire a visible light pulse wave, a signal “Flag = OFF” is received from the operation determination unit 116. However, when the engine is running, the vehicle is stopped, and the illuminance in the vehicle is a predetermined threshold value (for example, 50 lux) or less, the visible light waveform calculation unit 111 can acquire a visible light pulse wave. If not, the light source control unit 114 receives a “Flag = ON” signal from the operation determination unit 116. Then, the light source control unit 114 changes the amount of light emitted from the visible light source 121 at the moment when “Flag = OFF” is changed to “Flag = ON” so that the illuminance in the vehicle becomes, for example, 1000 lux. Enlarge. Thereafter, the light source control unit 114 controls the light amounts of the light sources of the visible light source 121 and the infrared light source 123 according to a signal from the correlation degree calculation unit 113, and details of the process will be described later.

  The light source control unit 114 receives the Drive signal independently of the Flag signal when the operation determination unit 116 determines that the vehicle is in operation. For example, a signal “Drive = ON” means that the user is driving the vehicle. Here, when the “Drive = ON” signal is received, the light source control unit 114 determines that the user is using navigation by the car navigation device, and does not turn off the visible light source 121 and Continue to irradiate light with an illuminance of about 50 lux.

  Note that the light source control unit 114 may change the amount of light emitted by the visible light source 121 between daytime and nighttime. Also, the determination of day and night may be based on a determination whether it is before or after a predetermined time (for example, 7:00 in the morning or 17:00 in the evening).

  For example, when the light source control unit 114 receives a “False, IR” signal, the light source control unit 114 can determine that the infrared light waveform calculation unit 112 has not properly acquired a predetermined feature point in the infrared light waveform. That is, for example, the “False, IR” signal indicates that there is a lot of noise in the infrared light waveform. For this reason, the amount of the light source in the visible light source 121 is not adjusted, and the amount of the light source in the infrared light source 123 is increased.

  Further, when the light source control unit 114 receives the “False, RGB” signal, the light source control unit 114 can determine that the visible light waveform calculation unit 111 has not properly acquired a predetermined feature point in the visible light waveform. In this case, the light source control unit 114 cannot determine whether or not the infrared light waveform calculation unit 112 can appropriately acquire a predetermined feature point in the infrared light waveform. Therefore, for example, if the standard deviation of the heartbeat interval time in the first predetermined period is equal to or smaller than the fourth threshold in the infrared light waveform, the light source control unit 114 decreases the light amount of the light source in the visible light source 121 and The light quantity of the light source in the external light source 123 is increased until the slope between the top and bottom of the infrared light waveform becomes A. Further, if the standard deviation in the infrared light waveform exceeds the fourth threshold value, the light source control unit 114 determines that both signals cannot be acquired, and changes the signal to “False, Both”.

  Further, when the light source control unit 114 receives the “FALSE, Both” signal, the light source control unit 114 can determine that a predetermined feature point has not been acquired in both the visible light waveform and the infrared light waveform. In this case, the light source control unit 114 increases the light amount of the visible light source 121 until the inclination from the top to the bottom of the visible light waveform reaches the first inclination A. The light source control unit 114 may increase the light amount of the visible light source 121 until the initial light amount of the visible light waveform is stored in the memory. Further, the light source control unit 114 reduces the light amount of the infrared light source 123 to zero. That is, the light source control unit 114 can obtain the light quantity of the visible light source 121 and the infrared light source 123 in a state where the predetermined feature points cannot be obtained most reliably in both the visible light waveform and the infrared light waveform. The light quantity is adjusted to the initial state and the light quantity is adjusted again.

  In other words, the light source control unit 114 has a case where the standard deviation of the plurality of first heartbeat interval times exceeds the fourth threshold value and the standard deviation of the plurality of second heartbeat interval times exceeds the fourth threshold value. When the difference between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the time series is smaller than the fifth threshold value ((−1) × third threshold value), the visible light in the visible light source 121 When the amount of infrared light is increased and the amount of infrared light in the infrared light source 123 is increased and the amount of infrared light is increased, the second slope in the infrared light waveform is stored in the memory. The amount of infrared light is increased until an inclination A of 1 is reached.

  In addition, the light source control unit 114 has a case where the standard deviation of the plurality of first heartbeat interval times exceeds the fourth threshold value, and the standard deviation of the plurality of second heartbeat interval times exceeds the fourth threshold value. When the difference between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the time series is larger than the sixth threshold (that is, the third threshold), the amount of infrared light in the infrared light source 123 When the amount of infrared light increases, the amount of infrared light is increased until the second slope in the infrared light waveform becomes the first slope A stored in the memory.

  In addition, the light source control unit 114 has a case where the standard deviation of the plurality of first heartbeat interval times exceeds the fourth threshold value, and the standard deviation of the plurality of second heartbeat interval times exceeds the fourth threshold value. When the difference between the first heartbeat interval time and the second heartbeat interval time corresponding to each other in the time series is a value between the fifth threshold value and the sixth threshold value, the visible light source 121 The amount of light is increased and the amount of infrared light in the infrared light source 123 is decreased.

  Note that the light source control unit 114 reduces the light amount of the infrared light source 123 to red unless a predetermined feature point cannot be acquired in both the visible light waveform and the infrared light waveform, such as “False, Both”. Although the second inclination of the external light waveform is increased until it reaches the first inclination A, the present invention is not limited to this. For example, when the average luminance value in the ROI exceeds a seventh threshold value, for example, 240, the light source control unit 114 embeds an image captured from the user's skin in noise information because the light amount of the light source is too strong. End up. For this reason, in this case, the light source control unit 114 is considered that the second inclination of the infrared light waveform exceeds the first inclination A, so that the second inclination becomes the first inclination A. The amount of infrared light may be reduced.

  FIG. 22 shows an example of the simplest step for reducing the light amount of the visible light source 121 to 0 and increasing the light amount of the infrared light source 123 to an appropriate light amount using the pulse wave measuring device. FIG. In all the graphs (a) to (d) in FIG. 22, the horizontal axis indicates time, and the vertical axis indicates luminance. In FIG. 22, the visible light waveform is represented as RGB, and the infrared light waveform is represented as IR.

  (A) of FIG. 22 is a figure which shows the visible light waveform and infrared light waveform which were acquired in the initial state which the user turned on the visible light source 121 by the vehicle-mounted display apparatus 10. FIG. The visible light waveform in (a) of FIG. 22 is a waveform having the largest inclination from the top to the bottom among the visible light waveforms in (a) to (d) of FIG. Therefore, the inclination from the top to the bottom of the visible light waveform at this time is stored in the memory as the first inclination A.

  At this time, the infrared light source 123 is OFF. For this reason, an infrared light waveform is hardly acquired. In this state, the correlation calculation unit 113 transmits, for example, a signal “False, IR” to the light source control unit 114. Therefore, the light source control unit 114 increases the light amount of the infrared light source 123 in the infrared light source 123. At this time, as the light amount of the infrared light source 123 is increased, the infrared light waveform calculation unit 112 can acquire a predetermined feature point of the infrared light waveform, and can acquire the second heartbeat interval time. In addition, the acquired standard deviation of the second heartbeat interval time falls within the fourth threshold. Then, as shown in FIG. 22 (b), while maintaining the standard deviation of the second heartbeat interval time within the fourth threshold, the second between the vertex and the bottom of the infrared light waveform is maintained. The light amount of the infrared light source 123 is increased until the inclination becomes the first inclination A. When the second inclination becomes the first inclination A, the correlation degree calculation unit 113 transmits, for example, a signal of “TRUE, AMP = A” to the light source control unit 114. For this reason, the light source control unit 114 once stops the adjustment of the light source when receiving the signal “TRUE, AMP = A”.

  Next, from the state of FIG. 22B, the light source control unit 114 decreases the light amount of the visible light source in the visible light source 121. FIG. 22C shows a state in which the standard deviation of the heartbeat interval time is equal to or smaller than the fourth threshold in the infrared light waveform calculation unit 112 and the light source in the visible light source 121 is OFF. FIG. 22D further shows a state where the light source in the visible light source 121 is OFF and the second slope in the infrared light waveform is the first slope A, that is, This is the final goal.

  In the process of changing from the state shown in FIG. 22B to the state shown in FIG. 22C, the amount of visible light is decreased at regular intervals, for example, by 1W. Then, every time the amount of visible light is decreased, the infrared light waveform calculation unit 112 and the correlation degree calculation unit 113 confirm whether or not a predetermined feature point can be appropriately acquired in the infrared light waveform. Further, if the infrared light waveform calculation unit 112 and the correlation calculation unit 113 can confirm that predetermined feature points can be appropriately acquired in the infrared light waveform, as shown in FIG. The amount of light in the light source of the external light source 123 is increased until the second slope in the infrared light waveform becomes the first slope A.

  Therefore, in the process of changing from the state of FIG. 22B to the state of FIG. 22C, the correlation calculation unit 113 sends a “True” signal to the light source control unit 114 or “False, The light source control unit 114 adjusts the light amount of the infrared light source 123 until it becomes “True” every time it receives the “False, IR” signal. When the light source control unit 114 receives “False, RGB” from the correlation degree calculation unit 113 by decreasing the light amount of the visible light source 121, the process is terminated.

  Alternatively, in the process of changing from the state of FIG. 22C to the state of FIG. 22D, the correlation calculation unit 113 transmits a “False, RGB” signal to the light source control unit 114, and the light source The control unit 114 continues to increase the light amount of the light source in the infrared light source 123 until the second slope in the infrared light waveform reaches the first slope A. For example, the visible light waveform cannot be obtained, and the first When the signal of “False, RGB, AMP = A” indicating that the inclination of 2 has become the first inclination A is received from the correlation calculation unit 113, the light source control by the light source control unit 114 is finished. .

  Further, the light source control unit 114 acquires, in the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112, two or more predetermined feature points that are continuous from the visible light waveform or the infrared light waveform, respectively. After completion, the light source is controlled. In other words, the light source control unit 114 in the visible light source 121 until each of the visible light waveform and the infrared light waveform is extracted within the second predetermined period for two or more predetermined feature points continuous from the waveform. It waits for the output of a control signal for controlling the amount of visible light or a control signal for controlling the amount of infrared light in the infrared light source 123.

  FIG. 23 illustrates waiting for light source control until two or more predetermined feature points continuous from the waveform are extracted within the second predetermined period in each of the visible light waveform and the infrared light waveform. FIG. The graph in FIG. 23 shows a visible light waveform or an infrared light waveform. In the graph of FIG. 23, the horizontal axis indicates time, and the vertical axis indicates luminance.

  When the light source controller 114 changes the light amount of the visible light source 121 or the infrared light source 123, the luminance gain of the visible light waveform or the infrared light waveform changes. When the luminance gain changes, the position of the pulse wave timing shifts, so that a large error occurs in the calculation of timing such as the heartbeat interval time. In the present disclosure, the heartbeat interval time is mainly used as a material for determining the degree of correlation between the visible light waveform and the infrared light waveform, and two consecutive peak points are necessary to calculate the heartbeat interval time. It is. Therefore, as shown in FIG. 23, the light source control unit 114 adjusts the light source amount after confirming that two or more peak points are continuously taken in the visible light waveform or the infrared light waveform.

(Biometric information calculation unit 115)
The biological information calculation unit 115 uses either one of the feature quantities of the visible light waveform acquired by the visible light waveform calculation unit 111 or the infrared light waveform acquired by the infrared light waveform calculation unit 112, and The biometric information is calculated. Specifically, when the visible light source 121 is ON and the visible light waveform calculation unit 111 can acquire a visible light waveform, the biological information calculation unit 115 receives the first heartbeat interval time from the visible light waveform calculation unit 111. To get. Then, the biological information calculation unit 115 calculates biological information such as a heart rate and a stress index using the first heartbeat interval time.

  On the other hand, the biological information calculation unit 115 is a case where the visible light source 121 is OFF or the visible light waveform calculation unit 111 cannot acquire a visible light waveform, and the infrared light waveform calculation unit 112 receives infrared light. When the waveform can be acquired, the second heartbeat interval time is acquired from the infrared light waveform calculation unit 112. Then, the biological information calculation unit 115 similarly calculates biological information such as a heart rate and a stress index using the second heartbeat interval time.

  The biological information calculation unit 115 can extract the feature amount (heartbeat interval time) of each waveform (visible light waveform and infrared light waveform) in both the visible light waveform calculation unit 111 and the infrared light waveform calculation unit 112. If it is, the biological information is calculated using the first heartbeat interval time from the visible light waveform calculation unit 111. This is because visible light is more robust to noise such as body movement and more reliable than infrared light.

  The biometric information to be calculated is a heart rate or a stress index, but is not limited to this. For example, an acceleration pulse wave may be calculated from the obtained pulse wave to calculate an arteriosclerosis index. Alternatively, the pulse wave timing may be accurately acquired from two different user sites, and the blood pressure may be estimated from the time difference (pulse wave propagation time). Further, the user's sleepiness may be calculated by calculating the superiority of the sympathetic nerve and the parasympathetic nerve from the fluctuation of the heartbeat interval time.

(Driving determination unit 116)
The driving determination unit 116 determines the driving state of the vehicle based on whether the vehicle engine is running, whether the user is driving the vehicle, and the like, and the pulse wave obtained from the visible light waveform calculation unit 111 Together with the information, it is transmitted to the light source control unit 114. And the light source control part 114 may control emission of the infrared light by the infrared light source 123, for example, when it is the driving | running state which shows the state which the vehicle has stopped.

  Specifically, the driving determination unit 116 acquires four signals and transmits a determination signal as to whether to turn on the visible light source 121.

  The first of the four signals is “Drive”. The driving determination unit 116 includes a speed sensor, and “Drive = ON” is set when the speed acquired by the speed sensor exceeds a predetermined threshold (for example, 5 km / h), and “Drive = OFF” is set otherwise. To do.

  The second of the four signals is “Engine”. The driving determination unit 116 determines whether the vehicle engine is on (ON) or not (OFF). For example, on the assumption that a current flows through the in-vehicle display device 10 when the vehicle engine is started, it is determined that the engine is ON when the current flows through the in-vehicle display device 10 and “Engine = ON”. Otherwise, it is determined that the engine is OFF, and “Engine = OFF” is set.

  The third of the four signals is “lx”. In the visible light waveform calculation unit 111, if the illuminance in the vehicle is equal to or less than a predetermined threshold (for example, 50 lux), “lx = OFF” is set; otherwise, “lx = ON” is set.

  The fourth of the four signals is “Pulse”. The driving determination unit 116 obtains the frequency at which the waveform of the visible light pulse wave of the user is acquired from the visible light waveform calculation unit 111. If the pulse wave acquisition frequency is sufficient, for example, if the standard deviation of the heartbeat interval time of the visible light pulse wave is equal to or smaller than the fourth threshold, “Pulse = ON” is set; otherwise, “Pulse = OFF” is set. And The pulse wave acquisition frequency is determined based on the standard deviation of the heartbeat interval time in visible light, but is not limited to this. For example, the determination may be made based on the number of pulse wave peaks in a predetermined time interval.

  The driving determination unit 116 comprehensively determines the above four signals, and sends a signal indicating how to control the visible light source 121 to the light source control unit 114. Specifically, for example, when (Drive, Engine, lx, Pulse) = (OFF, ON, OFF, OFF), a signal “Flag = ON” is transmitted to the light source control unit 114, while other than the above At this time, a signal indicating “Flag = OFF” is transmitted to the light source control unit 114.

  An example is shown in FIG. (A) and (b) of FIG. 24 show diagrams when the engine is running, that is, when “Engine = ON”. In FIG. 24A, the speedometer indicates about 60 km / h, so the user is driving. Therefore, since “Drive = ON”, “Flag = OFF”, and the visible light source 121 does not acquire the user's pulse wave by increasing the light amount by the light source control unit 114. At this time, the light source control unit 114 irradiates the visible light source 121 with light with an illuminance in the vehicle of about 50 lux, so that the user can use the navigation by the car navigation device.

  On the other hand, FIG. 24B shows a case where the engine is running but the speedometer is 0 km / h, that is, “Engine = ON” and “Drive = OFF”. Further, at this time, it is assumed that the vehicle interior is relatively dark (for example, the illuminance is about 40 lux) and the standard deviation of the user's pulse wave is larger than the fourth threshold value, and thus cannot be acquired. At this time, the operation determination unit 116 can determine that (Drive, Engine, lx, Pulse) = (OFF, ON, OFF, OFF). Therefore, the visible light source 121 is irradiated with light by the light source controller 114 so that the map information and the like disappear and the illuminance in the vehicle becomes 1000 lux, for example, in order to acquire the user's pulse wave.

(Display screen 104)
The display screen 104 presents the user's face image captured by the visible light imaging unit 122 and gives an instruction to the user so that the user's face is reflected on the visible light imaging unit 122. In addition, the biological information obtained from the biological information calculation unit 115 is presented. Specifically, the display screen 104 displays a heart rate, a stress index, user drowsiness information, and the like obtained from the biological information calculation unit 115. The display screen 104 includes a visible light source 121 as a light source for emitting pixels.

  The display screen 104 presents the biological information obtained from the biological information calculation unit 115, but is not limited thereto. The display screen 104 may always present the amount of light emitted from the visible light source 121 or the amount of light emitted from the infrared light source 123, for example. In addition, the display screen 104 may present the degree of coincidence at the current time point from the correlation degree calculation unit 113 as, for example,% display as the reliability. Specifically, the display screen 104 may present a correlation coefficient between the visible light waveform and the infrared light waveform.

  FIG. 25 shows an example of display on the display screen 104. In FIG. 25, the display screen 104 displays the user's heart rate, stress index, sleepiness index, current pulse wave acquisition reliability (reliability of the pulse wave acquired at the present time), visible light source 121 and infrared light source 123. And the user's arousal state. Here, the current pulse wave acquisition reliability indicates a correlation coefficient in the degree of coincidence of the heartbeat interval time between the visible light pulse wave and the infrared light pulse wave. The ratio of the light amount of the visible light source 121 and the infrared light source 123 is the ratio of the light amount of the visible light source 121 and the intensity of the light amount of the infrared light source 123 at the present time. The user's wakefulness state is the wakefulness state in the user's driving determined from each of the above information. For example, when the heart rate is 65 or less, the stress index is 40 or less, and the sleep depth is 40 or less, “GOOD” is set. . In addition to displaying these display contents, you may notify with a voice | voice etc., for example. Further, when the user's arousal index falls, an alarm sound may be given. This is because since the user is driving the vehicle, it may be more appropriate to obtain information as audio data than to obtain the information as visual data.

  FIG. 26 is a flowchart showing a flow of determination processing by the operation determination unit 116 for adjusting the light amount of the visible light source 121 in the in-vehicle display device 10 according to the present embodiment.

  In step S101, the driving determination unit 116 determines whether or not the engine of the vehicle on which the user gets is started. If the engine is running, the process proceeds to step S102, and if not, the process proceeds to step S107.

  In step S102, the driving determination unit 116 determines whether or not the speed of the vehicle on which the user gets is 5 km / h or more. If the vehicle speed is higher than 5 km / h, the process proceeds to step S106, and if not, the process proceeds to step S103.

  In step S103, the driving determination unit 116 determines whether the illuminance in the vehicle is 50 lux or more. If the illuminance in the vehicle is 50 lux or more, the process proceeds to step S107, and if not, the process proceeds to step S104.

  In step S104, the driving determination unit 116 determines whether or not the standard deviation of the heartbeat interval time of the visible light pulse wave obtained from the visible light waveform calculation unit 111 is equal to or less than a fourth threshold value. If the standard deviation of the heartbeat interval time of the visible light pulse wave is equal to or smaller than the fourth threshold value, the process proceeds to step S107, and if not, the process proceeds to step S105.

  In step S105, the driving determination unit 116 turns on the Flag signal and ends the determination process.

  In step S106, the driving determination unit 116 sets the Drive signal to ON. This is because it is determined in step S102 that the vehicle is driving.

  In step S007, the driving determination unit 116 turns off the Flag signal and ends the process. If it is determined in step S101 that the engine is not running, if it is determined in step S102 that the user is driving, if it is determined in step S103 that the illuminance in the vehicle is 50 lux or more, or step This is because it is not appropriate to irradiate the user with 1000 lux of light by the visible light source 121 when it is determined in S104 that the heartbeat interval time standard deviation of the visible light pulse wave is equal to or smaller than the fourth threshold.

  FIG. 27 is a flowchart showing the flow of processing from the light source control unit 114 after processing of FIG. 26 to the visible light source 121.

  In step S201, the light source control unit 114 determines whether or not the Flag signal received from the operation determination unit 116 is ON. If the Flag information is ON, the process proceeds to Step S203, and if the Flag information is not ON, that is, if it is OFF, the process proceeds to Step S202.

  In step S202, the light source control unit 114 determines whether the Drive signal received from the operation determination unit 116 is ON. If the Drive signal is ON, the process proceeds to step S204, and if not, the process ends.

  In step S203, the light source control unit 114 adjusts the amount of light emitted from the visible light source 121 so that the illuminance in the vehicle becomes 1000 lux, and ends the process.

  In step S204, the light source control unit 114 adjusts the amount of light in the visible light source 121 so that the illuminance in the vehicle becomes 50 lux, and ends the process.

  FIG. 28 is a flowchart showing a flow of processing for determining the amount of light emitted from the infrared light source 123 after the visible light source 121 is activated by the in-vehicle display device 10 according to the present embodiment.

  In step S300, the visible light source 121 is activated. At this time, the amount of light emitted by the visible light source 121 may be controlled so that the illuminance in the vehicle becomes 1000 lux so that a visible light pulse wave can be easily obtained.

  In step S301, the visible light waveform calculation unit 111 acquires a visible light image obtained by imaging a user irradiated with visible light by the visible light source 121 in the visible light region, and from the acquired visible light image, A visible light waveform that is a waveform indicating the user's pulse wave is extracted. The visible light waveform calculation unit 111 extracts a plurality of first feature points that are predetermined feature points in the visible light waveform. And the visible light waveform calculating part 111 calculates 1st heartbeat interval time as a feature-value of a visible light waveform. Further, the visible light waveform calculation unit 111 stores the inclination from the top to the bottom of the visible light waveform at this time in the memory as the first inclination A.

  In step S302, the infrared light waveform calculation unit 112 acquires and acquires an infrared light image obtained by capturing an image of a user irradiated with infrared light from the infrared light source 123 in the infrared light region. An infrared light waveform, which is a waveform indicating the user's pulse wave, is extracted from the infrared light image. The infrared light waveform calculation unit 112 extracts a plurality of second feature points that are predetermined feature points in the infrared light waveform. Then, the infrared light waveform calculation unit 112 calculates the second heartbeat interval time as the feature amount of the infrared light waveform.

  In step S303, the correlation degree calculation unit 113 determines a peak point. Specifically, the correlation degree calculation unit 113 determines whether or not there is an excessively acquired peak point for the first feature point extracted in the visible light waveform. Further, the correlation calculation unit 113 determines whether or not there is an excessively acquired peak point for the second feature point extracted from the infrared light waveform. Details of the peak point determination processing by the correlation degree calculation unit 113 will be described later.

  In step S304, the correlation degree calculation unit 113 calculates the degree of correlation between the visible light waveform and the infrared light waveform. The details of the correlation degree calculation processing by the correlation degree calculation unit 113 will be described later.

  In step S305, the light source control unit 114 adjusts the light amount of each light source. The light source control unit 114 outputs a control signal for controlling the light amount of each light source according to the result of adjusting the light amount. Details of the light amount adjustment processing by the light source control unit 114 will be described later.

  In step S306, the biological information calculation unit 115 calculates biological information from at least one of the feature amount of the visible light waveform and the feature amount of the infrared light waveform.

  In step S307, the biological information calculation unit 115 outputs the calculated biological information to the presentation device 40.

  FIG. 29 is a flowchart showing details of peak point excess acquisition determination processing in the present embodiment.

In step S401, the correlation degree calculation unit 113 calculates the standard deviation SD _RGB of the first heartbeat interval time.

In step S402, the correlation calculation unit 113 determines whether or not the standard deviation SD _RGB is equal to or smaller than a fourth threshold value.

In step S403, when the correlation calculation unit 113 determines that the standard deviation SD _RGB is equal to or smaller than the fourth threshold value (Yes in S402), the correlation degree calculation unit 113 calculates the standard deviation SD _IR of the second heartbeat interval time.

In step S404, the correlation calculation unit 113 determines whether or not the standard deviation _SDIR is equal to or less than the fourth threshold value.

In step S <b> 405, if the correlation degree calculation unit 113 determines that the standard deviation _SDIR is equal to or smaller than the fourth threshold (Yes in S <b> 404), it transmits a “False” signal to the light source control unit 114.

In step S406, the correlation calculation unit 113 determines that the standard deviation SD _RGB exceeds the fourth threshold (No in S402), or determines that the standard deviation SD _IR exceeds the fourth threshold ( No in S404), the absolute error e between the corresponding first heartbeat interval time and second heartbeat interval time is calculated.

  In step S407, the correlation calculation unit 113 determines whether or not the absolute error e is smaller than −200 [ms].

  In step S <b> 409, when the correlation degree calculation unit 113 determines that the absolute error e is greater than −200 [ms] (No in S <b> 407), the correlation level calculation unit 113 transmits a “False, RGB” signal to the light source control unit 114.

  On the other hand, in step S408, the correlation degree calculation unit 113 determines whether or not the absolute error e is larger than 200 [ms] when it is determined that the absolute error e is smaller than −200 [ms] (Yes in S407). To do.

  In step S <b> 410, when it is determined that the absolute error e is greater than 200 [ms] (Yes in S <b> 408), the correlation calculation unit 113 transmits a “False, IR” signal to the light source control unit 114.

  In step S411, the correlation degree calculation unit 113 transmits a “False, Both” signal to the light source control unit 114 when it is determined that the absolute error e is smaller than 200 [ms] (No in S408).

  FIG. 30 is a flowchart showing details of correlation degree calculation processing in the present embodiment.

  In step S501, the correlation degree calculation unit 113 calculates the degree of correlation between the plurality of first heart beat interval times and the plurality of second heart beat interval times.

  In step S502, the correlation degree calculation unit 113 determines whether or not the correlation degree obtained by the calculation is greater than the second threshold value.

  In step S <b> 503, when the correlation degree calculation unit 113 determines that the correlation degree is greater than the second threshold value (Yes in S <b> 502), the correlation level calculation unit 113 transmits a “True” signal to the light source control unit 114.

  In step S <b> 504, when the correlation degree calculation unit 113 determines that the correlation degree is equal to or less than the second threshold (No in S <b> 502), the correlation degree calculation unit 113 transmits a “False” signal to the light source control unit 114.

  FIG. 31 is a flowchart showing details of the light amount adjustment processing in the present embodiment.

  In step S <b> 601, the light source control unit 114 determines that the signal received from the correlation calculation unit 113 is any of “True”, “False”, “False, IR”, “False, RGB”, and “False, Both” signals. It is determined whether it is a signal.

  In step S <b> 602, when the received signal is a “True” signal, the light source control unit 114 decreases the amount of visible light and increases the amount of infrared light.

  In step S <b> 603, when the received signal is a “False” or “False, IR” signal, the light source control unit 114 increases only the amount of infrared light.

  In step S604, when the light source control unit 114 increases the amount of infrared light in step S602 or step S603, the second slope of the infrared light waveform becomes the first slope A stored in the memory. Determine whether they are equal.

  If the light source control unit 114 determines that the second inclination is equal to the first inclination (Yes in S604), the light amount adjustment processing ends.

In step S605, when the received signal is “False, RGB”, the light source control unit 114 determines whether or not the standard deviation _SDIR is less than or equal to the fourth threshold value.

If the light source control unit 114 determines that the standard deviation _SDIR is equal to or less than the fourth threshold value (Yes in S605), the light source control unit 114 performs the process of step S602.

In step S606, when the light source control unit 114 determines that the received signal is “False, Both” or that the standard deviation _SDIR is larger than the fourth threshold (No in S605), the light amount of visible light Is increased to return to the initial light amount, and the infrared light source 123 is turned off by decreasing the light amount of the infrared light.

  If the light source control unit 114 determines in step S604 that the second inclination is different from the first inclination A (No in S604) or if step S606 ends, the process returns to step S300.

  As described above, in the present embodiment, this device is used when the pulse wave cannot be obtained with visible light, for example, when the engine is started or when the ambient environment is dark during operation and is stopped. Visible light is emitted by illuminating the associated display. And the light quantity of an infrared light source is controlled so that a pulse wave can be acquired also with infrared light using the pulse wave information of visible light so that a user's pulse wave can be acquired even in the dark. Thereby, the user who is a driver can easily acquire a pulse wave not only at night but also in the shadow of a building or a tunnel in the daytime.

  As described above, the in-vehicle display device according to the present embodiment calculates the light amount of an infrared light source suitable for measuring a person's pulse wave, and irradiates the person with the calculated amount of infrared light. Get the pulse wave. In this way, the in-vehicle display device can acquire a user's pulse wave in a non-contact manner in the vehicle.

  Further, the in-vehicle display device controls the amount of infrared light applied to the person based on the magnitude of the correlation between the visible light pulse wave and the infrared light pulse wave. This makes it possible to change the amount of infrared light while maintaining a state in which there is a correlation between the visible light pulse wave and the infrared light pulse wave, and as a result, a light amount appropriate for acquiring a human pulse wave. It is possible to irradiate a person with infrared rays.

  In addition, the vehicle-mounted display device irradiates a person with visible light and infrared light when appropriate depending on whether the vehicle is in operation or the like, and provides an appropriate amount of light for pulse wave measurement using infrared light. Can be determined. Specifically, it is possible to perform processing for determining the amount of infrared light to be irradiated to a person at the time of starting the engine.

  In addition, the in-vehicle display device measures the correlation between the visible light pulse wave and the infrared light pulse wave while reducing the light amount of the visible light applied to the person, and provides an appropriate light amount for measuring the pulse wave by the infrared light. Can be determined.

  Moreover, when determining the appropriate amount of infrared light, the in-vehicle display device can suppress the human being feeling dazzled by irradiating the human eye with visible light by controlling the amount of light.

  In addition, when the on-vehicle display device determines an appropriate amount of infrared light, it suppresses that the person feels dazzling by irradiating human eyes with visible light by controlling the attitude of the display screen. Can do.

  In addition, the in-vehicle display device can determine an appropriate amount of light for pulse wave measurement using infrared light while the vehicle is stopped.

(Embodiment 2)
In the first embodiment, the example in which the user himself / herself controls the attitude of the in-vehicle display device has been described. In the present embodiment, the in-vehicle display device indicates the state of the vehicle or the user, for example, whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle, or the position of the seat that changes depending on the height or sitting height of the user. The visible light source, the visible light imaging unit, the infrared light source, or the infrared imaging unit is controlled accordingly.

  FIG. 32 is a block diagram showing a configuration of the in-vehicle display device 20 in the present embodiment. FIG. 33 is an external view of the in-vehicle display device 20 according to the present embodiment. In the present embodiment, components different from those in the first embodiment will be mainly described. The same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof may be omitted.

  As shown in FIG. 32, the in-vehicle display device 20 includes a visible light source 121, a visible light imaging unit 122, an infrared light source 123, an infrared light imaging unit 124, a visible light waveform calculation unit 111, and red. An external light waveform calculation unit 112, a correlation degree calculation unit 113, a light source control unit 114, a biological information calculation unit 115, an operation determination unit 116, a determination unit 131, and a drive unit 132 are provided. As in the first embodiment, the visible light source 121 is provided in the display screen 104.

  FIG. 33 shows an example of the appearance of the in-vehicle display device 20. As shown in FIG. 33, the in-vehicle display device 20 has a housing, and each component shown in FIG. 32 is arranged in the housing.

  A visible light camera 106 and an infrared light camera 108 are arranged side by side on the top of the housing. In addition, infrared LEDs 107A and 107B are provided on both sides of the display screen 104 in the horizontal direction.

(Visible light source 221)
The basic configuration of the visible light source 221 is substantially the same as that of the visible light source 121 in the first embodiment. The difference from the visible light source 121 according to the first embodiment is that the visible light source 221 has a light emitting method depending on whether the vehicle is a right-hand drive vehicle, a left-hand drive vehicle, or a user's sitting position. To change.

  As shown in FIG. 34, when the user gets on the vehicle with the in-vehicle display device 20 attached for the first time, the determination unit 131 asks the user whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle. . Then, according to the information, the visible light source 221 emits light from only half of the display screen 104.

  For example, as shown in FIG. 35 (a), when the user selects that the vehicle is a left steering wheel, the left half area 104A of the display screen 104 has an illuminance in the vehicle of, for example, 1000 lux. Irradiate with a large amount of light. At this time, the right half area 104B of the display screen 104 does not increase the amount of light.

  On the other hand, as shown in FIG. 35B, when the user selects that the vehicle is a right-hand drive vehicle, the amount of light in the right half area 104B of the display screen 104 is increased. At this time, the left half area 104A of the display screen 104 does not increase the amount of light.

  Thus, the visible light source 221 can determine the amount of light emitted by the visible light source 221 according to the position of the vehicle handle. When light is emitted from only the left half or the right half of the display screen 104, light is emitted from a half area as compared with the case where light is emitted from the entire area of the display screen 104. There is an advantage realized by power consumption.

  Further, the visible light source 221 may detect the position of the user's face or the position of the sheet in the determination unit 131 and change the light emission amount according to the detection result. Specifically, for example, the amount of light emission may be changed by controlling the voltage applied to each LED element, or the area of the display screen 104 that emits light may be changed. For example, when the vehicle is a right-hand drive vehicle, the sitting height is low, and the light irradiation range is lower than in the general case, the visible light source 221 is displayed on the display screen 104 as shown in FIG. The amount of light in the right half and lower half region 104D may be increased. At this time, the portion of the display screen 104 excluding the region 104D (region 104C) does not increase the amount of light.

  Note that the orientation of the display screen 104 may be changed based on the handle position information. Specifically, the display screen 104 may be tilted so that the light emission direction by the display screen 104 approaches the direction facing the seat having the handle.

(Infrared light source 223)
The basic structure of the infrared light source 223 is substantially the same as that of the infrared light source 123 in the first embodiment. The difference from the infrared light source 123 in the first embodiment is that the infrared light source 223 includes two infrared LEDs 107A and 107B arranged on both sides in the horizontal direction of the display screen 104. The two infrared LEDs 107A and 107B are controlled to emit light depending on the difference in the position of the steering wheel of the vehicle. In the present embodiment, the two infrared LEDs 107A and 107B correspond to two infrared light sources.

  For example, as shown in FIG. 34, for a user who uses the vehicle-mounted display device 20 for the first time, the determination unit 131 determines whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle, and the selection is made. Accordingly, the infrared LED that emits light is determined. For example, when the vehicle is a left-hand drive vehicle, only the left infrared LED 107A is lit as shown in FIG. On the other hand, when the vehicle is a right-hand drive vehicle, only the right infrared LED 107B is lit as shown in FIG.

  In this way, as with the visible light source 221, changing the infrared light source to be used according to the user's vehicle makes the angle irradiated to the user's cheek more accurate, and the irradiation As for the amount of light to be emitted, the pulse wave in the infrared light source can be acquired with less power consumption than when irradiating from only one point.

(Determination unit 131)
The determination unit 131 is a processing unit that determines the light emission positions and light emission means of the visible light source 221 and the infrared light source 223 based on the characteristics of the vehicle or the user. Specifically, whether the vehicle is a right-hand drive vehicle, a left-hand drive vehicle, or the light emission of the visible light source 221 and the infrared light source 223 from an imaging region that changes depending on the sitting height of the user and the position of the seat. The imaging position is determined by controlling the position, the light emitting means, and the driving unit 132.

  For example, when the user first rides on a vehicle to which the in-vehicle display device 20 is attached, the display screen 104 is used to ask the user whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle. As a specific method, for example, the display screen 104 of the in-vehicle display device 20 is provided with a touch sensor, and the number of times the touch sensor is touched is counted as “Count”. Then, when the count is changed from Count = 0 to Count = 1, the user may be asked and set which handle is shown in FIG.

  Note that the touch sensor determines whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle, but is not limited to this. For example, the visible light imaging unit 122 or the infrared light imaging unit 124 may determine which handle the user is. After determining the handle position, information indicating the determination result is transmitted to the visible light source 221, the infrared light source 223, and the drive unit 132.

  Note that whether the vehicle is a right-hand drive vehicle or a left-hand drive vehicle does not change in principle if it is set once at the time of initial setting. However, the driver who gets on the vehicle may change, and the seating height or the like varies accordingly. Therefore, it is necessary to adjust the imaging position. Therefore, the determination unit 131 sets the optimal posture of the display screen 104 based on the user's sitting height, seat position, and the like, and transmits a control signal to the driving unit 132 accordingly.

  An example is shown in FIG. FIG. 37A shows a case where the seat height of the current user is lower than the seat height of the previous user and the seat position has not changed. At this time, the size of the entire face of the user reflected in the visible light imaging unit 122 or the infrared light imaging unit 124 does not change greatly, but the imaging region is only on the upper side of the face due to the difference in sitting height. Therefore, it becomes difficult to appropriately obtain the visible light pulse wave or the infrared light pulse wave. In this case, for example, the position of the eye in the user's face is recognized, and the position of the eye at the vertical coordinate position of the display is set to a predetermined position (for example, a region that is a quarter from the lower side of the display screen 104). When the position is included, the determination unit 131 transmits information to the drive unit 132 so that the coordinate position of the eye is positioned at the center position as illustrated in FIG.

(Driver 132)
The drive unit 132 is a processing unit that controls the attitude of the in-vehicle display device 20 from the signal received from the determination unit 131. For example, it includes a motor that controls the horizontal direction of the drive unit 132 and the in-vehicle display device 20, and a motor that controls the vertical direction.

  The drive unit 132 receives the determination result of the handle position by the determination unit 131, and in the current posture of the in-vehicle display device 20, the pulse wave of the user is generated by the visible light waveform calculation unit 111 or the infrared light waveform calculation unit 112. When it is determined that it cannot be acquired, the posture of the in-vehicle display device 20 is controlled so that the face area of the user can be acquired.

  For example, even if it is determined that the vehicle is a right-hand drive vehicle and only the right side of both the visible light source 221 and the infrared light source 223 is lit, the in-vehicle display device 20 faces the front, There are cases where the face is not properly illuminated and the user's pulse wave cannot be acquired. At this time, the drive unit 132 controls the in-vehicle display device 20 to move rightward when viewed from the user so that the center of the user's face matches the center of the display.

  Even if the position of the steering wheel is determined, if fine adjustment is necessary for pulse wave acquisition due to a change in the user, control information is received from the determination unit 131, and the drive unit 132 is displayed on the vehicle. The angle that determines the attitude of the device 20 is controlled. For example, as shown in FIG. 37A, when the sitting height is low, when only the upper part of the user's face is reflected in the visible light imaging unit 122 or the infrared light imaging unit 124, the determination unit 131 displays the user's eyes. Information indicating the direction and angle (for example, 30 degrees upward) is transmitted to the drive unit 132 so that the position of the vehicle is controlled at the center position, and the drive unit 132 moves the in-vehicle display device 20 upward by 30 degrees. The user's pulse wave can be acquired by the visible light region and the infrared light region.

  FIG. 38 is a flowchart showing the flow of determination processing by the determination unit 131 and the processing of the visible light emitting unit and the infrared light emitting unit according to the determination processing in the present embodiment.

  In step S701, the determination unit 131 determines whether the Count signal has changed from 0 to 1. If it is determined that the Count signal has changed from 0 to 1, the process proceeds to step S602; otherwise, the process ends.

  In step S702, the determination unit 131 asks the user about handle information. This corresponds to the case where the Count signal has changed from 0 to 1 in step S701, that is, the initial setting state.

  In step S703, the determination unit 131 determines whether the handle position information selected by the user in step S702 indicates the right handle. If it indicates the right handle, the process proceeds to step S704, and if not, the process proceeds to step S706.

  In step S <b> 704, the determination unit 131 controls only the visible light source 221 to light only the right half of the display screen 104.

  In step S605, the determination unit 131 turns on only the infrared LED 107B disposed on the right side of the infrared light source 223, and ends the process.

  In step S <b> 606, the determination unit 131 controls only the visible light source 221 to light only the left half of the display screen 104.

  In step S607, the determination unit 131 turns on only the infrared LED 107A disposed on the left side of the infrared light source 223, and the process ends.

  As described above, the in-vehicle display device 20 according to the present embodiment determines the position of the vehicle steering wheel, and accordingly, the display screen 104 changes the region where visible light is emitted, and emits infrared light. The infrared LED that changes. Thereby, even if the steering wheel position of the vehicle is different, the pulse wave of the user who is the driver can be obtained more accurately only by installing and setting the in-vehicle display device 20, and living body monitoring in the vehicle Is possible.

  As described above, the in-vehicle display device according to the present embodiment appropriately acquires the pulse wave of the user who is in the driver's seat by changing the posture of the display screen according to the position of the vehicle handle. Can do.

  In addition, the in-vehicle display device appropriately acquires the pulse wave of the user riding in the driver seat of the left-hand drive vehicle or the right-hand drive vehicle by using one of the two infrared light sources arranged on the left and right. be able to.

  Further, the in-vehicle display device can acquire a user's pulse wave in a non-contact manner in a right-hand drive vehicle.

  Further, the in-vehicle display device can acquire a user's pulse wave in a non-contact manner in a left-hand drive vehicle.

(Embodiment 3)
In the present embodiment, only the essential components of the in-vehicle display device in each of the above embodiments are shown.

  FIG. 39 is a block diagram showing a configuration of the in-vehicle display device 30 in the present embodiment.

  As shown in FIG. 39, the in-vehicle display device 30 includes a display screen 104, a visible light receiving unit 142, an infrared light receiving unit 143, an infrared light source 144, a light source control unit 145, and a pulse wave calculation. Part 146. The light source control unit 145 and the pulse wave calculation unit 146 can be realized by the CPU 101 (see FIG. 1) or the like.

  The visible light receiving unit 142 receives visible light emitted from a person who has received light emitted from the display screen 104. The infrared light receiving unit 143 receives infrared light emitted from a person who has received infrared light emitted from the infrared light source 144.

  The light source controller 145 emits infrared light from the infrared light source 144 based on the amount of visible light received by the visible light receiver 142 and the amount of infrared light received by the infrared light receiver 143. To control. The pulse wave calculation unit 146 calculates pulse wave information related to a person's pulse wave from the amount of infrared light received by the infrared light receiving unit 143 after the above control, and outputs the calculated pulse wave information.

  Thereby, the vehicle-mounted display device 30 has the same effects as the vehicle-mounted display devices 10 and 20 in the above-described embodiments.

  Each component included in the in-vehicle display device may be a circuit. These circuits may constitute one circuit as a whole, or may be separate circuits. Each of these circuits may be a general-purpose circuit or a dedicated circuit. That is, in each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component.

  Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the control method of the in-vehicle display device according to each of the above embodiments is the following program.

  That is, this program is a method for controlling an in-vehicle display device in a computer, and the in-vehicle display device receives visible light emitted from a display screen and visible light emitted from a person who has received light emitted from the display screen. A light receiving unit, an infrared light source, an infrared light receiving unit that receives infrared light emitted from a person who has received infrared light emitted from the infrared light source, and a processor. The processor uses the infrared light source based on the waveform extracted from the visible light received by the visible light receiving unit and the waveform extracted from the infrared light received by the infrared light receiving unit. Control the emission of infrared light, calculate pulse wave information related to the human pulse wave from the waveform extracted from the infrared light received by the infrared light receiving unit after the control, and calculate the pulse wave Execute a control method that outputs information

  As mentioned above, although the vehicle-mounted display apparatus etc. which concern on the one or some aspect were demonstrated based on embodiment, this indication is not limited to this embodiment. Unless it deviates from the gist of the present disclosure, various modifications conceived by those skilled in the art have been made in this embodiment, and forms constructed by combining components in different embodiments are also within the scope of one or more aspects. May be included.

  For example, in the above-described embodiment, a process executed by a specific component may be executed by another component instead of the specific component. Further, the order of the plurality of processes may be changed, and the plurality of processes may be executed in parallel.

  The present disclosure is useful as an in-vehicle display device that acquires a user's pulse wave in a vehicle without contact. Specifically, the present invention is useful for a car navigation device, a vehicle rear seat monitor device, an electronic mirror device, and a reproducing device such as music.

10, 20, 30 On-vehicle display device 101 CPU
102 Main memory 103 Storage 104, 141 Display screen 104A, 104B, 104C, 104D Area 105 Visible light LED
106 Visible light camera 107, 107A, 107B Infrared LED
DESCRIPTION OF SYMBOLS 108 Infrared light camera 111 Visible light waveform calculating part 112 Infrared light waveform calculating part 113 Correlation degree calculating part 114, 145 Light source control part 115 Biometric information calculation part 116 Operation | movement determination part 121,221 Visible light source 122 Visible light imaging part 123 223, 144 Infrared light source 124 Infrared light imaging unit 131 Determination unit 132 Drive unit 142 Visible light receiving unit 143 Infrared light receiving unit 146 Pulse wave calculation unit

Claims (13)

A display screen;
A visible light receiving unit for receiving visible light emitted from a person who has received light emitted from the display screen;
An infrared light source;
An infrared light receiving unit that receives infrared light emitted from a person who has received infrared light emitted from the infrared light source;
With a processor,
The processor is
Infrared light emission from the infrared light source based on a waveform extracted from visible light received by the visible light receiving unit and a waveform extracted from infrared light received by the infrared light receiving unit Control
Calculating pulse wave information related to the pulse wave of the person from the waveform extracted from the infrared light received by the infrared light receiving unit after the control;
An in-vehicle display device that outputs the calculated pulse wave information. The processor is
In the control of the emission of the infrared light, the feature amount of the pulse wave component included in the waveform extracted from the visible light received by the visible light receiving unit and the infrared light received by the infrared light receiving unit are extracted. The degree of correlation with the feature quantity of the pulse wave component included in the waveform to be calculated is calculated, and the control of the emission of infrared light by the infrared light source is performed based on the calculated degree of correlation. In-vehicle display device. The processor further includes:
The in-vehicle display device according to claim 2, wherein the control of the emission of infrared light by the infrared light source is performed based on a driving state of a vehicle in which the in-vehicle display device is mounted. The processor further includes:
The display screen is controlled so that visible light emitted from the person is reduced, and a feature quantity of a pulse wave component is obtained from a waveform extracted from infrared light received by the infrared light receiving unit. The in-vehicle display device according to claim 1, wherein control of the display screen is stopped. The in-vehicle display device according to any one of claims 1 to 4, wherein the processor reduces the amount of light emitted from the display screen to a predetermined amount or less by reducing the amount of light emitted from the display screen. The in-vehicle display device according to any one of claims 1 to 5, wherein the processor changes a posture of the display screen to reduce a light amount irradiated to the eyes of the person to a predetermined value or less. The said processor performs the said control of the emission of the infrared light by the said infrared light source, when the acquired said driving state is a driving | running state which shows the state which the said vehicle has stopped. In-vehicle display device. The processor further includes:
Accepting handle position information indicating whether the vehicle on which the in-vehicle display device is arranged is a right-hand drive vehicle or a left-hand drive vehicle;
The in-vehicle device according to any one of claims 1 to 7, wherein the display screen is tilted so that an emission direction of light by the display screen approaches a direction facing a seat having a handle based on the received handle position information. Display device. The in-vehicle display device includes the two infrared light sources,
The in-vehicle display device according to any one of claims 1 to 8, wherein the two infrared light sources are arranged at positions sandwiching the display screen in a horizontal direction in a front view of the display screen. The processor further includes:
The in-vehicle display device according to claim 8, wherein when the received handle position information indicates that the vehicle is a right-hand drive vehicle, only a substantially half region on the right side in front view of the display screen is caused to emit light. The processor further includes:
The in-vehicle display device according to claim 8, wherein when the received handle position information indicates that the vehicle is a left-hand drive vehicle, only the left half of the display screen in front view is caused to emit light. A control method for an in-vehicle display device,
The in-vehicle display device is
A display screen;
A visible light receiving unit for receiving visible light emitted from a person who has received light emitted from the display screen;
An infrared light source;
An infrared light receiving unit that receives infrared light emitted from a person who has received infrared light emitted from the infrared light source;
With a processor,
In the control method, the processor
Infrared light emission from the infrared light source based on a waveform extracted from visible light received by the visible light receiving unit and a waveform extracted from infrared light received by the infrared light receiving unit Control
Calculating pulse wave information related to the pulse wave of the person from the waveform extracted from the infrared light received by the infrared light receiving unit after the control;
A control method for outputting the calculated pulse wave information.   The program for making a computer perform the control method of the vehicle-mounted display apparatus of Claim 12.

Images