JP6276170B2 - Display device and display method - Google Patents

Description

  The present invention relates to a display device and a display method.

  Conventionally, sun visors that block ultraviolet rays are used for beauty measures when going out and for preventing direct sunlight. Further, for example, in order to minimize ultraviolet rays hitting the face, a shading sun visor that covers most of the face is known (see, for example, Non-Patent Document 1).

JP 2007-36482 A

Movable visor, [online], [searched November 26, 2014], Internet <http://item.rakuten.co.jp/missa-more/841518/>

  However, the conventional sun visor described above has a problem that the user's eyes are hidden and the user's line of sight is unknown to others. For this reason, the movement of the user cannot be predicted from the user's face, and for example, when a bicycle is driven, there is a case where safe passage cannot be performed. In addition, although the technique of the projector which displays a predetermined projection image is known (for example, refer patent document 1), it does not display a user's eyes | visual_axis etc. on a sun visor etc.

  Therefore, the present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to make it possible for another person to determine the line of sight of a user whose face is covered with a shielding object such as a sun visor.

  In order to solve the above-described problems and achieve the object, the display device according to the present invention captures an image of the face of the user whose eyes are covered from the outside by a shielding object from the inside of the shielding object. A projection unit for projecting an image relating to the eye part detected by the detection unit onto a projection plane visible from outside And a section.

  Further, the display method according to the present invention is a display method executed by a display device, and is an image capturing method for capturing an image of a user's face, whose eyes are covered from the outside by a shield, from the inside of the shield. A step of detecting a user's eye part from a face image photographed by the photographing process, and a projection for projecting an image relating to the eye part detected by the detection process onto a projection surface visible from outside And a process.

  According to the display device and the display method of the disclosure, it is possible to make another person determine the line of sight of a user whose face is covered with a shield such as a sun visor.

FIG. 1 is a diagram for explaining an example of the configuration of the display device according to the first embodiment. FIG. 2 is a diagram for explaining the process of projecting the eye image as it is on the sun visor. FIG. 3 is a diagram illustrating an example of an image projected on the sun visor. FIG. 4 is a flowchart for explaining the processing procedure of the display apparatus according to the first embodiment. FIG. 5 is a diagram for explaining an example of the configuration of the display device according to the second embodiment. FIG. 6 is a diagram illustrating a process for determining an animation image from a captured eye image. FIG. 7 is a diagram illustrating a process of projecting an animated eye image on the sun visor. FIG. 8 is a diagram illustrating an example of an image projected on the sun visor. FIG. 9 is a flowchart for explaining the processing procedure of the display apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a process of projecting an animated eye image on the front of the bicycle. FIG. 11 is a diagram illustrating a computer that executes a display program according to the present embodiment.

  Exemplary embodiments of a display device and a display method according to the present invention will be described below in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments.

[Configuration of Display Device According to Embodiment 1]
First, the configuration of the display device 10 will be described with reference to FIG. FIG. 1 is a diagram for explaining an example of the configuration of the display device according to the first embodiment. As illustrated in FIG. 1, the display device 10 includes an input unit 11, a photographing unit 12, a control unit 13, a storage unit 14, and a projection unit 15. The processing of each of these units will be described below. Note that the configuration described here is merely an example, and various other forms can be implemented.

  The input unit 11 is used to input various setting changes, and includes buttons, a touch panel, and the like. For example, setting change such as lightness and darkness of a projected image is input.

  The storage unit 14 stores data required for various processes by the control unit 13 and various processing results by the control unit 13, and includes an image information storage unit 14a. The storage unit 14 is, for example, a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), or a flash memory, or a storage device such as a hard disk or an optical disk.

  The image information storage unit 14a stores information on an image photographed by the photographing unit 12 described later, and stores information on an image extracted by the extracting unit 13b described later. Specifically, the image information storage unit 14a stores an image of a user's face whose eye portion photographed by the photographing unit 12 is covered from the outside by a shielding object (for example, a sun visor or a helmet). The image of the eye portion extracted by the extraction unit 13b is stored.

  The imaging unit 12 captures an image of the face of the user whose eyes are covered from the outside by a shielding object from the inside of the shielding object, and includes a camera, a video, and the like. For example, the photographing unit 12 photographs a moving image or a still image of the face of the user whose eyes are covered from the outside by a sun visor. Here, as illustrated in FIG. 2, the imaging unit 12 captures an image of the user's face from the inside of the sun visor 20 in order to capture the user's face hidden by the sun visor 20. And the imaging | photography part 12 stores the image | photographed image at any time in the image information storage part 14a. The present invention is not limited to the case where the user to be imaged covers the face with a sun visor, and may be applied to the case where the face is covered with a helmet, sunglasses, or the like.

  The control unit 13 has an internal memory for storing a program that defines various processing procedures and necessary data, and performs various processes by using these programs. Particularly, as closely related to the present invention, It has a detector 13a and an extractor 13b.

  The detection unit 13 a detects an eye part from the face image photographed by the photographing unit 12. Specifically, the detection unit 13a sequentially reads out user face images from the image information storage unit 14a at predetermined time intervals, identifies preset eye feature points from the read face images, and A region of the image including the eye portion is detected.

  The extraction unit 13b extracts the image of the eye portion detected by the detection unit 13a from the face image. For example, the extraction unit 13b extracts an image of a predetermined region including the eye portion detected by the detection unit 13a from the face image, and stores the extracted image in the image information storage unit 14a.

  The projection unit 15 projects an image relating to the eye portion detected by the detection unit 13a onto a projection surface that can be viewed from the outside, and includes a projector and the like. Specifically, the projection unit 15 reads the image of the eye part extracted by the extraction unit 13b from the image information storage unit 14a, and projects the image of the eye part as an image related to the eye part. For example, referring to the example of FIG. 2, the projecting unit 15 projects an image of the eye portion photographed by the photographing unit 12 onto the surface of the sun visor 20. In the example of FIG. 2, the projection unit 15 projects from below the sun visor 20. However, the present invention is not limited to this. For example, the projection unit 15 may project from above or from the sun visor 20.

  Here, the example of the image | video projected on the sun visor 20 is demonstrated using the example of FIG. FIG. 3 is a diagram illustrating an example of an image projected on the sun visor. As illustrated in FIG. 3, the projecting unit 15 projects the real-time image of the eye portion photographed by the photographing unit 12 onto the surface of the sun visor 20 as it is. Thereby, other passers-by around the user who uses the sun visor 20 knows where the user's line of sight is facing even when the user's eyes are covered with the sun visor 20. Is possible.

[Processing by display device]
Next, processing performed by the display device 10 according to the first embodiment will be described with reference to FIG. FIG. 4 is a flowchart illustrating the processing operation of the display device according to the first embodiment.

  As illustrated in FIG. 4, when the imaging unit 12 of the display device 10 receives a power-on instruction (Yes at Step S <b> 101), the image of the face of the user whose eyes are covered from the outside by the sun visor 20 is displayed on the sun visor 20. The image is taken from the inside (step S102).

  Then, the detection unit 13a detects the eye part of the user from the face image photographed by the photographing unit 12 (step S103). Subsequently, the extraction unit 13b extracts the image of the eye portion detected by the detection unit 13a from the face image (step S104).

  Then, the projection unit 15 projects an image related to the eye part on a projection surface that is visible from the outside (step S105). Specifically, the projection unit 15 projects a real-time image of the eye portion extracted by the extraction unit 13 b onto the surface of the sun visor 20.

[Effect of Example 1]
As described above, the display device 10 captures an image of the face of the user whose eye part is covered from the outside by the sun visor 20 from the inside of the sun visor 20, and the user's eye part from the captured face image. And an image relating to the detected eye part is projected onto the surface of the sun visor 20 that is visible from the outside. Thereby, the line of sight of the user whose face is covered with a shield such as a sun visor can be discriminated by others. For this reason, it is possible for a user whose face is hidden by a shield such as a sun visor to provide information on his / her own line of sight to a person or the like who faces him / her, thereby realizing safe passage and the like. Further, the display device 10 is configured to be separated from the sun visor 20, can be operated for a long time with the projection unit 15 having a large power source, and is light in weight because the photographing unit 12 does not depend on the shape of the sun visor. can do.

  By the way, in the first embodiment described above, the case of projecting a real-time image of the user's eyes has been described, but the present invention is not limited to this, and an animation corresponding to the direction of the user's eyes is projected. You may make it do.

  Therefore, in Example 2 below, the direction of the user's eye line is specified from the image of the user's eye part, an animation image corresponding to the direction of the eye line is determined, and the determined animation image is an image related to the eye part. A case where the projection processing is performed will be described. Note that the description of the same configuration and processing as in the first embodiment will be omitted.

  First, the configuration of the display device 10A according to the second embodiment will be described with reference to FIG. FIG. 5 is a diagram for explaining an example of the configuration of the display device according to the second embodiment. As illustrated in FIG. 5, the display device 10A according to the second embodiment is different from the display device 10 according to the first embodiment in having a determination unit 13c and an animation image storage unit 14b.

  The animation image storage unit 14b stores an animation image corresponding to each eye direction. For example, the animation image storage unit 14b displays the animation image corresponding to each eye direction as an eye facing front, top, bottom, left, right, top left, top right, top left, bottom right, and bottom right. Each animation image is stored. Note that the animation image storage unit 14b may store, for example, different animations of eyes for men, women, and ages as types of animation. In this case, it is assumed that the type of animation can be set and changed by user designation.

  The determination unit 13c identifies the direction of the user's line of sight from the image of the eye portion detected by the detection unit 13a, and determines an animation image corresponding to the direction of the line of sight. Specifically, the determination unit 13c determines whether the direction of the user's line of sight from the image of the eye portion is any of front, top, bottom, left, right, top left, top right, bottom left, and bottom right. Is determined, and an animation image corresponding to the direction of the line of sight is determined.

  Here, the determination unit 13c detects, for example, where the position of the black eye relative to the center of the eyeball is as a process of specifying the direction of the eye line. For example, when there is a black eye near the center of the eyeball, the determination unit 13c identifies the direction of the eye as the front, and when the black eye is on the left with respect to the center of the eyeball, the direction of the eye is left. Is identified.

  Here, the process of determining an animation image from the captured image of the eye will be described using the example of FIG. FIG. 6 is a diagram illustrating a process for determining an animation image from a captured eye image. As shown in FIG. 6, for example, when the direction of the user's line of sight is the front from the image of the eye part, an animation image in which the direction of the line of sight is the front, that is, an animation image with black eyes near the center of the eyeball. Determine (see FIG. 6A).

  Also, for example, when the direction of the user's line of sight is upward from the image of the eye part, an animation image with the line of sight up is determined, that is, an animation image with a black eye above the center of the eyeball ( (See FIG. 6B). Also, for example, when the direction of the user's line of sight is downward from the image of the eye part, an animation image with the line of sight down is determined, that is, an animation image in which the black eye is below the center of the eyeball ( (See FIG. 6C).

  Further, for example, when the direction of the user's line of sight is left from the image of the eye part, an animation image with the line of sight left is determined, that is, an animation image with black eyes on the left with respect to the center of the eyeball ( (See FIG. 6D). Also, for example, when the user's eye direction is right from the eye image, an animation image having the right eye direction is determined, that is, an animation image having a black eye on the right with respect to the center of the eyeball ( (See (E) of FIG. 6).

  Then, the projection unit 15 projects the animation image determined by the determination unit 13c onto the surface of the sun visor 20 as an image relating to the eye part. Here, as illustrated in FIG. 7, the photographing unit 12 photographs an image of the user's face from the inside of the sun visor 20 in order to photograph the user's face hidden in the sun visor 20, as in the first embodiment. To do. Then, the projection unit 15 projects the animation image of the eye part on the surface of the sun visor 20.

  Here, the example of the image | video projected on the sun visor 20 is demonstrated using the example of FIG. FIG. 8 is a diagram illustrating an example of an image projected on the sun visor. As illustrated in FIG. 8, the projecting unit 15 projects an animation image corresponding to the direction of the user's eyes on the surface of the sun visor 20. Thereby, other passers-by around the user who uses the sun visor 20 knows where the user's line of sight is facing even when the user's eyes are covered with the sun visor 20. Is possible.

  Next, processing performed by the display device 10A according to the second embodiment will be described with reference to FIG. FIG. 9 is a flowchart illustrating the processing operation of the display device according to the second embodiment.

  As illustrated in FIG. 9, when the imaging unit 12 of the display device 10 </ b> A receives a power-on instruction (Yes in step S <b> 201), the image of the face of the user whose eyes are covered from the outside by the sun visor 20 is displayed on the sun visor 20. The image is taken from the inside (step S202).

  Then, the detection unit 13a detects the eye part of the user from the face image photographed by the photographing unit 12 (step S203). Subsequently, the determination unit 13c identifies the direction of the user's line of sight from the image of the eye portion detected by the detection unit 13a (step S204), and determines an animation image corresponding to the direction of the line of sight (step S205). . Specifically, the determination unit 13c determines whether the direction of the user's line of sight from the image of the eye portion is any of front, top, bottom, left, right, top left, top right, bottom left, and bottom right. Is determined, and an animation image corresponding to the direction of the line of sight is determined.

  Then, the projecting unit 15 projects the animation image determined by the determining unit 13c onto the surface of the sun visor 20 as an image relating to the eye part (step S206). For example, the projection unit 15 projects an animation image corresponding to the direction of the user's eyes on the surface of the sun visor 20.

  As described above, in the display device 10A according to the second embodiment, the direction of the user's line of sight is specified from the image of the eye part, the animation image corresponding to the direction of the line of sight is determined, and the determined animation image is used as the eye image. Project as an image of the part. For this reason, it is possible to project the line of sight of the user whose face is covered with a shielding object such as a sun visor to others easily. Furthermore, by changing the animation setting, it is possible to display an animation image of the eye portion according to the user's preference.

  Although the embodiments of the present invention have been described so far, the present invention may be implemented in various different forms other than the embodiments described above. Therefore, another embodiment included in the present invention will be described below as a third embodiment.

[Projection plane]
In the first and second embodiments, the case where the image relating to the eye portion is projected onto the surface of the sun visor 20 has been described. However, the present invention is not limited to this, and persons around the user wearing the sun visor 20 are identified. Any projection plane can be used. For example, as illustrated in FIG. 10, a projection surface may be provided on the front surface of the bicycle, and an image relating to the eye portion may be projected onto the projection surface.

[System configuration]
Of the processes described in this embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration may be functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

[program]
The display device 10 described in the above embodiment can also be realized by executing a program prepared in advance by a computer. Therefore, in the following, an example of a computer that executes a display program that realizes the same function as the display device 10 illustrated in FIG. 1 will be described.

  FIG. 11 is a diagram illustrating a computer 1000 that executes a display program according to the present embodiment. As shown in FIG. 11, the computer 1000 includes, for example, a memory 1010, a CPU (Central Processing Unit) 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network. Interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 11, the hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. The display program according to the present embodiment is stored in, for example, the hard disk drive 1090 as a program module 1093 in which a command executed by the computer 1000 is described.

  Data used for information processing by the display program is stored as program data 1094 in, for example, the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and program data 1094 stored in the hard disk drive 1090 to the RAM 1012 as necessary, and executes various procedures.

  The program module 1093 and the program data 1094 related to the display program are not limited to being stored in the hard disk drive 1090. For example, the program module 1093 and the program data 1094 are stored in a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. May be. Alternatively, the program module and program data relating to the display program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and are executed by the CPU 1020 via the network interface 1070. It may be read out.

  These embodiments and modifications thereof are included in the invention disclosed in the claims and equivalents thereof as well as included in the technology disclosed in the present application.

[Others]
Note that the access control program described in this embodiment can be distributed via a network such as the Internet. The control program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD, and being read from the recording medium by the computer.

DESCRIPTION OF SYMBOLS 10, 10A Display apparatus 11 Input part 12 Image | photographing part 13 Control part 13a Detection part 13b Extraction part 13c Determination part 14 Storage part 14a Image information storage part 14b Animation image storage part 15 Projection part 20 Sun visor

Claims (6)

A photographing unit for photographing an image of a user's face covered from the outside by a sun visor whose eye part is a shield, from the inside of the shield;
A detection unit for detecting a part of the user's eyes from the image of the face photographed by the photographing unit;
A projection unit that projects an image relating to the eye part detected by the detection unit onto the surface of the sun visor, which is a projection surface visible from the outside;
A display device comprising: An extraction unit that extracts an image of the eye portion detected by the detection unit from the face image;
The display device according to claim 1, wherein the projection unit projects an image of an eye part extracted by the extraction unit as an image related to the eye part. A determination unit that specifies the direction of the eye line of the user from the image of the eye part detected by the detection unit, and determines an animation image corresponding to the direction of the eye line;
The display device according to claim 1, wherein the projection unit projects the animation image determined by the determination unit as an image related to the eye part. A display method executed on a display device,
A photographing step of photographing an image of a user's face covered from the outside by a sun visor whose eye part is a shield, from the inside of the shield;
A detection step of detecting a part of the user's eyes from the image of the face photographed by the photographing step;
A projection step of projecting an image relating to the eye part detected by the detection step onto the surface of the sun visor, which is a projection surface visible from the outside;
The display method characterized by including. An extraction step of extracting an image of the eye part detected by the detection step from the face image;
The display method according to claim 4 , wherein the projecting step projects the image of the eye portion extracted by the extracting step as an image relating to the eye portion. A step of determining the direction of the user's line of sight from the image of the eye portion detected by the detection step, and determining an animation image corresponding to the direction of the line of sight;
The display method according to claim 4 , wherein the projecting step projects the animation image determined by the determining step as an image relating to the eye portion.

Images