JP2010282148A - Image display apparatus, image display method, and image display program - Google Patents

Abstract

An image display apparatus and an image display method capable of easily displaying image information corresponding to an emotion without the need for a registration operation in which image information or the like corresponding to the emotion state is stored in association with each other in advance. An image display program is provided.
SOLUTION: State information indicating the state of a person, correspondence information storage means for storing correspondence information associated with the state information, biological information acquisition means for acquiring biological information, and biological information acquired from the biological information acquisition means A state information acquisition unit that acquires state information based on the information, a correspondence information acquisition unit that acquires correspondence information stored in association with the state information acquired by the state information acquisition unit, from a correspondence information storage unit; Image information storage means for storing the image information, image information search means for searching for at least one image information from the image information storage means based on the correspondence information acquired by the correspondence information acquisition means, and image information search means Output means for outputting the image information retrieved by the above to the display means.
[Selection] Figure 1

Description

  The present invention relates to a technical field of an image display device, an image display method, and an image display program that can display image information corresponding to an emotional state.

  2. Description of the Related Art Conventionally, an image display device that determines an emotional state from audio information and displays image information or the like corresponding to the emotional state has been developed. For example, “bowlingals” that analyze emotional states from dog calls (voice information) and display emotional states have been developed. In addition, “KOTOANA” has been developed in which a flower-type terminal incorporating an LED (light-emitting diode) expresses the emotion of the speaker in the color of light. Specifically, emotion information is analyzed from the voices (speech information) of the speakers around it, and if the speaker feels “fun”, the flower-shaped terminal feels “yellow” or “excited” By reacting like "red", the emotional state of the speaker is expressed by the color of light. Further, Patent Document 1 discloses an invention that determines an emotional state from a voice uttered by a user and displays a landscape image corresponding to the emotional state, and Patent Document 2 includes a face image corresponding to a plurality of emotional states in advance. In the invention that prepares and judges the emotional state from the voice uttered by the user and displays the face image according to the emotional state, Patent Document 3 determines the emotional state from the loudness of the voice uttered by the user. An invention for displaying a character based on an emotional state is described.

JP 2005-352154 A JP-A-2005-352892 JP 2006-48352 A

  However, in any of the inventions described in Patent Documents 1 to 3, in order to display image information corresponding to the emotional state, a registration operation in which image information corresponding to the emotional state is stored in association with the user in advance. Is necessary, and the operation is complicated and inconvenient.

  The present invention has been made in view of such a situation, and there is no need for a registration operation to store image information or the like corresponding to the emotional state in advance by the user, and image information corresponding to the emotion can be easily obtained. It is a main object to provide an image display device, an image display method, and an image display program that can be displayed.

  According to a first aspect of the present invention for solving the above problem, in an image display device that displays image information on a display means, state information indicating a person's state and correspondence information associated with the state information are stored. Corresponding information storage means, biological information acquisition means for acquiring biological information, state information acquisition means for acquiring the state information based on the biological information acquired from the biological information acquisition means, and the state information acquisition means Correspondence information acquisition means for acquiring the correspondence information stored in association with the acquired state information from the correspondence information storage means, image information storage means for storing a plurality of image information, and the correspondence information acquisition means Based on the acquired correspondence information, the image information search means for searching for at least one image information from the image information storage means, and the image information search means And output means for outputting search has been image information to said display means, characterized by having a.

  According to a second aspect of the present invention for solving the above problem, in the image display device according to the first aspect, the correspondence information storage means associates the correspondence information with the feature information about the correspondence information. The feature information stored in association with the correspondence information obtained by the correspondence information obtaining means, and the feature information is obtained from the plurality of image information stored in the image information storage means. The image information retrieval unit retrieves at least one piece of image information from the image information storage unit based on a result of comparing the feature information acquired by the feature information acquisition unit. It is characterized by doing.

  According to a third aspect of the present invention for solving the above problem, in the image display device according to the second aspect, the correspondence information is color information, and the characteristic information about the color information is color classification information. The feature information acquisition means includes color information acquisition means, acquires color classification information as feature information stored in association with the color information acquired by the color information acquisition means, and the image information Color classification information is acquired from a plurality of pieces of image information stored in a storage unit, and the image information search unit includes color classification information stored in association with the color information acquired by the color information acquisition unit, The color classification information of the plurality of image information stored in the image information storage unit is compared, and the color classification information of each image information stored in the image information storage unit is acquired by the color information acquisition unit Was Wherein the retrieving image information is within a predetermined range from the image information storage means as compared to the color classification information stored in association with the information.

  The invention according to claim 4 for solving the above problem is the image display device according to claim 3, wherein the color information is a color that effectively works on a human emotional state indicated by the state information, or It is information which shows the color which represents the emotional state of the person which the said state information shows.

  According to a fifth aspect of the invention for solving the above problem, in the image display device according to the second aspect of the invention, the correspondence information is related information, and feature information about the related information is attached to the image information. The identification information as the characteristic information stored in association with the related information acquired by the previous related information acquisition means, and the characteristic information acquisition means has related information acquisition means, and The identification information is acquired from a plurality of pieces of image information stored in the image information storage means, and the image information search means is stored in association with the related information acquired by the related information acquisition means. And the identification information attached to each of the plurality of image information stored in the image information storage means, and from among each image information stored in the recording image information storage means, Above Wherein the retrieving the image information identification information is attached that matches the identification information stored in association with the relevant information acquired by the communication information acquiring means from the image information storage unit.

  The invention according to claim 6 for solving the above problem is the image display device according to claim 5, wherein the related information is information that effectively works on a person's emotional state indicated by the state information, or It is information expressing the emotional state of the person indicated by the state information.

  The invention according to claim 7 for solving the above problem is the image display device according to any one of claims 1 to 6, wherein the biological information is audio information, and the correspondence information storage is performed. The means further includes voice feature information acquisition means for storing the state information and word information related to the state information in association with each other, and acquiring word information included in the voice information acquired by the biological information acquisition means. And the status information acquisition means includes the status information based on the result of comparing the word information acquired by the voice feature information acquisition means and the word information stored in the correspondence information storage means. It is obtained from the correspondence information storage means.

  The invention according to claim 8 for solving the above problem is the image display device according to any one of claims 1 to 6, wherein the biological information is voice information, and the correspondence information storage is performed. The means stores personal information for identifying an individual and basic voice information corresponding to the personal information in association with each other, and the status information acquisition means is voice information for a certain period of time acquired by the biological information acquisition means. The state information is acquired based on change information over time and the basic voice information.

  The invention according to claim 9 for solving the above problem is the image display device according to claim 8, wherein the correspondence information storage means has at least one of loudness, height, and speed of voice. The basic voice information is stored.

  The invention according to claim 10 for solving the above problem is the image display device according to any one of claims 1 to 6, wherein the biological information is face image information, and the correspondence information. The storage unit stores the state information and the facial feature information related to the state information in association with each other, and acquires facial feature information included in the facial image information acquired by the biological information acquisition unit. An acquisition unit, wherein the state information acquisition unit is based on a result of comparing the facial feature information acquired by the facial feature information acquisition unit and the facial feature information stored in the correspondence information storage unit. The state information is obtained from the correspondence information storage means.

  The invention described in claim 11 for solving the above problem is the image display device according to claim 10, wherein the biological information is face image information, and the correspondence information storage means is an individual for identifying an individual. Information and basic face image information corresponding to the personal information are stored in association with each other, and the state information acquisition means is change information over time of the face image information acquired by the biological information acquisition means for a certain period of time. And acquiring the state information based on the basic face image information.

  The invention according to claim 12 for solving the above-described problem is an image display method for displaying image information on a display means, state information indicating a person's state, correspondence information associated with the state information, Corresponding information storage step for storing biometric information, biometric information acquisition step for acquiring biometric information, status information acquisition step for acquiring the status information based on the acquired biometric information, and the acquired status information in association with each other A correspondence information acquisition step for acquiring the stored correspondence information; an image information storage step for storing a plurality of pieces of image information; and image information for searching for at least one piece of image information based on the acquired correspondence information. A search process; and an output process for outputting the searched image information.

  The invention according to claim 13 for solving the above-described problem is a correspondence information storage means for storing state information indicating the state of a person, correspondence information associated with the state information, and biological information for acquiring biological information. The correspondence information stored in association with the state information acquired by the acquisition means, the state information acquisition means for acquiring the state information based on the biological information acquired from the biological information acquisition means, and the state information acquisition means Based on the correspondence information acquired by the correspondence information acquisition means, the image information storage means for storing a plurality of pieces of image information, the display means for displaying the image information, and the correspondence information acquisition means. Image information search means for searching for at least one image information from the image storage means, and image information searched by the image information search means for the display means Characterized in that to function as an output unit for outputting.

  According to the invention described in claims 1, 12 and 13, there is no need for a registration operation for storing image information or the like corresponding to the emotional state in advance by the user, and image information corresponding to the emotional state can be easily obtained. Can be displayed.

  According to the second aspect of the present invention, the correspondence information and the feature information are stored in association with each other, so that the correspondence information according to the user's state information is acquired and then the feature information according to the correspondence information. Since the image information can be acquired via the image information, the image information more suitable for the emotional state of the user can be acquired and displayed.

  According to the third and fourth aspects of the invention, the color information and the color classification information are stored in association with each other, so that the color information corresponding to the user's state information is acquired and then the color information is determined. Since the image information can be acquired through the color classification information, the image information more suitable for the emotional state of the user can be acquired and displayed.

  According to the fifth and sixth aspects of the invention, the related information and the drawing number information are stored in association with each other, and after acquiring the related information according to the state information of the user, the related information is responded. Since the image information can be acquired through the figure number information, the image information more suitable for the emotional state of the user can be acquired and displayed.

  According to the seventh aspect of the present invention, the image information can be searched based on the word information included in the voice information issued by the user of the image display device, so that the image information according to the emotional state of the user Can be displayed appropriately.

  According to the eighth and ninth aspects of the present invention, since it is possible to search for image information based on change information over time of audio information issued by the user of the image display device, a change in the emotional state of the user It is possible to appropriately display image information according to the above.

  According to the tenth aspect of the present invention, the image information can be searched based on the user's face image information, so that the image information corresponding to the emotional state of the user can be appropriately displayed.

  According to the eleventh aspect of the present invention, the image information can be searched based on the change information due to the time change of the user's face image information. Can be displayed.

It is a block diagram which shows an example of schematic structure of the image display apparatus 1 which concerns on this embodiment. It is a figure which shows a word information database. It is a figure which shows a corresponding | compatible information database. It is a figure which shows a color information database. It is a figure which shows a related information database. It is a figure which shows an image information database. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 1st embodiment. It is a figure which shows an audio | voice feature information database. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 2nd embodiment. It is a figure which shows a basic audio | voice information database. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 3rd embodiment. It is a figure which shows a face characteristic information database. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 4th embodiment. It is a figure which shows a basic face characteristic information database. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 5th embodiment. It is a figure which shows a basic biometric feature information database. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 6th embodiment. It is a flowchart which shows an example of the image display process performed by the control part in the image display apparatus 1 which concerns on 7th embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
With reference to FIG. 1, the configuration and function of an image display apparatus 1 according to an embodiment of the present invention will be described.

  FIG. 1 is a block diagram illustrating an example of a schematic configuration of an image display apparatus 1 according to the present embodiment.

  As shown in FIG. 1, the image display device 1 includes a microphone 11, a camera 12, a storage unit (an example of biological information storage means, correspondence information storage means, and image information storage means) 13, a CPU 15, a RAM 16, a ROM 17, and the like. A control unit 14, a display (an example of display means) 18, a communication unit 19, an external storage unit 21, a timer 22 and the like are provided. As such an image display device 1, for example, a personal computer, a digital photo frame device, various portable devices, and the like can be applied.

  For example, the microphone 11 collects sound emitted by the user of the image display device and outputs the sound to the control unit 14 as sound information. The camera 12 captures an image of a user's face or the like of the image display device and outputs it to the control unit 14 as user image information. The display 18 is, for example, a touch panel type color liquid crystal display.

  The storage unit 13 includes, for example, a nonvolatile semiconductor memory (for example, flash ROM), a hard disk drive, and the like, and stores an OS (Operating System), an image display program of the present invention, an application program, and the like.

  Examples of application programs stored in the storage unit 13 include an image display program that displays images, and a slide show program that defines display processing of a slide show that is displayed while switching images at predetermined time intervals.

  The word information database 10, the correspondence information database 20, and the color information database used for executing the image display program stored in the storage unit 13 and searching for image information corresponding to the emotional state of the user of the image display device. 30, a related information database 40 and an image information database 50 are constructed. The contents of each database will be described below.

<Word information database 10>
As shown in FIG. 2, the word information database 10 stores state information and word information related to the state information in association with each other.

  The state information is, for example, information indicating “anger”, “joy”, “sadness” or the like indicating the emotional state of a person. The word information is, for example, information such as “mukatsu”, “happy”, “sadness”, etc., indicating a phrase associated with the state information. In association with the state information, for example, “anger”, the word information “Mukatsu”, “I hate”, and “I am angry” regarding the state information “anger” are stored. In addition, state information, for example, word information “joy” and “smile” relating to the state information “joy” is stored in association with “joy”. In addition, state information, for example, word information “sadness” and “disillusion” regarding the state information “sadness” is stored in association with “sadness”.

<Correspondence information database 20>
As illustrated in FIG. 3, the correspondence information database 20 stores state information, color information as an example of correspondence information, and related information in association with each other.

  The state information is the same as that described above. Correspondence information is information on the state information, and color information as an example of correspondence information is, for example, a color that works effectively on the emotional state of the person indicated by the state information, or a person indicated by the state information It is the information which shows the color etc. which express an emotion state. The color information that indicates the color that works effectively on the emotional state of a person is “blue” (color information for calming anger), “green” (color information that relaxes joy and excitement), and “orange” (sadness) Color information that relaxes the color). The color information that indicates the color that represents the emotional state of a person is “black” (color information that represents anger), “red” (color information that represents joy and excitement), and “yellow” (color information that represents sadness). Etc. FIG. 2 shows color information “blue” in association with the state information, for example, “anger”, color information “red” in association with the state information “joy”, and further, the state information “sadness”. The color information “yellow” is stored in association with “”. The related information is, for example, information that works effectively on the emotional state of the person indicated by the state information, or information that expresses the emotional state of the person indicated by the state information. In addition, as described above, information other than color information indicating a color that effectively works on the emotional state of the person indicated by the state information, or color information indicating a color expressing the emotional state of the person indicated by the state information Information. Related information that works effectively on human emotional states are "sea", "sky" (related information to calm down anger), "mountain", "grass" (related information to relieve joy and excitement), “Animals” (related information to relieve sadness) and the like. Related information that expresses the emotional state of a person is “lightning”, “volcano” (related information that shows anger), “smile”, “festival” (related information that shows joy and excitement), “rain” (sadness) Related information). FIG. 2 is related to the state information, for example, “anger”, and related information “sea”, “sky”, state information “joy”, related information “smile”, “festival”, This indicates that the related information “animal” is stored in association with the state information “sadness”.

<Color information database 30>
As shown in FIG. 4, the color information database 30 stores color information and color classification information (an example of feature information) related to the color information in association with each other.

  The color information is the same as that described above. The color classification information is information indicating RGB values corresponding to the color information, for example. In association with the color information, for example, “blue”, color classification information (RGB value) recognized as a blue image, “R50 or less, G50 or less, B200 or more” is stored. In association with color information, for example, “red”, color classification information (RGB value) recognized as a red image, “R200 or more, G50 or less, B50 or less” is stored. In association with color information, for example, “yellow”, color classification information (RGB value) recognized as a yellow image, “R200 or more, G200 or more, 50 or less” is stored.

<Related information database 40>
As shown in FIG. 5, the related information database 40 stores related information and figure number information (an example of feature information) related to the related information in association with each other.

  The related information is the same as that described above. The figure number information is identification information attached to the image information, in other words, uniquely attached to the image information. The figure number information is given automatically, for example, when the image information is generated or by the input of the user of the image display device.

  In the related information database 40, for example, figure number information “FIG1”, “FIG11”,... Is stored in association with related information, “sea”, “sky”. Further, related information, for example, figure number information “Fig 2”, “Fig 12”,... Is stored in association with “smile”, “festival”. Corresponding information, for example, figure number information “FIG. 3”, “FIG. 13”,... Is stored in association with “animal”.

<Image information database 50>
As shown in FIG. 6, the image information database 50 stores image information, color classification information associated with the image information, and figure number information.

  The image information is, for example, an image (photograph) or other image information related to the user of the image display device, and a plurality of image files (for example, image files in a format such as TIFF, GIF, or JPEG) are stored. Yes. The number of colors of the image information and the image size are stored in the header of each image file.

  The color classification information is the same as that described above. The image information database 50 is a diagram showing the color classification information (RGB values) of the image information A and the figure number attached to the image information A in association with the image information A related to the user of the image display device. Number information (FIG. 1) is stored. In association with the image information B, color classification information (RGB values) of the image information B and figure number information (FIG. 2) indicating a figure number assigned to the image information B are stored. Also in the image information C, image information D,..., Image information (photographs), color classification information attached to the image information, and figure number information are stored.

  The external storage unit 21 may store image information stored in the image information database 50 constructed in the storage unit of the image display device.

  The communication unit 19 communicates with the external storage unit 21. For example, the communication unit 19 transmits an image information search request to the external storage unit 21 in response to transmission from the CPU 15, and receives image information corresponding to the image information search request from the external storage unit 21.

  The CPU 15 in the control unit 14 executes various processes according to the image display program of the present invention under execution of the OS. In particular, the CPU 15 functions as state information acquisition means, audio feature information acquisition means, correspondence information acquisition means, image information search means, face feature information acquisition means, and feature information acquisition means in the present invention in accordance with the image display program of the present invention. To do. The operation of the control unit 14 will be described in the following operation of the image display device.

  Next, the operation of the image display apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 7 is a flowchart illustrating an example of image display processing executed by the CPU 15 of the control unit 14 in the image display device 1 according to the first embodiment.

<Image display processing>
First, the CPU 15 acquires biometric information and temporarily stores it in the RAM 16 (hereinafter, “acquire” means that it is temporarily stored in the RAM 16) (step S11).

  The biometric information is, for example, audio information indicating a sound uttered by a user of the image information device or image information indicating a video such as a face of the user of the image information device. Hereinafter, the case where audio | voice information is acquired as an example of biometric information is demonstrated.

  For example, the CPU 15 acquires audio information collected by the microphone 11.

  Next, the CPU 15 acquires state information from the word information database 10 based on the acquired voice information (step S12).

  First, the CPU 15 acquires word information included in the voice information from the acquired voice information. For example, when the voice information “I hate fruits” is acquired, language analysis is performed on the voice information, and the voice information is decomposed into “fruit”, “ha”, “hate”, “is”, and the word Get information. Next, based on the result of collating the word information obtained by performing linguistic analysis and the word information stored in the word information database 10, the CPU 15 converts the word information determined to match. The state information stored in association with each other is acquired from the word information database 10. For example, the CPU 15 acquires the state information “anger” associated with the word information “dislike” from the word information database 10 as shown in FIG. 2 (similarly, the state information “joy” and “sadness”). Sometimes get.)

  Next, the CPU 15 determines whether or not the color information stored in association with the acquired state information is acquired from the correspondence information database 20 (step S13).

  For example, when the state information acquired from the word information database 10 is “anger”, the CPU 15 obtains blue (color information) associated with it from the correspondence information database 20 shown in FIG. 3. It is determined whether or not it has been acquired.

  When the color information stored in association with the acquired state information is acquired from the correspondence information database 20 (step S13: YES), the CPU 15 performs image processing based on the color information acquired from the correspondence information database 20. At least one piece of image information is searched from the information database 50 (step S14).

  First, the CPU 15 acquires color classification information stored in association with the acquired color information from the color information database 30, and obtains color classification information from a plurality of image information stored in the image information database 50. get. For example, if the color information acquired from the correspondence information database 20 is “blue”, the CPU 15 stores the color information stored in association with the color information “blue” from the color information database 30 shown in FIG. Classification information (R50 or less, G50 or less, B200 or more) is acquired, and color classification information (RGB values) stored in association with a plurality of pieces of image information is acquired from the image information database 50 shown in FIG. . Next, the CPU 15 compares the color classification information stored in association with the color information acquired from the color information database 30 and the color classification information of the plurality of image information stored in the image information database 50. Then, image information in which the color classification information of the image information stored in the image information database 50 is within a predetermined range compared to the color classification information stored in association with the acquired color information is displayed in the image information database. Search from 50. For example, the CPU 15 stores color classification information (R50 or less, G50 or less, B200 or more) stored in association with the color information “blue” acquired from the color information database 30 and the image information database 50. The color classification information (RGB value) of each image information is compared and associated with the color information “blue” acquired from the color information database 30 among the color classification information (RGB value) of each image information. Image information (for example, image information A,...) Corresponding to the color classification information within the range of the stored color classification information (R50 or less, G50 or less, B200 or more) is retrieved and acquired.

  On the other hand, when the CPU 15 does not acquire the color information stored in association with the state information acquired from the word information database 10 from the correspondence information database 20 (step S13: NO), the CPU 15 stores the word information database. It is determined whether or not the related information stored in association with the state information acquired from 10 is acquired from the correspondence information database 20 (step S15).

  For example, when the state information acquired from the word information database 10 is “anger”, the CPU 15 searches the correspondence information database 20 shown in FIG. ) Is acquired.

  When the CPU 15 acquires related information stored in association with the acquired state information from the corresponding information database 20 (step S15: YES), the CPU 15 is based on the related information acquired from the corresponding information database 20. Then, at least one piece of image information is searched from the image information database 50 (step S16).

  First, the CPU 15 acquires the figure number information stored in association with the related information acquired from the correspondence information database 20 from the related information database 40 and stores a plurality of pieces of image information stored in the image information database 50. Get figure number information from. For example, when the related information is “sea” and “sky”, the figure information (FIG. 1, FIG. 1) stored in association with the related information “sea” and “sky” from the related information database 40 shown in FIG. Fig. 11) is obtained, and from the image information database 50 shown in FIG. 6, the figure number information (FIG. 1, FIG. 11,...) Attached to each of a plurality of pieces of image information (image information A, B, C...).・) Get. Next, the CPU 15 compares the drawing number information stored in association with the related information acquired from the related information database 40 and the drawing number information of a plurality of pieces of image information stored in the image information database 50. Based on the result, at least one piece of image information is searched from the image information database 50. For example, the CPU 15 stores the figure number information (FIG 1, FIG 11,...) Stored in association with the related information “sea” and “sky” acquired from the related information database 40 and the image information database 50. The image number information of a plurality of image information is compared and stored in association with the related information acquired from the related information database 40 from among the image information stored in the image information database 50. Image information with the figure number information that matches the figure number information is retrieved and acquired.

  Next, the CPU 15 determines the order in which the searched image information is displayed (step S17), and outputs the image information to the display 18 (output step) (step S18).

  For example, when a plurality of pieces of image information are acquired, the CPU 15 may determine the display order based on a preset criterion, or may determine the display order at random. Thereafter, the CPU 15 outputs image information to the display 18, and the display 18 displays an image. Each image may be displayed on the display 18 in the slide show display process.

(Second embodiment)
Next, an image display device according to a second embodiment of the present invention will be described. The configuration and functions of the image display apparatus according to the second embodiment are basically the same as those in the first embodiment.

  In the image display device 1 according to the second embodiment, the storage unit 13 stores a voice feature information database 60 instead of the word information database 10 described in the first embodiment.

<Audio feature information database 60>
As shown in FIG. 8, the sound feature information database 60 stores state information and sound feature information related to the state information in association with each other.

  The voice feature information database 60 stores, for example, voice feature information corresponding to the state information “anger” (for example, a voice that gradually grows and includes a yelling voice) in association with the state information “anger”. The voice feature information (for example, laughter is included) corresponding to the state information “joy” is stored in association with the state information “joy”, and the state information “sadness” is associated with the state information “sadness”. Voice feature information corresponding to “sadness” (for example, voice gradually becomes smaller and crying is included) is stored.

  Next, the operation of the image display apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 9 is a flowchart illustrating an example of image display processing executed by the CPU 15 of the control unit 14 in the image display apparatus 1 according to the second embodiment.

<Image display processing>
First, the CPU 15 acquires audio information.

  For example, the timer 22 counts a certain period of time, and audio information generated during that time is acquired by the microphone 11. The CPU 15 acquires the audio information acquired by the microphone 11.

  Next, the CPU 15 acquires state information from the audio feature information database 60 based on the change information of the acquired audio information over a certain period of time and the audio feature information (step S22).

  First, the CPU 15 acquires change information over time of audio information for a certain period of time acquired by the audio information acquisition unit. For example, when the acquired voice information for a certain period of time is gradually increased with the passage of time, the CPU 15 acquires “change in voice” as the change information. Next, the CPU 15 acquires state information from the audio feature information database 60 based on a result of comparing the acquired change information and the audio feature information stored in the storage unit. For example, as illustrated in FIG. 8, the state information “anger” associated with the sound feature information “voice gradually increases” is acquired from the sound feature information database 60. When the acquired voice information for a certain period of time includes a yelling voice, the CPU 15 acquires “changing yelling is included” as change information. As shown in FIG. 8, the voice feature information database 60 acquires state information “anger” associated with the voice feature information “screams are included”. Similarly, state information “joy” and “sadness” may be acquired.

  In the process shown in FIG. 9, the processes in steps S23 to S28 are the same as the processes in steps S13 to S18 shown in FIG.

(Third embodiment)
Next, an image display apparatus according to a third embodiment of the present invention will be described. The configuration and functions of the image display apparatus according to the third embodiment are basically the same as those in the first embodiment.

  In the image display device 1 according to the third embodiment, the storage unit 13 stores a basic voice feature information database 70 instead of the word information database 10 described in the first embodiment.

<Basic voice feature information database 70>
As shown in FIG. 10, in the basic voice feature information database 70, personal information for identifying an individual and basic voice information corresponding to the personal information are stored in association with each other.

  In the basic voice feature information database 70, for example, “voice volume”, “voice pitch”, which are basic voice information corresponding to the personal information “Mr. A” in association with the personal information “Mr. A”, “Velocity of voice” is stored. Similarly, “voice volume”, “voice pitch”, and “voice speed”, which are basic voice information corresponding to the personal information “Mr. B”, are stored in association with the personal information “Mr. B”. Has been. Personal information and basic voice information corresponding to the personal information are stored in association with each other for C, D,.

  Next, the operation of the image display apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 11 is a flowchart illustrating an example of image display processing executed by the CPU 15 of the control unit 14 in the image display apparatus 1 according to the third embodiment.

<Image display processing>
First, the CPU 15 acquires audio information (step S31).

  For example, the timer 22 counts a certain period of time, and audio information generated during that time is acquired by the microphone 11. The CPU 15 acquires the audio information acquired by the microphone 11.

  Next, the CPU 15 acquires state information from the basic audio feature information database 70 based on the change information of the acquired audio information over a certain period of time and the basic audio information (step S32).

  First, the CPU 15 collates the acquired audio information S for a certain period of time with the basic audio information stored in the basic audio feature information database 70, and the basic audio information stored in the basic audio feature information database 70. Among them, basic voice information having similarity to the voice information S is acquired. For example, the CPU 15 compares the basic voice information similar to the voice information S for a certain period of time (assuming that it is Mr. A's voice information), and the voice information acquired during a certain period of time becomes voice information over time. When (voice) gradually increases, height increases, or speed increases, it is determined that Mr. A is angry and status information “anger” is acquired. In addition, the CPU 15 compares the basic voice information with the time when the volume of the voice information (voice) gradually becomes smaller, the height becomes lower, or the speed becomes slower. It is determined that Mr. A is sad and status information “sadness” is acquired. In this way, the state information is acquired by the change of the voice information with the passage of time.

  In the process shown in FIG. 11, the processes in steps S33 to S38 are the same as the processes in steps S13 to S18 shown in FIG.

(Fourth embodiment)
Next, an image display apparatus according to a fourth embodiment of the present invention will be described. The configuration and functions of the image display apparatus according to the fourth embodiment are basically the same as those in the first embodiment.

  In the image display device 1 according to the fourth embodiment, the storage unit 13 stores a face feature information database 80 instead of the word information database 10 described in the first embodiment.

<Facial feature information database 80>
As shown in FIG. 12, the face feature information database 80 stores state information and face feature information related to the state information in association with each other.

  In the face feature information database 80, for example, face feature information corresponding to the state information “anger” (for example, eyes are raised) is stored in association with the state information “anger”, and the state information “joy” is stored. The face feature information corresponding to the state information “joy” (for example, the mouth corner is raised) is stored in association with the face information corresponding to the state information “sadness” in association with the state information “sadness” ( For example, the eye is lowered) is stored.

  Next, the operation of the image display apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 13 is a flowchart illustrating an example of image display processing executed by the CPU 15 of the control unit 14 in the image display device 1 according to the fourth embodiment.

<Image display processing>
Hereinafter, a case where image information is acquired as an example of biological information will be described.

  First, the CPU 15 acquires image information (step S41).

  For example, a certain period of time is counted by the timer 22, and image information obtained during that time, particularly face image information, is acquired by the camera 12. The CPU 15 acquires the image information acquired by the camera 12.

  Next, the CPU 15 acquires state information from the face feature information database 80 based on the change information of the face image information over a certain period of time and the face feature information (step S42).

  First, the CPU 15 acquires change information over time of face image information for a certain period of time acquired by the image information acquisition unit. For example, when the acquired face image information for a certain period of time gradually increases as time passes, the CPU 15 acquires “changes in eyes” as the change information. Next, the CPU 15 acquires state information from the face feature information database 80 based on a result of comparing the acquired change information with the face feature information stored in the face feature information database 80. For example, as illustrated in FIG. 12, the CPU 15 acquires state information “anger” associated with the face feature information “eyes are lifting” from the face feature information database 80. Further, when the acquired face image information for a certain period of time gradually increases with the passage of time, the CPU 15 acquires “the mouth angle is gradually increasing” as the change information. As shown in FIG. 12, the face feature information database 80 acquires state information “joy” associated with the face feature information “mouth angle is gradually rising”. Further, when the acquired face image information for a certain period of time gradually decreases as time elapses, the CPU 15 acquires “change in eyes gradually” as change information. As shown in FIG. 12, the state information “sadness” associated with the face feature information “the eyes are gradually falling” is acquired from the face feature information database 80.

  In the process shown in FIG. 13, the processes in steps S43 to S48 are the same as the processes in steps S13 to S18 shown in FIG.

(Fifth embodiment)
Next, an image display apparatus according to a fifth embodiment of the present invention will be described. The configuration and functions of the image display apparatus according to the fifth embodiment are basically the same as those in the first embodiment.

  In the image display device 1 according to the fifth embodiment, a basic face image information database 90 is stored in the storage unit 13 instead of the word information database 10 described in the first embodiment.

<Basic facial feature information database 90>
As shown in FIG. 14, in the basic face feature information database 90, personal information for identifying an individual and basic face image information corresponding to the personal information are stored in association with each other.

  The basic face feature information database 90 stores, for example, “face image information A” in a normal state of Mr. A in association with personal information “Mr. A”. Similarly, “face image information B” in the normal state of Mr. B is stored in association with the personal information “Mr. B”. Personal information and basic face image information corresponding to personal information are also stored in association with each other for C, D,.

  Next, the operation of the image display apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 15 is a flowchart illustrating an example of image display processing executed by the CPU of the control unit in the image display apparatus 1 according to the fifth embodiment.

<Image display processing>
First, the CPU 15 acquires image information (step S51).

  For example, a certain period of time is counted by the timer 22, and image information obtained during that time, particularly face image information, is acquired by the camera 12. The CPU 15 acquires the face image information acquired by the camera 12.

  Next, the CPU 15 acquires state information from the basic face feature information database 90 based on the acquired change information of the face image information for a certain period of time and the basic face image information (step S52).

  For example, the CPU 15 collates face image information (still image) A at a certain moment of the acquired face image information for a certain period of time with basic face image information stored in the storage unit, and is stored in the storage unit. Among the basic face image information, the face image information having similarity is acquired.

  Here, since the method for determining the presence / absence of similarity is well known, detailed description thereof will be omitted. Extracted and a face graph connecting these feature points is created. Then, the ratio of the degree of coincidence between the created face graph and the face graph created in the same manner as described above based on the basic face image information (face image information A, B, C...) Is calculated, and the ratio Is more than a predetermined value, it is determined that there is similarity.

  Note that if the orientation of the face image information S and the basic face image information (face image information A, B, C...) Is different, not only the similarity is lost, but also the detection of face feature points is successful. Since the problem of being unable to do so arises, the face orientation is first estimated (for example, the face orientation is estimated from the shape of a triangle with the right eye, left eye, and mouth as vertices), and a face graph is displayed according to the orientation. The problem can be solved by converting.

  The CPU 15 obtains the face image information (moving image) for a certain period of time for a certain period of time compared to face image information (for example, face image information A) similar to the face image information (still image) S at a certain moment. When the face image information (moving image) gradually increases over time, it is determined that Mr. A is angry and the state information “anger” is acquired. Further, the CPU 15 determines that Mr. A is delighted when the face image information (moving image) gradually increases with the passage of time as compared with the basic face image information, and the state information “joy”. Is obtained. Further, the CPU 15 determines that Mr. A is sad when the face image information (moving image) gradually decreases over time as compared with the basic face image information, and the state information “sadness”. Is obtained.

  In addition, in the process shown in FIG. 15, the process of step S53-58 is the same as the process of step S13-18 shown in FIG. 7, Therefore It abbreviate | omits description.

(Sixth embodiment)
Next, an image display apparatus according to a fourth embodiment of the present invention will be described. The configuration and functions of the image display apparatus according to the sixth embodiment are basically the same as those in the first embodiment.

  In the image display device 1 according to the sixth embodiment, the basic biometric feature information database 100 is stored in the storage unit 13 instead of the word information database 10 described in the first embodiment.

<Basic biometric feature information database 100>
As shown in FIG. 16, the basic biometric feature information database 100 stores state information and basic biometric feature information related to the state information in association with each other.

  In the basic biometric feature information database 100, for example, the basic biometric feature information corresponding to the state information “anger” is associated with the state information “anger”, “the number of blinks is fast”, “pulse rate is fast”, “respiration” "Fast number" is stored, and the basic biometric information corresponding to the state information "sadness" is associated with the state information "sadness" "the number of blinks is slow", "the pulse rate is slow", "the respiratory rate is slow Is stored.

  Next, the operation of the image display device 1 according to the present embodiment will be described with reference to FIG.

  FIG. 17 is a flowchart illustrating an example of image display processing executed by the CPU of the control unit in the image display apparatus 1 according to the sixth embodiment.

<Image display processing>
First, the CPU 15 acquires image information (step S61).

  For example, a certain period of time is counted by the timer 22, and image information obtained during that time, particularly face image information, is acquired by the camera 12. The CPU 15 acquires the face image information acquired by the camera 12.

  Next, the CPU 15 acquires state information from the basic biometric feature information database 100 based on the change information of the blink information in the acquired face image information for a certain period of time and the basic biometric feature information ( Step S62).

  First, the CPU 15 acquires change information over time of face image information for a certain period of time acquired by the image information acquisition unit.

  For example, when the acquired face image information for a certain period of time gradually increases the number of blinks as time elapses, the CPU 15 acquires “the number of blinks is fast” as the change information.

  Next, the CPU 15 acquires state information from the storage unit based on a result of comparing the acquired change information and the basic biometric feature information stored in the storage unit.

  For example, the CPU 15 obtains the state information “anger” associated with the basic biometric feature information “the number of blinks is fast” from the basic biometric feature information database 100 stored in the storage unit as illustrated in FIG. 16. To do.

  In addition, when the acquired face image information for a certain period of time gradually decreases with the lapse of time, the CPU 15 acquires “change in the number of blinks” as change information. As illustrated in FIG. 14, the state information “sadness” associated with the basic biometric feature information “the number of blinks is slow” is acquired from the basic biometric feature information database 100 stored in the storage unit.

  In the process shown in FIG. 17, the processes in steps S63 to S68 are the same as the processes in steps S13 to S18 shown in FIG.

(Seventh embodiment)
Next, an image display apparatus according to a seventh embodiment of the present invention will be described. The configuration and functions of the image display apparatus according to the seventh embodiment are basically the same as those in the first embodiment.

  In the image display device 1 according to the seventh embodiment, instead of the microphone 11 and the camera 12 described in the first embodiment, a pulse rate measurement unit that measures a pulse rate (not shown) and a respiratory rate measurement unit that measures a respiratory rate are provided. I have. The pulse rate measurement unit and the respiration rate measurement unit exchange information with an interface unit (not shown) included in the image display device 1.

  In the image display device 1 according to the seventh embodiment, the basic biometric feature information database 100 is stored in the storage unit 13 instead of the word information database 10 described in the first embodiment (see the sixth embodiment). ).

  Next, the operation of the image display apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 18 is a flowchart illustrating an example of image display processing executed by the CPU of the control unit in the image display apparatus 1 according to the seventh embodiment.

<Image display processing>
First, the CPU 15 acquires biological information (step S71).

  For example, the CPU 15 acquires the pulse rate information by the pulse rate measuring unit or the respiratory rate information by the respiratory rate measuring unit.

  Next, the CPU 15 acquires state information from the basic biometric feature information database 100 based on the acquired pulse rate information or the change information of the respiratory rate information over time and the basic biometric feature information (step S72). .

  For example, the CPU 15 acquires pulse rate information for a certain period of time acquired by the pulse rate acquisition unit or the respiratory rate acquisition unit, or change information over time of the respiratory rate information. If the acquired pulse rate information or respiration rate information for a certain period of time is getting faster with the passage of time, “being fast” is acquired as feature information.

  Next, the CPU 15 acquires state information from the storage unit based on a result of comparing the acquired change information and the basic biometric feature information stored in the storage unit.

  For example, the CPU 15 acquires the state information “anger” associated with the basic biometric feature information “having been getting faster” from the basic biometric feature information database 100 stored in the storage unit as shown in FIG. .

  In addition, when the acquired pulse rate information or respiratory rate information for a certain time is gradually delayed as time elapses, the CPU 15 acquires “delayed” as change information. As illustrated in FIG. 16, the state information “sadness” associated with the basic biometric feature information “slower” is acquired from the basic biometric feature information database 100 stored in the storage unit.

  In the process shown in FIG. 18, the processes in steps S73 to S78 are the same as the processes in steps S13 to S18 shown in FIG.

  As described above, according to the embodiment of the present invention, there is no need for a registration operation to store image information or the like corresponding to the emotional state in advance by the user, and image information corresponding to the emotional state can be easily obtained. Can be displayed.

DESCRIPTION OF SYMBOLS 1 ... Image display apparatus 11 ... Microphone 12 ... Camera 13 ... Memory | storage part 14 ... Control part 15 ... CPU
16 ... RAM
17 ... ROM
18 ... Display 19 ... Communication unit 21 ... External storage unit 22 ... Timer

Claims (13)

In an image display device that displays image information on a display means,
State information indicating a person's state; correspondence information storage means for storing correspondence information associated with the state information;
Biometric information acquisition means for acquiring biometric information;
State information acquisition means for acquiring the state information based on the biological information acquired from the biological information acquisition means;
Correspondence information acquisition means for acquiring the correspondence information stored in association with the state information acquired by the state information acquisition means from the correspondence information storage means;
Image information storage means for storing a plurality of image information;
Image information search means for searching for at least one image information from the image information storage means based on the correspondence information acquired by the correspondence information acquisition means;
Output means for outputting the image information searched by the image information search means to the display means;
An image display device comprising: The image display device according to claim 1,
The correspondence information storage means stores the correspondence information and feature information related to the correspondence information in association with each other,
Feature information acquisition that acquires feature information stored in association with the correspondence information acquired by the correspondence information acquisition unit, and acquires feature information from a plurality of image information stored in the image information storage unit Further comprising means,
The image display device is characterized in that the image information search unit searches for at least one piece of image information from the image information storage unit on the basis of a result of comparing the feature information acquired by the feature information acquisition unit. The image display device according to claim 2,
The correspondence information is color information,
The feature information related to the color information is color classification information,
The feature information acquisition means includes color information acquisition means, acquires color classification information as feature information stored in association with the color information acquired by the color information acquisition means, and stores the image information storage Obtaining color classification information from a plurality of image information stored in the means,
The image information search means includes color classification information stored in association with the color information acquired by the color information acquisition means, and color classification information of a plurality of image information stored in the image information storage means. , And the color classification information of each image information stored in the image information storage means is predetermined in comparison with the color classification information stored in association with the color information acquired by the color information acquisition means. An image display apparatus that retrieves image information within the range from the image information storage means. The image display device according to claim 3,
The image display device, wherein the color information is information that indicates a color that effectively works on a person's emotional state indicated by the state information or a color that expresses a human emotional state indicated by the state information. The image display device according to claim 2,
The correspondence information is related information,
The feature information related to the related information is identification information attached to the image information,
The feature information acquisition means has related information acquisition means, acquires identification information as feature information stored in association with the related information acquired by the previous related information acquisition means, and the image information storage means To obtain identification information from a plurality of image information stored in
The image information search means is attached to each of the identification information stored in association with the related information acquired by the related information acquisition means and the plurality of image information stored in the image information storage means. Compare with the identification information,
Image information with identification information that matches the identification information stored in association with the related information acquired by the related information acquisition means from among the pieces of image information stored in the image information storage means. An image display device that searches from the image information storage means. The image display device according to claim 5,
The image display apparatus, wherein the related information is information that effectively works on a person's emotional state indicated by the state information, or information that represents a person's emotional state indicated by the state information. The image display device according to any one of claims 1 to 6,
The biological information is audio information,
The correspondence information storage means stores the state information and word information related to the state information in association with each other,
Voice feature information acquisition means for acquiring word information included in the voice information acquired by the biological information acquisition means;
The state information acquisition unit stores the state information based on a result of comparing the word information acquired by the voice feature information acquisition unit and the word information stored in the correspondence information storage unit. An image display device obtained from the means. The image display device according to any one of claims 1 to 6,
The biological information is audio information,
The correspondence information storage means stores personal information for identifying an individual and basic voice information corresponding to the personal information in association with each other,
The state information acquisition unit acquires the state information based on change information of the audio information of a certain time acquired by the biological information acquisition unit over time and the basic audio information. Display device. The image display device according to claim 8,
The correspondence information storage means stores basic voice information including at least one of loudness, height, and speed of voice. The image display device according to any one of claims 1 to 6,
The biological information is face image information,
The correspondence information storage means stores the state information and face feature information related to the state information in association with each other,
A facial feature information acquisition unit that acquires facial feature information included in the facial image information acquired by the biological information acquisition unit;
The status information acquisition unit is configured to convert the status information based on a result of comparing the facial feature information acquired by the facial feature information acquisition unit and the facial feature information stored in the correspondence information storage unit. An image display device obtained from an information storage means. The image display device according to claim 10.
The biological information is face image information,
The correspondence information storage means stores personal information for identifying an individual and basic face image information corresponding to the personal information in association with each other,
The state information acquisition unit acquires the state information based on change information of the face image information of a certain time acquired by the biological information acquisition unit over time and the basic face image information. An image display device. In an image display method for displaying image information on a display means,
A correspondence information storing step for storing state information indicating a person's state and correspondence information associated with the state information;
A biological information acquisition step of acquiring biological information;
A state information acquisition step for acquiring the state information based on the acquired biological information;
A correspondence information acquisition step of acquiring the correspondence information stored in association with the acquired state information;
An image information storage step for storing a plurality of pieces of image information;
An image information search step of searching for at least one image information based on the acquired correspondence information;
An output step of outputting the searched image information;
An image display method characterized by comprising: Correspondence information storage means for storing state information indicating a person's state and correspondence information associated with the state information;
Biometric information acquisition means for acquiring biometric information,
State information acquisition means for acquiring the state information based on the biological information acquired from the biological information acquisition means;
Correspondence information acquisition means for acquiring the correspondence information stored in association with the state information acquired by the state information acquisition means from the correspondence information storage means;
Image information storage means for storing a plurality of image information;
Display means for displaying the image information;
Image information retrieval means for retrieving at least one image information from the image storage means based on the correspondence information obtained by the correspondence information obtaining means;
An image display program that functions as output means for outputting image information searched by the image information search means to the display means.

Images