JP2009026125A - Emotion analysis device, emotion analysis method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an emotion analysis device which can estimate a variety of user's emotion on the basis of a variety of his or her contacting action which reflects his or her emotion. <P>SOLUTION: The emotion analysis device 1 includes a housing part 2 and a software part 3. The housing part 2 includes: a contacting member 4 which a user's hand contacts by his or her contacting action when he or she makes the contacting action which reflects his or her emotion; an image sensor 5 which detects image data relating to the contacting state of user's hand in his or her contacting action; and a pressure sensor 6 which detects pressure data relating to the contacting state of user's hand in his or her contacting action. The software part 3 is provided with: a characteristic pattern extraction part 10 which extracts a plurality of characteristic patterns on the basis of the image data and the pressure data; and a pattern analysis part 11 which executes pattern analysis for estimating the user's emotion from the plurality of characteristic patterns, by using a neural network. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an emotion analysis device having a function of extracting information related to a user's emotion from an action in which the user touches the device.

2. Description of the Related Art Conventionally, an information providing apparatus that provides information according to a driver's preference when providing various information such as route guidance and music to a driver (user) driving a vehicle is known (for example, Patent Document 1). In this conventional information providing apparatus, a user's emotion is analyzed based on a driver's (user's) reaction to the provided information, and the information provision rate is changed according to the analysis result. For example, the emotion of the user is analyzed based on the image of the user's face when the information is provided to the driver (user). Alternatively, analysis of the user's emotion is performed based on the volume (volume) of the user's voice when the information is provided to the driver (user) and the inflection of pronunciation.
Japanese Unexamined Patent Publication No. 2007-86880 (page 6-12, FIG. 1)

  However, in the conventional information providing apparatus, the emotion of the user is analyzed based on the image of the user's face when the information is provided to the driver (user), the magnitude (volume) of the user's voice, or the inflection of pronunciation. Is called. However, among the various emotions possessed by the user, there are those in which changes in the user's emotion are unlikely to appear in facial expressions or the like. With regard to such user emotions, changes in facial expressions and the like when the user emotions change are relatively small, so it is not easy to estimate the user emotions with conventional devices. In addition, the voice is easily affected by surrounding noise, and the face image is easily affected by ambient light, so the analysis performance may be deteriorated. Therefore, it is not easy to estimate the user's emotion using a conventional device in a situation where the influence of noise or ambient light is strong.

  The present invention has been made in view of the above problems, and an object thereof is to provide an emotion analysis device capable of estimating various emotions of a user.

  The emotion analysis device of the present invention is a contact member that a user contacts with a hand, and when the user performs a contact behavior that reflects the user's emotion, the contact that the user's hand contacts with the contact behavior A contact data detection means for detecting contact data relating to a contact state of the user's hand in the contact behavior, and a plurality of behavior elements included in the contact behavior based on the contact data A pattern that estimates the user's emotion from the plurality of feature patterns based on the relationship between the feature pattern and the user's emotion, as a feature quantity corresponding to Pattern analysis means for performing analysis.

  Thereby, the user's hand contact data is detected from the contact action of the user's hand against the contact member, a plurality of feature patterns are extracted from the contact data, and the user's emotion is estimated from the plurality of feature patterns. Various emotions of the user appear in the user's contact behavior. For example, when the user is frustrated, the user performs a contact action such as “tapping” the contact member. Based on various contact actions (striking, stroking, squeezing, etc.) reflecting such user's emotions, it is possible to estimate various user emotions (eg, joy, anger, sorrow, comfort).

  In the emotion analysis device of the present invention, the contact member may include a three-dimensional contact portion that allows the user to touch the hand.

  When the contact member includes a three-dimensional contact portion, the portion with which the user touches the hand has a three-dimensional shape, so that the user can be prevented from feeling that the user is operating. Many contact actions that reflect the user's emotions are unconsciously performed by the user. Therefore, the user can perform a contact action with a natural sense with respect to the three-dimensionally shaped contact portion. This three-dimensional shape includes, for example, a spherical shape, an elliptical spherical shape, an egg shape, a cubic shape, a rectangular parallelepiped shape, a cylindrical shape, a rectangular tube shape, and the like.

  In the emotion analysis device of the present invention, the three-dimensional contact portion may be a spherical contact spherical portion on which the user touches a hand.

  When a spherical contact spherical surface portion is used as the contact portion described above, since the portion with which the user contacts the hand is spherical, it is possible to suppress giving a sense that the user is operating. Many contact actions that reflect the user's emotions are unconsciously performed by the user. Therefore, the user can perform a contact action with a natural feeling on the spherical contact spherical surface portion.

  In the emotion analysis device of the present invention, the user's contact behavior is behavior in which the user makes a palm or a finger contact the contact member, and the pattern analysis means is based on the user's palm or finger contact behavior. Pattern analysis for estimating the user's emotion from the extracted feature patterns may be performed.

  Many contact actions that reflect the user's emotions are unconsciously performed by the user using palms or fingers. Therefore, the emotion of the user can be analyzed based on the contact behavior that the user performed with a natural sense on the contact member using the palm or finger.

  Further, in the emotion analysis device of the present invention, the contact behavior of the user may be a stroke of the contact spherical surface portion, stroking, squeezing, thus, rolling, pecking, repelling, or engraving a rhythm on the contact spherical surface portion. At least one may be included, and the pattern analysis means may perform a pattern analysis for estimating the user's emotion from the plurality of feature patterns extracted from each of the plurality of contact actions of the user.

  This makes it possible to extract various feature patterns from various user contact actions such as “stroking”, “tapping”, “squeezing”, “slicing”, “slicing rhythm”, “striking”, “strapping strongly”, “pecking”, “flicking”, etc. It becomes possible, and various emotions (joy, anger, sorrow, comfort, etc.) of the user can be estimated.

  The emotion analysis device of the present invention outputs a feedback reaction signal including at least one of light or vibration to the user as a feedback reaction signal according to the user's emotion estimated by the pattern analysis means. Feedback means may be provided.

  Thereby, when the user brings his / her hand into contact with the contact member, a feedback reaction signal corresponding to the user's emotion is returned to the user. The feedback reaction signal can attract the user's interest so that the user again touches the contact member.

  In addition, the emotion analysis device of the present invention includes at least one of light or vibration as a contact inducing signal for inducing a contact action of the user when the user does not touch the contact member with a hand. You may provide the contact induction means which outputs a contact induction signal with respect to the said user.

  Thereby, when the user does not touch the contact member with his / her hand, a contact inducing signal is issued to the user. With this contact inducing signal, the user can be alerted so that the user touches the contact member with his / her hand.

  In addition, the emotion analysis device of the present invention comprises learning update means for learning the personal characteristics of each user about the contact behavior and updating the relationship between the user's feature pattern and the user's emotion, The pattern analysis means may perform pattern analysis for estimating the user's emotion from the plurality of feature patterns based on the relationship between the updated feature pattern and the user's emotion.

  For example, even if the contact action is the same “tap”, the extracted feature pattern is different for different users. In such a case, by learning the personal characteristics of each user regarding the contact behavior and updating the relationship between the user's feature pattern and the user's emotion, the relationship between the feature pattern and the emotion is given to each user. Can be adapted.

  The emotion analysis device of the present invention is a network model for calculating a value of a predetermined output node from a combination of values of a plurality of input nodes, wherein the feature pattern is used as the input node, and the output node A network model storage unit that stores a network model in which a user's emotion is used may be provided, and the pattern analysis unit may estimate the user's emotion from the plurality of feature patterns using the network model.

  Thereby, a plurality of feature patterns extracted by the feature pattern extraction means are input, and pattern analysis for estimating the user's emotion is performed. As a pattern analysis method, a network model that calculates a value of a predetermined output node from a combination of values of a plurality of input nodes is used. Thus, the user's emotion can be accurately estimated from a plurality of feature patterns using the network model. As this network model, for example, a neural network or a Bayesian network is used.

  The emotion analysis method of the present invention detects contact data relating to a contact state of the user's hand in the contact behavior when the user performs a contact behavior that reflects the user's emotion on a contact member with which the user contacts the hand. Then, based on the contact data, a plurality of feature patterns are extracted as feature amounts corresponding to a plurality of behavior elements included in the contact behavior, and based on the relationship between the feature pattern and the user's emotion, Pattern analysis for estimating the user's emotion from the plurality of feature patterns is performed.

  Also by this method, various emotions (joy, anger, sorrow, comfort, etc.) of the user can be estimated based on various contact behaviors (striking, stroking, pinching, etc.) reflecting the emotions of the user.

  The emotion analysis program of the present invention is based on contact data relating to a contact state of the user's hand in the contact behavior when the user performs a contact behavior that reflects the user's emotion on a contact member that makes the hand contact. Then, in the computer, a process for extracting a plurality of feature patterns as feature amounts corresponding to a plurality of behavior elements included in the contact behavior from the contact data, and a relationship between the feature patterns and the emotion of the user. And a process of performing pattern analysis for estimating the user's emotion from the plurality of feature patterns.

  Also according to this program, various emotions (joy, anger, sorrow, comfort, etc.) of the user can be estimated based on various contact behaviors (striking, stroking, sipping, etc.) reflecting the emotions of the user. This program may cause a computer to update a network model used for pattern analysis by learning and execute processing for adapting the network model to the user.

  According to the present invention, various emotions of a user can be estimated based on various contact actions that reflect the emotions of the user.

  Hereinafter, an emotion analysis device according to an embodiment of the present invention will be described with reference to the drawings. In this embodiment, a case of an emotion analysis device used together with a vehicle navigation device or the like is illustrated. This emotion analysis device has a function of analyzing a user's emotion when various information (route guidance, music, etc.) is provided from the navigation device to the user (driver or passenger) and transmitting it to the navigation device. Yes. This function is realized by a program stored in the memory or HDD of the emotion analysis device.

  The configuration of the emotion analysis device according to the embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing the overall configuration of the emotion analysis device. FIG. 2 is an explanatory diagram showing the configuration of the casing of the emotion analysis device. FIG. 3 is a diagram illustrating an installation example of an emotion analysis device.

  As shown in FIG. 1, the emotion analysis device 1 includes a housing unit 2 and a software unit 3. The housing part 2 of the emotion analysis device 1 includes a contact member 4 that allows a user to touch the hand. The contact member 4 is provided with an image sensor 5, a pressure sensor 6, a light source 7, and a vibration motor 8. (See FIG. 2).

  When the user performs a contact action (described later) that reflects the user's emotion, the contact member 4 comes into contact with the user's hand by the contact action. As shown in FIG. 2, the contact member 4 has a substantially hemispherical contact spherical surface portion 9, and the size of the contact member 4 can be grasped by a user (for example, an adult man or an adult woman) with the palm of the hand. It is set to a size of about.

  An image sensor 5 composed of a CCD camera or the like is disposed at the approximate center below the contact member 4. The contact member 4 is transparent or translucent, and the image sensor 5 can capture the image of the contact state of the user's hand in the contact behavior by photographing the shadow of the user's hand during the contact behavior. In addition, a pressure sensor 6 made of pressure-sensitive conductive rubber or the like is disposed below the contact member 4, and the pressure applied to the contact member 4 during the contact action is measured, so that the user in the contact action Pressure data indicating the contact state of the hand can be obtained. Here, the image data and the pressure data correspond to the contact data of the present invention. Further, the image sensor 5 and the pressure sensor 6 correspond to contact data detection means of the present invention.

  A light source 7 that emits light of various luminances and colors is provided below the contact member 4. As will be described later, the light source 7 is controlled to emit light with a color, brightness, or rhythm corresponding to the analysis result of the user's emotion. The light source 7 is controlled to emit light with a color, brightness, or rhythm that induces the user's contact behavior when the user does not touch the contact member 4. This light source 7 corresponds to the feedback means of the present invention and also corresponds to the contact inducing means.

  The contact member 4 is provided with a vibration motor 8 that vibrates the contact member 4 with various strengths and rhythms. As will be described later, the vibration motor 8 is controlled to vibrate the contact member 4 with strength and rhythm according to the analysis result of the user's emotion. Further, the vibration motor 8 is controlled to vibrate the contact member 4 with such strength and rhythm that induces the user's contact behavior when the user does not touch the contact member 4 with his / her hand. The vibration motor 8 corresponds to the feedback means of the present invention and also corresponds to the contact inducing means.

  As shown in FIG. 1, the software unit 3 of the emotion analysis device 1 includes a CPU, a microcomputer, and the like, and includes a feature pattern extraction unit 10, a pattern analysis unit 11, a light source control unit 12, and a vibration control unit 13. Yes. The software unit 3 includes a network model storage unit 14 configured with a memory, an HDD, and the like, and a learning update unit 15 configured with a CPU, a microcomputer, and the like.

  Based on the image data of the image sensor 5 and the pressure data of the pressure sensor 6, the feature pattern extraction unit 10 uses various feature patterns (described later) as feature amounts corresponding to various behavior elements included in the user's contact behavior. To extract. The feature pattern extraction unit 10 corresponds to feature pattern extraction means of the present invention.

  The pattern analysis unit 11 performs pattern analysis for estimating a user's emotion from a plurality of feature patterns using a neural network (described later) representing the relationship between the feature pattern and the user's emotion. This neural network is a network model that calculates the value of an output node (user emotion) from a combination of values of a plurality of input nodes (feature patterns), and is stored in the network model storage unit 14. Here, the pattern analysis unit 11 corresponds to the pattern analysis unit of the present invention, and the network model storage unit 14 corresponds to the network model storage unit of the present invention.

  The light source control unit 12 controls the light source 7 so as to emit light with a color, brightness, or rhythm corresponding to the user's emotion estimated by the pattern analysis unit 11. Further, the light source control unit 12 controls the light source 7 to emit light with a color, brightness, or rhythm that induces the user's contact behavior when the user does not touch the contact member 4.

  The vibration control unit 13 controls the vibration motor 8 so as to vibrate the contact member 4 with the strength and rhythm according to the user's emotion estimated by the pattern analysis unit 11. Further, the vibration control unit 13 controls the vibration motor 8 so as to vibrate the contact member 4 with such strength and rhythm that induces the user's contact behavior when the user does not touch the contact member 4 with his / her hand. .

  The learning update unit 15 learns personal characteristics for each user regarding the contact behavior, and updates the network model stored in the network model storage unit 14. That is, it can be said that the learning update unit 15 performs a process of updating the network model so as to be adapted for each user. The learning update unit 15 corresponds to a learning update unit of the present invention.

  For example, even if the contact action is the same “tap”, the extracted feature pattern is different if the user is different, so that it may be erroneously recognized as “striking” depending on the user. In such a case, the recognition result is fed back to the user by the light source 7 and the vibration motor 8, the user is notified of the recognized result, and the user is inquired whether the recognition result is correct. For example, when the user tries the contact action again within a predetermined time after obtaining feedback from the light source 7 and the vibration motor 8, one or more previous contact actions are regarded as erroneous recognition. Accordingly, the network model is updated so that one or more previous contact actions are correctly recognized, and learning is performed so that the network model is more suitable for the user.

  The emotion analysis device 1 configured as described above is provided at a position where the user (driver or passenger) can easily touch the contact member 4 with a hand in the vehicle. For example, the emotion analysis device 1 is provided in the vicinity of the center console between the driver seat and the passenger seat, as shown in FIG.

(User contact behavior)
4A to 4H are explanatory diagrams illustrating examples of the user's contact behavior. FIG. 5 is an explanatory diagram showing the correlation between the user's contact behavior and the user's emotion. FIG. 4A is a diagram illustrating an example of the user's contact action “Nigiri”. In this figure, the user strongly grips the surface of the contact member 4 with his / her hand. This contact action “nigiri” is an action that is performed unconsciously when the user is happy or angry, for example. Therefore, it can be said that the contact action “Nigiri” reflects the user's emotion of “joy” or “anger” (see FIG. 5).

  FIG. 4B is a diagram showing an example of the user's contact action of “stroking”. In this figure, the user gently strokes the surface of the contact member 4 with the palm of the hand. FIG. 4C is a diagram showing an example of the user's contact action of “stroking vigorously”. In this example, the user is stroking the surface of the contact member 4 with his palm. This contact action of “stroking” or “stroking intensely” is, for example, an action that is performed unconsciously when the user is happy, sad, or happy. Therefore, it can be said that the contact behavior of “stroking” or “stroking strenuously” reflects the user's emotion of “joy”, “sorrow”, or “easy” (see FIG. 5).

  FIG. 4D is a diagram showing an example of the user's contact behavior “naguru”. In this figure, a state in which the user strongly hits the surface of the contact member 4 with a fist is shown. This contact action of “naguru” is an action performed unconsciously when the user gets angry, for example. Therefore, it can be said that the contact behavior of “naguru” reflects the user's emotion of “anger” (see FIG. 5).

  FIG. 4E is a diagram illustrating an example of the user's contact action “peck”. In this figure, the user gently taps the surface of the contact member 4 with the fingertip. This contact action “peck” is an action that is performed unconsciously when the user is sad, for example. Therefore, it can be said that the contact action “peck” reflects the user's emotion “sorrow” (see FIG. 5).

  FIG. 4F is a diagram illustrating an example of the user's contact action of “repelling”. FIG. 4F shows a state in which the user gently flicks the surface of the contact member 4 with the fingertip. This contact action “flick” is an action that is performed unconsciously when the user is sad, for example. Therefore, it can be said that the contact action “flick” reflects the user's feeling “sorrow”.

  FIG. 4G is a diagram showing an example of the user's contact action “Kaku”. In this figure, the user draws a heart mark on the surface of the contact member 4 with the fingertip. The contact action of “scratching” the heart mark is, for example, an action performed when the user is happy or happy. Therefore, it can be said that the contact action of “Kaku” with the heart symbol reflects the user's feeling of “joy” or “easy”. On the other hand, for example, the contact action of “scratching” the cross mark (× mark) is an action performed when the user is angry or sad, for example. Therefore, it can be said that the contact action of “scratching” the cross mark (× mark) reflects the user's feelings of “anger” and “sorrow”.

  FIG. 4H is a diagram illustrating an example of the user's contact action of “getting the rhythm”. In this figure, the user shows the rhythm on the surface of the contact member 4 with the finger. This contact action of “kick the rhythm” is an action that is performed unconsciously when the user is happy or happy, for example. Therefore, it can be said that the contact action of “kick the rhythm” reflects the user's feeling of “joy” or “easy” (see FIG. 5).

  FIG. 4 (i) is a diagram showing an example of the user's contact action of “tapping”. In this figure, the user taps the surface of the contact member 4 with the palm of the hand. This contact action of “tapping” is an action performed unconsciously when the user is happy or angry, for example. Therefore, it can be said that the contact action of “tapping” reflects the user's feelings of “joy” and “anger” (see FIG. 5).

  FIG. 4 (j) is a diagram showing an example of the user's contact action of “striking strongly”. In this figure, a state in which the user strikes the surface of the contact member 4 with the palm of the hand is shown. This contact action of “slamming” is an action that is performed unconsciously when the user is angry, for example. Therefore, it can be said that the contact action of “slamming” reflects the user's feeling of “anger” (see FIG. 5).

  When the user performs various contact actions as described above, image data indicating the contact state of the user's hand in the contact action is obtained by the image sensor 5, and the pressure indicating the contact state of the user's hand in the contact action Data is obtained by the pressure sensor 6.

  6A to 6D are explanatory diagrams illustrating examples of image data when various contact actions are performed. For example, when the user (passenger in the left seat) performs the contact action of “squeezing” with the right hand, image data as shown in FIG. 6A is obtained. Further, when the user (the driver in the right seat) performs the contact action of “squeezing” with the left hand, image data as shown in FIG. 6B is obtained. In this case, by analyzing the image data, it is possible to determine whether the user who performed the contact behavior is a driver or a passenger.

  Also, for example, when the user performs a contact action of “kick the rhythm” with the left hand, image data as shown in FIG. 6C is obtained. In this case, pressure data whose pressure value changes at a predetermined rhythm interval is obtained. Further, when the user performs a contact action of “pecking” with the left hand, image data as shown in FIG. 6D is obtained. In this case, pressure data in which the pressure value changes at regular time intervals is obtained.

(Feature pattern)
Various feature patterns are extracted from various contact data (image data and pressure data) as described above. For example, a feature pattern “fingertip position” is extracted from the image data of the contact behavior as a feature amount corresponding to each position (xy coordinate) of the fingertip. In addition, a feature pattern called “motion vector (x direction, y direction)” and a feature pattern called “time change period / amplitude of motion vector” are extracted as feature quantities corresponding to the movement of the entire palm from the image data of the contact action. Is done.

  In addition, various “shape feature amounts” are extracted from the contact action image data as feature amounts corresponding to the shape of the contact portion. The “shape feature amount” includes, for example, “moment feature amount” which is a low-dimensional feature amount indicating the shape of the contact portion, “perimeter length” indicating the contour length of the contact portion, and the area of the contact portion. The “area” shown is included. Further, from the image data of the contact behavior, a feature pattern “number of parts in contact” is extracted as a feature amount corresponding to the number of fingertips in contact with the contact member 4.

  On the other hand, from the pressure data of the contact behavior, a characteristic pattern of “load intensity” indicating the magnitude of the pressure, and a position where the pressure is applied (for example, the front side, the rear side, the right side, the left side, etc. of the contact member 4). A feature pattern of “loading position” is extracted. In addition, a characteristic pattern called “loading period” indicating a period when a load is periodically applied at regular time intervals is extracted from the pressure data of the contact action.

  For example, when the user performs a contact action of “stroking”, a feature pattern of “motion vector” as shown in FIG. 7A is extracted. Further, when the user performs a contact action of “stroking strenuously”, a feature pattern of “motion vector” as shown in FIG. 7B is extracted. As shown in FIGS. 7A and 7B, the period of the motion vector of “Stroke Stroke” is shorter than the period of the motion vector of “Stroke Stroke”, and the amplitude of the motion vector of “Stroke Stroke” is “ It is larger than the amplitude of the motion vector of “stroking”.

(neural network)
FIG. 8 is an explanatory diagram conceptually showing a neural network that estimates a user's emotion from a plurality of feature patterns. A neural network is a network model in which a plurality of nodes are combined in a hierarchy. In a neural network, numerical calculations are performed at the input layer node (input node), intermediate layer node (intermediate node), and output layer node (output node). Output is done.

  In the neural network shown in FIG. 8, the numerical values “A” to “F” of the feature patterns are the values of the input nodes, and the numerical values “a” to “e” of the user's contact behavior are the values of the intermediate nodes. The values of emotion “joy”, “anger”, “sorrow” and “easy” are the values of the output nodes. In FIG. 8, for convenience of explanation, the number of input nodes (feature patterns) is six, the number of intermediate nodes (contact behavior) is five, and the number of output nodes (user emotions). However, the number of nodes is not limited to this.

  In emotion analysis device 1 of the present embodiment, all feature pattern values obtained from contact data (image data and pressure data) are used as input node values. For example, as the numerical value “A” of the feature pattern, a numerical value of “fingertip position” (for example, the x coordinate or the y coordinate in the xy coordinate system with the center of the sphere as the origin) is used. Also, the value (for example, 1 second) of the “motion vector length change period” is used as the feature pattern value “B”. In addition, you may use the value of the one part feature pattern of the feature patterns obtained from contact data (image data or pressure data) as a value of an input node.

  Numerical calculation is performed on the value of the feature pattern input to the input node of the input layer, and the value of the intermediate node is determined. By this numerical calculation, numerical values indicating various contact actions performed by the user are reflected in the value of the intermediate node. For example, the numerical value “a” of the contact action means a numerical value indicating “nigiri” (for example, “1”). Further, the numerical value “b” of the contact action means a numerical value (for example, “2”) indicating “stroking”.

  A numerical calculation is performed on the value of the intermediate node of the intermediate layer, and the value of the output node is determined. By this numerical calculation, a numerical value indicating the user's emotion is reflected in the value of the output node. For example, when the user is pleased, the numerical value of “joy” of the output node is close to “1”, and the numerical values of “anger”, “sorrow”, and “easy” of the output node are close to “0”. When the user is angry, the numerical value of “angry” of the output node is close to “1”, and the numerical values of “joy”, “sorrow”, and “easy” of the output node are close to “0”. When the user is sad, the numerical value of “sorrow” of the output node is close to “1”, and the numerical values of “joy”, “anger”, and “easy” of the output node are close to “0”. When the user is having fun, the numerical value of “Easy” of the output node is close to “1”, and the numerical values of “joy”, “anger”, and “sorrow” are close to “0”.

(Feedback reaction signal, contact induction signal)
FIG. 9 is an explanatory diagram showing an example of a feedback reaction signal and a contact induction signal. As shown in FIG. 9, in the light feedback reaction signal corresponding to the user's emotion “joy”, the light color is “orange”, the light brightness is “high”, and the light emission rhythm is “lightly”. "Lightness changes." In the vibration feedback response signal corresponding to the user's emotion “joy”, the vibration strength is “strong” and the vibration rhythm is “small”.

  In the light feedback response signal corresponding to the user's emotion “anger”, the light color is “red”, the light luminance is “high”, and the light emission rhythm changes suddenly. In the vibration feedback response signal corresponding to the user's emotion “anger”, the vibration intensity is “strong” and the vibration rhythm is “suddenly”.

  Moreover, in the light feedback response signal corresponding to the user's emotion “sorrow”, the light color is “red”, the light brightness is “medium”, and the light emission rhythm “changes slowly” To do. The vibration feedback response signal corresponding to the user's emotion “sorrow” is “none”.

  In the light feedback response signal corresponding to the user's emotion “Easy”, the light color is “Red / Orange / Yellow”, the light brightness is “High”, and the light emission rhythm is “Lightly colored”. And brightness changes. " The vibration feedback response signal corresponding to the user's emotion “Raku” is “None”.

  In the light contact induction signal when inducing the user's contact behavior, the light color is “green / blue”, the light brightness is “high”, and the light emission rhythm is “lightly changes in color and brightness” " Further, in the vibration contact induction signal for inducing the user's contact behavior, the vibration intensity is “medium” and the vibration rhythm is “light”.

  An operation when the emotion analysis device 1 configured as described above is used together with the navigation apparatus will be described. Here, for example, when the navigation device provides music to the user in the automatic music selection mode, the operation when the emotion analysis device 1 analyzes the user's emotion and transmits it to the navigation device will be described.

  FIG. 10 is a flowchart for explaining operations of the emotion analysis device 1 and the navigation device when music is provided to the user in the automatic music selection mode. As shown in FIG. 10, when the automatic music selection mode is selected by the navigation device, a music piece is selected according to a predetermined probability (S1). Next, within a predetermined time in the early part of the music, the image sensor 5 is selected. Based on the image data from, it is determined whether or not the user has a contact action (S2). When it is determined that the user is not in contact with the contact member 4, music reproduction is continued as it is (S8). When it is determined that the user is in contact with the contact member 4, a feature pattern is extracted from the contact data, and the user's emotion is estimated from the feature pattern (S3). Then, a feedback reaction signal corresponding to the estimated emotion is transmitted (S4).

  Next, it is determined whether or not the user feels comfortable from the estimation result (S3) of the user (S5). When it is determined that the user does not feel comfortable, it is considered that the currently played music is not the user's preference, and thereafter the user's preference is learned by reducing the probability that this music is selected for automatic music selection. (S6). On the other hand, when it is determined that the user feels comfortable, the user is assumed to like the currently played music, and thereafter the user's preference is increased by increasing the probability that this music is selected for automatic music selection. Is learned (S7), and the currently reproduced music is continuously reproduced as it is (S8).

  Even after a predetermined time has passed since the music was reproduced by automatic music selection, the presence / absence of the user's contact action is determined based on the image data from the image sensor 5 as in the early stage of the music (S9). . When it is determined that the user is not in contact with the contact member 4, a contact induction signal is transmitted to prompt the user to touch the contact member 4 (S10). When it is determined that the user is in contact with the contact member 4, a feature pattern is extracted from the contact data, and the user's emotion is estimated from the feature pattern (S11). It is determined from the estimation result of the user's emotion whether or not the user likes the automatically selected music (S12). For example, when the user particularly likes the automatically selected music, the user performs a contact action such as “kick the rhythm in accordance with the music”, “stroking”, or “scratch a circle”. When a feature pattern is extracted from contact data of such contact behavior, and the user's emotion of “joy” or “easy” is estimated from the feature pattern, the user likes the music currently being played Thereafter, the user's preference is learned by increasing the probability that this music is selected by automatic music selection (S13). On the other hand, if the user does not particularly like the automatically selected music, learning is not performed and the current music continues to be played as it is.

  Then, the music is continuously played until the music is finished (S14). When the music ends, the next music is selected again by automatic music selection. That is, the next automatic music selection is performed (return to the start of the next automatic music selection).

  According to the emotion analysis device 1 of the present embodiment, various emotions of the user can be estimated based on various contact behaviors that reflect the emotion of the user.

  That is, in the present embodiment, contact data of the user's hand is detected from the contact action of the user's hand with respect to the contact member 4, and a plurality of feature patterns are extracted from the contact data, and the user's hand contact data is extracted from the plurality of feature patterns. Estimate emotions. Various emotions of the user appear in the user's contact behavior. For example, when the user is frustrated, the user performs a contact action such as “tapping” the contact member 4. Based on various contact actions (striking, stroking, squeezing, etc.) reflecting such user's emotions, it is possible to estimate various user emotions (eg, joy, anger, sorrow, comfort).

  Moreover, in this Embodiment, a user's emotion can be estimated with a sufficient precision from a some feature pattern using a neural network.

  Moreover, in this Embodiment, since the contact member 4 with which a user contacts a hand is spherical shape, it can suppress giving the sense that it is operating to a user. Many contact actions that reflect the user's emotions are unconsciously performed by the user. Therefore, the user can perform a contact action on the contact member 4 with a natural sense.

  In other words, when a spherical contact spherical surface is used as the contact member 4, the user can “spat”, “tap”, “niggle”, “kick”, “kick the rhythm”, “snuggle”, “tap strongly”, “peck”, “purge”, etc. Various contact actions can be performed. Thereby, various feature patterns can be extracted, and it becomes easy to estimate various emotions (joy, anger, sorrow, comfort, etc.) of the user.

  Moreover, in this Embodiment, when a user makes a hand contact the contact member 4, the feedback reaction signal according to a user's emotion will be returned to a user. This feedback reaction signal can attract the user's interest so that the user touches the contact member 4 again.

  Moreover, in this Embodiment, if the user has not made the hand contact the contact member 4, a contact induction signal will be emitted to a user. By this contact inducing signal, the user can be alerted so that the user touches the contact member 4 with his / her hand.

  Such light and vibration feedback reaction signals and contact-inducing signals can create an illusion for the user as if the device 1 has a personality. As a result, the device 1 and the car can be attached, and it is possible to make the user want to touch the device 1 many times or to get into the car again. In addition, the aesthetics of the interior of the vehicle can be improved by feedback reaction signals of light and vibration and contact induction signals. That is, it can be said that the device 1 also has a function as an interior of the vehicle.

  Hereinafter, the advantages of the present embodiment will be described in more detail by comparing with a conventional apparatus.

  In the conventional apparatus that analyzes the user's emotion based on the facial expression of the user, when the user is not facing the camera, information on the facial expression of the user cannot be obtained, and the user's emotion can be extracted. There was a problem that I could not. On the other hand, in this embodiment, even when the user is not facing the camera, the user's emotion can be estimated based on the user's contact behavior.

  In addition, a user driving a vehicle may face various directions. Further, the user may drive the vehicle wearing sunglasses or the like. In such a case, the conventional apparatus has a problem that it is difficult to sufficiently acquire information on the user's face, and analysis of the user's emotion is not easy. On the other hand, in this Embodiment, even in such a case, a user's emotion can be estimated based on a user's contact action.

  In addition, when sunlight or the like shines on the user's face from the window, the captured image of the user's face is significantly unclear in the conventional apparatus. In such a case, there is a problem that it is difficult to analyze the emotion of the user from the face image. On the other hand, in this Embodiment, even if the sunlight etc. are illuminating a user's face from a window, a user's emotion can be estimated based on a user's contact action.

  In a conventional device that analyzes the user's emotion based on the loudness (volume) of the user's voice and pronunciation, information on the volume of the user's voice (volume) and the inflection of the pronunciation when the user is not speaking Cannot be obtained, and the user's emotion cannot be extracted. On the other hand, in this Embodiment, even when the user is not talking, a user's emotion can be estimated based on a user's contact action.

  In addition, in a conventional device, when there is a lot of noise other than the user's voice in the driving car, it is difficult to obtain sufficient information on the volume of the user's voice and the inflection of the pronunciation, and as a result, the user There was a problem that analysis of emotions was not easy. On the other hand, in this Embodiment, a user's emotion can be estimated based on a user's contact action irrespective of the presence or absence of such noise.

  The embodiments of the present invention have been described above by way of example, but the scope of the present invention is not limited to these embodiments, and can be changed or modified according to the purpose within the scope of the claims. is there.

  For example, in the above description, an example in which a neural network is used as a network model for estimating a user's emotion from a plurality of feature patterns has been described. However, the scope of the present invention is not limited to this, and a network model such as a Bayesian network may be used as a network model for estimating a user's emotion from a plurality of feature patterns.

  As described above, the emotion analysis device according to the present invention has an effect that various emotions of the user can be estimated based on various contact behaviors that reflect the emotions of the user, and the vehicle navigation apparatus. Is useful together with etc.

It is a block diagram of the emotion analysis device in this Embodiment. It is explanatory drawing which shows the main structures of the housing | casing part of an emotion analysis device. It is explanatory drawing which shows the example of installation of an emotion analysis device. (A) It is explanatory drawing which shows an example of a user's contact action "Nigiri". (B) It is explanatory drawing which shows an example of a user's contact action "stroking". (C) It is explanatory drawing which shows an example of a user's contact action "striking intensely". (D) It is explanatory drawing which shows an example of a user's contact action "naguru". (E) It is explanatory drawing which shows an example of a user's contact action "pecking". (F) It is explanatory drawing which shows an example of a user's contact action "flick". (G) It is explanatory drawing which shows an example of a user's contact action "Kaku". (H) It is explanatory drawing which shows an example of a user's contact action "kick a rhythm." (I) It is explanatory drawing which shows an example of a user's contact action "tap". (J) It is explanatory drawing which shows an example of a user's contact action "striking strongly". It is explanatory drawing which shows the correlation of a user's contact action and a user's emotion. (A) It is explanatory drawing which shows an example of contact data (image data) when a user performs "nigiri" with a right hand. (B) It is explanatory drawing which shows an example of contact data (image data) when a user performs "nigiri" with a left hand. (C) It is explanatory drawing which shows an example of contact data (image data) when a user performs "kick a rhythm" with a left hand. (D) It is explanatory drawing which shows an example of contact data (image data) when a user performs "pecking" with the left hand. (A) It is explanatory drawing which shows an example of the characteristic pattern (motion vector) when a user performs "stroking". (B) It is explanatory drawing which shows an example of the characteristic pattern (motion vector) when a user performs "stroke strenuously". It is explanatory drawing of the neural network for estimating a user's emotion (output node) from a some feature pattern (input node). It is explanatory drawing which shows an example of a feedback reaction signal and a contact induction signal. It is a flowchart for operation | movement description of the emotion analysis device of this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Emotion analysis device 4 Contact member 5 Image sensor 6 Pressure sensor 7 Light source 8 Vibration motor 9 Contact spherical surface part 10 Feature pattern extraction part 11 Pattern analysis part 12 Light source control part 13 Vibration control part 14 Network model memory | storage part

Claims (11)

A contact member that a user contacts with a hand, and when the user performs a contact action that reflects the user's emotion, the contact member that the user's hand contacts with the contact action;
Contact data detection means provided on the contact member for detecting contact data relating to a contact state of the user's hand in the contact behavior;
Feature pattern extraction means for extracting a plurality of feature patterns as feature amounts corresponding to a plurality of behavior elements included in the contact behavior based on the contact data;
Pattern analysis means for performing pattern analysis to estimate the user's emotion from the plurality of feature patterns based on the relationship between the feature pattern and the user's emotion;
Emotion analysis device characterized by having   The emotion analysis device according to claim 1, wherein the contact member includes a three-dimensional contact portion that allows the user to touch the hand.   The emotion analysis device according to claim 2, wherein the three-dimensional contact portion is a spherical contact spherical portion with which the user contacts a hand. The contact action of the user is an action in which the user brings a palm or a finger into contact with the contact member,
The pattern analysis means performs pattern analysis for estimating the user's emotion from the plurality of feature patterns extracted from the contact action by the palm or finger of the user. The emotion analysis device described in Crab. The contact action of the user includes at least one of stroking, tapping, squeezing, stroking, plucking, repelling, and engraving a rhythm on the contact spherical part,
The pattern analysis unit performs pattern analysis for estimating the user's emotion from the plurality of feature patterns extracted from each of the plurality of contact actions of the user. The emotion analysis device described in Crab.   Feedback means for outputting a feedback reaction signal including at least one of light or vibration to the user as a feedback reaction signal according to the user's emotion estimated by the pattern analysis means is provided. The emotion analysis device according to claim 1.   When the user does not touch the contact member, a contact induction signal including at least one of light or vibration is output to the user as a contact induction signal for inducing the user's contact behavior The emotion analysis device according to any one of claims 1 to 6, further comprising contact inducing means. Learning personal characteristics for each of the users about the contact behavior, comprising learning update means for updating the relationship between the user's feature pattern and the user's emotions,
The pattern analysis means performs pattern analysis for estimating the user's emotion from the plurality of feature patterns based on the relationship between the updated feature pattern and the user's emotion. The emotion analysis device according to claim 7. A network model for calculating a value of a predetermined output node from a combination of values of a plurality of input nodes, wherein the feature pattern is used as the input node and the user's emotion is used as the output node Network model storage means for
The emotion analysis device according to claim 1, wherein the pattern analysis unit estimates the user's emotion from the plurality of feature patterns using the network model. When the user performs a contact action that reflects the user's feelings on the contact member that contacts the hand, contact data relating to the contact state of the user's hand in the contact action is detected,
Based on the contact data, a plurality of feature patterns are extracted as feature amounts corresponding to a plurality of behavior elements included in the contact behavior,
An emotion analysis method comprising: performing pattern analysis for estimating the user's emotion from the plurality of feature patterns based on the relationship between the feature pattern and the user's emotion. Based on contact data regarding the contact state of the user's hand in the contact behavior, when the user performs a contact behavior that reflects the user's feelings on the contact member that contacts the hand,
On the computer,
A process of extracting a plurality of feature patterns as feature quantities corresponding to a plurality of behavior elements included in the contact behavior from the contact data;
Based on the relationship between the feature pattern and the user's emotion, a process of performing pattern analysis for estimating the user's emotion from the plurality of feature patterns;
An emotion analysis program characterized by running