JP2013164745A - Communication terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow, when a stored image is displayed afterwards, profile information on a person in the image to be easily referred to.SOLUTION: A communication terminal includes: an image database 163 for storing image data; a profile information database 162 which, in association with person identification data for identifying a person in the image, stores profile information on the person; a face recognition collation unit 111 which refers to the profile information database 162 and identifies a person in the image; and a profile information adding unit 113 which adds the profile information on the identified person to the image data.

Description

  The present invention relates to a communication terminal apparatus having image capturing and storage functions.

  As a conventional communication terminal device, in order to easily store and manage image data acquired by shooting etc., a person appearing in the image is identified by face recognition processing, and the file name and folder name of the attribute related to that person There is known a technique for storing image data by using (see, for example, Patent Document 1).

  Also known is a technique for identifying a person by face recognition processing of a captured image and displaying the address information of the person so that the information in the address book stored in the communication terminal device can be easily used. (For example, refer to Patent Document 2).

  Furthermore, in an information processing device that recommends content such as broadcast programs to users, the user is identified using face recognition processing of captured images, and information on the preferences of a plurality of users is stored for each user. A technique for recommending content that suits the taste is also known (see, for example, Patent Document 3).

JP 2010-224901 A JP 2009-105652 A JP 2006-324809 A

  However, in the technique described in Patent Document 1, even when image data is stored with a file name having an attribute related to a person shown in the image, when the image is displayed later, the file name or the like is displayed. For other items, attributes related to the person cannot be displayed or referred to.

  In addition, in the techniques described in Patent Documents 2 and 3, face recognition processing is performed on the captured image, but the captured image itself is not stored, so that the image cannot be displayed later, The attribute related to the person cannot be displayed.

  The present invention has been made in view of such points, and its purpose is to easily display profile information related to a person included in a captured image when the captured and stored image is displayed later. It is to be able to let you.

The first invention is
A shooting section;
A captured image database for storing image data of captured images captured by the imaging unit;
A profile information database for storing profile information related to the person in association with the person specifying data for specifying the person included in the shot image based on the shot image;
With reference to the profile information database, a person specifying unit for specifying a person included in the photographed image,
A profile information adding unit for adding profile information of the identified person to the image data of the photographed image;
It is provided with.

  As a result, the image data of the photographed image is stored with the profile information added, so that when the photographed image is referred to later, the profile information related to the person included in the photographed image can be easily referred to. Can do. Therefore, for example, it is possible to easily remember the situation at the time of shooting or the person being photographed.

The second invention is
A communication terminal device according to a first invention,
The person identification data includes at least one of data of a person's face image and face information indicating a feature of the person's face extracted from the person's face image,
The person specifying unit includes at least one of face information extracted from image data of a person's face included in the person specifying data and face information included in the person specifying data, and a face extracted from the photographed image. It is configured to identify a person included in the captured image by collating with information.

  Thereby, it is possible to easily identify a person included in the captured image by collating face information based on the captured image and the person specifying data.

The third invention is
The communication terminal device according to any one of the first invention and the second invention,
The profile information adding unit adds profile information selected by the user to the image data of the photographed image when a plurality of profile information corresponding to the person specifying data is stored in the profile information database. It is configured.

  As a result, among the plurality of profile information stored in the profile information database, only what the user desires can be selectively added to the image data of the captured image.

The fourth invention is:
The communication terminal device according to any one of the first to third aspects, further comprising:
A display control unit for displaying a captured image stored in the captured image database on a display unit;
The display control unit is configured to perform display based on profile information corresponding to a person included in the photographed image in conjunction with display of the photographed image.

  Thereby, when the captured image is displayed later, the contents of the profile information related to the person included in the captured image can be easily displayed together.

The fifth invention is:
A communication terminal device according to a fourth invention,
The display control unit is configured to display an image according to the content of the profile information.

  This makes it easy to visually grasp and recognize the contents of the profile information.

The sixth invention is:
The communication terminal device according to any one of the fourth invention and the fifth invention,
The display control unit is configured to perform display based on profile information corresponding to the person selected by the user when the photographed image includes a plurality of persons.

  Thereby, even when a plurality of persons are shown in the photographed image, it is possible to easily display the contents of the profile information for each person desired by the user.

The seventh invention
The communication terminal device according to any one of the first to sixth aspects, further comprising:
Profile information update for updating the profile information added to the image data of the photographed image when the profile information stored in the profile information database is changed after being added to the image data of the photographed image It has the part.

  Thereby, it is possible to easily reflect the update of the profile information stored in the profile information database to the profile information added to the image data.

The eighth invention
A communication terminal device according to a seventh invention,
The profile information update unit maintains the state in which the profile information before update is added without being deleted from the image data of the captured image,
The profile information before and after the update can be displayed as a history.

  This makes it easy to display and confirm the profile information before and after the update.

The ninth invention
The communication terminal device according to any one of the first to eighth aspects, further comprising:
A communication unit is provided for transmitting image data of the photographed image to which the profile information is added to another device.

  As a result, the profile information is added to the image data, so other devices can refer to the profile information related to the person included in the captured image without referring to the profile information database of the transmission source device. Can do.

  According to the present invention, when a captured and stored image is displayed later, profile information related to a person included in the captured image can be easily displayed.

It is a block diagram which shows the structure of the communication terminal device 100 of embodiment of this invention. 4 is a flowchart illustrating a captured image recording operation of the communication terminal device 100. It is explanatory drawing which shows the example of a picked-up image. It is explanatory drawing which shows the example of face information and profile information. It is explanatory drawing which shows the example of a display of the selection screen of the profile information to add. It is explanatory drawing which shows the example of the image data to which profile information was added. 4 is a flowchart illustrating a captured image display operation of the communication terminal device 100. It is explanatory drawing which shows the example of the list display of a memory image. It is explanatory drawing which shows the example of a display of a memory | storage image and profile information. It is explanatory drawing which shows the example of the update of profile information. It is explanatory drawing which shows the example of a display of the updated profile information. It is explanatory drawing which shows the example of a display of the update log | history of profile information. It is explanatory drawing which shows the example of the image data transmitted to another apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Configuration of communication terminal apparatus 100)
As illustrated in FIG. 1, the communication terminal device 100 includes a control unit 110, an input unit 120, a photographing unit 130, a communication unit 140, a display unit 150, and a storage unit 160.

  The control unit 110 controls the following units.

  The input unit 120 performs an input operation for operating the communication terminal device 100 such as a touch panel and a hard key.

  The photographing unit 130 shoots an image such as a camera and generates image data. Further, it may have an image recognition function.

  The communication unit 140 is for communicating with the outside by wired or wireless communication such as so-called iC communication, Bluetooth (registered trademark) communication, infrared communication, communication using a telephone line network, or the like.

  The display unit 150 displays various information under the control of the control unit 110 such as a liquid crystal display.

  The storage unit 160 stores various information used for control of the control unit 110 such as a memory.

  Specifically, the storage unit 160 includes, for example, a face information database 161, a profile information database 162, an image database 163, and a profile auxiliary information database 164.

  The face information database 161 is a database that stores face information of a person registered in the address book, that is, information indicating facial features extracted from the face image of the person.

  The profile information database 162 is a database in which profile information related to a person registered in the address book, specifically, information such as name, telephone number, nickname, hobby, hometown, school, etc. is stored. is there. The face information database 161 and the profile information database 162 are used as, for example, a telephone book (address book).

  The image database 163 is a database in which images in the terminal are stored.

  The profile auxiliary information database 164 is a database in which auxiliary information, which is image data corresponding to the content of presumed profile information such as a hobby, is stored.

  Specifically, the control unit 110 specifically includes, for example, a face recognition / collation unit 111, a profile information quoting unit 112, a profile information adding unit 113, an information updating unit 114, an update history display unit 115, and profile assistance. An information display unit 116, an information update collation unit 117, and the like are provided.

  The face recognition collation unit 111 collates face information extracted from the photographed image photographed by the photographing unit 130 with face information obtained by referring to the face information database 161, and identifies the person being photographed. Is.

  The profile information quoting unit 112 quotes profile information related to the specified person from the profile information database 162.

  The profile information adding unit 113 adds the profile information acquired by the profile information quoting unit 112 to the image data of the photographed image.

  The information updating unit 114 updates the profile information added to the image data when the profile information stored in the profile information database 162 is updated after being added to the image data.

  The update history display unit 115 displays the profile information before and after the update added to the image data on the display unit 150 as an update history.

  The profile auxiliary information display unit 116 reads auxiliary data stored in the profile auxiliary information database 164, that is, image data corresponding to the content of the profile information, and displays the data on the display unit 150.

  The information update collation unit 117 checks whether update information is stored in the profile information database 162 of the storage unit 160.

(Operation when shooting images)
When a person registered in the telephone directory is photographed by the communication terminal apparatus 100 configured as described above, an operation as shown in FIG. 2 is performed.

  (S101) First, when the image capturing unit 130 is activated to capture an image of a person, the captured image is displayed on the display unit 150 as shown in FIG.

  (S102) The face recognition / collation unit 111 of the control unit 110 obtains facial information by extracting facial features from the human face image shown in the photographed image, and stores the facial information in the facial information database 161 as shown in FIG. Check the stored face information.

  (S103) The profile information quoting unit 112 reads profile information as shown in FIG. 4 and stored in the profile information database 162 in association with the person specified by the above collation, and displays the profile information in FIG. As shown in FIG. In the example shown in the drawing, a check box is displayed for each item, so that the user can select profile information to be added to the image data of the photographed image.

  (S104) As shown in FIG. 5B, the profile information adding unit 113 receives a user's selection operation using the check box.

  (S105) Further, as schematically shown in FIG. 6, the profile information adding unit 113 embeds the selected profile information in the image data of the photographed image and stores it in the image database 163.

(Operation when displaying images)
Next, when an image captured and stored as described above is displayed, an operation as shown in FIG. 7 is performed.

  (S201) First, an application for displaying an image, for example, called data BOX is activated.

  (S202) Therefore, as shown in FIG. 8, a list of images stored in the image database 163 (reduced display of a plurality of images) is displayed.

  (S203) Subsequently, when a user's image selection operation is accepted and a desired image is selected, normal display of the image is performed. In the following description, for example, it is assumed that an image in which profile information is embedded is selected as shown in FIG.

  (S204) Furthermore, an operation for selecting a subject for which the user wants to display profile information among one or more subjects included in the displayed image is accepted. For example, in the case of the communication terminal device 100 including a touch panel, an operation of tapping a subject is detected.

  (S205) The profile information added to the image data for the selected subject is displayed over the image as shown in FIG. 9, for example. Here, for example, hobby soccer or the like may be displayed as characters. For example, instead of displaying characters or displaying characters, the profile auxiliary information display unit 116 shows a profile as shown in FIG. An image of a soccer ball stored as auxiliary information in the auxiliary information database 164 may be displayed. By displaying such an image suitable for imagining soccer, the profile information can be easily visually recognized.

  In addition to the display as described above, an update button for instructing to update the profile information is displayed.

  (S206) Next, the operation of the update button is accepted.

  (S207) When the update button is operated, the information update collation unit 117 confirms whether the profile information stored in the profile information database 162 is updated.

  (S208) If there is no update information in the profile information database 162, the profile information added to the image data stored in the image database 163 is left as it is, and the display state of FIG. 9 is maintained.

  (S208) On the other hand, if there is update information in the profile information database 162, the information update unit 114 updates the profile information added to the image data stored in the image database 163. At this time, the information update unit 114 may overwrite the information after update on the information before update, but may add it as schematically shown in FIG. 10, for example. By leaving the information before update in this way, usually only the latest information is displayed as shown in FIG. 11 (for example, even if the hobby is changed from soccer to baseball, only baseball is displayed as a hobby. , Even if “● ● large” is added as a school of origin, “● ● large” is displayed instead of “● ● medium, ● ● high.” On the other hand, the update history is displayed as described below You can also.

  (S210) That is, when the profile information added to the image data has been updated at least once, a history button is displayed as shown in FIG.

  (S211) Then, an operation for selecting the history button is accepted.

  (S212) When the history button is operated, the update history display unit 115 displays the previous information to the latest information for each item side by side along the time axis as shown in FIG.

  As described above, it is not necessary to refer to the database or the like for the image data stored with the profile information added at the time when the image was taken, and the profile information can be easily displayed even in the offline state. Further, for example, when the image data as described above is transmitted to another communication terminal device 100 ′ or the like as shown in FIG. 13, the profile information can be displayed in the same manner. That is, since profile information is also exchanged by transmitting image data, it is easy to create new connections and communications.

  By using the communication terminal device 100 as described above, it is possible to easily improve user convenience, for example, in the following usage situation.

  That is, for example, consider a scene where a friend and two people take a commemorative photo. At this time, if the friend's current hobby can be recorded together with the photo, when he looks back at the photo a few years later, he can remember the memories of that time more clearly. In addition, if you can view the update contents of the embedded profile information and update history when displaying photos, you can also get a glimpse of the footprints of your life and help to activate communication. Here, it is troublesome to embed profile information in image information by inputting characters each time a photo is taken. Therefore, if profile information registered in the phone book in advance can be selected, it can be easily added to image information. In addition, the captured image is not only left on the terminal where the image was captured, but is also used for uploading to a social network service (SNS) or sending it to others. If such profile information is added at that time, it is possible to create a new communication that was not born by only exchanging images.

  In the above example, the face information database 161 stores information indicating facial features. However, the present invention is not limited to this, and the face image itself is stored and collated with the captured image. At this time, information indicating facial features may be extracted.

  Moreover, as long as it is information that can specify a person, not only the face but also information such as an image such as the upper body and the whole image may be used.

  Moreover, although the example in which the profile information is displayed so as to be superimposed on the image is shown, the present invention is not limited to this, and the profile information may be displayed in a separately provided display area.

  In the above example, the profile information is displayed after the subject selection operation is performed in (S204) of FIG. 7. However, when there is only one subject, (the subject is not specified). It may be displayed by a simple display instruction operation or may be automatically displayed simultaneously with the image display. Further, when a plurality of people are photographed, the profile information of all the members may be automatically displayed at the same time. Further, these displays may be selected by initial setting or the like.

  In the above example, an example in which whether the profile information in the profile information database 162 is updated after the update button is operated in (S206) in FIG. 7 has been described. The update button may be displayed only when the presence / absence of the update is confirmed when the message is displayed.

  Further, when the updated profile information is displayed in the profile information database 162, the profile information added to the image data is not immediately updated, but is updated after waiting for confirmation by the user. You may do it.

  As described above, the present invention is useful for communication terminal devices having image capturing and storage functions.

DESCRIPTION OF SYMBOLS 100 Communication terminal device 100 'Communication terminal device 110 Control part 111 Face recognition collation part 112 Profile information quotation part 113 Profile information addition part 114 Information update part 115 Update history display part 116 Profile auxiliary information display part 117 Information update collation part 120 Input part 130 photographing unit 140 communication unit 150 display unit 160 storage unit 161 face information database 162 profile information database 163 image database 164 profile auxiliary information database

Claims (9)

A shooting section;
A captured image database for storing image data of captured images captured by the imaging unit;
A profile information database for storing profile information related to the person in association with the person specifying data for specifying the person included in the shot image based on the shot image;
With reference to the profile information database, a person specifying unit for specifying a person included in the photographed image,
A profile information adding unit for adding profile information of the identified person to the image data of the photographed image;
A communication terminal device comprising: The communication terminal device according to claim 1,
The person identification data includes at least one of data of a person's face image and face information indicating a feature of the person's face extracted from the person's face image,
The person specifying unit includes at least one of face information extracted from image data of a person's face included in the person specifying data and face information included in the person specifying data, and a face extracted from the photographed image. A communication terminal device configured to collate with information to identify a person included in the captured image. The communication terminal device according to any one of claims 1 and 2,
The profile information adding unit adds profile information selected by the user to the image data of the photographed image when a plurality of profile information corresponding to the person specifying data is stored in the profile information database. A communication terminal device characterized by being configured. The communication terminal device according to any one of claims 1 to 3, further comprising:
A display control unit for displaying a captured image stored in the captured image database on a display unit;
The communication terminal apparatus, wherein the display control unit is configured to perform display based on profile information corresponding to a person included in the photographed image in conjunction with display of the photographed image. The communication terminal device according to claim 4,
The communication control device according to claim 1, wherein the display control unit is configured to display an image corresponding to the content of the profile information. The communication terminal device according to any one of claims 4 and 5,
The communication terminal apparatus, wherein the display control unit is configured to perform display based on profile information corresponding to a person selected by a user when the photographed image includes a plurality of persons. The communication terminal device according to any one of claims 1 to 6, further comprising:
Profile information update for updating the profile information added to the image data of the photographed image when the profile information stored in the profile information database is changed after being added to the image data of the photographed image A communication terminal device comprising a unit. The communication terminal device according to claim 7,
The profile information update unit maintains the state in which the profile information before update is added without being deleted from the image data of the captured image,
A communication terminal device configured to display profile information before and after update as a history. The communication terminal device according to any one of claims 1 to 8, further comprising:
A communication terminal apparatus comprising: a communication unit that transmits image data of the photographed image to which the profile information is added to another apparatus.

Images