JP2015191392A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of displaying a display image allowing a user to easily recognize relevance between an information terminal and the user, without burdening the user.SOLUTION: An image detection section 251 and a determination section 252 compare each of multiple pieces of picture information prescribed by image data D1 with each of face image information for the number of registrants, to obtain correlation image data D12 associated with personal information, the face image information of which matches a picture information unit. A communication detection section 253 obtains personal information identification data D23 formed by classifying the personal information for the number of registrants in phone book data D2 into multiple kinds of identification items to be identified based on communication history data D3. Image display means displays, on a display screen, a display image in a mode associating one of the multiple kinds of identification items with at least a part of the picture information prescribed by the image data D1 on the basis of user correlation diagram data D25.

Description

  The present invention relates to an image processing apparatus that performs image processing on image data including photographic information taken by an information terminal such as a smartphone or a mobile phone.

  In recent years, with the spread of information terminals such as digital cameras and smartphones, it has become possible to easily create digital photographs. Therefore, a large amount of camera image data exceeding 1000 images is often stored in each user's smartphone. In addition, the memory of smartphones and digital cameras in recent years has been increasing in capacity, and the number of camera image data that can be stored tends to increase more and more. Therefore, as a search method for finding desired photo information from a large amount of camera image data, means using the file name of camera image data and the shooting date and time of camera image data, and images taken using a smartphone or digital camera Since the position information is added to the data, there is a method of searching for the photograph information in the desired camera image data based on the position information where the photograph was taken.

  Further, in Patent Document 1, by uploading captured image data to a server, detecting a face image in photographic information in the uploaded image data, and detecting persons in the same photograph (information) A user correlation diagram creating apparatus that creates a user correlation diagram based on image data is disclosed.

JP 2009-251832 A

  However, in the user correlation diagram creation device disclosed in Patent Document 1, it is necessary to upload image data to the server in order to create the user correlation diagram, and as a previous procedure, the user is given to the service execution unit called the correlation information creation server device. Registration must be carried out. Further, in order to create a user correlation diagram, a procedure of uploading camera image data to the server occurs each time. As described above, there is a problem that many operations are required for the user, which places a burden on the user.

  In addition, in order to create a user correlation diagram, first, it is necessary to retrieve desired image data from a large number of images for uploading to a server. Furthermore, there is a problem that the user correlation diagram itself is limited to those based on image data.

  The present invention has been made to solve the above-mentioned problems, and can display a display image related to photographic information in a manner in which the relevance with the user of the information terminal can be easily understood without imposing a burden on the user. It is an object of the present invention to obtain an image processing apparatus that can be used.

  The image processing apparatus according to claim 1 of the present invention is based on image data including a plurality of pieces of photographic information and information terminal registration data, each of which is personal information for the number of registered persons associated with face image information. An image processing apparatus that performs processing, receives the image data and the information terminal registration data, compares each of the plurality of photo information with each of the face image information for the number of registered persons, An image determination unit that obtains correlation image data that associates the personal information that matches the face image information, and personal information identification that classifies personal information for the number of registered persons into a plurality of types of identification items based on the information terminal registration data. Based on the personal information identification unit for executing the identification item classification process for obtaining data, the correlation image data, and the personal information identification data, the duplication in the image data is performed. Of at least a portion of the picture information, and an image display means for displaying the display image of the aspect associated one of the plurality of types of identification items in the display screen.

  The image display means in the image processing apparatus according to claim 1 is at least part of a plurality of pieces of photographic information based on the correlation image data obtained from the image determination unit and the personal information identification unit and the personal information identification data. In addition, in order to display a display image in a form associated with one of a plurality of types of identification items on the display screen, the relationship between the photo information and the information terminal user is displayed from the display image displayed on the display screen. Can be easily understood. At this time, since the user of the information terminal does not need to perform any operation, it does not burden the user of the information terminal.

1 is a block diagram showing a configuration of a photographic print system including an image output apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating details of an internal configuration of the image output apparatus illustrated in FIG. 1. It is a flowchart which mainly shows the preparation procedure of the user correlation diagram data by the image output device of this Embodiment. It is explanatory drawing which shows the specific content of the telephone directory data shown in FIG.1 and FIG.2. It is explanatory drawing which shows typically the relationship between the face image detection process result by the determination part shown in FIG. 1, and correlation image data. It is explanatory drawing which shows typically the content of the communication determination process by the communication detection part shown in FIG. It is explanatory drawing (the 1) which shows the example of a display on the screen for a display by the image display means based on user correlation diagram data. It is explanatory drawing (the 2) which shows the example of a display on the screen for a display by the image display means based on user correlation diagram data.

<Embodiment>
(Configuration of correlation display creating means 25)
FIG. 1 is a block diagram showing the configuration of a photographic print system including an image output apparatus 1 which is an image processing apparatus according to an embodiment of the present invention. As shown in the figure, when the information terminal 2 is connected to the image output device 1 via the interconnection means 22, image data including a plurality of pieces of photographic information recorded in the storage medium 6 in the information terminal 2. The printout process based on D1 can be executed.

  The information terminal 2 may be a smartphone or a mobile phone that has a camera function, stores a plurality of photographed photo information as image data D1, and has a communication function such as a telephone function and an e-mail function. . Even if the information terminal 2 does not have a camera function, the information terminal 2 may be a terminal having an image data holding function that takes in photo information separately taken by a digital camera or the like as image data D1. Furthermore, a mode in which the media itself for storing the image data D1 is provided independently of the information terminal 2 is also conceivable.

  The interconnection means 22 is a hardware part of the image data reading means 7 and the mobile data reading means 21 to be described later. Specifically, as the interconnection means 22, a wired connection means such as a cable with the information terminal 2 or Wi- A wireless connection means such as Fi and a storage medium reading mechanism such as a card medium are conceivable.

  For example, when the storage medium 6 is a CD / DVD, a USB memory, a device capable of infrared communication, a Bluetooth device receiving data via Bluetooth (registered trademark), or a device capable of wireless LAN performing wireless communication with an information terminal. CD / DVD drive for reading CD / DVD, USB port for reading USB, infrared communication device for receiving data from mobile phone by infrared communication, Bluetooth device for receiving data by Bluetooth, wireless LAN and Wi-Fi for wireless communication with information terminal The interconnection means 22 may be constituted by -Fi or the like. Further, the interconnection means 22 may be a cable for directly connecting the information terminal 2 and performing data communication, or a scanner device. As described above, as the interconnection means 22, various configurations that can be connected to the storage medium 6 of the image output apparatus 1 and the information terminal 2 are conceivable.

  The correlation display creating means 25 which is a characteristic part in the image output apparatus 1 includes an image detection unit 251, a determination unit 252, and a communication detection unit 253, and a photo DB 254 and a communication history DB 255 as storage units. .

  The image detection unit 251 takes in the image data D1 via the interconnection means 22, performs face image detection processing for detecting the presence or absence of a face image for each of a plurality of pieces of photographic information defined in the image data D1, Face presence information D1f that identifies photo information in which an image exists is generated. Then, the image detection unit 251 outputs the image data D1 and the face presence information D1f to the determination unit 252.

  The determination unit 252 receives the image data D1, the face presence information D1f, and the phone book data D2, and indicates at least one of the plurality of pieces of photographic information in the image data D1 indicating that the face presence information D1f has detected a face image. Correlation image data in which each piece of photo information (face detection photo information) is compared with each face image information for the number of registered persons in the phone book data D2, and personal information matching the face image information is associated with each face detection photo information unit. D12 is registered in the photograph DB 254. The phone book data D2 is communication user information including personal information (personal information and personal information of the other party in the communication history data D3) for each registered person associated with the face image information.

  The image detection unit 251 and the determination unit 252 receive the image data D1 and the phone book data D2, and compare each of a plurality of pieces of photo information defined by the image data D1 with each of the face image information for the registered number of people. An image determination unit that obtains correlation image data D12 by associating personal information whose face image information matches with information units is configured.

  The communication detection unit 253 which is a personal information identification unit acquires the phone book data D2 and communication history data D3 which are mobile data D20 (information terminal registration data) from the information terminal 2 via the interconnection means 22.

  The communication history data D3 includes call history data D31, mail history data D32, and communication history data D33.

  The call history data D31, which is telephone history information, uses the telephone function originally provided in the information terminal 2 and indicates the use history of the telephone function with the other communication device by the user of the information terminal 2 by specifying the use partner. It is data. The mail history data D32, which is e-mail history information, uses the e-mail function originally provided in the information terminal 2 to specify the use history of the e-mail function used by the user of the information terminal 2 with other communication devices. It is data shown. The communication history data D33 uses the application communication function realized by the application software executable on the information terminal 2 and the use history of the application communication function with other communication devices and the like by the user of the information terminal 2 This data is specified and shown. In addition, the above-mentioned use other party can be specified by the personal information for the number of registered persons in the telephone directory data D2.

  Then, the communication detection unit 253 classifies personal information for the number of registered persons in the phone book data D2 into a plurality of types of identification items identified based on the communication history data D3 based on the phone book data D2 and the communication history data D3. The communication determination process (identification item classification process) for obtaining the personal information identification data D23 is executed, and the personal information identification data D23 is registered in the communication history DB 255.

  As the plurality of types of identification items described above, the communication function usage frequency based on the call frequency (phone function use frequency) based on the call history data D31, the e-mail use frequency based on the mail history data D32, and the application communication frequency based on the communication history data D3. There is. That is, based on the comprehensive evaluation, partial evaluation, or individual evaluation of the history data D31 to D33, a plurality of rankings based on the degree of use frequency of the communication function correspond to a plurality of types of identification items.

  In addition, it is basic to perform a plurality of types of identification items for each user. However, group information of family, friends, colleagues, etc. registered in the phone book data D2 (specific relationship of the user of the information terminal 2 is shown). (Information to be shown) may be classified by unit.

  Thus, the correlation display creating means 25 can obtain the user correlation diagram data D25 by the combination of the correlation image data D12 registered in the photograph DB 254 described above and the personal information identification data D23 registered in the communication history DB 255. it can.

  The image display means 13 (see FIG. 2), which is not shown in FIG. 1 and is a characteristic part of the image output apparatus 1, receives the above-described user correlation diagram data D25 in addition to the image data D1. Then, the image display unit 13 displays a display image in a form in which one of a plurality of types of identification items is associated with at least a part of a plurality of pieces of photographic information defined by the image data D1 based on the user correlation diagram data D25. Display on the display screen.

(Overall configuration of image output apparatus 1)
FIG. 2 is a block diagram showing details of the internal configuration of the image output apparatus 1 shown in FIG. As shown in FIG. 1, the image output apparatus 1 includes an image data receiving terminal (hereinafter referred to as “receiving terminal”) 3 and a printer 4 (printers 41 to 4n).

  The reception terminal 3 reads the image data D1, the phone book data D2, and the communication history data D3 taken by the information terminal 2 or a digital camera from the storage medium 6, and the image display unit 13 mounted on the image output device 1 This is a terminal that displays an image based on the user correlation diagram data D25 on the display screen and allows the user to select an image.

  The receiving terminal 3 includes a control unit 26 and a storage unit 10, and the control unit 26 includes an image data reading unit 7, a mobile data reading unit 21, and an image display unit 13 in addition to the correlation display creation unit 25 shown in FIG. , Voice reproducing means 14, accounting means 20, position detecting means 15, data selecting means 16, communication means 18, print data creating means 11, and print data transmitting means 12.

  The printer 4 is connected to the reception terminal 3 and prints out photos 5 (51 to 5n) that are photographic image outputs based on the image data D1 stored in the storage unit 10. As shown in the figure, the printer 4 is composed of n (n ≧ 2) printers 41 to 4n, and a photograph 5i is printed out by each printer 4i (i = 1 to n).

  The image data reading means 7 has the above-described interconnection means 22 as a hardware part, reads the image data D1 stored in the storage medium 6 of the information terminal 2, and stores it in the storage unit 10.

  The storage unit 10 stores the image data D1 read by the image data reading means 7 and adjusts the image quality of the definition data D9, which is a data group of screen display and voice guidance information, and the picture 5 printed by the printer 5. Image quality adjustment data D17 and mobile data D20 (phone book data D2 + communication history data D3) read via the mobile data reading means 21 are stored.

  Based on the print control signal SC13 from the image display means 13, the print data creation means 11 obtains print data composed of printing images selected from a plurality of photographic images in the image data D1 stored in the storage unit 10. At this time, image quality adjustment is performed as necessary based on the image quality adjustment data D17.

  The print data transmission unit 12 transmits the print data obtained from the print data creation unit 11 to the printer 4i, thereby causing the printer 4i to execute the printout process of the photograph 5i based on the image data D1.

  As described above, the image display means 13 receives the image data D1 read by the image data reading means 7, and displays a display image based on the user correlation diagram data D25 created by the correlation display creation means 25 on the display screen. This gives the user an opportunity to select the photograph 5 desired to be printed out by the printer 4.

  For example, a touch panel integrated with a display device such as a CRT or a liquid crystal panel corresponds to the heart wear portion of the image display means 13. The touch panel may be a multi-touch compatible touch panel. Therefore, the user can directly operate the display such as the touch using the touch panel function on the display on the display screen of the display image based on the user correlation diagram data D25 by the image display means 13.

  The voice reproducing means 14 is operated by voice guidance for the user to select the photograph information that the user desires to print or to operate the reception terminal 3 without being confused, or the BGM or the information terminal 2 during the printing of the photograph 5i. When not in use, a screen saver is displayed on the display screen, and a sound such as a catch voice that notifies the passerby of the function of the terminal is played. The audio reproducing means 14 has a speaker as a hardware part.

  The accounting unit 20 is a unit for paying the print fee for the order that the user of the image output apparatus 1 has ordered from the reception terminal 3. As a hardware part of the billing means 20, there are a part that accepts coins and banknotes, a function of holding the inserted coins and banknotes, and a change return of the difference between the amount of money inserted and the sales amount.

  The position detection means 15 is means for detecting position information where the image output apparatus 1 is installed. As a specific hardware part, for example, a position measurement method using GPS (Global Positioning System) or an IP address or a position measurement using a Wi-Fi base station is used to obtain the coordinates of the position of the image output apparatus 1, A configuration for outputting the result is conceivable. The coordinates here refer to latitude and longitude, but are not particularly limited as long as the position information of the image output apparatus 1 can be specified by other acquired position information.

  The data selection unit 16 selects definition data D9 for performing optimal screen display and voice guidance based on the position information detected by the position detection unit 15, and stores the definition data D9 in the storage unit 10. The data selection unit 16 performs data selection according to a program for the data selection unit 16 stored in the storage unit 10, for example.

  A plurality of types of definition data are prepared in the data server 19 outside the image output apparatus 1, and the data selection unit 16 communicates with the data server 19 via the communication unit 18 to thereby prepare a plurality of types of definition data prepared on the data server 19. Among the types of definition data, optimum definition data is stored in the storage unit 10 as definition data D9.

  Under the control of the data selection means 16, the communication means 18 communicates with the data server 19 in order to acquire or update the definition data D9 stored in the storage unit 10 described above. As a specific hardware configuration of the communication unit 18, a wired or wireless communication function may be used.

  The mobile data reading means 21 has the above-described interconnection means 22 as a hardware function, and reads the phone book data D2 and communication history data D3 of the information terminal 2 from the storage medium 6. Then, the mobile data reading means 21 stores data including the phone book data D2 and the communication history data D3 in the storage unit 10 as mobile data D20. The mobile data reading means 21 is not limited in its method and configuration as long as it can read the phone book data D2 and the communication history data D3 and store them in the storage unit 10 as the mobile data D20.

  In the communication history data D3, the call history data D31, which is call history information remaining in the information terminal 2 when the telephone communication inherent in the information terminal 2 is performed, and the electronic mail inherent in the information terminal 2 are transmitted or received. Mail history data D32 which is mail transmission / reception history information at the time, and communication history data D33 which is application communication history information remaining when communication with other users is performed using an application installed in the information terminal 2.

  The correlation display creating means 25 takes in the image data D1 and the mobile data D20 (phone book data D2 + communication history data D3) stored in the storage unit 10, and as described with reference to FIG. 1, the image data D1 and the mobile data Based on the data D20, user correlation diagram data D25 including correlation image data D12 and personal information identification data D23 is created.

  The main functions of each means described above by the control unit 26 can be realized by program processing by a CPU based on software. Hereinafter, this point will be described in detail. The connection itself between the image data reading means 7 and the mobile data reading means 21 and the storage medium 6 is realized by a hardware part which is the interconnection means 22. On the other hand, the image data reading process by the image data reading means 7 and the mobile data reading process by the mobile data reading means 21 are realized by a program process by a CPU based on software.

  As described above, the billing means 20 is implemented by hardware such as coin mech and bill validator for coins and bills and for storing coins and bills. Counting and paying out coins and bills that are input is realized by a program process by a CPU based on software.

  In addition, the processing procedure of each means such as the correlation display creating means 25 is realized mainly by program processing by a CPU based on software. As described above, the control unit 26 controls the entire image output apparatus 1 by the program processing based on software as described above, and corresponds to a CPU of a personal computer.

  The printer 4 is, for example, a high-resolution color printer, such as a sublimation printer or an inkjet printer, and the printing method is not limited. Further, the photo 5 printed by the printer 4 is not limited to an L size photo, and may be a large size such as a postcard size or 2L size, or a sticker print.

  The storage medium 6 refers to a card medium such as an SD card or a memory stick used in a digital camera or the like, a CD, a DVD, a USB memory, or the like. The storage medium 6 is incorporated in the information terminal in addition to the card medium described above, and is used to include a memory that stores camera image data and the like.

  The definition data D9 includes an image quality adjustment file for adjusting the color and texture of a photograph to be printed, and a picture of the image output apparatus 1 in addition to the screen display and voice guidance determined based on the position information acquired by the position detection means 15. Data corresponding to the price setting data that sets the sales amount of That is, the definition data D9 includes the position information obtained by the position detection means 15, for example, the screen display and voice guidance used in the latitude / longitude and the area, the color and texture of the photograph to be printed, and the image output. It means data in which the sales amount of the photograph of the device 1 is associated.

  The storage unit 10 refers to a mass storage device such as a hard disk. The storage unit 10 includes a program for controlling each component. For example, the storage unit 10 controls a program for reading the image data D1 from the storage medium 6 and temporarily storing it in the storage unit 10 (a program for realizing the image data reading unit 7), and a display image displayed on the display screen. And a program for editing and processing an image displayed on the display screen (a program for realizing the print data creating means 11). Further, the storage unit 10 includes a program for holding user print order information (a program for realizing the charging unit 20), and a program for transmitting an image to be printed to the printer 4 (a program for realizing the print data transmission unit 12). ), A program for exchanging data with the data server 19 (a program for realizing the data selection means 16 using the communication means 18), and a program for creating a user correlation diagram (to realize the correlation display creating means 25). For example).

  The user uses the touch panel function on the display screen of the image display means 13 for the selected photographic image, and performs image processing such as graffiti, stamp, and frame addition, and image enlargement or reduction, rotation, trimming, etc. Editing operations are possible. In these image editing operations, material data such as stamps and frames stored in the definition data D9 can be reflected in the print data created by the print data creation means 11.

(Operation of correlation display creating means 25)
FIG. 3 is a flowchart mainly showing a procedure for creating the user correlation diagram data D25 by the image output apparatus 1. Hereinafter, with reference to FIG. 3, a procedure for creating the user correlation diagram data D <b> 25 and a display operation by the image display unit 13 will be described.

  First, in step S <b> 1, the information terminal 2 is connected to the image output device 1. As described above, the information terminal 2 is a smartphone, a mobile phone, or the like that stores in the storage medium 6 the image data D1 having a plurality of pieces of photographic information that are camera images that the user desires to print.

  The connection method of the information terminal 2 to the image output apparatus 1 is as follows. As described in the interconnection means 22, a wired connection using a cable, a wireless connection using wireless such as Wi-Fi, a storage medium using a memory card, etc. There is no limitation on the connection type such as connection. That is, it is only necessary that the information terminal 2 can be connected to the image output apparatus 1 so that the image data reading means 7 can read the image data D1 and the mobile data reading means 21 can read the phone book data D2 and the communication history data D3.

  Next, in step S <b> 2, the image data reading means 7 reads the image data D <b> 1 stored in the storage medium 6 such as the information terminal 2 connected to the image output device 1 and stores it in the storage unit 10 of the image output device 1. Read processing of the image data D1 to be stored is executed.

  Thereafter, in step S 3, the mobile data reading means 21 reads the phone book data D 2 and the communication history data D 3 stored in the storage medium 6 of the information terminal 2 connected to the image output device 1. Mobile data reading processing stored as mobile data D20 in the storage unit 10 is executed. The mobile data D20 is the phone book data D2 and the communication history data D3.

  FIG. 4 is an explanatory diagram showing specific contents of the telephone directory data D2. As shown in the figure, the phone book data D2 is composed of registration data RD1 and RD11 to RD13 that define personal information of the number of registered persons. The registration data RD1 includes face image data FD1 that is the face image information of the person and personal data JD1 that defines personal information such as the person's name, telephone number, and mail address.

  The registration data RD11 to RD13 indicate the relationship between the face image data FD11 to FD13 which is the face image information of the user of the information terminal 2 and the name, telephone number, and mail address of the user, and the user. It consists of personal data JD11 to JD13 that define personal information such as group information.

  Although not shown in FIG. 4, the personal data JD1, JD11 to JD13 may include the date of birth of the person (user), address, comment for each user, etc. It is not limited to the contents of the data.

  Further, the communication history data D3 means communication history information that is used between a user of the information terminal 2 and a user such as another communication device to specify a communication partner and specify a use partner.

  As the communication history data D3, call history data D31 that indicates the use history of the telephone function using the telephone function originally provided in the information terminal 2 by specifying the other party, and the electronic using the electronic mail function inherent in the information terminal 2 Mail history data D32 that indicates the use partner of the mail function use history is included. Further, the communication history data D3 includes application communication history data D33 that indicates an application communication function that is installed in the information terminal 2 and performs application communication with other users by specifying a usage history.

  Thereafter, in step S4, the image detection unit 251 of the correlation display creating unit 25 executes a face image detection process for detecting whether or not a human face is included in the image data D1 held in the storage unit 10. That is, the image detection unit 251 performs image analysis as to whether or not a face image is included in each photographic image in the image data D1, and the presence of a face indicating the presence or absence of a face image in each of a plurality of pieces of photographic information as an image analysis result. A face image detection process for obtaining information D1f is executed. The image detection unit 251 outputs the face presence information D1f together with the image data D1 to the determination unit 252 at the next stage.

  The face image detection processing by these image detection units 251 is performed on all of the plurality of photographic images constituting the image data D1 held in the storage unit 10. Note that the face image detection process executed by the image detection unit 251 is realized by the control unit 26 executing, for example, a face detection program, an image analysis program, a face image comparison program, and the like stored in the storage unit 10. The

  Next, in step S5, it is confirmed whether or not a face image is detected by the face image detection process. That is, if the face presence information D1f indicates the presence of a face image in at least one piece of photographic information by the face image detection process in step S4 (YES), the process proceeds to the next step S6, and the face presence information D1f is all photos. When the presence of the face image in the information is not instructed (NO), the process proceeds to step S9. Here, the photo information in which the face presence information D1f indicates the presence of a face image among a plurality of pieces of photo information is referred to as “(at least one) face detection photo information”. In the case of NO at step S5, only the registration process of the personal information identification data D23 to the communication history DB 255 by the execution of steps S9 to S11 is performed.

  In step S6, which is executed in the case of YES in step S5, the determination unit 252 of the correlation display creating means 25 performs a comparison process with the face image (face image data FD) registered in the phone book data D2. That is, the determination unit 252 includes one face detection photo information and face image information (hereinafter referred to as “user face image”) of registered users (persons or users) stored in the phone book data D2. To determine whether the face detection photo information matches the user face image.

  Then, when there is a user face image that matches the face detection photo information, the determination unit 252 includes information that specifies a user (including a plurality) of the matched user face image and information that specifies the face detection photo information. Generate correlated correlation information.

  Thereafter, in step S7, confirmation processing is performed for the number of detected face images, that is, all of the face detection photo information, to determine whether the comparison processing in step S6 has been completed. If completed (YES), the process proceeds to step S8. If not completed (NO), the process returns to step S6. Thereafter, the processes in steps S6 and S7 are repeated until YES is determined in step S7.

  In step S8 executed in the case of YES in step S7, this is a collection of the correlation related information generated in step S6 when there is a matching user face image among at least one face detection photo information. Correlated image data D12 can be obtained, and the determination unit 252 registers the correlated image data D12 in the photograph DB 254.

  FIG. 5 is an explanatory diagram schematically showing the relationship between the face image detection processing result by the determination unit 252 and the correlation image data D12.

  As shown in FIG. 5A, among the plurality of pieces of image information defined by the image data D1, photographic data (photo information) PD1 to PD4 which are at least one face detection photographic information detected by the image detection unit 251. On the other hand, the face image detection processing result with the face image data FD11 to FD13 (face image information) of the users A to C defined by the registration data RD11 to RD13 of the phone book data D2 is shown. In FIG. 5A, when there is a user face image of the user corresponding to the photo data PD1 to PD4, the mark ○ indicates that the user face image of the user corresponding to the photo data PD1 to PD4 exists. The case where there was no is shown.

  For example, in the photo data PD1 (identification number 1), the determination unit 252 determines that the user A and the user B among the users A to C are captured, so that the user A is ○, the user B is ○, User C becomes x. Similarly, in the photo data PD2 (identification number 2), the user A is x, the user B is ◯, and the user C is x.

  When the determination result (aggregation of correlation related information in units of photograph data PD) by the determination unit 252 shown in FIG. 5A is obtained, the determination unit 252 generates correlation image data D12 based on the determination result. Become. That is, as shown in FIG. 5 (b), as user A related photo information 24A, the user A is given the identification number (numbering) of the photo image of the image data D1, 1, 3,. 4), and the user B related photo information 24B is associated with the user B as the photo image identification number 1, 2, 3, 4,... Of the image data D1, and the user C related photo information 24C is the user. Correlation image data D12 is generated by associating C, 3,..., Which is the identification number of the photographic image of image data D1, with C.

  The photograph data PD2 includes the face image (right face image) of the user that has not been detected from the phone book data D2. However, for such a face image, the creation target of the correlation image data D12 is as follows. do not do.

  As described above, in step S8, the determination unit 252 defines each user with the face detection photo information in the image data D1 based on the face image data FD in the registration data RD registered from the phone book data D2. Correlation image data D12 associated with whether or not it is reflected in the photograph is registered in the photograph DB 254. The photograph DB 254 may be provided in the storage unit 10 or may be provided separately in the image output apparatus 1 exclusively for the correlation display creating means 25.

  Subsequently, in step S9, the communication detection unit 253 performs group setting based on the group information of the phone book data D2. With group information, as shown in FIG. 4, the user of the information terminal 2 relates to the personal data JD11 to JD13 of each user registered in the phone book data D2 with the users such as friends, family / relatives, colleagues, etc. If the group information exists in the phone book data D2, it can be set for each group based on the group information. For example, users can be grouped according to friends, family / relatives, and colleagues. This group setting is an example, and an arbitrary group setting can be performed by the user of the information terminal 2.

  First, the communication detection unit 253 detects the group information of each user registered in the phone book data D2 held in the storage unit 10, and sets the group based on the group information if it exists.

  Next, in step S10, the communication detection unit 253 of the correlation display creating means 25 determines which user (utilization) the user of the information terminal 2 is based on the phone book data D2 and the communication history data D3 stored in the storage unit 10. A communication determination process (identification item classification process) is performed to determine whether the person is in close contact with the other party.

  That is, the communication detection unit 253, based on the determination result of the communication determination process, converts personal information for the number of registered persons in the image data D1 into a plurality of types of identification items (for example, communication items) in a format corresponding to the frequency of use of the communication function. Communication determination processing for obtaining personal information identification data D23 classified into identification items ranked based on the frequency of use of functions is executed.

  In step S11, the correlation display creating means 25 registers the personal information identification data D23 obtained in the communication determination process in step S10 in the communication history DB 255.

  FIG. 6 is an explanatory diagram schematically showing the contents of communication determination processing by the communication detection unit 253.

  As shown in FIG. 5A, the communication determination result D255 includes each user (A, B, C) registered in the phone book data D2, the group setting of each user detected in step S9, The number of times of communication function use with each user (A, B, C) (the number of communication by telephone, (electronic) mail, application (application communication)) is shown.

  In the communication history data D3, an application using a communication application installed in the information terminal, an incoming / outgoing call using the telephone number of each user registered in the telephone directory data D2, an e-mail transmission / reception using a mail address, Has a communication history. Even in the case of application communication using an application installed in the information terminal 2, the destination is determined using a telephone number or an e-mail address, and in order to perform communication, it is determined which user the user has communicated with Can do.

  Hereinafter, the user A in the communication determination result D255 of FIG. From the communication history data D3, it is possible to detect how many times the telephone number and mail address registered in the personal data JD11 of the user A shown in FIG. 4 are used. As for the number of telephone calls, if user A does not answer even though the call is made to user A, or if user A receives an incoming call to the user, the user cannot answer the call and the missed call If it becomes, the count is excluded. As a result, the user of the information terminal 2 makes five outgoing calls to the user A who is the other party, the user A makes four incoming calls to the user, and the user of the information terminal 2 sends three emails to the user A. Times, user A receives the mail from the information terminal 2 to the user three times, and the information terminal 2 uses the application communication from the user A to the user A once. On the other hand, there is one incoming call, and the total number of communication between the user of the information terminal 2 and the user A is 17 times.

  Further, group setting is also performed for the users B and C, and the communication determination process determines that the number of communication with the user B based on the communication history data D3 is 31 times and the number of communication with the user C is 20 times.

  As a result, when ranking the user with whom the information terminal 2 is communicating with the number of communication (judgment result) obtained, the user B ranks first in the communication function usage frequency rank (RANK), and the user C Can obtain the communication determination result D255 in which the user ranks second and the user A ranks third.

  As described above, from the telephone directory data D2 and the communication history data D3, it is possible to obtain the communication determination result D255 that is classified (ranked) by the user of the information terminal 2 and the usage frequency of the communication function.

  As a result, as shown in FIG. 6B, the communication detection unit 253 sets the personal information for the number of registered persons in the phone book data D2 in accordance with the communication determination result D255 into a plurality of identification items (such as ranking of communication frequency). The classified personal information identification data D23 can be obtained.

  For example, in the personal information identification data D23 shown in FIG. 6B, the group setting of the user A is “friend”, and the usage frequency of the communication function is classified into rank 3. User B's group setting is “Family / Relatives”, communication function usage frequency is classified into ranks, User C group setting is “colleagues”, and communication function usage frequency is classified as rank 2. The Therefore, when each group is represented by users A, B, and C, the usage frequency of the communication function in units of groups is classified into rank 1 for family / relatives, rank 2 for colleagues, and rank 3 for friends.

  The personal information identification data D23 obtained in this way is stored in the communication history DB 255. The communication history DB 255 may be stored in the storage unit 10 or provided exclusively for the correlation display creating means 25.

  Thereafter, in step S12, the image display means 13 is a user correlation diagram which is a combination of the correlation image data D12 stored in the photograph DB 254 in step S8 and the personal information identification data D23 stored in the communication history DB 255 in step S11. Based on the data D25, at least a part of a plurality of (face detection) photographic information defined by the image data D1 is associated with one of a plurality of types of identification items (ranks A to C) as necessary. Display the format display image on the display screen.

  Further, the image display means 13 prints a print control signal SC13 for instructing photo information to be printed among a plurality of pieces of photo information in the image data D1 in accordance with the selection operation contents using the touch panel function or the like of the display screen. Is output to the print data creation means 11.

(Example of display by image display means 13)
7 and 8 are explanatory views showing display examples of display images on the display screen by the image display means 13 based on the user correlation diagram data D25. FIG. 7 shows a display example for each user using the display screen as the image selection screen 27, and FIG. 8 shows a display example for each group using the display screen as the image selection screen 32.

  The user display button 31b of the display method button 31 in the upper left of the image selection screen 27 shown in FIG. 7 is displayed by user by performing a selection operation by touch or the like, and the group display button 31a is selected by performing a selection operation. It will be displayed separately.

  7 and 8 show user-specific and group-specific image selection screens 27 and 32, which are display screens for the user to select a desired print image in the image output apparatus 1. FIG. The left side of the image selection screen 27 (32) shows the relationship between the user of the information terminal 2 and other users.

  On the image selection screen 27, based on the personal information identification data D23, the user classified as the identification item with the highest frequency of use of the communication function is indicated by the thickness of the line and the order of the circle numbers. In the following, it is shown that user C and user A are in this order.

  In the example of FIG. 7 (FIG. 8), the rank (rank) is from 1st to 3rd. Of course, this is not restrictive, and the number of users to be displayed (number of groups) is correlated image data D12 and personal information identification data. You may change suitably by the relationship with D23. In FIG. 7, a first priority selection button 281, a second priority selection button 282, and a third priority selection button 283 are assigned to user B, user C, and user A. In FIG. A first priority selection button 331, a second priority selection button 332, and a third priority selection button 333 are assigned to relatives, colleagues, and friends.

  As shown in FIG. 7, on the right side of the image selection screen 27, photo data PD11 to PD14 when the user B is selected by the user operating the image output apparatus 1 with the first priority selection button 281 (displayed by a broken line). Is displayed. When the user B is selected for the selection operation by the user selection button 281, based on the correlation image data D 12 in the photograph DB 254, the photograph data PD 11 to PD 14 that are face detection photograph information in which the user B is captured in the image data D 1 are displayed. The image display means 13 displays the display image in a preferentially displayed form on the display screen. At this time, selection boxes SB1 to SB4 for specifying the user B in the photo data PD11 to PD14 are also displayed. The selection boxes SB <b> 1 to SB <b> 4 can be realized by the image display unit 13 when the correlation image data D <b> 12 further indicates the position information of the face image, for example.

  In this way, the photo data PD11 to PD14 (photo information), which are at least a part of the plurality of (face detection) photo information defined by the image data D1 as the photo information for the user B classified as rank 1, are associated. The display image of the selected format is displayed on the image selection screen 27.

  In the image selection screen 27, a period setting button 29 can be used to set a period as a generation condition of the personal information identification data D23 in the communication history DB 255. Thereby, the display image based on the user correlation diagram data D25 indicating the relevance of the communication function between the user of the information terminal 2 and another user in a specific designated period can be displayed on the image selection screen 27.

  As described above, when the user correlation diagram data D25 needs to be changed by the user's operation on the image display unit 13, the image display unit 13 appropriately sends operation information (new user correlation diagram data D25 to the correlation display creation unit 25). And the necessary information) are transferred to the correlation display creating means 25.

  The communication detection unit 253 that receives the operation information from the image display unit 13 sets the specified period set in the image display unit 13 as the determination target period in the communication determination process, and executes the above-described communication determination process. It has a function.

  That is, the communication detection unit 253 of the correlation display creating unit 25 performs the communication determination process in step S10 again using the specified period based on the setting content by the period setting button 29 as the generation condition of the personal information identification data D23.

  As a method for using the period setting, for example, it is assumed that the user of the information terminal 2 goes on a trip at a certain time. In this case, it can be assumed that the user of the information terminal 2 is in close contact with the travel companion before going on a trip. Therefore, when the designated period is set a short time before the travel period, the travel companion is ranked higher in the frequency of use of the communication function with the user of the information terminal 2 in the user correlation diagram data D25 (classification). Therefore, the image display means 13 can preferentially display the photo information with the travel companion as the display image on the image selection screen 27 based on the user correlation diagram data D25.

  An image sort button 30 is provided on the lower right side of the image selection screen 27, and the image data to be displayed can be further sorted in the order of the shooting date from the newest to the oldest, the ascending order or the descending order of the file name of the image data. The sort contents can be designated in the new order by the new order sort button 30a, the old order by the new reverse order sort button 30b, the file name ascending order by the name ascending order sort button 30c, and the file name descending order by the image sort button 30. These sort processes can also be easily realized by the control unit 26 executing the sort program stored in the storage unit 10 as the correlation display creating means 25.

  Next, referring to FIG. 8, by selecting and operating the group display button 31a of the display method button 31, the group by the image selection screen 32 shown in FIG. 8 is changed from the display by user on the image selection screen 27 shown in FIG. A case where the display is switched to another display will be described.

  On the left side of the image selection screen 32, a correlation diagram between the user of the information terminal 2 and the group setting in the phone book data D2 is shown. The correlation diagram with the group setting on the image selection screen 32 is determined depending on the number of users belonging to the group and the use frequency of the communication function.

  On the image selection screen 32, for convenience of explanation, as an identification item based on the frequency of use of the communication function, the family / relative group to which only the user B belongs is ranked first, the colleague group to which only the user C belongs is ranked second, only the user A This shows a case where the friend group to which No. 3 belongs is ranked third.

  In FIG. 8, when the group “friend” is selected by the third priority selection button 333, the photograph data PD21 to PD24 in which the user A whose group setting is the friend in the phone book data D2 shown in FIG. (Photo information) is preferentially displayed on the image selection screen 32. In the image selection screen 32, the image sorting by the image sort button 30 can be performed on the image of the user A displayed in the same manner as the image selection screen 27.

  When the operation of the image display means 13 (the operation of the selection buttons 281 to 283, 331 to 333, the period specifying operation by the period setting button 29, the sort processing operation of the image sort button 30) is performed, the user correlation diagram data D25 The content may need to be changed. In this case, as described above, the operation information of the image display means 13 is transmitted to the correlation display creation means 25, and the correlation display creation means 25 generates the user correlation diagram data D25 that matches the operation information from the image display means 13. Is done.

(Effects etc.)
As described above, according to the image output device 1 of the embodiment of the present invention, when the image data D1 stored in the information terminal 2 is displayed on the image output device 1, the information terminal 2 performs communication. In order to display the display image based on the user correlation diagram data D25 reflecting the contents of the communication history data D3 with the user who performed, image data with the user in close contact with the user of the information terminal 2 is given priority. It can be displayed and the procedure required for the user's image search can be simplified.

  In recent years, since the communication history data D3 of the information terminal 2 occupying a large role as a communication device between users is used, user correlation diagram data D25 in which the relationship between users is reflected in real time can be created.

  As described above, the image display unit 13 in the image output apparatus 1 according to the present embodiment has the user correlation diagram data obtained from the correlation display creation unit 25 in at least a part of a plurality of pieces of photographic information defined by the image data D1. Based on D25, a display image in a form in which any one of a plurality of types of identification items (communication use frequency order or the like) is associated is displayed on the display screen. Therefore, the user of the information terminal 2 can easily understand the relationship between the photographic information and the user of the information terminal 2 from the display image displayed on the display screen of the image display means 13. At this time, since the user of the information terminal 2 does not need to perform operations such as user registration and upload to the server as in the past, there is no burden on the user.

  In the image output apparatus 1, the photo printing unit including the print data creation unit 11, the print data transmission unit 12, and the printer 4 is based on the print control signal SC 13 from the image display unit 13, so that a plurality of types of identification items are obtained. It is possible to selectively print a photo highly relevant to any of the above.

  The communication detection unit 253 which is a personal information identification unit of the correlation display creating unit 25 classifies a plurality of types of identification items in a format corresponding to the frequency of use of the communication function of the user of the information terminal 2 based on the communication history data D3. A communication determination process (identification item classification process) is executed. For this reason, it is possible to display a display image on the display screen by the image display means 13 with relevance to the other party who frequently uses the communication function, and as a result, a person familiar with the user of the information terminal 2 The selection of photo information can be facilitated.

  In the present embodiment, ranking is performed according to the usage frequency of the overall communication function based on the call history data D31, the mail history data D32, and the application communication history data D33, but the call history data D31, the mail history data D32, and the application The communication history data D33 may be ranked according to the frequency of use of history data individually or in combination of two.

  For example, the communication detection unit 253 may perform communication determination processing by classifying a plurality of types of identification items in a format corresponding to the usage frequency of the telephone function of the user of the information terminal 2 based only on the call history data D31. It is done. In this case, it is possible to display the display image on the display screen by the image display means 13 with relevance to the other party who frequently uses the telephone function, and as a result, with the user of the information terminal 2 It is possible to easily select the photo information of the close person determined based on the telephone communication frequency.

  Further, the communication detection unit 253 classifies a plurality of types of identification items in a format corresponding to the usage frequency of the electronic mail function with the user of the information terminal 2 based on only the mail history data D32, and executes the communication determination process. It is possible. In this case, the display image can be displayed on the display screen with relevance to the other party who frequently uses the e-mail function, and as a result, the e-mail communication frequency of the user of the information terminal 2 can be increased. It is possible to facilitate selection of photograph information of a person who is close based on the determination.

  The communication detection unit 253 has a period designation classification function for executing the communication determination process using the designation period designated by the period setting button 29 of the image display means 13 as a generation condition, thereby limiting the information terminal 2 to the designation period. It is possible to easily select the photo information of a close person determined based on the frequency of use of the user's communication function.

  When the telephone directory data D2 includes group information, the communication detection unit 253 can execute a communication determination process in units of group information, and thus associates with a group having a specific relationship with the user of the information terminal 2 Thus, the display image can be displayed on the display screen, and as a result, it is possible to easily select the photo information of the use partner having a specific relationship with the user of the information terminal.

(Image processing method)
The processing procedure by the correlation display creating means 25 and the image display means 13 shown in FIG. 3 can be developed as an image processing method comprising the following steps (a) to (d).

(a) Image data D1 including a plurality of pieces of photo information and personal information for each registered person associated with the face image information, and mobile data D20 (image data D1 and phone book data registered in the information terminal 2) D2), and
(b) Image comparison is made between each of a plurality of pieces of photo information defined by the image data D1 and face image information corresponding to the number of registered persons in the phone book data D2, and personal information matching the face image information is associated with each photo information unit. Obtaining correlation image data D12,
(c) obtaining personal information identification data D23 obtained by classifying personal information corresponding to the number of registered persons into a plurality of types of identification items (such as frequency of use of communication function) based on mobile data D20. A combination with the information identification data D23 is defined as user correlation diagram data D25,
(d) Based on the user correlation diagram data D25 (correlation image data D12 + personal information identification data D23), any one of the plurality of types of identification items classified in step (c) is included in at least a part of the plurality of photographic information in the image data D1. An image processing method further comprising a step of displaying a display image in an associated form on the display screen.

  In step (d) in the image processing method, at least a part of a plurality of pieces of photographic information defined by the image data D1 is classified into a plurality of types based on the user correlation diagram data D25 obtained in steps (b) and (c). A display image in a form associated with any one of the identification items is displayed on the display screen. Therefore, the user of the information terminal 2 can easily understand the relationship between the photo information and the user of the information terminal 2 from the display image displayed on the display screen in step (d). Play.

  The processes (a) to (d) described above can be realized by program processing based on software by the control unit 26, for example.

(Other)
In the present embodiment, the telephone directory data D2 and the communication history data D3 are taken into the image output device 1 as mobile data D20, and the personal information identification data D23 is created by the correlation display creating means 25 of the image output device 1. showed that.

  However, the present invention is not limited to this method. An application for creating a user correlation diagram is installed in the information terminal 2, and a photo DB and a communication DB are provided in the information terminal 2, and the user correlation diagram data D25 is imaged together with the image data D1. A configuration in which the output device 1 is read is also possible.

  For example, a mode in which the correlation display creating means 25 and the storage unit 10 in the image output apparatus 1 are taken into the information terminal 2 is also conceivable. That is, the user correlation diagram data D25 may be created by executing an application program for the correlation display creating means 25 in the information terminal 2.

  In the above example, when the information terminal 2 executes the processes of steps S2 to S11 in FIG. 3, only the process of step S12 is executed by the image display unit 13 of the image output apparatus 1. Then, the image display means 13 in the image output device 1 receives the image data D1 and the user correlation diagram data D25 from the information terminal 2.

  When the information terminal 2 has a display screen, a configuration in which the function of the image display unit 13 is built in addition to the storage unit 10 and the correlation display creation unit 25 is also conceivable.

  In the above example, the information terminal 2 executes the processes of steps S2 to S12 in FIG. Then, the print data creation means 11 in the image output apparatus 1 receives the image data D1 and the print control signal SC13 from the information terminal 2.

  Also, using a card media storage medium such as a memory card, a photo DB and a communication DB are created in the memory card that is also used when printing image data, and simultaneously with the image data stored in the memory card. By taking the image into the image output device, the image output device 1 can display an image based on the user correlation diagram data D25. In this case, the card medium itself is treated as an equivalent configuration to the information terminal 2 incorporating the correlation display creating means 25 and the storage unit 10.

  Further, based on a plurality of pieces of photo information defined by the image data D1 stored in the information terminal 2, face information and individual closeness information are obtained from a snapshot or a group photo, and a communication history In addition to the registration information in the DB 255, user correlation diagram data D25 may be created.

  In the present invention, the embodiments can be appropriately modified and omitted within the scope of the invention.

  DESCRIPTION OF SYMBOLS 1 Image output device, 2 Information terminal, 3 Reception terminal, 4, 41-4n Printer, 6 Storage medium, 10 Storage part, 11 Print data creation means, 12 Print data transmission means, 13 Image display means, 25 Correlation display creation means , 26 control unit, 251 image detection unit, 252 determination unit, 253 communication detection unit, 254 photo DB, 255 communication history DB.

Claims (8)

An image processing device that performs image processing based on image data including a plurality of pieces of photo information and information terminal registration data that is personal information for each registered person associated with face image information,
Receiving the image data and the information terminal registration data, comparing each of the plurality of photo information with each of the face image information for the number of registered persons, and the personal information in which the face image information matches in units of the photo information An image determination unit for obtaining correlated image data associated with
Based on the information terminal registration data, a personal information identification unit for executing identification item classification processing for obtaining personal information identification data obtained by classifying personal information for the number of registered persons into a plurality of types of identification items;
Based on the correlation image data and the personal information identification data, a display image in a form in which any one of the plurality of types of identification items is associated with at least a part of the plurality of pieces of photographic information in the image data is displayed on the display screen. Image display means for displaying on
Image processing device. The image processing apparatus according to claim 1,
The image display means further outputs a print control signal for instructing photo information to be printed among the plurality of photo information,
The image processing apparatus includes:
A photo printing unit that selectively prints the plurality of photo information in the image data based on the print control signal;
Image processing device. The image processing apparatus according to claim 1 or 2,
The information terminal holding the information terminal registration data has a communication function with other communication devices,
The personal information for the registered number of people is
Communication user information associated with the face image information and indicating at least a communication partner of the communication function;
Further including communication history information that indicates a usage partner of the communication function usage history,
The personal information identification unit performs the identification item classification process for classifying the plurality of types of identification items in a format corresponding to the frequency of use of the communication function of the information terminal user based on the communication history information.
Image processing device. The image processing apparatus according to claim 3,
The communication function includes a telephone function with another communication device,
The communication history information includes telephone history information that indicates the usage history of the telephone function by identifying the other party,
The personal information identification unit performs the identification item classification process for classifying the plurality of types of identification items in a format corresponding to the frequency of use of the telephone function of the user of the information terminal based on the telephone history information.
Image processing device. The image processing apparatus according to claim 3,
The communication function includes an e-mail function with other communication devices,
The communication history information includes e-mail history information that identifies and shows a use history of the e-mail function,
The personal information identification unit executes the identification item classification process that classifies the plurality of types of identification items in a format corresponding to the usage frequency of the electronic mail function of the information terminal user based on the electronic mail history information. ,
Image processing device. The image processing apparatus according to any one of claims 3 to 5, wherein:
The personal information identification unit includes a period designation classification function for executing the identification item classification process in a format corresponding to the frequency of use of the communication function of the information terminal user in the designation period.
Image processing device. The image processing apparatus according to any one of claims 1 to 6,
The personal information for the registered number of people is
Further includes group information indicating the relationship with the user of the information terminal in a group unit,
The personal information identification unit executes the identification item classification process for the group information unit.
Image processing device. (a) inputting image data including a plurality of pieces of photo information, and information terminal registration data, each of which is personal information for the number of registered persons associated with face image information;
(b) Image comparison is made between each of a plurality of pieces of photo information defined by the image data and each of the face image information for the number of registered persons in the information terminal registration data, and the face image information is matched in units of the photo information. Obtaining correlated image data associated with personal information;
(c) obtaining personal information identification data obtained by classifying personal information for the number of registered persons into a plurality of types of identification items based on the information terminal registration data;
(d) Based on the correlation image data and the information terminal registration data, at least a part of the plurality of pieces of photographic information in the image data is associated with any of the plurality of types of identification items classified in the step (c). Displaying the display image of the aspect on the display screen;
An image processing method comprising:

Images