JP2011164141A - Automatic photograph producing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To apply a desirable pattern (typically, eye-catch) to user's eye without providing new components and without requiring a user to write the pattern (doodling). <P>SOLUTION: The automatic photograph producing device is configured, in eye-catch writing processing, to read a template image (S310), and compare the template image with a comparison object portion of a doodling object image (S320), thereby detecting a portion where matching degree with user's eye (image portion thereof) included in the doodling object image is equal to or above a threshold (S330), then calculate center coordinates of the iris of the eye included in the detected portion (S400), and properly stick the eye-catch image (S410). By thus automatically sticking the eye-catch image, a synthetic image where the eye-catch is applied to the user's eye is obtained in the same manner as the user's eye is charmingly photographed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an automatic photo creation apparatus, and more particularly to an automatic photo creation apparatus that takes a user with a camera and outputs a composite image generated based on the taken image as a photograph.

  2. Description of the Related Art Conventionally, there is known a photo creation device for a game where a user is photographed with a camera and the photographed image is output as a photograph. Such a play photo creating apparatus has high playability or entertainment, and therefore, various images prepared in advance according to the user's preference are prepared for images to be synthesized with the photographed images (hereinafter referred to as “additional images”). In many cases, a user can select from various background images and foreground images, or can freely draw by a user using a pen-type input unit such as a touch pen or a light pen. Since such an action by the user is performed on the photographed image, it is called “graffiti”. Such graffiti is generally provided with a time limit, and the user's preference can be selected from a variety of background images and foreground images (for example, frame images and stamp images) prepared in advance within the time limit. Various devices have been devised in the conventional automatic photo creating apparatus so that the corresponding image can be selected.

  In addition, the conventional game photo creation device appropriately sets the amount of light, the illumination angle, etc., such as a flash that emits a bright flash at the time of shooting to illuminate the user or a fluorescent lamp that always lights to illuminate the user. Some have been devised to take beautiful pictures of users.

  For example, in order to photograph a user's eyes (black eyes) beautifully, a screen is arranged in front of the user and a graphic image of a predetermined shape is projected from the video output device onto the screen. A device that captures and captures the figure (see Patent Document 1), and by appropriately arranging a plurality of light emitters so as to form a figure with a predetermined shape in front of the user, There is an apparatus that captures and captures a figure (see Patent Document 2). Such a high-luminance figure reflected in the eyes is called an eye catch (catch light).

JP 2002-77781 A JP 2002-56457 A

  However, the above-described conventional game photo creation device requires a new screen and image device for photographing the user's eyes (black eyes) or a plurality of parts such as a plurality of light emitters. Take it. Further, in the conventional apparatus, since the newly provided component must be appropriately disposed on the front surface of the housing, it is difficult to make a figure to be reflected in the eyes of the user into a desired shape.

  In this regard, a graphic having a desired shape with respect to the user's eyes (image) included in the photographed image, specifically, a figure that can make the user's eyes beautiful (without providing the above components) If the “eye catch” is written by the graffiti of the user (by hand), as a result, the same result (synthesized image) as if the user's eyes were photographed beautifully can be obtained.

  However, it is difficult for a user to write a figure having a shape (and color) that can make the eyes look beautiful with respect to the user's eyes (image) using a touch pen or the like, at least within the time limit It is extremely difficult to write quickly.

  Therefore, an object of the present invention is to provide a desired graphic (typically, without providing a new part such as an image device or a light emitter, and without requiring a user to write a graphic (doodle)). Is to provide an automatic photo creating apparatus capable of obtaining a composite image with an eye catch.

A first aspect of the invention is an automatic photo creation apparatus comprising a photographing means for photographing an image including a user who is a subject and an output means for outputting a composite image including a photographed image obtained by the photographing means as a photograph. ,
Eye detection means for detecting an image of the user's eye included in the captured image;
Storage means for storing an additional image indicating a figure to be combined with the eye image;
Image synthesis means for generating the synthesized image by synthesizing an image corresponding to the additional image stored in the storage means with the eye image detected by the eye detecting means. And

According to a second invention, in the first invention,
The eye detection means detects the center position of the pupil in the image of the user's eyes,
The image synthesizing unit aligns the predetermined position on the additional image corresponding to the center position of the pupil with the center position of the pupil detected by the eye detecting unit, and adds the additional position. An image corresponding to the image is synthesized.

According to a third invention, in the first or second invention,
The image synthesizing unit enlarges, reduces, or rotates the additional image stored in the storage unit in accordance with the black eye image of the eye image, thereby corresponding to the additional image. And an image corresponding to the additional image is combined with the black eye image.

According to a fourth invention, in any one of the first to third inventions,
The storage means further stores an image including an eye image to be a template image,
The eye detection unit generates a plurality of types of template images by enlarging, reducing, or rotating an image including an eye image stored in the storage unit, and is an image included in the captured image. An image that matches or approximates the plurality of types of generated template images is detected as an image of the user's eyes.

According to a fifth invention, in any one of the first to fourth inventions,
An input means for receiving an operation input by the user for selecting a figure to be combined with the eye image;
The storage means stores a plurality of additional images showing different figures from each other,
The image synthesizing unit is configured to detect, by the eye detection unit, an image corresponding to one of the plurality of additional images indicating a graphic selected according to the operation input received by the input unit. The synthesized image is generated by synthesizing the image.

According to a sixth invention, in any one of the first to fourth inventions, an input means for receiving an operation input by the user for erasing the additional image synthesized by the image synthesizing means from the synthesized image. In addition,
The image synthesizing unit restores the synthesized image to a state before the additional image is synthesized by the image synthesizing unit in response to an operation input received by the input unit.

A seventh aspect of the invention is a method for controlling an automatic photo creation apparatus, comprising: a photographing step for photographing an image including a user who is a subject; and an output step for outputting a composite image including a photographed image obtained in the photographing step as a photograph. Because
An eye detection step of detecting an image of the user's eye included in the captured image;
The synthesized image is generated by synthesizing an image corresponding to an additional image indicating a figure to be synthesized with the eye image stored in advance with the eye image detected by the eye detecting step. An image synthesis step.

  An eighth invention is a program that causes a computer to execute the control method of the automatic photo creating apparatus described in the seventh invention.

  According to the first aspect, the eye detection unit automatically detects the user's eye image included in the photographed image, and the image synthesis unit synthesizes an image corresponding to the additional image stored in the storage unit. As a result, a composite image is generated, so that the user's eyes are photographed beautifully without the need for new parts such as video equipment and light emitters, and without the need for drawing (doodles) by the user's hand. Similarly, it is possible to automatically generate a composite image with a desired figure (typically eye catch) attached to the user's eyes.

  According to the second aspect, the center position of the pupil in the image of the user's eyes is detected by the eye detection means, and the center position of the pupil is added to the center position of the pupil on the additional image by the image composition means. Since the corresponding positions are synthesized, the pupil can be easily and accurately detected. In particular, the pupil is not hidden by the eyelid during shooting. It is possible to detect and synthesize the additional image easily and accurately.

  According to the third aspect of the invention, the composition is performed by enlarging, reducing, or rotating the additional image in accordance with the black eye image of the eye image by the image composition means. It is easier to set the correct size or angle and align the correct position than when combining images of the whole eye (including the eyelids etc.) that are diverse (depending on people and facial expressions). Can do.

  According to the fourth aspect of the invention, since the image that matches or approximates a plurality of types of template images is detected by the eye detection means as the image of the user's eyes, accurate detection can be performed with a simple configuration. .

  According to the fifth aspect, the image corresponding to one of the plurality of additional images indicating the graphic selected according to the operation input received by the input means is synthesized with the eye image. Then, it is possible to generate a composite image with a favorite additional image selected by the user from various additional images (typically eye catches) prepared in advance.

  According to the sixth aspect, since the composite image is restored to the state before the additional image is synthesized in accordance with the operation input received by the input means, the user can easily synthesize the eye catch image. It becomes possible to cancel.

According to the seventh aspect, the same effect as that of the first aspect can be achieved in the method invention.

  According to the eighth aspect, the program can achieve the same effect as the first aspect.

It is a figure which shows the external appearance of the photo production apparatus for games which is an automatic photo production apparatus which concerns on one Embodiment of this invention. It is a front view of the imaging unit in the embodiment. It is a front view of the edit unit in the said embodiment. It is a block diagram which shows the structure which looked at the principal part of the photo production apparatus for play concerning the said embodiment from the functional surface. 6 is a flowchart illustrating a processing procedure of photographing processing in the embodiment. 5 is a flowchart illustrating a processing procedure for editing processing in the embodiment. In the said embodiment, it is a flowchart which shows the process sequence of an eye catch write process. It is a figure which shows the model image in the said embodiment. It is a figure for demonstrating the search procedure of the position of the eye by the template image in the said embodiment. It is a figure which shows the eye catch image in the said embodiment. It is a figure which shows the state which wrote in the eye part of the graffiti target image the eye catch image in the said embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
<1. Overall configuration>
FIG. 1 is a diagram showing an appearance of a game photo creation device that is an automatic photo creation device according to an embodiment of the present invention. More specifically, FIG. 1 (a) is an external side view of the play photo creation apparatus as viewed from the side, and FIG. 1 (b) is an external plan view as viewed from above. This play photo creation device accepts a shooting room 2 where a user enters, a shooting unit 3 that takes a picture of the user and receives selection of a background image and a foreground image, and an editing operation including graffiti (drawing operation) by the user. An editing unit 4 that generates a combined image combined with the photographed image and an output unit 5 that outputs the combined image are provided. FIG. 2 is a front view of the photographing unit 3, and FIG. 3 is a front view of the editing unit 4. Hereinafter, the overall configuration of the game photo creation device according to the present embodiment will be described with reference to FIGS. 1 to 3.

  The photographing room 2 has a substantially rectangular parallelepiped shape, and the photographing unit 3 is arranged along the front surface of the photographing room 2 which is a front surface when viewed from the user entering the inside. Note that an opening for a user to enter and exit, and a light-shielding curtain that covers a part or all of the opening are provided on a part of each of the left and right side surfaces of the photographing room 2. In addition, an electric roll curtain device 25 is disposed on the rear surface, which is the rear surface when viewed from the user entering the inside of the photographing room 2.

  The photographing unit 3 includes a camera 10 as an imaging unit for photographing a user, strobes 11, 12, 13 L, 13 R, and 14 that are arranged at the top, bottom, left and right positions of the camera 10 and emit flash light. It is provided with a shooting operation touch panel 20 that is arranged and receives operations from a user, displays shot images, and the like.

  The camera 10 is typically a digital camera that generates a digital image signal using a CCD (Charge Coupled Device), captures a user, and outputs an image signal representing the captured image. The flashes 11 to 14 emit a flash toward the user in order to obtain sufficient light for photographing. The shooting operation touch panel 20 is configured to provide an operation screen for accepting various operations by the user during shooting, and to display an image based on the image signal in real time.

  The shooting operation touch panel 20 has a real-time preview area for displaying a shot image in real time, a pose sample display area for displaying a pose sample, and a graffiti target image for displaying a graffiti target image obtained by shooting. A screen including a display area is displayed.

  The photographing unit 3 is built around a computer and incorporates a control device for controlling each part, an I / O control device, a network adapter for communicating with the editing unit 4, and the like. The photographing unit 3 includes a coin insertion slot 26 at the lower front side.

  The editing unit 4 is configured around a computer similar to the photographing unit 3, and includes a control device that controls each part and the like, and a network adapter that communicates with the photographing unit 3 and the like. Further, as shown in FIG. 1B, the editing unit 4 is divided into two units 4a and 4b so that two sets of users can play. One unit 4a includes an editing operation touch panel 400 including a graffiti area, an area for displaying a menu for graffiti, a tool, and the like, and touch pens 49L and 49R used for operations on the editing operation touch panel 400. Is provided. The other unit 4b has the same configuration. Here, the unit 4a seen in the foreground in FIG. 1A is referred to as “graffiti booth A”, and the unit 4b on the opposite side (the back side in the figure) is referred to as “graffiti booth B”.

  The editing operation touch panel 400 according to the present embodiment has a screen configuration in which two users can simultaneously doodle each unit 4a and 4b. In this description, components used by the user on the left side are given reference signs including “L”, and components used by the user on the right side are assigned reference signs including “R”. ing.

  The output unit 5 typically uses a user's mobile phone terminal by communication that does not require contact at a short distance (hereinafter referred to as “non-contact communication”), such as is performed using an infrared port provided in the mobile phone terminal. An output operation touch panel 30 that is operated by a user when transmitting an image, a non-contact communication port 31 that is disposed below the output operation touch panel 30 and performs infrared communication for image transmission, and the non-contact A speaker 32 is provided for notifying the user by voice of operation methods and sound effects necessary for communication. Further, the output unit 5 includes an outlet 33 for taking out a photo sticker, a photo card, or the like on which a composite image that has been edited by the editing unit 4 is printed.

  The output operation touch panel 30 performs various operations necessary for transmitting to a mobile phone terminal having the non-contact communication function when the user wants to see a photo sticker or the like on which a composite image is printed on the mobile phone terminal. It is configured to provide an operation screen for receiving.

  Such an output unit 5 is composed mainly of the same computer as the photographing unit 3, and includes a control device for controlling each part and the like, a network adapter for communicating with the editing unit 4, etc. A network printer 35 for printing an image as a photo sticker or the like is provided.

  In the configuration as described above, after shooting in the shooting room 2, the user uses the graffiti booth A of the editing unit 4 or the touch panel 400 for the editing operation of the graffiti booth B to generate based on the shot image. Graffiti is performed on the graffiti target image. Then, the user prints a composite image generated by graffiti using a network printer, transmits an image using a non-contact communication to a mobile phone terminal having a non-contact communication function, and receives an image received by the mobile phone terminal. Is displayed on the terminal screen.

<2. Functional configuration>
FIG. 4 is a block diagram showing a configuration of a main part of the play photo creating apparatus according to the present embodiment as seen from the functional aspect. As shown in FIG. 4, this play photo creating apparatus functionally has a photographing processing unit 7 for mainly performing a process of photographing a user (photographing process), and a user's graffiti mainly for a graffiti target image. An editing processing unit 8 for performing an editing process on the graffiti target image in response to an operation, and outputting the graffiti target image on which the editing process has been performed as a photo sticker or the like, And an output processing unit 9 that performs a process of outputting (output process).

  The imaging processing unit 7 includes a first control unit 70, an imaging unit 71, a first display / operation unit 72, an I / O control unit 73, an illumination unit 74, and a first communication unit 75. It is configured. The edit processing unit 8 includes a second control unit 80, second display / operation units 81 and 82, and a second communication unit 83. The output processing unit 9 includes a third control unit 90, a third display / operation unit 91, a print output unit 92, an audio output unit 93, a non-contact communication unit 94, and a third communication unit 95. It is constituted by. The first, second, and third communication units 75, 83, and 95 that are network adapters can communicate with each other via a network 6 that is a LAN (Local Area Network).

  The imaging unit 71 corresponds to the camera 10 configured using an imaging element such as a CCD, and captures an image in real time and outputs an image signal representing the image (captured image). This image signal is input to the first control unit 70 and temporarily stored as photographed image data in its internal memory. The captured image data is supplied as a captured image signal from the first control unit 70 to the first display / operation unit 72, and a captured image based on the captured image signal is displayed in real time.

  The first display / operation unit 72 corresponds to the shooting operation touch panel 20 and is used for selecting a background image and a foreground image to be combined with a shot image and determining a layout of a photo to be output. The user's operation and shutter operation are accepted. Signals indicating these operations are input to the first control unit 70 as operation signals. Here, when a predetermined process for photographing the user (corresponding to the selected photographing menu) is started, guidance for the user is displayed on the first display / operation unit 72, and thereafter Based on an instruction from the first control unit 70, a flash is emitted from the flashes 11 to 14 in the shooting direction of the camera 10 after a predetermined time of about several seconds. At that time, an image signal output from the imaging unit 71 as a signal representing a photographed image of the user is input to the first control unit 70, a memory in the first control unit 70, a hard disk device as an auxiliary storage device, or the like. Stored as captured image data.

  The illumination unit 74 corresponds to the strobes 11, 12, 13 </ b> L, 13 </ b> R, 14 disposed at the top, bottom, left, and right positions of the camera 10, and is turned on / off by the I / O control unit 73 based on an instruction from the first control unit 70. Turning off and dimming are controlled. The I / O control unit 73 corresponds to an I / O control device built in the photographing unit 3, and controls the illumination unit 74 based on an instruction from the first control unit 70. Further, an input signal such as a detection signal from a coin detection unit (not shown) described later is transferred to the first control unit 70. The first communication unit 75 corresponds to a network adapter built in the photographing unit 3 and functions as an interface for data transmission / reception via the network 6.

  The first control unit 70 is built in the photographing unit 3 and corresponds to a control device mainly composed of a computer including a CPU, a memory, a frame buffer, a timer, an auxiliary storage device and the like, and is stored in a predetermined memory stored in an internal memory. When the CPU executes the program, an instruction is given to each unit as described above in order to control each unit based on the operation signal or the like input as described above. A graffiti target image is generated based on the photographed image. The generated graffiti target image is written in the frame buffer and displayed on the first display / operation unit 72. Further, the first control unit 70 generates a composite image that is an image in which a predetermined background image or foreground image is drawn on the graffiti target image. The generated composite image is displayed on the first display / operation unit 72 by being written in the frame buffer. When the shooting and the generation of the composite image are completed in this way, the generated composite image is appropriately selected according to the input operation of the user, and then the second corresponding to the editing unit 4 via the first communication unit 75. It is sent to the control unit 80.

  In addition to the above-described components, the photographing unit 3 is further provided with a coin detection unit (not shown) for detecting coins inserted into the coin insertion slot 26 in the photographing unit 3, and the first control unit 70 includes Based on the detection result of the coin detection unit, each unit is controlled so as to allow the user to play with the present photo creation device, such as shooting, selection of the background image and foreground image, and graffiti for a predetermined time. Since the detection operation by the coin detection unit and the control operation by the first control unit 70 based on the detection result are the same as those of the conventional game photo creation device and are well known, detailed description thereof will be omitted.

  The second control unit 80 is a built-in editing unit 4 and corresponds to a control device that is configured around a computer including a CPU, a memory, a frame buffer, an auxiliary storage device, and the like, and is a predetermined program stored in an internal memory. Is executed by the CPU to perform overall control related to the editing process. In addition, the second control unit 80 generates a composite image that is an image obtained by drawing a predetermined image on the graffiti target image based on an operation signal for performing graffiti processing on the captured image sent from the first control unit 70. Generate. The generated composite image is displayed on the corresponding second display / operation unit 81 or 82 in accordance with a user instruction. When the generation of the composite image as described above is completed, the composite image is sent to the output unit 5. If the print output unit 92 is outputting another composite image, a message to that effect is displayed and sent after waiting for completion.

  The second display / operation units 81 and 82 correspond to the editing operation touch panel 400 for graffiti, and accept a user's operation using a touch pen. The second communication unit 83 corresponds to a network adapter built in the editing unit 4 and functions as an interface for data transmission / reception via the network 6.

  The third control unit 90 is built in the output unit 5 and corresponds to a control device mainly composed of a computer including a CPU, a memory, a frame buffer, a timer, an auxiliary storage device, and the like, and is stored in an internal memory. When the CPU executes a predetermined program, overall control relating to output processing is performed. The third control unit 90 stores the composite image sent from the second control unit 80 in the memory as composite image data. The print output unit 92 corresponds to the network printer 35 built in the output unit, and prints a plurality of composite image data stored in the memory as a photo sticker (or photo card) (appropriately laid out). The printed photo sticker or the like is taken out from an outlet 33 provided at the lower front of the output unit 5.

  Further, the third control unit 90 can start printing processing such as a photo sticker based on the composite image sent from the editing unit 4 and simultaneously download the composite image data created by the user to the mobile phone terminal. As described above, an operation screen for accepting a user's input operation is displayed on the third display / operation unit 91. The third display / operation unit 91 corresponds to the output operation touch panel 30 and functions as an input unit that inputs the received input operation to the third control unit 90 as an operation signal.

  Based on the input operation signal, the third control unit 90 issues an instruction to the non-contact communication unit 94 to transmit image data to the user's mobile phone terminal. The non-contact communication unit 94 corresponds to the non-contact communication port 31. The image data is directly transmitted from the non-contact communication unit 94 to the user's mobile phone terminal as a non-contact communication infrared signal. The third communication unit 95 corresponds to a network adapter built in the output unit 5 and functions as an interface for data transmission / reception via the network 6.

  Here, the third control unit 90 writes a demo image (demonstration image) or a mini game image stored in advance in the auxiliary storage device in the frame buffer while the input operation is not performed. Is displayed on the third display / operation unit 91. The audio output unit 93 corresponds to the speaker 32. The audio output unit 93 explains the input operation method to the user in conjunction with the operation screen displayed on the third display / operation unit 91, and the third display / operation unit 91 displays a demo image or a mini game. Play music, sound effects, etc. when displayed.

  Here, the predetermined program executed in each control device is provided by, for example, a DVD-ROM which is a recording medium on which the program is recorded. That is, a DVD-ROM as a recording medium for the predetermined program is attached to a DVD-ROM driving device built in the control device as an auxiliary storage device, and the predetermined program is read from the DVD-ROM as an auxiliary storage device. Installed on the hard disk drive. The predetermined program may be provided via a recording medium (CD-ROM or the like) other than a DVD-ROM or a communication line. Then, when an operation for starting the present photo creating device is performed, the predetermined program installed in the hard disk device is transferred to the memory in the control device and temporarily stored therein, It is executed by the CPU. Thereby, the control process of each part by the control device is realized.

<3. Processing procedure in a photo creation device for play>
As described above, the play photo creating apparatus includes the photographing unit 3, the editing unit 4, and the output unit 5. The photographing unit 3 performs photographing processing, the editing unit 4 performs editing processing, and the output unit 5 performs output processing. It should be noted that when a certain user is playing with the photographing unit 3, another user can play with the editing unit 4, and yet another user can output a composite image with the output unit 5. ing. In other words, the game photo creation device can perform shooting processing, editing processing, and output processing in parallel. Hereinafter, an outline of the processing procedure of the photographing process and the editing process will be described.

<3.1 Shooting process>
FIG. 5 is a flowchart showing a processing procedure of the photographing process in the present embodiment. When the play photo creation device is not used (when no play is performed), a demonstration image is displayed on the shooting operation touch panel 20. When the user inserts a coin into the coin insertion slot 26 while displaying the demo image, play is started (step S100).

  When the play is started, the first control unit 70 accepts selection of a shooting mode by the user (step S110). In step S110, for example, image quality (specifically, one of clear image quality with high contrast, soft image quality with softness, or cool image quality with transparency) is selected, brightness is selected, Whether to shoot automatically or manually is selected. When shooting automatically, a shooting theme is selected. In this case, the first control unit 70 displays on the shooting operation touch panel 20 a screen for allowing the user to select one or more shooting themes from a plurality of shooting themes prepared in advance. The selection operation by the person is accepted. Then, the first control unit 70 acquires selection information based on a user's selection operation, and determines a combination of a frame and a background to be used for shooting based on the selected shooting theme. When shooting manually, the user can freely determine the frame and background. Then, it progresses to step S120 and imaging | photography is performed. By this shooting, the shot image data is stored in the memory of the first control unit 70.

  In step S <b> 130, the graffiti target image generated based on the photographed image is displayed on the photographing operation touch panel 20. Specifically, each time the process of step S130 is performed, graffiti target images are additionally displayed sequentially in the graffiti target image display area 203 of the photographing operation touch panel 20 shown in FIG. Thereafter, the process proceeds to step S140, and the first control unit 70 determines whether or not a predetermined number of images have been taken. As a result of the determination, if the number of images has been shot, the process proceeds to step S150. If the number of images has not been shot, the process returns to step S120. In practice, a time limit for photographing (for example, 3 minutes) is provided.

  In step S150, an image to be actually graffitied is selected (by the user) from a plurality of graffiti subject images. Specifically, the first control unit 70 displays a list of graffiti target images on the shooting operation touch panel 20 in order to allow the user to select an image to be used for graffiti and printing, and performs a selection operation by the user. Accept. Then, the first control unit 70 sends the image selected by the user to the second control unit 80 as an actual graffiti target image. After step S150 ends, the process proceeds to step S160. In step S160, a guidance screen is displayed. Specifically, the first control unit 70 displays a screen for guiding the user to any one of the editing units 4 (4 a or 4 b) on the photographing operation touch panel 20. Thereby, the photographing process is completed.

<3.2 Editing process>
FIG. 6 is a flowchart showing a processing procedure of editing processing in the present embodiment. After completion of the above-described shooting process, the second control unit 80 acquires the graffiti target image (the image selected by the user in step S150 in FIG. 5) sent from the first control unit 70 via the network 6. (Step S200). Thereafter, the timer is set to a predetermined time (time for allowing the user to doodle) and the countdown is started (step S210).

  After the timer countdown starts, the editing unit 4 accepts a graffiti operation (drawing operation) by the user (step S220). In the present embodiment, as one of the graffiti operations, a figure having a shape that can make the user's eyes beautifully appears as an additional image, and the user's eyes (image) included in the graffiti target image are automatically used. It is characterized in that an operation for instructing writing can be performed.

  Specifically, a button for receiving the above instruction (hereinafter referred to as “eye catch button”) is displayed on a display screen (hereinafter referred to as “doodle screen”) for receiving a graffiti operation including the image to be doodled. Then, by performing an operation of selecting this button (that is, bringing the pen tip into contact with the displayed button image), a process of automatically performing the above writing (hereinafter referred to as “eye catch writing process”) is performed. The eye catch writing process will be described in detail below with reference to FIG.

  FIG. 7 is a flowchart showing the processing procedure of the eye catch writing process. In step S300 shown in FIG. 7, the second control unit 80 initializes parameters. This parameter includes a search position indicating a comparison target portion of a photographed image (here, a graffiti target image) used for comparison processing described later, a value indicating a combination of the size and angle of a template image described later, and the like.

  Subsequently, in order to search the position of the user's eyes from the captured image (here, the graffiti target image), the second control unit 80, for example, the eyes stored in advance in a hard disk device that is an auxiliary storage device and the like. An image serving as a nearby model (hereinafter referred to as “model image”) is read into a memory (RAM) (step S310).

  FIG. 8 is a diagram showing this model image. This model image is color and has the shape and color of a typical human eye and its vicinity. Since the model image is compared with the graffiti target image, its size and angle are changed according to parameters set as described later. This changed image (and the same unmodified image as the model image) is hereinafter referred to as a template image. Therefore, the model image is an image that should be a template image.

  Next, the template image is compared with the comparison target portion of the graffiti target image at the search position (step S320). This template image is generated by enlarging or reducing the model image according to the parameter indicating the size and further rotating according to the parameter indicating the angle, as will be described in step S360 described later. The second control unit 80 cuts out the image at the search position from the graffiti target image so as to be the same size as the template image generated in this way, compares the cut out image with the template image as a comparison target portion, The degree of coincidence indicating the degree of approximation is calculated. The method of calculating the degree of coincidence of images by comparison using template images in this way is a simple method called template matching, and various methods are well known. For example, each pixel in two images The degree of coincidence is calculated based on the gradation and color. More specifically, for example, the number of pixels determined to represent black eyes, white eyes, and surrounding skin colors included in the template image is compared with the number of pixels included in the clipped image. Thus, the degree of coincidence can be calculated.

  Subsequently, it is determined whether or not the degree of coincidence calculated in step 320 is equal to or greater than a predetermined threshold (step S330). As a result of this determination, if the coincidence is equal to or greater than the threshold (if Yes in step S330), the process proceeds to step S400. If the coincidence is less than the threshold (if No in step S330), the process is as follows. Proceed to step S340.

  In step S340, the second control unit 80 sets the next search position of the image to be compared with the template image. Specifically, the next search position is a position moved to the right by a predetermined pixel (for example, two pixels) from the current search position of the images to be compared, and the template image (left and right) is moved from this position to the right. When the position moved to the right by the width exceeds the right edge of the graffiti target image, the position moved downward by a predetermined pixel (for example, 2 pixels) from the current search position, and further moved from the position to the left edge of the graffiti target image It is. Thus, the search position changes according to the so-called raster order, but may be changed in any order as long as the entire screen can be searched.

  Next, the second control unit 80 determines whether or not the position moved downward by the height (up and down) of the template image from the next search position exceeds the lower end of the graffiti target image. It is determined whether or not the entire range of the target image has been searched (step S350). If it is determined that the entire range has been searched because the position exceeds the lower end (if Yes in step S350), the process proceeds to step S360, and the entire range is still searched because the position does not exceed the lower end. If it is determined that it is not (No in step S350), the process returns to step S320, and is repeated until the degree of coincidence is greater than or equal to the threshold in the process of step S330 or the entire range of the graffiti target image is searched. (S350 → S320 → S330 → S340 → S350).

  Subsequently, the second control unit 80 sets the size and angle of the template image to the next combination of size and angle, and generates a template image having a new size and angle (step S360). Here, the parameter indicating the size included in the combination indicates the enlargement ratio or the reduction ratio of the model image, and the parameter indicating the angle indicates the rotation angle of the model image. These values are set in advance at appropriate intervals within an appropriate range according to the size and angle of eyes included in a general captured image, and are stored in order (in an auxiliary storage device or the like) as the above combinations. Yes. If the order of this combination is the (statistical) frequency order in a general photographed image, it is possible to quickly search for a case where they match.

  When rotating the template image, the model image may be simply rotated. However, in order to facilitate image comparison processing, the model image is rotated and then rotated from the rotated model image. The template image is generated by cutting out the image so as to be a predetermined rectangular image (similar to that before rotation which is not a rhombus shape) including the image showing the eyes, and complementing the skin color region as necessary.

  Further, in the process of step S360, if there is no next (size / angle) combination, it means that all combinations have been searched. Therefore, no new template image is generated, and all combinations are found. Set a flag indicating that the search has been performed.

  Next, the second control unit 80 determines whether or not all combinations have been searched with reference to the flag set in step S360 (step S370). If it is determined that all combinations have been searched (Yes in step S370), the process proceeds to step S380, and if it is determined that all combinations have not yet been searched (if No in step S370), The process returns to step S320, and is repeated until the matching degree is equal to or greater than the threshold in the process of step S330 or until all combinations are searched (S370 → S320 → S330 → S340 →... → S370).

  The repetition process as described above is specifically performed as shown in FIG. FIG. 9 is a diagram for explaining a procedure for searching for an eye position using a template image. More specifically, FIG. 9A shows that the search for the eye position fails in the entire range of an image for a certain template image. FIG. 9B is a diagram showing an example in which the eye position search is successful for a certain template image. Note that the arrows in the figure indicate that the template image and the graffiti target image are sequentially compared as the search position moves to the right.

  In FIG. 9A, the template image TP1 has the same size and angle as the model image. As shown in FIG. 9A, it can be seen that an image portion matching the template image TP1 (more precisely, having a matching degree equal to or higher than a threshold) does not exist in the graffiti target image. Note that the shape of the female eye in the figure is similar to the template image TP1, but does not match because the size is different (the matching degree is equal to or less than the threshold). In the case shown in FIG. 9A, the search for the eye position in the entire range of the graffiti target image has failed, but as described above, various models can be obtained by appropriately enlarging / reducing or rotating the model image. A template image is generated and the search for eye positions is repeated.

  In FIG. 9B, the template image TP2 is generated based on an image obtained by rotating the model image 30 degrees counterclockwise. As shown in FIG. 9B, the image portion that matches the template image TP2 (more precisely, has a matching degree equal to or higher than the threshold value) includes the first region A1 and the second region in the graffiti target image. A2. An eye-catching image is written in the black eye included in the matched portion. This process will be described in detail later. As described above, the comparison is performed by changing the size and angle of the template image over the entire range of the graffiti target image, and when all the combinations of the size and angle are applied, the above iterative process ends. .

  Subsequently, the second control unit 80 determines whether or not the eye catch image has been written (even once) based on whether or not a search success flag set in step S420 described later has been set (step S420). S380). If the search success flag is set (Yes in step S380), the eye catch writing process ends, and the process returns to the graffiti operation acceptance process in step S220 shown in FIG. If the search success flag has not been set (No in step S380), the process proceeds to step S390.

  In step S <b> 390, the second control unit 80 performs error processing such as a display indicating that the eye catch writing cannot be performed. In other words, in this error processing, there is no template image that matches with a degree of coincidence equal to or higher than a predetermined threshold in the graffiti target image even though all combinations have been searched, so that the user's eyes can be seen beautifully. A display or the like for informing the user that the figure that can be made cannot be automatically written into the user's eye (image) included in the graffiti target image is performed. Thereafter, the eye catch writing process ends, and the process returns to the graffiti operation acceptance process in step S220 shown in FIG.

  Next, in step S400 (because it was determined in step S330 that the degree of coincidence of the images is equal to or greater than a predetermined threshold value), a predetermined eye-catching image is written to the black eye included in the graffiti target image. The center coordinates of the black eye are calculated. For the calculation of the center coordinates of the black eye, it is assumed that the center of the black eye included in the comparison target part in the graffiti target image is at the same relative position as the position where the center of the black eye should be in the template image, for example. That is, various known methods such as a method of calculating the center coordinates of the black eye based on the search position, size, and angle) can be considered.

  Here, any known method may be employed, but it is preferable to employ a method of detecting a circular pupil portion of black eyes included in the comparison target portion and calculating the center coordinates thereof. In this method, for example, a substantially black circular region included in the comparison target portion is detected using the fact that the pupil portion is substantially black, and the center coordinates of the circular region are set as the center coordinates of the black eye. The configuration for detecting the center coordinates of the pupil in this manner is preferable because the detection of the pupil can be easily and accurately, and is particularly preferable because the pupil is generally not hidden by the eye lid during photographing. .

  According to the above method, even if the iris portion is erroneously detected as the pupil portion, the center coordinates of the iris portion usually coincide with the center coordinates of the pupil. Therefore, the center coordinates of the black eye can be accurately calculated. Further, instead of the above method, the center coordinates of the iris portion may be detected. In this case, the center coordinates of the black eye can be accurately calculated even when the detection is impossible due to a bright pupil part. Furthermore, even when the iris is partially hidden by the eye lid or the like, the center coordinates can be easily calculated (from a part of the outer edge of the iris). As a result, it can be said that this method is also a method of detecting the center coordinates of the pupil.

  Subsequently, the second control unit 80 pastes a predetermined eye catch image (as an additional image) using the center coordinates of the black eye as a base point (step S410). Specifically, according to the size and angle of the template image, the eye-catching image is enlarged / reduced or rotated, and the obtained image is compared with the size of the user's black eye included in the graffiti target image (size). Write (overwrite) at the same angle. Hereinafter, a specific description will be given with reference to FIGS. 10 and 11.

  FIG. 10 is a diagram illustrating an eye catch image. The eye catch image Ge shown in FIG. 10 is composed of three figures G1 to G3 with bright gradations (so that the eyes can be seen vividly). The regions other than these three figures G1 to G3 are transparent, but may be a part of an image of an iris or pupil. Further, the center position of the eye catch image Ge is indicated by a one-dot chain line. The eye catch image Ge is a model in which the center position (here, the center position of the eye catch image Ge, but may be set to a predetermined position other than the center position) is aligned with the center position of the black eye in the model image. When the image is overwritten (overwritten), the three graphics G1 to G3 are arranged at appropriate positions on the model image on the iris. Therefore, if this eye-catching image Ge is used as an additional image and the black eye portion in the graffiti target image (which matches or approximates the model image) is appropriately changed in size and angle (overwritten), the eye looks beautiful. Can do.

  In addition, it is possible to change the size and angle of a circular black eye portion, that is, to appropriately enlarge, reduce, or rotate the model image and combine it with the entire eye image (including the eyelid). Easier to synthesize. Because the shape of the whole eye can vary (depending on the person and facial expression), but the shape of the round black eye does not change, so it is easy to set the correct size or angle and align it correctly. Because you can.

  Note that the eye catch image shown in FIG. 10 is an example, and other shapes such as a star shape and a heart shape are conceivable. However, the eye catch image has a shape that allows the user's eyes to be photographed beautifully. Specifically, the shape may be any shape that is suitable for photographing so that the photograph can be taken vividly or impressively, or the user's eyes can be enjoyed. Further, the color of the figure may be simply white, but it may be any color that is suitable for photographing as in the case of the above shape.

  FIG. 11 is a diagram illustrating a state in which the eye catch image is written in the eye portion of the graffiti target image. The partial eye image shown in FIG. 11 corresponds to, for example, an image obtained by enlarging the image of the area A1 shown in FIG. 9, and the center of the eye catch image is aligned with the center coordinates of the pupil region pp to further form the eye catch image 3 Two figures G1 to G3 are rotated 30 degrees counterclockwise and written in the iris area ir (overwritten and synthesized). Note that an eye-catching image may be written in the white-eye area wh shown in FIG. 11. However, since the high-brightness figure is originally difficult to appear in the white-eye area wh, the eye-catching image is the iris area ir (or the iris area ir and the pupil area). It is preferable to write in a black eye portion including pp). Since the black eye portion is circular, the eye catch image may be written without being rotated. However, the eye catch image is configured so that the above-mentioned figure does not reach the portion that is easily hidden by the eyelid or the like. In this case, the configuration of the present embodiment that is rotated is particularly preferable.

  Next, in step S420, a search success flag (used for determination in step S380) is set and the next search position is set. Unlike the next search position set in step S340, the next search position is set so as not to include a portion in the graffiti target image that has already been compared with the template image. This is to speed up the search and prevent false detection. Specifically, the next search position here is a position moved to the right by the left and right widths of the template image from the current search position of the images to be compared. When the position moved right by the width exceeds the right edge of the graffiti target image, the position is moved down by a predetermined pixel (for example, 2 pixels) from the current search position, and further moved from the position to the left edge of the graffiti target image. It is. Thereafter, the process proceeds to step S350. The processing content after step S350 is as described above. Note that the setting content of the next search position in step S420 may be the same as the processing content in step S340. When the eye catch writing process is performed according to the above processing procedure, the process returns to the graffiti operation acceptance process in step S220 shown in FIG.

  As shown in FIG. 6, when the remaining time for allowing the user to doodle is 60 seconds thereafter, the second control unit 80 determines whether or not to extend the doodle time (step S230). Whether or not to extend the graffiti time is determined by whether or not the user who will play next in the unit 4a or 4b is in the shooting room 2 (whether or not the shooting process is performed in the shooting unit 3). Is done. As a result, if it is determined to extend the graffiti time, the process proceeds to step S240. On the other hand, if it is determined that the graffiti time is not extended, the process proceeds to step S250. In step S240, the timer is set to a predetermined time (time for allowing the user to further doodle), and the countdown is started again. After step S240 ends, the process proceeds to step S250.

  In step S250, as in step S220, the editing unit 4 accepts a graffiti operation by the user. Therefore, it is possible to perform eye catch writing processing. When the graffiti time reaches 0 (ends), the division pattern of the output photo is selected (step S260). Specifically, the second control unit 80 displays on the editing operation touch panel 400 a screen for allowing the user to select one of the plurality of division patterns prepared in advance, and the user can select the division pattern. The selection operation by is accepted. And the 2nd control part 80 acquires selection information based on a user's selection operation. Thereafter, the process proceeds to step S270.

  In step S270, a guidance screen is displayed. Specifically, the second control unit 80 displays a screen for guiding the user to the output unit 5 on the editing operation touch panel 400. Thereby, the editing process ends. Thereafter, the output processing described above is performed in the output unit 5.

<4. Effect>
As described above, in the present embodiment, in the eye catch writing process shown in FIG. 7, the template image and the comparison target portion of the graffiti target image are compared, so that the user's eyes ( Corresponding image portion) is automatically detected, and an eye catch image is appropriately pasted to the center of the detected black eye. As a result, it does not require any new parts such as a video device or a light emitter, and does not require drawing of the figure by hand (doodle). It is possible to generate a composite image in which a desired figure (typically an eye catch) is attached to the user's eyes.

<5. Modification>
In the above embodiment, the eye catch writing process is performed by accepting an operation for selecting the eye catch button by the user during the graffiti operation accept process (S220, S250). A configuration may be adopted in which the above operation is accepted and eye catch writing processing is performed at a time other than the graffiti operation, such as at the time of selection (S110), at the time of selection of the graffiti target image (S150), or during output processing.

  The eye catch button may be omitted, and an eye catch image may be always written without waiting for the user's selection. For example, the configuration shown in FIG. 7 may always be performed before the graffiti operation acceptance processing (S220), or may be always performed after the graffiti ends.

  Furthermore, instead of the eye catch button, a plurality of eye catch selection buttons may be provided, and a configuration in which a plurality of eye catch images can be selected by a user operation may be adopted. For example, when different eye catch images stored in advance are associated with a plurality of eye catch buttons and the user selects an eye catch button corresponding to a certain eye catch image, the selected eye catch button is selected. A configuration in which an eye catch writing process using an eye catch image corresponding to a catch button is performed may be used. With such a configuration, the user can select a preferable eye catch image from various eye catch images prepared in advance.

  Furthermore, in addition to the eye catch button, a button that erases the eye catch image may be provided. For example, an eye catch image already synthesized by the user's operation may be erased from the synthesized image. For example, when a captured image of (eye) before writing an eye-catching image is saved and the delete button is selected, the saved image before writing is rewritten and restored to the original state. A configuration in which processing is performed may be used. With such a configuration, the user can easily cancel the synthesis of the eye catch image.

  In the above embodiment, one model image stored in advance is read (S310), and the template image is generated by enlarging / reducing and rotating the read model image. It may be stored. In this configuration, the eye catch writing process shown in FIG. 7 is repeated for each model image.

  In addition, various image recognition methods that can determine the position of the eye from the captured image, such as a well-known feature extraction method that does not use a template, can also be used. For example, face detection is performed (based on a well-known method such as using the shape of a skin color region), the position of an eye that exists in the vicinity of the eye position that should exist on the face as a detection result is detected, and the detected eye It may be configured to write an eye catch image at the position. If the eye is detected after face detection is performed in this manner, eye detection can be performed at a higher speed and more accurately than when eye detection is performed on the entire image, and a well-known face detection LSI chip is used. By using hardware such as the above, it is possible to speed up eye detection compared to software processing.

  In the above embodiment, the center position of the eye catch image stored in advance is aligned with the center position of the user's black eye in the graffiti target image, and the eye catch image is appropriately enlarged / reduced (and rotated) and pasted to the black eye. Although it is a structure, it is possible to use various methods which can affix an eye catch image suitably. For example, the configuration may be such that the outer edge (shape) of the user's iris in the graffiti target image is detected, and the eye catch image is appropriately enlarged / reduced and pasted in accordance with the detected outer edge (size).

  In addition, when the user's eyes are half-eye, etc., when part of the black eye (iris part and pupil part) is hidden by the eyelid, the eye-catching image is displayed only on the black eye part that is not hidden by the eyelid. The structure which cuts out and affixes suitably may be sufficient. For example, since there is a difference in the brightness (gradation value) of the pixel between the skin-colored eyelid and the nearly black eye, when you paste a (typically) bright figure that makes up an eye-catching image, graffiti When the target pixel in the target image is brighter than a predetermined threshold value, the place may be the eye lid, and the bright figure may not be pasted.

  In the above embodiment, the eye-catching image is written at the eye position of the graffiti target image. However, the shape (and color) suitable for shooting is such that the eyes of the user other than the eye-catching image can be seen beautifully or happily. Any configuration for writing an image may be used. For example, the shape of the pupil included in the graffiti target image is stored in advance at the position of the pupil so that it is a vertical slit shape (like a cat's eye) or a horizontal slit shape (like a goat's eye). Alternatively, a configuration for writing the slit-shaped pupil figure may be used.

  Note that the configuration of the embodiment and the modification example described above partially changes the image included in the graffiti target image, such as changing the iris color included in the graffiti target image or changing the size of the eye. It differs from the structure which deform | transforms. That is, the configuration for partially deforming the image is different from the configuration of the present embodiment in which a graphic image stored in advance is written. Therefore, even if the structure of deforming the image can make the user's eyes look beautiful or fun, it is merely a deformation. Therefore, it is easily predictable for the user and is less surprising, so that the playability or entertainment is less than in the case of this embodiment, and it is difficult to give a strong impression. Therefore, it is preferable to write an image of a shape (and color) suitable for photographing so that the user's eyes can be seen beautifully or happily on the graffiti target image as in the present embodiment and its modifications.

3 ... Shooting unit 4 ... Editing unit 5 ... Output unit 6 ... Network (LAN)
DESCRIPTION OF SYMBOLS 10 ... Camera 20 ... Touch panel for photography operation 30 ... Touch panel for output operation 35 ... Network printer 31 ... Non-contact communication port (infrared port)
32 ... Speaker 70 ... First control unit 72 ... First display / operation unit (touch panel for photographing operation)
75. First communication unit (network adapter)
80 ... second control unit 81, 82 ... second display / operation unit (touch panel for editing operation)
90 ... third control unit 91 ... third display / operation unit (touch panel for output operation)
92 ... Print output section (network printer)
93 ... Audio output unit (speaker)
94 ... Non-contact communication part (non-contact communication port)
80 ... second control unit 81, 82 ... second display / operation unit (touch panel for editing operation)
83 ... 2nd communication part (network adapter)

Claims (8)

An automatic photo creation device comprising a photographing means for photographing an image including a user who is a subject, and an output means for outputting a composite image including a photographed image obtained by the photographing means as a photograph,
Eye detection means for detecting an image of the user's eye included in the captured image;
Storage means for storing an additional image indicating a figure to be combined with the eye image;
Image synthesis means for generating the synthesized image by synthesizing an image corresponding to the additional image stored in the storage means with the eye image detected by the eye detecting means. An automatic photo creation device. The eye detection means detects the center position of the pupil in the image of the user's eyes,
The image synthesizing unit aligns the predetermined position on the additional image corresponding to the center position of the pupil with the center position of the pupil detected by the eye detecting unit, and adds the additional position. The automatic photo creation apparatus according to claim 1, wherein an image corresponding to the image is synthesized.   The image synthesizing unit enlarges, reduces, or rotates the additional image stored in the storage unit in accordance with the black eye image of the eye image, thereby corresponding to the additional image. The automatic photo creating apparatus according to claim 1, wherein an image corresponding to the additional image is synthesized with the black eye image. The storage means further stores an image including an eye image to be a template image,
The eye detection unit generates a plurality of types of template images by enlarging, reducing, or rotating an image including an eye image stored in the storage unit, and is an image included in the captured image. The automatic photo creation according to any one of claims 1 to 3, wherein an image that matches or approximates a plurality of types of generated template images is detected as an image of the user's eyes. apparatus. An input means for receiving an operation input by the user for selecting a figure to be synthesized with the eye image;
The storage means stores a plurality of additional images showing different figures from each other,
The image synthesizing unit is configured to detect, by the eye detection unit, an image corresponding to one of the plurality of additional images indicating a graphic selected according to the operation input received by the input unit. 5. The automatic photo creating apparatus according to claim 1, wherein the synthesized image is generated by synthesizing the image. An input unit for receiving an operation input by the user for erasing the additional image synthesized by the image synthesizing unit from the synthesized image;
The image synthesizing unit restores the synthesized image to a state before the additional image is synthesized by the image synthesizing unit in response to an operation input received by the input unit. The automatic photo production apparatus according to any one of claims 1 to 4. A method for controlling an automatic photo creation apparatus, comprising: a photographing step for photographing an image including a user who is a subject; and an output step for outputting a composite image including a photographed image obtained in the photographing step as a photograph.
An eye detection step of detecting an image of the user's eye included in the captured image;
The synthesized image is generated by synthesizing an image corresponding to an additional image indicating a figure to be synthesized with the eye image stored in advance with the eye image detected by the eye detecting step. And an image synthesizing step.   A program causing a computer to execute the control method of the automatic photo creating apparatus according to claim 7.

Images