JP2017004258A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform finer image processing of a human face image. When a touch operation is performed in a state where an image including a person's face is displayed on a touch display unit 6, a control unit 1 specifies a part of the person's face specified by the touch position of the touch operation And different image processing (make-up process) is performed on the part depending on the part. For example, if the part specified by the touch operation is skin, image processing that makes wrinkles and rough skin less noticeable by performing exposure correction, saturation and lightness correction, soft focus correction, etc. on the image part Execute. [Selection] Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing an image including a human face.

  Recently, in an imaging apparatus such as a digital camera or a video camera, an opportunity for photographing itself is increasing. In particular, because there is a strong demand for more beautiful images for women, if the face of a subject (eg, a photographer) is detected when photographing a person and it is determined that the detected face is a woman, for example, exposure compensation, software An image pickup apparatus that performs processing such as focus correction, skin beautification correction, and correction of making a person look slim is proposed as disclosed in Patent Document 1. However, since the above-described processing performs correction with a uniform intensity automatically when it is determined that the face is a woman's face, there is a problem that the correction processing cannot be performed with the intensity desired by the photographer or the subject. there were. In this way, in order to perform correction processing at the intensity desired by the photographer or subject, it is necessary to manually set the correction intensity for each correction process and shoot. Patent Document 2 is disclosed as a technique that makes it possible to more easily set the correction intensity of processing.

Japanese Patent No. 5239126 Japanese Patent No. 5083116

According to Patent Document 2 described above, correction strengths of a plurality of types of correction processing can be set according to the beautiful face level set by a key operation, and different image processing can be performed on a human face for each beautiful face level. It becomes possible.
By the way, people who don't like whitening and patchy eyes, people who care about wrinkles rather than wrinkles, etc. have different facial consciousness depending on the person, and even if the same person is better off your mouth When it is better to tighten, there are cases where it is desired to change only a part of the face rather than the beautiful face.

  An object of the present invention is to enable finer image processing of a human face image.

In order to solve the above-described problems, the present invention
An image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Touch detection means for detecting a touch operation on the touch operation means;
It is determined whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person in the image including the face of the person displayed on the display means, or designated by the touch position. Discriminating means for discriminating a part of the face
Image processing means for executing, on the face area or part, different image processing depending on whether or not the face part of the person is discriminated by the discrimination means or on the face part;
It is characterized by providing.

  According to the present invention, a person's face image can be processed more finely.

The block diagram which showed the basic component of the imaging device applied as an image processing apparatus. (1), (2) is an external view showing an imaging device (digital camera) configured to be able to take a self-photographing photographer himself while viewing a live view image. The figure for demonstrating the makeup table 3c. The figure for demonstrating concretely the content of the various makeup processes memorize | stored in the makeup table 3c. The flowchart which showed the operation | movement started when it switches to imaging | photography mode (REC mode). 6 is a flowchart for explaining in detail a makeup process (step A6 in FIG. 5). (1)-(3) is the figure which showed the example of an image display in the process which transfers to makeup processing. The figure for demonstrating the makeup table 3c in 2nd Embodiment. 6 is a flowchart for explaining in detail makeup processing (step A6 in FIG. 5) according to the second embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
First, a first embodiment of the present invention will be described with reference to FIGS.
In the present embodiment, in a state in which an image including a person's face is displayed on the touch display unit, a makeup process corresponding to the part of the person's face corresponding to the touch-operated position is performed on the face of the displayed person's face. It is characterized by being performed on a part. The touch operation is not limited to a single tap that is touched once and released, but a make-up process can be executed according to the trajectory of a tracing operation that moves the touch position while maintaining the touch state, thereby enabling fine image processing.
This embodiment exemplifies a case where the present invention is applied to an imaging apparatus as an image processing apparatus, and FIG. 1 is a block diagram illustrating basic components of the imaging apparatus.
This imaging apparatus is a digital camera provided with an imaging function capable of photographing a subject with high definition, an image reproduction function for arbitrarily reading and reproducing a captured image (recorded image) recorded and stored. The control unit 1 operates by supplying power from the power source unit (secondary battery) 2 and controls the overall operation of the imaging apparatus (digital camera) according to various programs stored in the storage unit 3. The control unit 1 is provided with a CPU (Central Processing Unit) and a memory (not shown).

  The storage unit 3 includes, for example, a ROM, a flash memory, and the like, and stores programs and various applications for realizing the present embodiment in accordance with operation procedures shown in FIGS. 5 and 6 to be described later. A program memory 3a, a work memory 3b for temporarily storing data such as flags, and a makeup table 3c described later. The storage unit 3 may be configured to include a removable portable memory (recording medium) such as an SD card or an IC card that can store a plurality of captured images (saved image group), for example, Although not shown, a storage area on a predetermined server device side may be included when connected to a network via a communication function. A portable memory such as an SD card is a memory that stores a plurality of high-quality captured images (recorded images).

  Although not shown, the operation unit 4 includes a power key for turning on / off the power as various hard keys, a mode capable of shooting (REC mode), and a mode for playing back a captured image (recorded image) (PLAY mode). ) And various hard keys such as a mode change key for switching to a makeup mode for executing a process for making up a person's face in the image in the REC mode, and a release key for instructing photographing. Therefore, the control unit 1 executes, for example, a mode change process, a photographing process, etc. according to the operation of the hard key.

  Although not shown, the imaging unit 5 can capture a subject with high definition by forming a subject image from the photographing lens 5a on an image sensor (CCD, CMOS, etc.). It has elements, various sensors, an analog processing unit, and a digital processing unit. The control unit 1 performs normal image processing such as white balance adjustment, color interpolation, gradation conversion, and contour enhancement on the image captured by the image capturing unit 5. Makeup processing (image processing) for performing image processing for making up or shaping the face is performed. Further, the control unit 1 performs image compression processing on the image processed image, and then records and saves the image on the recording medium of the storage unit 3.

  The touch display unit 6 includes a display unit 6a such as high-definition liquid crystal or organic EL (Electro Luminescence) for displaying an image, and a touch operation unit 6b having a transparent touch panel, and the touch operation unit 6b is stacked on the display unit 6a. The arranged touch screen is configured, and various software keys (icons, touch keys) are assigned and arranged to display their function names, or a touch operation with a finger or the like is detected and an operation signal corresponding to the touch operation is input. To do. The display screen of the touch display unit 6 may be a monitor screen (live view screen) that displays captured images (live view images) in real time or a playback screen that plays back captured images (recorded images). .

  The touch operation unit 6b employs, for example, an electrostatic capacity method or a resistance film method, and the control unit 1 performs a touch operation unit based on a contact presence / absence signal (touch-on signal / touch-off signal) from the touch operation unit 6b. The coordinate position (contact position), the contact direction, the contact distance, and the contact time on 6b are detected to determine the presence or absence of the touch operation and the type of the touch operation. Note that the types of touch operations include “single tap” in which an arbitrary position is touched and immediately released, and “race” in which the user continues to touch and moves in an arbitrary direction.

  Note that the touch operation described above is not limited to a touch operation that directly touches with the finger or pen on the touch operation unit 6b, but as an operation that approximates the touch operation, for example, an electrostatic due to the approach or movement of the finger or pen This means that it includes a proximity operation that detects the position of a finger or pen in accordance with a change in capacitance. That is, not only a contact-type touch panel that detects a contact operation but also a proximity-type touch panel or a device may be used. However, in the present embodiment, a contact operation on a contact-type touch panel is illustrated as a touch operation. Is shown.

FIG. 2 is an external view of an image pickup apparatus (digital camera) configured to perform so-called self-photographing of a photographer himself / herself while viewing a live view image.
A housing 11 constituting an imaging device (digital camera) connects a thin rectangular display block 11a, a rectangular frame frame block 11b surrounding the display block 11a, and the display block 11a and the frame block 11b. In the configuration having the lens block 11c, one end central portion (upper center portion in the illustrated example) of the display block 11a and one end central portion (lower end central portion in the illustrated example) of the display block 11a are a hinge portion for lateral rotation. It is attached so as to be able to rotate laterally through (not shown). Further, the inner sides of both ends of the frame block 11b (inside the left and right sides in the example of FIG. 2A) and the both ends of the lens block 11c (the left and right sides in the example shown in the figure) are hinge portions for vertical rotation (not shown). It is attached so as to be able to rotate vertically.

  The display block 11a is provided with a touch display unit 6 at the center thereof, and the lens block 11c is provided with a photographing lens 5a constituting the image pickup unit 5, and the above-described lateral rotation hinge unit. By rotating the display block 11a through 180 degrees with respect to the lens block 11c, the touch display unit 6 and the photographing lens 5a can be directed in the same direction or in the opposite direction. The display block 11a and the lens block 11c are integrated into a main body block. The inner side of both ends of the frame block 11b (inside the upper left and right sides in the example of FIG. 2 (1)) and the both ends of the main body block (left and right in the example shown in the figure). The side portion may be attached so as to be rotatable in the vertical direction via a hinge portion (not shown) for vertical rotation. 2 (1) shows a state in which the frame block 11b is closed with respect to the display block 11a in a state where the touch display unit 6 and the photographing lens 5a are directed in the same direction in order to enable self-portrait. ) Shows a state in which the frame block 11b is opened with respect to the display block 11a.

FIG. 3 is a diagram for explaining makeup table 3c.
The makeup table 3c is a table used at the time of makeup processing (image processing for applying makeup or shaping) on a person's face in an image, and is associated with each part constituting the person's face. Information for accessing the makeup process is stored. The makeup table 3c has items of “a part of a person's face”, “position or state in the part”, “person attribute: female”, and “person attribute: male”. Can be changed arbitrarily.

  The “person's face part” is an item for storing the part of the face to be subjected to the makeup process, and the example shown in the figure is “skin”, “eyes”, “eyebrows”, “睫”, “eyebrows” , “Teeth” is stored as a processing target (selection candidate). When the face of the person in the image is touched, the control unit 1 determines the face part corresponding to the first touch position, refers to the makeup table 3c, "Skin", "eyes", "eyebrows", "” "," eyebrows ", and" teeth "shown by The make-up process is executed.

  The “position or state within the part” is an item that stores the position or state within the part in association with each part indicated by “the part of the person's face”. This corresponds to the case where “wrinkles, rough skin, pores, etc.”, “stains, dullness, wrinkles, etc.” are stored as information indicating the state of the portion corresponding to “skin” of “part”. Further, this corresponds to the case where “inside of eyes” and “outside of eyes” are stored as information indicating the position in the part corresponding to “eyes”. In the example shown in the figure, the “position or state in the part” corresponding to “eyebrows”, “” ”,“ eyebrows ”,“ tooth ”is not stored, but“ position or state in the part ”is not stored. May be stored. When detecting a touch operation in the face of a person, the control unit 1 determines the state if the touch position is “skin”, and determines the position if the touch position is “eyes”. The makeup table 3c is referred to and it is determined whether it corresponds to one of the “positions or states in the part”.

  “Person attribute: female” and “Person attribute: male” are items for storing information for accessing makeup processing for each gender as a person attribute. In the illustrated example, “person's face part” is In the case of “skin”, when the “position or state in the part” is “wrinkle, rough skin, pores, etc.”, corresponding to “person attribute: female”, “person attribute: male” “Skin treatment (1)” is stored as a common makeup process, and “stain, dullness, wrinkle, etc.” corresponds to “person attribute: female” and “person attribute: male”. In this example, “skin treatment (2)” is stored as a common process. In addition, when the “part of the human face” is “eyes” and the “position or state within the part” is “inside of the eyes”, the “eye attribute: female” corresponds to the “eyes”. Process (1) ”is stored, and“ eye process (3) ”is stored as another process different from that for female corresponding to“ person attribute: male ”. In the case of “outside of eyes”, “eye processing (2)” is stored as a common process corresponding to “person attribute: female” and “person attribute: male”. Yes.

  Furthermore, when “the face part of the person” is “eyebrow”, “eyebrow processing (1)” is stored corresponding to “person attribute: female”, and “person attribute: male” is supported. In this example, “eyebrow process (2)” is stored as another process different from that for a woman. If the “person's face part” is “睫”, the makeup process of “睫 treatment” is stored in correspondence with “person attribute: female”, and “person attribute: male” is stored. Correspondingly, a case where makeup processing is not stored is shown. When “portion of person's face” is “mouth”, “mouth processing (1)” is stored corresponding to “person attribute: female”, and “person attribute: male” is stored. In this example, “mouth processing (2)” is stored as another processing different from that for a woman. In the case where “person's face part” is “tooth”, “tooth treatment” is stored as a common process corresponding to “person attribute: female” and “person attribute: male”. ing.

  In a state where a live view image including a person's face is displayed on the touch display unit 6, the control unit 1, when a predetermined part in the face is touched, the attribute of the person corresponding to the touch position And the face part, the touch position or state in the part is determined, and the makeup table 3c is searched based on the determination result (touch position or state in the part or part). The corresponding makeup process is specified, and this makeup process is executed for that part.

FIG. 4 is a diagram for specifically explaining the contents of various makeup processes stored in makeup table 3c.
“Skin treatment (1)” and “Skin treatment (2)” are makeup processes (skin-beautifying processes) performed on the captured image. For example, the exposure of the image is corrected, and the saturation and brightness are adjusted. Image processing is performed to make the skin white by performing correction processing, or to make wrinkles, rough skin, and spots less noticeable by performing soft focus, etc. In particular, “skin processing (1)” “Skin treatment (2)” is a process for making spots, dullness, etc. inconspicuous. “Eye Processing (1)”, “Eye Processing (2)”, and “Eye Processing (3)” are for deforming the eyes and correcting the saturation and brightness of the eyes. “Process (1)” is a process that opens the eyes, and “Eye process (2)” is a process that reduces the redness of the white eye by correcting the saturation and brightness of the white eye. The eye processing (3) "is processing for sharpening the eyes.

“Eyebrow treatment (1)” and “eyebrow treatment (2)” deform eyebrows and correct the saturation and brightness of eyebrows. In particular, “eyebrows treatment (1)” This is a process of thinning out the useless hair, and “eyebrow process (2)” is a process of thickening the eyebrows. In addition, the “wrinkle process” is a process that makes the wrinkle longer and curls, and the “mouth process (1)” is a process that reduces the mouth and breaks the mouth, and “mouth process (2)” This is a process of tightening the mouth. “Tooth treatment” is a process for whitening yellowed teeth. Note that the makeup process is not limited to the above example, and for example, eye shadow may be applied to the upper eyelid or lipstick may be applied to the lips.
These makeup processes are known techniques, and in the present embodiment, the known techniques are used. Therefore, detailed description thereof will be omitted.

  Next, an operation concept of the imaging apparatus (digital camera) in the first embodiment will be described with reference to flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. The same applies to other embodiments described later. In addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium. FIG. 5 is a flowchart showing an outline of the operation of the characteristic part of the present embodiment in the entire operation of the imaging apparatus (digital camera). When the flow of FIG. Return to the flow (not shown).

FIG. 5 is a flowchart showing an operation that is started when switching to the photographing mode (REC mode). Hereinafter, the operation will be described with reference to display examples shown in FIGS. Shall be explained. FIG. 7 is a diagram showing an image display example in the process of shifting to the makeup process.
First, the control unit 1 starts an operation of displaying the captured image acquired from the imaging unit 5 on the touch display unit 6 as a live view image (step A1). FIG. 7A shows a display example of a live view image including a human face. In this state, it is checked whether the mode switching operation for instructing the transition to the makeup mode has been performed (step A2), whether the mode switching operation to exit from the makeup mode has been performed (step A7), or by the release key It is checked whether a photographing instruction operation has been performed (step A9), or whether a mode switching operation for ending the REC mode has been performed (step A11).

  If a mode switching operation for instructing the transition to the makeup mode is performed (YES in step A2), a captured image displayed as a live view image at the time of the operation is acquired and image analysis is performed. After performing the face detection process (step A3) for detecting the face area of the person, the captured image is enlarged and displayed with the detected face as the center (step A4). Note that this face detection process is a technique that is generally used in cameras, and in the present embodiment, the well-known technique is used. Therefore, a specific description thereof will be omitted.

  FIG. 7B shows a case where the face of the person in the live view image shown in FIG. In this case, for example, when a person's face is shown at the edge of the screen, the position of the person may be temporarily moved so that the face is at the center of the screen. Thus, in the state where the face of the person is enlarged and displayed on the touch display unit 6, it is checked whether the touch operation to the touch display unit 6 has been performed (step A5), and if the touch operation has not been performed (step A5). The process proceeds to step A7 described above, but when a touch operation is performed (YES in step A5), the process proceeds to a makeup process (step A6). That is, when a touch operation is performed on a touch screen on which a person's face is magnified (face magnified display), it is determined that the target of makeup processing has been designated by the user, and makeup processing (image processing) Move on.

FIG. 6 is a flowchart for explaining the makeup process (step A6 in FIG. 5) in detail.
First, the control unit 1 checks whether the touch position of the touch operation is within the face area of the person (step B1). If an area that is not a person is touch-operated, or an area that is not a face is touch-operated even if it is a touch operation on a person (NO in step B1), the process proceeds to step B2, and the touch operation continues. Check whether it is touching (touching). In this case, if the touch operation continues (YES in step B2) regardless of whether the touch position is moved, the process returns to the first step B1.

  FIG. 7 (3) shows the “skin” portion of the face in the state where the face area of the person is enlarged as shown in FIG. 7 (2), in the example shown, the cheek (skin on the chin) ) Is shown as an example when the user's index finger is touched. If the touch position is within the face area of the person as shown (YES in step B1), the captured image at that time is acquired. The image analysis is performed, and the face at the touch position, the ornament, the hairstyle, the clothes, etc. are comprehensively determined to determine the gender as the attribute of the person (step B3). Further, the captured image is analyzed to determine which part of the face has been touched (step B4).

  The discrimination process in step B4 is repeatedly executed at predetermined timings while the touch operation is continued (YES in step B2). Based on the discrimination result, the discrimination process is the same as the previous time, that is, at the current timing. It is checked whether or not the determined part is the same as the part determined at the previous timing (step B5). This step B5 is for checking whether the touched part has changed due to the movement of the touch position. For example, during the tracing operation, makeup is applied to the skin that is the first touch position. If the finger has been moved to the eye despite being performed, if it is different from the previous part so as not to inadvertently perform makeup processing on the eye (NO in step B5) ), And proceeds to step B2 in order to invalidate the makeup process for this touch operation.

  On the other hand, if the current part is the same as the previous part (YES in step B5), the process proceeds to a process of determining the touch position or state in the part (step B6). For example, if the part at the touch position is “skin”, “skin” is set as the processing target, and the state within the part is determined as wrinkles, rough skin, spots, dullness, etc., and the part at the touch position is “eyes”. If there is, “inside of eyes” and “outside of eyes” are discriminated as the touch position in the part with “eye” as the processing target. Note that this determination process is a publicly-known technique that is generally used, and in the present embodiment, the known technique is used, and therefore a specific description thereof will be omitted.

  Next, the position or state in the part determined at the current timing is compared with the position or state in the part determined at the previous timing to check whether they are the same (step B7). This step B7 is to check whether or not the position or state in the part has changed due to the movement of the touch position. For example, from the part with the eyelid to the part without the eyelid, from the inside of the eye to the outside of the eye Thus, even if the touch is in the same part, if the finger is moved, the position or state in the previous part is different (NO in step B7), so the makeup process for the touch operation in this case is performed. In order to make it invalid, it moves to step B2.

  On the other hand, if it is the same as the touch position or state in the previous part (YES in step B7), makeup is performed based on the above-described determination results (person attributes, face parts, position or state in the part). The corresponding makeup process is searched with reference to the table 3c (step B8). If there is no corresponding makeup process (NO in step B9), for example, if the attribute of the person is “male” and the facial part is “睫”, there is no corresponding makeup process. Therefore, the process proceeds to step B2, but if there is a corresponding makeup process (YES in step B9), for example, the attribute of the person is “female”, the facial part is “skin”, and the state in the part is “皺, If it is “rough skin, pores, etc.”, “Skin treatment (1)” exists as the corresponding makeup process, so the process moves to the next step B10, and another window is opened on the face enlargement display, and the user performs the touch operation. The image part hidden by the finger is further magnified and displayed in this separate window, and the contents of the corresponding makeup process are displayed in guidance by superimposing a character string message on the predetermined area of the face magnified display. .

  For example, as shown in FIG. 7 (3), when the cheek (skin) part on the chin is touched with the finger of the user, the image part of the cheek on the chin hidden by the finger is specified. The image portion is further enlarged and displayed in a separate window on the face enlargement display, and the text message “Make the wrinkle, rough skin, etc. inconspicuous” appears as the content of the corresponding makeup process “skin treatment (1)”. Guidance is displayed in a predetermined area of the enlarged display. Note that an image part including the finger contact area and its peripheral area may be an image part hidden behind the finger. The character string message to be displayed for guidance is the same as the processing content shown in FIG. 4, but the present invention is not limited to this, and an icon indicating the processing content may be displayed.

Next, the corresponding makeup processing is executed (step B11). If the makeup processing is, for example, “skin processing (1)”, the facial skin is processed and the image portion is processed. By performing exposure correction, saturation and lightness correction, soft focus correction, etc., image processing that makes wrinkles and rough skin less noticeable is executed. In the case of “eye processing (2)”, image processing for reducing redness of the white eye is executed by correcting the saturation, lightness, etc. of the image portion of the white eye. When makeup processing is performed in this way, after that, instead of face enlargement display, the image after makeup is switched and displayed (step B12).
The position at which the makeup process is performed may be a predetermined range centered on the point detected by the touch position of the touch operation, or when the touch operation is performed with a finger, the touch position may be It may be detected as a surface, and the surface may be used as a range, or the range may be changed as appropriate depending on the part. When the touch operation is “race”, the makeup process may be executed on the trace locus that is the touch position that moves continuously.

  Thereafter, the process proceeds to step B13 to check whether the touch operation is released (finger is released) or not. If the touch operation is continuing (NO in step B13), the process waits until the touch operation is released. Therefore, during that time, the user can confirm the enlarged face display after makeup. If the touch operation is released (NO in step B13), the process of canceling (erasing) the above-described separate window display and guidance display is performed (step B14), and then the flow of FIG. 6 is exited.

  When such makeup processing (step A6 in FIG. 5) is completed, the process returns to step A5 in FIG. 5 to check whether a touch operation for performing makeup processing on another part of the person's face has been performed. When a touch operation for performing makeup processing on another part of the person's face is performed (YES in step A5), the above-described makeup processing of FIG. 6 is performed and no touch operation is performed. Moves to step A7 described above.

  When an operation for exiting the makeup mode is performed (YES in step A7), the process of canceling the above-described face enlarged display and returning to the normal display state (step A8) is performed, and then the process returns to step A2. When a shooting instruction operation using the release key is performed (step A9), the process proceeds to a shooting process (step A10) in which the captured image is converted into a shot image and compressed and recorded, but the makeup process (step A6) described above is performed. With respect to the applied captured image, the captured image after the makeup process is converted into a captured image and compressed and recorded. Thereafter, the process returns to step A2. Further, when an operation for ending the REC mode is performed (YES in Step A11), the flow exits from FIG.

  As described above, in the first embodiment, the control unit 1 is designated by the touch position of the touch operation when the touch operation is performed in a state where an image including a human face is displayed on the touch display unit 6. The person's face part is identified, and different image processing (make-up processing) is executed for that part, so that it is suitable for the part specified by the touch operation. Make it possible to perform makeup processing and make it possible to make more detailed facial images, even if the same person is better when the mouth is broken and when it is better to tighten It is possible to easily cope with a case where only a part of the face is desired to be changed rather than a beautiful face.

  In addition to determining the part of the person's face specified by the touch position of the touch operation, the control unit 1 further determines the position or state in the part, and according to the part and the position or state in the part Since different makeup processes are executed for the part, even if the same part is designated by the touch operation, it is possible to perform the makeup process suitable for the position or state in the part.

  Since the control unit 1 performs different makeup processing on the face depending on the face part even if the type of the touch operation is the same, the type of touch operation is changed for each different makeup process. There is no need to change, and a makeup process suitable for the part can be performed on the part specified by the touch operation in the same type of touch operation (for example, a single tap or a tracing operation).

  In addition to determining the face part of the person specified by the touch position of the touch operation, the control unit 1 further determines the attribute of the person, and performs different makeup processing according to the attribute of the part and the person. Therefore, the makeup process suitable for the part and person attributes can be executed or prohibited for the part designated by the touch operation. For example, when gender is discriminated as an attribute of a person, women tend to seek beauty and men tend to seek coolness, so the content of makeup processing can be changed for each part depending on the attribute, In some cases, for example, by prohibiting the makeup process, it is possible to easily cope with a difference in preference depending on attributes.

  Since the face of the person is detected from the image including the face of the person displayed on the touch display unit 6 and the face of the person is enlarged, the user wants the desired face within the face of the person. When a part is designated as a makeup processing target, the designation operation can be easily performed.

  Since the contents of the makeup process are displayed on the touch display unit 6 at a position different from the touch position of the touch operation, the same screen shows what makeup process is performed for the user. It can be easily confirmed above.

  Since the face part including the touch position of the touch operation is displayed on the touch display unit 6 at a position different from the touch position of the touch operation, a part of the face is hidden by a finger or the like during the touch operation. Even then, the user can easily confirm the hidden image portion on the same screen.

  The control unit 1 determines whether or not the touch position of the touch operation is another position outside the person's face area, and executes the makeup process when the touch position is outside the person's face area. Since the prohibition is performed, the makeup process can be executed on the assumption that a desired processing target is designated by a touch operation.

  In a state where the image captured by the image capturing unit 5 is displayed as a live view image on the touch display unit 6, the part of the face specified by the touch position of the touch operation is determined in the captured image (live view image). As a result, makeup processing can be performed on the live view image, and various makeup can be tried before recording and saving the image.

  The control unit 1 determines the attribute of the person specified by the touch position of the touch operation in the image including the face of the person, and performs different makeup processing on the face of the person according to the attribute of the person. Since it is executed, it is possible to make up the face image in more detail according to the attributes of the person.

  The control unit 1 executes different image processing on the face part according to the attribute of the person specified by the touch position of the touch operation and the part of the face. Make-up suitable for the part of the person's face becomes possible.

  In the first embodiment described above, “tracing” and “single tap” are illustrated as touch operations. However, the touch operation is not limited thereto, and may be “double tap”, “pinch out”, or the like.

(Second Embodiment)
A second embodiment of the present invention will be described below with reference to FIGS.
In the first embodiment described above, the part of the person's face specified by the touch position of the touch operation is determined, the position or state in the part is determined, and the position or state in the part is determined. In the second embodiment, different makeup processes are executed depending on the type of touch operation even when the same part is designated. Here, in both the embodiments, the same or the same names are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the second embodiment. .

FIG. 8 is a diagram for explaining a makeup table 3c according to the second embodiment.
The makeup table 3c stores information for accessing various makeup processes in association with each part constituting a person's face, and includes “person's face part”, “person attribute: female”, “ Each item has “person attribute: male”, and the “person attribute: female” and “person attribute: male” items are “touch operation type: tracing” and “touch operation type: single tap”. It is divided into items. The contents of makeup table 3c can be arbitrarily changed by a user operation. As in the first embodiment described above, “person's face part” is an item for storing the part of the face to be subjected to the makeup process. In the illustrated example, “skin”, “eyes”, “eyebrows” It shows a case where “”, “睫”, “brow”, “mouth”, and “tooth” are stored as processing targets (selection candidates).

  “Person attribute: female” and “Person attribute: male” are items for storing information for accessing the makeup process for each gender and each type of touch operation as a person attribute. In the example shown in the figure, “skin treatment (1)” is stored corresponding to “touch operation type: tracing” of “person attribute: female” when “part of human face” is “skin”, In this case, “skin treatment (2)” different from the “tracing” touch operation is stored corresponding to the “touch operation type: single tap”. Similarly, “person attribute: male” stores “skin treatment (1)” corresponding to “touch operation type: tracing”, and “skin treatment corresponding to the“ touch operation type: single tap ”. (2) “” is stored.

  Further, when “the face part of the person” is “eyes”, “eye processing (1)” is stored in correspondence with “touch operation type: tracing” of “person attribute: female”. A case where “eye processing (2)” different from the touch operation of “tracing” is stored corresponding to the touch operation type: single tap ”is shown. Also, “eye processing (3)” is stored corresponding to “touch operation type: tracing” of “person attribute: male”, and “race” is corresponding to the “touch operation type: single tap”. The case where “eye processing (2)” different from the touch operation is stored is shown.

  When the “part of the person's face” is “eyebrows”, “、”, “mouth”, or “teeth”, the “person attribute” is used regardless of the type of touch operation, as in the first embodiment. : Female "and" person attribute: male "are stored separately. That is, when “the face part of a person” is “eyebrows”, “eyebrow processing (1)” is stored corresponding to “person attribute: female”, and “person attribute: male” is supported. In this example, “eyebrow processing (2)” is stored. When “person's face part” is “睫”, “make-up process” is stored corresponding to “person attribute: female” and “person attribute: male” This shows a case where makeup processing is not stored. When “portion of person's face” is “mouth”, “mouth processing (1)” is stored corresponding to “person attribute: female”, and “person attribute: male” is stored. “Mouth treatment (2)” is stored. When “the face part of a person” is “tooth”, “tooth processing” is stored as a common processing corresponding to “person attribute: female” and “person attribute: male”. The contents of the various makeup processes described above are the same as in the first embodiment (see FIG. 4).

  FIG. 9 is a flowchart for explaining in detail the makeup process (step A6 in FIG. 5) in the second embodiment. In the second embodiment, the same operation as in the flow of FIG. 5 is performed, and thus detailed description thereof is omitted. In the second embodiment, steps C6 and C7 shown in FIG. 9 are used instead of steps B6 and B7 shown in FIG. 6, and the others are basically the same. The description will focus on the part.

  First, when an image including a person's face is displayed and a touch operation is performed on the live view screen, the control unit 1 determines that the touch position is within the person's face area (step S1). C1 is YES), the attribute (gender) of the person is determined (step C3), and which part of the face is touched is determined (step C4). If the part determined at the current timing is the same as the part determined at the previous timing (YES in step C5), whether the tracing operation is performed or the single tap operation is performed as the type of the touch operation. (Step C6). Then, in order to determine the identity of the touch operation, it is checked whether the type of the touch operation determined at the current timing is the same as the type of the touch operation determined at the previous timing (step C7).

  Here, if it is different from the previous part (NO in step C5), or if the type of touch operation is different from the previous part (NO in step C7), the touch operation is continuing as in the flow of FIG. If it is the same as the type of the previous touch operation (YES in step C7), based on the determination result (person attribute, face part, type of touch operation). Then, the corresponding makeup process is searched with reference to the makeup table 3c (step C8). If there is no corresponding makeup process (NO in step C9), the process proceeds to step C2. If there is a corresponding makeup process (YES in step C9), the process proceeds to step C10, which is the same as in the first embodiment. Similarly, the image portion hidden by the user's finger at the time of the touch operation is enlarged and displayed in another window, and the contents of the corresponding makeup processing are guided and displayed by a character string message. When the type of touch operation is “tracing”, the display content of the separate window also changes following the movement of the finger.

Thereafter, as in the first embodiment, the corresponding makeup process is executed (step C11), and at the same time, the image after the makeup is switched to be displayed instead of the live view image (step C12). Then, it is checked whether the touch operation is released (finger is released) (step C13). If the touch operation is continuing (NO in step C13), the touch operation is in a standby state until it is released. When the touch operation is released (NO in step C13), the process of releasing (erasing) the above-described separate window display and guidance display is performed (step C14), and then the process exits from the flow of FIG.
When the touch operation is “single tap”, the position where the makeup process is executed is executed for a range determined to be continuous with the touch position as a base point. For example, in the case of the eye process (2), a makeup process for reducing white eye hyperemia is performed on the entire white eye portion inside the eye. Further, when the touch operation is “tracing”, makeup processing is executed on the trajectory of the tracing that is the touch position that moves continuously as in the first embodiment.

  As described above, in the second embodiment, when the touch operation is performed in a state where an image including a human face is displayed on the touch display unit 6, the control unit 1 determines the face specified by the touch operation. Even if the part is the same, if the type of touch operation is different, different image processing (make-up process) is performed on the part of the face depending on the type of touch operation. Thus, the desired makeup can be obtained for the same part only by changing the type of touch operation.

  In the second embodiment described above, “race” and “single tap” are exemplified as the types of touch operations. However, the types of touch operations are not limited thereto, and differences in the number of fingers during multi-touch are used as the types of touch operations. Also good.

In each embodiment described above, the control unit 1 performs a makeup process on the captured image in a state where the image captured by the imaging unit 5 is displayed on the touch display unit 6 as a live view image. However, the makeup process may be executed on a photographed image (recorded image) recorded and stored in an SD card or the like. In this case, in a state in which the recorded image is read and reproduced and displayed, the part of the face of the person specified by the touch position of the touch operation is determined in the recorded image, and different makeup is performed depending on the part of the face. What is necessary is just to make it perform a process with respect to the site | part. This makes it possible to make up a person's face according to his / her preference for images recorded and stored.
In this case, it is desirable that the makeup-processed image is recorded and stored as an image different from the recorded and stored image.

  In each of the above-described embodiments, the part of the face of the person specified by the touch position of the touch operation is determined, and different makeup processing is performed on the part depending on the part. It is determined whether the touch position of the operation is within the face area of the person, and makeup processing is performed on the face area depending on whether the touch position is within the face area of the person, or the makeup process May not be executed. Further, different image processing may be performed depending on whether or not the face area is within a person. As a result, even if the touch position of the touch operation extends from the inside of the person's face area to the outside of the area, only the inside of the face area can be accurately and finely processed.

  In each of the embodiments described above, a case where “skin”, “eyes”, “eyebrows”, “睫”, “eyebrows”, and “teeth” are set as processing targets (selection candidates) as human face parts. As an example, “nose”, “forehead”, etc. may be used as other examples, and for example, makeup processing for raising the nose or makeup processing for rounding the forehead can be executed corresponding to the part. Good.

  In each of the embodiments described above, “inside of the eye” and “outside of the eye” are exemplified as “position in the region”, but not limited to this, for example, when the region is “skin”, The nasal muscles may be determined as positions in the skin, and when the person is a woman, the nasal muscles are softened so as not to disappear, and when the person is a male, makeup processing that makes the nasal muscles clear may be performed. In addition, the “state in the part” is also arbitrary. For example, when the part is the eye (pupil), the size of the eye (pupil) with respect to the standard size is set to the “state in the part” and is smaller than the standard size. The eyes (pupils) may be made larger than the standard size.

  In each of the above-described embodiments, the makeup process is performed regardless of gender, but the makeup process may be performed only for women.

  In each of the above-described embodiments, the gender is exemplified as the attribute of the person, but the make-up process is performed in the direction that looks younger in the case of old age, and in the direction that looks younger in the case of young age. You may make it perform. Further, the other person's attribute may be a hair style or the like.

  In each of the above-described embodiments, the case where the present invention is applied to an imaging device (digital camera) has been described. However, the present invention is not limited to this, and a personal computer, a PDA (personal portable information communication device), a tablet terminal device, a smartphone, and the like. Mobile phones, electronic games, music players, etc., or a dedicated image playback device.

  Further, the “apparatus” and “unit” shown in each of the above-described embodiments may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
An image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Touch detection means for detecting a touch operation on the touch operation means;
It is determined whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person in the image including the face of the person displayed on the display means, or designated by the touch position. Discriminating means for discriminating a part of the face
Image processing means for executing, on the face area or part, different image processing depending on whether or not the face part of the person is discriminated by the discrimination means or on the face part;
It is characterized by providing.
(Claim 2)
The invention according to claim 2 is the image processing apparatus according to claim 1,
In addition to determining whether the touch position of the touch operation detected by the touch detection means is within the face area of the person or determining the facial part specified by the touch position, the determining means When the position of the part is determined, the position or state in the part is further determined,
The image processing means performs different image processing on the face area or part depending on whether or not the face area of the person is discriminated by the discrimination means or on the face part and the position or state in the part. Run,
It is characterized by that.
(Claim 3)
The invention according to claim 3 is the image processing apparatus according to claim 1,
Even if the type of touch operation detected by the touch detection unit is the same, the image processing unit differs depending on whether or not it is within the face area of the person determined by the determination unit or the face part Performing image processing on the face region or part;
It is characterized by that.
(Claim 4)
According to a fourth aspect of the present invention, in the image processing apparatus according to the first aspect,
Whether the image processing means is in the face area of the person determined by the determination means or the type of touch operation detected by the touch detection means is different even if the face part is the same. Depending on the type of touch operation, different image processing is performed on the face region or part.
It is characterized by that.
(Claim 5)
The invention according to claim 5 is the image processing apparatus according to claim 1,
The determination means determines whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person, or determines the part of the face specified by the touch position. Further determine the attributes,
The image processing unit executes or performs different image processing on the face region or part depending on whether or not it is within the face area of the person determined by the determination unit or the face part and the person attribute. Prohibit execution,
It is characterized by that.
(Claim 6)
The invention according to claim 6 is the image processing apparatus according to any one of claims 1 to 5,
Face detection means for detecting a person's face from within an image including the face of the person displayed on the display means;
First display control means for performing control to enlarge and display the face of the person detected by the face detection means on the display means;
In addition,
It is characterized by that.
(Claim 7)
The invention according to claim 7 is the image processing apparatus according to any one of claims 1 to 6,
On the display means, a second display for controlling the display means to display the content of the image processing executed by the image processing means at a position different from the touch position of the touch operation detected by the touch detection means. A control means;
It is characterized by that.
(Claim 8)
The invention according to claim 8 is the image processing apparatus according to any one of claims 1 to 7,
On the display means, a face portion including the touch position of the touch operation detected by the touch detection means is displayed on the display means at a position different from the touch position of the touch operation detected by the touch detection means. Further comprising third display control means for performing control,
It is characterized by that.
(Claim 9)
The invention according to claim 9 is the image processing apparatus according to any one of claims 1 to 8,
The determination means further determines whether or not the touch position of the touch operation detected by the detection means is another position outside the human face,
The image processing unit prohibits execution of the image processing when the determining unit determines that the touch position is another position deviating from a person's face;
It is characterized by that.
(Claim 10)
According to a tenth aspect of the present invention, in the image processing apparatus according to any one of the first to ninth aspects,
An image pickup means for picking up a person's face is further provided,
The display means displays an image including a human face imaged by the imaging means,
The determination means determines whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person in the captured image displayed on the display means or is designated by the touch position To identify the part of the face
It is characterized by that.
(Claim 11)
According to an eleventh aspect of the present invention, in the image processing apparatus according to any one of the first to ninth aspects,
The display means displays a recorded image,
The discriminating unit discriminates whether or not the touch position of the touch operation detected by the detecting unit is within the face area of the person in the recorded image displayed on the display unit or is designated by the touch position. To identify the part of the face
It is characterized by that.
(Claim 12)
The invention according to claim 12
An image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Detecting means for detecting a touch operation on the touch operating means;
A discriminating unit for discriminating an attribute of a person specified by a touch position of a touch operation detected by the detecting unit in an image including a human face displayed on the display unit;
Image processing means for executing different image processing on the face of the person according to the attribute of the person determined by the determination means;
It is characterized by that.
(Claim 13)
The invention according to claim 13 is the image processing apparatus according to claim 12,
The determination means further determines a part of the face of the person specified by the touch position of the touch operation detected by the touch detection means,
The image processing means executes different image processing on the face part depending on the attribute of the person determined by the determination means and the part of the face.
It is characterized by that.
(Claim 14)
The invention according to claim 14
An image processing method in an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Processing for detecting a touch operation on the touch operation means;
In the image including the face of the person displayed on the display means, it is determined whether or not the touch position of the detected touch operation is within the face area of the person in the image including the face of the person. A process of determining the face part specified by the touch position;
A process of executing different image processing on the face region or part depending on whether or not it is within the face region of the person to be discriminated, or a part of the face;
It is characterized by including.
(Claim 15)
The invention according to claim 15 is:
For a computer of an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
A function of detecting a touch operation on the touch operation means;
In the image including the face of the person displayed on the display means, it is determined whether or not the touch position of the detected touch operation is within the face area of the person in the image including the face of the person. A function to determine the face part specified by the touch position;
A function of executing different image processing on the face region or part depending on whether or not it is within the face region of the person to be discriminated;
It is a program for realizing.
(Claim 16)
The invention described in claim 16
An image processing method in an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Processing for detecting a touch operation on the touch operation means;
A process of determining an attribute of a person specified by a touch position of the detected touch operation in an image including the face of the person displayed on the display means;
Processing to perform different image processing on the face of the person according to the attribute of the person to be determined;
It is characterized by including.
(Claim 17)
The invention described in claim 17
For a computer of an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
A function of detecting a touch operation on the touch operation means;
A function of determining an attribute of a person specified by a touch position of the detected touch operation in an image including the face of the person displayed on the display means;
A function of executing different image processing on the face of the person according to the attribute of the person to be determined;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 1 Control part 2 Power supply part 3 Memory | storage part 3a Program memory 3b Work memory 3c Makeup table 4 Operation part 5 Imaging part 6 Touch display part 6a Display part 6b Touch operation part

Claims (17)

An image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Touch detection means for detecting a touch operation on the touch operation means;
It is determined whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person in the image including the face of the person displayed on the display means, or designated by the touch position. Discriminating means for discriminating a part of the face
Image processing means for executing, on the face area or part, different image processing depending on whether or not the face part of the person is discriminated by the discrimination means or on the face part;
An image processing apparatus comprising: In addition to determining whether the touch position of the touch operation detected by the touch detection means is within the face area of the person or determining the facial part specified by the touch position, the determining means When the position of the part is determined, the position or state in the part is further determined,
The image processing means performs different image processing on the face area or part depending on whether or not the face area of the person is discriminated by the discrimination means or on the face part and the position or state in the part. Run,
The image processing apparatus according to claim 1. Even if the type of touch operation detected by the touch detection unit is the same, the image processing unit differs depending on whether or not it is within the face area of the person determined by the determination unit or the face part Performing image processing on the face region or part;
The image processing apparatus according to claim 1. Whether the image processing means is in the face area of the person determined by the determination means or the type of touch operation detected by the touch detection means is different even if the face part is the same. Depending on the type of touch operation, different image processing is performed on the face region or part.
The image processing apparatus according to claim 1. The determination means determines whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person, or determines the part of the face specified by the touch position. Further determine the attributes,
The image processing unit executes or performs different image processing on the face region or part depending on whether or not it is within the face area of the person determined by the determination unit or the face part and the person attribute. Prohibit execution,
The image processing apparatus according to claim 1. Face detection means for detecting a person's face from within an image including the face of the person displayed on the display means;
First display control means for performing control to enlarge and display the face of the person detected by the face detection means on the display means;
In addition,
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. On the display means, a second display for controlling the display means to display the content of the image processing executed by the image processing means at a position different from the touch position of the touch operation detected by the touch detection means. A control means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. On the display means, a face portion including the touch position of the touch operation detected by the touch detection means is displayed on the display means at a position different from the touch position of the touch operation detected by the touch detection means. Further comprising third display control means for performing control,
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The determination means further determines whether or not the touch position of the touch operation detected by the detection means is another position outside the human face,
The image processing unit prohibits execution of the image processing when the determining unit determines that the touch position is another position deviating from a person's face;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. An image pickup means for picking up a person's face is further provided,
The display means displays an image including a human face imaged by the imaging means,
The determination means determines whether or not the touch position of the touch operation detected by the touch detection means is within the face area of the person in the captured image displayed on the display means or is designated by the touch position To identify the part of the face
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The display means displays a recorded image,
The discriminating unit discriminates whether or not the touch position of the touch operation detected by the detecting unit is within the face area of the person in the recorded image displayed on the display unit or is designated by the touch position. To identify the part of the face
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. An image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Detecting means for detecting a touch operation on the touch operating means;
A discriminating unit for discriminating an attribute of a person specified by a touch position of a touch operation detected by the detecting unit in an image including a human face displayed on the display unit;
Image processing means for executing different image processing on the face of the person according to the attribute of the person determined by the determination means;
An image processing apparatus. The determination means further determines a part of the face of the person specified by the touch position of the touch operation detected by the touch detection means,
The image processing means executes different image processing on the face part depending on the attribute of the person determined by the determination means and the part of the face.
The image processing apparatus according to claim 12. An image processing method in an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Processing for detecting a touch operation on the touch operation means;
In the image including the face of the person displayed on the display means, it is determined whether or not the touch position of the detected touch operation is within the face area of the person in the image including the face of the person. A process of determining the face part specified by the touch position;
A process of executing different image processing on the face region or part depending on whether or not it is within the face region of the person to be discriminated, or a part of the face;
An image processing method comprising: For a computer of an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
A function of detecting a touch operation on the touch operation means;
In the image including the face of the person displayed on the display means, it is determined whether or not the touch position of the detected touch operation is within the face area of the person in the image including the face of the person. A function to determine the face part specified by the touch position;
A function of executing different image processing on the face region or part depending on whether or not it is within the face region of the person to be discriminated;
A program to realize An image processing method in an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
Processing for detecting a touch operation on the touch operation means;
A process of determining an attribute of a person specified by a touch position of the detected touch operation in an image including the face of the person displayed on the display means;
Processing to perform different image processing on the face of the person according to the attribute of the person to be determined;
An image processing method comprising: For a computer of an image processing apparatus comprising display means for displaying an image including a human face and touch operation means,
A function of detecting a touch operation on the touch operation means;
A function of determining an attribute of a person specified by a touch position of the detected touch operation in an image including the face of the person displayed on the display means;
A function of executing different image processing on the face of the person according to the attribute of the person to be determined;
A program to realize

Images