WO2008126948A1

WO2008126948A1 - System for making caricature and method of the same and recording medium of the same

Info

Publication number: WO2008126948A1
Application number: PCT/KR2007/001788
Authority: WO
Inventors: Jin Kook Choi; Kediy Kim
Original assignee: JDF Group
Current assignee: JDF Group
Priority date: 2007-04-12
Filing date: 2007-04-12
Publication date: 2008-10-23
Anticipated expiration: 2009-10-12
Also published as: JP2009521065A

Abstract

A caricature generation system, method, and recording medium of generating a caricature by using image data, the system including: a face shape generator which extracts, from the image data inputted from a user, face shape information including face contour information generated by distinguishing a face area from a background area, and generates a face shape; a character image generator which extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes; and a caricature generator which combines the face shape and the character images, and generates the caricature. Alternatively, the system may include: a shadow plate generator which generates a shadow plate including a face contour and a face curve generated by dividing a face area and a background area from the image data; a character image generator; and a caricature generator.

Description

SYSTEM FOR MAKING CARICATURE AND METHOD OF THE SAME AND RECORDING MEDIUM OF THE SAME

Technical Field

The present invention relates to a caricature generation system and method, and more particularly, to a caricature generation system and method which can generate a caricature by receiving image data from a user and extracting face information.

Background Art As users using Internet community sites, Internet shopping mall sites, and

Internet game sites increase recently, a tendency of identifying, with themselves, characters which may show themselves on Internet, for example, avatars, game characters, and the like is becoming common. In order to fulfill the above-described requirements of the users, Internet sites provide the users with various face shapes, hair styles, shapes of ears, eyes, mouths, noses, facial expressions, and the like, so that the users may generate characters similar to the users' own appearances by combining each component.

However, a method of generating a character by combining provided components as described above cannot completely fulfill the users' desires even though various components are provided, and the users' characters are generally similar for each other. Accordingly, there is a limitation of providing the users with characters having strong individuality. Also, Internet sites require to provide work such as design manufacturing, image manufacturing, and the like, with much labor and time in order to provide the users with a maximum amount of various components. Accordingly, methods of automatically generating caricatures by using image data including pictures and the like, and generating characters similar to the users by using the generated caricatures are introduced. For example, a caricature manufacturing method disclosed in Korean Granted Patent Application No. 10-376760 manufactures a caricature by a method of analyzing each portion of a picture by using a source picture provided by a user, extracting a caricature model of each portion from a database, combining the extracted each portion, and subsequently combining the combined portions and a necessary background image. FIG. 1 is a flowchart illustrating a caricature manufacturing method according to a conventional art. Referring to FIG. 1, a source picture for manufacturing a caricature is inputted in a picture input of an automatic caricature manufacturing device, in step 100 of inputting the picture for conventional caricature manufacturing. The picture may be a famous entertainer's picture, and the like, and may be generally a user's own face. When the source picture is inputted, a face shape, eyes, a nose, a mouth, and the like are analyzed as each portion of the picture, i.e. each portion configuring a face, in step 102, and a model appropriate for a character of analyzed each portion is extracted in order to be used for the caricature, in step 104. Extraction of the model is progressed by using a database previously constructed with respect to each portion of a person. For example, an entire face shape is completed by information extracted by calculating light and shadows on the face in the case of a face shape, and a location and a ratio of each portion are calculated, and are extracted from the database in the case of eyes, a nose, and a mouth. When extraction of each portion of the face is completed, the face is combined in step 106, and a hairstyle appropriate for the combined face is selected from a hairstyle database by inputting data of the combined face, in step 108. Also, the face is completed in step 109. Next, a body of the caricature is selected in step 112, and other portions of the body may be also selected by a similar method. A background image is combined with the caricature in step 114, and the caricature is completed in step 116.

When the above-described method is used, the user can easily generate the character similar to the user's own shape, from an automatic device including a built-in program by merely inputting the user's own picture into the image data. However, since an existing automatic caricature generation method cannot generally and accurately determine a location and a range of an actual face in the picture, a face contour cannot be accurately identified. In particular, there is a problem that a boundary of a jaw line portion of the face cannot be accurately identified in the image data since a neck and a jaw generally show a similar color.

Also, when the automatic device including the built-in program is used, the user can merely select, from the prestored database, each portion required for generating the caricature, and cannot directly modify light and shadows on, or a contour of the use's face. Accordingly, there is a drawback that the caricature similar to an actual person cannot be generated since a three-dimensional effect and reality are reduced.

Disclosure of Invention

Technical Goals The present invention provides a caricature generation system and method which enables a user to easily generate a caricature by merely inputting the user's own picture into image data.

The present invention also provides a caricature generation system and method which can extract a face contour by using a difference between a face area color and a background area color when the caricature is generated, thereby generating a caricature similar to an actual user.

The present invention also provides a caricature generation system and method which can generate a caricature similar to an actual user by accurately extracting a face contour when the caricature is generated. The present invention also provides a caricature generation system and method which can generate a caricature which a user desires by enabling the user to modify an automatically extracted contour with a simple mouse click.

The present invention also provides a caricature generation system and method which can extract a contour very similar to an actual user's contour with reference to an estimated face width and a face height by estimating the face width and the face height, based on a pupils location and a lips location when a face contour is extracted in order to generate a caricature.

The present invention also provides a caricature generation system and method which can extract a contour in which a face area color and a background area color are similar, such as a jaw line, by extracting a contour in a face range when a face contour is extracted in order to generate a caricature.

The present invention also provides a caricature generation system and method which can express a three-dimensional shape of an actual face by using a shadow plate including a face contour and a face curve when a caricature is generated. The present invention also provides a caricature generation system and method which can easily generate a face curve by posterizing a face area in image data inputted from a user. The present invention also provides a caricature generation system and method which can generate a caricature similar to an actual face by using a color of an actual face area when a face area is posterized from image data inputted from a user in order to generate a face curve. The present invention also provides a caricature generation system and method which can generate a caricature having various moods by receiving, from a user, a transparency of a shadow plate and determining a combination ratio when the shadow plate and a character image are combined.

Technical solutions

According to an aspect of the present invention, there is provided a caricature generation system for generating a caricature by using image data inputted from a user, the system including: a face shape generator which extracts, from the image data, face shape information including face contour information generated by distinguishing a face area from a background area, and generates a face shape; a character image generator which extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes; and a caricature generator which combines the face shape and the character images, and generates the caricature. The face shape generator includes: a face component location identifier which identifies pupils location information and lips location information from the image data; a face length estimator which estimates a face width and a face height, based on the identified pupils location and the identified lips location; a face contour information extractor which distinguishes the face area from the background area in an estimated face range by using the estimated face width and the face height, and extracts the face contour information; and a face shape former which processes the face contour information or the image data, and forms the face shape.

According to another aspect of the present invention, there is provided a caricature generation system for generating a caricature from image data inputted from a user, the system including: a shadow plate generator which generates a shadow plate including a face contour and a face curve generated by dividing a face area and a background area from the image data; a character image generator which extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes; and a caricature generator which combines the shadow plate and the character images, and generates the caricature.

According to still another aspect of the present invention, there is provided a caricature generation method of generating a caricature by using image data inputted from a user, the method including: a step of distinguishing a face area from a background area, extracting, from the image data, face shape information including face contour information, and generating a face shape; a step of extracting face component shapes from the image data, and generating character images corresponding to the extracted face component shapes; and a step of combining the face shape and the character images, and generating the caricature.

The step of distributing includes: identifying pupils location information and lips location information from the image data; estimating a face width and a face height, based on the identified pupils location and the identified lips location; distinguishing the face area from the background area in an estimated face range by using the estimated face width and the face height, and extracting the face contour information; and forming the face shape, based on the extracted face contour information, generating an image of ears, eyes, mouth, and nose corresponding to eyes, eyebrows, nose, mouth, or ears shape of the image data, combining the face shape with the image of ears, eyes, mouth, and nose, and generating the caricature.

According to yet another aspect of the present invention, there is provided a caricature generation method of generating a caricature from image data inputted from a user, the method including: a step of generating a shadow plate including a face contour and a face curve generated by dividing a face area and a background area from the image data; a step of extracting face component shapes from the image data, and generating character images corresponding to the extracted face component shapes; and a step of combining the shadow plate and the character images, and generating the caricature.

According to another aspect of the present invention, there is provided a computer-readable recording medium storing a program for implementing the above- described caricature generation method. Brief Description of Drawings

FIG. 1 is a flowchart illustrating a caricature manufacturing method according to a conventional art;

FIG. 2 illustrates roughly, operations of a caricature generation system according to an exemplary embodiment of the present invention;

FIG. 3 is a block diagram illustrating an internal configuration of a caricature generation system according to a first exemplary embodiment of the present invention;

FIG. 4 is a block diagram illustrating an internal configuration of a face shape generator of FIG. 3; FIG. 5 illustrates a method of estimating face lengths, based on a pupils location and a lips location according to an exemplary embodiment of the present invention;

FIG. 6 is a diagram illustrating data from experiments for acquiring a correlation between a distance between a female's eyes, and a face width, and a correlation between a distance between the female's eyes and lips, and a face height according to an exemplary embodiment of the present invention;

FIG. 7 is a diagram illustrating data from experiments for acquiring a correlation between a distance between a male's eyes, and a face width, and a correlation between a distance between the male's eyes and lips, and a face height according to an exemplary embodiment of the present invention; FIG. 8 illustrates a method of distinguishing a face area from a background area in a face range according to an exemplary embodiment of the present invention;

FIG. 9 is a flowchart illustrating flows of a caricature generation method according to a first exemplary embodiment of the present invention;

FIG. 10 is a flowchart illustrating step S720 of generating a face shape of FIG. 9 in detail;

FIG. 11 is a block diagram illustrating an internal configuration of a caricature generation system according to a second exemplary embodiment of the present invention;

FIG. 12 illustrates a shadow plate generated by using a posterizing according to an exemplary embodiment of the present invention;

FIG. 13 illustrates a caricature changed depending on a transparency of a shadow plate according to an exemplary embodiment of the present invention; FIG. 14 is a diagram illustrating a character image database according to an exemplary embodiment of the present invention;

FIG. 15 illustrates a picture in which a selected hairstyle and selected eye shapes of a caricature are inputted from a user according to an exemplary embodiment of the present invention;

FIG. 16 illustrates an example of using, in a mobile terminal, a caricature generated according to an exemplary embodiment of the present invention; and

FIG. 17 is a flowchart illustrating flows of a caricature generation method according to a second exemplary embodiment of the present invention.

Best Mode for Carrying Out the Invention

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

FIG. 2 illustrates rough operations of a caricature generation system according to a first exemplary embodiment of the present invention. As illustrated in FIG. 2, a caricature generation system 120 generates a caricature 130 by using image data 110 inputted from a user The image data 110 may include all data showing a static picture. For example, the image data 110 may be digital data including a form such as a Joint Photographic Experts Group (JPEG) image, a Graphics Interchange Format (GIF) image, a Tagged Image File Format (TIFF), a bitmap (BMP), and the like, or may be analog data such as a printed picture. The digital data may be inputted by using a digital medium such as a floppy disk, a compact disc read-only memory (CD-ROM), a Universal Serial Bus (USB)-memory, a communication network, and the like, and the analog data may be inputted after being converted into the digital data by using a conversion apparatus such as a scanner, and the like. Also, a movie may be used for the image data 110 by capturing the static picture. The caricature generation system 120 may be operated being installed in a user's terminal. Also, the caricature generation system 120 may be installed in a server, receive the image data 110 by using the user's terminal connected with the server via the communication network, and generate the caricature 130. After generating the caricature 130, the caricature generation system 120 may transmit the caricature 130 to the user's terminal via the communication network, or may store the caricature 130 in an internal storing apparatus of the server, and enable the user to transmit the caricature 130 to the user's terminal as required.

FIG. 3 is a block diagram illustrating an internal configuration of the caricature generation system 120 of FIG. 2. Referring to FIG. 3, the caricature generation system 120 includes a face shape generator 210 which extracts, from the image data, face shape information including face contour information, and generates a face shape, a character image generator 220 which extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes, and a caricature generator 230 which combines the face shape and the character images, and generates the caricature. The face shape information includes extracted pupils location information, lips location information, face width information, and face height information including the face contour information.

A caricature is a word derived from the Italian word 'caricatura' denoting "exaggeration and distortion", and denotes a satirical drawing, a drawing drawn as a cartoon, and the like. Although the caricature was generally used for interestingly and exaggeratedly depicting shapes of celebrities such as entertainers, politicians, athletes, and the like, in the past, the caricature is widely used on the Internet recently for images showing users' characteristics. The caricature in the present specification denotes a face image drawn resembling an actual shape of a specific person by using a facial characteristic of the specific person, and may be widely understood by being not limited to a specific form, a number of components, and an expression form. FIG. 4 is a block diagram illustrating an internal configuration of the face shape generator in the caricature generation system of FIG. 3 in detail. As illustrated in FIG. 4, the face shape generator 210 includes a face component location identifier 210-1, a face length estimator 210-2, a face contour information extractor 210-3, and a face shape former 210-4. Hereinafter, a function of each component is described in detail. The face component location identifier 210-1 identifies pupils location information and lips location information from image data. The pupils location information and the lips location information identified by the face component location identifier 210-1 may include a coordinate of a pixel corresponding to a location of a pupil center, and a coordinate of a pixel corresponding to a location of a center of lips in the image data. The pupils location information and the lips location information may be identified by using a difference between a color of pixels corresponding to a pupils location and a lips location, and a color of skin around pupils and lips. Also, a coordinate of a clicked pixel may be identified by providing a user with the image data, and receiving, from the user, the pupils location and the lips location by a click.

The face length estimator 210-2 estimates a face width and a face height, based on the pupils location and the lips location identified by the face component location identifier 210-1. For an approximation, a correlation between the pupils location and the lips location extracted from a plurality of sample image data, and the face width and the face height may be used. The correlation is respectively acquired depending on sex differentiation of a person in sample image data, and the face width and the face length may be estimated by using the correlation corresponding to the sex differentiation of the person in image data inputted from the user. For this, the face length estimator 210-2 may further include a user sex differentiation information database including sex differentiation information.

For example, the face length estimator 210-2 may estimate the face width, based on a distance between two pupils, and estimate the face length, based on a distance between pupils and lips. For an approximation, the face length estimator 210- 2 may respectively extract the distance between two pupils, and the face width from a plurality of image data, and statistically acquire a correlation between the distance between two pupils, and the face width by using the extracted distance between two pupils, and the face width. The face length estimator 210-2 may store, in a separate storing apparatus, the acquired correlation as described above, and when the user inputs the image data, the face length estimator 210-2 may estimate the face width by using the stored correlation, based on the distance between two pupils extracted from the inputted image data. The face height may be estimated by an identical method by using the correlation between pupils and lips.

The face contour information extractor 210-3 estimates a face range in the image data by using the face width and the face height estimated by the face length estimator 210-2. The face range may be a range of a rectangle having a minimum size including a face, and the face width and the face height may be respectively established as a width and a height of the rectangle. A location of the rectangle may be estimated by using the pupils location information and the lips location information identified by the face component location identifier 210-1.

Also, the face contour information extractor 210-3 distinguishes the face area from the background area in the face range, and extracts the face contour information. The face area may be distinguished from the background area by using a difference between color data of a pixel corresponding to the face area, and color data of a pixel corresponding to the background area. Also, information with respect to a boundary line of the distinguished area may be the face contour information. However, when a color of the face area and a color of the background area are indistinguishable, for example, a jaw line, areas may not be clearly distinguishable by using only the difference of the color data. Accordingly, the areas may be distinguished more accurately than an existing method by distinguishing the areas in the face range. Therefore, there is an effect to generate the caricature similar to the actual person by extracting the accurate face contour information.

Also, the face contour information extractor 210-3 may provide the user with the extracted face contour information, receive contour change information from the user, and modify the contour information, based on the contour change information. A method of estimating the face width and the face height by the face length estimator 210-2 uses a statistical method. Since the face contour information extractor 210-3 estimates the face range by using the face width and the face height, and extracts the face contour information in the face range, the contour information different from the actual face contour may be extracted when an exception to statistics is generated. In this case, the caricature similar to an actual photo may be generated by enabling the user to modify the face contour extracted by a simple mouse click, and to be in accordance with an actual contour.

The face shape former 210-4 forms the face shape, based on the face contour information extracted from the face contour information extractor 210-3. The face shape may be generated by covering, with a color similar to an actual face, an internal portion of the face contour, the face contour being extracted from the face contour information extractor 210-3, or extracting the internal portion of the face contour from the image data inputted from the user and passing through a predetermined process. Referring to FIG. 3 again, the character image generator 220 extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes. The face components may denote characteristic components in a face, and may include eyes, eyebrows, a nose, a mouth, ears, and the like. Also, the character images may be images showing characteristic shapes of the face components, and may include a form of a digital image which may be combined with other digital images.

For this, the character image generator 220 may extract the face components from an identified location after identifying the location of the face components. The location of the face components may be identified by using a difference between a color of a pixel corresponding to the location of the face components, and a color of surrounding skin. Also, a coordinate of a clicked pixel may be identified by providing a user with the image data, and receiving, from the user, the pupils location information and the lips location information by click. When the location of the face components is identified as described above, the contour of the face components is extracted by using a color difference between the face components and the surrounding skin in the identified location. Accordingly, the face component shapes may be extracted.

Also, the character image generator 220 generates character images corresponding to the extracted face component shapes. For this, the character image generator 220 may extract the character images corresponding to the extracted face component shapes, from a character image list stored in the character image database 240, or may provide the user with the character image list, and receive the selected character images from the user. Also, the character image generator 220 may generate the character images by using user information including sex differentiation information of the user, information concerning whether double eyelids exist, and the like. Also, the user information may be stored in a predetermined user information database, and may be referred when the character images are generated.

The caricature generator 230 combines the face shape and the character images of the face components, and generates the caricature. Face component images may include images showing shapes of eyes, eyebrows, a nose, lips, ears, and the like. A plurality of face component images may be stored in a predetermined storing apparatus, and the caricature generator 230 may provide the user with a list, and receive the selected desirable images from the user. Also, the face components may be identified from the image data inputted from the user, and the face component images similar to the identified face components may be extracted in the storing apparatus.

Also, the caricature generator 230 may determine a location to combine the images, based on the pupils location and the lips location identified by the face component location identifier 210-1 when the character images of the face components are combined with the face shape.

FIG. 5 illustrates a method of estimating face lengths, based on pupils location and lips location in order to perform the first exemplary embodiment of the present invention including the caricature generation system 120 of FIG. 3. Reference numeral 310 illustrates the method of estimating the face lengths, based on the pupils location and the lips location in an actual picture, and reference numeral 320 expresses an identical illustration with a picture instead of a photograph, in order to be easily recognized. As illustrated in FIG. 5, the caricature generation system according to the present invention estimates a distance between two pupils, e.g. width_eyes, and a distance between pupils and lips, e.g. height_eyemouth, by identifying the pupils location and the lips location from the image data, and estimates a face width, e.g. width_face, and a face height, e.g. height_face, by using the estimated distance between two pupils and the distance between pupils and lips. For an approximation, a correlation between the distance between two pupils, e.g. width eyes, and the face width, e.g. widthjface, and a correlation between the distance between pupils and lips, e.g. height_eyemouth, and the face height, e.g. height_face, may be used.

For example, the caricature generation system according to the present invention estimates the distance between two pupils, the distance between pupils and lips, the face width, and the face height from a plurality of image data, acquires correlations of estimated values, and stores the correlations in the predetermined storing apparatus. When the user generates the caricature by inputting the image data, the caricature generation system may estimate the distance between two pupils and the distance between pupils and lips from the image data inputted from the user, and estimate the face width and the face height by inputting the estimated values in the stored correlations. In this instance, the correlations may be separately stored by being variously acquired depending on sex differentiation, and correlations corresponding to sex differentiation of an inputted image data thereto may be applied.

FIGS. 6 and 7 are diagrams illustrating data from experiments for acquiring a correlation between a distance between eyes, and a face width, and a correlation between a distance between eyes and lips, and a face height according to an exemplary embodiment of the present invention. FIG. 6 illustrates data of females from experiments, and FIG. 7 illustrates data of males from experiments.

As illustrated in FIGS. 6 and 7, the caricature generation system according to the present invention calculates a ratio of a distance between two pupils, and a face width, e.g. a ratio eye/face, and a ratio of a distance between pupils and lips, and a face height, e.g. a ratio height/face, for each sample image data by calculating the distance between two pupils, the distance between pupils and lips, the face width, and the face height from a plurality of sample image data.

According to the illustrated experiment data, since it is understood that the ratio eye/face and the ratio height/face have a nearly uniform value for each image data, it may be identified that the distance between two pupils, and the face width, and the distance between pupils and lips, and the face height have a first linear correlation. Accordingly, when the distance between two pupils, and the distance between pupils and lips can be estimated, the face width and the face height may be inferred by applying the estimated ratio.

Also, when data of FIG. 6 and data of FIG. 7 are compared, it is understood that the ratio eye/face and the ratio height/face are different in males and females. According to the data, the ratio eye/face ranges from 0.4 to 0.5 in males, and the ratio eye/face ranges from 0.3 to 0.35 in females. The ratio height/face ranges from 0.3 to 0.4 in males, and the ratio height/face ranges from 0.4 to 0.5 in females. Since there is a characteristic that the correlations are changed depending on sex differentiation, the face width and the face height may be accurately estimated when sex differentiation information of the image data here is used in estimation of the face width and the face height by using the distance between two pupils and the distance between pupils and lips.

FIG. 8 illustrates a method of distinguishing a face area from a background area in a face range according to an exemplary embodiment of the present invention. As illustrated in an image 601, the caricature generation system according to the present invention estimates a face range 610 by using a face width and a face height.

The face range 610 may be a range of a rectangle having a minimum size that includes a face, and the face width and the face height may be respectively established as a width and a height of the rectangle.

As illustrated in an image 602, the caricature generation system according to the present invention extracts, in the face range 610, a face contour 640 by distinguishing a face area 620 from a background area 630. The face area 620 and the background area 630 may be distinguished by respectively using a difference between the face area color and the background area color, and a boundary of the face range may be the face contour in a portion in which the face contour 640 is not clearly distinguishable, for example, a jaw line.

FIG. 9 is a flowchart illustrating flows of a caricature generation method according to the first exemplary embodiment of the present invention, and FIG. 10 is a flowchart illustrating step S720 of FIG. 9 in detail.

Image data is inputted from a user, in step S710, and face shape information including face contour information is generated in step S 720.

When step S710 is described with reference to FIG. 10, pupils location information and lips location information are identified from the image data first inputted from the user, in step S720-1. The identified pupils location information and the lips location information may include a coordinate of a pixel corresponding to a location of a pupil center, and a coordinate of a pixel corresponding to a location of a center of lips in the image data. The pupils location information and the lips location information may be identified by using a difference between a color of pixels corresponding to a pupils location and a lips location, and a color of skin around pupils and lips. Also, a coordinate of a clicked pixel may be identified by providing a user with the image data, and receiving, from the user, the pupils location and the lips location with a click.

In step S720-2, a face width and a face height are estimated, based on the pupils location and the lips location identified in step S720-1. For an approximation, a correlation between the pupils location and the lips location extracted from a plurality of sample image data, and the face width and the face height may be used. The correlation is respectively acquired depending on sex differentiation of a person in sample image data, and the face width and the face length may be estimated by using the correlation corresponding to the sex differentiation of the person in image data inputted from the user. In step S720-3, a face range in the image data is estimated by using the face width and the face height estimated in step S720-2. The face range may be a range of a rectangle having a minimum size including a face, and the face width and the face height may be respectively established as a width and a height of the rectangle. A location of the rectangle may be estimated by using the pupils location information and the lips location information identified in step 720-1.

Also, in step S720-3, the face area is distinguished from the background area in the face range, and the face contour information is extracted. The face area may be distinguished from the background area by using a difference between color data of a pixel corresponding to the face area, and color data of a pixel corresponding to the background area. Also, information with respect to a boundary line of the distinguished area may be the face contour information. However, when a color of the face area and a color of the background area are indistinguishable, for example, a jaw line, areas may not be clearly distinguishable by using only the difference of the color data. Accordingly, the areas may be distinguished more accurately than an existing method by distinguishing the areas in the face range. Therefore, there is an effect to generate the caricature similar to the actual person by extracting the accurate face contour information.

Also, in step S720-3, the user may be provided with the extracted face contour information, contour change information may be received from the user, and the contour information may be modified, based on the contour change information.

In step S720-4, the face shape is formed, based on the face contour information extracted in step S 720-3. The face shape may be generated by filling in, with a color similar to an actual face, an internal portion of the face contour, the face contour being extracted in step 720-3, or extracting the internal portion of the face contour from the image data inputted from the user and passing through a predetermined process.

Referring to FIG. 9 again, in step of 730, face component shapes are extracted from the image data, and character images are generated corresponding to the face component shapes. Face component shapes may include images showing shapes of eyes, eyebrows, a nose, lips, ears, and the like. A plurality of face component images may be stored in a predetermined storing apparatus, the user may be provided with a list, and the selected desirable images may be received from the user. Also, the face components may be identified from the image data inputted from the user, and the face component images similar to the identified face components may be extracted in the storing apparatus.

In step S740, the generated character images of the face components are combined with the face shape, and the caricature is generated. The pupils location and the lips location identified in step S720-1 may be used for the combination.

Hereinafter, a caricature generation system and a caricature generation method according to a second exemplary embodiment of the present invention are described in detail with reference to FIGS. 11 and 12.

FIG. 11 is a block diagram illustrating an internal configuration of a caricature generation system according to the second exemplary embodiment of the present invention. As illustrated in FIG. 11, the caricature generation system may include a shadow plate generator 210', a character image generator 220, and a caricature generator 230. The caricature generation system may further include a character image database 240. The shadow plate generator 210' generates a shadow plate including a face contour and a face curve from image data inputted from a user. The image data may be converted into a form which may use data inputted from the user in the shadow plate generator 210'. For example, when the user inputs the image data including a compressed form such as a JPEG format, a GIF format, and the like, the image data may be used by being converted into an image form including color data of all pixels such as a BMP format form. When the user inputs analog data such as an actual picture, the analog data may be converted into digital data by using a conversion apparatus such as a scanner, and the like. Also, the face contour denotes a boundary line of a face and a background, and shows a plane characteristic of a face. The face curve denotes an extent in which a face is protruded and concaved, and shows a three-dimensional characteristic of the face.

The shadow plate generator 210' extracts the face contour from the image data in order to generate the shadow plate. For extraction, the shadow plate generator 210' may identify the boundary line by dividing a face area and a background area from the image data. For example, a portion in which a color data value is significantly changed may be identified as the boundary line by comparing the color data values for each pixel of the image data.

Also, the shadow plate generator 210' generates the face curve from the image data in order to generate the shadow plate. In order to generate the face curve, the shadow plate generator 210' may use a posterizing method. Posterizing denotes reducing a range of a value of light and shade which each pixel may have in an image, and when the image data passes through a posterizing process, the image data is converted into using only a predetermined number of colors.

Accordingly, the image data having passed through the posterizing process is generally and indistinctly shown, a specific shape of each portion can not be clearly distinguishable. Therefore, when posterizing of the face area is performed, face components are imperceptibly erased, and only the general face curve remains. Also, the posterizing uses only the predetermined number of colors, and different images may be generated depending on which color the image is based upon when the color is determined. Accordingly, the shadow plate generator 210' may determine a color being a basic color of the posterizing, and the color may be determined, based on a color of the face area from the image data inputted from the user. Also, when there is a method of generating the face contour similar to the actual face, based on the image data, any method may be applied to the present invention. For example, a method of extracting the face contour in the caricature generation system according to the first exemplary embodiment of the present invention may be applied to the present invention. A character image generator 220 is referred to a description of the caricature generation system according to the above-described first exemplary embodiment of the present invention.

A caricature generator 230 combines the shadow plate generated by the shadow plate generator 210', and the character images generated by the character image generator 220, and generates the caricature. The shadow plate and the character image may have a BMP format form having color data for each pixel, and may combine two images by respectively adding the color data of the overlapped pixel depending on a predetermined ratio. Also, the color data of the pixel may be divided into three values including red (R), green (G), and blue (B), and combination may be performed by respectively adding the RGB values depending on the predetermined ratio.

Also, the caricature generator 230 may combine the shadow plate and the character images, based on a transparency value of the shadow plate by receiving the transparency value of the shadow plate inputted from the user. The transparency value of the shadow plate may denote a ratio of the color data of the pixel corresponding to the shadow plate when the color data of each pixel are added. The transparency value of the shadow plate may have a value from 0 to 1 , and combination may be performed, based on Equation as follows: [Equation] dstR = picR x alpha + chaR x (1 -alpha) dstG = picG x alpha + chaG x (1 -alpha) dstB = picB x alpha + chaB x (1 -alpha). In Equation, alpha denotes the transparency value of the shadow plate, dstR, dstG, and dstB denote the RGB value of the combined caricature, picR, picG, and picB denote the RGB value of the shadow plate, and chaR, chaG, and chaB denote the RGB value of the character images.

A character image database 240 stores character images corresponding to face components. The character image database 240 stores a plurality of character images.

When the character image generator 220 extracts the character images corresponding to face component shapes from a character image list, the character images corresponding to face component shapes are retrieved and are provided. The character image database 240 may be divided into each face component such as eyes, eyebrows, noses, mouths, ears, and the like, and store the character images.

FIG. 12 illustrates a shadow plate generated by using a posterizing introduced in a caricature generation system according to the second exemplary embodiment of the present invention.

As illustrated in FIG. 12, the caricature generation system according to the present exemplary embodiment of the present invention may generate the shadow plate by posterizing image data by a predetermined color. Reference numeral 301 illustrates a face area extracted from image data inputted from a user, and reference numeral 302 illustrates a shadow plate generated by posterizing the face area, based on a skin color of the face area in the image data. Although the shadow plate generated by the posterizing includes a face curve, face components are indistinctively shown. Accordingly, although the shadow plate is combined with character images afterwards, original face components may be not shown being overlapped with character images.

FIG. 13 illustrates a caricature changed depending on a transparency of a shadow plate according to an exemplary embodiment of the present invention.

As illustrated in FIG. 13, a caricature generation system according to the present invention may determine a combination ratio, based on the transparency value of the shadow plate when the shadow plate and the character images are combined. The transparency value of the shadow plate may denote a ratio in which the shadow plate occupies in combination, and may have a value from 0 to 1. Reference numeral 401 corresponds to a case where the transparency value of the shadow plate is 0, reference numeral 404 corresponds to a case where the transparency value of the shadow plate is 1, and reference numerals 402 and 403 show that the transparency value of the shadow plate are intermediate values. The transparency value of the shadow plate is increased as the caricature progresses from reference numeral 401 to reference numeral 404.

As described above, since the face curve is not shown when the transparency value of the shadow plate is low, the caricature appears like a drawing, and since the face curve is shown when the transparency value of the shadow plate is high, the caricature is felt like an actual photograph. The transparency value of the shadow plate may be received from the user, and may be used.

FIG. 14 is a diagram illustrating a character image database 240 according to an exemplary embodiment of the present invention. As illustrated in FIG. 14, the character image database 240 may store character images corresponding to face components such as eyes, eyebrows, noses, mouths, and ears, and may separately store character images for each face component. As an example, when the character image generator 220 extracts a shape of eyes from image data inputted from a user, the character image generator 220 may extract character images most similar to the extracted shape of eyes from among records in which a face component field corresponds to eyes in the character image database 240, and may generate the character images. As another example, when the selected character images corresponding to the shape of eyes are received from the user, the user is provided with a record list in which the face component field corresponds to eyes from the character image database 240, and the selected character images from among the list are received. FIG. 15 illustrates a picture in which a selected hairstyle and selected eye shapes of a caricature are inputted from a user according to an exemplary embodiment of the present invention. Reference numeral 610 is a caricature generated in the caricature generation system according to the present invention, reference numeral 620 is a picture of selecting eye shapes of the caricature, reference numeral 630 is a picture of selecting a hairstyle of the caricature, and reference numeral 640 illustrates a caricature depending on the hairstyle selected in reference numeral 630.

As illustrated in FIG. 15, since the caricature generation system according to the present invention enables a user to freely select face component images of the caricature, to select hairstyles, accessories, clothes, and the like, and to freely adorn the user's caricature, there is an effect that the user can desirably express himself/herself by using the caricature.

FIG. 16 illustrates an example of using, in a mobile terminal, a caricature generated according to an exemplary embodiment of the present invention.

As illustrated in FIG.16, the caricature generated by the present invention may be used for expressing oneself in the mobile terminal, and may be variously used as an image showing a user on Internet communities, personal homepages, Internet game sites, and the like. According to the present invention, since the caricature fully showing one's characteristic may be generated by using the shadow plate including the face curve, the caricature generated by the present invention may be generally used as image expressing oneself on the Internet wired/wirelessly connected.

FIG. 17 is a flowchart illustrating flows of a caricature generation method according to the second exemplary embodiment of the present invention.

In step S 801, a shadow plate including a face contour and a face curve is generated from image data inputted from a user. The face contour of the shadow plate may be generated by using a difference between a face area color and a background area color in the image data, and the face curve may be easily generated by using posterizing. The posterizing denotes reducing a range of a value of light and shade in which each pixel may have in an image, and is performed, based on a predetermined color. Also, the predetermined color may be determined by using a color of the face area.

In step S 802, face component shapes are extracted from the image data, and character images corresponding to the extracted face component shapes are generated. The face component shapes may be extracted by using a color difference between face components and surrounding skin. The character images corresponding to the extracted face component shapes may be extracted from a character image list, or the user may be provided with the character image list, and the selected character images are received from the user. User information including sex differentiation information of the user, information concerning whether double eyelids exist, and the like may be used when the character images are generated. The user information may be stored in a predetermined user information database, and may be referred to when the character images are generated.

In step S803, the shadow plate generated in step S801, and the character images generated in step S802 are combined, and the caricature is generated. The shadow plate and the character image may have a BMP format form having color data for each pixel, and may combine two images by respectively adding the color data of the overlapped pixel depending on a predetermined ratio. Also, the color data of the pixel may be divided into three values including R, G, and B, and combination may be performed by respectively adding the RGB values depending on the predetermined ratio.

Also, in step S 803, the shadow plate and the character images may be combined, based on a transparency value of the shadow plate by receiving the transparency value of the shadow plate inputted from the user. The transparency value of the shadow plate may denote a ratio of the color data of the pixel corresponding to the shadow plate when the color data of each pixel are added. The transparency value of the shadow plate may have a value from 0 to 1, and combination may be performed, based on

Equation as follows:

[Equation] dstR = picR x alpha + chaR x (1 -alpha) dstG = picG x alpha + chaG x ( 1 -alpha) dstB = picB x alpha + chaB x (1 -alpha). In Equation, alpha denotes the transparency value of the shadow plate, dstR, dstG, and dstB denote the RGB value of the combined caricature, picR, picG, and picB denote the RGB value of the shadow plate, and chaR, chaG, and chaB denote the RGB value of the character images.

A caricature generation method according to the first exemplary embodiment and the second exemplary embodiment of the present invention may be recorded in computer-readable media including program instructions to implement various operations embodied by a computer. The media may also include, alone or in combination with the program instructions, data files, data structures, and the like. The media and program instructions may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer- readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD ROM disks and DVD; magneto-optical media such as optical disks; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory, and the like. The media may also be a transmission medium such as optical or metallic lines, wave guides, and the like, including a carrier wave transmitting signals specifying the program instructions, data structures, and the like. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter. The described hardware devices may be configured to act as one or more software modules in order to perform the operations of the above-described embodiments of the present invention.

According to the present invention, there is provided a caricature generation system and method which enables a user to easily generate a caricature by merely inputting the user's own picture into image data.

Also, according to the present invention, there is provided a caricature generation system and method which can generate a caricature similar to an actual user by accurately extracting a face contour when the caricature is generated. Also, according to the present invention, there is provided a caricature generation system and method which can generate a caricature which a user desires by enabling the user to modify an automatically extracted contour with a simple mouse click.

Also, according to the present invention, there is provided a caricature generation system and method which can extract a face contour by using a difference between a face area color and a background area color when the caricature is generated, thereby generating a caricature similar to an actual user.

Also, according to the present invention, there is provided a caricature generation system and method which can extract a contour very similar to an actual user's contour with reference to an estimated face width and a face height by estimating the face width and the face height, based on a pupils location and a lips location when a face contour is extracted in order to generate a caricature.

Also, according to the present invention, there is provided a caricature generation system and method which can extract a contour in which a face area color and a background area color are similar, such as a jaw line, by extracting a contour in a face range when a face contour is extracted in order to generate a caricature. Also, according to the present invention, there is provided a caricature generation system and method which can express a three-dimensional shape of an actual face by using a shadow plate including a face contour and a face curve when a caricature is generated.

Also, according to the present invention, there is provided a caricature generation system and method which can easily generate a face curve by posterizing a face area in image data inputted from a user.

Also, according to the present invention, there is provided a caricature generation system and method which can generate a caricature similar to an actual face by using a color of an actual face area when a face area is posterized from image data inputted from a user in order to generate a face curve.

Also, according to the present invention, there is provided a caricature generation system and method which can generate a caricature having various moods by receiving, from a user, a transparency of a shadow plate and determining a combination ratio when the shadow plate and a character image are combined. Although a few embodiments of the present invention have been shown and described, the present invention is not limited to the described embodiments. Instead, it would be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A caricature generation system for generating a caricature by using image data inputted from a user, the system comprising: a face shape generator which extracts, from the image data, face shape information including face contour information generated by distinguishing a face area from a background area, and generates a face shape; a character image generator which extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes; and a caricature generator which combines the face shape and the character images, and generates the caricature.

2. The system of claim 1, wherein the face shape information comprises pupils location information, lips location information, face width information, and face height information.

3. The system of claim 1, wherein the face shape generator comprises: a face component location identifier which identifies pupils location information and lips location information from the image data; a face length estimator which estimates a face width and a face height, based on the identified pupils location and the identified lips location; a face contour information extractor which distinguishes the face area from the background area in an estimated face range by using the estimated face width and the face height, and extracts the face contour information; and a face shape former which processes the face contour information or the image data, and forms the face shape.

4. The system of claim 3, wherein the face length estimator estimates the width by using a distance between two pupils, and estimates the length by using a distance between pupils and lips.

5. The system of claim 3, wherein the face length estimator extracts, from a plurality of sample image data, a correlation between the distance between two pupils, and the face width, and a correlation between the distance between pupils and lips, and the face height, and estimates the width and the length by using the correlations.

6. The system of claim 3, wherein the face length estimator further comprises: a user sex differentiation information database which stores user information including sex differentiation information.

7. The system of claim 6, wherein the face length estimator extracts the sex differentiation information of the user with reference to the database, and estimates the width and the length by further considering the extracted sex differentiation information.

8. The system of claim 3, wherein the face contour information extractor distinguishes the face area and the background area by using a difference between a face area color and a background area color.

9. The system of claim 3, wherein the face contour information extractor extracts a face contour from the face area.

10. The system of claim 3, wherein the face contour information extractor provides the user with the extracted contour information, receives contour change information from the user, and modifies the contour information, based on the contour change information.

11. The system of claim 3, wherein the face shape former performs a process of filling in, with a color similar to an actual face, an internal portion of the face contour, the face contour being extracted from the face contour information extractor, and forms the face shape.

12. The system of claim 3, wherein the face shape former extracts, from the image data, an internal portion of the face contour, and forms the face shape.

13. The system according to any one of claims 1 and 3, further comprising: a character image database which stores a plurality of character images having a plurality of face component shapes, wherein the character image generator generates the character image corresponding to the extracted plurality of face component shapes with reference to the character image database.

14. The system according to any one of claims 1 and 3, further comprising: a character image database which stores a plurality of character images having a plurality of face component shapes, wherein the character image generator provides the user with a character image list stored in the character image database, receives a selection of character images from the user, and generates the character images, based on the selected character images.

15. The system according to any one of claims 1 and 3, wherein the character image generator generates the character images by further considering user information including sex differentiation information of the user, or information concerning whether double eyelids exist.

16. The system according to any one of claims 1 and 3, further comprising: a user information database which stores user information including sex differentiation information or information concerning whether double eyelids exist, wherein the character image generator extracts the sex differentiation information of the user, or the information concerning whether double eyelids exist with reference to the user information database, and generates the character images by further considering the extracted sex differentiation information, or the information concerning whether double eyelids exist.

17. The system of claim 3, wherein the caricature generator determines a location to combine the character images, based on the identified pupils location and the identified lips location in the face component location identifier.

18. A caricature generation system for generating a caricature from image data inputted from a user, the system comprising: a shadow plate generator which generates a shadow plate including a face contour and a face curve generated by dividing a face area and a background area from the image data; a character image generator which extracts face component shapes from the image data, and generates character images corresponding to the extracted face component shapes; and a caricature generator which combines the shadow plate and the character images, and generates the caricature.

19. The system of claim 18, wherein the shadow plate generator extracts the face contour, posterizes the face area specified by the face contour, based on a predetermined color, and generates the face curve.

20. The system of claim 18, wherein the predetermined color is determined, based on a color of the face contour area.

21. The system of claim 18, wherein the caricature generator determines a location to combine the character images in the shadow plate, based on a location of face components.

22. The system of claim 18, wherein the caricature generator combines the shadow plate and the character images depending on a predetermined combination ratio, and the combination ratio is determined on a transparency value of the shadow plate inputted from the user.

23. The system of claim 22, wherein the caricature generator generates the caricature by considering a red, green, and blue (RGB) value of the shadow plate, and the RGB value of the character images.

24. The system of claim 23, wherein the transparency value of the shadow plate corresponding to an alpha has a value from 0 to 1, and the RGB value of each pixel of the combined caricature is determined by Equation as follows:

[Equation] dstR = picR x alpha + chaR x (1 -alpha) dstG = picG x alpha + chaG x (1 -alpha) dstB = picB x alpha + chaB x (1 -alpha), where dstR, dstG, and dstB denote the RGB value of the caricature, picR, picG, and picB denote the RGB value of the shadow plate, and chaR, chaG, and chaB denote the RGB value of the character images.

25. The system of claim 18, further comprising: a character image database which stores a plurality of character images having a plurality of face component shapes, wherein the character image generator generates the character images corresponding to the extracted plurality of face component shapes with reference to the character image database.

26. The system of claim 18, further comprising: a character image database which stores a plurality of character images having a plurality of face component shapes, wherein the character image generator provides the user with a character image list stored in the character image database, receives the selected character images from the user, and generates the character images, based on the selected character images.

27. The system of claim 18, wherein the character image generator generates the character images by further considering user information including sex differentiation information of the user, or information concerning whether double eyelids exist.

28. The system of claim 18, further comprising: a user information database which stores user information including sex differentiation information or information concerning whether double eyelids exist, wherein the character image generator extracts the sex differentiation information of the user, or the information concerning whether double eyelids exist with reference to the user information database, and generates the character images by further considering the extracted sex differentiation information, or the information concerning whether double eyelids exist.

29. A caricature generation method of generating a caricature by using image data inputted from a user, the method comprising: a step of distinguishing a face area from a background area, extracting, from the image data, face shape information including face contour information, and generating a face shape; a step of extracting face component shapes from the image data, and generating character images corresponding to the extracted face component shapes; and a step of combining the face shape and the character images, and generating the caricature.

30. The method of claim 29, wherein the step of distributing comprises: identifying pupils location information and lips location information from the image data; estimating a face width and a face height, based on the identified pupils location and the identified lips location; distinguishing the face area from the background area in an estimated face range by using the estimated face width and the face height, and extracting the face contour information; and forming the face shape, based on the extracted face contour information, generating an image of ears, eyes, mouth, and nose corresponding to eyes, eyebrows, nose, mouth, or ears shape of the image data, combining the face shape with the image of ears, eyes, mouth, and nose, and generating the caricature.

31. A caricature generation method of generating a caricature from image data inputted from a user, the method comprising: a step of generating a shadow plate including a face contour and a face curve generated by dividing a face area and a background area from the image data; a step of extracting face component shapes from the image data, and generating character images corresponding to the extracted face component shapes; and a step of combining the shadow plate and the character images, and generating the caricature.

32. A computer-readable recording medium storing a program for implementing the method according to any one of claims 29 through 31.