JP2006301779A - Image processing system, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing system for improving precision of face candidate region extraction processing. <P>SOLUTION: The image deciding part 5 of an image processing system 100 extracts a skin color region from image information, and decides whether or not the extracted skin color region is a face candidate region by using a neural network learning the face candidate region extraction processing of a person by using each characteristic value of the skin color region as an input signal. Then, the face candidate region is emphasized and displayed on an image to be displayed by a display part 10 so as to be clearly discriminated from the other regions. An operator decides whether or not each face candidate region is the face region, and selects the face region from the face candidate region via an instruction input part 9. Then, the selection result of the selected face region is stored in a teacher data storage part 6 as teacher data. A neuro learning part 7 executes the learning of the neural network on the basis of those teacher data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing system, an image processing method, and an image processing program that perform image processing on image information.

  In recent years, digital still cameras have become widespread and their performance has greatly improved. In connection with this, the request | requirement with respect to the quality improvement of a picked-up image is also increasing. Many of the subjects to be photographed are people, but in this case, there is a particularly high demand for higher quality for the face portion being photographed. In response to such a request, many techniques for extracting a face area from a photographed image and performing correction processing based on the extracted face area information are disclosed. For example, it is determined whether or not the face candidate area is a face by determining whether the positional relationship of the characteristic part of the face candidate area matches the characteristic part (eyes, etc.) of the face, by Kohonen's self-organization A method of determining using a method has been proposed (see, for example, Patent Document 1 and Patent Document 2).

In addition to traditional photo studios, photography studios are also diversifying according to various needs, such as a children's photo studio and a women's photo studio. In such a photo studio, since the concept and target user are different for each photo studio, there is a bias in the number, gender, and age of the person who is the subject. In addition, because we are trying to create images that match different concepts and target users, shooting conditions such as lighting and composition such as shooting light source and light distribution vary greatly from photo studio to photo studio. The characteristics of the captured original image data are different.
Japanese Patent Laid-Open No. 5-282457 JP-A-6-214970

  However, in the conventional technology, for each system with different environments (trends such as shooting conditions), the extraction conditions of face candidate area extraction processing in the system are customized according to the environment to improve accuracy. Was difficult due to heavy human load.

  The present invention has been made in view of the above-described problems in the prior art, and an object of the present invention is to improve the accuracy of face candidate area extraction processing according to each system.

  In order to solve the above-mentioned problem, the invention according to claim 1 is characterized in that display means for displaying an image based on acquired image information, and face candidate region extraction processing in which extraction conditions are determined by learning, from the image information. Face candidate area extracting means for extracting a face candidate area, display control means for displaying the extracted face candidate area on an image displayed on the display means, and an operator from the displayed face candidate area A face area selecting means for selecting the image, an image correcting means for performing a predetermined correction process on at least the selected face area to generate a corrected image, and the face candidate area extracting process based on the selection result. An image processing system comprising learning means for performing learning.

  According to a second aspect of the present invention, in the image processing system according to the first aspect, when a learning result of the learned face candidate region extraction process satisfies a predetermined condition, the learning result is extracted as the face candidate region. A reflection means for reflecting in the processing is provided.

  According to a third aspect of the present invention, in the image processing system according to the first or second aspect, the face candidate area extraction process is a face candidate area extraction process using a neural network.

  According to a fourth aspect of the present invention, a face candidate region is extracted from the image information by display means for displaying an image based on the acquired image information and a first face candidate region extraction process in which an extraction condition is determined by learning. A face candidate area extracting means for displaying, a display control means for displaying the extracted face candidate area on an image displayed on the display means, and an operator for selecting a face area from the displayed face candidate areas Different from the first face candidate area extraction process based on the selection result, the image correction means that performs a predetermined correction process on at least the selected face area to generate a corrected image, and the selection result An image processing system comprising: learning means for performing learning of a second face candidate region extraction process having a configuration and having an extraction condition determined by learning.

  According to a fifth aspect of the present invention, in the image processing system according to the fourth aspect, the first face candidate is obtained when a learning result of the learned second face candidate region extraction process satisfies a predetermined condition. The image processing apparatus includes a replacement unit that replaces the area extraction process with the second face candidate area extraction process.

  The invention according to claim 6 is the image processing system according to claim 4 or 5, wherein the first face candidate area extraction process is a face candidate area extraction process using a first neural network, The second face candidate area extraction process is a face candidate area extraction process using a second neural network having a configuration different from that of the first neural network.

  The invention according to claim 7 is a display unit that displays an image based on the acquired image information, a face candidate region extraction unit that extracts a face candidate region from the image information by a predetermined region extraction process, and the display unit Display control means for displaying the extracted face candidate area on an image displayed on the screen, face area selection means for an operator to select a face area from the displayed face candidate area, and at least the selected Image correction means for performing a predetermined correction process on the face area and generating a correction image; and learning means for performing learning of a face candidate area extraction process in which an extraction condition is determined by learning based on the selection result. An image processing system is provided.

  In the image processing system according to claim 7, when the learning result of the learned face candidate region extraction process satisfies a predetermined condition, the face candidate region reflecting the learning result is provided. An introduction means for introducing an extraction process is provided.

  The invention according to claim 9 is the image processing system according to claim 7 or 8, wherein the face candidate area extraction process is a face candidate area extraction process using a neural network.

  According to a tenth aspect of the present invention, in the image processing system according to any one of the first to ninth aspects, the learning means sets the size of the face candidate region to be determined as to whether or not the face is a face. According to this, learning is performed.

  The invention according to claim 11 is the image processing system according to any one of claims 1 to 10, wherein the learning means performs learning according to a light distribution condition when the image is captured. It is characterized by doing.

  According to a twelfth aspect of the present invention, there is provided display means for displaying an image based on acquired image information, and face candidates for extracting face candidate regions from the image information by face candidate region extraction processing in which extraction conditions are determined by learning. Area extraction means, display control means for displaying the extracted face candidate area on an image displayed on the display means, and face area selection for an operator to select a face area from the displayed face candidate areas Means for correcting the face candidate area extraction process based on the selection result, and a correction intensity calculation means for calculating a correction intensity by a correction intensity calculation process in which a calculation condition is determined by learning. Correction processing means for performing correction processing on the image information based on the calculated correction strength, and a correction strength for correcting the calculated correction strength in accordance with an operator instruction. And correction means, on the basis of the modification result is an image processing system characterized by comprising a second learning means for performing learning of the correction intensity calculation process.

  According to a thirteenth aspect of the present invention, there is provided a face candidate region extraction step of extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning, and an image is displayed on the display means based on the image information A display step of displaying the extracted face candidate region on the image, a face region selecting step of allowing the operator to select a face region from the displayed face candidate region, and at least the selected face region An image processing method comprising: an image correction step of performing a predetermined correction process for generating a corrected image; and a learning step of performing learning of the face candidate region extraction process based on the selection result It is.

  The invention described in claim 14 is the image processing method according to claim 13, wherein when the learned result of the learned face candidate region extraction process satisfies a predetermined condition, the learned result is extracted as the face candidate region. It is reflected in the processing.

  According to a fifteenth aspect of the present invention, in the image processing method according to the thirteenth or fourteenth aspect, the face candidate area extraction process is a face candidate area extraction process using a neural network.

  According to a sixteenth aspect of the present invention, there is provided a face candidate region extraction step of extracting a face candidate region from image information by a first face candidate region extraction process in which an extraction condition is determined by learning, and an image based on the image information. A display step of displaying the extracted face candidate region on the image, and a face region selecting step of allowing the operator to select a face region from the displayed face candidate region. The image correction process for performing a predetermined correction process on the face area and generating a corrected image, and the first face candidate area extraction process based on the selection result, the extraction condition is determined by learning. And a learning step of performing learning of the determined second face candidate region extraction process.

  The invention according to claim 17 is the image processing method according to claim 16, wherein the first face candidate is obtained when a learning result of the learned second face candidate region extraction process satisfies a predetermined condition. The area extraction process is replaced with the second face candidate area extraction process.

  The invention according to claim 18 is the image processing method according to claim 16 or 17, wherein the first face candidate area extraction process is a face candidate area extraction process using a first neural network, The second face candidate area extraction process is a face candidate area extraction process using a second neural network having a configuration different from that of the first neural network.

  According to a nineteenth aspect of the present invention, there is provided a face candidate region extracting step of extracting a face candidate region from image information by a predetermined face candidate region extraction process, and displaying an image on a display unit based on the image information, A display step for displaying the extracted face candidate region on the top, a face region selecting step for allowing an operator to select a face region from the displayed face candidate region, and a predetermined correction process for at least the selected face region. And an image correction process for generating a corrected image, and a learning process for performing learning of a face candidate region extraction process in which an extraction condition is determined by learning based on the selection result. Is the method.

  According to a twentieth aspect of the present invention, in the image processing method according to the nineteenth aspect, when a learning result of the learned face candidate region extraction process satisfies a predetermined condition, the face candidate region reflecting the learning result It is characterized by introducing face candidate region extraction processing by extraction processing.

  The invention according to claim 21 is the image processing method according to claim 19 or 20, wherein the face candidate area extraction process is a face candidate area extraction process using a neural network.

  According to a twenty-second aspect of the present invention, in the image processing method according to any one of the thirteenth to twenty-first aspects, in the learning step, the size of a face candidate region that is a determination target of whether or not the face is a target is determined. According to this, learning is performed.

  The invention according to claim 23 is the image processing method according to any one of claims 13 to 22, wherein, in the learning step, learning is performed according to a light distribution condition when the image is taken. It is characterized by doing.

  According to a twenty-fourth aspect of the present invention, there is provided a face candidate region extraction step of extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning, and an image is displayed on the display means based on the image information. Based on the display result, a display step of displaying the extracted face candidate region on the image, a face region selection step of selecting a face region from the displayed face candidate region, and the selection result. Based on the first learning step for performing the learning of the face candidate region extraction processing, the correction strength calculating step for calculating the correction strength by the correction strength calculation processing in which the calculation condition is determined by learning, and the calculated correction strength A correction processing step for correcting the image information, a correction strength correction step for correcting the calculated correction strength in accordance with an operator instruction, and the correction result Based on a second learning step of performing learning of the correction intensity calculation process, an image processing method, which comprises a.

  According to a twenty-fifth aspect of the present invention, there is provided a face candidate region extraction function for extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning, and an image based on the image information. A display function for causing the display means to display the extracted face candidate area on the image, and a face area selection function for causing the operator to select a face area from the displayed face candidate area, at least the selected Image processing for realizing an image correction function for performing a predetermined correction process on the face area and generating a corrected image, and a learning function for performing learning of the face candidate area extraction process based on the selection result It is a program.

  The invention according to claim 26 is the image processing program according to claim 25, wherein, when the learning result of the learned face candidate region extraction process satisfies a predetermined condition, the learning result is stored in the computer. It is characterized by realizing a function to be reflected in the face candidate area extraction process.

  The invention according to claim 27 is the image processing program according to claim 25 or 26, wherein the face candidate area extraction process is a face candidate area extraction process using a neural network.

  The invention according to claim 28 is based on the face candidate area extraction function for extracting a face candidate area from image information by a first face candidate area extraction process in which an extraction condition is determined by learning, and the image information. A display function for displaying the image on the display means and displaying the extracted face candidate area on the image, a face area selection function for allowing the operator to select a face area from the displayed face candidate area, and at least An image correction function that performs a predetermined correction process on the selected face area to generate a corrected image, and a configuration different from the first face candidate area extraction process based on the selection result, An image processing program for realizing a learning function for performing learning of a second face candidate region extraction process in which an extraction condition is determined.

  According to a twenty-ninth aspect of the present invention, in the image processing program according to the twenty-eighth aspect, when the learning result of the learned second face candidate region extraction process satisfies a predetermined condition in the computer, the first It is characterized in that a function of replacing one face candidate area extraction process with the second face candidate area extraction process is realized.

  The invention according to claim 30 is the image processing program according to claim 28 or 29, wherein the first face candidate area extraction process is a face candidate area extraction process using a first neural network, The second face candidate area extraction process is a face candidate area extraction process using a second neural network having a configuration different from that of the first neural network.

  According to a thirty-first aspect of the present invention, a computer displays a face candidate region extraction function for extracting a face candidate region from image information by a predetermined face candidate region extraction process, and displays an image on a display unit based on the image information. A display function for displaying the extracted face candidate area on the image, a face area selection function for allowing the operator to select a face area from the displayed face candidate area, and at least a predetermined face area for the selected face area An image for realizing an image correction function that performs correction processing and generates a corrected image, and a learning function that performs learning of face candidate region extraction processing in which an extraction condition is determined by learning based on the selection result It is a processing program.

  According to a thirty-second aspect of the present invention, in the image processing program according to the thirty-first aspect, the learning result is reflected when the learning result of the learned face candidate region extraction process satisfies a predetermined condition in the computer. The present invention is characterized in that a function for introducing face candidate area extraction processing by the face candidate area extraction processing performed is realized.

  The invention according to claim 33 is the image processing program according to claim 31 or 32, wherein the face candidate area extraction process is a face candidate area extraction process using a neural network.

  A 34th aspect of the present invention is the image processing program according to any one of the 25th to 33rd aspects, wherein when the learning function is realized, a face candidate region that is a target for determining whether or not it is a face. Learning is performed according to the size of.

  According to a thirty-fifth aspect of the present invention, in the image processing program according to any one of the twenty-fifth to thirty-fourth aspects, when the learning function is realized, according to a light distribution condition when the image is captured. It is characterized by carrying out learning.

  According to a thirty-sixth aspect of the present invention, a face candidate region extraction function for extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning, and an image based on the image information. A display function for displaying the extracted face candidate area on the image, a face area selecting function for allowing the operator to select a face area from the displayed face candidate area, and the selection result. Based on a first learning function for performing learning of the face candidate region extraction process, a correction intensity calculating function for calculating a correction intensity by a correction intensity calculating process in which a calculation condition is determined by learning, and the calculated correction A correction processing function for correcting the image information based on the intensity, and a correction intensity correction function for correcting the calculated correction intensity according to an operator instruction , On the basis of the modification result is an image processing program for realizing a second learning function implementing learning of the correction intensity calculation process.

  According to the invention described in claims 1, 13, and 25, the face candidate area extraction process is learned based on the face area selection result by the operator, so that the face candidate area extraction process is performed according to the running environment. By automatically optimizing the extraction conditions, the accuracy of the face candidate region extraction process can be improved.

  According to the invention described in claims 2, 14, 26, the accuracy of the face candidate area extraction process can be improved by reflecting the learning result in the face candidate area extraction process.

  According to the third, fifteenth and twenty-seventh aspects of the present invention, the face candidate region extraction process is automatically optimized by automatically optimizing the extraction conditions of the face candidate region extraction process using the neural network according to the running environment. Accuracy can be improved.

  According to the invention described in claim 4, 16, 28, the second face candidate area extraction based on the result of the operator selecting the face area from the face candidate areas extracted by the first face candidate area extraction process. Since the learning of the process is performed, the accuracy of the second face candidate area extraction process is improved by automatically optimizing the extraction condition of the second face candidate area extraction process according to the running environment. Can do.

  According to the invention described in claims 5, 17 and 29, the accuracy of the face candidate area extraction process can be improved by replacing the first face candidate area extraction process with the second face candidate area extraction process. .

  According to the invention described in claims 6, 18, and 30, by automatically optimizing the extraction condition of the face candidate area extraction process using the second neural network according to the running environment, The accuracy of the face candidate area extraction process can be improved.

  According to the seventh, nineteenth and thirty-first aspects of the present invention, learning of the face candidate area extraction process is performed based on the result of the operator selecting the face area from the face candidate areas extracted by the predetermined area extraction process. Therefore, the accuracy of the face candidate area extraction process can be improved by automatically optimizing the extraction conditions of the face candidate area extraction process according to the running environment.

  According to the invention described in claims 8, 20, and 32, the accuracy of the face candidate area extraction process can be improved by introducing the face candidate area extraction process reflecting the learning result.

  According to the invention described in claims 9, 21 and 33, the face candidate region extraction processing is automatically optimized by automatically optimizing the extraction conditions of the face candidate region extraction processing using the neural network according to the running environment. Accuracy can be improved.

  According to the tenth, twenty-second, and thirty-fourth aspects of the present invention, the accuracy of face candidate region extraction processing can be improved for face candidate regions of any size, and the extraction accuracy as a whole can be improved. In addition, since the extraction accuracy increases with respect to the size of the face candidate region having a high appearance frequency, the extraction accuracy can be improved for the face candidate region having a high importance.

  According to the invention described in claims 11, 23, and 35, the accuracy of face candidate region extraction processing can be improved for all light distribution conditions, and the extraction accuracy can be improved as a whole. In addition, since the extraction accuracy increases for light distribution conditions with high appearance frequency, the extraction accuracy can be improved for light distribution conditions with high importance.

  According to the invention described in claims 12, 24, and 36, the accuracy of the face candidate region extraction process can be improved and the accuracy of the correction strength calculation process can be improved according to the running environment.

[First Embodiment]
First, a first embodiment to which the present invention is applied will be described.
FIG. 1 shows a configuration of an image processing system 100 according to the first embodiment. An image processing system 100 is a photographic print system that optically prints a developed and photographed image on a photographic paper. As shown in FIG. 1, a light source unit 1, an imaging unit 2, an exposure control unit 3, and image information. The acquisition unit 4, the image determination unit 5, the teacher data storage unit 6, the neuro learning unit 7, the image processing unit 8, the instruction input unit 9, and the display unit 10 are configured.

  The light emitted from the light source unit 1 is transmitted through the color adjustment filters (Y, M, C), and the color tone is adjusted. The light is diffusely reflected in the light diffusing member, homogenized, and irradiated onto the film. The light transmitted through the film passes through the lens, forms an image on the film on the photographic paper, and is exposed under the control of the exposure control unit 3. The exposed photographic paper is developed by a development processing apparatus (not shown), and a photograph is completed. The light transmitted through the film is appropriately guided to the imaging unit 2 by a movable mirror. The imaging unit 2 collects the optical image input from the movable mirror and outputs it to the image information acquisition unit 4 as film image information.

  The exposure control unit 3 determines the exposure time of the film and the color balance of the light source based on the correction processing performed in the image processing unit 8, and controls the color adjustment filter and the shutter according to the determined exposure time and color balance. To do.

  The image information acquisition unit 4 acquires the image information input from the imaging unit 2 and outputs it to the image determination unit 5.

  The image determination unit 5 calculates various characteristic values (average values of pixel signal values, statistical values such as dispersion, light distribution conditions, etc.) from the image information acquired from the image information acquisition unit 4, and calculates the calculated characteristic values. A hierarchical neural network is obtained from the neuro-learning unit 7 as an input signal and a value indicating the likelihood of a face as an output signal, and a face candidate region is extracted using the hierarchical neural network. Here, face candidate region extraction processing using a neural network is used as face candidate region extraction processing in which extraction conditions are determined by learning. Note that “a face candidate area extraction process in which an extraction condition is determined by learning” refers to a face candidate area extraction process in which an extraction condition can be uniquely determined according to a predetermined algorithm.

  The teacher data storage unit 6 stores teacher data necessary for learning of the neural network, including information on whether or not the face region is input by the instruction input unit 9.

  The neuro-learning unit 7 performs learning of the hierarchical neural network using the information about whether or not the face area is stored in the teacher data storage unit 6 as teacher data. Further, the neuro-learning unit 7 executes a neuro-replacement process for replacing the hierarchical neural network with the hierarchical neural network that has further performed learning (see FIGS. 15 and 16). The hierarchical neural network applied in the present embodiment will be described later with reference to FIGS.

  When displaying the image data to be corrected on the display unit 10, the image processing unit 8 creates display data in which the face candidate area extracted by the image determination unit 5 is displayed on the display image. Output to. Further, the image processing unit 8 performs correction processing on the image information to be corrected based on the information on whether the face candidate region is extracted by the image determination unit 5 and whether the face region is input by the instruction input unit 9. Thus, a corrected image is created, and the created corrected image is output to the display unit 10. Further, when there is an instruction to output a corrected image, the image processing unit 8 outputs the corrected image information to the exposure control unit 3.

  The instruction input unit 9 includes an input device such as a keyboard and a mouse, and outputs an operation signal generated by operating the input device to the image processing unit 8. Note that the instruction input unit 9 may include a touch panel provided to cover the display screen of the display unit 10. The touch panel detects coordinates instructed by a touch based on a reading principle such as an electromagnetic induction type, a magnetostriction type, and a pressure sensitive type, and outputs the detected coordinates to the image processing unit 8 as a position signal.

  The display unit 10 includes a display screen such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and performs a required display according to a display control signal input from the image processing unit 8.

  The image processing system to which the present invention is applied is not limited to the image processing system 100 that is a photo print system for optical printing as shown in FIG. FIG. 2 shows a configuration of an image processing system 200 as another example of the image processing system to which the present invention is applied. Hereinafter, in the image processing system 200, the same components as those in the image processing system 100 in FIG. 1 are denoted by the same reference numerals, and the description of the functions of the components having the same functions is omitted.

  As shown in FIG. 2, the image processing system 200 includes an image information acquisition unit 4, an image determination unit 5, a teacher data storage unit 6, a neuro learning unit 7, an image processing unit 8, an instruction input unit 9, a display unit 10, and a reduction unit. The image forming unit 11, DSC (digital still camera) 12, document scanner 13, recording medium driver 14, communication control unit 15, image information output unit 16, printer unit 17, and recording medium writing unit 18 are configured.

  The image information acquisition unit 4 acquires digital image information input via the DSC 12, the document scanner 13, the recording media driver 14, and the communication control unit 15, and outputs the digital image information to the reduced image creation unit 11 or the image processing unit 8.

  The image processing unit 8 performs correction processing on the image information to be corrected based on the face candidate area extracted by the image determination unit 5 and the information indicating whether the face area is input by the instruction input unit 9. Then, a corrected image is created, and the created corrected image is output to the display unit 10. If there is an instruction to output a corrected image, the image processing unit 8 creates a corrected image corresponding to the designated output destination and outputs the corrected image to the image information output unit 16.

  The reduced image creating unit 11 creates a reduced image (thumbnail image) from the image information acquired from the image information acquiring unit 4 as necessary (for example, when an instruction is given from the operator), and sends it to the image determining unit 5. Output.

  The DSC 12 includes an optical lens unit for photographing including a focusing lens, a zoom lens, a shutter, a diaphragm, a photoelectric conversion unit including an imaging element such as a CCD (Charge Coupled Device), and the like, and is obtained by photographing a subject. The captured image is photoelectrically converted to acquire a digital image signal, and the acquired digital image signal is output to the image information acquisition unit 4.

  The document scanner 13 includes a light source, a CCD, an A / D converter, and the like. The document scanner 13 irradiates a document such as a photographic print placed on a document table with light from the light source, and the reflected light is an electrical signal (analog signal) by the CCD. The digital signal is acquired by converting the analog signal into a digital signal by the A / D converter, and the acquired digital image signal is output to the image information acquisition unit 4.

  The recording media driver 14 is a CD-R, a memory stick (registered trademark), a smart media (registered trademark), a compact flash (registered trademark), a multimedia card (registered trademark), an SD memory card (registered trademark), and the like. It is configured to be able to mount media, reads image signals recorded on these recording media, and outputs the read results to the image information acquisition unit 4.

  The communication control unit 15 controls communication between an image processing system 200 and an external device connected to a communication network N such as a LAN (Local Area Network) or the Internet. Specifically, the communication control unit 15 performs control for outputting the image information received from the external device to the image information acquisition unit 4, and sets the image information acquired from the image information output unit 16 to a designated address. Control to send.

  The image information output unit 16 outputs the image information processed by the image processing unit 8 to a designated output destination (any one of the communication control unit 15, the printer unit 17, and the recording medium writing unit 18).

  The printer unit 17 exposes the photosensitive material based on the image information input from the image information output unit 16, develops the exposed photosensitive material, and dries it to create a photographic print.

  The recording medium writing unit 18 has a configuration in which various recording media can be mounted, and records the image information processed by the image processing unit 8 on the mounted recording medium.

<neural network>
Next, a neural network used in the image processing according to the present embodiment will be briefly described. As shown in FIG. 3, this neural network is a hierarchical neural network (hereinafter simply referred to as a neural network) having an input layer, an intermediate layer, and an output layer. The neural network preferably uses a digital neural network chip, but can also be realized by using a general-purpose DSP (Digital Signal Processor) and a dedicated emulation program, or using a normal CPU and an emulation program. It doesn't matter.

ここで、ｆ（ｘ）は一般に式（２）に示すような非線形のシグモイド関数が用いられるが、必ずしもこれに限定されるわけではない。

Each layer of the neural network is composed of structural units called neurons. Except for the input layer, an arbitrary neuron receives a large number of inputs called synapses as shown in FIG. 4 and multiplies each input value x _i by a predetermined weight w _i called bond strength to obtain the sum, It functions as a functional element that gives the output y as a result of evaluating it with a predetermined output function f. The output function f is expressed as Equation (1).

Here, f (x) is generally a non-linear sigmoid function as shown in Equation (2), but is not necessarily limited thereto.

Here, the input weight w _i of each neuron is important information for determining the relationship between the input and output of the neural network, in other words, the operation of the neural network, and is determined by performing a learning operation on the neural network. In FIG. 3, only one intermediate layer is shown, but this can be configured in one or more arbitrary layers. Hierarchical networks are known to be able to achieve any function that can be synthesized if there is only one intermediate layer, but increasing the number of intermediate layers is advantageous in terms of learning efficiency and the number of neurons. It has been known.

The neural network has the following characteristics.
(1) A multi-input / multi-output nonlinear system can be realized with a relatively simple configuration.
(2) Each neuron in each layer can be operated independently, and high-speed operation can be expected by parallel processing.
(3) Arbitrary input / output relations can be realized by providing appropriate teacher data for learning.
(4) The system has generalization ability. That is, there is an ability to give a generally correct output even to an unlearned input pattern that is not necessarily given as teacher data.

  Here, the teacher data is a pair of an input pattern and a desired output pattern for the input pattern, and a plurality of teacher data are usually prepared. The operation of determining a desired output pattern for a specific input pattern is generally determined by relying on the subjective judgment of an expert. Major applications of neural networks include (a) nonlinear function approximation and (b) clustering (identification, recognition, classification).

ここで、式（３）の関数ｆ（ｘ）は、式（２）に示した非線形のシグモイド関数、Ｙ_iLはニューロンの出力、Ｕ_iLはニューロンの内部状態、Ｙ_i＜Ｌ−１＞はＬ−１層のニューロンの出力＝Ｌ層のニューロンの入力、ｗ_ij＜Ｌ＞は結合強度である。 Next, a learning method using teacher data for the neural network will be described. The state of the i-th neuron in the L-th layer with respect to the p-th input pattern is expressed as Equation (3) and Equation (4).

Here, the function f (x) in Expression (3) is the nonlinear sigmoid function shown in Expression (2), Y _iL is the output of the neuron, U _iL is the internal state of the neuron, and Y _i <L−1> is L-1 layer neuron output = L layer neuron input, w _ij <L> is the connection strength.

ただしｋは出力層の番号であり、Ｔ_i（ｐ）はｐ番目の入力パターンに対する望ましい出力パターンである。 In such a definition, the widely used error back propagation learning method (back propagation learning method, hereinafter referred to as BP method) uses the mean square error as an error evaluation function, and the error Define E.

Where k is the output layer number and T _i (p) is the desired output pattern for the pth input pattern.

ここで、∂は偏微分記号（ラウンド）、ηは学習率を表す係数である。Δｗ_ij＜Ｌ＞を各々のｗ_ij＜Ｌ＞に加算することによって学習が進む。なお、この形式のＢＰ法では学習が進んだ結果、教師データとの誤差の絶対値が小さくなってくると、Δｗの絶対値も小さくなり、学習が進まなくなるという現象が指摘されており、その問題点を回避するために、種々の改良が提案されている。 In this case, the correction amount Δw of each bond strength based on the BP method is determined by the following equation.

Here, ∂ is a partial differential symbol (round), and η is a coefficient representing a learning rate. Learning proceeds by adding Δw _ij <L> to each w _ij <L>. In this type of BP method, it has been pointed out that when learning progresses, the absolute value of the error from the teacher data decreases, the absolute value of Δw also decreases, and learning does not proceed. Various improvements have been proposed in order to avoid problems.

式（７）の第二項は慣性項（モーメンタム項）と呼ばれ、現在の修正量だけでなく、過去の修正量も考慮することで収束時の振動を抑え、学習を高速化する効果がある。安定化定数αは1．0以下の正の実数で、過去の修正量の考慮の度合いを規定する係数である。 For example, there is a proposal to change the definition of Δw as follows.

The second term in equation (7) is called the inertial term (momentum term), and it has the effect of suppressing the vibration at the time of convergence by considering not only the current correction amount but also the past correction amount, and speeding up learning. is there. The stabilization constant α is a positive real number of 1.0 or less, and is a coefficient that defines the degree of consideration of past correction amounts.

Alternatively, there is also a proposal of dynamically changing the learning rate η so as to satisfy the following expression according to the number of learnings n.

  Furthermore, there is also a proposal to make the error evaluation in a form that cancels out the derivative of the sigmoid function instead of the mean square error. In any case, an output approximate to the teacher data can be obtained by performing the learning operation a sufficient number of times using the BP method.

Next, the operation in the first embodiment will be described.
First, the overall flow of image processing executed in the image processing system 100 or 200 will be described with reference to the flowchart of FIG.

  First, image information input to the image information acquisition unit 4 (hereinafter referred to as input image information) is acquired (step S1). Next, in the image determination unit 5, a skin color region (skin color hue region) is extracted from the input image information acquired in step S1 (step S2).

  The skin color region extraction method is not limited, but using color values such as lightness, saturation, and hue value, a region in which a specific color in a predetermined range continues is used as a skin color region, or an edge is extracted from an image. For example, a method for extracting a specific shape pattern such as an ellipse, a method using pattern matching or a neural network, a method combining these, or the like can be used.

  For example, the original image disclosed in Japanese Patent Laid-Open No. 4-346332 is divided into a large number of pixels, a histogram of hue value and saturation value is created from the BGR value of each pixel, and the histogram is divided from the shape. A method of dividing an image into regions composed of pixels corresponding to each part and estimating a region corresponding to a face from a plurality of divided regions, Japanese Patent Laid-Open Nos. 6-309433 and 6-67320 A skin color area is determined based on a histogram of hue value, saturation value, luminance value, etc. disclosed in Japanese Patent Laid-Open No. 5-158164 and Japanese Patent Laid-Open No. 5-165120, and this skin color area is defined as a face area. The method disclosed in JP-A-8-184925 is subjected to processing such as binarization to divide the image into a plurality of regions, and the region having the highest probability of being a region corresponding to a face is determined from the plurality of regions. Line process disclosed in JP-A-8-221567, JP-A-2000-20694, JP-A-2000-32272, JP-A-2000-201158, JP-A-2000-20769 The method using the method is mentioned.

  Also, edges and low-frequency images are generated from color digital images, and initial pixels whose predetermined parameters (for example, saturation, hue, and brightness values) match predetermined initial conditions (values in each predetermined range) are extracted. It is also possible to perform simple region expansion from the initial pixel extracted for the low-frequency image, and forcibly end simple region expansion when the image edge is reached, thereby performing region extraction. Also, logarithmic polar coordinate conversion is performed on the image centered on the pixels of the human eye area, and simple area expansion is performed using the image after logarithmic polar coordinate conversion, or a face area is extracted from the image after logarithmic polar coordinate conversion It is also possible to perform region extraction by performing pattern matching using a template for the purpose.

  When the skin color area is extracted, various characteristic values such as [area area / peripheral length] representing the form of the skin color area, [shortest diameter / longest diameter], and dispersion of pixel signal values are calculated. Then, using each characteristic value as an input signal, the extracted skin color is obtained by using a hierarchical neural network in which human face candidate region extraction processing is learned (hereinafter, the hierarchical neural network is abbreviated as “neuro”). A process of determining whether or not the area is a face candidate area (hereinafter referred to as “neuro determination process”) is performed (step S3). As shown in FIG. 6, a face candidate region is extracted using a neuron having each characteristic value as an input signal (input parameter) and a face reliability parameter as an output signal. The face reliability parameter indicates the likelihood of a face with a numerical value from 0 to 1, and the closer to 1, the higher the possibility that the face is. When the face reliability parameter is equal to or greater than a predetermined threshold, the face candidate area is extracted.

  Next, the input image information acquired in step S1 is displayed on the display unit 10, and the extracted face candidate area is highlighted on the image so as to be clearly distinguishable from other areas (step). S4). As a method for emphasizing and displaying, a method of clearly specifying the target region by surrounding the region to be emphasized with a line or changing the color in the region is preferable. For example, in the correction target image 21 obtained by photographing the person shown in FIG. 7A, the face candidate regions 22a, 22b, and 22c extracted by the image determination unit 5 are surrounded by lines as shown in FIG. 7B. Displayed. Here, the face candidate areas 22a and 22b are correctly determined, but the face candidate area 22c is erroneously determined that the pattern portion of the kimono is the face candidate area. In FIG. 7, the face candidate area is displayed by being surrounded by a line along the shape of the face candidate area extracted in step S <b> 3. However, the face candidate area is an area that can be extracted according to the face determination method. It is sufficient if the display is sufficient to identify the rectangle (rectangle, ellipse, perfect circle, etc.).

  Next, the operator determines whether or not the face candidate areas 22 a, 22 b, and 22 c on the correction target image 21 displayed on the display unit 10 are face areas, and via the instruction input unit 9, the face candidate area A face area is selected from 22a, 22b, and 22c (step S5). Specifically, the face area is selected by moving the mouse pointer to a face candidate area determined to be a face area by a mouse operation of the instruction input unit 9 and clicking. It is indicated that the selected face area is selected by changing the color in the area. FIG. 7C shows an example in which the face candidate areas 22a and 22b are determined to be face areas and are selected. Here, the face candidate area determined to be a face area is selected by a mouse operation. However, if there is no face candidate area extracted by mistake, the return key is pressed. Alternatively, a method may be used in which the mouse pointer is moved to what is determined to be a face candidate area that the operator has determined is not a face area, and clicked to remove it from the selection.

  Next, a predetermined correction process is performed on at least the selected face area by the image processing unit 8, and a corrected image is generated (step S6). Here, not only image processing suitable for the face area is performed, but also image processing is performed so that other areas do not feel uncomfortable. Specifically, the density correction processing is performed on the entire image so that the density of the face area becomes the target density, and the color correction processing is performed on the entire image so that the color of the face area becomes the target color. Apply correction processing only to the face area so that the density and color of the face will be the target density and color, enlarge or reduce the entire image so that the face area has the appropriate size, and make the face area the correct position Various correction processes such as position correction of the entire image, setting of the default value of the number of prints to the number of extracted faces, or a combination thereof can be performed.

  Each time image information is input, the processing of steps S1 to S6 is performed, and the selection result of the face area selected in step S5 is accumulated in the teacher data storage unit 6 as teacher data (step S7). The teacher data is a pair of a desired value of the neuro output (that is, 1 if it is a face area, 0 if it is not a face area) and a corresponding input signal group (that is, each characteristic value of the skin color area). .

  When the number of teacher data stored in the teacher data storage unit 6 exceeds a predetermined number, neuro learning is performed based on these teacher data (step S8). Here, “learning” refers to calculating each coupling strength in the neuro based on teacher data. The learning in step S8 may be performed periodically.

  When the learning result in step S8 satisfies a predetermined condition, the learning result is reflected in the neuron. Reflecting means replacing with a neuro that uses each new connection strength obtained by learning the neuro.

In the captured image, since a large image of a person is often important, when learning a neuro, depending on the size of the face candidate area to be determined whether or not it is a face, It is preferable to carry out learning.
FIG. 8 is a flowchart showing a neuro-learning process according to the size of the face candidate area.

First, input parameters (1 to n) are acquired (step S11).
Next, a neuro to be used for the face candidate area extraction process is selected according to the size of the face candidate area (step S12).

  When the size of the face candidate area is large, a set of input parameters and correct answer values is added to the teacher data for the large face candidate area, and after a predetermined number of teacher data is accumulated, neuro A learning is performed. (Step S13). When the size of the face candidate area is medium, the set of input parameters and correct values is added to the teacher data for the medium face candidate area, and after a predetermined number of teacher data is accumulated, Learning is performed (step S14). When the size of the face candidate area is small, a set of input parameters and correct values is added to the teacher data for the small face candidate area. After a predetermined number of teacher data is accumulated, neuro C learning is performed. (Step S15).

  Learning of each neuro may be performed when a predetermined number of images are processed and a predetermined number of teacher data is accumulated. In this case, the extraction accuracy is particularly improved for the size of the face candidate region having a high appearance frequency, and the extraction accuracy can be improved for the face candidate region having a high importance.

  Further, when a predetermined number of teacher data for large, medium, and small face candidate areas are accumulated (for example, each of large, medium, and small 500 images or more), learning of each neuro may be performed. Good. In this case, the extraction accuracy is ensured even for the size of the face candidate region having a low appearance frequency, and the accuracy of the face candidate region extraction process can be improved for the face candidate region of any size. In addition, in order to ensure the extraction accuracy for the size of the face candidate region with high learning difficulty and the extraction accuracy for the size of the face candidate region with high importance, the teacher accumulated until learning of each neuro is performed It is also possible to change the number of data for each face candidate area size.

Further, when performing neuro learning, it is preferable to perform learning in accordance with the light distribution condition when the image is taken. It is preferable that the light distribution conditions at the time of photographing include forward light, backlight, strobe (flash light use state), under, and the like.
FIG. 9 is a flowchart showing a neuro-learning process according to the light distribution condition.

First, input parameters (1 to n) are acquired (step S21).
Next, a neuro used for the face candidate area extraction process is selected according to the light distribution condition (step S22). The light distribution condition determination process will be described later with reference to FIG.

  When the shooting scene is a front light scene, a set of input parameters and correct answer values is added to the front light teacher data, and after a predetermined number of teacher data has been accumulated, neuro D learning is performed (step S23). ). When the shooting scene is a strobe scene, a set of input parameters and correct answer values is added to the strobe teacher data, and after a predetermined number of teacher data has been accumulated, neuro E learning is performed (step S24). When the shooting scene is a backlight scene, a set of input parameters and correct answer values is added to the backlight teacher data, and after a predetermined number of teacher data has been accumulated, neuro F learning is performed (step S25). When the shooting scene is an underscene, a set of input parameters and correct values is added to the under teacher data, and after a predetermined number of teacher data is accumulated, neuro G learning is performed (step S26).

  Learning of each neuro may be performed when a predetermined number of images are processed and a predetermined number of teacher data is accumulated. In this case, the extraction accuracy is particularly improved for light distribution conditions (photographing scenes) having a high appearance frequency, and the extraction accuracy can be improved for light distribution conditions having a high importance.

Further, learning of each neuro may be performed when a predetermined number of teacher data for each light distribution condition is accumulated. In this case, extraction accuracy is ensured even for light distribution conditions with low appearance frequency, and the accuracy of face candidate region extraction processing can be improved for all light distribution conditions. Also, for the purpose of ensuring the extraction accuracy for light distribution conditions with a high degree of learning difficulty and the extraction accuracy for light distribution conditions with a high degree of importance, the number of teacher data accumulated until each neuro learning is carried out is distributed. It is also possible to change for each condition.
In addition, it is preferable to classify according to the photographing light source (sunlight, strobe light, fluorescent light, tungsten light).

Here, the light distribution condition determination processing will be described with reference to FIG.
First, image data is divided into predetermined image areas, and an occupancy ratio calculation process is performed to calculate an occupancy ratio (first occupancy ratio, second occupancy ratio) indicating the ratio of each divided area to the entire image data. (Step S31). Details of the occupation rate calculation process will be described later with reference to FIGS.

  Next, the occupancy ratio (first occupancy ratio, second occupancy ratio) calculated in step S31, at least the average brightness value of the skin color in the screen center portion of the image data, and the shooting conditions are set in advance. Based on the coefficients, indexes (indexes 1 to 6) for specifying light distribution conditions (quantitatively representing light source conditions and exposure conditions) are calculated (step S32). The calculation method of each index in step S32 will be described in detail later.

  Next, the light distribution condition of the image data is determined based on the index calculated in step S32 (step S33), and the main light distribution condition determination process ends. A method for determining the light distribution condition based on each index will be described in detail later.

  Next, with reference to FIG. 11, the first occupation rate calculation process executed in step S31 of FIG. 10 will be described in detail.

  First, RGB (Red, Green, Blue) values of image data are converted into the HSV color system (step S41). Next, the image data is divided into regions composed of a combination of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S42).

  Lightness (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is found that the contribution to the discrimination of the shooting scene is small. The flesh color hue area is further divided into a flesh color area (H1) and other areas (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following equation (10) is defined as the flesh-colored region (H1), and the region that does not satisfy the equation (10) is ( H2). However, the values of the input image data are InR, InG, and InB.

10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance (Y) = InR x 0.30 + InG x 0.59 + InB x 0.11 (9)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) + 0.7 (10)

  Therefore, the number of divided areas of the image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the expressions (9) and (10).

  When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S43). Let Rij be the first occupancy calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj.

Next, a method for calculating the index 1 and the index 2 will be described.
Table 1 shows, for each divided area, the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting, that is, the brightness state of the face area at the time of strobe shooting. The coefficient of each divided area shown in Table 1 is a weighting coefficient by which the first occupation ratio Rij of each divided area is multiplied, and is set in advance according to the shooting conditions.

When the first coefficient in the lightness region vi and the hue region Hj is Cij, the index 1 is defined as in Expression (11).

Table 2 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy as backlight photographing, that is, the brightness state of the face region at the time of backlight photographing. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy rate Rij of each divided area is multiplied, and is set in advance according to the shooting conditions.

When the second coefficient in the lightness area vi and the hue area Hj is Dij, the index 2 is defined as in Expression (12).

  Next, the second occupancy rate calculation process executed to calculate the index 3 will be described in detail with reference to FIG.

  First, the RGB values of the image data are converted into the HSV color system (step S51). Next, the image data is divided into regions each consisting of a combination of the distance from the outer edge of the captured image screen and the brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S52).

  FIGS. 13A to 13D show four regions n1 to n4 divided according to the distance from the outer edge of the screen of the image data. A region n1 illustrated in FIG. 13A is an outer frame, a region n2 illustrated in FIG. 13B is a region inside the outer frame, and a region n3 illustrated in FIG. 13C is further added to the region n2. A region n4 shown in FIG. 13D is an inner region and is a central region of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the image data is divided into regions each having a combination of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

  When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S53). The second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is defined as Qij.

Next, a method for calculating the index 3 will be described.
Table 3 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the second occupancy rate Qij of each divided area is multiplied, and is set in advance according to the shooting conditions.

When the third coefficient in the brightness area vi and the screen area nj is Eij, the index 3 is defined as in Expression (13).

Next, a method for calculating the index 4 will be described.
First, the luminance Y is calculated from the RGB (Red, Green, Blue) values of the image data using Equation (9). Also, the average luminance value x1 of the skin color area in the center of the screen of the image data is calculated. Here, the screen center portion is, for example, a region constituted by a region n3 and a region n4 in FIG. Next, a difference value x2 between the maximum luminance value and the average luminance value of the image data is calculated.

Next, the standard deviation x3 of the luminance of the image data is calculated, and the average luminance value x4 at the center of the screen is calculated. Also, a comparison value x5 between the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the image data and the average luminance value Yskin_ave of the skin color area is calculated. This comparison value x5 is expressed as the following equation (14).
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (14)

Next, the index 4 is calculated by multiplying each of the calculated values x1 to x5 by a fourth coefficient set in advance according to the photographing condition and taking the sum. The index 4 is defined as the following formula (15).
Index 4 = 0.06 × x1 + 1.13 × x2 + 0.02 × x3 + (− 0.01) × x4 + 0.03 × x5−6.50 (15)

  Further, the average luminance value of the skin color area in the center of the screen of the image data is set as an index 4 '. Here, the center of the screen is, for example, an area composed of the areas n2, n3, and n4 in FIG.

In addition, the index 5 is defined as in Expression (16) using the index 1, index 3, and index 4 ′, and the index 6 is defined as in Expression (17) using the index 2, index 3, and index 4 ′. Defined in
Index 5 = 0.46 × index 1 + 0.61 × index 3 + 0.01 × index 4′−0.79 (16)
Index 6 = 0.58 x Index 2 + 0.18 x Index 3 + (-0.03) x Index 4 '+ 3.34 (17)
Here, the weighting coefficient by which each index is multiplied in Expression (16) and Expression (17) is set in advance according to the shooting conditions.

Next, a method for determining light distribution conditions in step S33 of FIG. 10 will be described.
In FIG. 14A, 60 images are taken under each light source condition of forward light, backlight, and strobe, and index 5 and index 6 are calculated for a total of 180 digital image data. The values of index 6 are plotted. According to FIG. 14A, when the value of the index 5 is larger than 0.5, there are many flash shooting scenes, the value of the index 5 is 0.5 or less, and the value of the index 6 is larger than −0.5. When there are many backlight scenes, the value of the index 5 is 0.5 or less, and the value of the index 6 is −0.5 or less, it is understood that there are many backlight scenes. In this way, the shooting scene can be determined quantitatively based on the values of the index 5 and the index 6.

  FIG. 14B is a graph in which index 4 and index 5 of an image with index 5 larger than 0.5 are calculated and plotted among 60 captured images of the strobe shooting scene and the under shooting scene. According to FIG. 14B, it can be seen that when the value of the index 4 is greater than 0, there are many flash shooting scenes, and when the value of the index 4 is 0 or less, there are many under shooting scenes.

  Next, with reference to the flowcharts of FIGS. 15 and 16, a neuro replacement process for replacing the current neuro (hereinafter referred to as an old neuro) with a post-learning neuro (hereinafter referred to as a new neuro) will be described. In this embodiment, two types of neuro replacement processing will be described.

First, the neuro replacement process 1 executed in the neuro learning unit 7 will be described with reference to the flowchart of FIG.
First, a new neuro is acquired (step S61), and an old neuro is acquired (step S62). Next, determination image information used to determine the output results of the new neuron and the old neuron is acquired (step S63).

  Next, with respect to the image information for determination acquired in step S63, a determination result as to whether or not the skin color region is a face candidate region, which is performed on each skin color region using the new neuro, is acquired. The determination result is similarly acquired using the neuro, and the difference information between the face determination results output from the new neuro and the old neuro is acquired (step S64). The determination result of whether or not the skin color area is a face candidate area is the above-described neuro output value, and is indicated by a continuous value of 0 to 1.

  Next, an average value of difference information for a predetermined number of past scenes is calculated (step S65), and the average value calculated up to the previous time is compared with the average value calculated this time (step S66). It is determined whether or not the amount of change has converged (step S67).

  In step S67, when it is determined that the change amount of the average value of the difference information is not converged (step S67; NO), the teacher data using the old neuro face determination result as the desired value of the output signal of the new neuro is used. Thus, the new neuro learning is performed (step S68). Then, a new neuro that is finely corrected by the execution of learning in step S68 is set (step S69), and the process returns to step S63, and the processes of steps S63 to S67 are repeated for the set new neuro.

  In step S67, when it is determined that the amount of change in the average value of the difference information has converged (step S67; YES), the old neuro is replaced with the new neuro (step S70), and the present neuro replacement process 1 ends.

Next, the neuro replacement process 2 executed in the neuro learning unit 7 will be described with reference to the flowchart of FIG.
First, a new neuro is acquired (step S71), and an old neuro is acquired (step S72). Next, determination image information used to determine the output results of the new neuron and the old neuron is acquired (step S73).

  Next, for the determination image information acquired in step S73, a face determination result is acquired using the new neuron, and a face determination result is acquired using the old neuron (step S74).

  Next, based on the old neuro face determination result, an image in which the face candidate area is displayed on the captured image is displayed on the display unit 10 (step S75).

  Next, when the operator observes the face candidate area display image displayed on the display unit 10 and determines that correction is necessary to determine whether or not it is a face area, the operator corrects it by operating the instruction input unit 9. The determination (correct value) of the face determination is input, and the correct value determined by the operator is acquired (step S76).

Next, a difference value (first difference value) between the correct answer value and the old neuro face determination result is calculated (step S77).
(First difference value) = (correct value) − (old neuro face determination result)

Also, a difference value (second difference value) between the correct answer value and the new neuro face determination result is calculated (step S78).
(Second difference value) = (correct value) − (new neuro face determination result)

  Next, an average value of the first difference value and the second difference value of the past predetermined number of scenes is calculated, and it is determined whether or not the average value of the first difference value is larger than the average value of the second difference value (step S79). ).

  In step S79, when it is determined that the average value of the first difference value is equal to or less than the average value of the second difference value (step S79; NO), the teacher data having the correct value as a desired value of the output signal of the new neuron is obtained. In this way, new neuro learning is performed (step S80). Then, a new neuro that is finely corrected by the execution of learning in step S80 is set (step S81), and the process returns to step S73, and the processes in steps S73 to S79 are repeated for the set new neuro.

  If it is determined in step S79 that the average value of the first correction value is greater than the average value of the second correction value (step S79; YES), the old neuron is replaced with the new neuron (step S82), and the current neuron The replacement process 2 ends.

  In the neuro replacement process 2, automatic processing can be performed by omitting steps S75 and S76 and using values prepared in advance as correct values used in steps S77 and S78.

  Note that a process obtained by connecting the neuro replacement process 1 in FIG. 15 and the neuro replacement process 2 in FIG. 16 may be executed. In this case, when the change amount of the average value of the difference information is converged in step S67 in FIG. 15 (step S67; YES), the process proceeds to the neuro replacement process 2 in FIG. A new neuron learned in the replacement process 1 is acquired. As described above, when the neuro replacement process 1 and the neuro replacement process 2 are combined, the neuro replacement process 1 completes a large amount of learning at a high speed. In the neuro replacement process 2, a small amount of more advanced learning is performed. Therefore, advanced learning at the time of neuro replacement can be performed in a short time.

  Further, when the change amount of the average value of the difference information does not converge in step S67 of the neuro replacement process 1 in FIG. 15 (step S67; NO), or in step S79 of the neuro replacement process 2 in FIG. When it is determined that the average value is equal to or less than the average value of the second difference values (step S79; NO), the process of FIG. 5 may be further advanced without replacing the old neuron with the new neuron.

  Further, the neuro replacement process 1 in FIG. 15 or the neuro replacement process 2 in FIG. 16 may be performed while the operator is running. However, since the process may become heavy, the neuro replacement process may be performed collectively at night or the like. It is good.

  As described above, according to the image processing systems 100 and 200 of the first embodiment, neuro learning is performed based on the face region selection result by the operator, and the learned neuro learning result is a predetermined value. When the condition is satisfied, the learning result is reflected in the neuro, so that the accuracy of the face candidate area extraction process is improved by automatically optimizing the extraction condition of the face candidate area extraction process according to the running environment. be able to.

[Second Embodiment]
Next, a second embodiment to which the present invention is applied will be described.
In the image processing system shown in the second embodiment, the same components as those in the image processing systems 100 and 200 shown in the first embodiment are denoted by the same reference numerals, and the configuration is shown and described. Omitted. Hereinafter, a configuration and processing characteristic of the second embodiment will be described.

  The image determination unit 5 calculates various characteristic values from the image information acquired from the image information acquisition unit 4, uses the calculated characteristic value as an input signal, and uses a first neuron as a value indicating the facialness. Is acquired from the neuro-learning unit 7, and a face candidate region is extracted using the first neuro. After the first neuron is replaced with the second neuron having a configuration different from that of the first neuron, the image determination unit 5 acquires the second neuron from the neurolearning unit 7, and the second neuron is obtained. To extract a face candidate area. Here, the different configuration includes a case where the number of intermediate layers is different and a case where the number and types of input parameters are different.

  The neuro-learning unit 7 learns the second neuron having a configuration different from that of the first neuron, using the information on whether or not the face region is stored in the teacher-data storage unit 6 as teacher data. Further, the neuro-learning unit 7 executes a neuro replacement process for replacing the first neuro with the second neuro.

Next, the operation in the second embodiment will be described.
First, the flow of the entire image processing executed in the image processing system according to the second embodiment will be described with reference to the flowchart of FIG.

  First, the image information input to the image information acquisition unit 4 is acquired (step S91). Next, in the image determination unit 5, a skin color region is extracted from the input image information acquired in step S91 (step S92).

  When the skin color area is extracted, various characteristic values of the skin color area are calculated. Then, by using each characteristic value as an input signal and using the first neuron learned from the human face candidate area extraction process, it is determined whether or not the extracted skin color area is a face candidate area. A neuro determination process is performed (step S93).

  Next, the input image information acquired in step S91 is displayed on the display unit 10, and the extracted face candidate region is highlighted on the image so as to be clearly distinguishable from other regions (step). S94).

  Next, the operator determines whether the face candidate area on the correction target image displayed on the display unit 10 is a face area, and selects the face area from the face candidate area via the instruction input unit 9. (Step S95).

  Next, a predetermined correction process is performed on at least the selected face area by the image processing unit 8, and a corrected image is generated (step S96).

  Each time image information is input, the processing of steps S91 to S96 is performed, and the selection result of the face area selected in step S95 is accumulated in the teacher data storage unit 6 as teacher data (step S97).

  When the number of teacher data accumulated in the teacher data storage unit 6 exceeds a predetermined number, learning of the second neuro is performed based on these teacher data (step S98).

If the learning result in step S98 satisfies a predetermined condition, the first neuron can be replaced with the second neuron.
The process of replacing the first neuron with the second neuron is the same process as the process shown in FIGS. 15 and 16 and will not be described. However, in FIG. 15 and FIG. 16, “old neuro” is read as “first neuro” and “new neuro” is read as “second neuro”.

  As described above, according to the image processing system of the second embodiment, the operator selects a face area from the face candidate areas extracted by the face candidate area extraction process using the first neuron. Then, when the learning of the second neuro is performed and the learning result of the learned second neuro satisfies the predetermined condition, the first neuro is replaced with the second neuro, so depending on the running environment The accuracy of the face candidate area extraction process can be improved by automatically optimizing the extraction conditions of the face candidate area extraction process using the second neuron.

  In the second embodiment, the case where the first neuron is replaced with a second neuron different from the first neuron has been described. However, in the case where the face candidate region extraction process uses a template matching method, a template is used. And the configuration of the face candidate area extraction process may be changed by changing the size of the template or changing the configuration of the template (for example, changing the template of the eyes and mouth to the eyes, mouth, and eyebrows).

[Third Embodiment]
Next, a third embodiment to which the present invention is applied will be described.
In the image processing system shown in the third embodiment, the same components as those in the image processing systems 100 and 200 shown in the first embodiment are denoted by the same reference numerals, and the configuration is shown and described. Omitted. Hereinafter, a configuration and processing characteristic of the third embodiment will be described.

  The image determination unit 5 extracts a face candidate region from the image information acquired by the image information acquisition unit 4 by a predetermined region extraction process. After the neuro determination process is introduced, the image determination unit 5 acquires a neuro from the neuro learning unit 7 and extracts a face candidate region using the neuro.

  The neuro-learning unit 7 performs neuro-learning using information on whether or not the face area is stored in the teacher data storage unit 6 as teacher data. Further, the neuro-learning unit 7 introduces a neuro-determination process using a neuro that reflects the learning result.

Next, the operation in the third embodiment will be described.
First, the flow of the entire image processing executed in the image processing system according to the third embodiment will be described with reference to the flowchart of FIG.

  First, the image information input to the image information acquisition unit 4 is acquired (step S101). Next, in the image determination unit 5, a skin color area is extracted from the input image information acquired in step S101 (step S102).

  When the skin color area is extracted, various characteristic values of the skin color area are calculated. Then, a human face candidate area extraction process is performed based on each characteristic value by a predetermined area extraction process (step S103).

  Next, the input image information acquired in step S101 is displayed on the display unit 10, and the extracted face candidate region is highlighted and displayed on the image so that it can be clearly distinguished from other regions (step). S104).

  Next, the operator determines whether the face candidate area on the correction target image displayed on the display unit 10 is a face area, and selects the face area from the face candidate area via the instruction input unit 9. (Step S105).

  Next, a predetermined correction process is performed on at least the selected face area by the image processing unit 8, and a corrected image is generated (step S106).

  Each time image information is input, the processing of step S101 to step S106 is performed, and the selection result of the face area selected in step S105 is accumulated in the teacher data storage unit 6 as teacher data (step S107).

  When the number of teacher data stored in the teacher data storage unit 6 exceeds a predetermined number, neuro learning is performed based on these teacher data (step S108).

When the learning result in step S108 satisfies a predetermined condition, a neuro candidate face extraction process (neuro determination process) reflecting the learning result is introduced into the face candidate area extraction process in step S103.
The process of introducing the neuro-determination process reflecting the learning result is the same as the process shown in FIG. 15 and FIG. 16, but in this process there is nothing equivalent to the old neuro, so the face of the old neuro The determination result may be executed as a preset constant.

  As described above, according to the image processing system shown in the third embodiment, neuro learning is performed based on the result of the operator selecting a face area from the face candidate areas extracted by the face candidate area extraction process. When a learned neurolearning result is implemented and meets a predetermined condition, a face candidate region extraction process using the neuron reflecting the learning result is introduced, so the extraction of the face candidate region extraction process is performed according to the running environment By automatically optimizing the conditions, the accuracy of the face candidate area extraction process can be improved.

[Fourth Embodiment]
Next, a fourth embodiment to which the present invention is applied will be described.
In the image processing system shown in the fourth embodiment, the same components as those in the image processing systems 100 and 200 shown in the first embodiment are denoted by the same reference numerals, and the configuration is shown and described. Omitted. Hereinafter, a configuration and processing characteristic of the fourth embodiment will be described.

  The image determination unit 5 calculates various characteristic values from the image information acquired from the image information acquisition unit 4, uses the calculated characteristic value as an input signal, and a neuro (hereinafter referred to as “output signal”) indicating a face-like value. (Referred to as face determination neuron) from the neurolearning unit 7, and a face candidate region is extracted using the face determination neuron. The image determination unit 5 uses the characteristic value calculated from the image information and the determination result of the face determination neuron as an input signal, and a neuron (hereinafter referred to as a neuron that uses a tone correction parameter for tone conversion of the image information as an output signal). , Referred to as a correction neuron) is acquired from the neurolearning unit 7, and a tone correction parameter is calculated using the correction neuron.

  The neuro-learning unit 7 performs face-determination neuro-learning using information on whether or not the face area is stored in the teacher data storage unit 6 as teacher data. Further, the neuro-learning unit 7 learns correction neuron using the additional correction value (correction amount) stored in the teacher data storage unit 6 as teacher data.

Next, the operation in the fourth embodiment will be described.
First, an overall flow of image processing executed in the image processing system according to the fourth embodiment will be described with reference to the flowchart of FIG.

  First, the image information input to the image information acquisition unit 4 is acquired (step S111). Next, in the image determination unit 5, a skin color region is extracted from the input image information acquired in step S111 (step S112).

  When the skin color area is extracted, various characteristic values of the skin color area are calculated. Then, by using the face determination neuron that learned the human face candidate area extraction process using each characteristic value as an input signal, a neuro determination process for determining whether or not the extracted skin color area is a face candidate area. Performed (step S113).

  Next, the input image information acquired in step S111 is displayed on the display unit 10, and the extracted face candidate area is displayed on the image (step S114).

  Next, the operator determines whether the face candidate area on the correction target image displayed on the display unit 10 is a face area, and selects the face area from the face candidate area via the instruction input unit 9. (Step S115).

  Each time image information is input, the processes of steps S111 to S115 are performed, and the selection result of the face area selected in step S115 is accumulated as teacher data in the teacher data storage unit 6 (step S116).

  When the number of teacher data stored in the teacher data storage unit 6 exceeds a predetermined number, learning of the face determination neuron is performed based on these teacher data (step S117).

  When the learning result in step S117 satisfies a predetermined condition, the learning result is reflected in the face determination neuron.

  After step S115, the tone correction parameter is calculated using the correction value using the characteristic value calculated from the image information and the determination result in the neuro determination process of step S113 as the input signal (input parameter) and the tone correction parameter as the output signal. Is calculated (step S118). The tone correction parameter indicates the correction strength in the tone correction process. Input parameters include image size, resolution, various statistical values, and the like, but preferably include face extraction results (automatic extraction or operator selection results).

  FIG. 20 shows a case where the gradation correction parameters that are output signals of the correction neuron are the brightness correction parameter and the contrast correction parameter. In the case of the image processing system 100 shown in FIG. 1, the opening / closing time of the shutter is controlled by the brightness correction parameter calculated by the correction neuron. In the case of the image processing system 200 shown in FIG. 2, the signal value conversion characteristics are controlled based on the brightness correction parameter and / or the contrast correction parameter calculated by the correction neuron, and the gradation correction of the image information is performed. Applied.

  Note that the gradation correction parameter calculated by the correction neuron may be either a brightness correction parameter, a contrast correction parameter, or another gradation correction parameter, and according to the number of gradation correction parameters to be output, The number of neurons in the output layer is adjusted.

  Next, a correction value for the input image information is set based on the gradation correction parameter calculated in step S118 (step S119), and the input image information is corrected based on the set correction value, thereby performing a correction process. A corrected image is created (step S120).

  Next, the corrected image created in step S120 is displayed on the display unit 10 (step S121). The operator observes the correction image displayed on the display unit 10, visually determines whether or not the correction result is good, and inputs the determination result through the instruction input unit 9.

  If a determination result indicating that the correction result of the corrected image by visual determination is not good is input (step S122; NO), and if there is additional correction (step S123; YES), additional correction is performed by operating the instruction input unit 9 by the operator The value is input, the process returns to step S119, the input additional correction value is set (step S119), and the processes of steps S120 to S123 are repeated.

  When a determination result indicating that the correction result of the corrected image by visual determination is good is input (step S122; YES), or when there is no additional correction in step S123 (step S123; NO), the main image processing is performed. finish.

  Further, the addition value of the gradation correction parameter and the additional correction value is accumulated as teacher data in the teacher data storage unit 6 (step S124). When the number of accumulated teacher data exceeds a predetermined number, learning of the correction neuron is performed based on these teacher data (step S125), and the process of step S118 is performed using the learned correction neuron. Is called. The learning in step S125 may be performed periodically.

  As described above, according to the image processing system of the fourth embodiment, learning of the face determination neuro is performed based on the selection result of the face area by the operator, and the learning result of the learned face determination neuro is predetermined. When the above condition is satisfied, the learning result is reflected in the face determination neuron, so that the accuracy of the face candidate area extraction process can be improved.

  Further, since the additional correction value instructed by the operator is learned as teacher data of the correction neuron, and the image information is corrected using the learned correction neuron, the correction intensity calculation process (steps) is performed according to the running environment. The accuracy of the tone correction parameter calculation process) can be improved, and an image suitable for the operator's preference can be obtained.

  In addition, for the face determination neuron and the correction neuron, it is possible to acquire the teacher data by one operation and individually improve the learning performance.

  In the fourth embodiment, the case where the first half of the process shown in FIG. 19 is the same as the process shown in FIG. 5 of the first embodiment has been described. However, the first half of the process shown in FIG. 19 is performed. Instead, the process shown in FIG. 17 of the second embodiment or the process shown in FIG. 18 of the third embodiment may be performed.

  In the fourth embodiment, as an example of correction processing, the case of performing gradation correction processing for controlling the brightness and contrast of an image has been described. However, color correction processing, noise removal processing, sharpness correction processing, face correction processing, and the like are described. Enlargement / reduction processing, trimming processing, or the like for making the size and face position appropriate may be performed.

  The descriptions in the above embodiments are examples of the image processing system according to the present invention, and the present invention is not limited to this. The detailed configuration and detailed operation of each part constituting the system can be changed as appropriate without departing from the spirit of the present invention.

  FIG. 21 shows a configuration of an image processing system 300 including a plurality of physically independent devices. As shown in FIG. 21, the image processing system 300 includes an image input device 30, an image processing device 31, an image output device 32, a system control device 33, and a communication control unit 15. In FIG. 21, the same components as those of the image processing apparatus 200 of FIG. 2 are denoted by the same reference numerals, and the description of the functions of the components having the same functions is omitted.

  The image input device 30 includes an image information acquisition unit 4, a DSC 12, a document scanner 13, and a recording media driver 14.

  The image processing device 31 includes an image determination unit 5, a teacher data storage unit 6, a neuro learning unit 7, an instruction input unit 9, a display unit 10, a reduced image creation unit 11, and an image processing unit 34.

  The image determination unit 5 calculates various characteristic values from the image information, acquires a face determination neuron using the calculated characteristic value as an input signal and a value indicating facialness as an output signal from the neuro learning unit 7, A candidate face area is extracted using the determination neuron. Further, the image determination unit 5 uses the characteristic value calculated from the image information and the face determination result as an input signal, and uses the neurolearning unit 7 as a correction neuron having a tone correction parameter for converting the tone of the image information as an output signal. And the tone correction parameter is calculated using the correction neuron.

  The image processing unit 34 includes information on whether or not the face candidate area extracted by the image determination unit 5 and the face region input by the instruction input unit 9, the tone correction parameter calculated by the image determination unit 5, and Based on the additional correction value input from the instruction input unit 9, a correction image is generated by correcting the correction target image information, and the generated correction image is output to the display unit 10.

  The image output device 32 includes an image information output unit 16, a printer unit 17, a recording medium writing unit 18, and an image processing unit 35. The image processing unit 35 is designated based on the gradation correction parameter calculated by the image determination unit 5 and / or the additional correction value input from the instruction input unit 9 according to the control signal input from the system control device 33. The tone conversion for output to the output destination is performed to create a corrected image, and the generated corrected image is output to the image information output unit 16.

  The system control device 33 comprehensively controls various processes in the communication control unit 15, the image input device 30, the image processing device 31, and the image output device 32.

  Note that a plurality of the image input device 30, the image processing device 31, and the image output device 32 may be arranged as necessary. For example, when the processing capability of the image input device 30 is inferior to the image processing capability of the entire system, a plurality of image input devices 30 can be arranged and operated in parallel under the control of the system control device 33.

  In each of the above embodiments, the face candidate area extraction process using a neural network is used as the face candidate area extraction process in which the extraction condition is determined by learning. The neural network uses the characteristics of data to be determined. This is one of the typical methods of reflecting the determination conditions by learning, which is preferable. As the neural network, the hierarchical neural network described in each of the above embodiments, the non-hierarchical neural network (competitive learning type vector quantization neural network, etc., an example of competitive learning is Kohonen's self-organization, etc.), etc. There are various methods that can be applied to the present invention.

  In addition, various methods known in the art can be used as face candidate region extraction processing in which extraction conditions are determined by learning. For example, it is also possible to apply to the present invention a method for determining whether a feature parameter of an extracted region (skin color region) is a face based on a predetermined threshold value. Specifically, face and non-face input data of a certain parameter are individually made into appearance frequency histograms, threshold values are determined using a statistical method such as a discrimination criterion method, and the correct answer rate when using the threshold values is determined. Update if you are smarter. The parameters may be data such as those used in neuro input parameters, or may be statistical values such as Mahalanobis distances that are secondarily calculated from data such as those used in input parameters. In addition, statistical methods of multivariate analysis such as discriminant analysis and principal component analysis can be applied.

It is a block diagram which shows an example of a structure of the image processing system in the 1st Embodiment of this invention. It is a block diagram which shows the other structural example of the image processing system in the 1st Embodiment of this invention. It is a figure which shows typically the structure of a hierarchical neural network. It is a figure which shows the structure of the neuron which comprises a neural network. It is a flowchart which shows the image processing performed in the image processing system of 1st Embodiment. It is a figure which shows the hierarchical neural network applied in 1st Embodiment. It is an example of a display of the face candidate area | region displayed on a correction object image. It is a flowchart which shows the neuro learning process according to the magnitude | size of a face candidate area | region. It is a flowchart which shows the neurolearning process according to light distribution conditions. It is a flowchart which shows a light distribution condition discrimination | determination process. It is a flowchart which shows a 1st occupation rate calculation process. It is a flowchart which shows a 2nd occupation rate calculation process. It is a figure which shows the area | region n1-n4 determined according to the distance from the outer edge of the screen of image data. It is a plot figure which shows the relationship between an imaging | photography scene (forward light, strobe, backlight, under) and the indices 4-6. It is a flowchart which shows the neuro replacement process 1. FIG. It is a flowchart which shows the neuro replacement process 2. FIG. It is a flowchart which shows the image processing performed in the image processing system of 2nd Embodiment. It is a flowchart which shows the image processing performed in the image processing system of 3rd Embodiment. It is a flowchart which shows the image processing performed in the image processing system of 4th Embodiment. It is a figure which shows the hierarchical neural network which calculates the gradation correction parameter applied in 4th Embodiment. It is a figure which shows the modification of the image processing system of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Light source part 2 Imaging part 3 Exposure control part 4 Image information acquisition part 5 Image determination part 6 Teacher data storage part 7 Neuro learning part 8 Image processing part 9 Instruction input part 10 Display part 11 Reduced image creation part 12 DSC
13 Document Scanner 14 Recording Media Driver 15 Communication Control Unit 16 Image Information Output Unit 17 Printer Unit 18 Recording Media Writing Unit 21 Correction Target Images 22a, 22b, 22c Face Candidate Area 30 Image Input Device 31 Image Processing Device 32 Image Output Device 33 System Control device 34, 35 Image processing unit 100, 200, 300 Image processing system N Communication network

Claims (36)

Display means for displaying an image based on the acquired image information;
Face candidate region extraction means for extracting a face candidate region from the image information by face candidate region extraction processing in which an extraction condition is determined by learning;
Display control means for displaying the extracted face candidate area on an image displayed on the display means;
A face area selection means for an operator to select a face area from the displayed face candidate areas;
Image correction means for performing a predetermined correction process on at least the selected face area and generating a corrected image;
Learning means for performing learning of the face candidate area extraction process based on the selection result;
An image processing system comprising: The image processing system according to claim 1,
An image processing system comprising: a reflecting unit configured to reflect a learning result in the face candidate region extraction process when a learning result of the learned face candidate region extraction process satisfies a predetermined condition. The image processing system according to claim 1 or 2,
The face candidate area extraction process is a face candidate area extraction process using a neural network. Display means for displaying an image based on the acquired image information;
Face candidate region extraction means for extracting a face candidate region from the image information by a first face candidate region extraction process in which an extraction condition is determined by learning;
Display control means for displaying the extracted face candidate area on an image displayed on the display means;
A face area selection means for an operator to select a face area from the displayed face candidate areas;
Image correction means for performing a predetermined correction process on at least the selected face area and generating a corrected image;
Learning means for performing learning of a second face candidate area extraction process having an arrangement different from that of the first face candidate area extraction process based on the selection result, wherein an extraction condition is determined by learning;
An image processing system comprising: The image processing system according to claim 4,
When a learning result of the learned second face candidate area extraction process satisfies a predetermined condition, a replacement unit is provided that replaces the first face candidate area extraction process with the second face candidate area extraction process. An image processing system characterized by that. The image processing system according to claim 4 or 5,
The first face candidate area extraction process is a face candidate area extraction process using a first neural network,
2. The image processing system according to claim 1, wherein the second face candidate area extraction process is a face candidate area extraction process using a second neural network having a configuration different from that of the first neural network. Display means for displaying an image based on the acquired image information;
Face candidate area extraction means for extracting a face candidate area from the image information by a predetermined area extraction process;
Display control means for displaying the extracted face candidate area on an image displayed on the display means;
A face area selection means for an operator to select a face area from the displayed face candidate areas;
Image correction means for performing a predetermined correction process on at least the selected face area and generating a corrected image;
Learning means for performing learning of a face candidate region extraction process in which an extraction condition is determined by learning based on the selection result;
An image processing system comprising: The image processing system according to claim 7,
An image processing system comprising: an introduction unit that introduces a face candidate area extraction process that reflects the learning result when a learning result of the learned face candidate area extraction process satisfies a predetermined condition. The image processing system according to claim 7 or 8,
The face candidate area extraction process is a face candidate area extraction process using a neural network. In the image processing system according to any one of claims 1 to 9,
The image processing system according to claim 1, wherein the learning unit performs learning according to a size of a face candidate region that is a determination target of whether or not the face is a face. In the image processing system according to any one of claims 1 to 10,
The image processing system according to claim 1, wherein the learning unit performs learning according to a light distribution condition when the image is captured. Display means for displaying an image based on the acquired image information;
Face candidate region extraction means for extracting a face candidate region from the image information by face candidate region extraction processing in which an extraction condition is determined by learning;
Display control means for displaying the extracted face candidate area on an image displayed on the display means;
A face area selection means for an operator to select a face area from the displayed face candidate areas;
First learning means for performing learning of the face candidate region extraction processing based on the selection result;
A correction strength calculation means for calculating a correction strength by a correction strength calculation process in which a calculation condition is determined by learning;
Correction processing means for performing correction processing on the image information based on the calculated correction strength;
Correction intensity correction means for correcting the calculated correction intensity in accordance with an operator instruction;
Second learning means for performing learning of the correction strength calculation processing based on the correction result;
An image processing system comprising: A face candidate region extraction step of extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning;
A display step of displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection step for allowing an operator to select a face area from the displayed face candidate areas;
An image correction step of performing a predetermined correction process on at least the selected face area to generate a corrected image;
A learning step of performing learning of the face candidate area extraction process based on the selection result;
An image processing method comprising: The image processing method according to claim 13.
When the learning result of the learned face candidate area extraction process satisfies a predetermined condition, the learning result is reflected in the face candidate area extraction process. The image processing method according to claim 13 or 14,
The face candidate area extraction process is a face candidate area extraction process using a neural network. A face candidate region extraction step of extracting a face candidate region from image information by a first face candidate region extraction process in which an extraction condition is determined by learning;
A display step of displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection step for allowing an operator to select a face area from the displayed face candidate areas;
An image correction step of performing a predetermined correction process on at least the selected face area to generate a corrected image;
A learning step of performing learning of a second face candidate region extraction process having a configuration different from that of the first face candidate region extraction process based on the selection result, wherein an extraction condition is determined by learning;
An image processing method comprising: The image processing method according to claim 16.
When the learning result of the learned second face candidate area extraction process satisfies a predetermined condition, the first face candidate area extraction process is replaced with the second face candidate area extraction process. Image processing method. The image processing method according to claim 16 or 17,
The first face candidate area extraction process is a face candidate area extraction process using a first neural network,
2. The image processing method according to claim 1, wherein the second face candidate area extraction process is a face candidate area extraction process using a second neural network having a configuration different from that of the first neural network. A face candidate region extraction step of extracting a face candidate region from the image information by a predetermined face candidate region extraction process;
A display step of displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection step for allowing an operator to select a face area from the displayed face candidate areas;
An image correction step of performing a predetermined correction process on at least the selected face area to generate a corrected image;
A learning step of performing learning of a face candidate region extraction process in which an extraction condition is determined by learning based on the selection result;
An image processing method comprising: The image processing method according to claim 19, wherein
When the learning result of the learned face candidate area extraction process satisfies a predetermined condition, a face candidate area extraction process based on the face candidate area extraction process reflecting the learning result is introduced. The image processing method according to claim 19 or 20,
The face candidate area extraction process is a face candidate area extraction process using a neural network. The image processing method according to any one of claims 13 to 21,
In the learning step, learning is performed in accordance with the size of a face candidate region that is a determination target of whether or not it is a face. The image processing method according to any one of claims 13 to 22,
In the learning step, learning is performed according to a light distribution condition when the image is photographed. A face candidate region extraction step of extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning;
A display step of displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection step for allowing an operator to select a face area from the displayed face candidate areas;
A first learning step of performing learning of the face candidate region extraction process based on the selection result;
A correction strength calculation step of calculating a correction strength by a correction strength calculation process in which a calculation condition is determined by learning;
A correction processing step for correcting the image information based on the calculated correction strength;
A correction strength correction step of correcting the calculated correction strength in accordance with an operator's instruction;
A second learning step of performing learning of the correction strength calculation process based on the correction result;
An image processing method comprising: On the computer,
A face candidate region extraction function for extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning;
A display function for displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection function that allows an operator to select a face area from the displayed face candidate areas;
An image correction function for performing a predetermined correction process on at least the selected face area and generating a corrected image;
A learning function for performing learning of the face candidate area extraction process based on the selection result;
An image processing program for realizing The image processing program according to claim 25,
In the computer,
An image processing program for realizing a function of reflecting a learning result in the face candidate area extraction process when a learning result of the learned face candidate area extraction process satisfies a predetermined condition. In the image processing program according to claim 25 or 26,
An image processing program characterized in that the face candidate area extraction process is a face candidate area extraction process using a neural network. On the computer,
A face candidate region extraction function for extracting a face candidate region from image information by a first face candidate region extraction process in which an extraction condition is determined by learning;
A display function for displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection function that allows an operator to select a face area from the displayed face candidate areas;
An image correction function for performing a predetermined correction process on at least the selected face area and generating a corrected image;
A learning function for performing learning of a second face candidate area extraction process having a different configuration from the first face candidate area extraction process based on the selection result, and an extraction condition determined by learning;
An image processing program for realizing The image processing program according to claim 28, wherein
In the computer,
Realizing a function of replacing the first face candidate region extraction process with the second face candidate region extraction process when a learning result of the learned second face candidate region extraction process satisfies a predetermined condition An image processing program characterized by the above. The image processing program according to claim 28 or 29,
The first face candidate area extraction process is a face candidate area extraction process using a first neural network,
The image processing program characterized in that the second face candidate area extraction process is a face candidate area extraction process using a second neural network having a configuration different from that of the first neural network. On the computer,
A face candidate area extraction function for extracting a face candidate area from image information by a predetermined face candidate area extraction process;
A display function for displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection function that allows an operator to select a face area from the displayed face candidate areas;
An image correction function for performing a predetermined correction process on at least the selected face area and generating a corrected image;
Based on the selection result, a learning function for performing learning of face candidate area extraction processing in which extraction conditions are determined by learning;
An image processing program for realizing The image processing program according to claim 31, wherein
In the computer,
When a learning result of the learned face candidate area extraction process satisfies a predetermined condition, a function of introducing a face candidate area extraction process by a face candidate area extraction process reflecting the learning result is realized. Image processing program. In the image processing program according to claim 31 or 32,
An image processing program characterized in that the face candidate area extraction process is a face candidate area extraction process using a neural network. The image processing program according to any one of claims 25 to 33,
An image processing program for performing learning according to the size of a face candidate region that is a determination target of whether or not a face is present when realizing the learning function. The image processing program according to any one of claims 25 to 34,
An image processing program for performing learning according to a light distribution condition when the image is photographed when the learning function is realized. On the computer,
A face candidate region extraction function for extracting a face candidate region from image information by a face candidate region extraction process in which an extraction condition is determined by learning;
A display function for displaying an image on a display unit based on the image information, and displaying the extracted face candidate region on the image;
A face area selection function that allows an operator to select a face area from the displayed face candidate areas;
A first learning function for performing learning of the face candidate region extraction process based on the selection result;
A correction strength calculation function for calculating a correction strength by a correction strength calculation process in which a calculation condition is determined by learning; and
A correction processing function for performing correction processing on the image information based on the calculated correction strength;
A correction strength correction function for correcting the calculated correction strength according to an operator's instruction;
An image processing program for realizing a second learning function for performing learning of the correction intensity calculation processing based on the correction result.

Images