JP2019032654A - Image processing apparatus, image processing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of estimating lost information from an input image and generating an image having high image quality. An image processing device holds a plurality of learning information 10 in which a first image (in-camera developed image 220) and a second image (user-developed image 210) generated from the same image pickup data are associated with each other. .. In addition, the second image is classified based on the image taste. In addition, an image generator that generates a high-quality image is trained for each classification result of classifying the second image. [Selection diagram] Fig. 1

Description

  The present invention relates to a technique for generating a high-quality image from an input image.

For example, as a method of capturing a high-quality image (high-quality image) using a digital camera which is a kind of imaging device, there is a method of saving a RAW image (Raw image format) at the time of imaging.
Specifically, image data in a recording format called RAW data at the time of imaging, that is, raw data that has been acquired by the image sensor is acquired and stored. Then, an image engine (image processing unit) included in the digital camera performs development processing on the RAW data. The image after the development process (for example, the development process for converting RAW data into a JPEG image) is presented on a finder, a display, or the like included in the digital camera.

Here, the development processing is processing configured by various image processing such as demosaicing processing, exposure adjustment, white balance adjustment, color adjustment, noise removal processing, for example, converting pixel data in a Bayer array into RGB 3-channel pixel values. This is a preset process.
The RAW data stores optical information acquired from the image sensor and imaging parameters separately. For this reason, it is possible to correct white balance, exposure, and the like even after imaging. In addition, since RAW data holds color information of higher gradation than image data of other recording formats, image quality tends not to deteriorate due to image processing.
Furthermore, by holding the RAW data at the time of imaging, it is possible to convert the saved RAW data into a favorite image by setting a parameter according to preference and performing development processing. In this case, it is possible to generate a high-quality developed image according to the user's intention, which is different from the development processing by the digital camera.

However, depending on the configuration of the imaging device (for example, a digital camera), there is a configuration that does not have a function of saving RAW data during imaging. Moreover, since the capacity of the RAW data is large, there are cases where it is not possible to save the RAW data of all captured images due to the limitation of the storage capacity such as the memory. In addition, various information held in the RAW data is lost in the image developed by the digital camera.
For example, there is a method of estimating information lost from an image using a machine learning technique such as estimating color information lost from a monochrome image and generating a color image (see Patent Document 1 and Non-Patent Document 1). ).

JP 2005-167978 A

SatoshiIizuka, Edgar Simo-Serra, and Hiroshi Ishikawa, "Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic ImageColorization with Simultaneous Classification", ACM Transaction on Graphics (Proc. Of SIGGRAPH 2016)

  However, when the user further performs image processing on the image developed by the digital camera, it is very difficult to convert the image into an image having the same image quality as the image that has been developed by the user based on the RAW data. There is a problem that it is.

  The main object of the present invention is to provide an image processing apparatus that learns an image generator that estimates information lost from an input image and generates a high-quality image.

  The present invention is an image processing apparatus that learns an image generator that generates an output image having a relatively high image quality from an input image having a relatively low image quality, and a first image generated from the same imaging data, Storage means for storing a plurality of learning information associated with the second image, first classification means for classifying the second image based on characteristics of the image, and a classification result obtained by classifying the second image Learning means for learning an image generator for generating the output image each time.

  According to the present invention, it is possible to learn an image generator that estimates information lost from an input image and generates a high-quality image.

1 is a diagram illustrating an example of a configuration of an image processing apparatus according to a first embodiment. The flowchart which shows an example of the process sequence of the learning process which an image processing apparatus performs. 6 is a flowchart illustrating an example of a processing procedure of image generation processing performed by the image processing apparatus. The figure which shows an example of a structure of the image processing apparatus in 2nd Embodiment. The flowchart which shows an example of the process sequence of the learning process which an image processing apparatus performs. 6 is a flowchart illustrating an example of a processing procedure of image generation processing performed by the image processing apparatus. The figure which shows an example of a structure of the image processing apparatus which concerns on 3rd Embodiment. 6 is a flowchart illustrating an example of a processing procedure of processing performed by the image processing apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, embodiment described below shows an example at the time of implementing this invention concretely, and is not restricted to this.

The image processing apparatus to which the present invention is applied can output a high-quality image that matches the content of the input image for an input image that does not have RAW data. That is, an output image with relatively high image quality can be generated from an input image with relatively low image quality.
Note that the image processing apparatus can also be mounted on a digital camera which is a kind of imaging apparatus. In this case, even if the imaging result of the digital camera is an input image that does not have RAW data, it is possible to output a high-quality image that matches the content of the input image. That is, even when an image having no RAW data is targeted, it is possible to output a high-quality image as if the user performed the RAW development processing.

The RAW development processing is processing for converting a RAW image into an RGB image by appropriately setting various parameters such as exposure adjustment, white balance adjustment, color adjustment, and noise removal processing.
Hereinafter, a processed image in which image data captured by a device such as a digital camera is automatically RAW developed by the digital camera is referred to as an “in-camera developed image” (first image).
On the other hand, an image after processing in which the user arbitrarily performs RAW development processing based on the imaging data is referred to as a “user development image” (second image).
An image having an image quality equivalent to that of a user developed image generated by the image processing apparatus to which the present invention is applied is referred to as a “high quality image” (output image).

[First Embodiment]
FIG. 1 is a diagram illustrating an example of a configuration of an image processing apparatus according to the present embodiment.
The image processing apparatus 1000 according to the present embodiment mainly executes two processes, a learning process and an image generation process. The operations of various functional units included in the image processing apparatus 1000 are mainly controlled by a CPU (Central Processing Unit) 100.

The image processing apparatus 1000 includes a learning image pair storage unit 110, an image taste classification unit 120, an image generator learning unit 130, and an image generator selection table creation unit 140 as components mainly related to learning processing.
The learning image pair storage unit 110 stores a learning image pair group 10 prepared in advance. The learning image pair group 10 is learning information that associates a plurality of in-camera developed images and user developed images generated from the same imaging result (imaging data).
The learning image pair storage unit 110 stores a learning image pair group 10 as learning information.

  The image taste classification unit 120 is information (hereinafter referred to as image taste) that represents the characteristics of each image of the user developed image group among the learning image pair group (learning image pair group 10) stored in the learning image pair storage unit 110. ) Based on). Specifically, the image taste classification unit 120 classifies the user developed image group into one or more clusters. Details of the classification process based on the taste of the image will be described later.

The image generator learning unit 130 learns an image generator for each cluster of user developed images classified by the image taste classification unit 120. Details of the learning process will be described later.
The image generator selection table creation unit 140 determines an image generator to be used when generating an image. Specifically, an image generator selection table (table) that stores associations between a plurality of image generators generated by the image generator learning unit 130 and the in-camera developed image used for learning is created. Details of the process will be described later.

The image processing apparatus 1000 includes an image generator selection unit 150, an image generation unit 160, and an image display unit 170 as a configuration mainly related to image generation processing.
The image generator selection unit 150 selects an image generator that performs image generation based on an image (input image 20) input as a processing target by the user and a table created by the image generator selection table generation unit 140.
The image generation unit 160 generates an image using the image generator selected by the image generator selection unit 150.
The image display unit 170 displays the high-quality image generated by the image generation unit 160 on a predetermined display screen. For example, the image display unit 170 may be configured to control display on a display screen of an external display device (not shown) connected to the image processing apparatus 1000.
In the above description, the configuration for performing the learning process and the image generation process by an integrated image processing apparatus has been described. However, each process may be performed by separate image processing apparatuses.

  FIG. 2 is a flowchart illustrating an example of a processing procedure of learning processing performed by the image processing apparatus 1000. 2 is mainly executed by the CPU 100. The user developed image group 210 and the in-camera developed image group 220 referred to in the processing are image information included in the learning information of the learning image pair group 10 described above.

The CPU 100 performs the classification process on the user-developed image group among the learning image pair group included in the image processing apparatus 1000 via the image taste classification unit 120 (S201).
In this processing, the user developed image group 210 received by the image processing apparatus 1000, that is, the image after the RAW development is performed by the user, is classified according to the taste of the image. Hereinafter, this point will be specifically described.

“Supervised learning”, one of the machine learning methods, is a method that performs learning using teacher data consisting of pairs of inputs and outputs, and predicts correct outputs for input cases whose outputs are unknown. . Therefore, an output value uniquely determined for the input is required as the teacher data.
However, the development method in the development process varies depending on the object and the imaging scene appearing in the image, and even for the same image, various images can be generated depending on adjustment of parameters in the development process.
That is, user-developed images having various image tastes exist depending on the user's intention, and there is no optimal user-developed image that is uniquely determined for a certain input image. Therefore, in the image processing apparatus according to the present embodiment, teacher data is classified for each taste of a user developed image.

Further, the “image taste” described above represents an impression received from an image such as “soft” and “astringent”. For example, when creating a “soft image”, a development method that combines development processes such as increasing exposure, decreasing contrast, and changing white balance is often employed. In addition, in the case of “astringent image”, a development method that combines development processes such as lowering saturation and increasing contrast is often employed.
As described above, the “image taste” of an image has a greater influence on the hue and distribution of luminance values (luminance distribution) than on the content of the image itself.

  Therefore, in the image processing apparatus according to the present embodiment, color and luminance distribution information (luminance distribution information) is used as a similarity criterion for classifying images for each image taste. Specifically, histograms of H (hue), S (saturation), and V (brightness) when the image is expressed in the HSV color space are used as feature amounts. These three histograms are combined into one feature vector, and images are classified based on the feature vector.

For example, the classification can be performed using any existing method. A representative method is a k-average method. In this case, the user-developed image group is classified into k clusters having different tastes.
Note that the feature amount used for classification is not limited to the HSV histogram feature amount, and any feature amount based on color information that is an element constituting the taste can be used.
Further, for example, even when there are a plurality of user developed images (= teacher data) for the same in-camera developed image, it is possible to perform learning by uniquely determining the teacher data for the in-camera developed image.

The CPU 100 creates a plurality of learning sets (learning information) based on the classification results in the process of step S201 (S202).
In the present embodiment, the learning set includes a plurality of in-camera developed images and user developed images. Therefore, the cluster for each image taste created in the process of step S201 is set as one unit of the learning set. Specifically, a set of user developed images included in each image taste cluster and in-camera developed images that are a pair of the user developed images is defined as one learning set.

The CPU 100 learns an image generator for each learning set created in the process of step S202 via the image generator learning unit 130 (S203).
The image generator is configured to generate a high quality image as an output when an image is input. In the process of step S203, the image generator is learned using a user developed image that is a pair of the in-camera developed image as teacher information when the in-camera developed image is input. Further, since a learning set is created for each image taste, image generators that generate high-quality images of a specific image taste from the input image are created for the number of image taste clusters classified in the process of step S201. Will be.

  As a method for estimating lost information based on the input image and generating a new image based on the estimated information, for example, it can be realized using a deep neural network (DNN) (for example, Non-patent document 1). Since the image generation method using DNN is an existing method, detailed description thereof is omitted here. If the input / output requirements are satisfied, the learning method of the image generator is not limited to DNN, and any machine learning method may be used.

The CPU 100 associates each in-camera developed image (camera developed image group 220) in the learning data with the image generator created in step S203 via the image generator selection table creating unit 140 (S204). ).
Assume that the in-camera developed image and the image generator are associated with the in-camera developed image used as learning data in the learning process of each image generator.
By this processing, the developed image in the camera and the image generator are associated with each other in a one-to-one relationship. The association result is stored in the image processing apparatus 1000 as an image generator selection table.
Through the learning process described above, an image generator for generating high-quality images with a plurality of different image tastes can be created.

  FIG. 3 is a flowchart illustrating an example of a processing procedure of image generation processing performed by the image processing apparatus 1000. 3 is mainly executed by the CPU 100. In addition, the image processing apparatus 1000 accepts an input image 20 input by the user as an image to be processed.

The CPU 100 allocates the received input image 20 to any one of the developed images in the camera via the image generator selection unit 150 (S301).
In the process of step S203 described above, the image generator is learned in order to generate a high-quality image having a specific image taste. However, when an unknown image is input, what image taste should be converted into what is called a good image (high-quality image) means the meaning of the image, the subject in the image, and the imaging scene It is greatly influenced by.

  For example, a “fluffy” image taste is often applied to portraits and flower photos, and an “astringent” image taste is often applied to street corner photos. Therefore, in order to determine what kind of image taste to generate a high-quality image from the input image, in this embodiment, the input image is prepared in advance based on the image content such as the image capture scene and the captured object information. It is assumed that it is assigned to one of the developed images in the camera.

Specifically, feature amounts are extracted from the input image and the in-camera developed image group, and the input image is assigned to the in-camera developed image having the most similar feature amount extracted from the input image.
As the feature amount, for example, an arbitrary feature amount derived from image content such as an image capturing scene (image capturing location or image capturing situation), an object, or a composition can be used. For example, GIST feature values often used in scene estimation can be mentioned. Alternatively, scene estimation processing and object detection processing may be actually performed on the input image and the in-camera developed image, and the in-camera developed image with the highest matching degree (similarity) may be assigned to the input image.
By this processing, the input image 20 and the in-camera developed image with similar image contents are selected and associated.

The CPU 100 uses the image generator selection table in the process of step S204 via the image generator selection unit 150 to use the image generator associated with the in-camera developed image assigned to the input image in the process of step S301. Is selected (S302).
By this processing, an image generator that generates a high-quality image with an appropriate image taste is selected for an image whose content is close to that of the input image (= assigned in-camera developed image).

The CPU 100 generates a high-quality image from the input image using the image generator selected in step S302 via the image generation unit 160 (S303).
With this processing, a high-quality image is generated from the input image with an appropriate image taste according to the content of the input image.
The CPU 100 presents the high-quality image generated by the image generation unit 160 to the user via the image display unit 170 (S304).

As described above, the image processing apparatus 1000 according to the present embodiment can generate a high-quality image that the user has performed RAW development even for an input image that does not have RAW data.
As a result, even if the image has been developed in the camera, it is as if the RAW development processing was performed with an appropriate image taste according to the content of the image using the RAW data of the image before the development processing. It becomes possible to output a high-quality image.
In the present embodiment, a description has been given of a mode in which the same image processing apparatus 1000 performs learning processing and image generation processing. However, the present embodiment is not limited to such a configuration, and the learning process and the image generation process may be performed by different devices. In this case, for example, in a learning device configured by a PC or the like, the image generator is learned in advance by learning processing. In an image processing apparatus corresponding to an imaging device such as a camera, the learned image generator may be held in a holding unit, and an image may be generated using the image generator.

[Second Embodiment]
In the first embodiment, generation of a high-quality image having an appropriate image taste according to the input image has been described. However, there are generally a plurality of appropriate image tastes for a certain input image. That is, even when the same image is targeted, when the development process is performed based on the RAW data, if the user changes, the processing contents will be different, and the taste of the developed image will also be different. The quality of the developed image is the preference of the user, and it is considered that the image having any image taste is a good image (high quality image) for the user who performed the processing.
Therefore, in the present embodiment, an image processing apparatus capable of generating a high-quality image having a plurality of different image tastes according to the image content of the input image will be described. In addition, while the same thing as the functional structure demonstrated in 1st Embodiment is attached | subjected the same code | symbol, the description is abbreviate | omitted.

FIG. 4 is a diagram illustrating an example of the configuration of the image processing apparatus according to the present embodiment.
The image processing apparatus 4000 according to the present embodiment is different from the image processing apparatus 1000 according to the first embodiment in that it includes an in-camera developed image classification unit 420 in addition to the configuration of the image processing apparatus 1000 in the first embodiment.

  The image processing apparatus 4000 mainly includes a learning image pair storage unit 410, an image content classification unit 420, an image taste classification unit 430, an image generator learning unit 440, and an image generator selection table creation unit 450 as a configuration related to learning processing. In addition, about the learning image pair memory | storage part 410, the image taste classification | category part 430, and the image generator learning part 440, the learning image pair memory | storage part 110 of 1st Embodiment, the image taste classification | category part 120, the image generator learning part 130, an image Since the configuration is similar to that of the generator selection table creation unit 140, the description thereof is omitted.

The image content classification unit 420 performs a classification process based on the image content on the in-camera developed image group out of the learning image pair group (learning image pair group 10) stored in the learning image pair storage unit 410, and stores the image in the camera. The developed image group is classified into one or more clusters.
The image generator selection table creation unit 450 associates a plurality of image generators generated by the image generator learning unit 440 with each in-camera developed image cluster classified by the image content classification unit 420. The associated result is stored in the image processing apparatus 4000 as an image generator selection table (table). Details of the processing will be described later.

The image processing apparatus 4000 includes an image generator selection unit 460, an image generation unit 470, and an image display unit 480 as components mainly related to image generation processing.
The image generator selection unit 460 selects one or more image generators that perform image generation based on the image input by the user (the input image 20) and the table created by the image generator selection table creation unit 450. To do.
The image generation unit 470 generates a high quality image using one or more image generators selected by the image generator selection unit 460.
The image display unit 480 displays one or more high-quality images generated by the image generation unit 470 on the display screen.

FIG. 5 is a flowchart illustrating an example of a processing procedure of learning processing performed by the image processing device 4000. Each process shown in FIG. 5 is mainly executed by the CPU 100. Further, the in-camera developed image group 510 and the user developed image group 520 referred to in the processing are learning information of the learning image pair group 10 described above.
Also, the processing from step S502 to step S504 shown in FIG. 5 is the same processing as the processing from step S201 to step S203 according to the first embodiment, and thus the description thereof is omitted.

The CPU 100 performs classification processing on the in-camera developed image group among the learning image pair group included in the image processing device 4000 via the image content classification unit 420 (S501).
In this process, the in-camera developed image group 510 received by the image processing apparatus 4000 is classified according to the similarity of the semantic content of the image. Hereinafter, this point will be specifically described.

  In this process, the same process as the process of searching for a developed image in the camera similar to the input image in the process of step S301 in the first embodiment is performed. Specifically, the in-camera developed image is classified into a plurality of clusters by using an arbitrary classification method using a feature amount derived from an image capturing scene, an object, and a composition as a reference for similarity. For example, GIST feature values are extracted from all in-camera developed images, and classification is performed by the k-average method based on the distance between the GIST feature values.

Further, a representative feature amount of the cluster is generated for each classified cluster. As the representative feature amount, a feature amount extracted from an arbitrary in-camera developed image included in the cluster may be selected. Or it is good also considering what averaged the feature-value of all the development images in a camera contained in a cluster as a representative feature-value.
By this processing, the in-camera developed image group is classified into a plurality of different clusters (in-camera developed image cluster) based on the similarity of image contents.

The CPU 100 associates the in-camera developed image cluster created in the process of step S501 with the image generator created in the process of step S504 via the image generator selection table creating unit 450 (S505).
Assume that the in-camera developed image cluster and the image generator are associated with a cluster including the in-camera developed image used in the learning process of each image generator. For example, there are learning image pairs that are developed into user-developed images of different image tastes even if they are in-camera developed images that are included in the same cluster (= image contents are similar). Therefore, the same in-camera developed image cluster is associated with a plurality of different image generators. The result of association is stored in the image processing apparatus 4000 as an image generator selection table.

  Note that only the image generators in which a certain number or more of the in-camera developed images included in the cluster are used in the learning process may be associated with the in-camera developed image cluster. Alternatively, the in-camera developed image included in the cluster may be weighted in association with the in-camera developed image cluster and the image generator according to the ratio used in the learning process of each image generator.

  FIG. 6 is a flowchart illustrating an example of a processing procedure of image generation processing performed by the image processing device 4000. Each process shown in FIG. 6 is mainly executed by the CPU 100. Further, the image processing device 4000 receives the input image 20 input by the user as an image to be processed.

The CPU 100 assigns the received input image 20 to any cluster (developed image cluster in the camera) classified in the process of step S501 via the image generator selection unit 460 (S601).
Specifically, the same feature amount as that when the in-camera developed image is classified in the processing of step S501 described above is extracted from the input image, and the representative feature amount created for each cluster and the feature amount extracted from the input image are obtained. In comparison, the input image is assigned to the most similar cluster.
With this process, an image content cluster having similar image content to the input image is selected and associated.

The CPU 100 uses the image generator selection unit 460 to select an image generator associated with the cluster assigned to the input image in the process of step S601, using the image generator selection table in the process of step S505. (S602).
In the process of step S505 described above, each cluster is associated with one or more image generators. Therefore, one or more image generators that generate a high-quality image with an appropriate image taste for an image whose content is close to that of the input image (= an image included in the assigned cluster) are selected by this processing.

The CPU 100 generates a high-quality image from the input image using the one or more image generators selected in step S602 via the image generation unit 470 (S603).
By this processing, one high quality image is generated from each image generator. Therefore, a plurality of high-quality images having different image tastes are output for one input image.

The CPU 100 presents one or more high-quality images generated by the image generation unit 470 to the user via the image display unit 480 (S604).
For example, when weighting is performed for each image generator in the process of step S505 in the learning process, control is performed so that images are displayed in an order corresponding to the order of the weighting. This makes it possible to preferentially present a high-quality image having an image taste that is frequently used for the input image to the user.

  As described above, the image processing apparatus 4000 according to the present embodiment can generate high-quality images having a plurality of different image tastes according to the image content of the input image.

[Modification]
Hereinafter, a case where the image processing apparatus is configured to perform image processing specialized for a specific image taste will be described.
For example, a photographer's work has an image taste unique to the photographer. Therefore, by learning the in-camera developed image and the user (= photographer) developed image for the RAW data captured by a specific photographer, a high-quality image imitating the photographer's image taste can be generated.
Specifically, when there are a large number of images of the target photographer, only the image of the photographer is used as learning data.
On the other hand, if the photographer's image is small relative to the learning data prepared in advance, the image of the professional photographer can be obtained by performing additional learning using the photographer's image data on the image generator that has already been learned. The image generator that imitates the taste is learned.
In this case, first, the photographer's user-developed image is assigned to one of the image taste clusters classified in the processing of step S502. The assignment is performed based on the similarity of the image taste as in the classification standard performed in the process of step S502. Then, additional learning is performed with respect to the image generator corresponding to the assigned image taste cluster, using a pair of the developed image in the camera of the photographer and the user developed image as teacher data.
By controlling in this way, it is possible to generate a high-quality image imitating the image taste of a specific photographer instead of a high-quality image having a general image taste with respect to the input image.

[Third Embodiment]
In the present embodiment, an image processing apparatus configured to include the functions of the image processing apparatus described in the second embodiment will be described.
FIG. 7 is a diagram illustrating an example of the configuration of the image processing apparatus according to the present embodiment.
In the image processing apparatus 7000 according to the present embodiment, for example, a user uploads an arbitrary image via an information terminal (for example, a tablet PC) 9 connected to a cloud environment (Internet 8), and the image is processed on a server in the cloud environment. It is assumed that a high-quality image is generated using the image stored in. Further, it is assumed that the user downloads a favorite image via the information terminal 9 among the high-quality images generated by the image processing device 7000.
The same functional configurations as those described in the first and second embodiments are denoted by the same reference numerals and description thereof is omitted.

The image processing device 7000 includes the same functional configuration as that of the image processing device 4000 described in the second embodiment. The information processing apparatus 7000 includes an image selection unit 710 and a learning data addition unit 720.
The image processing device 4000 generates a high-quality image having one or more different image tastes based on the input image 20 from the user based on the learning result using the learning image pair group 10 prepared in advance.
The image selection unit 710 specifies an image arbitrarily selected by the user from one or more high-quality images generated by the image processing device 4000.
The learning data adding unit 720 adds the received pair of the input image 20 and the image specified by the image selecting unit 710 as learning information to the learning image pair group 10. In this way, the learning data adding unit 720 functions as learning information adding means.

  FIG. 8 is a flowchart illustrating an example of a processing procedure of processing performed by the image processing apparatus 7000 according to the present embodiment. Note that each process shown in FIG. 8 is mainly executed by the CPU 100. Further, the image processing device 7000 receives an input image 20 input by the user as an image to be processed via the Internet 8.

The CPU 100 generates one or more high-quality images having an appropriate image taste according to the contents of the received input image 20 (S801).
The CPU 100 presents one or more high-quality images generated in the process of step S801 to the user via the image display unit 480 (S802). For example, the high-quality image can be presented to the user by uploading the high-quality image to the Internet 8 via the image display unit 480 and checking the user via the information terminal 9.

The user selects an arbitrary number of favorite images from the presented high-quality images. Further, when a favorite image is not presented, the selection may not be performed. Further, the selected image may be controlled to be downloaded to the user information terminal 9.
At this time, the user ID, the classification result of the input image, and the image generator that generated the selected high-quality image may be recorded. By recording these pieces of information, when the same user newly inputs an image from the next time onward, it is possible to preferentially present a high-quality image having an image taste that matches the user's preference.

  The CPU 100 adds a pair of the received input image 20 and an image selected by the user among the generated high-quality images to a learning image pair group prepared in advance as learning data (addition of learning information) (S803). By performing additional learning using a learning image pair group in which learning data is newly added in this way, it is possible to generate a high-quality image with higher user satisfaction.

  The present invention supplies a computer program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus store the computer program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The embodiment described above is for explaining the present invention more specifically, and the scope of the present invention is not limited to these examples.

Claims (10)

An image processing apparatus for learning an image generator that generates an output image with relatively high image quality from an input image with relatively low image quality,
Storage means for storing a plurality of pieces of learning information in which a first image and a second image generated from the same imaging data are associated;
First classification means for classifying the second image based on characteristics of the image;
An image processing apparatus comprising: learning means for learning an image generator that generates the output image for each classification result obtained by classifying the second image. The image content of the input image is information representing the image content including an object included in the input image, a shooting location, a shooting situation, and a composition of the input image.
The image processing apparatus according to claim 1. Selecting means for selecting the image generator based on the image content of the input image;
Generating means for generating the output image using the selected image generator;
The image processing apparatus according to claim 1. Second classification means for classifying based on the image content of the first image;
The selecting means selects one or more image generators based on a classification result of the input image and the second classification means,
The image processing apparatus according to claim 3. The second classifying unit classifies based on one or more similarities among information representing an object included in the first image, the first image shooting location, a shooting situation, and a composition. And
The image processing apparatus according to claim 4. The first classifying means classifies based on one or more similarities of information representing color information, saturation, contrast, and luminance distribution of the second image,
The image processing apparatus according to claim 1. Display means for displaying the generated output image to the user;
A specifying means for specifying an image selected by the user among the presented output images;
Learning information adding means for storing learning information associating the input image with the identified image in the storage means,
The image processing apparatus according to claim 3 or 4. The imaging data is image data having RAW data, the first image is a processed image that has been automatically subjected to RAW development processing in the imaging apparatus, and the second image is a user based on the image data. Is a processed image that has been arbitrarily subjected to RAW development processing,
The image processing apparatus according to claim 1. An image processing method performed by an image processing apparatus that learns an image generator that generates an output image with relatively high image quality from an input image with relatively low image quality,
Storing a plurality of pieces of learning information associating a first image and a second image generated from the same imaging data;
Classifying the second image based on features of the image;
Learning an image generator for generating the output image for each of the classified classification results;
It is characterized by having
Image processing method. A computer program for operating a computer as an image processing device for learning an image generator that generates an output image with relatively high image quality from an input image with relatively low image quality,
The computer,
Storage means for storing a plurality of learning information in which a first image and a second image generated from the same imaging data are associated;
First classification means for classifying the second image based on the characteristics of the image;
For each classification result obtained by classifying the second image, the learning unit is made to function as a learning unit that learns an image generator that generates the output image.
Computer program.

Images