JP2018017611A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing technique for correcting uneven gloss in an image representing the gloss of an object without using a reference plate. A first acquisition unit that acquires first image data representing the reflection intensity of reflected light from an object, a division unit that divides the first image data into regions of a predetermined size, and a division unit that divides the first image data. The calculation means for calculating the average value of the reflection intensity in each area, the extraction means for extracting one value from the average value, and the average value of the reflection intensity in each area is the one value. And an output means for outputting corrected image data obtained by correcting the reflection intensity in each of the regions so as to have the same value. [Selection] Figure 5

Description

  The present invention relates to an image processing technique for uneven gloss in reflected light from an image.

  In recent years, there has been a demand for a technique for acquiring the color and gloss of an object for the purpose of reproducing and storing art such as paintings. As a technique for obtaining the gloss of an object, an image showing the distribution of gloss intensity by switching between the condition of making the polarization direction of the polarizer of the illumination system and the direction of polarization of the polarizer of the light receiving system orthogonal and the condition of making it parallel is used. There is a way to get. However, in the method described above, since the geometric conditions such as the incident angle of illumination light and the emission angle of reflected light change for each position of the object, unevenness occurs in the gloss indicated by the image. On the other hand, Patent Document 1 discloses a technique for calculating a correction coefficient by capturing an image of a reference plate and correcting the gloss indicated by an image obtained by capturing an object placed on the reference plate using the correction coefficient.

JP 2012-002601 A

  However, in the case of an object having fine irregularities on its surface, such as an oil painting, the light reflection direction changes for each position of the object. Therefore, if a reference plate having similar unevenness is not used, the geometric condition at the time of imaging as described above for each pixel of the data representing the correction coefficient obtained using the reference plate and each pixel of the image of the object is used. However, the gloss unevenness cannot be corrected with high accuracy.

  Accordingly, an object of the present invention is to provide an image processing technique for correcting uneven gloss in an image representing the gloss of an object without using a reference plate.

  In order to solve the above-described problem, an image processing apparatus according to the present invention includes first acquisition means for acquiring first image data representing the reflection intensity of reflected light from an object, and the first image data in an area of a predetermined size. A dividing unit that divides the average value of the reflection intensity in each region divided by the dividing unit, an extracting unit that extracts one value from the average value, and each region. Output means for outputting corrected image data obtained by correcting the reflection intensity in each of the regions so that an average value of the reflection intensity is the same as the one value. To do.

  According to the present invention, it is possible to correct uneven gloss in an image representing the gloss of an object without using a reference plate.

FIG. 2 is a block diagram showing an example of a hardware configuration of the image processing apparatus 1 Block diagram showing a functional configuration example of the image processing apparatus 1 Flow chart of processing executed by image processing apparatus 1 Schematic diagram showing how the object was imaged under two different geometric conditions Flowchart of processing for correcting gloss unevenness The figure which shows an example of the process divided | segmented into a block Flow chart of processing to connect images The figure which shows an example of the positional relationship of two image data to connect. The figure which shows an example of the data which the coordinate holding part 208 hold | maintains Flowchart of processing for correcting gloss unevenness The figure which shows an example of the reflective characteristic of an object Flowchart of processing for correcting gloss unevenness The figure which shows an example of UI for acquiring the reflective characteristic of a target object The figure which shows an example of the table showing the correspondence of the reflective characteristic of a target object, and block size

  Embodiments of the present invention will be described below with reference to the drawings. The following examples do not limit the present invention. In addition, not all the combinations of features described in the present embodiment are essential for the solving means of the present invention. In addition, about the same structure, the same code | symbol is attached | subjected and demonstrated.

[Example 1]
FIG. 1 is a hardware configuration example of an image processing apparatus 1 in the present embodiment. The image processing apparatus 1 is a computer, for example, and includes a CPU 101, a ROM 102, and a RAM 103. The CPU 101 executes an OS (operating system) and various programs stored in the ROM 102, HDD (hard disk drive) 15 and the like using the RAM 103 as a work memory. The CPU 101 controls each component via the system bus 107. Note that the processing according to the flowchart to be described later is executed by the CPU 101 after the program code stored in the ROM 102, the HDD 15, or the like is expanded in the RAM 103. An input device 12 such as a mouse and a keyboard and a printer 13 are connected to the general-purpose I / F (interface) 104 via the serial bus 11. The SATA (serial ATA) I / F 105 is connected to a general-purpose drive 16 that reads and writes the HDD 15 and various recording media via the serial bus 14. The CPU 101 uses various recording media mounted on the HDD 15 or the general-purpose drive 16 as a storage location for various data. A display 17 is connected to the video I / F 106. The CPU 101 displays a UI (user interface) provided by the program on the display 17 and receives an input such as a user instruction received via the input device 12.

  FIG. 2 is a block diagram illustrating a functional configuration of the image processing apparatus 1 according to the present embodiment. The image processing apparatus 1 includes a first acquisition unit 201, a second acquisition unit 202, an image holding unit 203, a generation unit 204, a correction unit 205, a connecting unit 206, an output unit 207, and a coordinate holding unit 208. The first acquisition unit 201 acquires first gloss image data representing the gloss of the object obtained by imaging the object from the image holding unit 203. In the present embodiment, a plurality of pieces of image data representing images obtained by dividing an object into regions so as to include overlap are acquired. The second acquisition unit 202 stores second gloss image data representing the gloss of the object obtained by imaging the object under an imaging condition different from that obtained when the first gloss image data is obtained. 203. In the present embodiment, a plurality of pieces of image data obtained by capturing the same region as the region of the object included in each first gloss image data acquired by the first acquisition unit 201 under different conditions are acquired. The generation unit 204 uses the first gloss image data and the second gloss image data acquired by the first acquisition unit 201 and the second acquisition unit 202 to use regular reflection image data representing the gloss regular reflection component of the object. Is generated. The correction unit 205 corrects the gloss unevenness with respect to the regular reflection image data generated by the generation unit 204. The gloss unevenness in the present embodiment is a change in gloss that occurs when geometric conditions such as the incident angle of illumination light and the exit angle of reflected light during imaging change for each position of the object. The connecting unit 206 connects a plurality of specular image data (corrected image data) corrected by the correcting unit 205 based on the coordinate information held in the coordinate holding unit 208 in the overlapping area, and outputs the data to the output unit 207. Output.

  Hereinafter, processing executed in the present embodiment having the above-described configuration will be described with reference to the flowchart of FIG. Note that the object handled in this embodiment is made of the same material, such as an oil painting, and the gloss is substantially uniform on a macro scale, but microscopically depending on the position of the object due to surface irregularities. The object has different gloss intensity. Hereinafter, each step (process) is represented by adding S before the reference numeral.

  In step S <b> 301, the first acquisition unit 201 captures an image with the polarization direction of the polarizer of the imaging device parallel to the polarization direction of the illumination polarizer as glossy image data representing the distribution of the reflection intensity of the regular reflection light on the object. The obtained first glossy image data is acquired from the image holding unit 203. The first glossy image data in this embodiment is image data in which the reflection intensity of light is stored in each pixel. The reflection intensity stored in each pixel by the first glossy image data includes not only the regular reflection light but also the reflection intensity of diffuse reflection light that has passed through the polarizer. FIG. 4A is a diagram illustrating an example of imaging conditions for obtaining first glossy image data. In FIG. 4A, the light emitted from the light source 501 passes through the polarizing filter 504 and enters the target object 503 as irradiated light (incident light). The reflected light reflected by the target object 503 is received by the light receiving unit 502 of the imaging apparatus via the polarizing filter 505. The polarization direction of the polarization filter 504 and the polarization direction of the polarization filter 505 are arranged in the same direction, that is, in parallel. First gloss image data obtained by imaging an object under the conditions as shown in FIG. 4A is held in the image holding unit 203 in advance.

  In S <b> 302, the second acquisition unit 202 captures an image with the polarization direction of the polarizer of the imaging device orthogonal to the polarization direction of the illumination polarizer as glossy image data representing the distribution of the reflection intensity of the diffusely reflected light on the object. The second gloss image data thus obtained is acquired from the image holding unit 203. FIG. 4B is a diagram showing an example of imaging conditions for obtaining the second glossy image data. In FIG. 4B, reference numerals 501 to 504 are the same as those in FIG. The light receiving unit 502 receives the reflected light of the object via the polarizing filter 506 arranged in such a direction that the polarization direction of the polarizing filter 504 and the polarization direction are orthogonal to each other. The second gloss image data obtained by imaging the object under the conditions as shown in FIG. 4B is held in the image holding unit 203 in advance. If the polarization direction of the polarizer of the imaging device is one direction different from the polarization direction of the illumination polarizer, the two directions do not have to be orthogonal.

In step S303, the generation unit 204 uses the first gloss image data in each region of the target acquired in step S301 and the second gloss image data acquired in step S302 to obtain a specular reflection image of the target using Equation 1. Generate data. The pixel value of the first gloss image data is I _N p, the pixel value of the second gloss image data is I _N c, the pixel value of the specular reflection image data is I _N g, and N is the area of the object Represents a number. The regular reflection image data with the reflection intensity of only the regular reflection light obtained by removing the reflection intensity of the diffuse reflection light from the reflection intensity of light as the pixel value can be obtained by Expression 1. The specular reflection image data is data having uneven glossiness because the incident / exit angles of illumination at the time of imaging differ depending on the pixel position.

I _N g (x, y) = α I _N p (x, y) −I _N c (x, y) (Formula 1)
Note that α in Expression 1 is a constant for matching the exposure of the imaging device when the first glossy image data is captured with the exposure of the imaging device when the second glossy image data is captured. One regular reflection image data is generated every N, that is, for one set of the first gloss image data and the second gloss image data.

  In step S304, the correction unit 205 corrects gloss unevenness for the plurality of regular reflection image data generated in step S303. Details of S304 will be described later. In step S305, the connecting unit 206 connects the plurality of specular image data after correcting the gloss unevenness in step S304 with overlapping regions. Details of S305 will be described later. In step S306, the output unit 207 outputs the connected image data connected in step S305 and ends the process.

  Hereinafter, the process of correcting the gloss unevenness in S304 will be described with reference to the flowchart of FIG. Note that the process of S304 is performed for each specular image data.

  In S701, the regular reflection image data generated in S303 is divided into blocks having a preset number of pixels of m × n. FIG. 6A is specular image data divided into blocks, and FIG. 6B is a histogram representing the gloss intensity distribution in the area surrounded by the block 801 in FIG. 6A. The regular reflection image data is data in which gloss unevenness appears in gloss because the incident / exit angles at the time of imaging change for each position of the object. On the other hand, if it is within the range of the small area (area in the block in this embodiment), the geometric condition at the time of imaging is similar between adjacent pixels, and therefore the gloss unevenness in the area is negligibly small. Therefore, the gloss intensity distribution as shown in the histogram of FIG. 6B is not a distribution due to a difference in geometric conditions at the time of imaging, but represents a magnitude relationship of the gloss intensity caused by fine irregularities on the surface of the object. It can be said that.

  In S702, the average value of the pixel values in the block according to Equation 2

Is calculated.

In Expression 2, M represents a block number for identifying each block of regular reflection image data.

  In S703, the average pixel value for each block calculated in S702

The maximum value V _max is extracted from the _list . In an object composed of substantially the same material, the reflection intensity of specular reflection light has the highest value. Therefore, it can be said that the maximum value Vmax of the average pixel value is a value that most reflects the regular reflection light of the material.

  In S704, using Vmax extracted in S703, the pixel values in the blocks of each block are corrected by Equation 3 so that the average pixel value of all the blocks becomes Vmax, and the corrected regular reflection image data is obtained. obtain. In Expression 3, (i, j) is the coordinates of each pixel in the block M, and corresponds to the coordinates (x, y) of the regular reflection image data.

By this processing, the macroscopic gloss intensity is equalized to Vmax in each block, so that gloss unevenness can be corrected while maintaining the gloss intensity for each position of the object. The gloss unevenness can be corrected even if the gloss intensity of all the blocks is equalized with the average value of other blocks in each block instead of the maximum average value.

Hereinafter, the connection process of regular reflection image data in S305 will be described with reference to the flowchart of FIG. In the present embodiment, in order to simplify the description, a case where specular reflection image data D _1g ′ and D _2g ′ having overlapping regions in the horizontal direction as shown in FIG. Similarly, when there are overlapping areas in the vertical direction, connection can be made by performing processing in the vertical direction.

In step S <b> 901, among the regular reflection image data corrected by the correction unit 205, first regular reflection image data D _1g ′ that is a connection destination in image connection is acquired. In S902, among the regular reflection image data corrected by the correction unit 205, the second regular reflection image data I _2g ′ having an overlapping area with the regular reflection image data acquired in S901 is used as a concatenation target. 203.

  In S903, coordinate information indicating the position on the surface of the object of the first specular image data acquired in S901, and coordinates indicating the position on the surface of the object of the second specular image data acquired in S902. Information is acquired from the coordinate holding unit 208. Incidentally, the coordinate information on the surface is data in which the relative position in the whole object corresponding to the four corners (upper left, upper right, lower left, lower right) of the image is described in millimeters as shown in FIG. Each of x10, y10, etc. has a value.

  In S904, a region Sx overlapping in the x direction and a region Sy overlapping in the y direction of the first regular reflection image data and the second regular reflection image data are calculated by Equations 4 and 5, respectively.

x ₂₀ ≦ S _x ≦ x ₁₁ (Formula 4)
y ₂₁ ≦ S _y ≦ y ₁₀ (Expression 5)
Based on the calculated Sx and Sy, an overlap area of Sx × Sy can be specified.

In S905, for the pixel value I _1g ′ (x, y) of the first specular image data and the pixel value I _2g ′ (x, y) of the second specular image data included in the overlapping area, 6 is used to calculate the evaluation value R. In Equation 6, k is the total number of pixels of the first regular reflection image data included in the overlapping region. As for the total number of pixels included in the overlapping area, the first specular image data and the second specular image data are the same, and therefore k is the pixel of the second specular image data included in the overlapping area. The total number of

In S906, the magnitude relation between the evaluation value R calculated in S905 and the variable Rmin representing the minimum value of the evaluation value R calculated for each overlapping region is compared. If R is smaller than Rmin, the process proceeds to S907; otherwise, the process proceeds to S908. In S907, the value of the variable Rmin is updated to R using Expression 7.

R _min = R (Expression 7)
In step S908, it is determined whether the search has been completed for a preset search range. If the processing has been completed for all the search ranges, the process proceeds to S910; otherwise, the process proceeds to S909. In this embodiment, the search range represented by the black frame in FIG. 8B is a range corresponding to an area surrounded by 10% of the long side and the short side of the second regular reflection image data. It is not limited to.

  In S909, within the search range, the second specular image data is shifted and moved by one pixel distance (Δx, Δy) in the x direction or the y direction, and the overlapping regions Sx, Sy are expressed by Expressions 8 and 9. Update and return to S905.

x ₂₀ + Δx ≦ S _x ≦ x ₁₁ (Expression 8)
y ₂₁ + Δy ≦ S _y ≦ y ₁₀ (formula 9)
In S910, the first regular reflection image data and the second regular reflection image data are connected within the range of the overlapping region corresponding to Rmin. At this time, a known interpolation operation is performed in the overlapping region. As a known interpolation method in this embodiment, a linear density conversion method is used, but is not limited to this. For example, a histogram matching method or an average density difference correction method may be used. In S911, it is determined whether or not all the regular reflection image data have been connected. If concatenation has been completed, the process is terminated; otherwise, the process returns to S902.

  As described above, by correcting the pixel value in each block so that the average value for each block of the regular reflection image data is constant, the image has the relative relationship of the gloss intensity for each position of the object. Uneven gloss can be corrected.

  In this embodiment, the object is imaged by dividing it into a plurality of regions, and the connection is performed after correcting the uneven glossiness. However, the present invention is not limited to the above example. If it is not necessary to divide the object into a plurality of areas and capture images, one first gloss image data and one second gloss image data are obtained, and gloss unevenness correction is performed on one regular reflection image data. Then, the regular reflection image data may be output in S306 without performing the connection process in S305.

[Example 2]
In the first embodiment, in the gloss unevenness correction process, the gloss image is divided into predetermined blocks of a predetermined size. However, when the block size is large, gloss unevenness due to a difference in geometric conditions may be included in the block. In such a case, the block size may be adaptively changed according to the glossy image. In the present embodiment, a difference from the first embodiment will be mainly described with respect to a method for appropriately determining a block size according to a glossy image.

  Hereinafter, the gloss unevenness correction processing in S304 will be described with reference to the flowchart of FIG. This process is a process for determining a block size based on a statistical value of pixel values in a block in order to determine a block size with less influence of gloss unevenness due to a difference in geometric conditions from a plurality of types of block sizes.

  In step S1201, a pixel range (2a × 2b region) from −a to + a in the x direction and −b to + b in the y direction is extracted with the center coordinate of the regular reflection image data as the center. In S1202, the standard deviation σ of the pixel value in the block extracted in S1201 is calculated by Expression 10 and Expression 11. Icg is the pixel value in the block extracted in S1201.

In S1203, the magnitude relation between σ calculated in S1202 and the variable σmin representing the minimum value of the standard deviation is compared. If Expression 12 is satisfied, the process proceeds to S1204, and if not, the process proceeds to S1205.

σ <σmin (Formula 12)
In S1204, the value of σmin is updated to σ. In step S1205, it is determined whether the standard deviation σ has been calculated for all combinations of a and b in the block size. The combination of a and b is from (a, b) = (1, 1) to a combination of values in which the 2a × 2b region is equal to the entire region of the regular reflection image data. For example, when the regular reflection image data is 6 × 6 pixels, (1, 1), (2, 1), (1, 2), (2, 2), (3, 2) for a and b , (2, 3), (3, 3) are all combinations. If the standard deviation has been calculated for all the combinations, the process proceeds to S1207, and if not, the process proceeds to S1206.

  In S1206, the value of a or b is changed so that an unprocessed combination is obtained for a and b, and the process returns to S1202. Since S1207 to S1210 are the same as S701 to S704 in the first embodiment, description thereof is omitted.

  Through the above processing, the uneven glossiness of the image can be corrected more accurately by adaptively determining the block size according to the standard deviation of the pixel value for each block of the glossy image.

[Example 3]
In the second embodiment, in the gloss unevenness correction process, the block size is determined according to the standard deviation of the pixel value for each block of the glossy image. However, the method for determining the block size is not limited to this. FIG. 11 is a diagram showing a difference in reflection characteristics depending on characteristics of an object. In FIG. 11, the horizontal axis represents the reflection angle, and the 0 degree direction is the regular reflection direction. The vertical axis represents the reflection intensity. As shown in FIG. 11A, in the case of an object having a large change in reflection intensity for each angle, such as metal, the reflection intensity changes greatly only by slightly deviating from the angle indicating the regular reflection direction. On the other hand, as shown in FIG. 11B, when the change in the reflection intensity for each angle such as an oil painting is small, the reflection intensity does not change greatly even if the angle deviates from the angle indicating the regular reflection direction. In such a case, the block size may be adaptively determined according to the change in the reflection intensity for each angle of the object. In the present embodiment, the difference from the first embodiment will be mainly described with respect to the method for determining the block size according to the characteristics of the object.

  Hereinafter, the gloss unevenness correction processing in S304 will be described with reference to the flowchart of FIG.

  In S1401, the reflection characteristic for each angle of the object is acquired. FIG. 13 shows a user interface (UI) for acquiring characteristics of an object. In this embodiment, it is assumed that three types of characteristics are preset. The first is “metal” with a large change in reflection intensity at each angle, the second is “oil painting” with a medium change in reflection intensity at each angle, and the third is the reflection intensity at each angle. “Others” is a state in which the change in is small. Based on the user instruction result indicating which of the above three states the object is obtained via the UI of FIG. 13, the reflection characteristic is acquired. Note that the reflection characteristics for each preset angle are not limited to three.

  In step S1402, the block size corresponding to the characteristic acquired in step S1401 is acquired by referring to the table representing the correspondence between the characteristic of the object and the block size shown in FIG. Since steps S1403 to S1406 are the same as steps S701 to S704 in the first embodiment, description thereof is omitted.

  Through the above processing, the uneven glossiness of the image can be corrected with higher accuracy by adaptively determining the block size according to the characteristics of the object.

  Note that, instead of acquiring the reflection characteristic of the object based on the user's instruction result as in the present embodiment, the reflection characteristic (reflection intensity) obtained by imaging the object may be acquired. . In this case, a block size corresponding to the characteristics of the object can be determined by preparing in advance a table in which the value of the reflection intensity is associated with the block size.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1 Image Processing Device 205 Correction Unit 207 Output Unit

Claims (14)

First acquisition means for acquiring first image data representing reflection intensity of reflected light from an object;
Dividing means for dividing the first image data into regions of a predetermined size;
Calculating means for calculating an average value of the reflection intensity in each region divided by the dividing means;
Extracting means for extracting one value from the average value;
Output means for outputting corrected image data obtained by correcting the reflection intensity in each region such that an average value of the reflection intensity in each region is the same as the one value;
An image processing apparatus comprising: Second acquisition means for acquiring second image data representing a reflection intensity of light obtained by polarizing the reflected light with respect to the incident light to the object polarized in a predetermined one direction in the same direction as the predetermined one direction; ,
Third acquisition means for acquiring third image data representing a reflection intensity of light obtained by polarizing the reflected light with respect to the incident light to the object polarized in the first direction in a direction different from the predetermined one direction. When,
Generating means for generating the first image data based on the second image data and the third image data;
The image processing apparatus according to claim 1, wherein the first acquisition unit acquires the first image data generated by the generation unit.   The image processing apparatus according to claim 2, wherein the first image data represents a reflection intensity of a regular reflection component of the reflected light.   The generation means calculates a value of the reflection intensity represented by the first image data by subtracting a value of the reflection intensity represented by the third image data from a value of the reflection intensity represented by the second image data. The image processing apparatus according to claim 2.   The third image data representing the reflection intensity of the light obtained by polarizing the reflected light with respect to the incident light on the object polarized in the first direction in a direction orthogonal to the predetermined one direction. The image processing apparatus according to claim 2, wherein:   6. The image processing apparatus according to claim 1, wherein the predetermined one value extracted by the extraction unit is a maximum value among average values of the reflection intensities. A calculation means for calculating a standard deviation of the reflection intensity in each region of a plurality of sizes in the first image data;
Setting means for setting the size of the region where the value of the standard deviation is the smallest value to the predetermined size used for the division of the dividing means;
The image processing apparatus according to claim 1, wherein the dividing unit divides the first image data into regions of the predetermined size set by the setting unit. . Fourth acquisition means for acquiring the reflection characteristic of the object;
Determining means for determining the size of the area to be divided by the dividing means based on the reflection characteristics;
The image processing apparatus according to claim 1, wherein the dividing unit divides the first image data into regions having a size determined by the determining unit.   The image processing apparatus according to claim 8, wherein the reflection characteristic is a reflection characteristic specified by a user instruction.   The image processing apparatus according to claim 8, wherein the reflection characteristic is a reflection intensity obtained by imaging the object. The first image data is a plurality of image data obtained by imaging the object divided into a plurality of regions,
And a connecting means for connecting a plurality of the corrected image data,
The image processing apparatus according to claim 1, wherein the output unit outputs the corrected image data connected by the connecting unit. Fifth acquisition means for acquiring, for each of the correction image data, coordinate information indicating the reflection intensity represented by the correction image data at which position in the object;
A specifying means for specifying an overlapping area in the plurality of corrected image data based on the coordinate information; and
The image processing apparatus according to claim 11, wherein the connecting unit connects the corrected image data in the overlapping area specified by the specifying unit.   A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 12. A first acquisition step of acquiring first image data representing a reflection intensity of reflected light from an object;
A dividing step of dividing the first image data into regions of a predetermined size;
A calculation step of calculating an average value of the reflection intensity in each region divided by the dividing unit;
An extraction step of extracting one value from the average value;
An output step of outputting corrected image data obtained by correcting the reflection intensity in each area so that an average value of the reflection intensity in each area is equal to the one value;
An image processing method for an image processing apparatus, comprising: