TWI405148B - Method of realism assessment of an image composite - Google Patents

Abstract

A method of realism assessment of an image composite is disclosed. The foreground and the background are transformed into a color space, such as YCbCr, followed by projecting the foreground and the background to a subspace represented by the axes representing chromatic information. The image composite is assessed in the projected subspace, according to linearity of color distributions of the foreground/the background, or according to distance between the color distributions of the foreground/the background and a center of the transformed color space.

Description

Realistic image evaluation method for synthetic images

The present invention relates to image composition, and more particularly to realism assessment and re-coloring of an image composite.

Image synthesis is a post-photographic operation that is usually performed by cut-and-paste. However, synthetic images produced often encounter color incompatibility between the inserted or pasted object and the background of the image. Therefore, when judging by the human eye, it is felt that the synthetic image is not real enough.

In order to make the inserted object more compatible with the image background, some methods have been proposed to solve this problem. However, traditional methods are often too complex, inaccurate, and require the intervention of many users, or require multiple auxiliary images for realistic evaluation. Since conventional methods cannot effectively evaluate synthetic images, there is a need to propose a novel method for performing realistic evaluation of synthetic images economically, accurately, and/or automatically.

In view of the above, one of the objects of embodiments of the present invention is to propose a method for performing an evaluation of image realism by providing some objective metrics. Furthermore, the composite image can be re-colored according to the results of the realism evaluation to improve the compatibility between the inserted object and the image background.

According to one of the embodiments of the present invention, first, the foreground and the background are converted into a color space, wherein the converted color space has a first axis to represent the photometric information, and has a second axis and a third axis to represent the chrominance information. . Projecting the foreground and background to a subspace represented by a second axis and a third axis. The composite image located in the subspace is evaluated based on the linearity of the color distribution of the foreground and background, and the correlation between the color distribution of the foreground and the background.

In accordance with another embodiment of the present invention, the foreground and background are converted to a color space, wherein the converted color space has a first axis to represent the luminosity message and has a second axis and a third axis to represent the chrominance information. The distance between the color distribution of the foreground and the background and the center of the converted color space are separately measured to obtain a gray point. The gray point of the gray point and the background is measured according to the foreground to evaluate the composite image located in the conversion subspace.

According to yet another embodiment of the present invention, the foreground and background are reduced to obtain an area color distribution of the foreground and the background. The histogram distance is used to determine the color similarity between the reduced foreground and the reduced background.

11-21B‧‧‧Steps

27‧‧‧Reduced background

28‧‧‧Insert objects

29‧‧‧ prospects for reduction

30, 40, 60 ‧ ‧ foreground components

32, 42, 62‧‧‧ background components

51-56‧‧‧Steps

Y‧‧‧ brightness

Cb‧‧‧ Chroma

Cr‧‧‧ Chroma

The flowchart of the first figure shows a method for evaluating the realism of a synthetic image according to an embodiment of the present invention.

The second image shows the reduced foreground and reduced background.

The third to third C-pictures show some foreground and background components of the CbCr subspace.

Figures 4A through 4B show some foreground and background components of the YCbCr color space.

The flowchart of the fifth figure shows a re-coloring method of the synthetic image of the embodiment of the present invention.

Figures 6A through 6B show some foreground and background components of the color subspace.

The flowchart of the first figure shows a method for evaluating the realism of a synthetic image according to an embodiment of the present invention. This embodiment proposes two evaluation mechanisms of realism: color similarity and consistency of color tendency. When evaluating synthetic images, one or two of the evaluation mechanisms can be selected (step 11).

Regarding color similarity, first, in step 12, the foreground (such as an inserted object) and/or the reduced background are shrunk to obtain the regional color distribution of the foreground and background. As shown in the second figure, the reduced background 27 is a band region outside the boundary of the inserted object 28, and the reduced foreground 29 is a banded region inside the boundary of the inserted object 28. The main principle of reducing the foreground and reducing the background is that the background far from the inserted object and/or the message located in the center of the foreground is usually not related to the realistic evaluation of the synthesized image. It is worth noting that reducing the outer boundary of background 27 and/or reducing the inner boundary of foreground 29 does not require a similar shape to the boundary of insert article 28. Furthermore, the size of the area for reducing the foreground and reducing the background may depend on the needs of the particular application.

Next, at step 13, color similarity between the reduced background 27 and the reduced foreground 29 is obtained. In this embodiment, the color similarity S of the synthesized image is obtained by a histogram intersection [or a histogram distance], and the color similarity S can be expressed as:

Among them, h ^B (.) and h ^F (.) are respectively the International Commission on Illumination (CIE) L*a*b* color space reduction background and 3D reduction of foreground (3D) ) histogram, i _L , i _a and i _b represent the histogram indices of the L, a and b channels, respectively, and N _L , N _a and N _b represent the number of bins of the histograms of the L, a and b channels, respectively. , M ^B and M ^F represent the magnitude of the background and the foreground, respectively.

Further techniques for histogram intersection/distance can be found in related literature, for example, "Color indexing" by MJ Swain et al., International Journal of Computer Vision (November 1991), vol. 7, no. 1, page 11-32.

Regarding the consistency of the color tendency, in step 14, the foreground and background are converted to a color space, such as the YCbCr color space. In the YCbCr color space, the Y axis represents the luminance information, and the Cb axis and the Cr axis represent the chrominance information. Although the present embodiment is exemplified by the YCbCr color space, other suitable color spaces may be employed, such as a YUV color space or an L*a*b* color space similar to the YCbCr color space.

This embodiment provides two color tendency metrics: linear and grayness. When determining the consistency of the color tendency, one or two of the color tendency metrics may be selected (step 15).

Regarding the linear metric, the foreground and background of the composite image are projected to the CbCr subspace of the YCbCr color space (step 16). For example, in a CbCr subspace, a grayish image containing more achromatic pixels would fall around the origin. The CbCr distribution of such an image can be represented by a straight line passing through the origin. On the other hand, for example, a greenish image containing more colorless pixels and green pixels falls on the negative side of the Cr axis, so the CbCr distribution of such an image can be approximated by a straight line. In step 17, the composite image is evaluated based on the linearity of the foreground and background color distributions and the correlation between the foreground and background color distributions.

The third A diagram illustrates the foreground component 30 and the background component 32 of the CbCr subspace. In this example (step 18A), both the foreground component 30 and the background component 32 are linear and substantially uniform (or coincident with each other), which indicates that the foreground component 30 has a high degree of color consistency with the background component 32, thus synthesizing The image appears to be real. The third B diagram illustrates the foreground component 30 and the background component 32 of another CbCr subspace. In the present example (step 18B), both the foreground component 30 and the background component 32 are linear, but the color distribution is significantly different or inconsistent, which indicates that the foreground component 30 and the background component 32 have a low degree of color consistency, thus synthesizing the image. It seems unreal. The third C diagram illustrates the foreground component 30 and the background component 32 of yet another CbCr subspace. In this example (step 18C), neither foreground component 30 nor background component 32 has a linear color distribution. Therefore, synthetic images may appear to be real.

Regarding the gray point metric, it measures the color distribution of the composite image and the distance from the center of the color space (eg, the three-dimensional CbCr color space) (step 19). In this embodiment, the gray point of the color distribution is defined as the closest point from the center of the YCbCr space to the color distribution of the synthesized image. Negative distance. In this embodiment, the gray points of the foreground and the background are respectively measured. If the background and foreground of the synthetic image have a different color orientation for the gray point, a hollow area is formed between the foreground and the background. At step 20, the composite image is evaluated based on the presence or absence of a hollow region. The fourth A diagram illustrates the foreground component 40 and the background component 42 of the YCbCr color space. In the present example (step 21A), the foreground component 40 and the background component 42 have significantly different color distributions relative to the center of the color space and form a hollow region, which indicates that the foreground component 40 and the background component 42 have a low color consistency. Therefore, the synthetic image does not appear to be true. The fourth B picture simultaneously shows the foreground and background components of the YCbCr space. In the present example (step 21B), the foreground component and the background component have a similar color distribution with respect to the center of the color space, and no hollow region is formed, which indicates that the foreground component 40 and the background component 42 have a high color consistency, and thus are synthesized. The image appears to be real.

According to the aforementioned method for real-time evaluation of synthetic images, the realism of the synthesized images can be objectively and accurately evaluated according to color similarity and/or color tendency (for example, linear or gray point), in particular, only by synthesizing images by a single image. The message can be evaluated. Furthermore, the foregoing embodiments can also be used for automation of realistic evaluation.

According to the one or more measurement results, the synthetic image can be re-colored as needed to improve its realism. The flowchart of the fifth figure shows a re-coloring method of the synthetic image of the embodiment of the present invention. This embodiment proposes two types of re-coloring: consistency of color similarity and color tendency (such as linearity and gray point). One or two re-coloring methods (step 51) may be selected to improve the authenticity of the synthesized image.

Regarding color similarity, a reduced background and a reduced foreground are obtained in step 52 in a manner similar to step 12 of the first figure. Next, the histogram of the reduced foreground (e.g., the inserted object) is adjusted such that the distribution of the reduced foreground matches the reduced background distribution (step 53). For example, the adjustment can be made by shifting and/or expanding the histogram distribution of the reduced foreground. In another embodiment, the histogram of the reduced background may be adjusted, or the histogram of the reduced foreground and background may be adjusted simultaneously. In this embodiment, the color of each foreground pixel is converted in a channel-by-channel in the CIE L*a*b* color space, which can be expressed as follows: c'=(σ _B /σ _F )(c-μ _F )+μ _B ,

Where c represents the pixel value of the input image of a channel, c' represents the corrected pixel value, μ represents the average value, σ represents the standard deviation, subscript B represents the reduced background, and subscript F represents the reduced foreground.

Regarding the consistency of the color tendency, in step 54, the image containing the inserted object is converted to the YCbCr color space. Next, in step 55, the color distribution of the foreground is first rotated along the Y-axis such that the color distribution of the foreground and the color distribution of the background can be aligned or parallel in the CbCr subspace. This rotation removes the inconsistency of the hue between the foreground and the background. Figure 6A illustrates the foreground component 60 and background component 62 of the CbCr subspace. In the present example, the foreground component 60 is rotated in a clockwise direction, and its rotation angle is φ.

Next, in step 56, the foreground color distribution after the first rotation is rotated a second time along the Cb or Cr axis, so that the color distribution of the foreground and the color distribution of the background can be aligned in the YCr or/and YCb subspace or parallel. This rotation handles objects that are over-colorful in the image. The sixth B diagram illustrates the foreground component 60 and the background component 62 of the YCr subspace. In the present example, the foreground component 60 is rotated in a counterclockwise direction, and its rotation angle is θ.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the invention should be included in the following Within the scope of the patent application.

11-21B‧‧‧Steps

Claims (20)

A method for evaluating a realistic image of a synthetic image, comprising: providing a composite image having a foreground and a background; converting the foreground and the background to a color space, wherein the converted color space has a first axis to represent luminosity a message having a second axis and a third axis to represent a chrominance message; projecting the foreground and the background to a subspace represented by the second axis and the third axis; and according to the foreground The linearity of the color distribution with the background, and the correlation between the foreground and the color distribution of the background, is used to evaluate the composite image located in the subspace. The method for evaluating a realistic image of a synthetic image according to claim 1, wherein the foreground is an insert of the synthetic image. The method for evaluating a realistic image of a synthetic image according to claim 1, wherein the converted color space is a YCbCr color space, the first axis is a Y axis, the second axis is a Cb axis, and the third axis is Cr axis. The method for evaluating a realistic image of the synthetic image according to claim 1, wherein the color space of the conversion is a YUV color space or an L*a*b* color space. The method for evaluating the quality of a synthetic image as described in claim 1 is characterized in that when the foreground has a linear color distribution consistent with the background or when the foreground and the background have no linear color distribution, the composite image is evaluated as True; when the foreground has a linear color distribution that is inconsistent with the background, then the composite image is evaluated as unreal. For example, the method for evaluating the realism of the synthetic image described in the first application of the patent scope includes: A first rotation of the foreground or/and the color distribution of the background along the first axis is such that the color distribution of the foreground and the color distribution of the background are aligned or parallel to each other in the projection subspace. The method for evaluating a realistic image of the synthetic image according to claim 6 further includes: performing a second rotation of the foreground or/and the color distribution of the background along the second axis or the third axis, so that the foreground The color distribution and the color distribution of the background are distributed in the subspace represented by the first axis, the third axis, or the subspace represented by the first axis and the second axis, and are aligned or parallel to each other. A method for evaluating a realistic image of a synthetic image, comprising: providing a composite image having a foreground and a background; converting the foreground and the background to a color space, wherein the converted color space has a first axis to represent luminosity a message having a second axis and a third axis to represent a chrominance message; respectively measuring a distance between the foreground and the color distribution of the background and a center of the converted color space to obtain a gray point; The measurement gray point of the foreground and the measurement gray point of the background are used to evaluate the composite image located in the conversion subspace. The method for evaluating a realistic image of a synthetic image according to claim 8, wherein the front view is an insert of the synthetic image. The method for evaluating a realistic image of the synthetic image according to claim 8 , wherein the converted color space is a YCbCr color space, the first axis is a Y axis, the second axis is a Cb axis, and the third axis is Cr axis. A method for evaluating the realism of a synthetic image described in claim 8 of the patent application, The color space of the above conversion is a YUV color space or an L*a*b* color space. The method for evaluating a realistic image of a synthetic image according to claim 8 wherein the gray point is a negative number from a distance from a center of the converted color space to a closest point of the foreground or background color distribution. The method for evaluating the quality of a synthetic image according to claim 8 is that when a hollow region exists between the foreground and the color distribution of the background, the synthesized image is evaluated as being unreal; otherwise, the composite image is The assessment is true. The method for evaluating a realistic image of the synthetic image according to claim 8 further includes: performing a first rotation of the foreground or/and the color distribution of the background along the first axis, such that the color distribution of the foreground is The color distribution of the background is aligned or parallel to each other in the projection subspace. The method for evaluating a realistic image of the synthetic image according to claim 14 further includes: performing a second rotation of the foreground or/and the color distribution of the background along the second axis or the third axis, so that the foreground The color distribution and the color distribution of the background are distributed in the subspace represented by the first axis, the third axis, or the subspace represented by the first axis and the second axis, and are aligned or parallel to each other. A method for evaluating a realistic image of a synthetic image, comprising: providing a composite image having a foreground and a background; reducing the foreground and the background to obtain a color distribution of the foreground and the background; and determining by a histogram distance a color similarity S between the reduced foreground and the reduced background; wherein the color similarity S determined above is expressed as: Where h ^B (.) and h ^F (.) are respectively the reduced background of the International Lighting Organization (CIE) L*a*b* color space and the three-dimensional (3D) histogram of the reduced foreground, i _L , i _a And i _b represent the histogram indicators of the L, a, and b channels, respectively, and NL, Na, and Nb represent the number of bins of the histograms of the L, a, and b channels, respectively, and M ^B and M ^F represent the background and the The size of the foreground. The method for evaluating a realistic image of a synthetic image according to claim 16, wherein the front view is an insert of the synthetic image. The method for evaluating a realistic image of a synthetic image according to claim 17, wherein the reduced background is a strip-shaped region outside the boundary of the inserted object, and the reduced foreground is a strip-shaped region inside the boundary of the inserted object. The method for evaluating the realistic image of the synthetic image according to claim 16 further includes: adjusting the reduced background and the histogram of the reduced foreground such that the distribution of the reduced background matches the distribution of the reduced foreground. The method for evaluating the realism of the synthesized image according to claim 19, wherein the adjustment of the histogram comprises: in the CIE L*a*b* color space, one channel and one channel according to the following formula (channel-by- Channel) converts the color of each foreground pixel: c'=(σ _B /σ _F )(c-μ _F )+μ _B , where c represents the pixel value of the input image of the channel, and c′ represents the corrected pixel value. μ represents the average value, σ represents the standard deviation, subscript B represents the reduced background, and subscript F represents the reduced foreground.