JPH06236440A - Image processing method - Google Patents

Abstract

(57) [Abstract] [Purpose] Color, pattern, and color of the object area of a color natural image
Provided is a simulation method for changing the texture and surface shape separately or in combination. An image processing method of processing image data representing a color image stored in an image storage device by a computer, wherein the image is processed in a color space having a color component value representing a pixel of the image data as a coordinate system. Step 1 of extracting a color vector representing the distribution of pixel values in a specific area in the data
02, and a step 1 of obtaining a component decomposition value obtained by decomposing the pixel value of the specific region with the color vector
02, and steps 105 and 10 for changing the above-mentioned component decomposition image.
7 and a step 110 of reconstructing pixel values of the specific region based on the modified component decomposition image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for simulating a change in surface shape and a change in material of an object in a color natural image, and particularly to an image suitable for product design, sales presentation, etc. Regarding processing method.

[0002]

2. Description of the Related Art Texture mapping is known as a typical method of processing image data representing a color image by a computer and changing the surface shape, material, color or pattern of an object in the image to another one. ing.

Three-dimensional CG (computer graphics)
Seiko Denshi Kogyo Co., Ltd. Electronic Equipment Division "Practical computer graphics" (Publication example 1), pp438
-446, consider mapping (mapping) a texture image (a texture is a detail of the surface of an object) onto the surface of the object defined in the three-dimensional space. At this time, Find the pixels of the texture image that can be seen from the viewpoint. Further, in CG, as described in pp 446-450 of the above-mentioned known example 1, by changing the surface normal by adding a pattern expressing fine irregularities on the surface in the normal direction of the surface of the object. , A bump mapping method for expressing a fine surface shape is known.

Texture mapping is also generally used in a method of converting the material of the specific object region using a natural image such as a photograph as an original image. A process of replacing a pixel of an object region to be converted in an original image with a pixel of a texture image is widely performed by using an image obtained from a photograph of a target material as a texture image. Morita et al., "Examination of texture mapping method reflecting shadow",
1992, IEICE Spring National Convention (Publication 2)
In some cases, a natural image is generated by adding shadows extracted in advance from the object region of the original image after mapping.

The image processing methods described in Japanese Patent Application Laid-Open No. 3-41570 (Publication 3) and Japanese Patent Application No. 3-338167 (Prior Art) are similar to those of the present invention in the object area of a color image. A color design simulation of the object region is performed by converting the distribution of pixel values in the color space according to the reflection model. Since the method is based on the reflection property of the object, it is possible to perform a color change simulation while maintaining the gloss and the shadow of the object. Particularly, in the known example 3, the single object color of the object area is changed. Further, the preceding example is a method in which the texture mappings of the known example 3 and the known example 2 are integrated, and it is possible to change not only a single object color but also the pattern / pattern.

A known example 3 will be briefly described below. In an image obtained by photographing an object such as colored plastic, the pixel value C (x, y) of this object region is input bias vector B = 0 and color texture vector Cd = 0 in order to simplify the description in the known example 3. Then, in the color space as shown in FIG. 3, it can be expressed by a linear sum of three vectors of the object color Cb, the light source color Cs, and the environment color Ca.

[0007]

[Equation 1]

As shown in FIG. 13, the incident light 32 which strikes the surface 33 of the object from the main light source is reflected by the surface 33 of the object (specular reflection) 34 and enters the object and the color particles 3 are reflected.
Light that goes out of the object after repeated collisions with 5 (diffuse reflection)
Divided into 36. The specularly reflected light is reflected at a predetermined angle determined by the incident angle and the surface direction of the object, while the diffusely reflected light diffuses from the surface 33 in all directions. Specular reflected light may be considered to have the same three primary color component ratio as light from the main light source, and diffuse reflected light has three primary color component values determined by the main light source and the specific reflectance of the object. Therefore, the specular reflection 34 represents the light source color, and the diffuse reflection 36 diffused by the color particles 35 inside the object represents the object color. Diffuse reflectance Co for each R, G, B component of the object

[0009]

[Equation 2]

As an incident light I

[0011]

[Equation 3]

Then, the object color Cb is

[0013]

[Equation 4]

It can be expressed as follows.

Further, in general, the light emitted from the main light source hits the substance around the object and is reflected and reaches the object. For example, as shown in FIG. 14, the light reaching the object includes light from the blue sky (background), and there is reflection of this light by the object. The color of this reflected light is called the environmental color. This environmental color C
Also for a, the ambient light Ia from the background

[0016]

[Equation 5]

Then,

[0018]

[Equation 6]

It becomes

Further, in an object such as colored plastic, the light source color Cs is the same as the incident light I.

[0021]

[Equation 7]

K is a coefficient. The component decomposition values ms (x, y) and mb (x, y) for the vectors Cs and Cb are calculated for all pixel values C (x, y) in the object area. An image having this component decomposition value as a pixel value is called a component decomposition image (FIG. 12).

In the known example 3, when the color of the object is changed,
The object color is changed by changing the object color Cb, and also changing other vectors that need to be changed based on this, and reconstructing the pixel value of the object region by the equation (1).

In the preceding example, Cb in equation 1 for each pixel x, y
Define (x, y),

[0025]

[Equation 8]

By the above, conversion for attaching patterns and patterns is realized.

In the known example 3 and the prior art, the diffuse reflection color and the specular reflection color are the object color Cb and the light source color Cs.
Reflected in.

[0028]

The conventional methods described above have the following problems.

The CG method of the publicly known example 1 is flexible by using various attributes such as material, surface shape, color and pattern, which are advantages of CG, and three-dimensional information of an object, light source, viewpoint and the like. Although it can be simulated, there is a problem that precise modeling is necessary to generate a realistic image.

In the known example 2, when there is an image such as a photograph of a real object having a material, a surface shape, a color and a pattern to be expressed, the simulation is simply performed by mapping the photograph of the object. There is an advantage that can be. However, it is not practical to store as many images as there are attribute combinations, and there is also the problem that it is not possible to perform flexible design simulation such as specifying the material, surface shape, color, and pattern to be changed separately. .

When the methods of the known example 3 and the prior example are used, it is not necessary to retain a large number of texture images when changing the object color of a single color, as compared with the known example 2.
Since the simulation is performed on the model based on the reflection property, there is an advantage that a more realistic image can be obtained. However, similarly to the known example 2, although the color elements such as the color and pattern of the object area can be specified separately from the shadow to perform the simulation, the geometric surface shape of the object surface other than the color elements and There is a problem that it is not possible to perform a simulation that combines materials.

An object of the present invention is to provide an image processing method capable of performing a simple and realistic simulation based on a general color natural image such as a photograph.

Another object of the present invention is to provide an image processing method capable of performing a simulation that combines surface shapes and materials in addition to colors and patterns.

[0034]

In order to achieve the above object, the present invention is an image processing method for processing image data representing a color image stored in an image storage device by a computer. In a color space having a color component value representing a pixel of a coordinate system as a coordinate system, a process of extracting a color vector representing a distribution of pixel values of a specific region in the image data, and a pixel value of the specific region being the color vector A step of obtaining a component decomposition value decomposed into components as a component decomposition image, a step of changing the component decomposition image, and a step of reconstructing pixel values of the specific region based on the changed component decomposition image. It is characterized by

The specific area is an object area and can be distinguished from other areas by a mask for distinguishing individual pixels.

The color vectors are, for example, a diffuse reflection color vector representing the diffuse reflection color of the object and a specular reflection color vector representing the specular reflection color.

By changing the component decomposition value by the specular reflection color vector and the component decomposition value by the diffuse reflection color vector according to the material of the object in a specific relation, the reflection characteristic of the object is changed to obtain a desired material appearance. To get

A method for changing the surface shape of an object by changing the component decomposition image is also provided.

A combination of color vector changes can be used to change the component decomposition image.

[0040]

As described in the known example 3, each pixel value C (x, y) of the object region is calculated by the component decomposition value mb by the object color Cb and the component decomposition value ms by the light source color Cs.
Is expressed by Equation 9.

[0041]

[Equation 9]

Is represents the pixel color due to specular reflection, and Id represents the pixel color due to diffuse reflection. The light source color Cs represents the color of specular reflection, where the incident light is I = (Ri, Gi, Bi) and the specular reflectance is Cos = (ros, gos, bos).
As described in (1), the light source color Cs is as shown in Expression 10.

[0043]

[Equation 10]

In this specification, the light source color Cs is a specular reflection color,
Alternatively, it is called a specular reflection vector. The object color Cb represents the color of diffuse reflection, and the diffuse reflectance is Co = (r
o, go, bo), Cb becomes as shown in Formula 11, as described in the known example 3.

[0045]

[Equation 11]

In the present specification, Cb is a diffuse reflection color,
Alternatively, it is called a diffuse reflection vector.

Component decomposition values ms (x, y), mb (x,
y) is considered to be shadow information other than color elements such as the color of the light source and the color of the object, and as shown in FIG. 4, a point P on the object corresponding to points x and y in the object area in the image. , The angle θ formed by the direction N with the incident direction I of the light source, the angle e formed by the viewpoint directions V and N, and the geometric information determined by the angle g formed by I and V. Therefore, by changing the component decomposition values ms and mb, it is possible to express a shadow when the surface shape such as fine irregularities on the surface of the object is changed.

Further, the diffuse component decomposition value by the diffuse reflection color and the component decomposition value by the specular reflection color express the reflection characteristics of the object, and the distribution state of the specular reflection component and the diffuse reflection component and the specular reflection component depending on the material. Characteristically, the rate of changes. For example, in the case of a smooth plastic object, highlights are strongly observed on a part of the surface of the object, which means that in the component decomposition value, the specular component decomposition is ms> 0 corresponding to a specific diffuse reflection component value mb. Indicates that the value is present. A metal object has a large specular reflection component, and a matte object such as rubber has a larger diffuse reflection component. By paying attention to the reflection characteristic of such an object and changing the diffuse reflection component decomposition value and the specular reflection component decomposition value in association with each other, it is possible to change the material appearance of the object region related to the reflection characteristic.

That is, it is possible to change the surface shape and material appearance of an object and the color / pattern by changing the component decomposition image and the color vector. In Equation 9, a simulation that combines the two is possible. The change in color / pattern is as described in the known example 3 and the preceding example, and will not be described in detail here.

In the present invention, the following technique for changing the surface shape and material appearance of an object is applied to a natural image.

(1) Smoothing of diffuse reflection component The color at a certain point P on the object due to diffuse reflection can be expressed as in Equation 12 using Lambert's law (the vector N representing the direction of the surface is of magnitude Shall be normalized).

[0052]

[Equation 12]

[0053]

[Equation 13]

Θ is the angle formed by the incident light I at the point P in FIG. 4 and the direction N of the surface. kd is a diffuse reflection coefficient depending on the material. Similarly to the known example 3, the color of the incident light I is R,
It is represented by G and B components, I = (Ri, Gi, Bi), and kd is also R,
Substituting diffuse reflectance Co = (ro, go, bo) of the G and B component objects and using Equations 9 and 11,

[0055]

[Equation 14]

[0056]

[Equation 15]

Is obtained.

Here, θ is an angle formed by the incident light I and the surface direction N in FIG. 4, and cos θ corresponds to the component decomposition value mb. In this model, mb is the surface direction N and the light source direction I.
Is the geometrical information according to, and the viewpoint direction V is irrelevant.
From Equations 13 and 15, when the light source is sufficiently far, the diffusion component decomposition value mb is smoothed by weighted addition, which corresponds to the processing of smoothing the direction of the surface. Therefore, there is an effect that the surface looks smooth. It can also be used for noise removal of diffuse reflection components.

(2) Exponentiation of specular reflection component Regarding the specular reflection, Phong's model,
And the Blinn model is often used. On a well-polished surface such as a mirror, incident light reflects in a specular reflection direction J, which is the opposite direction to the orientation of the surface. In the case of a general object, it spreads around the specular reflection direction due to fine irregularities on the surface. In the Phong model, the angle α formed by the viewpoint direction V and the regular reflection direction J is considered, and the viewpoint direction is deviated from the regular reflection direction, and the nth power of cos α (n is an arbitrary real number) is viewed from the regular reflection direction. It is said that it is decreasing than when.

[0060]

[Equation 16]

Ks is a specular reflection coefficient depending on the material. If incident light I = (Ri, Gi, Bi) is set and ks is also replaced by specular reflectance Cos = (ros, gos, bos) of an object of R, G, B components, and Equations 9 and 10 are used,

[0062]

[Equation 17]

[0063]

[Equation 18]

In Bryn's model, the surface of an object is considered to be a set of minute planes having fluctuations in the direction. At this time, a vector H having a ½ angle direction of V and I
And the specular reflected light seen from the viewpoint direction V on the point P is
It is assumed to be due to specular reflection of a minute surface facing the H direction. At this time,
At the point P, the ratio of the minute surface facing the H direction determines the amount of specular reflection. Assuming that the ratio of the minute faces in the H direction is maximum when the directions N and H of the faces coincide with each other, and the angle between H and N is β, the ratio of the minute faces facing the H direction is n of cosβ.
Express by the power of square.

[0065]

[Formula 19]

As in the Phong model, the incident light I = (Ri,
Gi, Bi) and specular reflectance Cos = (ros, gos, bos), using Equations 9 and 10,

[0067]

[Equation 20]

[0068]

[Equation 21]

According to the equations (18) and (21), ms varies depending on the value of n as shown in FIG. The smoother the material is, such as a well-polished mirror, the less irregular the surface is, so that the spread of specular reflection in the specular reflection direction can be suppressed. At this time, n takes a larger value.

From the equations (18) and (21), if the component decomposition value ms of the specular reflection component is the k-th power (k is an arbitrary real number: different from k in equation (7)), n is changed to k * n. . Therefore, when the component decomposition value ms of the specular reflection component is raised to the power of k, the object approaches a mirror-like object having a smooth surface specular reflection characteristic.

(3) Representation of unevenness The component decomposition values mb and ms are the object surface direction, the light source direction,
Since the geometrical information about the viewpoint direction is represented, the effect equivalent to bump mapping in the three-dimensional CG can be obtained by multiplying or adding the function of the texture representing the minute unevenness to the component decomposition image. As shown in FIG. 6, rendering is performed on a three-dimensional shape representing unevenness of the surface using appropriate specular reflection and diffuse reflection models such as Equations 12 and 16, and a specular reflection image and diffuse reflection image are obtained, respectively. Generate an image. By multiplying each pixel value of the original component decomposed image by the ratio of each pixel value of this image to the maximum pixel value, the three-dimensional rendering image representing the surface irregularities is reflected in the component decomposed image. At this time, the three-dimensional shape representing the unevenness of the surface can be generated by using the bump function or by mapping the bump function on the surface of a simple object such as a hemisphere using the bump mapping method.

Further, as shown in FIG.
Given a dimensional structure, the surface decomposition of the approximate 3D object can be used to modify the component decomposed image. The variation Δmb of the diffuse reflection component due to bump mapping with respect to the surface normal is calculated, and the original component decomposition value mb (x,
y), the diffuse reflection component mb ′ (x, y) after bump mapping is obtained.

[0073]

[Equation 22]

From equations 13 and 15,

[0075]

[Equation 23]

Here, θ (x, y) is the angle formed by the incident direction I and the surface normal N ′ changed by bump mapping at the point P on the object corresponding to the pixel x, y. In the following description, the description of x and y is omitted and

[0077]

[Equation 24]

[0078]

[Equation 25]

The surface Q (u, w) of the three-dimensional object and the normal N
If (u, w), the surface changes as follows due to the bump function f (u, w). At this time, u (x, y), w
Let (x, y).

[0080]

[Equation 26]

At this time, the normal line N is

[0082]

[Equation 27]

Here, Qu is u in the plane Q (u, w).
Partial differential in the direction

[0084]

[Equation 28]

Qw is a partial differential in the w direction on the surface Q (u, w), and

[0086]

[Equation 29]

X represents an outer product. Similarly, Q '(u,
The modified normal N ′ of w) is

[0088]

[Equation 30]

And Qu 'and Qw' are

[0090]

[Equation 31]

[0091]

[Equation 32]

And fu and fw are

[0093]

[Expression 33]

[0094]

[Equation 34]

It is As described above, using Equations 24 to 34, mb ′ in Equation 25 is as follows.

[0096]

[Equation 35]

Therefore, the variation Δmb of mb is

[0098]

[Equation 36]

By assuming the object Q (u, w) and the light source I, the diffuse reflection component after mapping can be obtained. Since the amount of change on the assumed object is added to the component decomposition image of the original image, the effect can be expected even if the assumed object is not exact.

(4) Change of Material Next, the material feeling is changed by changing the reflection characteristic. The reflection characteristic of the object to be changed is modeled, and the component decomposition image is processed according to the model. In CG, the plastic may be modeled by a linear sum of specular reflection and diffuse reflection, and the metal may be modeled by specular reflection using the mathematical formulas 12 and 16. "Recognition of Reflection Characteristics of Object Surface Using Color Image" by Hashimoto et al., Preprints of the Second Signals Systems
As described in Control Symposium) ('89 .1.24-25), the distribution of pixel values in the color space of the object region where the color nature of the material is characteristic is characteristic. This is almost in agreement with the result of modeling in CG. The pixel value distributions in the color space of plastic, metal, rubber and cloth are shown in FIGS. 8, 9, 10, and 11. In order to change the material appearance, each pixel value of the component decomposition value ms (x, y), mb
By changing (x, y) and the color vectors Cs and Cb, a distribution having characteristics of the material is obtained. The characteristics of these materials are described below for plastic, metallic, metal, rubber and cloth, and how the component decomposition image and color vector should be changed in order to express the texture.

(A) Plastic Object The characteristics of the pixel value distribution in the color space of the plastic object shown in FIG. 8 are shown below.

In the color space, the pixel values are distributed almost linearly on the two vectors of the specular reflection vector Cs and the diffuse reflection vector Cb.

Since the specular reflection has the same color as the light source, the direction of the specular reflection color vector represented by the distribution of pixel values is the same as the color of the light source.

From the above, the component decomposed image and the color vector may be set as follows.

In the equation 9, the specular reflection component ms sufficiently larger than 0 exists only for the pixels x and y having the diffuse reflection component value mb in the specific range.

The direction of Cs is the same as that of the light source. In other words, in equation 10, ros = gos = bos.

In order to obtain a glossy material appearance, in order to increase the ratio of the specular reflection component, Cs in Equation 9 should be increased, or in order to narrow the spread of the highlight portion on the object,
The specular reflection component Ms in Expressions 18 and 21 may be raised to the n-th power.

(B) The distribution of pixel values in the metal object color space is set as shown in FIG. That is, the metal has the following characteristics.

In the color space, the pixel values are distributed almost linearly on the specular reflection vector Cs.

The metal color is reflected in the color of the specular reflection component.

Specular reflection is observed at each point on the object, and the change in the specular reflection component decomposition value ms is gentler than that of plastic.

From the above, the component decomposed image and the color vector may be set as follows.

In Equation 9, Cs is increased to increase the specular reflection component, and Cb is decreased to reduce the diffuse reflection component.

In the equation 17, the reflectance (ros, go
(s, boss) is defined as a metal color, and a specular reflection vector Cs different from the color of the light source is defined.

The change in the specular reflection component value ms is made gentle.

(C) Metallic object The metallic paint is modeled as shown in FIG. That is, metallic has the following characteristics.

There are specular reflection and diffuse reflection, but the distribution of pixel values in the color space does not clearly separate specular reflection and diffuse reflection.

The metal color is reflected in the color of the specular reflection component.

These are because minute metal pieces are randomly scattered in the medium of the metallic paint, and the specularly reflected light is incident on the starting point from various angles by the minute metal pieces. Therefore, the texture of the metallic paint is expressed as follows.

The ms picked up at regular intervals or at irregular intervals are set to 0 <ms <1 and other ms are set to almost 0.

In equation 17, the reflectance (ros, go
(s, boss) is defined as a metal color, and a specular reflection vector Cs different from the color of the light source is defined.

(D) Matte object such as cloth, rubber, etc. The distribution of pixel values in the color space is set as shown in FIG. That is, cloth and rubber have the following characteristics.

In the color space, the pixel values are distributed almost linearly on the diffuse reflection vector Cb.

As a result, the following may be carried out.

In Equation 9, the specular reflection vector Cs is reduced to reduce the specular reflection component.

In addition to the above processing, the surface shape of various materials (fabric, stone, etc.) can be expressed by the processing of "(3) Expression of unevenness".

[0127]

EXAMPLE An example of the present invention will be described in detail. In this embodiment, by changing the pixel value of the object area of the color natural image, the appearance of the material of the object, the surface shape of the unevenness, the color
It is intended to simulate a pattern change.

First, FIG. 2 shows a hardware configuration of a system in which the image processing of the present invention is executed. This system includes a computer 202 that executes the image processing program of the present invention, an external storage device 205 that stores the program and data, a keyboard 203 and a mouse 204 that receive input operations from a user, and a display unit that displays characters and image data ( A CRT) and a scanner (or video camera) 206 for inputting original image information.

FIG. 1 shows the processing procedure of an embodiment of the present invention.

In step 101, the image data stored in the storage device 205 is input. In this embodiment, this image data is original image data of a natural image captured by a video camera, as shown in FIG. 15, for example. That is, as shown in FIG. 12A, the original image is represented by R, G, B values for each pixel corresponding to the two-dimensional coordinates (x, y). It is assumed that, in addition to the color information, information different from the color called a mask value is added to each pixel in the image data. By giving a special mask value only to the pixels in the area where the image simulation is performed, it is possible to determine whether or not the pixel of interest is a target to be changed thereafter. In the present embodiment, an example in which an image obtained by shooting a plastic object is used as an input image will be described.

At step 102, from the target object area,
Diffuse reflection color (object color) Cb in the same manner as in the known example 3,
The specular reflection color (light source color) Cs and the environment color Ca are extracted (FIG. 12B). Next, these vectors are calculated by computer 2
02 is stored in the main memory (not shown).

In step 103, the specular reflection component decomposition value (ms) and diffuse reflection component decomposition value (mb) are obtained for each pixel in the target object region by the same method as in the known example 3 (FIG. 12 (c)). , And stores it in the main memory in the computer 202.

The following three sets of processing of steps 104 and 105, steps 106 and 107, and steps 108 and 109 can be executed independently or in any combination.

At step 104, the intended material is designated. First, using the menu displayed on the CRT 201, plastic, metal, metallic, or cloth is selected with the mouse 204 or the keyboard 203. After that,
The surface shape, color, and pattern are specified for each of these materials. This is done by selecting a menu item displayed on the CRT 201 with the mouse 204 or the keyboard 203.

In step 105, the distribution of pixel values in the color space corresponding to the material designated in step 104 (FIGS. 8 to 11), as described in "(4) Change of material" in the operation.
The component decomposition value or color vector is changed so that The case where the material is plastic, metallic metal, or rubber will be described below.

(1) Plastic Object When the input image is a plastic object as in the present embodiment, the process of changing the component decomposition image and the color vector in this step is basically unnecessary. However, here, a modification example will be described in which an object such as rubber having no specular reflection component serves as an input image. First, a specular reflection vector Cs of the same color as the light source is defined, and then the specular reflection component ms has a sufficiently large (non-zero) value within a limited range of the diffuse reflection component value mb of the component. To change. Pixel x,
In the diffuse reflection component value mb (x, y) corresponding to y, t1 <mb <t2 (Equation 37) (t1, t2 are positive real numbers from 0 to 1), and t1 <
With t2) as a domain, consider an appropriate convex function f such as a Gaussian distribution having a maximum value of 1 such that ms (x, y) = f (mb (x, y)) (Equation 38), and t1, The specular reflection component decomposed image ms (x, y) is created so that ms has an appropriate size in the range of t2.

Regardless of the material of the input object, the color vector C
The feeling of material can be finely corrected by changing the sizes of s and Cb, or by changing the spread of the specular reflection by raising the specular reflection component value to the nth power by the equations 18 and 21 as described in the operation. The narrower the extent of specular reflection by increasing the ratio of the specular reflection component or increasing n, the generally glossier the material.

(2) Metal object When the plastic is the input image, the plastic is more specular reflection component ms than the metal with respect to the geometrical conditions (face direction, viewpoint, light source direction) of each point on the object. Since the change of is drastic, for example, a lookup table that outputs by increasing the ms value of a small value is provided, and using this,
Make the change in the specular reflection component value gradual. Further, the magnitude of the specular reflection vector Cs is larger than that of the diffuse reflection vector Cb, and the metal color is defined as the reflectance (ros, gos, boss) in Expression 17, which is different from the color of the light source. Change to the specular reflection vector Cs.

(3) Metallic Object First, the same treatment as for metal is performed. That is, the component decomposition value is changed so that the change in the specular reflection component value becomes gentle, and the magnitude of the specular reflection vector Cs is larger than the diffuse reflection vector Cb, and the color of the metal is reflected. Change it to the color you specified. Next, as described in the operation, in the color space, the specular reflection component decomposition value ms corresponding to the diffuse reflection component decomposition value mb picked up at regular intervals or irregular intervals is set to a small value such as 0 It causes variations in the value of reflection. The ms to be converted can also be determined by the interval of pixel values in the image.
In the above, a random number may be used when deciding ms at an indefinite interval.

(4) Object such as cloth or rubber When the input image is plastic, the specular reflection vector Cs
It is sufficient to reduce the specular reflection component by reducing. In the case of cloth, in the next steps 106 and 107, the pattern of unevenness according to the weaving of the yarn is used to
A more realistic material feeling can be expressed by the processing described in "Expression of unevenness".

The component-separated image and color vector changed in step 105 are stored in the main memory of the computer 202.

At step 106, the surface shape is designated. For example, keyboard 2 while displaying on CRT 201
Designate fine irregularities on the surface by freely writing the cross section with 01. Alternatively, the sectional view is stored in the external storage device 205 and used.

In step 107, the component decomposition image is changed based on the shape of the unevenness designated in step 106, as described in the operation "(3) Expression of unevenness". By changing the component decomposition image, unevenness is given to the object region, and it seems that the surface shape is changed. A specific method of changing the component decomposition image will be described below.

First, as shown in FIG. 6, a method of rendering a concavo-convex shape and multiplying the resulting image by the component decomposition image will be described. The component decomposition image for diffuse reflection is generated as follows. The generation of the rendering image is calculated by setting the kd and the size of the light source I to 1 in Equation 12 and setting the incident direction of the light source I to the viewpoint direction (camera direction). Next, each pixel of the diffuse reflection image rendered by Equation 12 is divided by the maximum value of the pixel and normalized to a numerical value of 1 or less (Pb (u, w), where u (x, y), w (X, y)) to the computer 20
Diffuse reflection component decomposed image mb stored in main memory 2
Each pixel of (x, y) is multiplied to obtain a diffuse reflection component decomposed image. The same applies to the generation of the component decomposition image for the specular reflection component. That is, in Expression 16, both ks and the size of the light source I are set to 1, and the rendering direction is generated by setting the incident direction of the light source I to be the same as in the case of diffuse reflection. Each pixel of the rendered specular reflection image is divided by the maximum value of the pixel and normalized to a numerical value of 1 or less (P
s (u, w), where u (x, y), w (x, y))
Is multiplied by each pixel of the specular reflection component decomposed image ms (x, y) stored in the main memory of the computer 202 to obtain a specular reflection component decomposed image.

Secondly, a method of changing the surface shape by using the surface normal of the approximate three-dimensional object described in the operation will be described. At this time, after the diffuse reflection component decomposition value is determined, the specular reflection component decomposition value is determined by the following method. First, using the already determined diffuse reflection component decomposition value mb, ms corresponding to mb is obtained from the distribution in the color space of the object region in the image. Alternatively, in FIG. 7, the bump function f (u, w)
A specular reflection pattern is determined in advance by using the first method in correspondence with the shape pattern shown in, and this pattern is defined as ms of the original image. Or this pattern ms
It is reflected in ms by multiplying by.

When smoothness is designated in this step,
As described in "(2) Exponentiation of specular reflection component" of the action, the specular reflection component decomposition value ms is raised to the power of k so that n in Expression 16 increases as the smoothness increases. Regarding the diffuse reflection component, "(1) Smoothing the diffuse reflection component" of the action
As described above, the averaging process is performed in the vicinity of the pixel of interest by performing weighted addition or the like so that the normal line changes smoothly. For example, when the value of the target pixel x, y is C (x, y), the value C (i,
j) (i = x-1, x, x + 1, j = y-1, y, y +
In 1), they are averaged as in the following equation.

[0147] However, Is.

The component-decomposed image modified as described above is stored in the main memory of the computer 202.

At step 108, a single color is designated by the keyboard 203 or the mouse 204, or a texture image showing a pattern is selected.

In step 109, the diffuse diffuse reflection vector Cb and other color vectors are changed from the already designated color or pattern using the methods of the known example 3 and the prior art.

In step 110, based on the changed color vector and the component-separated image stored in the main memory of the computer 202, the changed pixel value is calculated by using Equation 9, and the changed image is reconstructed. . This image is CRT20
1, and the generated image is stored in the external storage device 205.

An alternative method is possible in which the component-separated image in this embodiment is used as an "image of a ratio to the standard color" (hereinafter, referred to as a ratio image) that represents a shade in the preceding example. In the prior art, the standard color is the standard color of the object area.
The standard color can be determined by the user inputting the RGB component values with the keyboard or by designating one point in the object area in the image with the mouse. This standard color C
The “ratio image” is obtained by calculating, for each pixel, the ratio F (x, y) of the value Cav corresponding to each pixel value C (x, y) of the object region with respect to r. Here, Cav is obtained by averaging pixel values in the vicinity of the pixels x and y. The image of the ratio to the standard color represents the shading. In the object region in the image with a small amount of specular reflection, an image of this ratio can be used as the component decomposition image of diffuse reflection in this embodiment.

[0153]

According to the present invention, it is possible to change the color, pattern, surface shape, and material appearance of an object area in a natural image, which is effective for design simulation.

[Brief description of drawings]

FIG. 1 is a flowchart showing an embodiment of the present invention.

FIG. 2 is a hardware configuration for implementing the embodiment.

FIG. 3 is an explanatory diagram of a conventional technique.

FIG. 4 is an explanatory diagram of geometric information at a point P.

FIG. 5 is an explanatory diagram of changes in boundary reflection due to n.

FIG. 6 is an explanatory diagram of a surface shape changing method 1.

FIG. 7 is an explanatory diagram of a surface shape changing method 2.

FIG. 8 is an explanatory diagram of a reflection model of plastic.

FIG. 9 is an explanatory diagram of a metallic reflection model.

FIG. 10 is an explanatory view of a reflection model of metal.

FIG. 11 is an explanatory diagram of a cloth / rubber reflection model.

FIG. 12 is an explanatory diagram of a component decomposition image.

FIG. 13 is an explanatory diagram of specular reflection and diffuse reflection.

FIG. 14 is an explanatory diagram of environmental colors

FIG. 15 is an explanatory diagram of capturing a natural image with a camera.

[Explanation of symbols]

 Reference numeral 101 ... Input of image 102 ... Extraction and retention of color vector 103 ... Extraction and retention of component decomposition image 104 ... Designation of material 105 ... Change according to material of color vector of component decomposition image 106 ... Designation of surface shape 107 ... Component decomposition image Change according to the surface shape of 108. Designation of color / pattern 109 ... Change of color 110 ... Reconstruction of changed image

Claims (25)

[Claims] 1. An image processing method for processing image data representing a color image stored in an image storage device by a computer, in a color space having a color component value representing a pixel of the image data as a coordinate system. A step of extracting a color vector representing a distribution of pixel values of a specific area in the image data, and a step of obtaining a component decomposition value obtained by dividing the pixel values of the specific area with the color vector as a component decomposition image, An image processing method comprising: a step of changing the component decomposed image; and a step of reconstructing a pixel value of the specific region based on the changed component decomposed image. 2. The image processing method according to claim 1, wherein the specific area is an object area and is distinguished from other areas by a mask for distinguishing individual pixels. 3. The image processing method according to claim 2, wherein the color vectors are a diffuse reflection color vector representing a diffuse reflection color of an object and a specular reflection color vector representing a specular reflection color. 4. The image processing method according to claim 1, further comprising a step of changing the color vector after changing the component-separated image and before reconstructing pixel values of the specific region. 5. In the process of changing the component decomposition image, with respect to the component decomposition value by the diffuse reflection color vector,
The image processing method according to claim 1, wherein pixel values in the vicinity of the pixel of interest are weighted and added. 6. The image processing method according to claim 1, wherein a power of a component decomposition value is obtained in the process of changing the component decomposition image. 7. In the process of changing the component decomposed image, the pixel value of the component decomposed image at the position of coordinates x, y is m.
When (x, y), by giving a function P (u, w) that is u (x, y), w (x, y), the following expression m ′ (x, y) = m (x, y) The image processing method according to claim 1, wherein the changed pixel value m ′ (x, y) is generated by) + P (u, w). 8. In the process of changing the component decomposition image, the pixel value of the component decomposition image at the position of coordinates x, y is m.
When (x, y), by giving a function P (u, w) that is u (x, y), w (x, y), the following expression m ′ (x, y) = m (x, y) 3. The image processing method according to claim 1, wherein the changed pixel value m ′ (x, y) is generated by * P * (u, w) (* represents a scalar product). 9. The image processing method according to claim 7, wherein the parameters u and w of the function P are u = x and w = y in the process of changing the component decomposition image. 10. The component decomposition value by the diffuse reflection color vector is changed in the process of changing the component decomposition image, and the component decomposition value by the specular reflection color vector is changed by using the changed component decomposition value. The image processing method according to claim 2. 11. The image processing method according to claim 10, wherein the component decomposition value based on the specular reflection color vector is changed by using a function having the component decomposition value based on the diffuse reflection color vector as a variable. 12. The image processing method according to claim 10, wherein the component separation value based on the specular reflection color vector is obtained from the component separation value based on the diffuse reflection color vector using a distribution of pixel values of the specific region in a color space. . 13. In the process of changing the component decomposition image, a specific pixel in the specific area is picked up,
The component decomposition value of the picked-up pixel by the specular reflection color vector is set to an intermediate value between the maximum value and the minimum value, and the component decomposition value of the non-picked-up pixel by the specular reflection color vector is set to almost the minimum value. 2. The image processing method described in 2. 14. The specific pixel in the specific region is a pixel extracted at regular intervals of the coordinate value of the pixel.
3. The image processing method described in 3. 15. The specific pixel in the specific area is a pixel extracted at every irregular interval of the coordinate value of the pixel.
3. The image processing method described in 3. 16. The image processing method according to claim 13, wherein the specific pixel in the specific region is a pixel extracted at constant intervals of component decomposition values by the dispersed reflection color vector. 17. The image processing method according to claim 13, wherein the specific pixel in the specific region is a pixel extracted at an indefinite interval of component decomposition values by the dispersed reflection color vector. 18. The image processing method according to claim 15, wherein the indefinite interval is obtained by a random number. 19. The image obtained by rendering the three-dimensional shape for a diffuse reflection component by giving a three-dimensional shape, a reflection model representing a material, and rendering the three-dimensional shape for the function P. The described image processing method. 20. The function P is an image obtained by giving a three-dimensional shape, a reflection model expressing a material, and rendering the three-dimensional shape for specular reflection components. The described image processing method. 21. The image processing method according to claim 19, wherein the three-dimensional shape is an object generated by processing a surface shape of a three-dimensional object serving as a model. 22. The function P includes a model object, a light source, and
Assuming a reflection model expressing the material, the component decomposition value when the three-dimensional shape is processed for the model object,
9. The method according to claim 7 or 8, which is generated from the difference between the component decomposition values before performing.
The described image processing method. 23. In the process of changing the component-separated image, the components of the specular reflection color vector of the pixels of the specific region are sufficiently large only for the pixels having the component decomposition value of the diffuse reflection color vector of the specific range. The image processing method according to claim 2, wherein the resolution value is given and the direction of the specular reflection color vector is made the same as the color of the light source. 24. In the process of changing the component decomposition image, for the pixels of the specific region, the component decomposition value by the specular reflection color vector is increased, the component decomposition value by the diffuse reflection color vector is decreased, and the specular reflection color vector is reduced. The method according to claim 2, wherein a change in the component separation value due to the color vector is moderated.
The described image processing method. 25. The image processing method according to claim 2, wherein, in the process of changing the component-separated image, the component-separated value by the specular reflection color vector is reduced for the pixels in the specific region.