JP2000201358A - Image data converting method and recording medium in which image data conversion program is recorded - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an object image to be included in an original image into parts to be easily reused. SOLUTION: Plural still images to be a moving image, are read (a step 300), a reference image is specified and encoded (steps 302, 304). Next, the next still image is read (a step 306) and difference between the reference image and the still image is calculated (a step 308). A change object is specified by using a calculated differential value (a step 310), vector information is extracted (a step 312) and the change object is encoded for the read still image (a step 314). This processing is executed for all of plural still images to be an object as the moving images (by repeating each processing until it is judged as affirmative in a step 316). When the processings of all still images are completed, moving image data is generated by synthesizing pieces of data of each encoded still image (a step 318).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for converting image data and a recording medium on which an image data conversion program is recorded, and more particularly to a method for converting a plurality of original images into image data capable of presenting a moving image. The present invention relates to an image data conversion method and a recording medium that stores an image data conversion program.

[0002]

2. Description of the Related Art In recent years, with the development of computer technology, color images have been digitized as image data, and the image data may be stored or distributed for use. The digitized image data includes image data representing a photographed image generated by photographing with a digital photographing device such as a digital camera, and image data representing a scanned image generated by scanning a color original with a scanner. Etc.

By the way, it is known that a moving image can be presented by presenting a plurality of continuous still images at regular intervals. Therefore, if a plurality of continuous digitized image data are presented at regular intervals, a digitized moving image can be presented.

However, when a color image is digitized as image data, the data capacity occupied by one still image is large, so that the data capacity increases in accordance with the number of still images presented as moving images. For this reason, a large-capacity storage device must be prepared. Further, in order to present each digitized still image as an image, image conversion for presentation is required for each image, and it is necessary to execute the conversion in real time to present it as a moving image. For this reason, the apparatus becomes large-scale and requires high-speed processing.

To solve this problem, there is the so-called MPEG standard. According to the MPEG standard, for a moving image composed of a plurality of still images, a difference or the like between successive still images is obtained with the first still image as a reference (reference still image), and the difference value or the like is stored instead of a continuous still image. ing. By doing so, the storage capacity can be reduced, and a moving image can be presented in real time.

[0006]

However, the development based on the difference between successive still images is effective for approximate still images, but may involve a large motion in an image or a case where a different element is newly inserted. Not suitable. In addition, when a change in resolution such as enlargement / reduction is accompanied, a sense of incongruity may occur.

In view of the above, an object of the present invention is to provide an image data conversion method capable of reproducing a moving image composed of a plurality of continuous images without a sense of incongruity, and a recording medium storing an image data conversion program. It is.

[0008]

According to an aspect of the present invention, there is provided an image data converting method for representing a moving image by using image data including color information of a plurality of continuous original images. An image data conversion method for converting into image data, wherein discontinuity information of an original image representing discontinuity of color information represented by a line process is generated based on image data of the original image, and the generated discontinuity is generated. Extracting the outline of the object image included in the original image based on the information, obtaining outline information obtained by outlining the outline of the object image based on the discontinuity information, and extracting the object image using the extracted outline. The texture information to be represented is obtained, and the image data is encoded including the outline information and the texture information.

According to a second aspect of the present invention, there is provided the image data converting method according to the first aspect, wherein the outline of the object image is outlined by a parametric curve, and the color information is encoded by a parametric curved surface. Features.

According to a third aspect of the present invention, there is provided the image data conversion method according to the first or second aspect, wherein the discontinuity information includes an outer peripheral irregularity corresponding to a contour of an object image included in the original image. It is characterized in that the continuous information and the internal discontinuity information inside the object image are generated as the discontinuity information of the original image.

According to a fourth aspect of the present invention, there is provided the image data conversion method according to any one of the first to third aspects, wherein a reference original image is selected from the plurality of continuous original images. Determined, the coding for the determined reference original image, obtained vector information representing the variation of the object image included in the reference original image for the other original image, the image data and vector information encoded for the reference original image To image data representing a moving image.

According to a fifth aspect of the present invention, there is provided a recording medium having recorded thereon an image data conversion program for converting image data including color information of a plurality of continuous original images into image data representing a moving image by a computer. Wherein the image data conversion program generates discontinuity information of the original image representing discontinuity of the color information represented by the line process, based on the image data of the original image including the color information. The outline of the object image included in the original image is extracted based on the discontinuity information, and the outline information of the outline of the object image is obtained based on the discontinuity information, and the extracted outline is used. To obtain texture information representing the object image, and to encode image data including the outline information and the texture information. To.

According to a sixth aspect of the present invention, there is provided a recording medium storing the image data conversion program according to the fifth aspect, wherein the outline of the object image is outlined by a parametric curve, and the color information is a parametric curved surface. Is encoded.

According to a seventh aspect of the present invention, there is provided a recording medium storing the image data conversion program according to the fifth or sixth aspect, wherein the discontinuity information is a contour of an object image included in an original image. Is generated as discontinuity information of the original image.

According to an eighth aspect of the present invention, there is provided a recording medium on which the image data conversion program according to any one of the fifth to seventh aspects is recorded, wherein: The reference original image is determined, the determined reference original image is coded, the vector information representing the variation of the object image included in the reference original image is obtained for the other original images, and the code is obtained for the reference original image. The image data is converted from image data and vector information into image data representing a moving image.

In the image data conversion method according to the first aspect, image data including color information of a plurality of continuous original images is converted into image data representing a moving image. On the basis of the color information of the original image, discontinuity information of the original image representing discontinuity of the color information represented by the line process is generated. The line process represents discontinuity of color information, that is, whether color information is continuous or discontinuous. An energy function is defined by the color information and the line process, and energy is minimized using the energy function. Then, the line process has a value at a place where the color information is discontinuous in the original image. The discontinuity information of the original image can be represented by a line process having this value. Since the discontinuity information indicates a place where the color information is discontinuous in the original image, it becomes a boundary of the color information in an adjacent color region formed of different color information which is not formed of the same or similar color information.

Note that color information can be used as density information when dealing with a single color. An example is image information of a binary image. Accordingly, discontinuity information appears at the outline of a color region composed of the same or similar color information. The object image included in the original image may be configured with the same or similar color information, or may be configured with a plurality of predetermined color information. Therefore, the contour of the object image included in the original image is extracted based on the discontinuity information.

Next, outline information in which the outline of the object image is outlined based on the discontinuity information is obtained. The outline information is information whose size can be changed, for example, enlarged or reduced, while maintaining most of the original information (that is, information on the outline). For example, the outline information includes shape information composed of points and vector information such as line segments and surface information. Note that this shape information can also be described by mathematical expressions. In this way, by describing the outline of the object image with the outline information, the outline of the object image can be expressed without depending on the resolution.

The color information of the object image is included in the outline of the object image described by the outline information, and can be encoded by the expression of the continuous information when the object image is in a continuous state without discontinuity. Therefore, texture information representing an object image is obtained using the extracted contour. This texture information includes
There are bitmap data and image data formats.
An example of the image data format is encoding that approximates color information, for example, and includes an image data format known as JPEG or the like. The image data is encoded including the outline information and the texture information.

As described above, since the outline of the object image is described by the outline information and the image data is encoded including the texture information of the original image or the object image, a plurality of original images are reproduced, that is, presented continuously. In this case, the image data can be converted into a format that can be used in a form that maintains the contour of the original object image without depending on the resolution.

In addition to obtaining outline information,
Based on the discontinuity information, the degree of penetration indicating the degree of penetration when the object image is combined with another image may be obtained. This degree of penetration represents the degree of penetration of the object image with respect to the portion on the original image corresponding to the discontinuity information,
By using it at the time of image synthesis or enlargement / reduction, the original image can be enlarged / reduced or the object image can be blended into another image without causing a sense of incongruity at the boundary portion. As a result, when a synthesis operation such as pasting to another image or enlargement / reduction is performed, the contour does not cause a sense of discomfort at the boundary between the object image and the other image.

When an image is represented, its outline is the outermost periphery of the image and is a two-dimensional curve. The color information is
It is included in the outline of the image and becomes continuous. The color information can be three-dimensionally represented by its position and density for each color. Therefore, continuous color information can be made to correspond to a three-dimensional curved surface. It is preferable that such curves and curved surfaces are expressed in a format that is easy to handle.

According to the second aspect of the present invention, the outline of the object image is outlined by a parametric curve, and the color information is encoded by a parametric curved surface. By doing so, the contour and color information of the object image can be expressed in a simple description format using mathematical expressions, and can be easily used.

Here, most of the outline of the object image included in the original image exists near the outer periphery, but has a characteristic inside the object image, or only the inside of the object image slightly changes in shape. Or may be. For example, in a person image, parts such as eyes, nose, mouth, etc. have characteristics, and the outline of the head does not substantially change, but the facial expression of the person changes due to changes in parts such as eyes, nose, mouth, etc. May be.

According to the third aspect of the present invention, as the discontinuity information, the outer peripheral discontinuity information corresponding to the contour of the object image included in the original image and the internal discontinuity information inside the object image are converted into the original image. Is generated as the discontinuity information. in this way,
By describing the outline and the internal discontinuity information of the object image by the outline information, the outline and the internal discontinuity representing the feature of the object image can be extracted and can be expressed without depending on the resolution.

By the way, when image data including color information of a plurality of continuous original images is converted into image data representing a moving image, outline information representing an outline of an object image and outline information representing the outline of the object image or an If the image data is encoded including the texture information of the image, the processing load increases.

According to a fourth aspect of the present invention, a reference original image is determined from a plurality of continuous original images, the determined reference original image is encoded, and other original images are included in the reference original image. Is obtained, and image data representing a moving image is converted from the image data coded for the reference original image and the vector information. The object image may be composed of a plurality of element images. For example, the background may not change, but only the person may move, or only the expression of the person may change. In this case, it is efficient to store only the changing image.
Therefore, by encoding the reference original image and obtaining vector information indicating the variation of the object image included in the reference original image for the other original images, the capacity of the image data representing the moving image can be reduced. In addition, the processing load for converting the encoded reference original image image data and vector information into image data representing a moving image is reduced.

The encoding of the original image by the image data conversion method can be realized on a computer by executing an image data conversion program recorded on a recording medium according to the present invention. In detail, a recording medium recording an image data conversion program for converting into image data representing a moving image using image data including color information of a plurality of continuous original images by a computer, wherein the image data conversion The program generates discontinuity information of the original image indicating discontinuity of the color information represented by the line process based on the image data of the original image including the color information, and generates the discontinuity information of the original image based on the generated discontinuity information. The outline of the object image included in the image is extracted, and the outline information of the outline of the object image is determined based on the discontinuity information.The extracted outline is used to obtain texture information representing the object image. Then, the image data is encoded including the outline information and the texture information. Thus, on a computer, when reproducing a plurality of original images, that is, presenting them successively, the outline of the object image can be used in a form that maintains the outline of the original object image without depending on the resolution. Can convert the image data.

According to a sixth aspect of the present invention, the contour of the object image is encoded so as to be outlined by a parametric curve, and the color information is encoded so as to be encoded by a parametric surface. Can be.

[0030] As described in claim 7, the discontinuity information includes outer periphery discontinuity information corresponding to the contour of the object image included in the original image and internal discontinuity information inside the object image. , Can be generated as discontinuity information of the original image.

Further, as set forth in claim 8, a reference original image is determined from the plurality of continuous original images, the determined reference original image is encoded, and the other original images are encoded. Vector information representing the variation of the object image included in the reference original image can be obtained, and image data representing a moving image can be converted from the image data and vector information encoded for the reference original image.

[0032]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, the present invention is applied to an image conversion device that generates image data representing a moving image from a plurality of color original images.

As shown in FIG. 2, an image conversion apparatus 10 of the present embodiment includes a display device 12 such as a display device for displaying an image, and an input device such as a keyboard for inputting commands and data from the outside. 14, the device body 16,
And an image reading device 30 for reading a color original image from a color original.

The apparatus body 16 includes a CPU 18, a RAM 2
0, a ROM 22, and an input / output port (I / O) 28, each of which is connected by a bus 26 so that commands and data can be exchanged. The ROM 22 stores a processing routine to be described later, which is executed in the apparatus main body 16.

A memory 24 for storing image data is connected to an input / output port 28 of the apparatus main body 16. The input device 14 and the display device 12 are connected to the input / output port 28 of the device main body 16. An image reading device 30 such as a color scanner is connected to the input / output port 28.

The image reading device 30 can read a multi-valued color original image from a color original such as a printed matter, or a photographic film in which a negative image or a positive image is developed after photographing a subject.

The input / output port 28 is connected to a floppy disk unit (hereinafter referred to as FDU) into which a floppy (registered trademark) disk (hereinafter referred to as FD) as a recording medium can be inserted and removed. Note that processing routines and the like described below can be read from and written to the FD using the FDU. Therefore, the processing routine to be described later
Instead of storing it in M22, record it in the FD in advance,
A processing program recorded in the FD via the FDU may be executed. Also, a large-capacity storage device (not shown) such as a hard disk device is connected to the device main body 16, and the processing program recorded in the FD is stored (installed) in the large-capacity storage device (not shown) and executed. Is also good.
As a recording medium, there is an optical disk such as a CD-ROM or a magneto-optical disk such as an MD or MO.
A ROM device, an MD device, an MO device, or the like may be used.

In this embodiment, as an example, a case where a multi-valued color original image is input by an image reading device 30 such as a color scanner will be described. However, the present invention is not limited to this. Image data stored in advance in a storage medium such as an FD may be input. Also, a signal exchange device such as a network board may be connected to the input / output port 28 to form a so-called LAN that enables signal exchange with other devices, and image data may be received from other devices. Good.

The multi-valued color original image includes an image such as a real image or a natural image from a monochrome image or a binary image formed by combining different colors.

Next, the operation of the present embodiment will be described. First, encoding of a still image as an original image in image conversion will be described. The following processing (encoding processing routine) may be provided as a computer-executable application stored in a recording medium such as an FD and executed by an execution instruction.

First, in step 100, a color original image (hereinafter, referred to as an original image) is read by reading a color original placed on the image reading device 30.
In the present embodiment, it is assumed that the RGB data is output from the image reading device 30 and the output RGB data is used. When the image data is image data represented in another color system such as CMYK data,
What is necessary is just to perform RGB conversion.

FIG. 4 shows an original image 40 that has been read.
An example was shown. In the original image 40, flowers 50 are scattered around a person 52, a plant 48 is located in the vicinity thereof, a mountain 42 and a small mountain 44 are located far (above the image), and a cloud 46 is located above. ing.

In the next step 102, discontinuity information is detected from the original image by a line process. It is a virtual element that indicates line process (line element) discontinuity. In the present embodiment, a case where discontinuous information of an image is detected by a neural network using a line process will be described. First. The line process will be described in detail.

FIG. 5A shows an example of one-dimensional data obtained from the relationship between the position and the density of the pixel. The points P1 to P6 and the points P7 to P12 are located substantially in the vicinity. There is a large distance between P6 and point P7. When obtaining a curve along these one-dimensional data by an approximation method such as the least squares method or an interpolation method, as shown in FIG. 5B, points P1 to P6 and points P7 to P12 are continuously formed by a curve. Therefore, the curve 34 is required to be far from the points P6 and P7 but to be smooth.

However, since the points P6 and P7 are far away from the curve 34, the obtained characteristics (ie, the curve 34) do not conform to the actual data. Therefore, as shown in FIG. 5C, curves 34a and 34b are obtained along points P1 to P6 and points P7 to P12 located substantially in the vicinity. When obtained in this manner, a discontinuity occurs between the point P6 and the point P7 which are far apart. A discontinuous point 3 is located between the point P6 and the point P7.
If the curves 34a and 34b are determined together with 6, the characteristics according to the actual data can be obtained. That is, the line process indicating that the discontinuity occurs at the discontinuity point 36 at the position between the points P6 and P7 is turned on.
Further, between each of the points P1 to P6 and the points P7 to P12, that is, the points P1 and P2, the points P2 and P3,.
5, point P6, point P7 and point P8, point P8 and point P9,.
.. The line process is turned off between each of the points P11 and P12.

Although the case of one-dimensional data has been described above, when a line process is applied to an image represented by two-dimensional data, the line process is defined as a virtual function between two pixels. Therefore, when the defined line process is turned on or off by a local density difference, the discontinuous portion in the image can be extracted as discontinuous information, and the connected portion of the discontinuous portion can be extracted as the contour of the object image. it can.

Using this line process, the energy function is calculated and minimized by a neural network to obtain discontinuity (information) of the image. This will be described in detail.

FIG. 6 is a conceptual configuration diagram for explaining a neural network using a line process. FIG.
The neural network, as shown in, three neuron f _i for one pixel corresponding to the image when the input is an _{image, j, h i, j,} v i, j correspond.

Where f _{i, j} is the value of the pixel,
h _{i, j} and v _{i, j} are hidden functions called line processes indicating the existence and non-existence of the discontinuity between f _{i, j} . That is, f _{i, j} represents the intensity change of the input value, h _{i, j} , v
_{i, j} are the continuity of f _{i, j in} the horizontal and vertical directions _, respectively.
Indicates discontinuity.

With these three variables f _{i, j} , h _{i, j} and v _{i, j} , the energy function of the whole system is defined as shown in the following equation (1).

[0051]

(Equation 1)

Where E _I : the continuity of the curved surface (depth) data E _D : the reliability of the observation data (initial value) E _V : the line process goes to the angle (0 or 1) of the hypersphere E _P : the parallel near E _C : Condition for a single line process to be 1 E _C : Condition for a single line process to be 1 E _L : Condition for preferring a continuous multi-line process and dislike crossing and discontinuity E _G : Equation (2) conditions for m, n does not diverge in the _{C D, C V, C P} , C C, C L, C G: parameter value g (): sigmoid function d _{i, j:} initial value the time law of variable It was defined by the following equation (2).

[0053]

(Equation 2)

Here, g (U _i ) = 1 / e (−2λU _i ) where e (−2λU _i ) is an exponential function, and the inside of () represents an exponent part hi _{, j} = g (m _{i , j} ) v _{i, j} = g (n _{i, j} ) m, n: Internal state variables Examples of calculation of the partial derivative of the above equation (2) are shown in the following (3), (4),
Equation (5) shows.

[0055]

(Equation 3)

The calculation results of the above equations (3) to (5) are small.
That is, it is close to 0 or takes a value of 0.
Is the adjacent value f_{i, j + 1}And the value f_{i, j}, And f_{i + 1, j}
And f _{i, j}Have substantially the same value. Therefore,
Energy E expressed by equation (1)_IFor f_{i, j + 1}≒
f_{i, j}, And f_{i + 1, j}≒ f_{i, j}, Energy
Lugi E_IIs relatively small, so the line process h
_{i, j}, V_{i, j}Need not have a value, so h_{i, j}, V_{i, j}
Is a relatively small value.

[0057] On the other hand, when adjacent values f _{i, j + 1} and the value f _{i, j,} and the value f _{i + 1, j} and the value f _i, the difference between _j is large, that the boundary between adjacent values When there is (fi _{, j + 1} -f
_{i, j} ) ² and (fi _{+ 1, j-} fi _{, j} ) ² are large.
Therefore, to reduce the energy E _I , hi _{, j} or vi _{, j} has a value, and (1-hi _{, j} ) or (1-
v _{i, j} ) needs to be relatively small. Thus, adjacent values fi _{, j} and fi _{, j + 1} or fi _{, j} and f
If there is a difference in _{i + 1, j} , the line processes h _{i, j} , v _{i, j} between the values have values, and a boundary line appears between regions of different colors. Become.

The conditions under which the line process occurs from this coefficient
Decide the matter. As an example, the single line shown in FIG.
The condition under which the process occurs is E_P= 0, E_C= 0, E_L
= 2C_L, The continuous line process shown in FIG.
The condition that occurs is E_P= 0, E _C= 0, E_L= C_L, FIG.
The condition in which parallel line processes shown in (C) occur.
The case is E_P= C_P, E_C= C_C, E_L= 2C_LAnd
Line processes crossing each other as shown in FIG.
Is a condition that E_P= 0, E_C= 2C_C, E_L= 10
C_LIt is.

According to the above equations (1) to (5), the result of the energy learning in which the calculation for minimizing the energy of the entire system is repeated is the solution of the neural network for the given input.

In this neural network, (a) when the input is an image to which noise is added, f _{i, j} at the obtained minimum energy corresponds to the restored image, and (b) the input is the brightness image. In the case, h _ij and v _ij at the obtained minimum energy correspond to the contour, and (c) when the input is survey data such as a mountain,
The obtained minimum energy f _ij indicates the altitude of each point estimated from the survey points. In the present embodiment,
This is an example used for image input of (b).

The above neural network has expandability that can be applied to various inverse problems depending on what is assigned to a variable. Since the algorithm is realized by local computation, parallel processing hardware such as light is used. It also has the advantage that the device can be easily formed and high-speed processing can be performed.

Therefore, in step 102 of FIG. 3, the energy function and its minimization are performed by the neural network using the above-described line process, and the discontinuity is detected to extract the contour portion (discontinuous) of the image. You. That is, when the internal state variables are updated according to the equation (2), the total energy of the equation (1) always decreases and the minimum value is calculated. H _ij and v _ij obtained at the minimum correspond to discontinuities (contours) in the horizontal and vertical directions.

Next, in step 104, a color area of a similar color in the original image is set. The similar color setting process is a process of labeling, for example, a color or a color group specified by a combination of similar colors, the same color, and a predetermined color on the original image. For example, for each pixel on the original image, a color space of a coordinate system (hereinafter, referred to as an HLS color space) including a hue value axis, a saturation value axis, and a lightness value axis that are orthogonal to each other.
There is a method of performing an integration process or the like by an iterative type area expansion method based on the distance obtained in the above.

Specifically, first, any one of the original images
Select one pixel. Next, one pixel around the pixel (so-called one pixel in the vicinity of 8) is selected, and when the selected pixel is already included in an area to which any label has been assigned, the label is changed. Give. On the other hand, when the two pixels are not included, the distance between the two selected pixels in the HLS color space, that is, the similarity of lightness, saturation, and hue of the two pixels is obtained. The longer the distance, the lower the similarity, and the shorter the distance, the higher the similarity. When the distance is less than the predetermined value, the two pixels are considered to be similar, and the same label is assigned to the two pixels (labeling). When the value is equal to or more than the predetermined value, it is determined that the objects are not similar and without labeling.
The above processing is executed for other pixels in the neighborhood of 8. When all the pixels in the vicinity of 8 have been completed, the outermost one pixel of the area (in the original image) to which the same label is added is selected, and the above processing is repeatedly executed. The above processing is executed for all pixels. By sequentially executing the same label assignment for pixels having high similarity from the outermost peripheral pixel of the area assigned with the same label, a color area is set in the original image by pixels having similar hue, lightness and saturation. I do. In addition,
For the pixel group to which the same label is assigned, the respective average values of the hue value, the saturation value, and the brightness value are obtained, and the obtained average values are used as the hue value, the saturation value, and the brightness value of the label.

The color area specified by the same label is
Since there is no discontinuous portion, at least the discontinuity, that is, the contour detected in step 102 is not divided.

As described above, after the color area is set in step 104 based on the similarity of hue, lightness, and saturation for the adjacent pixels of the original image, in the next step 106, the original image is classified. You. This classification is a process of determining a relationship between similar color regions set in the original image. That is, pixel groups with the same or similar hues, lightness, and saturation, which are color regions to which one label is assigned, are likely to be the same object, but are not adjacent to each other and are located on the original image at distant positions. Are likely to be objects of the same type. Therefore, values and ranges of hue, lightness, and saturation for classifying (segmenting) similar color regions are determined in advance, and the original image is classified into classes by classifying the color regions according to the determined values and ranges. Divide.

In the present embodiment, an example will be described in which a relationship is determined between color regions of similar colors set in the original image and classification is performed, but the discontinuity information corresponds to the outline of the object image. Therefore, the classification can be performed based on the similarity of the discontinuous information (contour).

Specifically, taking the original image 40 shown in FIG. 4 as an example, a landscape class CA including a mountain 42, a small mountain 44, and a cloud 46, an environment class CB including scattered flowers 50 and plants 48, Classification into a person class CC including the person 52 is possible.

In the next step 108, an image is extracted for each classification. Specifically, taking the original image 40 shown in FIG. 4 as an example, the landscape class CA includes a mountain 42, a small mountain 44,
And a class image 54 including the cloud 46.
This class image 54 includes a mountain 42, a small mountain 44, and a cloud 46.
Are maintained as they are, and a large area is set so that these images are not divided at least. The background on the original image 40 may be left as it is in the class image 54, or the image may be pasted on a new background, for example, a predetermined background color. Similarly, the environment class CB is extracted as a class image 56 including the scattered flowers 50 and the plants 48, and the person class CC is extracted as a class image 58 including the person 52.

A class image extracted as described above may include a plurality of object images. Therefore, although a detailed specific example will be described later, first, in step 110, one of the class images extracted in step 108 is designated. Next, in step 112, one object image is extracted, and in the next step 114
Outlines discontinuity information using a Bezier curve. When the outline is completed, the next step 115
In, the degree of penetration of the object image into the background is determined using the discontinuity information.

In the next step 116, the object image is described by the outline of the outline, the degree of penetration, and the color in which the color information (texture information) of the object image is encoded. In the next step 118, it is determined whether or not all the object images included in the class specified in the above step 110 have been extracted, and when an unextracted object image remains, the determination is negative and the process returns to step 112. When the extraction of all object images is completed, the process proceeds to the next step 120. In step 120, it is determined whether or not all of the above-described processing of the class images included in the original image has been completed. This routine ends when the processing is completed.

When the texture information is stored in advance as the information including the color information of the object image, the processing such as the classification is unnecessary, and the texture information may be used as it is.

Next, the processing after step 110 in FIG. 3 will be specifically described with reference to FIG. The class image 54 of the landscape class CA shown in FIG. 4 includes object images of the mountain 42, the small mountain 44, and the cloud 46. These object images are separated from the class image 54 and extracted. This separation extraction
This is done using discontinuous information. Since the discontinuous information corresponds to the contour of the object image, an area surrounded by the discontinuous information is set as the object image. The region surrounded by the discontinuity information is a closed region or a closed space formed by a curve or a curved surface connecting discontinuous points corresponding to the contour portion of the object image,
It is not limited to one closed region or closed space. That is, it may be a group including a plurality of closed regions and closed spaces. In addition, as described later, a closed region or a closed space may be generated by a curve or a curved surface obtained by connecting a parametric curve or a curved surface, as will be described later.

For example, in the class image 54, three areas surrounded by discontinuous information are set. The three regions are sequentially set, and the discontinuity information and the color information of the region surrounded by the discontinuity information, that is, the hue, lightness, and saturation values of the pixels in the region are extracted. The image 62 and the cloud image 64 are extracted. In the example of FIG. 4, for each of the mountain image 60, the mountain image 62, and the cloud image 64, an image obtained by adding a background image to each extracted image is set as an object image.

Similarly, from the environment class CB, the flower image 6
6 and the plant image 68 are extracted, and from the person class CC,
A head image 70, an arm image 72, a body image 74, and a leg image 76 of the person are extracted.

Next, the outline (discontinuous information) of the extracted image
Is outlined as follows. In order to outline discontinuous information, a parametric curve that can be easily described is adopted. In particular, in the present embodiment, a Bezier curve that is easy to handle among parametric curves is employed. Therefore, the outline (discontinuous information: h _ij , v _ij ) of the extracted image is binarized and outlined using a Bezier curve. An equation that determines this Bezier curve P (t) is shown in the following equation (6).

[0077]

(Equation 4)

[0078] ^{_{However, B i n (t) =}} (1-t) · B i n-1 (t) + t · B i-1 n-1 (t) B j n (t) = 0 (j ≒ n ) = 1 (j = n) P _i : control point (discontinuous point) t: parameter

By outline forming the outline portion of the object image in this way, even if the object image is enlarged or reduced, the outline shape is maintained. By describing the object image with the outline that has been converted into outlines and the color based on the color information (preferably encoded) of the object image, the object image can be described in a reusable format (componentized). Become.

Next, the degree of penetration of the object image into the background is determined using discontinuous information corresponding to the contour of the extracted image. In the present embodiment, the horizontal penetration Mh (i
j) and the vertical penetration Mv (ij) are determined using the following equation (7).

Mh (ij) = β (m _ij ) Mv (ij) = β (n _ij ) (7) where β () is a nonlinear function such as a sigmoid function or a linear function 0 ≦ Mh (ij) ≦ 10 0 ≦ Mv (ij) ≦ 1 By determining the degree of penetration of the object image in this way, even when the object image is enlarged or reduced, the outline of the object image can be blended into the background. become.
The object image is described in a format that can be reused without a sense of incongruity by describing the object image with the outline that has been converted into outlines, the degree of penetration, and the color based on the color information (preferably encoded) of the object image. (Parts).

The penetration degree of the object image will be further described by taking the mountain image 60 shown in FIG. 10A as an example of the object image obtained by outlining the outline portion of the object image and calculating the penetration degree. The ridgeline 60A of the mountain represents a contour portion and has a degree of penetration. As understood from the above equation, the degree of penetration gradually changes between the maximum value and the minimum value. FIG. 10B is an enlarged view of the region Ar, and shows a transition of the density change of the ridgeline 60A by contour lines. FIG. 10C is a sectional view taken along the line ii in the region Ar (FIG.
(See (B)). As shown in FIG. 10B, the mountain image 6 shown in FIG.
At 0, referring to an area Ar including the mountain ridgeline 60A, the vicinity of the ridgeline 60A, which is the boundary of the original mountain image, is the ridgeline 6A.
As the distance from 0A increases, the density gradually decreases so as to dissolve into the background 60B (see FIG. 10C). Therefore, the contour portion of the object image gradually blends into the background without forming a clear ridge line.

In the above description, an example has been described in which an object image is described by outlines that have been converted into outlines and colors based on the color information of the object image. However, the present invention is not limited to this. Information for the operation may be added.
For example, when the object image includes an image such as a line drawing or a pattern, or when the object image has been subjected to a certain process such as a mask process or a focus process, the image or the process may belong thereto. That is, the description about the object image is
It is also possible to subdivide it by a pattern such as a line or gradation, or to subdivide it by the applied processing.

In the above description, the case where a parametric curve is used has been described. However, the present invention is not limited to the parametric curve, and another approximate curve may be used.

Next, the case of enlarging a componentized object image will be described. In the following description, the color information of the object image is set to N
A case where encoding is performed using a URBS (Non Unifom Rational B-Spline) surface will be described as an example. The following equation (8) shows an equation that determines NURBS as this parametric surface.

[0086]

(Equation 5)

Hereinafter, referring to FIG. 8 and FIG.
The case where the resolution is doubled (the area ratio is quadrupled) using the RBS will be described.

At step 200 in FIG. 9, an original image corresponding to step 100 in FIG. 3 is read. In the next step 202, the read original image is subjected to discontinuity detection by a line process corresponding to step 102 in FIG.
The outline corresponding to step 114 of FIG. In the next step 205, the calculation of the degree of penetration corresponding to step 115 in FIG. 3 is performed.

Next, at step 206, the discontinuous information that has been outlined is enlarged, and at the next step 207, the degree of penetration is enlarged (details will be described later). In the next step 208, the resolution is increased using the expanded discontinuous information and the parametric surface. Then, in the next step 210, an image whose resolution is increased is output.

FIG. 8 shows that an image 80 is composed of four pixels (16 pixels in total) that form a part of the original image, and each pixel is defined as a control point P _ij (0 ≦ i ≦ 3, 0 ≦ j ≦ 3). The curve 82 represents the discontinuity information extracted by the line process described above,
A curve obtained when the image is directly enlarged, and a curve 84
Is a curve obtained when the above-described discontinuous information is outlined and then enlarged.

The image has half the white pixels and the other black pixels.
Assume that each pixel value has a control point P₀₀, P_Ten,
P₂₀, P₀₁, P₁₁, P₂₀Is white, control point P₃₀, P_{twenty one},
P₃₁, P ₁₂, P_{twenty two}, P₃₂, P₀₃, P₁₃, P_{twenty three}Is a black value
Is set.

In this embodiment, a new pixel is added as a virtual point S within the control points P _{00 to} P ₃₃ at the time of the above enlargement. For example, a virtual point S _mn (0 ≦ m ≦ 3, 0 ≦ n ≦ 3) is added in a range surrounded by the control points P ₁₁ , P ₁₂ , P ₂₁ , and P ₂₂ .

The pixel values of these added virtual points S are
With reference to the pixel value of the original image around (the pixel value of the control point)
Desired. This embodiment considers discontinuities in the original image
Thus, the pixel value is determined. That is, it is discontinuous information
The pixel value of the pixel exceeding the curve 84 is not used.
For example, the virtual point S₂₀When calculating the pixel value of the control point P ₁₁
Is the virtual point S₂₀The straight line that connects
Since there is no control point P_{twenty two}Is the virtual point S₂₀And
Since the connecting straight line intersects with the curve 84, it is not referred to. More details
Specifically, the weight in the equation (8) representing NURBS is
By setting it to “0”, it is reflected in the calculation. By this
And the virtual point S₂₀Are the values of the white pixels. Likewise
To calculate pixel values by calculating other virtual points.

As described above, when discontinuous information is outlined and enlarged, an edge with reduced blur can be obtained. As a result, jagged edges (so-called jagged edges) at the time of image enlargement can be suppressed.

Next, the degree of penetration is determined using the discontinuity information. The degree of penetration exists only in discontinuous information existing between pixels before enlargement. Therefore, first, the degree of penetration between the pixels after the enlargement, that is, a value for discontinuity (discontinuous information) is obtained.

The penetration before the enlargement is determined by the penetration Mh ₂₀ based on the discontinuity information existing between the control points P ₂₀ and P ₂₁ corresponding to the curve 82 and the discontinuity existing between the control points P ₁₁ and P _12. Penetration degree Mh ₁₁ by information, penetration degree Mv ₁₁ by discontinuous information existing between control points P ₁₁ and P ₂₁ , vertical penetration degree Mv by discontinuous information existing between control points P ₀₂ and P _{12. 02} .

Next, the degree of penetration corresponding to the enlarged curve 84 is determined. This penetration degree is weighted by the distance between adjacent penetration degrees to determine a new penetration degree. The new penetration degree is as follows (9), (10)
It can be obtained by at least one of the equations.

Mv _ik = α · Mv _i + β · Mh _j (9) Mh _ik = γ · Mv _i + ε · Mh _j (10) where, i: enlargement in the horizontal direction closest to the reference side A value representing the position of the previous penetration degree j: A value representing the position of the penetration degree immediately before the reference side in the vertical direction before the enlargement k: A value representing the position of the penetration degree after the enlargement α, γ: the horizontal direction in FIG. Coefficients β and ε representing the proportional relationship of the distance in FIG. 8: the coefficients representing the proportional relationship of the distance in the horizontal direction in FIG. 8 Specifically, the penetration degree Mv _11-1, M around the virtual point S ₂₀
v _11-2 and Mh _20-1 can be obtained as follows.

[0099] _{Mv 11-1 = 0.4 · Mv 11 +0.6} · Mh 20 Mv 11-2 = 1.0 · Mv 11 +0.0 · Mh 20 Mh 20-1 = 0.25 · Mv 11 +0. The 75 · Mh ₂₀ penetration degree Mv _11-1 considers the position Pp between the virtual point S ₂₀ and the control point P ₂₁ as a reference of the distance, and the penetration degree Mv ₁₁ ,
Determining the distance ratio to each penetration degree Mh _20. When the distance between the control point is assumed as "1", the distance ratio to penetration of Mv ₁₁ is 1/3, the distance ratio to penetration of Mh ₂₀ is 1/2. If these sums are normalized so as to be “1”, α = 0.4 and β = 0.6. Other degrees of penetration can be determined in the same manner.

Here, when the degree of penetration is not used,
For example, when determining the pixel value of the virtual point S _20, the control point P ₁₁
Although references since the straight line connecting the virtual point S ₂₀ does not intersect the curve 84, the control point P ₂₂ does not refer since the straight line connecting the virtual point S ₂₀ intersects the curve 84. In detail, when referring to another control point, it indicates NURBS (8)
The weight w _ij in the equation is set to “1”, and when the reference is made, the weight w _ij is set to “0” to be reflected in the calculation. Thus, the pixel value of the virtual point S ₂₀ becomes a value of white pixels.

On the other hand, in the present embodiment, the degree of penetration is used. Therefore, the weight w _ij in the equation (8) is changed to the following (1)
Equations (1) and (12) are used. However, the degree of penetration for control points that do not intersect with the discontinuity information is w _ij = 1, as in the above.

W _ij = 1−Mv _ik (11) w _ij = 1−Mh _ik (12) The calculation of the weight w _ij is performed, for example, in the case of the virtual point S ₂₀ by using each control point. The discontinuity information at which the straight line connecting the two intersects, that is, the degree of penetration corresponding to the intersection of the curve 84 after the enlargement is used. In the case where a plurality of pieces of discontinuous information intersect, the maximum value of the degree of penetration is used.

As described above, by weighting the equation (8) and calculating, the object image can be enlarged in consideration of the degree of penetration with the background.

In the above description, the example of the white pixel and the black pixel has been described. However, in the case of a pixel having a density or a pixel having a color, a pixel value is obtained by obtaining a predicted value by averaging or gradient. Is also good.

As described above, discontinuity information is detected from the original image using the line process, and the contour of the object image included in the original image is extracted using the discontinuity information. Further, even if the outline of the object image is outlined using the discontinuity information and the resolution is changed, for example, when the image is enlarged, the image can be generated without blurring or roughening the outline of the object image. . Furthermore, since the degree of dissolution of the object image is obtained using the discontinuity information, even when the image is scaled or combined with another image, or when the resolution is changed, for example, when the background is synthesized with enlargement. The image can be generated such that the object image blends. As a result, the object image can be described as parts without depending on the resolution, and reuse can be facilitated.

In the above embodiment, the description of the consideration of the component of the pixel value is omitted.
The above processing may be performed on each component of GB, or may be performed on components of hue and saturation.

[0107] Although the case where the scaling is performed in the same image has been described, the present invention is also applicable to the case where an object from which the background portion in the object image is removed is cut out and combined with another background image. In this case, the degree of penetration at the time of clipping may be stored, and the stored degree of penetration may be used when combining with another background image. By doing so, it is possible to generate a composite image with less discomfort.

Next, a description will be given of a conversion process for generating moving image data using the above-described encoding and scaling of image data. Note that the following processing routine may be provided as a computer-executable application stored in a recording medium such as an FD, and may be executed by an execution instruction.

First, when the power is turned on to the image conversion apparatus 10, the processing routine shown in FIG. 1 is executed. The processing routine may be stored in a recording medium such as an FD and provided as a computer-executable application, and may be executed according to an execution instruction.

At step 300 in FIG. 1, a plurality of still images to be processed are read as moving images. In the next step 302, the reference image is read out of the plurality of read still images, and in the next step 304, the reference image is encoded. As the reference image, a temporally first still image among a plurality of still images can be adopted. Further, the processing of step 304 is the same as the above-described encoding processing (FIG.
Reference).

When the coding of the reference image is completed, in the next step 306, a still image that is continuous with the reference image is read out from a plurality of still images. In the next step 308, the difference between the reference image and the still image read in step 306 is determined. This step 308 is for specifying how much the read still image has changed from the reference image.

Next, in step 310, an object image (changed object) changed from the reference image is specified by using the obtained difference value, and in the next step 312, vector information of the object image is extracted. When the read still image changes from the reference image, it appears as a difference. The similarity of the object image included in the reference image is determined from the difference value, the position, and the shape, and the position variation and the size variation are determined. These are set as vector information. When the vector information is stored in advance, the processing of this step is unnecessary.

In the next step 314, step 306 is executed.
The changing object is encoded with respect to the still image read by. This encoding may be performed only with information indicating that the image is based on the reference image and vector information. That is, if there is a reference image and vector information, the read still image can be reproduced.

The processes from step 306 to step 314 are repeated (each process is repeated until the determination is affirmative in step 316), and all of the plurality of still images to be processed as moving images are executed. In the above processing, the vector information is obtained for the reference image. However, the processing may be performed for adjacent still images. In this case, the object image included in the immediately preceding still image fluctuates with the vector information from the object image included in the reference image. Also,
An object image newly appearing in a plurality of still images that is not included in the reference image can be newly encoded as an object image at that time (when the difference having no similarity is obtained).

When the above processing is completed for all still images, in the next step 318, moving image data is generated by synthesizing the data of the still images encoded as described above. In this combination, moving image data may be generated by arranging data in order, or moving image data may be generated by describing data according to a predetermined format.

Next, the processing of FIG. 1 will be described with reference to FIGS.
This will be specifically described with reference to FIG. 11 (A) to 11
(C) shows a plurality of still images 40A and 40 as original images.
B and 40C show an example in which a moving image is formed. In this moving image, the object images such as the mountain 42 and the flower 50 do not change, but the person 52 gradually approaches the flower 50 from the mountain 42. At this time, the position and the size of the object image of the person 52 change.

Therefore, the moving image data includes a background image 40S obtained by removing the person 52 from the original image 40 (still image 40A) shown in FIG. 12A, an object image of the person 52 shown in FIG. It can be expressed from the vector information 53. The vector information represents information on the position and the size.

By obtaining discontinuity information for the original image 40 (still image 40A), it is possible to extract the contour and characteristic portion of the object image. FIG. 13A is an enlarged top view of the person 52. When this discontinuity information is obtained, FIG.
As shown in FIG. 3B, the outline and expression of the person and the features of each part can be extracted. That is, it is not possible to find a change in expression or a feature only by the outline of a person, but it is possible to obtain a characteristic portion of the image by obtaining discontinuity information inside the image.

FIG. 14 shows a state in which the inside of the image of the person (only the head) is changed (a state during conversation).
FIG. 14A shows the case where discontinuity information is obtained for an image when the mouth is open, and FIG. 14B shows the case where discontinuity information is obtained for an image when the mouth is closed. As can be seen, features appear around the mouth 55.

In this way, the characteristic portion (discontinuous information) of the extracted image is outlined. By contouring the contour portion of the object image and the discontinuous portion inside the object image, the contour shape and the characteristic portion are maintained even if the original image or the object image is enlarged or reduced. Therefore, even when synthesizing color information (texture information) with a contour shape or a characteristic portion, an image can be expressed without being affected by the resolution.

As described above, in the present embodiment,
Detect discontinuity information of original image using line process,
Using the discontinuity information, the outline of the object image included in the original image and the discontinuity inside the image are extracted. Using the discontinuity information to outline the outline of the object image and the discontinuity inside the image and change the resolution, for example, even when the enlargement is performed, the outline of the object image is not blurred or coarse,
Images can be generated.

The image data representing a moving image can be described by outline data of the outline of the image and discontinuity in the image, and texture information such as color information included in the outline. Even when the image is transmitted and the image is reconstructed on the transmission side, it is possible to provide a high-quality image that is not blurred or coarse and does not depend on the resolution. Further, even when the image is enlarged or reduced, it is possible to provide a high-quality image independent of the resolution.

[0123]

As described above, according to the present invention, the outline of an object image is described by outline information, and the image is encoded including texture information representing the object image according to the outline. When playing back multiple original images or object images as a moving image, even if the image is enlarged or reduced, the outline of the original object image can be maintained without depending on the resolution, and even when playing back moving images There is an effect that the image data can be converted without causing a feeling of strangeness.

Further, the outline of the object image and the discontinuity inside the object image are described by outline information, and the image is coded by including the texture information representing the object image in accordance with the outline. There is an effect that minute movements inside these object images can be faithfully reproduced from the fluctuation of the discontinuous information.

[Brief description of the drawings]

FIG. 1 is a flowchart illustrating a flow of processing executed by an image conversion apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a schematic configuration of an image conversion apparatus according to the embodiment of the present invention.

FIG. 3 is a flowchart showing a flow of an encoding process in the image conversion device according to the embodiment of the present invention.

FIG. 4 is an image diagram showing a process of extracting an object image from an original image.

FIG. 5 is an explanatory diagram for explaining a line process.

FIG. 6 is a conceptual configuration diagram for explaining a neural network using a line process.

FIG. 7 is an explanatory diagram for explaining a line process applied to an image.

FIG. 8 is an explanatory diagram for describing a configuration of an image when the resolution is enlarged using NURBS.

FIG. 9 is a flowchart illustrating a flow of a process of enlarging an original image.

FIG. 10 is an explanatory diagram for explaining the degree of penetration using a mountain image.

FIG. 11 is an image diagram in which a moving image is decomposed into still images.

FIG. 12 is an explanatory diagram for explaining vector information.

FIG. 13 is an image diagram illustrating an object image for explaining discontinuity information.

FIG. 14 is an explanatory diagram for describing discontinuity information inside an object image.

[Explanation of symbols]

 Reference Signs List 10 Image conversion device 30 Image reading device 40 Original image 54 Class image 60 Mountain image (object image)

Continued on the front page F-term (reference) 5B057 CA01 CA12 CB01 CB12 CC02 CE15 CG01 DB02 DB06 DC16 5C054 FB03 FC12 FC14 FD00 GB12 GB14 GB15 5C057 AA11 EA01 EA07 EC01 ED06 EE04 EF05 EM07 5C059 KK37 PP38 MB03

Claims (8)

[Claims] 1. An image data conversion method for converting image data including color information of a plurality of continuous original images into image data representing a moving image, wherein the image data is converted by a line process based on the image data of the original image. Generating discontinuity information of the original image representing the discontinuity of the represented color information, extracting the contour of the object image included in the original image based on the generated discontinuity information, and Image data conversion method for obtaining outline information obtained by outlining the outline of an object image based on the extracted outline, obtaining texture information representing the object image using the extracted outline, and encoding image data including the outline information and the texture information . 2. The image data conversion method according to claim 1, wherein the outline of the object image is outlined by a parametric curve, and the color information is encoded by a parametric curved surface. 3. The discontinuity information generates, as discontinuity information of the original image, outer peripheral discontinuity information corresponding to the contour of the object image included in the original image and internal discontinuity information inside the object image. 3. The image data conversion method according to claim 1, wherein: 4. A reference original image is determined from the plurality of continuous original images, the coding is performed on the determined reference original image, and a variation of an object image included in the reference original image is determined on other original images. 4. The method according to claim 1, further comprising obtaining vector information to be represented, and converting the encoded image data and the vector information of the reference original image into image data representing a moving image. 5. Image data conversion method. 5. A recording medium recording an image data conversion program for converting image data containing a plurality of continuous original images into image data representing a moving image by using a computer, the image data comprising: The conversion program generates discontinuity information of the original image representing discontinuity of the color information represented by the line process based on the image data of the original image including the color information, and generates the discontinuity information based on the generated discontinuity information. The outline information of the object image included in the original image is extracted, the outline information of the outline of the object image is obtained based on the discontinuity information, and the outline information is obtained. Wherein the image data is encoded including the outline information and the texture information. A recording medium recording the program. 6. The recording medium according to claim 5, wherein the outline of the object image is outlined by a parametric curve, and the color information is encoded by a parametric curved surface. 7. The discontinuity information generates, as discontinuity information of the original image, outer peripheral discontinuity information corresponding to the contour of the object image included in the original image and internal discontinuity information inside the object image. A recording medium on which the image data conversion program according to claim 5 is recorded. 8. A reference original image among the plurality of continuous original images is determined, the determined reference original image is encoded, and the other original images are included in the reference original image. 8. The method according to claim 5, wherein vector information representing a change is obtained, and image data representing a moving image is converted from image data and vector information encoded for a reference original image. A recording medium on which the image data conversion program according to the item is recorded.