JP2003125194A - Data processing device and data processing method, and program and recording medium - Google Patents

Abstract

(57) [Summary] [Problem] Embed more data. SOLUTION: An embedding encoder 3 rotates a horizontal line and a vertical line of the image data in response to data to be embedded which is embedded in the image data, and furthermore, converts a pixel value of a pixel forming the vertical line into a pixel value. By performing a bit plane swap for exchanging a bit plane for each bit of the bit string to be represented, embedded encoded data in which the data to be embedded is embedded in the image data is generated. The decompressor 6 performs bit plane swapping and rotation of the vertical line of the embedded coded data to restore the image data in which the horizontal line and the vertical line have been rotated. To restore the original image data and the embedded data embedded therein.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing device, a data processing method, a program, and a recording medium, and particularly to, for example, embedding a large amount of data in image data and an image in which the data is embedded. The present invention relates to a data processing device and a data processing method, a program, and a recording medium that enable data and embedded data to be restored.

[0002]

2. Description of the Related Art For example, as a method of embedding another data series as other information into a certain data series as information without increasing the data amount, for example, digital audio data (audio data) There is one that converts the least significant bit, the least significant 2 bits, and the like into embedded information. This method takes advantage of the fact that the lower bits of digital audio data do not significantly affect the sound quality, and simply replaces the lower bits with the data that makes up another data sequence, and therefore, during playback, The digital audio data in which other data series are embedded is output as it is without returning the lower bits. That is, in this case, it is difficult to restore the lower bits in which other data series are embedded, and
Since the lower bits do not affect the sound quality so much, the digital audio data is output in a state where other data series are embedded.

However, according to the method described above, audio data different from the original audio data is output.
That is, since the other data embedded in the original audio data becomes noise with respect to the original audio data, the sound quality of the audio data embedded with the other data is deteriorated.

In addition to voice data, image data
It is possible to embed other data in the same manner as in the case of the audio data described above, but in this case as well, the image data in which the other data is embedded has deteriorated image quality.

[0005]

Therefore, the applicant of the present invention restores the original image data and other data embedded in the original image data without deteriorating the image quality (minimizing the deterioration of the image quality). We have previously proposed a possible embedding method.

That is, now, image data to be embedded with other data is embedded data, other data to be embedded in the embedded data, embedded data, and embedded data with respect to the embedded data. Data obtained by embedding is referred to as embedded coded data.

In this case, in the embedding process for embedding the data to be embedded in the embedding target data, for example, as shown in FIG.
As shown in A, the horizontal line that constitutes the image data as the embedding target data is
Sequentially, the line of interest. Then, the line of interest is rotated to the right by an amount corresponding to the embedded data, so that the embedded data becomes
It is embedded in (the attention line of) the image data as the embedding target data. Here, FIG. 1A shows a case where “2” is embedded as embedded data,
The line of interest corresponds to the embedded data “2” and is 2
It is rotated to the right by the amount of pixels.

In the embedding process, as described above, the image data as the embedding target data is rotated corresponding to the embedding target data, so that the embedding target data is embedded in the embedding target data. Embedded encoded data is generated. Since this embedded coded data is obtained by rotating the horizontal line of the image data as the embedding target data in the horizontal direction (here, to the right), the data amount is
It is the same as the embedding target data. Therefore, according to the embedding process described above, the embedding target data and the embedded data are compressed to the data amount of the embedding target data.

In the restoration processing for restoring the embedded coded data as described above into the original embedding target data and the embedded data, for example, as shown in FIG. 1B, image data that is embedded coded data is formed. The horizontal lines are sequentially set as the attention line from the upper side to the lower side. Further, a horizontal line one line above the line of interest is used as a reference line, and the line of interest is rotated pixel by pixel in the left direction, which is the opposite direction to the case of the embedding process. Also, while rotating the line of interest, the sum of the correlation between each pixel forming the line of interest and the corresponding pixel forming the reference line is calculated as the correlation between the line of interest and the reference line, and the correlation is maximized. The amount of rotation to be calculated is required. And the reference line is
It is returned to the original position by being rotated by the rotation amount that maximizes the correlation, and the rotation amount is restored as the embedded data embedded in the line of interest.

In the restoration process, as described above, each horizontal line forming the image data as the embedded coded data is rotated to a position where the correlation with the reference line is maximized, so that the data to be embedded is selected. Image data is restored, and the embedded data embedded in the embedded coded data is restored based on the rotation amount of each horizontal line.

As described above, according to the embedding process and the restoring process proposed above, the embedding target data and the embedding data are compressed into the embedding coded data having the data amount of the embedding target data, that is, The embedded data can be compressed by the amount corresponding to the embedded data, and the embedded coded data thus compressed can be restored to the original data to be embedded and the embedded data.

By the way, although it is convenient if more embedded data can be embedded, the data amount of the embedded data that can be embedded in the image data is only by the operation rule of rotating the line.
The number of lines forming the image data is limited.

The present invention has been made in view of such a situation, and embeds more embedded data in embedding target data, and embedded coded data obtained by the embedding process. To
The restoration processing is performed to restore the original embedding target data and the embedded data.

[0014]

The first data processing apparatus of the present invention performs an operation so that the information embedded with other information is restored to the original information by utilizing the energy bias of the information. According to the first operation rule to be performed, the embedding target data is manipulated in correspondence with any embedded data,
According to a first embedding means for generating embedded coded data according to the first operation rule in which arbitrary embedding data is embedded in the embedding target data, and a second operation rule different from the first operation rule, the first embedding means The second operation rule in which the embedded coded data according to the operation rule of No. 1 is operated corresponding to any other embedded data, and the embedded coded data according to the first operation rule is embedded with any other embedded data. And a second embedding means for generating embedded coded data according to.

The first data processing method of the present invention is a first operation rule for performing an operation so that information having other information embedded therein is restored to the original information by utilizing the energy bias of the information. According to the first embedding step, the embedding target data is manipulated corresponding to the arbitrary embedding data, and the embedding coded data is generated according to the first operation rule in which the embedding target data is embedded with the arbitrary embedding data. And according to a second operation rule different from the first operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, and the first operation rule is used. Second embedding step of generating embedded coded data according to the second operation rule in which other arbitrary embedded data is embedded in the embedded coded data Characterized in that it comprises a.

The first program of the present invention follows the first operation rule for performing an operation so that the information in which other information is embedded is restored to the original information by utilizing the energy bias of the information. A first embedding step of operating the embedding target data in correspondence with arbitrary embedding data and generating embedding coded data according to the first operation rule in which the embedding target data embedding the arbitrary embedding data, According to a second operation rule different from the first operation rule, the embedded coded data by the first operation rule is operated corresponding to any other embedded data, and the embedded code by the first operation rule is operated. A second embedding step of generating embedded encoded data according to a second operation rule in which other arbitrary embedded data is embedded in the encoded data. Characterized in that it obtain.

The first recording medium of the present invention has the first operation rule for performing an operation so that the information in which other information is embedded is restored to the original information by utilizing the bias of the energy of the information. Accordingly, a first embedding step of operating the embedding target data in correspondence with arbitrary embedding data, and generating embedded coded data according to the first operation rule in which the embedding target data embedding the arbitrary embedding data, , According to a second operation rule different from the first operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, and embedded according to the first operation rule. A second embedding step of generating embedded coded data according to a second operation rule in which other arbitrary embedded data is embedded in the encoded data. Characterized in that that program is recorded.

The second data processing apparatus of the present invention operates the embedded coded data according to the second operation rule according to the second operation rule, and also operates according to the first operation rule. First restoration means for restoring the embedded coded data according to the first operation rule and other arbitrary data to be embedded by utilizing the energy bias of the obtained data, and the embedded coded data according to the first operation rule According to the first operation rule, and utilizing the bias of the energy of the data obtained by the operation, embedding target data and second restoring means for restoring arbitrary embedded data. And

According to the second data processing method of the present invention, the embedded coded data according to the second operation rule is operated according to the second operation rule, and also operated according to the first operation rule. A first restoration step of restoring the embedded coded data according to the first operation rule and any other embedded data using the bias of the energy of the obtained data, and the embedded coded data according to the first operation rule According to the first operation rule, and utilizing the energy bias of the data obtained by the operation, the second restoration step of restoring the embedding target data and arbitrary embedded data. And

The second program of the present invention is obtained by operating the embedded coded data according to the second operation rule according to the second operation rule and the first operation rule. A first restoration step of restoring the embedded coded data according to the first operation rule and any other embedded data by utilizing the bias of the energy of the data, and the embedded coded data according to the first operation rule, It is characterized by comprising a second restoring step of restoring data to be embedded and arbitrary embedded data by operating in accordance with the first operation rule and utilizing the energy bias of the data obtained by the operation. .

The second recording medium of the present invention operates the embedded coded data according to the second operation rule according to the second operation rule and the first operation rule, and obtains the operation. The first restoration step of restoring the embedded coded data according to the first operation rule and any other embedded data by using the bias of the energy of the stored data, and the embedded coded data according to the first operation rule. , A program including an embedding target data and a second restoring step of restoring arbitrary embedded data by operating the first operation rule and utilizing the energy bias of the data obtained by the operation. It is characterized by

In the first data processing device and data processing method, the program and the recording medium of the present invention, the information in which other information is embedded is restored to the original information by utilizing the energy bias of the information. According to the first operation rule for performing the operation as described above, the embedding target data is manipulated corresponding to the arbitrary embedding data, and the embedding is performed according to the first operation rule in which the embedding target data embeds the arbitrary embedding data. Encoded data is generated. Then, according to the second operation rule different from the first operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, and the first operation rule is applied. The embedded coded data according to the second operation rule is generated by embedding other arbitrary embedded data in the embedded coded data.

In the second data processing device and data processing method, and the program and the recording medium of the present invention, the embedded coded data according to the second operation rule is:
In addition to being operated according to the second operation rule, the operation is performed according to the first operation rule, and by utilizing the energy bias of the data obtained by the operation, the embedded coded data and other Any embedded data is restored. Then, the embedded coded data according to the first operation rule is operated according to the first operation rule, and the embedding target data and any embedded data are restored by utilizing the energy bias of the data obtained by the operation. To be done.

[0024]

FIG. 2 shows a configuration example of an embodiment of an embedded coding / decoding system to which the present invention is applied.

The embedded coding / decoding system is composed of a coding device 11 and a decoding device 12. The encoding device 11 generates and outputs embedded coded data by embedding embedding data in image data as embedding target data, for example.
The decoding device 12 decodes the embedded coded data into original embedding target data and embedded data.

That is, the encoding device 11 is composed of an embedding target database 1, an embedding database 2 and an embedding encoder 3.

The embedding target database 1 stores, for example, digital image data as embedding target data. Then, the image data stored therein is read from the embedding target database 1 and supplied to the embedding encoder 3.

The embedded database 2 stores embedded data (digital data). Then, the embedded data stored therein is also read from the embedded database 2 and supplied to the embedded encoder 3.

The embedded encoder 3 receives the image data from the embedding target database 1 and the embedded data from the embedded database 2. Further, the embedded encoder 3 converts the image data into the embedded data from the embedded database 2 so that the image data from the embedded database 1 can be decoded by utilizing the bias of the energy of the image data. By performing the corresponding operation, the embedded data is embedded in the image data, and the embedded coded data is generated and output. The embedded coded data output by the embedded encoder 3 is recorded on a recording medium 4 such as a semiconductor memory, a magneto-optical disk, a magnetic disk, an optical disk, a magnetic tape, a phase change disk, or, for example, Ground wave, satellite line, CATV (Cable Television)
The data is transmitted via the transmission medium 5 such as a network, the Internet, or a public line, and provided to the decoding device 12.

The decoding device 12 comprises a decompressor 6 in which the embedded coded data provided via the recording medium 4 or the transmission medium 5 is received. Further, the decompression device 6 decompresses the embedded coded data into the original image data and the embedded data by utilizing the energy bias of the image data as the embedding target data. The restored image data is supplied to and displayed on a monitor (not shown), for example.

The embedded data may be, for example, text data related to the original image data as the data to be embedded, voice data, reduced image data obtained by reducing the image data, or the like. Irrelevant data can also be used.

Next, FIG. 3 shows the embedded encoder 3 of FIG.
The example of composition of is shown.

The image data as the embedding target data supplied from the embedding target database 1 is supplied to the frame memory 21, and the frame memory 21 stores the image data from the embedding target database 1, for example, Temporarily store in frame units.

The embedding unit 22 receives the embedding data stored in the embedding data base 2 and performs embedding processing for embedding the embedding data in the image data as the embedding target data stored in the frame memory 21, As a result, embedded coded data that can be restored by utilizing the energy bias of the image data is generated.

The frame memory 21 is composed of a plurality of banks so that a plurality of frames (or fields) can be stored therein. By switching the banks, the frame memory 21 can be embedded in the database 1 to be embedded. So that the image data supplied from the device, the image data subjected to the embedding process by the embedding unit 22, and the image data after the embedding process (embedded encoded data) can be output at the same time. Has become. As a result, even if the image data supplied from the embedding target database 1 is a moving image, the embedded coded data can be output in real time.

Next, FIG. 4 shows the embedded encoder 3 of FIG.
2 shows an example of the configuration of the decompressor 6 of FIG. 2 that reconstructs the embedded coded data output from the original image data and the data to be embedded by utilizing the energy bias of the image data.

The embedded coded data, that is, the image data in which the embedded data is embedded (hereinafter, also referred to as embedded image data as appropriate) is supplied to the frame memory 31, and the frame memory 31 The embedded image data is temporarily stored in frame units, for example. The frame memory 31 is also configured similarly to the frame memory 21 of FIG. 3, and by performing bank switching, even if the embedded image data is a moving image, real-time processing thereof can be performed.

The restoration unit 32 performs restoration processing for restoring the embedded image data stored in the frame memory 31 into the original image data and the embedded data by utilizing the bias of the energy of the image data.

Next, the principle of the embedding process performed by the embedded encoder 3 and the restoration process performed by the decompressor 6 will be described.

Generally, what is called information has a bias (universality) of energy (entropy), and this bias is recognized as information (valueful information). That is, for example, an image obtained by shooting a certain landscape is recognized by a person as an image of such a landscape because the image (the pixel value of each pixel forming the image, etc.) This is because there is a corresponding bias in energy, and an image with no bias in energy is nothing but noise and is not useful as information.

Therefore, even if some kind of operation is performed on valuable information and the original bias of energy possessed by the information is destroyed, so to speak, the bias of the destroyed energy is returned to the original state and some operation is performed. The information that has been given can also be returned to the original information. That is, the data obtained by manipulating the information can be restored to the original valuable information by utilizing the original energy bias of the information.

Here, the energy possessed by the information (bias)
For example, there are correlation, continuity, similarity, and the like.

The information correlativity means the correlation (for example, autocorrelation, a certain constituent element and another constituent element) of the information (for example, in the case of an image, pixels and horizontal lines constituting the image). Distance with components).
For example, what represents the correlation between images is a correlation between horizontal lines of an image, and the correlation value representing this correlation is, for example, 2 which is the difference between the pixel values of corresponding pixels in two horizontal lines. Reciprocal numbers such as sum of multiplications can be used.

That is, for example, as shown in FIG.
When there is an image having a number of horizontal lines, the correlation between the first horizontal line from the top (first horizontal line) and other horizontal lines is generally the first as shown in FIG. 6A.
A horizontal line closer to the horizontal line (horizontal line on the upper side of the image in FIG. 5) becomes larger as shown as a correlation for the M-th horizontal line, and a horizontal line farther from the first horizontal line (FIG. 5). The lower horizontal line in the image) becomes smaller as shown as the correlation for the Nth horizontal line. Therefore, there is a bias in the correlation that the closer to the first horizontal line the greater the correlation with the first horizontal line, and the farther from the first horizontal line the smaller the correlation.

Therefore, in the image of FIG.
Pixels of the Mth horizontal line relatively close to the horizontal line and the Nth horizontal line relatively far from the first horizontal line are exchanged (1 <M <N ≦ H), and the image after the exchange is When the correlation between one horizontal line and another horizontal line is calculated, it becomes as shown in FIG. 6B, for example.

That is, in the image after the replacement, the correlation with the M-th horizontal line near the first horizontal line (the N-th horizontal line before the replacement) becomes small, and the N-th horizontal line far from the first horizontal line (before the replacement). (M-th horizontal line).

Therefore, in FIG. 6B, the bias of the correlation is destroyed, in which the correlation increases as the distance from the first horizontal line increases and the correlation decreases as the distance from the first horizontal line increases. However,
For an image, generally, the bias of the correlation that is destroyed can be restored by utilizing the bias of the correlation that the correlation increases as the distance from the first horizontal line increases and the correlation decreases as the distance from the first horizontal line increases. That is, in FIG. 6B,
The fact that the correlation with the M-th horizontal line close to the first horizontal line is small and the correlation with the N-th horizontal line far from the first horizontal line is large is apparently inferior from the original bias of the correlation that the image has. It is natural (funny) and the Mth horizontal line and the Nth horizontal line should be interchanged. Then, by replacing the M-th horizontal line and the N-th horizontal line in FIG. 6B, it is possible to restore the image having the original bias of the correlation as shown in FIG. 6A, that is, the original image.

Here, in the case described with reference to FIGS. 5 and 6, the horizontal line replacement operation embeds embedded data in image data. Then, when embedding the data to be embedded, for example, operations on the image data such as what horizontal line is moved and which horizontal line is exchanged are performed.
It is determined corresponding to the data to be embedded. On the other hand, the horizontal line of the embedded image data (embedded encoded data) in which the embedded data is embedded can be replaced with the original position by using the correlation of the original image data to restore the original image data. become. Then, at the time of the restoration, for example, the embedded data embedded in the image data is restored based on an operation on the embedded image data such as which horizontal line has been moved or which horizontal line has been exchanged. .

Next, with reference to FIGS. 7 to 9, an example of the embedding process and the restoration process using the correlation of the image data will be described.

In the embodiment shown in FIGS. 7 to 9, in the embedding process, some pixels forming the image data as the embedding target data are selected as the process target pixels, and the selected pixels are embedded. By operating in the level direction corresponding to the data (by operating the pixel value), the embedded data is embedded in the pixel. On the other hand, in the restoration process, some pixels that form the embedded image data obtained by the embedding process (pixels at the same position as that selected in the embedding process)
Is selected as a pixel to be processed, and as shown in FIG.
The pixel value is changed by performing the operation opposite to that in the embedding process on the pixel to be processed. Further, as shown in FIG. 7, the correlation value R1 between the pixel to be processed before the pixel value change and its surrounding pixels (pixels adjacent to each other in the embodiment in FIG. 7) is calculated, and The correlation value R2 between the pixel to be processed whose value has been changed and the peripheral pixels of that pixel is calculated. Then, based on the magnitude relationship between the correlation values R1 and R2, one of the processing target pixels before or after the pixel value change is set as the restoration result of the original image data, and the processing is performed. The embedded data embedded in the target pixel is restored.

FIG. 8 is a flow chart showing an example of embedding processing performed by the embedment encoder 3 of FIG. 3 by utilizing the correlation of image data.

In the embedding target database 1, the image data as the embedding target data stored therein is read out and supplied to the frame memory 21 and stored therein, for example, in units of one frame.

On the other hand, the embedding unit 22 receives the embedded data from the embedded database 2, for example, bit by bit, and in step S1, the pixel to be embedded (processing target) is embedded. Pixel) is selected from the image stored in the frame memory 21.

Here, in the present embodiment, for example, from the image data stored in the frame memory 21, the pixels are sequentially selected in the form of a five-eye grid and are set as the pixels to be processed. .

After that, the embedding section 22 carries out step S2.
At, it is determined whether the embedded data received from the embedded database 2 is 1 or 0. In step S2, the embedded data is
If it is determined to be 0 out of 1 or 0, the process returns to step S1. That is, when the embedded data is 0, the embedding unit 22 selects the pixel to be processed as
Without performing any operation (adding 0 as a predetermined constant), the process returns to step S1, waits for the next 1-bit embedded data to be transmitted from the embedded database 2, and then A pixel to be processed is selected, and the same process is repeated thereafter.

If it is determined in step S2 that the embedded data is, for example, 1 out of 1 or 0, the process proceeds to step S3, and the embedding section 22
A predetermined operation is performed on the pixel to be processed. That is, the embedding unit 22 adds, for example, 2 as a predetermined constant, to the power of the number of bits assigned to the pixels forming the image data, to the pixel value of the pixel to be processed.

Therefore, if, for example, 8 bits are assigned as the pixel value of the pixel forming the image data, 2 ⁷ is added to the pixel value of the pixel to be processed in step S3. .

In addition, in this addition, the pixel value is, for example, Y
When represented by UV or the like, it may be performed on either the luminance component Y or the color components U and V. Also,
The addition may be performed on any of R, G, and B when the pixel value is represented by RGB, for example. That is, when the pixel value is composed of a plurality of components, the addition in step S3 may be performed on one or two or more of the plurality of components, or may be performed on all the components. good. Further, the addition in step S3 can be performed independently for each of the plurality of components.

After adding 2 ⁷ to the pixel value of the pixel to be processed in step S3, the process proceeds to step S4,
It is determined whether the addition result overflows. When it is determined in step S4 that the addition result does not overflow, step S5 is skipped, and the process proceeds to step S6, where the embedding unit 22 sets the addition result as the pixel value of the pixel to be processed as the frame memory 21. To (overwrite) and return to step S1.

In step S4, the addition result is
If it is determined that an overflow has occurred, that is, this implementation
, The pixel value is 8 bits, the addition result
But 2 ⁸When it becomes above, it progresses to step S5 and fills it.
The inclusion unit 22 corrects the added value. That is, step
In step S5, the overflowed addition value is, for example,
Overflowed amount (2 from the added value that overflowed⁸
Is subtracted). Then, in step S6
Then, the embedding unit 22 processes the addition result after the correction.
Write to the frame memory 21 as the pixel value of the target pixel
Embedded, the next 1-bit embedded data is embedded
Wait for the data to be sent from database 2 and then
Return to step S1.

Incidentally, the data stored in the frame memory 21,
After the embedding process is performed on the image data of a certain one frame, the image data of the one frame, that is, the embedded image data is read from the frame memory 21 and output. Then, the embedding unit 22 continues the embedding process for the image data of the next one frame stored in the frame memory 21.

Next, referring to the flowchart of FIG.
The restoration processing by the restoration device 6 for restoring the embedded image data obtained by the embedding processing of FIG. 8 will be described.

In the frame memory 31, the embedded image data supplied thereto is sequentially stored, for example, in 1-frame units.

On the other hand, in step S11, the restoration unit 32 selects a pixel (processing target pixel) to be restored from the embedded image data of a certain frame stored in the frame memory 31.

Here, in the restoration unit 32, the embedding unit 22
Similarly, the pixels are sequentially selected from the embedded image data stored in the frame memory 31 in a five-eye grid pattern. That is, the same pixel is selected as the pixel to be processed in the embedding process and the restoring process.

After that, the process proceeds to step S12, and the restoration unit 3
2 performs the operation opposite to the operation performed by the embedding unit 22 in FIG. 3 on the pixel to be processed. That is, the embedding part 22 is
A predetermined constant, for example, 2 to the power of the number of bits assigned to the pixels forming the image minus 1 is subtracted from the pixel value of the processing target pixel.

Therefore, as described above, in the case where, for example, 8 bits are assigned as the pixel value of the pixels forming the embedded image data, step S12
Then, 2 ⁷ is subtracted from the pixel value of the processing target pixel.

In this subtraction, the pixel value is, for example, Y
When represented by UV or the like, it may be performed on either the luminance component Y or the color components U and V. Also,
The subtraction may be performed on any of R, G, and B when the pixel value is expressed in RGB, for example. However, the subtraction in step S12 is performed in step S of FIG.
It must be done for the same thing that the addition in 3 was done. That is, the pixel value is expressed by, for example, YUV, and the addition in step S3 of FIG.
For example, when the Y component of YUV is performed, the subtraction in step S12 also needs to be performed for the Y component.

After 2 ⁷ is subtracted from the pixel value of the pixel to be processed in step S12, the process proceeds to step S13, and it is determined whether or not the subtraction result is underflowed. When it is determined in step S13 that the subtraction result does not underflow, step S14
Is skipped and the process proceeds to step S15.

If it is determined in step S13 that the subtraction result is underflow, that is, if the addition result is less than 0, the process proceeds to step S14.
The restoration unit 32 corrects the subtracted value. That is, in step S14, the underflowed subtraction value is corrected to, for example, a value obtained by adding 2 ⁸ to the subtraction value.
Go to 5.

In step S15, the pixel value of the pixel to be processed (for which 2 ⁷ has not been subtracted in step S12) (hereinafter referred to as the first pixel value as appropriate) P1 and 2 ⁷ are subtracted from that pixel value. Subtracted value (hereinafter, the value corrected in step S14 is also included) (hereinafter, appropriately referred to as the second value).
For each of P2 (referred to as the pixel value of the pixel), a correlation value representing the correlation with, for example, the adjacent pixels on the left and right of the pixel to be processed is calculated.

That is, in step S15, for example, the absolute value of the difference between the first pixel value P1 of the pixel to be processed and the pixel values of the left and right pixels is calculated, and the absolute value of the two absolute values obtained as a result is calculated. The reciprocal of the added value is obtained as the correlation value R1 for the first pixel value P1 of the pixel to be processed.
Further, in step S15, also with respect to the second pixel value P2 of the processing target pixel, the reciprocal of the addition value of the absolute values of the differences between the pixel values of the left and right pixels is calculated, which is the It is obtained as the correlation value R2 of the second pixel value P2.

The pixels used to obtain the correlation with the pixel to be processed in step S15 are not limited to the pixels adjacent to the left and right, but may be the pixels adjacent to each other in the vertical direction. The pixels may be adjacent in time. Further, it is not always necessary that the pixels are spatially or temporally adjacent to each other. However, when obtaining the correlation with the pixel to be processed, it is desirable to use the pixel in which the embedded data is not embedded in the embedded image data. This is because even if the correlation between the pixel to be processed and the pixel in which the embedded data is embedded is obtained,
Since the correlation for the original image data cannot be obtained and therefore the correlation of the image data cannot be used, the original pixel value and the embedded data can be accurately calculated from the pixel in which the embedded data is embedded. It will be difficult to restore to. Further, as long as the processing target pixel is restored using the correlation of the image data, the pixel used to obtain the correlation value with the processing target pixel has a close spatial or temporal distance to the processing target pixel. Is desirable.

After calculating the correlation value R1 for the first pixel value P1 and the correlation value R2 for the second pixel value P2,
In step S16, the restoration unit 32 compares the correlation values R1 and R2.

In step S16, the correlation value R1 is
When it is determined that the correlation value R2 is larger than (or equal to or larger than) R2, the process proceeds to step S17, where the restoration unit 32
As a result of restoring the embedded data, 0 is output and the process returns to step S11. Then, in this case, the stored value of the frame memory 31 is not rewritten, and therefore the restoration result of the pixel value of the processing target pixel remains the pixel value P1.

That is, the fact that the correlation value R1 for the first pixel value P1 is larger than the correlation value R2 for the second pixel value P2 means that the pixel value of the pixel to be processed is
Since the pixel value P1 is more likely than the pixel value P2, the restoration result of the pixel value of the pixel to be processed is the certain pixel value P1. Furthermore, since the pixel value P1 has not been subtracted by 2 ⁷ in step S12,
It is considered that 2 ⁷ is not added in step S3 of FIG. Then, in the embedding process of FIG. 8, when the embedded data is 0, 2 ⁷ is not added, so the correlation value R1 for the first pixel value P1 is larger and the pixel value P1 is When the pixel value of the pixel to be processed is likely, the embedded data embedded therein is 0.

On the other hand, in step S16, the correlation value R
If it is determined that 2 is greater than (is greater than or equal to) the correlation value R1, the process proceeds to step S18, and the restoration unit 32 determines that the pixel value of the pixel to be processed stored in the frame memory 31 is 2 ⁷ from that pixel value. The subtraction value obtained by subtracting, that is, the second
Is rewritten to the pixel value P2. Therefore, in this case, the restoration result of the pixel value of the pixel to be processed is the pixel value P2. Then, the process proceeds to step S <b> 19, the restoring unit 32 outputs 1 as the restoration result of the embedded data, and the process returns to step S <b> 11.

That is, the fact that the correlation value R2 for the second pixel value P2 is larger than the correlation value R1 for the first pixel value P1 means that the pixel value of the pixel to be processed is
Since the pixel value P2 is more likely than the pixel value P1, the restoration result of the pixel value of the pixel to be processed is the likely pixel value P2. Further, since the pixel value P2 is obtained by subtracting 2 ⁷ from the pixel value P1 in step S12, it is considered that 2 ⁷ is added to the original pixel value in step S3 of FIG. . Then, in the embedding process of FIG. 8, when the embedded data is 1, 2 ⁷
Since the correlation value R2 for the second pixel value P2 is larger and the pixel value P2 is more likely to be the pixel value of the pixel to be processed, the embedded data embedded therein is added. Will be 1.

Here, when the difference between the correlation values R1 and R2 obtained as described above is small, the pixel values P1 and P2
Which of the above is likely to be the pixel value of the pixel to be processed cannot be generally stated. Therefore, in such a case, not only the pixels adjacent to the left and right of the pixel to be processed,
By using other pixels as well, the correlation value for each of the pixel values P1 and P2 is obtained, and by comparing the correlation values, which of the pixel values P1 and P2 is likely to be the pixel value of the processing target pixel. You can decide.

As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data by utilizing the correlation of the images. The embedded image data can be restored to the original image data and the embedded data without the overhead for the restoration. Therefore, basically, the restored image (decoded image) does not deteriorate in image quality due to the embedding of the embedded data.

That is, according to the embedding process of FIG. 8, a predetermined value (0 or 2 ^{7 in} the embodiment of FIG. 8) corresponding to the value of the embedded data is added to the pixel value of the image data, When the added value overflows, the embedding data is embedded in the image data according to the operation rule of performing the correction, and the embedded image data is generated.

On the other hand, according to the restoration process of FIG. 9, according to the operation rule in the embedding process, the operation opposite to that in the embedding process is performed on the embedded image data, and the correlation of the data after the operation is obtained. Based on this, the operation performed in the embedding process is, so to speak, specified. Then, according to the specified operation, the embedded image data is restored to the original image data and the embedded data.

Therefore, it can be said that the operation rule in the embedding process of FIG. 8 is an operation rule that can restore the embedded image data in which the embedded data is embedded by utilizing the correlation of the image data. However, in the embedding process, by adopting such an operation rule, it is possible to embed the image data in the image data without deteriorating the image quality of the image data (with the minimum deterioration) and without increasing the data amount. Data can be embedded.

That is, the pixel in which the embedded data is embedded can be used as an original pixel without overhead by utilizing the correlation of the image, that is, the correlation with the pixel in which the embedded data is not embedded here. It is possible to restore (restore) the pixel and the embedded data. Therefore, basically, the restored image (decoded image) obtained as a result does not deteriorate in image quality due to embedding of the embedded data.

In the embodiments of FIGS. 7 to 9, the reciprocal of the absolute value of the difference between the pixel values is used as the correlation value representing the correlation between the pixel to be processed and other pixels. However, the correlation value is not limited to this.

Further, in the embodiments of FIGS. 7 to 9, the pixels are selected from the image data in the form of a five-eye grid and the embedded data is embedded in the pixels. The selection pattern of pixels to be embedded is not limited to this. However, when restoring the pixel in which the embedded data is embedded, as described above, it is desirable to obtain the correlation by using the pixel in which the embedded data is not embedded, and the correlation between the pixels is basically The smaller the spatial or temporal distance between them, the smaller. Therefore, from the viewpoint of performing accurate restoration, it is desirable to select the pixels in which the embedded data is embedded so as to be spatially or temporally so-called sparse. On the other hand, from the viewpoint of embedding a large amount of embedded data, that is, the compression rate, it is necessary to increase the number of pixels in which the embedded data is embedded to some extent. Therefore,
Pixels for embedding embedded data are accurate for restoration,
It is desirable to select a balance with the compression rate.

Further, in the embodiment of FIGS. 7 to 9,
Although 1-bit embedded data is embedded in one pixel selected as a processing target pixel, 2 pixels are embedded in one pixel.
It is also possible to embed embedded data of more than one bit. For example, when 2-bit embedded data is embedded in one pixel, for example, 0, 2 ⁶ , 2 ⁷ , 2 ^{6 are} embedded according to the 2-bit embedded data.
Any one of +2 ⁷ may be added to the pixel value.

Further, in the embodiments of FIGS. 7 to 9, by adding either 0 or 2 ^{7 to} the pixel value (either not adding 2 ⁷ to the pixel value or adding it). Although the embedded data is embedded, the value to be added to the pixel value is not limited to 2 ⁷ .
However, when a value that affects only the lower bits of the pixel value is added, the added value and the original pixel value do not differ significantly, and therefore step S15 in FIG. Also, the correlation values R1 and R2 obtained in step 2 do not become very different. Since this deteriorates the accuracy of the restoration result of the pixel value and the embedded data, the value to be added to the pixel value according to the embedded data is a value that affects the upper bits of the original pixel value. Is desirable.

Further, in the embodiment of FIGS. 7 to 9,
Although the embedded data is embedded by adding a predetermined value to the pixel value, the embedded data is embedded in operations other than addition (for example, bit inversion and horizontal line rotation described above). Can also be performed by applying to the pixel value. However, as described above, from the viewpoint of preventing the deterioration of the accuracy of the restoration result of the pixel value and the embedded data, the operation performed on the pixel value is
It is desirable that the correlation with respect to the original image and the correlation with respect to the image subjected to the operation are significantly different.

In the embodiments shown in FIGS. 7 to 9, the 1-bit embedded data is embedded in one pixel selected as the pixel to be processed.
It is also possible to embed embedded data of bits. That is, the same operation corresponding to 1-bit embedded data can be performed on two or more adjacent pixels.

Next, the embedding process and the restoring process using the continuity of information will be described.

For example, when attention is paid to one horizontal line having image data, the line of interest is shown in FIG.
Assuming that a waveform WAVE # 1 in which the pixel value change pattern is continuous as shown in 0A is observed, a pixel value change pattern different from the attention line is observed on other horizontal lines apart from the attention line. To be done. Therefore, the line of interest and the other horizontal line apart from the line of interest have different pixel value change patterns, and the continuity is biased. That is, when attention is paid to the pixel value change pattern of a certain portion of the image, a similar pixel value change pattern exists in a portion adjacent to the attention portion, and a different pixel value change pattern is obtained as the distance from the attention portion increases. There is a continuity bias that exists.

Therefore, a portion of the waveform WAVE # 1 in which the pixel value change pattern is continuous in the horizontal line of the image data shown in FIG. 10A is separated as shown in FIG. 10B. Waveform W on horizontal line
Replace with AVE # 2.

In this case, the bias in the continuity of the image data is destroyed. However, it is possible to restore the disrupted bias of continuity by using the bias of continuity in which the pixel value change patterns of the adjacent portions are continuous and the pixel value change patterns differ as the distance increases. it can.
That is, in FIG. 10B, the change pattern of the pixel value of a part of the waveform WAVE # 2 is significantly different from the change pattern of the pixel value of the other part of the waveform because the original continuity of the waveform is biased. It's obviously unnatural,
Part W different from the pixel value change pattern of other parts
The AVE # 2 should be replaced with a waveform similar to the change pattern of the pixel value of the other part. Then, by performing such a replacement, the bias of continuity is restored, whereby the original waveform shown in FIG. 10A can be restored from the waveform shown in FIG. 10B.

Here, in the case described with reference to FIG. 10, it is possible to replace a part of the waveform with a waveform of a pixel value change pattern which is significantly different from the pixel value change pattern in the vicinity thereof. The embedded data is embedded. Further, in the embedding, for example, which part of the waveform to change the pixel value change pattern and how much to change the pixel value change pattern are determined according to the embedded data. On the other hand, the embedded image data (embedded encoded data) in which the embedded data is embedded, that is, the waveform having a largely different pixel value change pattern as a part, the peripheral pixel value change patterns are continuous and are separated from each other. The original waveform is restored by using the bias of continuity that the change patterns of the pixel values are different to restore the original waveform. Then, at the time of restoration, for example, in which part of the waveform the change pattern of the pixel value has changed significantly, and how much the change pattern of the pixel value has changed, that is, the embedded image data, The original image data (waveform)
The embedded data that was embedded in the image data is restored depending on whether or not the data has been returned to.

Next, FIG. 11 is a flow chart showing an example of embedding processing in the embedding encoder 3 when embedding utilizing the continuity of images is performed.

First, in step S21, the embedding section 22 causes the frame memory 21 to store one frame of image data as embedding target data stored in the embedding target database 1. Further, in step S21, the embedding unit 22 determines that the frame memory 21
The continuous area of the image data stored in is detected, and the continuous area data indicating the position of the continuous area is stored in a work memory (hereinafter referred to as a work memory) not shown.

That is, for example, as shown in FIG. 12A, in the case where the continuous area is detected for the image data in which the width × height is 224 (= 7 × 32) × 1600 pixels, the embedding unit is used. 22 indicates image data, for example, 3
The image block is divided into 2 × 1 pixel blocks, DCT (Discrete Cosine Transform) processing is performed on each image block, and the DCT coefficient for each image block is calculated.

Further, the embedding unit 22 scans the image blocks in the raster scan order and sequentially sets them as the target block. For example, the difference between the DCT coefficient corresponding to the target block and the corresponding DCT coefficient of the adjacent image block on the left side. The sum of absolute values of is calculated in order and stored in the work memory as the continuity evaluation value of the block of interest. Further, the embedding unit 22 recognizes an image block whose calculated continuity evaluation value (difference value) is equal to or less than a predetermined threshold as a continuous area, and stores the position of the image block in the work memory.

Here, since there is a possibility that a part of the image block which should originally be a continuous area is judged not to be a continuous area due to the influence of noise, etc., such a judgment is prevented. For this purpose, the embedding unit 22 performs correction processing after detecting the continuous area, and the adjacent area is adjacent to the continuous area.
An image block that is a non-continuous area is converted into an image block of a continuous area. That is, according to the correction process, for example,
An image block of a short non-contiguous area sandwiched between two continuous areas or one (or a small number) of image blocks of non-contiguous areas adjacent to the continuous area is converted into an image block of the continuous area (continuous area). Image blocks of two continuous areas and the area sandwiched between them (image blocks of non-continuous areas) are corrected to one continuous area as a whole, or one continuous area. Area and
The image blocks of the non-contiguous areas adjacent to it are corrected into one continuous area as a whole.

After that, in step S22, the embedding section 22 receives the embedded data from the embedded data database 2 by 6 bits (3 bits + 3 bits), for example, and proceeds to step S23.

In step S23, the embedding section 22
An image to be processed is selected and extracted as a pixel group in which the 6-bit embedded data is embedded. That is, in step S23, for example, one frame of image data stored in the frame memory 21 is virtually divided into upper and lower half regions (hereinafter, appropriately referred to as upper region and lower region, respectively). Then, the continuous regions of the respective upper and lower regions, for example, horizontal lines at the same position from above are selected and extracted.

Specifically, for example, the frame memory 21
The image data of one frame stored in FIG.
As shown in FIG. 2A, when it is composed of 1600 horizontal lines, the area from the first horizontal line to the 800th horizontal line is the upper area, and the area from the 801st horizontal line to the 1600th horizontal line. Is the lower area. Then, for the image data stored in the frame memory 21, when the process of step 23 is first performed, as indicated by hatching in FIG. 12A, the first horizontal line which is the first row of the upper region. A continuous area of the line and the 801st horizontal line, which is the first row of the lower area, is selected and extracted.

It should be noted that the embedding section 22 executes the step S21.
In reference to the continuous area data stored in the work memory, only the continuous areas of the first horizontal line and the 801st horizontal line are selected and extracted. Here, in the embodiment of FIG. 12, the entire area of the first horizontal line and the 801st horizontal line is selected and extracted as a continuous area.

Then, in step S24, the embedding section 22 replaces the image of the first horizontal line and the image of the 801st horizontal line, which are the images to be processed, with the image data to be embedded data. Embed.

That is, FIG. 12B shows the pixel value of the image data of the first horizontal line before the embedded data is embedded. Further, FIG. 12C shows the pixel value of the image data of the 801st horizontal line before the embedded data is embedded. As shown in FIGS. 12B and 12C, the pixel value change patterns (frequency characteristics) are different between the first horizontal line region and the 801st horizontal line region.

For example, if it is assumed that 6-bit embedded data in which the upper 3 bits are 2 and the lower 3 bits are 6 is embedded in the image data, the embedding section 22 embeds 2 of the upper 3 bits. Since the image block of the second block from the left of the first horizontal line shown in FIG. 12B is selected and 6 of the lower 3 bits are embedded,
The sixth image block from the left of the 801st horizontal line shown in C is selected. Furthermore, the embedding part 22
Performs an operation of exchanging the selected second block of the first horizontal line with the selected sixth block of the 801st horizontal line. As a result, the change pattern of the pixel value of the first horizontal line is changed from FIG. 12B to FIG.
12C to 12E, the change pattern of the pixel value of the 801st horizontal line is changed as shown in FIG. 12C, and the upper 3 bits are 2 in the first horizontal line and the 801st horizontal line. And the lower 3 bits represent 6 and the 6-bit embedded data is embedded.

Thereafter, in step S25, the embedding section 22 writes (overwrites) the image data of the first horizontal line and the 801st horizontal line in which the embedded data is embedded into the frame memory 21, and proceeds to step S26.

In step S26, the embedding section 22
It is determined whether or not the embedded data is embedded in all the horizontal lines of the image data of one frame stored in the frame memory 21. If it is determined in step S26 that the embedded data has not been embedded in all the horizontal lines, the process returns to step S22.
The embedding unit 22 receives the new embedded data,
It proceeds to step S23. In step S23, the embedding unit 22 selects the next horizontal line from the upper region and the lower region, that is, in this case, the second horizontal line and the 802nd horizontal line, and the same processing is performed thereafter. repeat.

On the other hand, when it is determined in step S26 that the embedded data is embedded in all the horizontal lines of the image data of one frame stored in the frame memory 21, that is, in step S23, the first horizontal line is detected. And the set of the 801st horizontal lines to the set of the 800th and 1600th lines are selected as the processing target images, the embedding unit 22 determines that the image data stored in the frame memory 21, that is, the embedded data. The embedded image data (embedded image data) is read out and output, and the embedding process ends.

The embedding process shown in FIG. 11 is repeated for each frame for the image data stored in the frame memory 21.

Next, FIG. 13 is a flow chart showing an example of a restoration process for restoring the embedded coded data in which the embedded data is embedded in the image data as the embedding target data by utilizing the continuity of the images. Is.

First, in step S31, the restoration unit 32 stores the embedded coded data (embedded image data) in units of one frame in the frame memory 31. Furthermore, in step S31, the restoration unit 32
Similar to the case of step S21 of FIG. 11, a continuous area is extracted from the embedded coded data, and the position of the image block is stored in a work memory (not shown) as continuous area data.

Here, in the present embodiment, as described with reference to FIG. 12, the embedded data of 3 bits is embedded in one horizontal line. This embedding is performed by one image block in one horizontal line. This is performed by an operation of changing to an image block of another horizontal line. Therefore, at least one of the image blocks forming one horizontal line is a non-contiguous region due to the change operation for embedding the embedded data. However, when extracting the continuous area, since the correction processing is performed as described in step 21 of FIG. 11, the image block that is made into the non-continuous area by the change operation for embedding the embedded data is Converted to continuous areas (considered as continuous areas).

After that, the process proceeds to step S32, and the restoration unit 3
2 refers to the continuous area data stored in the work memory in step S31, and the embedded image data of one frame stored in the frame memory 31 is compared with the upper area and the lower area in the same manner as in step S23 of FIG. It is virtually divided into regions, and continuous regions of horizontal lines at the same position are sequentially selected and extracted from each of the upper region and the lower region.

Therefore, for the embedded image data stored in the frame memory 31, when the process of step S32 is performed first, the first horizontal line which is the first row of the upper region and the first line of the lower region. The 801st horizontal line is selected. Here, as described in step S23 of FIG. 11, assuming that the first horizontal line of the image data as the embedding target data and the entire region (all image blocks) of the 801st horizontal line are continuous regions, S
At 32, the entire regions of the first horizontal line and the 801st horizontal line are extracted.

Then, the process proceeds to step S33, and the restoration unit 3
In step 2, the image block in 32 × 1 pixel units, which forms the continuous area of the first horizontal line and the continuous area of the 801st horizontal line extracted from the upper area and the lower area in step S32, is displayed by, for example, DCT. After processing, the DCT coefficient as an evaluation value for evaluating the continuity of the image data is calculated, and the process proceeds to step S34.

In step S34, the restoration unit 32 sets the image blocks constituting the continuous area of the first horizontal line extracted from the upper area as the target block sequentially from, for example, the left side. From the DCT coefficient of the block of interest, the DC of the image block on the left
The T coefficient is subtracted, and the absolute value sum of the difference values of the DCT coefficients is calculated. Further, similarly, for the continuous area of the 801st horizontal line extracted from the lower area, the restoration unit 32 similarly sequentially sets the image blocks forming the continuous area as the block of interest, and with respect to the block of interest, the difference in DCT coefficient between them. Calculate the sum of absolute values. Then, the restoration unit 32 stores the calculated absolute value sum of the difference values of the DCT coefficients in the work memory.

If there is no image block on the left of the target block, that is, if the leftmost image block forming the continuous area is the target block, the difference value of the DCT coefficient for the target block is determined. The sum of absolute values of is set to 0, for example.

After that, the process proceeds to step S35, and the restoration unit 3
2 is an image block in which the sum of absolute values of the difference values of the DCT coefficients of the image blocks forming the continuous area of each of the two horizontal lines stored in the work memory (sum of absolute differences) is the second largest; Two image blocks of the second largest image block are detected, and the positions of the image blocks are stored in the work memory. That is, for example, in the work memory, the sum of absolute differences between the DCT coefficients of the image blocks forming the continuous area of the first horizontal line and the difference of the DCT coefficients of the image blocks forming the continuous area of the 801st line in the work memory. When the sum of absolute values is stored, in the first horizontal line, the image block in which the sum of absolute difference between the DCT coefficients is the largest (hereinafter, as appropriate,
1st image block) and 2nd largest image block (hereinafter, appropriately referred to as 2nd image block)
Is detected, the positions of the first and second image blocks are stored in the work memory, and the first image block and the second image block are detected in the 801st horizontal line. Then, the positions of the first and second image blocks are stored in the work memory.

Then, the process proceeds to step S36, and the restoration unit 3
2 restores and outputs 6-bit embedded data based on the positions of the first and second image blocks for each of the two horizontal lines stored in the work memory.

That is, the restoration unit 32 determines that the first and second image blocks are adjacent to each other, and the difference absolute value sums of the DCT coefficients of the first and second image blocks are similar values ( If the difference between the absolute sums of the differences is less than or equal to a predetermined threshold value), the position of the image block located on the left side of the adjacent first and second image blocks is set to the embedded data by the embedding process of FIG. Is stored in the work memory as the position of the image block (hereinafter, appropriately referred to as an embedding position) that is replaced by embedding, and the value corresponding to the embedding position is output as the restoration result of the embedded data.

Specifically, for example, in the embedding process of FIG. 11, for the continuous area of the first horizontal line, FIG.
As shown in B and FIG. 12D, when the replacement operation of the third image block from the left and the image block of the 801st horizontal line is performed, the second and third images from the left are replaced.
For the second image block, the sum of absolute differences between the DCT coefficients becomes large. Therefore, for the first horizontal line, in step S35 of FIG.
The two adjacent image blocks, the second and the fourth, are detected as the first and second image blocks (or the second and first image blocks), and the image block located to the left of them, that is, Based on the position (embedding position) of the third image block from the left, the upper 3 bits of the 6-bit embedded data are restored to 2 (= 3-1).

In addition, for example, in the embedding process of FIG. 11, for the continuous area of the 801st horizontal line, FIG.
And as shown in FIG. 12E, when the 7th image block from the left and the image block of the first horizontal line are exchanged, the 7th and 8th image blocks from the left are: The sum of absolute differences of the DCT coefficients becomes large. Therefore, for the 801st horizontal line, in step S35 of FIG. 13, the two adjacent image blocks, the seventh and the eighth from the left, are the first and second image blocks (or the second and the second image blocks). 1st image block), and the lower 3 bits of the above-mentioned 6-bit embedded data based on the position (embedding position) of the image block located on the left side thereof, that is, the 7th block from the left Is restored to 6 (= 7-1).

Here, in the embedding process of FIG.
When the embedded data is embedded by replacing the image block at the left end or the image block at the right end of the continuous region of horizontal lines with the image block of another horizontal line, the sum of absolute difference of DCT coefficients is , The size of the replaced image block is increased to some extent, and other blocks are not increased so much. Therefore, when the positions of the first and second image blocks are not adjacent to each other, or when the first and second image blocks have the same difference absolute value sum of the DCT coefficients. If the value is not such a value (when the difference between the absolute sums of differences is equal to or larger than a predetermined threshold value), the embedded data is restored based on the position of the first image block. That is, in this case, when the first image block is the leftmost or rightmost image block, 0 or 7 is restored as the data to be embedded, respectively.

When the embedded data is restored based on the embedding position as described above, step S37.
Then, the restoration unit 32 follows the two embedding positions used to restore the 6-bit embedded data stored in the work memory in step S35,
The positions of the image blocks in the two horizontal lines selected in step S32 of the embedded image data stored in the frame memory 31 are exchanged.

That is, for example, in the embedding process of FIG. 11, as described with reference to FIG. 12, the third image block from the left of the first horizontal line, which is the first row of the upper region, and the first row of the lower region. If the 7th image block from the left of the 801st line is replaced, the embedding position indicating the 3rd from the left is stored for the 1st horizontal line in step S35 of FIG. For the horizontal line, the embedding position representing the seventh from the left is stored. Therefore, in this case, in step 37, the first
The third image block from the left of the horizontal line,
The 7th image block from the left of the horizontal line is replaced, and thereby the 1st horizontal line and the 801st horizontal line are restored.

After that, the restoration unit 32 writes the restored two horizontal lines in the frame memory 31 in a form of overwriting, and proceeds to step S38. In step S38, the restoration unit 32 determines whether the restoration of the image data as the embedding target data and the embedded data embedded therein has been completed for the one frame of the embedded image data stored in the frame memory 31. To judge. 1 stored in the frame memory 31 in step S38
When it is determined that the restoration of the embedded image data of the frame is not completed, the process returns to step S32, and the horizontal line of the next row is extracted from each of the upper region and the lower region in the embedded image data stored in the frame memory 31. After being selected, similar processing is repeated thereafter.

If it is determined in step S38 that the restoration of the embedded image data of one frame stored in the frame memory 31 is completed, the process proceeds to step S39 and the restoration unit 32 is stored in the frame memory 31. The image data that has been restored, that is, the image data that is the restored embedding target data is output, and the processing ends.

According to the embodiment of FIG. 13, the image data as the embedding target data is restored for every two horizontal lines, that is, for each horizontal line of the same row in each of the upper area and the lower area thereof. It Therefore, the restored image data can be output after all of the one-frame image data stored in the frame memory 31 is restored as described above, but two horizontal lines, which are restoration units, are restored. It is also possible to output every time.

As described above, since the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data by utilizing the continuity of the image, The embedded image data can be restored to the original image data and the embedded data without the overhead for the restoration. Therefore, basically, the restored image (decoded image) does not deteriorate in image quality due to the embedding of the embedded data.

That is, according to the embedding process of FIG.
Of the image block at the position corresponding to the upper 3 bits of the 6-bit embedded data in the horizontal line of
According to the operation rule of replacing the image block at the position corresponding to the lower 3 bits of the 6-bit embedded data in one horizontal line, the embedded data is embedded in the image data to generate the embedded image data. It

On the other hand, according to the restoration processing of FIG. 13, the DCT coefficient of the embedded image data is used as the continuity evaluation value.
Utilizing the continuity, the operation opposite to that in the embedding process is performed on the embed image data according to the operation rule in the embedding process. Then, the operation restores the embedded image data to the original image data and the embedded data.

Therefore, the operation rule in the embedding process of FIG. 11 can be said to be an operation rule that can restore the embedded image data in which the embedded data is embedded by utilizing the continuity of the image data. But,
In the embedding process, by adopting such an operation rule, the embedded data can be embedded in the image data without deteriorating the image quality of the image data (with the minimum deterioration) and without increasing the data amount. Can be embedded.

In the above case, the DCT coefficient of the embedded image data is used as the continuity evaluation value, but other than the DCT coefficient can be used as the continuity evaluation value. Is.

Next, the embedding process and the restoring process utilizing the similarity of information will be described.

For example, part of an image of a landscape is
It is known that it can be generated by utilizing the fractal nature (self-similarity) of an image. Then, for example, in an image in which the sea R1 and the forest R2 are displayed as shown in FIG. 14A, a change pattern (for example, an edge shape) of pixel values of the entire sea R1 and some pixels of the sea R1. Although there is a high degree of similarity with the value change patterns, there is a similarity bias in that those change patterns have a low degree of similarity with the pixel value change patterns of the forest R2 distant from the sea R1.

Therefore, a part R3 of the sea R1 and a part R4 of the forest R2 shown in FIG. 14A are replaced.

In this case, the bias in the similarity of images is destroyed, and an image as shown in FIG. 14B is obtained. However, by utilizing the similarity bias that the variation patterns of the pixel values of the adjacent portions have a high similarity, and the similarity of the variation patterns of the pixel values decreases as the distance increases, the bias of the destroyed similarity is eliminated. Can be restored. That is, FIG.
4B, a part of the image of the sea R1 is a part R4 of the image of the forest R2 having low similarity to the sea R1, and a part of the image of the forest R2 has low similarity to the forest R2. Sea R1
The fact that it is a part R3 of the image is obviously unnatural from the perspective of the original similarity of the image.
Specifically, in FIG. 14B, in the image of the sea R1,
The similarity of a part R4 of the image of the forest R2 is extremely lower than the similarity of other parts of the sea R1, and the similarity of the image of the sea R1 in the image of the forest R2. The similarity for part R3 is also extremely lower than the similarity for other parts of the forest R2. On the other hand, the sea R1
The similarity between the part R4 of the forest R2 inside the forest and the forest R2, and the similarity between the part R3 of the sea R1 inside the forest R2 and the sea R1 are both high.

Therefore, in view of the similarity bias inherent in the image, it is part of the image of the forest R2, which is part of the image of the sea R1, and part of the image of the forest R2. ,
A part R3 of the image of the sea R1 should be replaced. Then, by performing such replacement, the bias in similarity of the images is restored, and thus the original image shown in FIG. 14A can be restored from the image shown in FIG. 14B.

Here, in the case described with reference to FIG. 14, a part R of the image of the sea R1 as the embedding target data.
Replacing 3 with part R4 of the image of forest R2,
The embedded data is embedded in the embedding target data. Also, when embedding it, for example, sea R
Which part (position on the screen) of the image No. 1 and which part of the image of the forest R2 are to be replaced is determined according to the embedded data. On the other hand, the embedded image data (embedded encoded data) in which the embedded data is embedded, that is, part of the sea R1 is part of the forest R2 and part of the forest R2 is part of the sea R1. An image with a part R3 has a high similarity in the variation pattern of the surrounding pixel values, and the similarity of the variation pattern of the pixel values becomes lower as the distance from the image is reduced. To restore the original image, the original image is restored. Then, at the time of the restoration, for example, which part of the sea image and which part of the forest image were exchanged, that is, how the embedded image data was restored to the original image data Thus, the embedded data embedded in the image data is restored.

Next, the similarity will be further described with reference to FIG.

Now, for example, when a fractal image as shown in FIG. 15A is used as embedding data and embedded data is embedded therein, a part of the fractal image corresponding to the embedding data is a part of the fractal image. The embedded data is embedded in the fractal image by replacing the image with an image that is not similar to.
That is, in FIG. 15A, a fractal image in the shape of a tree leaf is set as embedding target data. However, when embedding data to be embedded, a part of the fractal image becomes a triangle as shown in FIG. 15B. It is done by being replaced. That is, in FIG. 15, the portions indicated by D1 and D2 in FIG. 15B are triangular, and in this way, the position of the fractal image to be replaced by the triangle, the size and number of the triangle to be replaced with the fractal image, etc. Determined according to the embedded data.

On the other hand, the embedded coded data in which the embedded data is embedded in the fractal image as the embedding target data as described above is restored to the fractal image and the embedded data as follows, for example. It That is, for example, in FIG. 15B, a portion surrounded by a dotted-line quadrangle is not replaced with a triangle, and the portion surrounded by the dotted-line quadrangle is similar to the teacher image as a teacher image. The non-part (here, a triangle) is searched. Further, the triangle that is not similar to the teacher image is replaced with the image (fractal image) generated from the reference figure of the fractal image included in the teacher image, thereby restoring the original fractal image (FIG. 15A). To be done. Then, the embedded data to be embedded is restored based on the position, size, number, etc. of the retrieved triangle.

In the above case, it is necessary to detect the reference figure of the fractal image included in the teacher image, but this is performed as follows, for example. That is, the reference figure of the fractal image is searched for in the image of FIG. 15B based on the similarity between the teacher image and the other part in the image of FIG. 15B and the self-similarity of the teacher image. A figure that can most efficiently represent the part other than the teacher image is detected as the reference figure.

In the embodiment of FIG. 15, the original fractal image generation rule is recognized, and an image to be replaced with a triangle is generated using a reference figure based on the generation rule. . That is, at the time of restoring the fractal image as the embedding target data and the embedded data, in order to generate an image to be replaced with a triangle,
The size, position, rotation amount, etc. of the reference graphic are specified based on the generation rule, and the reference graphic is operated according to the specified size, position, rotation amount, etc. of the reference graphic, and the image is replaced with a triangle ( Fractal image) shall be generated.

Next, FIG. 16 is a flow chart showing an example of processing in the embedding encoder 3 when embedding is performed using the similarity of images.

In step S51, the embedding section 22
Transfers one frame of image data from the embedding target database 1 to the frame memory 21 and stores it therein.
Further, the similar area of the image data is detected, and the similar area data indicating the position of the similar area is stored in the work memory.

That is, for example, as shown in FIG.
When embedding data to be embedded by using a fractal image composed of 6 × 1600 pixels as embedding target data, the embedding unit 22 firstly outputs the 56 × 160.
A fractal image consisting of 0 pixels is divided into image blocks of, for example, 8 × 8 pixels, and the image blocks are sequentially set as target blocks. Furthermore, the embedding part 22 is
The pixels forming the target block are scanned, for example, in raster scan order, and the similarity between the target block and each of the two peripheral image blocks that are image blocks adjacent to the left and right of the target block is calculated.

Specifically, the embedding unit 22 changes the size, position, and rotation amount of the block of interest by a predetermined amount, and the block of interest and the image block on the left side or on the right side (peripheral image block), respectively. And the reciprocal of the sum of absolute differences between the pixels forming the target block and the corresponding pixels forming the peripheral image block is obtained as the matching degree. Further, the embedding unit 22 averages two matching degrees of the target block and each of the two peripheral image blocks,
The average value is stored in the work memory as an evaluation value (similarity evaluation value) representing the similarity with the periphery of the block of interest. Here, when the image block at the left end or the right end is the block of interest, there is no peripheral image block on the left side or the right side, and therefore, for example, one obtained with the right or left peripheral image block is used. The matching degree is directly used as the similarity evaluation value of the target block. The embedding unit 22 recognizes an image block in which the calculated similarity evaluation value (matching degree) is equal to or higher than a predetermined threshold as a similar region, and stores the position of the image block in the work memory.

Here, there is a possibility that a part of the image block, which should originally be a similar region, may be judged not to be a similar region due to the influence of noise or the like, so such a judgment is prevented. For this purpose, the embedding unit 22 performs correction processing after detecting the similar region, and the adjacent region is adjacent to the similar region.
An image block that is a dissimilar region is converted into an image block of a similar region. That is, according to the correction process, for example,
An image block of a non-similar region in a short section sandwiched between two similar regions or an image block of one (or a small number) of non-similar regions adjacent to the similar region is converted into an image block of the similar region (similar region Image blocks of two similar regions and the region sandwiched between them (the image block of the non-similar region) is corrected to one similar region as a whole, or one similar region. Area and
The image blocks in the non-similar region adjacent to it are corrected into one similar region as a whole.

Next, in step S52, the embedding section 22 receives the embedded data from the embedded database 2 by 6 bits (3 bits + 3 bits), for example.

Then, in step S53, the embedding section 22 selects and extracts a processing target image as an image block group in which the 6-bit embedded data is embedded. That is, in step S53, for example, one frame of image data stored in the frame memory 21 is virtually divided into upper half and lower half areas (upper area and lower area), and the upper area and lower area are divided. , The similar regions in the same row from above are selected and extracted.

Specifically, for example, the frame memory 21
The image data of one frame stored in FIG.
As shown in FIG. 7A, when the image block is composed of 1600 horizontal lines and is divided into image blocks of 8 × 8 pixels in step S51, the area of the image block from the first row to the 100th row is the upper area. , Line 101 to line 20
The area of the image block up to the 0th row is the lower area. Then, for the image data stored in the frame memory 21, when the process of step 53 is first performed, the image block of the first row of the upper region and the similar region of the image block of the first row of the lower region Are selected and extracted.

It should be noted that the embedding section 22 performs step S51.
In reference to the continuous area data stored in the work memory, only the similar areas of the image block of the first row and the image block of the 101st row are selected and extracted. Here, FIG.
In the seventh embodiment, all image blocks in the first row and the 101st row are selected and extracted as being similar areas.

After that, the process proceeds to step S54, and the embedding unit 22 replaces, for example, one of the image blocks in the first row and one of the image blocks in the 101st row, which is the image to be processed. , Frame memory 2
The embedded data is embedded in the image data stored in 1.

That is, FIG. 17A shows the image data as the data to be embedded before the embedded data is embedded, and FIG. 17B shows the embedded coded data after the embedded data is embedded. .

In the embedding data shown in FIG. 17A, the pixel value change pattern in the image block area of the first row and the image block area of the 101st row, that is,
The figures in the image block have different similarities.

If, for example, 6-bit embedded data in which the upper 3 bits are 2 and the lower 3 bits are 6 is embedded in the image blocks of the first row and the 101st row, the embedding unit 22 is assumed. Selects the third image block from the left in the first row of FIG. 17A in correspondence with the upper 3 bits 2 of the 6-bit embedded data, and also selects the lower part of the 6-bit embedded data. The 7th image block from the left in the 101st row of FIG. 17A is selected corresponding to 3 of 6 bits. Furthermore, the embedding part 22
Is the third image block from the left of the selected first row,
The 7th image block from the left of the 101st row is replaced, whereby 6-bit embedded data is embedded in the image blocks of the 1st row and the 101st row, as shown in FIG. 17B.

Here, the i-th image block from the left will be appropriately referred to as the i-th image block hereinafter.

After embedding the data to be embedded in step S54, the process proceeds to step S55, where the embedding unit 22 stores the image block in the first row and the image block in the 101st row in which the data to be embedded is embedded in the frame memory. 21 is written in a form of overwriting, and the process proceeds to step S56.

In step S56, the embedding section 22
It is determined whether or not the embedded data is embedded in all the image block rows of the image data of one frame stored in the frame memory 21. If it is determined in step S56 that the embedded data has not been embedded in all the image block rows, step S56.
Returning to 52, the embedding unit 22 receives the new embedded data, and proceeds to step S53. In step S53, the embedding unit 22 selects the image block of the next row from the upper area and the lower area, that is, in this case, the image block of the second row and the image block of the 102nd row, Hereinafter, similar processing is repeated.

On the other hand, if it is determined in step S56 that the embedded data is embedded in all the image block rows of the image data of one frame stored in the frame memory 21, that is, in step S53, the first When the set of the image blocks of the 100th row and the 101st row to the set of the image blocks of the 100th and 200th rows are selected as the processing target images, the embedding unit 22 determines that the image data stored in the frame memory 21. That is, the image data in which the embedded data is embedded (embedded image data) is read out and output, and the embedding process ends.

The embedding process of FIG. 16 is appropriately repeated for the frame of the image data as the embedding target data.

Next, FIG. 18 is a flow chart showing an example of the restoration process for restoring the embedded coded data in which the embedded data is embedded in the image data as the embedding target data by utilizing the similarity of the images. Is.

First, in step S61, the restoration unit 32 sequentially stores the embedded coded data (embedded image data) in units of one frame in the frame memory 31. Furthermore, in step S31, the restoration unit 32
In the same manner as in step S51 of FIG. 16, extracts a similar region from the embedded coded data,
The position of the image block is stored in a work memory (not shown) as similar area data.

Here, in the present embodiment, as described with reference to FIG. 17, 3-bit embedded data is embedded in one row of an image block. This embedding is performed among the image blocks forming one row. Is performed by an operation of changing one image block (operation of replacing). Therefore, at least one of the image blocks forming one row becomes a dissimilar region due to the change operation for embedding the data to be embedded. However, at the time of extracting the similar region, the correction process is performed as described in step 51 of FIG. 16, so that the image block that has been made a dissimilar region by the change operation for embedding the embedded data is Converted to a similar area (considered as a similar area). Therefore, the similar regions extracted in step S31 include the image blocks replaced in the embedding process of FIG.

After that, the process proceeds to step S62, and the restoration unit 6
2 refers to the similar area data stored in the work memory in step S61, and the embedded image data of one frame stored in the frame memory 31 is compared with the upper area and the lower area in the same manner as in step S53 of FIG. It is virtually divided into regions, and from each of the upper region and the lower region, similar regions of image blocks in the same row are sequentially selected and extracted.

Therefore, for the embedded image data stored in the frame memory 31, when the process of step S62 is performed first, the first row of the upper area and the first row of the lower area are selected. Here, step S53 of FIG.
As described above, assuming that the image block of the first row and the image block of the 101st row of the image data as the embedding target data are all similar areas, step S
In 62, all the image blocks in the first row and the 101st row are
It is extracted as a similar region.

Then, the process proceeds to step S63, and the restoration unit 6
2 is similar to the case described in step S51 of FIG. 16 and 8 × 8 forming the similar area of the first row and the similar area of the 101st row extracted from the upper area and the lower area in step S62. The pixel-by-pixel image blocks are sequentially set as the target block, and the target block is matched with the peripheral image blocks. As a result, the restoration unit 62 obtains the degree of matching between the target block and the surrounding image blocks, and further, from the degree of matching, in the same manner as in the case described in FIGS. 16 and 17,
Obtain a similarity evaluation value.

In the above case, the average value of the matching degrees of the target block and the left and right image blocks (peripheral image blocks) is set as the similarity evaluation value of the target block. It is also possible to adopt the larger or smaller matching degree of each of the target block and the left and right image blocks as the similarity evaluation value.

As described above, the restoration unit 62 obtains the similarity evaluation value of each of the image blocks that form the similarity area of the first row and the similarity area of the 101st row extracted from the upper area and the lower area. If you ask, the similarity evaluation value,
It is stored in the work memory, and the process proceeds to step S65.

In step S65, the reconstructing section 32 sets the similarity regions of the image blocks of two rows (one row in the upper area and one row in the lower area corresponding to the row, a total of two rows) stored in the work memory. From each of them, the image block with the highest similarity evaluation value is detected, and the position of the image block is stored in the work memory. That is, for example, now, the work memory stores the similarity evaluation value for the image block forming the similarity region of the first row and the similarity evaluation value for the image block forming the similarity region of the 101st row. If there is, the image block with the highest similarity evaluation value in the first row (hereinafter referred to as the maximum similarity image block)
Then, in the 101st row, the maximum similarity image block is detected, and the position of the maximum similarity image block is stored in the work memory.

Then, the process proceeds to step S66, and the restoration unit 3
2 restores and outputs 6-bit embedded data based on the positions of the maximum similarity image blocks in each of the two rows stored in the work memory.

That is, when the maximum similarity image block in the row in the upper area or the lower area is the i-th image block, the restoring unit 32 sets i−1 to the upper 3 bits or the lower 3 bits of the embedded data of 6 bits. As 3 bits, they are respectively restored, and further, from the upper 3 bits and the lower 3 bits, 6
The bit embedded data is constructed and output.

Here, for example, in the case shown in FIG. 17B, the maximum similarity image block in the first row is the third image block, and the maximum similarity image block in the 101st row is
This is the seventh image block. Therefore, the upper 3 bits and the lower 3 bits of the 6-bit embedded data are 2
And becomes 6.

As described above, when the embedded data is restored based on the position of the maximum similarity image block, the process proceeds to step S67, and the restoring unit 32 proceeds to step S67.
In step S62, the embedded image data stored in the frame memory 31 is selected according to the positions of the two maximum similarity image blocks used to restore the 6-bit embedded data stored in the work memory in 65. Swap the positions of the image blocks of the two rows that have been created.

That is, for example, in the embedding process of FIG. 16, as described with reference to FIG. 17, the third of the first row in the upper region is processed.
Image block and the first line of the lower area (the 101st line of the whole)
If the seventh image block of FIG.
In step S65 of 8, for the first row, the third row
The position of the image block is stored, and for the 101st row, the position of the seventh image block is stored. Therefore, in this case, in step 67, the third image block in the first row and the seventh image block in the 101st row are exchanged,
As a result, the image data of the first row and the 101st row are restored.

Thereafter, the restoration section 32 writes the restored two lines of image data (image block) in the frame memory 31 in a form of overwriting, and proceeds to step S68. In step S68, the restoration unit 32 determines whether the restoration of the image data as the data to be embedded and the embedded data embedded therein has been completed for the embedded image data of one frame stored in the frame memory 31. To judge. In step S68, the frame memory 3
When it is determined that the restoration of the embedded image data of one frame stored in No. 1 is not completed, the process returns to step S62, and from the upper region and the lower region in the embedded image data stored in the frame memory 31, The image block in the row is selected, and the same processing is repeated thereafter.

When it is determined in step S68 that the restoration of the embedded image data of one frame stored in the frame memory 31 is completed, the process proceeds to step S69, and the restoration section 32 is stored in the frame memory 31. The image data that has been restored, that is, the image data that is the restored embedding target data is output, and the processing ends.

In the embodiment of FIG. 18 as well, as in the case of the embodiment of FIG. 13, all the image data of one frame stored in the frame memory 31 is restored as the restored image data. It can be output later, or can be output each time the image block of two rows, which is the restoration unit, is restored.

As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data by utilizing the similarity of images. The embedded image data can be restored to the original image data and the embedded data without the overhead for the restoration. Therefore, basically, the restored image (decoded image) does not deteriorate in image quality due to the embedding of the embedded data.

That is, according to the embedding process of FIG.
The image block at the position corresponding to the upper 3 bits of the 6-bit embedded data in the row image block, and the lower 3 of the 6-bit embedded data forming another row
According to the operation rule of replacing the image block at the position corresponding to the bit, the embedded data is embedded in the image data to generate the embedded image data.

On the other hand, according to the restoration processing of FIG. 18, the degree of matching of the embedded image data is used as the evaluation value of the similarity, the similarity is used, and the embedded image data is processed according to the operation rule in the embedding processing. , The operation opposite to that in the embedding process is performed. Then, the operation restores the embedded image data to the original image data and the embedded data.

Therefore, it can be said that the operation rule in the embedding process of FIG. 16 is an operation rule that can restore the embedded image data in which the embedded data is embedded by utilizing the similarity of the image data. But,
In the embedding process, by adopting such an operation rule, the embedded data can be embedded in the image data without deteriorating the image quality of the image data (with the minimum deterioration) and without increasing the data amount. Can be embedded.

In the above-mentioned case, the matching degree of the embedded image data is used as the similarity evaluation value, but the similarity evaluation value other than the matching degree can be used. .

As described above, in the embedded coder 3, the image data is stored in accordance with the operation rule for performing the operation so that the restoration can be performed by utilizing the bias of the energy of the image data as the data to be embedded. When the embedded data is embedded in the image data and the embedded coded data is output by operating in accordance with the embedded data, the decompressor 6 stores the embedded coded data in the energy of the image data. By utilizing the bias of, there is no overhead for restoration,
The original image data and the embedded data can be restored.

By embedding the data to be embedded in the image data as the data to be embedded, the embedded image data (embedded encoded data) obtained as a result of the embedding becomes an image different from the original image data. Since it is not an image that a person can recognize as valuable information, the image data as the embedding target data can be encrypted without overhead.

Furthermore, a completely reversible digital watermark can be realized. That is, in the conventional digital watermark, for example,
Although the lower bits of the pixel value, which does not significantly affect the image quality, are simply changed to the values corresponding to the digital watermark, in this case, it is difficult to restore the lower bits to the original value. Therefore, the image quality of the restored image is deteriorated to some extent by changing the lower bit as the digital watermark. On the other hand, when the embedded coded data is restored by utilizing the energy bias of the image data as the embedding target data, the original image data and the embedded data without deterioration can be obtained, Therefore, by using the embedded data as a digital watermark, the image quality of the restored image does not deteriorate due to the digital watermark.

The embedded data embedded in the embedding target data can be retrieved (restored) by restoring the embedding target data from the embedded coded data. Can be provided. In other words, since the embedded data can be embedded in the embedding target data without the overhead of extracting the embedded data, the embedded coded data obtained as a result of the embedding is compressed (embedded). It can be said that it is compressed. Therefore, for example, if half of a certain image is the embedding target data and the other half is the embedding data, the remaining half image can be embedded in the half image that is the embedding target data. In that case, the image would simply be compressed in half.

Further, since the embedded coded data is restored by utilizing the bias of the energy of the information, that is, the statistical amount, it is highly resistant to errors.
That is, robust coding (statistical coding), which is highly robust coding, can be realized.

Further, for example, while the image data is set as the embedding target data, the data to be embedded is, for example, audio data of a medium different from the image data is used. It becomes possible to provide. That is, on the encoding device 11 side of FIG. 2, for example, as data to be embedded such as the voice “open sesame” uttered by the contractor, it is embedded in the image data that is the embedding target data, and on the restoration device 12 side, it is displayed to the user. , The voice “open sesame” is uttered, and the speaker recognition is performed using the voice and the voice embedded in the image data. By doing so, for example, as a result of speaker recognition, it is possible to automatically present an image only when the user is a contractor. In this case, the voice as the embedded data can use the voice waveform itself instead of the so-called characteristic parameter.

Further, for example, by using the audio data as the embedding target data, and by using the image data of a medium different from the audio as the embedding data, for example, the audio is provided using the image as a key. It is possible to do what is done (for example, voice response after face recognition).
That is, on the encoding device 11 side in FIG. 2, for example, an image of the user's face is embedded in the voice as a response to the user, and on the restoration device 12 side, the user's face is photographed and the resulting image is obtained. By outputting a voice in which a face image matching with is output, it is possible to realize a voice response system that makes a different voice response for each user.

Furthermore, it is also possible to embed information of a certain medium, such as embedding a voice in a voice or an image in an image, to embed information in the same medium. Alternatively, if the voice of the contractor and the face image are embedded in the image, the image is presented only when the voice of the user and the face image match those embedded in the image. It is also possible to realize a dual key system, so to speak.

Also, for example, it is possible to embed one of the synchronized image and sound which compose the television broadcast signal in the other, so that the information of different media is integrated. In other words, so-called integrated coding can be realized.

Further, in the embedding processing, as described above, the more the information is characterized by its energy bias,
Many embedded data can be embedded. Therefore, for example, for two pieces of information, the one having a characteristic energy bias is adaptively selected, and the selected one is
By embedding the other one, it becomes possible to control the entire data amount. That is, between two pieces of information, one piece of information can absorb the amount of information of the other, so to speak. As a result of being able to control the total amount of data in this manner, it becomes possible to perform information transmission (environmentally compliant network transmission) with the amount of data that matches the transmission band of the transmission path, the usage status, and other transmission environments.

Further, for example, by embedding a reduced image of an image in an image (or by embedding a thinned version of the voice in a voice), the so-called hierarchical coding is performed without increasing the data amount. (Encoding that generates information in the upper layer by reducing the amount of information in the lower layer) can be realized.

Further, for example, by embedding an image serving as a key for retrieving the image in the image, it is possible to realize a database for retrieving the image based on the image serving as the key. Become.

As described above, there are correlations, continuity, similarity, etc. to represent the energy (bias) of the information, but in the following, in order to simplify the explanation,
For example, only the correlation will be explained. Therefore,
First, image data is used as embedding target data,
Three methods (first, second, and third methods) will be introduced as other examples of embedding processing and restoration processing using the correlation of the image data.

In the first method, in the embedding process,
The operation rule is to replace (pixel swap) the positions of the pixels forming the image data as the embedding target data in units of one column (pixel column arranged in the vertical direction) in accordance with the embedded data. According to the operation rule, the image data as the embedding target data is operated, so that the embedded data is embedded in each column.

That is, in the embedding process of the first method, the position of each column of the image data of one frame, which is the embedding target data, is calculated based on the data to be embedded.
Pixel swap information indicating how to replace is generated. Specifically, for example, one frame of image data as embedding target data is composed of pixels of M rows and N columns, and the nth column (nth column from the left) of the image data.
Is replaced with the n'th column, pixel swap information in which n and n'are associated with each other is generated (n, n ').
Is an integer of 1 or more and N or less).

Here, when the number of columns of the image data of one frame is N columns as described above, the replacement method is N! If all the columns are to be replaced.
There are only streets (! Represents factorial). Therefore, in this case, at most log ₂ (N!) Bits (however, the fractional part is rounded down) can be embedded in one frame.

After the pixel swap information is generated, the position of each column of the image data as the embedding target data is exchanged (pixel swap) according to the pixel swap information, so that the embedding target data is embedded in the embedding target data. Are embedded, and embedded coded data (embedded image data) is generated.

Here, in the first method, as described above,
The embedding data is embedded according to the operation rule that each column of the image data as the embedding target data is pixel swapped corresponding to the embedding data. Therefore, this first method will be referred to as the pixel swapping method hereinafter as appropriate. Say.

Next, with reference to the flow chart of FIG. 19, the embedding process when the embedding encoder 3 of FIG. 3 performs the embedding process by the pixel swap method will be described.

The image data as the embedding target data stored therein is read from the embedding target database 1 in units of, for example, one frame, and is sequentially supplied and stored in the frame memory 21.

Then, the embedding section 22 carries out step S8.
In step 1, the embedded data having a data amount that can be embedded in the image of one frame is read from the embedded database 2. That is, for example, as described above, when the number of columns of an image in one frame is N columns and all the columns are to be subjected to pixel swap (replacement), at most log _{2 in} one frame. Since it is possible to embed (N!) Bits of embedded data, such embedded data having the number of bits (below) is read from the embedded database 2.

Then, the embedding section 22 carries out step S8.
2, the pixel swap information is generated based on the embedded data read in step S81. That is, the embedding unit 22 determines, for example, the second to Nth columns other than the first column among the first to Nth columns of the frame of the image data stored in the frame memory 21 based on the embedded data. Pixel swap information representing the number of columns to which each pixel is swapped is generated.

After that, the process proceeds to step S83, and the embedding unit 22 pixel-swaps the position of each column of the frame to be processed stored in the frame memory 21 according to the pixel swap information. Then, the frame in which the pixel swap of the column is performed is read from the frame memory 21,
It is output as embedded coded data.

The pixel swap of each column of the frame is
This can be done by changing the storage position of the image data (the pixels forming the image data) in the frame memory 21,
In addition, for example, by controlling the address when reading the frame from the frame memory 21, it is possible to read the frame in which the pixel swap of the column is performed from the frame memory 21.

Further, here, in the pixel swap information,
Each of the second to Nth columns includes information indicating to which column the pixel swap is performed, but does not include information indicating to which column the pixel swap of the first column is performed. Therefore, in the embedding unit 22, the pixel swap of each of the second column to the Nth column is performed, but the pixel swap of the first column is not performed. The reason why the pixel swap of the first column is not performed will be described later.

When the embedded coded data is output from the frame memory 21 in step S83, the process proceeds to step S84, and it is determined whether or not the frame memory 21 stores a frame that has not been processed yet. When it is determined that the frame is stored, the process returns to step S81, and the same process is repeated for the frame that has not been processed.

If it is determined in step S84 that the frame memory 21 does not store any frame that has not yet been processed, the embedding process ends.

As described above, the positions of the pixels in each column as a set of one or more pixels forming the image stored in the frame memory 21 are pixel-swapped in accordance with the data to be embedded. When embedding data is embedded in a column, the original image data can be restored by performing the reverse pixel swap (here, column exchange), and what kind of pixel swap is performed. The embedded data can be restored on the basis of whether or not it is high. Therefore, it is possible to embed the embedded data in the image data while minimizing the deterioration of the image quality of the image data and without increasing the data amount.

That is, each column of image data (embedded image data), which is the image data in which the data to be embedded is embedded and in which the pixel position of the column has been swapped, has a correlation of the image data, that is, here. , For example, by utilizing the correlation between the original image data and the column in the same correct position, the pixel can be swapped to the original position without overhead, and the method of pixel swapping (each The embedded data can be restored according to which column the pixel is swapped. Therefore, basically, in the restored image obtained as a result, the image quality does not deteriorate due to the embedding of the embedded data.

If the embedded coded data does not have a column at the correct position, it is generally possible to restore the image data and the embedded data by utilizing the correlation between the image data as described above. It is difficult. Therefore, here, in the embedding process of FIG. 19, the first column of each frame is not replaced and is output as it is as the embedded coded data.

However, it is also possible to perform embedded coding with all columns including the first column as the target of replacement, and in this case, for example, at least one of the columns after replacement is replaced.
By including the above original position as the overhead in the embedded coded data, the original image data and the embedded data can be easily restored.

The embedded data may be embedded in the image data by replacing all the columns other than the first column of the image data at once, or the columns other than the first column of the image data may be sequentially embedded. It is also possible to embed it in the image data by replacing it.

Next, the embedded image data obtained by the embedding processing by the pixel swap method is restored to the original image and the embedded data by the following restoration processing by the pixel swap method using the correlation of the image data. can do.

That is, in the restoration processing by the pixel swap method, among the columns forming the embedded image data, the correlation between the latest restored column and another column is calculated, and the correlation with the restored column is maximized. By performing pixel swapping on the column to the right of the already restored column for all columns that make up the embedded image data, the original image data is restored, and the embedded image data The embedded data is restored based on the way of pixel swapping of each column of the embedded image data when restoring the original image data.

Next, with reference to the flow chart of FIG. 20, the restoration processing when the restoration processing by the pixel swap method is performed in the restoration device 6 of FIG. 4 will be described.

In the frame memory 31, the embedded image data supplied thereto are sequentially stored, for example, in 1-frame units.

Then, in step S91, the restoring section 32 sets a variable n for counting the number of columns in the frame.
For example, 1 is set as an initial value, and the process proceeds to step S92 to determine whether the variable n is equal to or less than N-1 obtained by subtracting 1 from N, which is the number of columns of the frame.

In step S92, the variable n is N-1.
When it is determined that the following is true, the process proceeds to step S93,
The decompression | restoration part 32 is the pixel (pixel row) of the nth column from the frame of the embedded image data memorize | stored in the frame memory 31.
For each pixel of the n-th column (pixel value of the n-th column) and arranged as an element (hereinafter, appropriately referred to as a column vector)
Generate v _n . Here, when the frame is composed of M rows of pixels, the column vector v _n (similarly to the column vector v _k described later) is an M-dimensional vector.

Thereafter, in step S94, the restoration section 32 sets the variable k for counting the column on the right side of the nth column to n + 1 as an initial value, and proceeds to step S95. In step S95, as in step S93, the restoration unit 32 reads out the pixel in the k-th column from the frame of the embedded image data, and reads the k-th pixel.
A column vector v _k having pixels in the column as elements is generated, and the process proceeds to step S96.

In step S96, the restoration unit 32 uses the column vectors v _n and v _k to obtain a correlation value representing the correlation between the nth column and the kth column.

That is, the restoring unit 32 firstly sets the column vector v
The distance d (n, k) between _n and v _k is calculated according to the following equation.

D (n, k) = | v _n −v _k | = (Σ (A (m, n) −A (m, k)) ² ) ^1/2 (1) However, the above equation In, Σ represents the summation when m is changed from 1 to M. A (i, j) is the i-th frame of the embedded image data frame to be processed.
Represents a pixel (pixel value) in the row and j-th column.

Further, the restoration unit 32 calculates the reciprocal 1 / d (n, k) of the distance d (n, k) between the column vectors v _n and v _k as
It is determined as a correlation value indicating the correlation between the nth column and the kth column.

After calculating the correlation value between the n-th column and the k-th column, the process proceeds to step S97, where the restoration unit 32 determines that the variable k is equal to or less than N-1 obtained by subtracting 1 from N, which is the number of columns in the frame. Determine if there is. When it is determined in step S97 that the variable k is N-1 or less, the process proceeds to step S98, the restoration unit 32 increments the variable k by 1, and the process returns to step S95. The processes of steps S95 to S98 are repeated until it is determined that k is not N-1 or less. That is, by this, the correlation value between the n-th column and each of the columns of the embedded image on the right side thereof is obtained.

Thereafter, in step S97, the variable k
If it is determined that is not less than or equal to N-1, the process proceeds to step S99, where the restoration unit 32 maximizes the correlation value with the n-th column.
And proceeds to step S100. Then, if k that maximizes the correlation value with the n-th column is expressed as, for example, K,
In step S100, the restoration unit 32 pixel swaps the n + 1th column and the Kth column of the frame of the embedded image data to be processed stored in the frame memory 31, that is, the Kth column to the right of the nth column. Swap to the (n + 1) th column.

After that, the procedure goes to step S101, in which the restoring unit 32 increments the variable n by 1 and returns to step S92.
Until it is determined that is not N-1 or less, step S92.
The process from S101 to S101 is repeated.

Here, in the present embodiment, the first column of the embedded image data remains the first column of the original image data. Therefore, when the variable n is 1, which is the initial value, the first column is set. The column of the embedded image data having the highest correlation with the column is pixel-swapped to the second column on the right of the first column. The column having the highest correlation with the first column is basically the second column of the original image data because of the correlation of the image data. In this case, in the embedding process of FIG. The second column of the original image data that has been pixel-swapped to that column will be returned (restored) to the original position (column).

When the variable n becomes 2, the column of embedded image data having the highest correlation with the second column that has been pixel-swapped in the original position as described above is the second column to the right of the second column. Pixels are swapped to 3 columns. The column having the highest correlation with the second column is basically the third column of the original image data due to the correlation of the image data. In this case, in the embedding process of FIG. The third column of the original image data that has been pixel swapped to some column of will be returned to its original position.

Thereafter, similarly, the embedded image data stored in the frame memory 31 is restored to the original image data.

Then, in step S92, the variable n
Is determined to be not less than or equal to N−1, that is, all of the second to Nth columns forming the embedded image data are returned to their original positions by using the correlation of the image data, whereby the frame When the stored contents of the memory 31 are restored to the original image data, the process proceeds to step S102 and the restoration unit 3
2 stores the restored image data in the frame memory 3
Read from 1. Furthermore, in step S102,
The restoration unit 32 embeds the embedded image data in the embedded image data based on the pixel swapping method (pixel swap information) of each of the second to Nth columns of the embedded image data when the embedded image data is restored to the original image data. The embedded data that was stored is restored and output.

After that, the procedure goes to step S103, in which the restoration unit 32 determines whether or not the frame of the embedded image data which has not been processed yet is stored in the frame memory 31, and determines that the frame is stored. If
Returning to step S91, the same processing is repeated for the frame of the embedded image data that has not been processed yet.

If it is determined in step S103 that the frame memory 31 does not store a frame of embedded image data that has not yet been processed, the restoration process ends.

As described above, the embedded image data, which is the image data in which the embedded data is embedded, is restored to the original image data and the embedded data by using the correlation of the image data. The encoded data can be restored to the original image and the embedded data without the overhead for the restoration. Therefore, basically, the image quality is not deteriorated by embedding the embedded data in the restored image.

In the restoration process of FIG. 20, the correlation between the latest restored column (in the case of n = 1, the first column which is not pixel-swapped during the embedding process) and the column which has not been restored yet. And based on that correlation,
I tried to detect the column that should be pixel swapped at the position next to the right of the latest column that has already been restored.
It is also possible to detect the column to be pixel-swapped to the right of the latest restored column by calculating the correlation between each column that has already been restored and the column that has not been restored. It is possible.

Further, in the above case, the pixels constituting the image data as the embedding target data are pixel-swapped on a column-by-column basis (by vertical pixel swapping) based on the embedding data. Although the embedding data is embedded, the embedding of the embedding data is not limited to this. For example, the image data as the embedding target data may be pixel-swapped row by row (horizontal line pixel swapping) or time. It is also possible to perform pixel swapping for pixel rows at the same position in a predetermined number of frames arranged in the direction.

Further, in the embedding process by the pixel swap method, for example, the image data as the embedding target data is pixel-swapped in the column unit, and further, the pixel-swap is performed in the row unit to embed the embedding data. Is.

Here, regarding the embedded image data obtained by performing pixel swapping of both columns and rows, for example, considering a restoration process for returning the column position to the original position, in the restoration process, equation (1) is used. In Σ, only the order of the added terms changes, and the added terms themselves do not change. Therefore, the distance d (n, k) obtained by the equation (1) is
There is no difference between pixel swapping of only the columns of image data and pixel swapping of both columns and rows.Therefore, embedded image data obtained by pixel swapping both columns and rows is also pixel swapped only for columns. Similar to the embedded image data obtained as described above, the original image data and the embedded data can be restored by the restoration process of FIG. That is, for the embedded image data obtained by pixel swapping both the columns and the rows, the restoration processing of FIG.
By performing each independently, the original image data and the embedded data can be restored.

As described above, in the embedding process, when the embedded data is embedded in the image data by pixel swapping both the column and the row, which pixel swap of the row and the column is performed first or later. Does not affect the restoration process. Therefore, in the embedding process,
Either of the row and column pixel swaps may be performed first or later, and in the restoration process, either of the row and column pixel swaps may be performed first or later. It is also possible to alternately perform row and column pixel swap.

However, in the embedding process, when pixel swapping is performed only for columns, the method of pixel swapping of columns of embedded image data when the embedded image data is restored to the original image data is The result will be restoration, but if you do both row and column pixel swapping,
Restoration of embedded data depends on which position (m ', n') in the restored image (original image data) the pixel at the position (m, n) in the m-th row and n-th column of the embedded image data is pixel swapped. Will result.

Further, in the embedding process of FIG. 19, only the first column of the image data as the embedding target data is fixed, and in the restoration process of FIG. However, if the restoration standard is set in advance by embedding processing and restoration processing,
It may be the last Nth column or any other column. Furthermore, the restoration criterion does not have to be all the pixels in one column, but may be one pixel in the extreme.

For example, when the pixel swap of the embedded image data is performed by utilizing the correlation of the image data with the first column as the so-called restoration reference, if the pixel swap of the first column is wrong, There is a high possibility that the pixel swap of the subsequent column (here, the column on the right side of the column where the pixel swap is erroneous) is also erroneous. In this case, since the original image data cannot be restored, the embedded data cannot be restored.

Therefore, in the embedding process, a plurality of columns (for example, a plurality of columns selected at equal intervals from all columns) are
It is possible to leave it as a reference for restoration (not to be a target of pixel swap).

Next, in the second method, in the embedding process, the bit plane for each bit of the bit string representing the pixel value of a plurality of pixels forming the image data as the embedding target data corresponds to the embedding data. Bit plane swap is performed as an operation rule, and the image data as the embedding target data is operated according to the operation rule, and the embedded data is embedded in the image data. Data is embedded.

That is, in the embedding process of the second method, as shown in FIG. 21, the image data as the embedding target data is divided into image blocks composed of a plurality of pixels, and the image blocks are It is divided into bit planes for each bit of a bit string representing the pixel value of each pixel forming the image block.

Here, assuming that, for example, 8 bits are assigned to each pixel value forming the image data as the embedding target data, the image block has the least significant bit and the least significant bit from the least significant bit to 2 bits. Bit of the eye, 3
.., the seventh bit, and the most significant bit (hereinafter, appropriately referred to as first to eighth bits), respectively.

Then, the bit planes of the image block are rearranged in accordance with the embedded data (bit plane swapping), so that the embedded data is embedded in the image data.

Note that, for example, as described above, when 8 bits are assigned to the pixel value forming the image data as the embedding target data, 8 bit planes are obtained from one image block. To be Since there are 40320 (= 8! (! Represents factorial)) ways of rearranging the eight planes of bit planes (method of swapping bit planes), one image block can have a maximum of log ₂ It is possible to embed embedded data of (8!) Bits (however, the fractional part is truncated). Therefore, in the embedding process, the data to be embedded is divided into such number of bits and embedded in each image block that constitutes the image data as the embedding target data.

Here, in the second method, as described above,
Since the embedded data is embedded according to the operation rule of swapping the bit plane of each image block of the image data as the embedding target data in correspondence with the embedded data, this second method will be described below. Appropriately called the bit plane swap method.

Next, with reference to the flow chart of FIG. 22, the embedding processing when the embedding encoder 3 of FIG. 3 performs the embedding processing of the bit plane swap method will be described.

First, in step S111,
The embedding unit 22 reads the embedded data that can be embedded in one frame image from the embedded database 2 and formats it into a unit that can be embedded in one image block (divides the embedded data). Further, in step S111, the embedding unit 22 causes the frame memory 21 to store the image data of the frame that is not yet the processing target among the frames of the image data as the embedding target data stored in the embedding target database 1.

Then, proceeding to step S112, the embedding section 22 divides the one-frame image stored in the frame memory 21 into image blocks of, for example, 4 × 4 pixels in width × length, and proceeds to step S113. Step S11
In 3, the embedding unit 22 divides each image block forming one frame of image data stored in the frame memory 21 into bit planes, and proceeds to step S114.

In step S114, the embedding section 22 is used.
Is an image block divided into bit planes,
For example, in raster scan order, a block that has not yet been set as a block of interest is set as a block of interest, and the bit plane of the block of interest is swapped with the bit planes corresponding to one unit of embedded data obtained in step 111,
The image block after the bit plane swap is written in the frame memory 21 in a form of overwriting.

After that, the process proceeds to step S115, where the embedding unit 22 determines whether all the image blocks forming one frame of the image stored in the frame memory 21 have been processed as a target block (process of rearranging bit planes). judge. If it is determined in step S115 that all blocks of one frame have not been processed yet, the process returns to step S114 and the embedding unit 2
In the raster scan order 2, the image block that is not the target block is newly set as the target block, and the same process is repeated thereafter.

On the other hand, if it is determined in step S115 that all blocks of one frame have been processed as the block of interest, the process proceeds to step S116, where the embedding unit 22 performs bit plane swap corresponding to the embedded data. One frame of image data composed of image blocks, that is, embedded image data, is read from the frame memory 21 and output, and the process ends.

The above-mentioned embedding process is repeated for each frame of the image data as the embedding target data stored in the embedding target database 1.

As described above, by bit-plane swapping the bit planes of each image block corresponding to the embedding data, when embedding the embedding data in each image block, the opposite bit plane swap is performed. By doing so, the original image data can be restored, and further, the embedded data can be restored based on what kind of bit plane swap was performed. Therefore, it is possible to embed the embedded data in the image while minimizing the deterioration of the image quality of the image and without increasing the data amount.

That is, in the image, the correlation between the bits in the bit planes forming the image block is
Basically, the higher the bit plane of the upper bit, the higher (the lower the bit plane of the lower bit). Therefore, by utilizing the correlation of such an image, that is, the correlation between the bits forming the bit planes here, the bit planes swapped with the bit planes corresponding to the embedded data have no overhead, You can rearrange them in their original order, and depending on how you rearrange them,
The embedded data can be restored. Therefore, basically, the restored image does not deteriorate in image quality due to the embedding of the embedded data.

Next, the embedded image data obtained by the embedding processing by the bit plane swap method is restored to the original image data by the restoration processing by the bit plane swap method using the correlation of the image data as follows. Can be restored to data.

That is, in the restoration processing by the bit plane swap method, the embedded image data for each frame is
The image blocks are divided into the same image blocks as in the embedding process by the bit plane swap, and each image block is further divided into bit planes for each bit of the bit string representing the pixel value of the pixels forming the image block. Then, the correlation value representing the correlation between the bits in each bit plane forming each image block is obtained.

Specifically, for example, as shown in FIG. 23A, first, the bit plane for calculating the correlation value is taken out from the image block as the bit plane of interest.
Then, if the image block is composed of 4 × 4 pixels as described above, for example, the target bit plane will also be composed of 4 × 4 bits in width × length.
From the target bit plane of 4 × 4 bits, for example,
As shown in FIG. 23B, 4 of a predetermined size centering on each of the 2 × 2 bits A, B, C and D in the central portion thereof.
One small block is constructed. Here, FIG. 23B shows a small block of 3 × 3 bits configured around the bit A.

In the restoration processing by the bit plane swap method, for each small block obtained from the bit plane of interest, the number of bits having the same value as the central bit is counted, and the count value is the central bit of the small block. And other bits are obtained as a correlation value.
Here, in the small block centered on the bit A shown in FIG. 23B, the central bit (here, bit A) is 0, and the upper left, upper, upper right, right, lower right, lower, The lower left and left bits are 1, 0, 1, respectively
It is 0, 0, 1, 0, 0. Therefore, the center bit having the same value as 0 has the same value as the upper, left, lower right, lower left, and left 5 bits of the center bit. The correlation value between them becomes 5.

In the restoration processing by the bit plane swap method, the correlation value is obtained for each small block centered on each of the bits A to D obtained from the bit plane of interest as described above. Further, the correlation values of the small blocks centered on the respective bits A to D thus obtained are added, and the added value is set as the correlation value of the bit plane of interest.

Each bit plane forming an image block is sequentially set as a target bit plane, and the correlation value is obtained as described above. Then, when the correlation values of all the bit planes of the image block can be obtained, as shown in FIG. 24, the bit plane having the high correlation value is located on the most significant bit side, and the bit plane having the low correlation value is the least significant bit. The bit planes are bit plane swapped so that they are located on the side, that is, in ascending order of the correlation values. As a result, the image block is restored. here,
In FIG. 24, a bit plane having a lower density represents a bit plane having a higher correlation value.

After the image block is restored, the arrangement of bit planes forming the restored image block (hereinafter, appropriately referred to as a restoration block) and the arrangement of bit planes forming the image block before restoration are set. By comparing, when restoring the image block, it is detected how each bit plane was bit plane swapped, and the embedded data embedded in the restored block is detected based on the way of the bit plane swap. Restored.

Next, referring to the flowchart in FIG.
The restoration processing when the restoration processing by the bit plane swap method is performed in the restoration device 6 in FIG. 4 will be described.

In step S121, the restoration section 32 stores the embedded image data of one frame in the frame memory 31, and proceeds to step S122. In step S122, the restoration unit 32 blocks the embedded image data of one frame stored in the frame memory 31 into image blocks as in the embedding process, and proceeds to step S123. In step S123,
The restoration unit 32 divides each image block into bit planes, and proceeds to step S124.

In step S124, the restoration unit 32 sequentially sets each image block divided into bit planes, for example, in the raster scan order as a target block, and calculates the correlation value of each bit plane of the target block.
The calculation is performed as described with reference to FIG. 23, and the process proceeds to step S125. In step S125, the restoration unit 32 bit-plane swaps the bit planes of the block of interest so that the correlation values of the bit planes are arranged in ascending order, and as a result,
Restore the image blocks that make up the original image data. Furthermore, in step S125, the restoration unit 32 restores the restored image block (restored block) to the frame memory 31.
In the embedding process, what bitplane swap is performed for each restored block by comparing the bitplane arrangement of the restored block with the arrangement of the bitplanes of the image block before restoration. Recognize what was done. Then, the restoration unit 32 restores the embedded data embedded in the restoration block based on the recognition result.

After that, proceeding to step S126, the restoration section 32 determines whether or not all the image blocks of the embedded image data of one frame stored in the frame memory 31 have been processed as the block of interest, and one frame If it is determined that all blocks have not been processed, the process returns to step S124, and an image block that has not yet been set as a target block in the raster scan order is set as a new target block, and the same process is repeated.

On the other hand, if it is determined in step S126 that all the image blocks of one frame stored in the frame memory 31 have been processed, the process proceeds to step S127, in which the restoring unit 32 restores the restored blocks stored in the frame memory 31. The image data of one frame constituted by, that is, the restored original image data is read and output. Further, in step S126, the restoration unit 32 outputs the embedded data restored in step S125,
The restoration process ends.

The above restoration process is repeated for each frame of embedded image data.

As described above, the embedded image data in which the embedded data is embedded by performing the bit plane swapping on the bit planes of the image data as the embedding target data is subjected to the bit plane swap and the image correlation is used. Thus, since the bit planes are returned to the original image data, the embedded image data can be restored to the original image data and the embedded data without overhead. Therefore, basically, the image quality is not deteriorated by embedding the embedded data in the restored image.

Next, in the third method, in the embedding process, the lines (horizontal lines or vertical lines, or lines composed of a plurality of pixels lined up in the time direction) forming the image data to be embedded are The operation rule is to rotate the line corresponding to the data to be embedded (thus performing the operation in the space direction or the time direction), and the image data as the embedding target data is operated according to the operation rule. Then, the embedded data is embedded in the image data.

That is, in the embedding process of the third method, for example, each line in the horizontal direction (a series of pixels arranged in the horizontal direction) (horizontal line) of the image data as the embedding target data corresponds to the embedded data. The embedded data is embedded in each horizontal line by performing line rotation that shifts in the horizontal direction by an amount corresponding to that.

Here, in the third method, as described above,
Since the embedding data is embedded according to the operation rule of rotating the line of the image data as the embedding target data in correspondence with the embedding data, the third method will be appropriately referred to as a line rotation method hereinafter.

Next, with reference to the flow chart of FIG. 26, the embedding process when the embedding encoder 3 of FIG. 3 performs the embedding process by the line rotation method will be described.

The image data as the embedding target data stored therein is read from the embedding target database 1 and sequentially supplied and stored in the frame memory 21.

Then, the embedding section 22 carries out step S1.
At 31, the horizontal line which constitutes the image data stored in the frame memory 21 is read out as a line of interest as a line of interest, and the process proceeds to step S132. In step S132, the embedding unit 22 determines whether the line of interest is the first horizontal line (the uppermost horizontal line of the image data of one frame). In step S132, the attention line is the first
If it is determined that the line is a horizontal line, step S133.
Then, the embedding unit 22 writes the first horizontal line in the frame memory 21 as it is by overwriting, and proceeds to step S137. That is, the first horizontal line is not specifically operated (line rotation), and therefore, the embedded data is not embedded.

If it is determined in step S132 that the line of interest is not the first horizontal line, that is,
If the line of interest is any horizontal line after the second horizontal line, the process proceeds to step S134, the embedding unit 22 reads the embedded data to be embedded in the line of interest from the embedded database 2, and the process proceeds to step S135. . In step S135, the embedding unit 2
2 rotates the line of interest in the horizontal direction by, for example, the number of pixels corresponding to the embedded data read in step S134.

That is, for example, assuming that the N-th horizontal line (N ≠ 1) is the target line as shown in FIG. 27A, the embedding section 22 displays the N-th horizontal line as shown in FIG. 27B. The horizontal line is slid (shifted) by the same number of pixels as the value of the embedded data in the left or right direction, which is the horizontal direction, for example, in the right direction. And
By the slide, the embedding portion 22 fits the portion of the Nth horizontal line protruding to the right of the frame into the left side of the Nth horizontal line, as shown in FIG. 27C.

The embedding unit 22 rotates the line of interest, and when the embedded data is embedded in the line of interest, the process proceeds to step S136, and the line of interest after the rotation is overwritten in the frame memory 21. , And proceeds to step S137.

[0287] In step S137, the embedding unit 22
Determines whether all the horizontal lines of the image data stored in the frame memory 21 have been processed as the lines of interest. If it is determined in step S137 that all the horizontal lines of the image data stored in the frame memory 21 have not been set as the target line, the process returns to step S131, and the horizontal lines that are not set as the target line (for example, until now). The horizontal line, which is one line below the line of interest, is newly set as the line of interest, and the same processing is repeated thereafter.

If it is determined in step S137 that all the horizontal lines of the image data stored in the frame memory 21 are the target lines, that is, the one frame of image data stored in the frame memory 21 When the embedded data is embedded in each horizontal line (excluding the first horizontal line in this case) to generate embedded image data, the embedding unit 22 stores the embedded image data of the one frame in the frame memory. The data is read out from 21 and output, and the process ends.

The embedding process described above is repeated for each frame of the image data stored in the frame memory 21.

According to the embedding processing of FIG. 26, one frame of image data is the following embedded image data.

That is, for example, image data as shown in FIG.
In the case of embedding 00, ..., As shown in FIG. 28B, the first horizontal line is directly output and the second horizontal line is output.
The horizontal line is rotated to the right by 10 pixels, which is the same value as the first embedded data. Further, the third horizontal line is rotated to the right by 150 pixels, which is the same value as the second embedded data, and the fourth horizontal line is 200 pixels, which is the same value as the third embedded data. Just rotated to the right. Similarly, after the fifth horizontal line, the pixels are rotated rightward by the number of pixels corresponding to the embedded data.

As described above, the horizontal lines forming the image stored in the frame memory 21 are rotated rightward by the number of pixels corresponding to the data to be embedded, so that the data to be embedded in each horizontal line. When embedding, the original image data can be restored by performing the reverse rotation (reverse rotation), and further, based on the rotation amount when the reverse rotation is performed, The embedded data that was embedded can be restored. Therefore, it is possible to embed the embedded data in the image data while minimizing the deterioration of the image quality of the image data and without increasing the data amount.

That is, the horizontal line in which the embedded data is embedded is rotated by the rotation amount (the number of pixels) corresponding to the embedded data from the original position. Such a horizontal line is correlated with the image data. By taking advantage of the correlation between the sex, ie here the horizontal line in its original position, it can be returned to its original position without overhead. Furthermore, by returning the horizontal line to the original position, the rotation amount by which the horizontal line is rotated can be recognized by the embedding process, and based on the rotation amount, the embedding target embedded in the horizontal line is embedded. Data can be restored. Therefore, basically, in the restored image obtained as a result, the image quality does not deteriorate due to the embedding of the embedded data.

When the embedded image data does not have a horizontal line at the original position, it is generally used to restore the image data and the embedded data by utilizing the correlation of images as described above. It is difficult. Therefore, here, in the embedding process of FIG. 26, the embedded data is not embedded (not rotated) in the first horizontal line of each frame, and is directly used as the embedded image data.

Next, the embedded image data obtained by the embedding processing by the line rotation method is converted into the original image data and the embedding data by the restoration processing by the line rotation method utilizing the correlation of the image data as follows. Can be restored.

That is, in the restoration processing by the line rotation method, each horizontal line of the embedded image data is sequentially set as the target line from the first horizontal line, and the horizontal line one line above the target line is selected. Is the reference line.

The attention line is the horizontal line which is about to be returned to its original position. In this case, the horizontal lines are sequentially set as the attention line from the upper side to the lower side. The reference line is a horizontal line that has already been returned to its original position.

In the restoration processing by the line rotation method, the line of interest is then rotated pixel by pixel in either the right or left direction, and the correlation between the line of interest after rotation and the reference line is determined. Is calculated.

Specifically, for example, as shown in FIG. 29A, the line of interest is not rotated (the line of interest is rotated by 0 pixels), and the pixel values of the pixels forming the line of interest are set as follows: The reciprocal of the sum of the absolute differences between the pixel values of the corresponding pixels forming the reference line and the pixel value of the corresponding line is calculated as the correlation value between the reference line and the reference line. Further, as shown in FIG. 29B, the line of interest is rotated by one pixel to either the right side or the left side (in FIG. 29, it is rotated in the left direction which is the reverse direction to that described with reference to FIG. 27). ), The correlation value with the reference line is calculated for the line of interest after the rotation. Similarly, the line of interest is the original position (the original position in the embedded image data).
The correlation value between the reference line and the reference line is calculated while sequentially rotating the line of interest to 2 pixels, 3 pixels, ...

As described above, after the correlation value for the target line rotated by each rotation amount (here, the number of pixels forming the horizontal line) is calculated, the target line is selected from each rotation amount. The amount of rotation that maximizes the correlation value for is detected. Then, the detected rotation amount is determined as the rotation amount for returning the line of interest to the original position (the determined rotation amount will be referred to as
The target line is rotated by the determined rotation amount in the same direction as in the case of calculating the correlation value (in the above case, leftward), and is returned to the original position. That is, the line of interest is restored by this.

Further, the embedded data embedded in the line of interest is restored based on the determined rotation amount.

Next, referring to the flowchart in FIG. 30,
The restoration processing when the restoration processing by the line rotation method is performed in the restoration device 6 of FIG. 4 will be described.

In the frame memory 31, the embedded image data supplied thereto is sequentially stored, for example, in 1-frame units.

Then, the restoration unit 32 carries out step S141.
In, in the embedded image data stored in the frame memory 31, the horizontal line of the upper row of the horizontal lines not yet set as the target line is read as the target line, and the process proceeds to step S142. Step S14
In 2, the restoration unit 32 determines whether the line of interest is the first horizontal line. When it is determined in step S142 that the line of interest is the first horizontal line, the process proceeds to step S143, and the restoration unit 32 overwrites the line of interest, which is the first horizontal line, on the frame memory 31 as it is. Write and proceed to step S150. That is, in the embedding process of FIG. 28, as described above, since the embedded data is not embedded in the first horizontal line (therefore, it is not rotated), the first horizontal line is not particularly operated.

If it is determined in step S142 that the line of interest is not the first horizontal line, that is, if the line of interest is any horizontal line after the second horizontal line, the process proceeds to step S144. The restoration unit 32 reads out from the frame memory 31 the horizontal line one line above the line of interest as the reference line, and proceeds to step S145. In step S145, the restoration unit 32
Calculates the correlation value (line correlation value) between the line of interest and the reference line, stores it in its built-in memory, and proceeds to step S146.

In step S146, the restoration unit 32 rotates the line of interest by, for example, one pixel in the left direction, and proceeds to step S147. In step S147, the restoration unit 32 rotates the target line in step S146 so that each pixel of the target line after rotation has returned to the position of each pixel of the target line stored in the frame memory 31. Determine whether

If it is determined in step S147 that each pixel of the line of interest after rotation has not returned to the position of each pixel of the line of interest stored in the frame memory 31, the process returns to step S145, and after rotation. The correlation value between the line of interest and the reference line is calculated, and the same process is repeated thereafter.

Further, in step S147, each pixel of the line of interest after rotation is changed to the frame memory 3
If it is determined that the pixel has returned to the position of each pixel of the line of interest stored in 1, the process proceeds to step S148, and the restoration unit 3
2 indicates steps S145 to S1 for the line of interest.
The maximum value is obtained from the correlation values for each rotation amount obtained by performing the loop processing of 47. Further, in step S148, the restoration unit 32 detects the rotation amount that gives the maximum correlation value,
The rotation amount is determined as the determined rotation amount. Then, the restoration unit 32 restores and outputs the embedded data embedded in the line of interest based on the determined rotation amount, that is, outputs the determined rotation amount as it is as embedded data, and proceeds to step S149. move on.

In step S149, the restoring unit 32 restores the line of interest by reading the line of interest stored in the frame memory 31 and rotating it to the left by the determined rotation amount (the line of interest is the original image. Return to position in data). Further, in step S149, the restoration unit 32 writes the line of interest after rotation in the frame memory 31 in a form of overwriting, and proceeds to step S150.

In step S150, the restoration section 32 determines whether or not all the horizontal lines of the embedded image data stored in the frame memory 31 have been processed as lines of interest. When it is determined in step S150 that all the horizontal lines of the embedded image data stored in the frame memory 31 have not been set as the target line, the process returns to step S141, and the line below the one line that has been set as the target line until now is displayed. The horizontal line is newly set as the line of interest, and the same processing is repeated thereafter.

If it is determined in step S150 that all the horizontal lines of the embedded image data stored in the frame memory 31 are the target lines,
That is, the embedded data embedded in each horizontal line (but excluding the first horizontal line here) of the embedded image data of one frame stored in the frame memory 31 is restored, and the embedded image data is restored. ,
When the image data is restored to the original image data, the restoration unit 32 reads the restored image data from the frame memory 31, outputs the read image data, and ends the process.

The above restoration process is repeated for each frame of the embedded image data stored in the frame memory 31.

As described above, the embedded image data in which the embedded data is embedded by rotating the embedded horizontal line is rotated, the embedded image data is rotated, and the horizontal line is moved to the original position by utilizing the correlation of images. Since the restoration is performed, the embedded image data can be restored to the original image data and the embedded data without overhead. Therefore, basically, the image quality is not deteriorated by embedding the embedded data in the restored image.

In the restoration processing of FIG. 30, the sum of absolute differences between corresponding pixels (sum of absolute differences) is used as the correlation value representing the correlation between the line of interest and the reference line. However, it is also possible to use, for example, the sum of squares of pixel differences.

In the embedding process of FIG. 26, since one horizontal line is rotated according to the embedding data, one horizontal line has a value within the range of the number of pixels forming the horizontal line. It is possible to embed the data to be embedded. However, even when embedding embedded data having a value within a range larger than the number of pixels forming one horizontal line, for example, a plurality of horizontal lines such as two horizontal lines are rotated according to the embedded data. It is possible to do it.

By the way, in the embedding processing by the line rotation method of FIG. 26 described above, the horizontal line formed by the pixels lined up in the horizontal direction of the image data as the embedding target data is rotated in the horizontal direction to embed the embedded data. Although the embedded data is embedded, the embedded data is embedded in other
It is also possible to rotate a vertical line composed of pixels arranged in the vertical direction of the image data in the vertical direction, or rotate a line composed of pixels arranged in the oblique direction in the oblique direction. Is.

In the case of the line rotation method, the embedded data is embedded by rotating either the horizontal line or the vertical line, and then the other of the horizontal line or the vertical line is rotated. As a result, it is possible to further embed the embedded data.

That is, for example, as shown in FIG. 31A, in the embedding processing by the line rotation method, the horizontal line of the image data as the embedding target data is converted into a predetermined embedding data (hereinafter, referred to as the first embedding data as appropriate). By embedding the first embedded data by rotating it corresponding to
The vertical line of the image data as the resulting embedded coded data (hereinafter, appropriately referred to as first embedded coded data) is converted into another embedded data (hereinafter,
It is possible to embed the second embedded data by rotating the data corresponding to the second embedded data.

Now, the embedded coded data generated by embedding the second embedded data by rotating the vertical line of the image data as the first embedded coded data is converted into the second embedded coded data. In that case, since the embedded data is embedded in the second embedded coded data by rotating not only the horizontal lines of the original image data but also the vertical lines, the embedding process of FIG. As described above, more embedded data can be embedded as compared with the case of rotating only the horizontal line of the image data.

That is, for example, when the number of horizontal lines and vertical lines of the image data as the embedding target data is the same, both horizontal and vertical lines of the image data are generated by rotation. In the second embedded coded data, it is possible to embed embedded data having a data amount twice as large as that in the first embedded coded data generated by rotating only horizontal lines of image data.

The restoration process for restoring the above-mentioned second embedded coded data to the embedding target data and the first and second embedded data also uses the correlation of images to Can be done as

That is, when the horizontal line of the image data as the embedding target data is rotated, the correlation in the image data in the vertical direction which is the spatial direction orthogonal to the horizontal line is destroyed. Therefore, with respect to the embedded coded data generated by rotating only the horizontal line of the image data, the horizontal line is restored to the original correlation in the vertical direction (in the restoration process of FIG. 30, the reference line is used). The original image data can be restored by rotating so that the correlation value between and becomes maximum.

The embedded coded data generated by rotating only the vertical lines of the image data also has the same original code as the embedded coded data generated by rotating the horizontal lines. Image data can be restored.

That is, when the vertical line of the image data as the embedding target data is rotated, the correlation in the horizontal direction which is the spatial direction orthogonal to the vertical line in the image data is destroyed. Therefore, the embedded coded data generated by rotating the vertical line of the image data is restored to the original image data by rotating the vertical line so as to restore the horizontal correlation. be able to.

As described above, when the horizontal line of the image data is rotated, the vertical correlation is destroyed.
Horizontal correlation is not affected. Further, when the vertical line of the image data is rotated, the correlation in the horizontal direction is destroyed, but the correlation in the vertical direction is not affected. Then, the embedded coded data generated by rotating the horizontal line of the image data can be restored to the original image data by using only the correlation in the vertical direction, and the vertical line of the image data is rotated. The embedded coded data generated by doing so can be restored to the original image data by using only the correlation in the horizontal direction.

Therefore, as shown in FIG. 31A, the horizontal line of the image data as the embedding target data is set to the first line.
The first embedded coded data is generated by rotating the first embedded coded data corresponding to the first embedded coded data, and the vertical line of the image data as the first embedded coded data corresponds to the second embedded data. As shown in FIG. 31B, the second embedded coded data generated by rotating the first embedded code by rotating the vertical line to restore the correlation in the horizontal direction to the original one. Data and second embedded data can be restored. Then, the first embedded coded data can be restored to the embedding target data and the first embedded data by rotating the horizontal line so as to restore the vertical correlation. That is, the second embedded coded data can be restored to the original embedding target data and the first and second embedded data.

The embedding by the rotation of the horizontal line and the embedding by the rotation of the vertical line are both based on the line rotation method in which the embedding target data is operated according to the operation rule of rotating the line corresponding to the embedding data. Although it is embedded, horizontal line rotation only affects vertical correlation, and vertical line rotation only affects horizontal correlation, so horizontal line rotation and vertical line rotation Rotation can be said to be an independent operation or an orthogonal operation. As described above, since the rotation of the horizontal line and the rotation of the vertical line are independent operations, the horizontal line of the image data is rotated corresponding to the first embedded data, and the vertical line is further rotated. Second obtained by rotation corresponding to the second embedded data
The embedded coded data can be restored to the original image data and the first and second embedded data.

By the way, since the second embedded coded data is obtained by rotating the horizontal line and the vertical line of the image data as the embedding target data, the so-called spatial correlation of the image data is all It has been destroyed in the spatial direction.

Therefore, the image data as the second embedded coded data is operated not only by the line rotation method but also by other operation rules, for example, the above-mentioned pixel swap method or bit plane swap. If the embedded data is further embedded according to the method, it is difficult to restore the second embedded coded data and the embedded data embedded in the second embedded coded data by the restoration processing by the corresponding method. Becomes

That is, for the image data as the second embedded coded data, another embedded data (hereinafter, referred to as the third
Embedded data), the third embedded data is embedded by rotating a horizontal line or a vertical line of the second embedded encoded data in correspondence with the third embedded data. Be done.

However, the rotation of the horizontal line or the vertical line corresponding to the third embedded data is the rotation of the horizontal line for embedding the first embedded data or the embedding of the second embedded data. It is not possible to distinguish whether the rotation of the horizontal line is due to the embedding of the first or third embedded data, because the operation overlaps the rotation of the vertical line for performing Alternatively, it is not possible to distinguish whether the rotation of the vertical line is due to the embedding of the second or third embedded data, and therefore, the second embedded coded data also has the third embedded data. The embedded data of cannot be restored.

Further, in the decoding processing by the pixel swap method or the bit plane swap method, as described above, the correlation between the spatially adjacent pixels is used, and the original image data and the object embedded in the original image data are used. The embedded data is restored.

However, with respect to the image data as the second embedded coded image data, since the correlation in the spatial direction is destroyed in all directions, such second embedded coded data is The embedding of the third embedded data by the pixel swap method or the bit plane swap method has no bias in energy, for example, noise is used as the data to be embedded by the pixel swap method or the bit plane swap method. It is equivalent to embedding data.

Therefore, with respect to the embedded coded data obtained by such embedding, the spatial correlation of the image data (image data having value as information) cannot be used, and the pixel swap method is used. Or, depending on the restoration process by the bit plane swap method,
The original data to be embedded and the embedded data cannot be restored.

As described above, the third embedded data is embedded in the second embedded coded data in which the spatial correlation is destroyed in all directions by the pixel swap method or the bit plane swap method. ,
When the embedded coded data is generated, the correlation of the image data as the second embedded coded data is used for the embedded coded data (hereinafter, appropriately referred to as the third embedded coded data). In some cases, the second embedded coded data and the third embedded data cannot be restored.

By the way, in the case where the third embedded data is embedded by the same line rotation method to the second embedded coded data generated by the embedding processing by the line rotation method, it is described above. As described above, the operation performed when generating the second embedded coded data and the operation performed when generating the third embedded coded data may overlap with each other. Even if the image data as the encoded data (or the embedding target data) can be restored, it is difficult to restore all the first to third embedded data.

On the other hand, with respect to the second embedded coded data generated by the embedding processing by the line rotation method, the third embedding data is further different from the line rotation method, for example, by the bit plane swap method. When embedded, the operation performed when generating the second embedded coded data does not overlap with the operation performed when generating the third embedded coded data.

Therefore, in this case, if the image data as the first embedded coded data (or the embedding target data) can be restored, the first to third
All embedded data of can also be restored.

That is, if the first embedded coded data can be restored from the third embedded coded data, the operation of generating the third embedded coded data from the first embedded coded data is performed. Specified. And
The operation of generating the third embedded coded data from the first embedded coded data includes the operation of embedding the second embedded data in the first embedded coded data and the second embedded data obtained by the operation. The operation includes embedding the third embedded data in the embedded coded data, and in this case, the operation of embedding the second embedded data and the operation of embedding the third embedded data overlap. There is nothing to do.

Therefore, if the operation for generating the third embedded coded data from the first embedded coded data is specified, the third embedded data is embedded in the second embedded coded data based on the operation. The operation of embedding the data (also the operation of embedding the second embedded data in the first embedded coded data) can be specified.

Therefore, with reference to FIG. 32, the operation for generating the third embedded coded data from the first embedded coded data is specified, and further, the third embedded coded data is processed by the third embedded coded data. A method of identifying the operation in which the embedded data is embedded will be described.

In the following, for example, the first embedded coded data is generated by rotating the horizontal line of the image data as the embedding target data in correspondence with the first embedded data, and the second embedded coded data. The embedded coded data is generated by rotating the vertical line of the image data as the first embedded coded data in correspondence with the second embedded data. Furthermore, it is assumed that the third embedded coded data is generated by bit plane swapping the image data as the second embedded coded data in correspondence with the third embedded data.

Further, in the embodiments shown in FIGS. 21 to 25, the bit planes of the pixel groups forming the image block of 4 × 4 pixels are adopted as the bit planes.
In the following, for example, a bit plane of a pixel group that constitutes one vertical line is adopted.

That is, in the following, in the embedding processing and the restoration processing by the bit plane swap method, 1
It is assumed that one vertical line is regarded as one image block, and bit planes for each bit of the bit string representing the pixel value of each pixel forming the vertical line are bit plane swapped. However, the image block that is the target of the bit plane swap is not limited to the vertical line.

In this case, the embedding process for embedding the first to third embedded data in the image data as the embedding target data is performed as shown in FIG. 32A.

That is, first, as explained in FIG. 31A,
The horizontal line of the image data as the embedding target data is rotated corresponding to the first embedding data, so that the first embedding coded data in which the first embedding data is embedded in the embedding target data. Is generated, and the vertical line of the image data as the first embedded coded data is rotated corresponding to the second embedded data, so that the first embedded coded data is rotated with respect to the first embedded coded data. Second embedded coded data in which the two embedded data are embedded is generated.

Then, as shown in FIG. 32A, each vertical line of the image data as the second embedded coded data is used as an image block, and the bit plane of each image block is associated with the third embedded data. By performing the bit plane swap, the third embedded coded data in which the third embedded data is embedded in the second embedded coded data is generated.

In the image data as the second embedded coded data, it is assumed that a certain vertical line is at the original position (the original position in the image data as the first embedded coded data). Considering that the vertical line at the position of is the reference line, and the vertical line to the right of the reference line is the attention line, and the attention line is returned to the original position, the attention line is Similar to the case described above, the original position can be restored by performing the restoration process by the line rotation method.

That is, the line of interest is rotated in the vertical direction while changing the rotation amount, and the correlation value with the reference line is calculated for each rotation amount. The position rotated by the rotation amount that maximizes the correlation value is the original position of the line of interest.

By the way, as described above, the correlation value obtained for the line of interest is calculated with respect to the rotation amount of the line of interest, and is therefore a function with the rotation amount as an argument.

Now, the function representing this correlation value is referred to as a correlation value function, and the Nth vertical line from the left (Nth vertical line) of the second embedded coded data is taken as the attention line, and the attention line is As for the correlation value function, the correlation value function is as shown in FIG. 32B, for example. Note that in FIG. 32 (similarly in FIGS. 33 and 41 described later), the correlation value function is not a function representing the correlation value itself for each rotation amount but a function representing the reciprocal of the correlation value.

In FIG. 32B, the correlation value function has a steep concave portion in the vicinity of the rotation amount where the function value becomes the minimum. The correlation value function of FIG. 32B is a function that represents the reciprocal of the correlation value, as described above,
Since the rotation amount that minimizes the function value is the rotation amount that maximizes the correlation value, it is the rotation amount (determined rotation amount) that returns the target line to the original position. Therefore, if the correlation value function is a function that represents the correlation value itself, the correlation value function has a sharp convex portion, and the rotation amount that maximizes the function value is the determined rotation amount.

In the image data as the second embedded coded data, the correlation value function of the line of interest is shown in FIG.
As described above, the image data has a feature of having a steep concave portion. Such a feature is that the image data before the second embedded data is embedded in the second embedded coded data, that is, This is due to the horizontal correlation (here, the correlation between the line of interest and the reference line) possessed by the image data as the first embedded coded data.

That is, the first embedded coded data is obtained by rotating the horizontal line of the original image data (embedding target data), and as described above, its vertical correlation is destroyed. The horizontal correlation is not destroyed, and the original horizontal correlation of the image data is maintained. Therefore, when the vertical line of the second embedded coded data is rotated, when the vertical line is rotated near the position (original position) in the first embedded coded data, the correlation value becomes Increase sharply. Therefore, the correlation value function has a steep concave portion as shown in FIG. 32B.

Here, as described above, the function value of the correlation value function is the reciprocal of the correlation value. Therefore, considering the correlation value, the correlation value is in the vicinity of the rotation amount where the value becomes maximum,
A steep convex portion appears.

On the other hand, as shown in FIG. 32A, each vertical line of the image data as the second embedded coded data is used as an image block, and the bit plane of each image block corresponds to the third embedded data. The third embedded coded data in which the third embedded data is embedded in the second embedded coded data by the bit plane swapping is performed as compared with the case of the above-mentioned second embedded coded data. Similarly, when the N-th vertical line is set as a line of interest and a correlation value function representing the correlation between the line of interest and a reference line that is the vertical line adjacent to the left is drawn, the correlation value function is, for example, Three
2C as shown in FIG.

In FIG. 32C, the correlation value function does not have the feature that a sharp concave portion appears unlike the correlation value function of FIG. 32B.

As described above, the correlation value function for the vertical line of the third embedded coded data does not have a sharp recess unlike the correlation value function for the vertical line of the second embedded coded data. For the following reasons.

That is, the third embedded coded data is obtained by bit-plane swapping the bit planes of the image block with the vertical line of the second embedded coded data as an image block. For example, the spatial correlation of the image data is destroyed. Then, in order to restore the correlation of the image data in which the correlation is destroyed by the bit plane swap, it is necessary to perform the bit plane swap in which the bit planes are returned to the original alignment, and the vertical line is rotated to which position. Even then, the correlation that has been destroyed by the bitplane swap cannot be restored.

Therefore, the correlation value between the vertical line of the image data as the third embedded coded data and the reference line does not change so much depending on the rotation amount of the vertical line. Therefore, the correlation value function, which is the function that represents the reciprocal of the correlation value,
As shown in FIG. 32C, there is no abrupt recess, FIG.
It becomes flat compared to the case of.

By the way, a certain vertical line of the third embedded coded data is set as a target line, the target line is set as an image block, the bit plane swap is performed, and the correlation value is calculated for the target line after the bit plane swap. When the function is obtained, if the bit plane of the line of interest returns to the original arrangement (arrangement in the second embedded coded data) due to the bit plane swap, the correlation value function is shown in FIG. 32B. As described above, it has a sharp recess.

On the other hand, if the bit planes of the line of interest do not match the original alignment, the correlation value function is the same as that when bit plane swap is not performed, that is, as shown in FIG. 32C, it is steep. There will be no concave portion.

Therefore, in the third embedded coded data,
With a certain vertical line as the attention line, the bit planes of the attention line are bit plane swapped in a predetermined pattern, and the resulting vertical line (hereinafter, appropriately,
When a correlation value function of a temporary restoration vertical line) is obtained and the correlation value function has a steep concave portion, the temporary restoration vertical line is rotated by the rotation amount that minimizes the function value. The position is the original position of the temporary restored vertical line, and the arrangement of the bit planes forming the temporary restored vertical line is the original arrangement.

From the above, if the correlation value function having the steep concave portion is obtained for the temporary restored vertical line, the temporary restored vertical line becomes the restored result of the target line, and further,
The third embedded data embedded in the target line is restored based on the bit plane swap pattern of the bit plane of the target line (hereinafter, appropriately referred to as a swap pattern) when the provisional restoration vertical line is obtained from the target line. can do. That is, as shown in FIG. 32D, the third embedded coded data can be restored to the second embedded coded data and the third embedded data.

Here, although the correlation value function shown in FIGS. 32B and 32C is a schematic one, when the correlation value function is obtained for actual image data on which a natural image is displayed,
The correlation value function is as shown in FIG. 33, for example.

That is, in FIG. 33, horizontal line rotation and vertical line rotation are performed only for the Y component of image data composed of 1920 × 1035 pixels, and further vertical line bit plane swap is performed. A vertical line of the obtained third embedded coded data is set as a target line, and a correlation value function obtained for the target line is shown.

In FIG. 33, the correlation value function indicated by F ₁ is
This is a case where the bit plane of the line of interest is different from the original arrangement, and is relatively flat without any sharp recess.

The correlation value function indicated by F ₂ in FIG. 33 is used when the bit plane of the line of interest matches the original arrangement, and has a very steep concave portion.

Next, FIG. 34 is a diagram showing a case where third embedded coded data is generated by performing embedding by the line rotation method and further by embedding by the bit plane swap method as described above. 3 shows a configuration example of the embedded unit 22 of No. 3.

In the embodiment shown in FIG. 34, the embedding section 22 is used.
Is composed of a horizontal line rotation embedding unit 41, a vertical line rotation embedding unit 42, and a bit plane swap embedding unit 43.

Horizontal line rotation embedding section 41
Reads the embedded data, which can be embedded by rotating a horizontal line of one frame, as the first embedded data from the embedded database 2. Further, the horizontal line rotation embedding unit 41 sequentially sets the horizontal line of the image data (original image data) of one frame as the embedding target data stored in the frame memory 21 as the attention line, and sets the attention line as the first line. The first embedded data is embedded in the line of interest by rotating (horizontally) corresponding to the embedded data. Then, the horizontal line rotation embedding unit 41
Of the image data as the embedding target data,
For example, using all the horizontal lines except the first horizontal line as the line of interest, embedding the first embedded data,
As a result, when the first embedded coded data is obtained, the first embedded coded data is output to the vertical line rotation embedding unit 42.

Vertical line rotation embedding section 42
Is the second embedded data, which is the embedded data that can be embedded by rotating a vertical line of one frame from the embedded database 2 (excluding the first embedded data). Read as. Further, the vertical line rotation embedding unit 42 sequentially sets the vertical line of the image data of one frame as the first embedded coded data supplied from the horizontal line rotation embedding unit 41 as the attention line, and the attention line, By rotating (in the vertical direction) corresponding to the second embedded data, the second embedded data is embedded in the line of interest. Then, the vertical line rotation embedding unit 42 embeds the second embedded data by using, for example, all vertical lines other than the first vertical line in the image data as the first embedded coded data as the target line. Thus, when the second embedded coded data is obtained, the second embedded coded data is
The data is output to the bit plane swap embedding unit 43.

Bit plane swap embedding unit 43
Is the embedded data from the embedded database 2 that can be embedded by bit-plane swapping a vertical line of one frame (excluding the first and second embedded data). Three
Is read as embedded data. Further, the bit plane swap embedding unit 43 sequentially sets the vertical line of the image data of one frame, which is the second embedded coded data supplied from the vertical line rotation embedding unit 42, as the attention line, Bit plane swapping (in the level direction) corresponding to the second embedded data embeds the third embedded data in the line of interest. Then, the bit plane swap embedding unit 43, for example, among the image data as the second embedded coded data,
When all the vertical lines except the first vertical line are the target lines, the third embedded data is embedded, and when the third embedded coded data is obtained, the third embedded coded data is stored in the frame memory. Write over 21.

Next, the embedding processing by the embedding section 22 in FIG. 34 will be described with reference to the flowchart in FIG.

First, in step S161,
The horizontal line rotation embedding unit 41 reads out one frame of image data as embedding target data from the frame memory 21. Further, in step S161, the horizontal line rotation embedding unit 41, the vertical line rotation embedding unit 42, and the bit plane swap embedding unit 43 are processed by
Through to the third embedded data, respectively.

Then, proceeding to step S162, the horizontal line rotation embedding unit 41 sequentially sets the horizontal line of the image data of one frame as the embedding target data read from the frame memory 21 as the attention line, and the attention line thereof. By rotating corresponding to the first embedded data, the first embedded data is embedded in the line of interest. And
The horizontal line rotation embedding unit 41 sets all the horizontal lines other than the first horizontal line in the image data as the embedding target data as the first line of interest.
When the first embedded coded data is generated by embedding the data to be embedded, the first embedded coded data is transferred to the vertical line rotation embedding unit 4.
2, and the process proceeds to step S163.

In step S163, the vertical line rotation embedding unit 42 sets the vertical line of the image data of one frame as the first embedded coded data supplied from the horizontal line rotation embedding unit 41,
The second embedding data is embedded in the target line by sequentially rotating the target line as the target line corresponding to the second embedding data. Then, the vertical line rotation embedding unit 42 embeds the second embedded data by using all the vertical lines other than the first vertical line in the image data as the first embedded coded data as the target line. , The second embedded coded data is generated, the second embedded coded data is output to the bit plane swap embedding unit 43, and step S1 is performed.
Proceed to 64.

In step S164, the bit plane swap embedding unit 43 sequentially sets the vertical lines of the one frame of image data as the second embedded coded data supplied from the vertical line rotation embedding unit 42 as the attention lines. The line of interest is bit-plane swapped in correspondence with the second embedded data to embed the third embedded data in the line of interest.

That is, when the pixel value of each pixel forming the line of interest is, for example, as shown in FIG. 36A, the pixel value of each pixel of the line of interest becomes
By the bit plane swap corresponding to the third embedded data, the data is changed to that shown in FIG. 36B.

Here, in FIG. 36 (see FIG.
The same applies to 1), assuming that the pixel value is 8 bits,
Each bit of the pixel value of each pixel forming the line of interest is represented by a square with the left side as the upper bit and the right side as the lower bit. Also, in FIG. 36, among the squares representing each bit of the pixel value, those with a shadow have a value of 0, and those without a shadow have a value of 1, Represent each.

Returning to FIG. 35, in step S164, the bit plane swap embedding unit 43 embeds the third embedded data with all the vertical lines of the image data as the second embedded coded data as the target lines. Thus, when the third embedded coded data is generated, the third embedded coded data is written in the frame memory 21 in a form of overwriting.

As described above, the third embedded coded data stored in the frame memory 21 is read from the frame memory 21, and the image data of the next frame stored in the frame memory 21 will be described below. The process of FIG. 35 is repeated as the data to be embedded.

As described above, the first and second embedded data are embedded by the line rotation method,
Furthermore, when embedding the third embedded data by the bit plane swap method, it is possible to embed more embedded data as compared with the case where only the line rotation method or only the bit plane swap method is adopted. .

Next, FIG. 37 shows a configuration example of the bit plane swap embedding unit 43 of FIG.

The input data holding unit 51 temporarily stores the vertical line rotation embedding unit 42 (the second embedded coded data supplied from FIG. 349).

The line-of-interest selection unit 52 selects from the vertical lines which form one frame of image data as the second embedded coded data stored in the input data holding unit 51,
Select the line of interest and swap information acquisition unit 5
4 and the bit plane swap unit 55.

The embedded data holding section 53 temporarily stores the third embedded data read from the embedded database 2 (FIG. 34).

The swap information acquisition section 54 reads from the embedded data holding section 53 the third embedded data by the data amount which can be embedded by performing the bit plane swap of the target line, and the third embedded data is read. Swap information representing the bit swapping method (swap pattern) of the line of interest is generated corresponding to the data. That is, for example, when the pixel value is 8 bits, there are 40320 swap patterns as described above, but which swap pattern is to be adopted for bit plane swapping the target line is embedded. Uniquely corresponding to the value of the third embedded data,
It is predetermined. Then, the swap information acquisition unit 54 generates swap information representing a swap pattern uniquely associated with the third embedded data read from the embedded data holding unit 53, and supplies it to the bit plane swap unit 55. .

Here, in the present embodiment, the line of interest is a vertical line of one frame, and therefore is composed of a relatively large number of pixels. Therefore, it is extremely rare in probability that a bit plane composed of a certain bit of pixel values of such a large number of pixels is the same as a bit plane composed of other bits.

However, even if it is extremely rare, there is a possibility that the bit plane of one bit of the line of interest coincides with the bit plane of another bit.

Now, the bit plane of the i-th bit (the i-th bit from the least significant bit) of the line of interest and the j-th (≠
i) If the bit plane of the bit has the same configuration as the bit plane, the bit plane of the line of interest is not changed even if the bit plane swap is performed to replace the bit plane of the i th bit and the bit plane of the j th bit. The arrangement does not change (substantially does not change) before and after the bit plane swap.

Therefore, in this case, the bit plane swap according to the swap pattern for exchanging the bit plane of the i-th bit and the bit plane for the j-th bit and the bit plane swap with the swap pattern without exchanging are determined from the bit plane swap result. Indistinguishable.

Therefore, the swap information acquisition section 54 selects one of the two or more bit plane swap swap patterns that cannot be distinguished.
That is, for example, only the swap pattern in which the same bit planes are not exchanged is validated, and the other swap patterns are invalidated.

As a result, the swap information acquisition section 54
In the above example, the swap information representing the swap pattern for exchanging the bit planes of the i-th bit and the j-th bit is not generated.

In this case, since the total number of swap patterns is smaller than that in the case where there is no invalid swap pattern, the data amount (bits) of the third embedded data that can be embedded in the attention line (bit number)
Will also decrease.

Further, the swap information acquisition unit 54 refers to the bit plane of each bit of the attention line supplied from the attention line selection unit 52 to determine whether or not there is a swap pattern which cannot be distinguished for the attention line. Determined by

The bit plane swap unit 55 performs the bit plane swap on the bit plane of the line of interest supplied from the line of interest selection unit 52 according to the swap pattern represented by the swap information supplied from the swap information acquisition unit 54. The third embedded data corresponding to the swap information is embedded. Then, the bit-plane swap unit 55 embeds the third embedded data in all the vertical lines forming the image data as the second embedded coded data,
That is, when the third embedded coded data is generated, the third embedded coded data is stored in the frame memory 21.
(Fig. 34) Output and write.

Next, the processing of the bit plane swap embedding unit 43 of FIG. 37 will be described with reference to the flowchart of FIG. Note that the processing of FIG. 38 is processing performed by the bit-plane swap embedding unit 43 in step S164 of FIG.

In the bit plane swap embedding unit 43, first, in step S171, the input data holding unit 51 temporarily stores the second embedded coded data supplied from the vertical line rotation embedding unit 42 (FIG. 34). To do. Furthermore, in step S171,
The embedded data holding unit 53 stores the third embedded data read from the embedded database 2 (FIG. 34).

Then, in step S172, the line-of-interest selection unit 52 selects a line-of-interest from the vertical line forming one frame of image data as the second embedded coded data stored in the input data holding unit 51. Select what you want to do. That is, the line-of-interest selection unit 52 sequentially selects the right vertical line from the first vertical line of the image data of one frame as the second embedded coded data as the line of interest. In step S172, the line-of-interest selection unit 52
Among the vertical lines forming one frame of image data as the second embedded coded data, a vertical line in the more leftward direction that is not the target line yet (to the right of the vertical line that has been the target line up to now) Vertical line) is selected as the line of interest. This attention line is changed from the attention line selection unit 52 to the swap information acquisition unit 5
4 and the bit plane swap unit 55.

After that, proceeding to step S173, the swap information acquisition section 54 reads from the embedded data holding section 53 the third embedded data by the data amount that can be embedded by bit plane swapping the line of interest. , Swap information representing a swap pattern corresponding to the third embedded data is generated.

The swap information generated by the swap information acquisition section 54 is supplied to the bit plane swap section 55,
When the bit plane swap unit 55 receives the swap information, the process proceeds from step S173 to S174, and the bit plane of the line of interest supplied from the line of interest selection unit 52 is represented by the swap information supplied from the swap information acquisition unit 54. By performing bit plane swap according to the pattern, the third embedded data corresponding to the swap information is embedded.

Then, in step S175, the attention line selection unit 52 sets all vertical lines of the image data of one frame as the second embedded coded data stored in the input data holding unit 51 as attention lines. If it is determined that all the vertical lines of the image data of one frame as the second embedded coded data have not been set as the attention line, it is determined in step S172.
Return to. In this case, in step S172, as described above, in the line-of-interest selection unit 52, among the vertical lines of the image data of one frame as the second embedded coded data, the line which is not yet set as the line of interest is
The line of interest is newly selected, and the same process is repeated thereafter.

If it is determined in step S175 that all the vertical lines of the image data of one frame as the second embedded coded data are the target lines, the process proceeds to step S176, and the bit plane swap unit 55 , The data in which the third embedded data is embedded in all the vertical lines forming one frame of image data as the second embedded encoded data, that is, the third embedded encoded data,
1 (FIG. 34) to overwrite and overwrite the process.

Next, FIG. 39 shows an embedding section 22 of FIG.
In the case of restoring the third embedded coded data generated in (3) into the image data as the embedding target data and the first to third embedded data embedded therein, the restoration unit 32 of FIG. A configuration example is shown.

In the embodiment of FIG. 39, the restoration unit 32
The bit plane swap return unit 61, the vertical line rotation return unit 62, and the horizontal line rotation return unit 63 are included.

The bit plane swap return unit 61 reads the third embedded coded data stored in the frame memory 31, and the third embedded coded data is
As described with reference to FIG. 32, the second embedded coded data and the third embedded data are restored.

That is, the bit plane swap return unit 61
Performs the vertical line of the image data of one frame as the third embedded coded data as a target line in sequence, operates the target line according to the bit plane swap method, and further operates according to the line rotation method.

[0409] Specifically, the bit-plane swap return unit 61 swaps the line of interest with the bit-plane to obtain the complete array of the bit-planes. Further, the bit plane swap return unit 61 obtains the line of interest rotated by the total rotation amount by rotating the line of interest formed of the bit planes in each row. Then, the bit-plane swap return unit 61 obtains a correlation value indicating the correlation with the reference line with respect to each rotation amount for the line of interest constituted by each array of bit planes, and changes the reciprocal of the correlation value with respect to the rotation amount. Find the correlation value function to represent.

Thereafter, the bit plane swap return unit 6
1 is a correlation value function obtained for a line of interest composed of bit lines of each row, and obtains one having the above-mentioned steep recesses, and arranges the bit planes corresponding to the correlation value function. A swap pattern for swapping the bit planes of the line of interest is detected. Then, the bit plane swap return unit 6
1 is a bit line swap of the bit plane of the line of interest according to the detected swap pattern, so that the line of interest as the third embedded coded data is the vertical line of the image data as the second embedded coded data. To the third line embedded in the line of interest based on the detected swap pattern.
Restore the embedded data of.

The bit plane swap return section 61 performs the above-mentioned processing by using all the vertical lines of the image data of one frame as the third embedded coded data as the attention lines, and thereby the second embedded coded data is obtained. Image data of one frame and the third embedded data embedded therein are restored. Then, the bit plane swap return unit 61 supplies the restored second embedded coded data to the vertical line rotation return unit 62.

The vertical line rotation returning unit 62 and the horizontal line rotation returning unit 63 restore the second embedded coded data supplied from the bit plane swap returning unit 61 by the line rotation method as described in FIG. 31B. To restore the embedding target data and the first and second embedded data.

That is, the vertical line rotation return unit 6
Reference numeral 2 denotes a vertical line of the image data as the second embedded coded data supplied from the bit plane swap returning unit 61, which is sequentially used as a line of interest, and is subjected to restoration processing by the line rotation method as described in FIG. 31B. As a result, the first embedded coded data and the second embedded data embedded therein are restored. Then, the vertical line rotation return unit 62
The restored first embedded coded data is supplied to the horizontal line rotation returning unit 63.

The horizontal line rotation return section 63 is
The horizontal line of the image data as the first embedded coded data supplied from the vertical line rotation returning unit 62 is sequentially treated as a line of interest, and restoration processing by the line rotation method as described in FIG. 31B is performed. , The image data as the embedding target data and the first embedded data embedded therein are restored. Then, the horizontal line rotation returning unit 63 writes the restored embedding target data in the frame memory 31 in a form of overwriting.

Next, the restoration processing by the restoration unit 32 in FIG. 39 will be described with reference to the flowchart in FIG.

First, in step S181, the bit plane swap return unit 61 reads out one frame of image data as the third embedded coded data from the frame memory 31, and proceeds to step S182.

In step S182, the bit plane swap return unit 61 restores the third embedded coded data read from the frame memory 31 into the second embedded coded data and the third embedded data.

That is, as shown in FIG. 41A, the bit-plane swap return unit 61 swaps the target line with the bit-plane, and further rotates the target line composed of the bit planes in each row, A correlation value function (in this case, a correlation value function having the inverse of the correlation value as its function value) representing a correlation with a reference line which is a vertical line adjacent to the left side is obtained. And
The bit-plane swap return unit 61 obtains a correlation value function having a steep concave portion as shown in FIG. 41B from the correlation value functions obtained for the line of interest composed of each array of bit planes, and corresponds to the correlation value function. A swap pattern for swapping the bit planes of the line of interest and the bit plane of the line of interest is detected. After that, the bit plane swap return unit 61 performs bit plane swapping on the bit plane of the line of interest in accordance with the detected swap pattern, thereby setting the line of interest as the third embedded coded data to the second embedded coded data. Is restored to the vertical line of the image data, and further, the third embedded data embedded in the line of interest is restored based on the detected swap pattern.

The bit plane swap return section 61 performs the above-mentioned processing by using all the vertical lines of the image data of one frame as the third embedded coded data as the attention line, and thereby the second embedded coded data is obtained. When the image data of one frame and the third embedded data embedded therein are restored, the restored second embedded coded data is supplied to the vertical line rotation returning unit 62, and the process proceeds to step S183. move on.

In step S183, the vertical line rotation return unit 62 sequentially sets the vertical lines of the image data as the second embedded coded data supplied from the bit plane swap return unit 61 as the target line in FIG. 31B. The restoration processing by the line rotation method as described above is performed, and thereby the first embedded coded data and the second embedded data embedded therein are restored. Then, the vertical line rotation returning unit 62 supplies the restored first embedded coded data to the horizontal line rotation returning unit 63,
It proceeds to step S184.

In step S184, the horizontal line rotation returning unit 63 sequentially sets the horizontal lines of the image data as the first embedded coded data supplied from the vertical line rotation returning unit 62 as the attention lines in FIG. 31B. The restoration processing by the line rotation method as described above is performed, and thereby, the image data as the embedding target data and the first embedded data embedded therein are restored.

Then, proceeding to step S185, the horizontal line rotation returning section 63 outputs the restored embedding target data to the frame memory 31 and writes it in a form of overwriting. Furthermore, in step S185, the bit plane swap return unit 61, the vertical line rotation return unit 62, and the horizontal line rotation return unit 63.
Outputs the restored first to third embedded data, respectively, and ends the process.

As described above, the embedding target data written in the frame memory 31 is the frame memory 3
The process of FIG. 40 is repeated for the third embedded coded data of the next frame, which is read out from the first frame 1 and is stored in the frame memory 31.

As described above, in the bit plane swap return unit 61, the third embedded coded data is operated according to the bit plane swap method and the line rotation method, and the correlation value of the data after the operation is performed. Since the restoration process is performed by using the function, the third embedded coded data can be restored to the second embedded coded data and the third embedded data embedded therein. Further, in the vertical line rotation returning unit 62, the second embedded coded data is restored to the first embedded coded data and the second embedded data embedded therein, and the horizontal line rotation unit 63. , The first embedded coded data is embedded in the embedding target data and It can be restored to the first of the embedded data are.

That is, for the embedding target data, the first
It is possible to restore the third embedded coded data in which a large amount of data, that is, to the third embedded data, are embedded into the original embedding target data and the first to third embedded data.

Next, FIG. 42 shows a detailed configuration example of the bit plane swap return unit 32 of FIG.

The input data holding unit 71 temporarily stores the image data of one frame as the third embedded coded data supplied from the frame memory 31 (FIG. 39).

The reference line selecting section 72 selects a predetermined vertical line from the vertical lines which are stored in the embedded coded data holding section 82 and which constitute the image data which has already been restored and serves as the second embedded coded data. Is selected and read as a reference line. That is, the reference line selection unit 72
A vertical line adjacent to the left of the vertical line selected as the target line in the target line selection unit 73, which will be described later, and already restored to the second embedded coded data is selected as the reference line. Then, the reference line selection unit 72 supplies the reference line to the vertical line correlation value calculation unit 76.

The line-of-interest selection unit 73 sequentially selects and reads out vertical lines constituting the image data as the third embedded coded data stored in the input data holding unit 71, as the line-of-interest, and performs bit-plane swapping. Supply to the section 74. Here, the line-of-interest selection unit 73 is configured to sequentially select the right vertical line as the line of interest from the first vertical line of the frame of the image data as the third embedded coded data. There is.

The bit plane swap unit 74 bit-plane swaps the bit plane of the target line according to the swap pattern corresponding to the optimum swap information supplied from the optimum swap information selection unit 80, and the target line after the bit plane swap. As the restoration result of the vertical line forming the second embedded coded data,
It is supplied to the embedded coded data holding unit 82. In addition, the bit plane swap unit 74 bit-plane swaps the bit plane of the attention line according to the swap pattern corresponding to the provisional swap information supplied from the provisional swap information determination unit 81, and selects the attention line after the bit plane swap. , And is supplied to the line rotation unit 75 as the above-mentioned temporary restoration vertical line.

The line rotation unit 75 rotates the temporary restored vertical lines supplied from the bit plane swap unit 74 by all the rotation amounts, and rotates the temporary restored vertical lines by each rotation amount (hereinafter, appropriately referred to as rotation line). Is supplied to the vertical line correlation value calculation unit 76.

Here, the vertical line forming the frame of the image data as the third embedded coded data is N
When it is composed of pixels, the line rotation unit 7
5 obtains the rotation line rotated by each rotation amount of 0 to N-1 pixels.

The vertical line correlation value calculation unit 76 calculates the correlation value between the rotation line of each rotation amount supplied from the line rotation unit 75 and the reference line supplied from the reference line selection unit 72, and the correlation value It is supplied to the holding unit 77. That is, the vertical line correlation value calculation unit 76 calculates, for example, the absolute difference value between the pixel value of a pixel forming a rotation line having a certain rotation amount and the pixel value of a pixel forming a reference line corresponding to the rotation line. Then, the reciprocal of the sum (sum of absolute differences) is calculated. Then, the inter-vertical line correlation value calculation unit 76 obtains the reciprocal of the sum of the absolute differences described above for the rotation lines of each rotation amount, and the correlation value of the rotation line of each rotation amount (correlation value with the reference line). Is supplied to the correlation value holding unit 77.

The function value holding unit 77 temporarily stores the correlation value of the rotation line of each rotation amount supplied from the vertical line correlation value calculation unit 76.

The correlation value function shape feature amount calculating unit 78 refers to the correlation value stored in the function value holding unit 77, and represents the correlation with the reference line when the temporary restored vertical line is rotated by each rotation amount. The feature amount of the correlation value function (correlation value function for the temporary restored vertical line) is obtained and supplied to the correlation value function feature amount holding unit 79.

Here, the correlation value function for the temporary restoration vertical line represents the correlation between the temporary restoration vertical line rotated by each rotation amount and the reference line, as described above. Function value holding unit 77
The correlation value stored in is the reciprocal of the function value at each rotation amount of the correlation value function for the temporary restored vertical line.

The correlation value function shape feature amount calculation unit 78 refers to the correlation value stored in the function value holding unit 77 as described above to determine, for example, the minimum function value of the correlation value function for the temporary restored vertical line. The value is obtained as the characteristic amount of the correlation value function.

Although the minimum value of the function value of the correlation value function is obtained here, the feature value of the correlation value function is not limited to this. That is, the feature quantity of the correlation value function may be any one that can determine whether or not the correlation value function has a shape having a steep concave portion, as shown in FIG. As the feature amount of the correlation value function, it is possible to adopt, for example, a rate of change (differential value of the function) or the like, in addition to the minimum value of the function value.

The correlation value function shape feature amount holding unit 79 uses the correlation value function feature amount for the temporary restored vertical line supplied from the correlation value function shape feature amount calculation unit 78 (here, as described above, the function value (Minimum value of) is temporarily stored.

The correlation value function shape feature amount holding unit 79 is supplied from the correlation value function shape feature amount calculation unit 78 with the correlation value function feature amount for the temporary restored vertical line.
The temporary swap information determining unit 81 is supplied with temporary swap information representing a swap pattern when the temporary restored vertical line is generated. Then, the correlation value function shape feature amount storage unit 79 associates the correlation value function for the temporary restored vertical line with the temporary swap information representing the swap pattern when the temporary restored vertical line is generated, and the correlation value function A set of temporary swap information is stored.

The optimum swap information selecting section 80 refers to the feature value of the correlation value function stored in the correlation value function shape feature value holding section 79 and associated with the temporary swap information representing each swap pattern. Similarly, from the temporary swap information stored in the correlation value function shape feature amount storage unit 79, the temporary swap information representing the swap pattern of the temporary restored vertical line for which the correlation value function of the shape having the steep concave portion is obtained is selected. Then, the selected tentative swap information is used as swap information indicating a swap pattern for optimal bit plane swap for the attention line, that is, swap information indicating a swap pattern for restoring the original arrangement of the bit planes on the attention line. The bit plane swap unit 74 and the target Supplies in order inclusive data holding unit 83.

That is, in the present embodiment, the minimum value of the function value is adopted as the feature amount of the correlation value function. In this case, the target line is bit plane swapped according to each swap pattern, and the temporary restored vertical line is used. It can be said that, of the correlation value functions of, the one having the smallest minimum value of the function values has a shape having a sharp concave portion.

Therefore, the optimum swap information selection unit 80
Of the feature values of the correlation value function stored in the correlation value function shape feature amount storage unit 79, the smallest feature amount is detected, and the temporary swap information associated with the smallest feature amount is selected. Then, the optimum swap information selection unit 80 reads the selected temporary swap information from the correlation value function shape feature amount holding unit 79 and supplies it to the bit plane swap unit 74 and the embedded data holding unit 83 as optimum swap information. .

The optimum swap information selection section 80 detects the smallest feature value of the correlation value function as described above.
It is equivalent to detecting the maximum correlation value.

The temporary swap information determining section 81 determines a swap pattern for bit plane swapping the target line and supplies the swap information representing the swap pattern to the bit plane swap section 74 as temporary swap information.

That is, the bit plane swap unit 74
Swap patterns for performing bit plane swaps for sequentially rearranging the bit planes of the line of interest are sequentially set, and swap information representing the swap patterns is supplied to the bit plane swap unit 74 as temporary swap information. .

The embedded coded data holding unit 82 temporarily stores the line of interest subjected to the bit plane swap in accordance with the swap pattern supplied from the bit plane swap unit 74 and represented by the optimum swap information.

The embedded data holding unit 83 restores and temporarily stores the third embedded data embedded in the line of interest based on the optimum swap information supplied from the optimum swap information selecting unit 80.

Next, with reference to the flowchart in FIG. 43, the processing of the bit plane swap return unit 61 in FIG. 42 will be described. It should be noted that in the processing of FIG. 43, the bit plane swap return unit 61 performs the processing in step S1 of FIG.
This is the process performed at 82.

First, the input data holding section 71 temporarily stores the image data of one frame as the third embedded coded data input from the frame memory 31,
It proceeds to step S192.

In step S192, the line-of-interest selecting unit 73 has not yet set the line of interest among the vertical lines of the image data of one frame as the third embedded coded data stored in the input data holding unit 71. The vertical line on the left side is selected as the line of interest and supplied to the bit plane swap unit 74. Furthermore, in step S192, the reference line selection unit 72 causes the embedded coded data storage unit 82 to store the line of interest among the vertical lines of the image data already restored to the second embedded coded data. The vertical line on the left side of is selected as the reference line and supplied to the inter-vertical line correlation value calculation unit 76.

Here, as described above, the first vertical line is not subject to processing in the embedding processing, so the third embedded coded data is not embedded. Therefore, the line-of-interest selection unit 73 has step S1
In 92, when the first vertical line is selected as the target line, the first vertical line is directly supplied to the embedded coded data holding unit 82 via the bit plane swap unit 74 and stored therein. Then, the line-of-interest selection unit 73 returns to step S192, and selects the vertical line next to the image data of one frame as the third embedded coded data, that is, the second vertical line, as the line of interest.

After that, proceeding to step S193, the temporary swap information determining section 81 generates temporary swap information and supplies it to the bit plane swap section 74.

That is, for example, as described above, when the pixel value is composed of 8 bits, the line of interest is also composed of 8 bit planes, and the arrangement thereof is 8! However, in this case, in step S193, the temporary swap information determination unit 81 outputs the 8! One of the normal bit planes and the swap pattern for swapping the bit planes, one of which has not yet generated the temporary swap information is determined as the swap pattern to be used for the bit plane swap of the attention line, and the temporary swap information is represented. Swap information is generated and supplied to the bit plane swap unit 74 and the correlation value function shape feature amount holding unit 79.

Then, proceeding to step S194, the bit-plane swap unit 74 bit-plane swaps the line of interest according to the swap pattern represented by the temporary swap information supplied from the temporary determination unit 81, and outputs the bit-plane swap result. As a temporary restoration vertical line,
Supply to the line rotation unit 75, step S1
Proceed to 95.

In step S195, the line rotation unit 75 sets the rotation amount to a value obtained by adding 1 to the previous rotation amount, and rotates the temporary restored vertical line by the rotation amount.

Here, in step S195, when the first rotation is performed on the new temporary restored vertical line, the line rotation unit 75 initializes the rotation amount to 0.

When the line rotation unit 75 rotates the temporary restoration vertical line, the line rotation unit 75 changes the rotation line that is the rotated temporary restoration vertical line to
It supplies to the correlation value calculation part 76 between vertical lines, and step S1
Proceed to 96.

In step S196, the inter-vertical line correlation value calculation unit 76 calculates the correlation value between the rotation line from the line rotation unit 75 and the reference line from the reference line selection unit 72, and the correlation value holding unit It is supplied to 77 for storage.

[0460] After that, proceeding to step S197, the line rotation unit 75 determines whether or not the temporary restored vertical line has been rotated by all the rotation amounts. When it is determined in step S197 that the temporary restored vertical line is not yet rotated by the entire rotation amount, the process returns to step S195, and the line rotation unit 75, as described above,
The rotation amount is set to a value obtained by adding 1 to the previous rotation amount, and the same processing is repeated thereafter.

If it is determined in step S197 that the temporary restored vertical line has been rotated by all the rotation amounts, that is, the temporary restored vertical line is rotated by each rotation amount and the reference line. When all the correlation values are obtained and stored in the correlation value holding unit 77, the process proceeds to step S198,
The correlation value function shape feature amount calculation unit 78 includes a function value holding unit 77.
As described above, the minimum value of the function value of the correlation value function for the provisional restored vertical line is obtained as a feature amount of the correlation value function by referring to the correlation value stored in. Then, the correlation value function shape feature amount calculation unit 78 supplies the correlation amount function shape feature amount holding unit 79 with the correlation value function shape amount regarding the temporary restored vertical line, and the temporary swap information determination unit 8
The temporary swap information output by 1 is stored in association with the temporary swap information, and the process proceeds to step S199.

[0462] In step S199, the temporary swap information determination unit 81 determines whether or not temporary swap information has been generated for all the arrangements of the bit planes forming the line of interest. When it is determined in step S199 that temporary swap information has not yet been generated for all the arrangements of the bit planes forming the line of interest, the process returns to step S193 and the temporary swap information determination unit 8
As described above, 1 determines one of the swap patterns for the row for which the temporary swap information has not yet been generated among all the row bit plane rows as the swap pattern to be used for the bit plane swap of the line of interest. Then, the same processing is repeated thereafter.

On the other hand, in step S199, when it is determined that the temporary swap information is generated for all the arrangements of the bit planes forming the attention line, that is, the correlation value function shape feature amount holding unit 79 Regarding, regarding the temporary swap information corresponding to the entire array of bit planes and the feature amount of the correlation value function corresponding to the temporary swap information, the process proceeds to step S200, and the optimum swap information selecting unit 80 determines the correlation value. By referring to the feature amount of the correlation value function stored in the function shape feature amount holding unit 79 and associated with the temporary swap information representing each swap pattern, the correlation value function shape feature amount holding unit 79 also stores the same. The swap pattern of the tentative restoration vertical line for which the correlation value function of the shape having a steep recess is obtained from the tentative swap information Select temporary swap information indicating the down, determined as the optimum swap information, and supplies the bit plane swapping section 74, and the embedded data holding unit 83.

That is, in step S200, the optimum swap information selection unit 80 has the correlation value function shape feature amount storage unit 79.
The smallest one of the feature quantities of the correlation value function stored in is detected, and the temporary swap information associated with the smallest feature quantity is selected. Then, the optimum swap information selection unit 80 reads the selected temporary swap information from the correlation value function shape feature amount holding unit 79 and supplies it to the bit plane swap unit 74 and the embedded data holding unit 83 as optimum swap information. .

Then, the flow proceeds to step S201, and the bit plane swap unit 74 and the embedded data holding unit 8
3 restores the vertical line of the second embedded coded data corresponding to the line of interest and the third embedded data embedded therein, based on the optimum swap information.

That is, in step S201, the bit plane swap unit 74 performs bit plane swapping on the bit plane of the line of interest according to the swap pattern corresponding to the optimal swap information supplied from the optimal swap information selecting unit 80. The vertical line forming the second embedded coded data is restored and supplied to the embedded coded data holding unit 82 for storage. Furthermore, in step S201, the embedded data holding unit 83
Based on the optimum swap information supplied from the optimum swap information selection unit 80, the third embedded data embedded in the line of interest is restored and temporarily stored.

After that, proceeding to step S202, the attention line selection unit 73 selects all vertical lines of the frame of the image data as the third embedded coded data stored in the input data holding unit 71 as attention lines. It is determined whether or not. If it is determined in step S202 that all the vertical lines of the frame of the image data as the third embedded coded data stored in the input data holding unit 71 have not yet been selected as the line of interest, step S192 Returning to FIG. 5, the line-of-interest selecting unit 73 selects the vertical line on the left side of the vertical line of the image data of one frame as the third embedded coded data stored in the input data holding unit 71, which is not the line of interest yet. The line is selected as the line of interest, and the same process as the above case is repeated.

On the other hand, when it is determined in step S202 that all the vertical lines of the frame of the image data as the third embedded coded data stored in the input data holding unit 71 are selected as the target line, that is, , The image data of one frame as the third embedded coded data stored in the input data holding unit 71 is the second
Of the embedded coded data and the third embedded data embedded therein, and the second embedded coded data and the third embedded data are embedded in the embedded coded data holding unit 82 and the embedded data. When the data is stored in the data holding unit 83, the process proceeds to step S203, where the embedded coded data holding unit 82 reads the second embedded coded data stored therein, and the vertical line rotation return unit 62 (FIG. 39). Supply to. further,
In step S203, the embedded data holding unit 83
However, the third embedded data stored therein is read out and output, and the process ends.

In this embodiment, in the embedding process, the bit plane swap is performed to allow all the arrangements of the bit planes of the target line.
It is possible to prohibit a bit plane swap in which bit planes of lower bits are only exchanged (exclude such a bit plane swap from the swap pattern).

That is, in the vertical line, only the bit planes of the lower bits are exchanged (for example, when the bit plane of the least significant bit is exchanged with the bit plane of the bit one bit higher). ), And when the bit planes are not replaced, a large difference may not appear in the correlation value function. Then, in this case, the detection of the correlation value function having the steep concave portion may be erroneous, and the embedded coded data may not be restored correctly (restored).

Such a restoration error can be prevented by prohibiting the replacement of only the lower bit planes.

It should be noted that the above-mentioned restoration error causes almost no change in the pixel value between the case where the bit planes of the lower bits of the vertical line are exchanged and the case where the bit planes are not exchanged. Due to not doing.

Before and after such a bit plane swap, a restoration error resulting from a small change in pixel value inhibits (does not perform) such a bit plane swap with a small change in pixel value. In addition, the binary code representing the pixel value is converted into a binary code in which the pixel value is assigned such that the intervals between the pixel values represented by the binary code having the same number of 0s (or 1s) are uniformly coarse. Then, the embedding process and the restoring process can also be performed to prevent this.

That is, for example, for the sake of simplicity, assume that a vertical line as a line of interest is composed of one pixel and that pixel value is represented by 8 bits.

In this case, if the pixel value of the pixel forming the line of interest is, for example, "2", its binary code is "00000010", and this binary code is obtained by bit plane swapping. The binary code as a bit plane is "000000
01 "," 00000010 "," 0000010 "
0 "," 00001000 "," 00010000 ",
"001000000", "01000000", "10
There are 8 types of "000000".

Then, the eight binary codes "0
"0000001", "00000010", "0000"
0100 "," 00001000 "," 0001000 "
0 "," 001000000 "," 01000000 ",
The pixel value represented by "10000000" is
"1", "2", "4", "8", "16", "3"
2 "," 64 ", and" 128 ", and the intervals of the pixel values corresponding to these eight patterns of binary code are dense or coarse. That is, in a portion having a small pixel value, the intervals between the pixel values are dense, and in a portion having a large pixel value, the intervals between the pixel values are coarse.

Therefore, for a pixel value with a close interval such as a pixel value "2" forming the line of interest, the pixel value will not change even if bit plane swap is performed to replace only the lower bits. It doesn't change much.

That is, for example, if a bit plane swap is performed in which the least significant bit (the bit plane) in the binary code “00000010” of the pixel value “2” and the one bit upper bit (the bit plane) thereof are exchanged, Although the binary code “00000001” is obtained, the pixel value represented by the binary code “00000001” after the bit plane swap is “1”, and only 1 is different from the pixel value “2” before the bit plane swap.

Therefore, the correlation values for the line of interest before and after the bit plane swap are not so different, and as a result, the possibility of erroneous restoration increases.

Therefore, a binary code (hereinafter referred to as a new code, as appropriate) in which the pixel values are assigned so that the intervals between the pixel values expressed by the same binary code in which the number of 0s (or 1s) is the same are coarse. ) Is adopted, and a binary code (hereinafter,
It is possible to obtain pixel values whose values are significantly different before and after the bit plane swap even if it is a bit plane swap in which only the lower bits are exchanged by converting the conventional code) into a new code. Becomes

That is, FIG. 44 and FIG. 45 show the pixel value represented by the conventional code for the 8-bit binary code,
The correspondence with the pixel value assigned to the new code is shown.

In FIGS. 44 and 45, the code column represents a new code and a conventional code as an 8-bit binary code, and the old column represents the binary code in the code column as a conventional code. It represents the pixel value represented by the conventional code. In addition, the new column shows the binary code in the code column as a new code,
It represents the pixel value assigned to the new code.

Regarding the new code, the pixel values of the new code having the same number of 0s or 1s are set so that the intervals between the pixel values corresponding to the new codes are uniform and are as coarse as possible. As a result, even in a bit plane swap in which only the lower bits are exchanged, it is possible to obtain pixel values whose values are significantly different before and after the bit plane swap.

That is, in FIG. 46, among the new 8-bit codes shown in FIGS. 44 and 46, all of the arrangements of those in which “1” is only 6 bits (thus, “0” is only 2 bits) 28 The pattern and the corresponding pixel value are shown.

For example, the sixth binary code “01111101” and the seventh binary code “01111110” in FIG. 46 have a relationship in which the least significant bit and the one bit higher bit are interchanged. , In the conventional code, the pixel values are "125" and "12", respectively.
6 ", the difference is only 1.

On the other hand, the binary code "0111"
In the new code, “1101” and “01111110” represent pixel values “49” and “62”, respectively, so the difference is 13.

Therefore, according to the new code, it is possible to obtain pixel values having greatly different values before and after the bit plane swap, and the embedding process and the restoration process are performed by using the new code, It is possible to prevent a restoration error due to the pixel value not changing much before and after the bit plane swap.

Next, the series of processes described above can be performed by hardware or software. When performing a series of processing by software, the program that constitutes the software,
It is installed on a general-purpose computer or the like.

Therefore, FIG. 47 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

The program is stored in the hard disk 105 or the ROM 1 as a recording medium built in the computer.
03 can be recorded in advance.

Alternatively, the program is a flexible disk, a CD-ROM (Compact Disc Read Only Memory),
MO (Magneto Optical) disc, DVD (Digital Versatile)
Disc), magnetic disk, semiconductor memory, or other removable recording medium 111 can be stored (recorded) temporarily or permanently. Such removable recording medium 111 can be provided as so-called package software.

The program is installed in the computer from the removable recording medium 111 as described above, and is also wirelessly transferred from the download site to the computer via an artificial satellite for digital satellite broadcasting or LAN (Local Area). Network), via a network such as the Internet, and transferred to a computer by wire. In the computer, the program thus transferred can be received by the communication unit 108 and installed in the built-in hard disk 105.

The computer is a CPU (Central Processing).
Unit) 102 is built in. CPU 102 has a bus 1
The input / output interface 110 is connected via 01, and the CPU 102 causes the user to operate the input unit 107 including a keyboard, a mouse, a microphone, etc. via the input / output interface 110. When a command is input, the ROM (Read O
nly Memory) 103 executes the program stored in it. Alternatively, the CPU 102 may execute a program stored in the hard disk 105, a program transferred from a satellite or a network, received by the communication unit 108 and installed in the hard disk 105, or a removable recording medium 111 mounted in the drive 109. The program read out and installed in the hard disk 105 is loaded into the RAM (Random Access Memor
y) Load in 104 and execute. As a result, the CPU 10
2 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 102 sends the processing result to the processing result as needed, for example, via the input / output interface 110.
The data is output from the output unit 106 configured by an LCD (Liquid CryStal Display), a speaker, or the like, or transmitted from the communication unit 108, and further recorded on the hard disk 105.

Here, in the present specification, the processing steps for writing a program for causing a computer to perform various processes do not necessarily have to be processed in time series in the order described in the flow chart, but in parallel. Alternatively, it also includes processes that are executed individually (for example, parallel processes or processes by objects).

Further, the program may be processed by one computer or may be processed in a distributed manner by a plurality of computers. Further, the program may be transferred to a remote computer and executed.

The embedded data is not particularly limited, and includes, for example, image data, audio data,
Text, computer programs, control signals, and other data can be used as embedded data.

Further, in the present embodiment, image data is adopted as the embedding target data, and the embedding data is
Although it is embedded in the image data, it is also possible to adopt, for example, audio data or the like as the embedding target data. That is, for example, the embedded data can be embedded in the audio by dividing the time-series audio data into appropriate frames and replacing the audio data of each frame according to the embedded data.

Furthermore, in the present embodiment, the horizontal line of the embedding target data is rotated, and further the second embedded coded data obtained by rotating the vertical line is subjected to the third by the bit plane swap method. I embedded the embedded data of
In addition, the third embedded data can be embedded in the embedded coded data obtained by rotating only one of the horizontal line and the vertical line of the embedding target data.

Further, in the present embodiment, the embedding by the line rotation method and the embedding by the bit plane swap method are performed to embed more embedded data, but in the reverse order. It is also possible to embed.
Further, such a large amount of embedded data can be embedded by employing the embedding of any other two methods. Further, the embedding can be performed by using a plurality of methods of 3 or more. However, as described above, it is desirable that the embedding of a plurality of methods to be adopted does not have an overlapping operation as an operation for embedding.

[0500]

According to the first data processing device, data processing method, program, and recording medium of the present invention, the information in which other information is embedded is utilized as the original information by utilizing the energy bias of the information. The first operation in which the embedding target data is manipulated corresponding to the arbitrary embedded data according to the first operation rule for performing the operation so that the embedded data is embedded in the embedding target data. Embedded coded data according to the rule is generated. Then, according to the second operation rule different from the first operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, and the first operation rule is applied. The embedded coded data according to the second operation rule is generated by embedding other arbitrary embedded data in the embedded coded data. Therefore, it becomes possible to embed more embedded data.

According to the second data processing apparatus and data processing method, and the program and recording medium of the present invention, the embedded coded data according to the second operation rule is
In addition to being operated according to the second operation rule, the operation is performed according to the first operation rule, and by utilizing the energy bias of the data obtained by the operation, the embedded coded data and other Any embedded data is restored. Then, the embedded coded data according to the first operation rule is operated according to the first operation rule, and the embedding target data and any embedded data are restored by utilizing the energy bias of the data obtained by the operation. To be done. Therefore, it becomes possible to restore the embedded coded data in which more embedded data is embedded, to the original embedding target data and the embedded data.

[Brief description of drawings]

FIG. 1 is a diagram illustrating an outline of an embedding process and a restoring process by a line rotation method.

FIG. 2 is a block diagram showing a configuration example of an embodiment of an embedded coding / restoration system.

FIG. 3 is a block diagram showing a configuration example of an embedded encoder 3.

FIG. 4 is a block diagram showing a configuration example of a restorer 6.

FIG. 5 is a diagram showing image data as embedding target data.

FIG. 6 is a diagram for explaining embedding / restoration using correlation.

FIG. 7 is a diagram for explaining embedding / restoration using correlation.

FIG. 8 is a flowchart illustrating an embedding process.

FIG. 9 is a flowchart illustrating a restoration process.

FIG. 10 is a diagram illustrating embedding / restoration using continuity.

FIG. 11 is a flowchart illustrating an embedding process.

FIG. 12 is a diagram illustrating embedding processing and restoration processing.

FIG. 13 is a flowchart illustrating a restoration process.

FIG. 14 is a diagram for explaining embedding / restoration using similarity.

FIG. 15 is a diagram for explaining embedding / restoration using similarity.

FIG. 16 is a flowchart illustrating an embedding process.

FIG. 17 is a diagram illustrating embedding processing and restoration processing.

FIG. 18 is a flowchart illustrating a restoration process.

FIG. 19 is a flowchart illustrating an embedding process.

FIG. 20 is a flowchart illustrating a restoration process.

FIG. 21 is a diagram for explaining embedding processing by the bit plane swap method.

FIG. 22 is a flowchart illustrating an embedding process.

[Fig. 23] Fig. 23 is a diagram for describing a method of calculating a bit-plane correlation.

[Fig. 24] Fig. 24 is a diagram for describing restoration processing by the bit plane swap method.

FIG. 25 is a flowchart illustrating a restoration process.

FIG. 26 is a flowchart illustrating an embedding process.

FIG. 27 is a diagram for explaining rotation.

FIG. 28 is a diagram for explaining a result of embedding processing.

[Fig. 29] Fig. 29 is a diagram for describing restoration processing by the line rotation method.

FIG. 30 is a flowchart illustrating a restoration process.

FIG. 31 is a diagram for explaining embedding processing and rotation processing for rotating both horizontal and vertical lines.

[Fig. 32] Fig. 32 is a diagram for describing embedding processing and restoration processing that can be restored using a correlation value function.

FIG. 33 is a diagram showing a correlation value function of an actual image.

FIG. 34 is a block diagram showing a configuration example of an embedding unit 22.

FIG. 35 is a flowchart illustrating an embedding process performed by the embedding unit 22.

FIG. 36 is a diagram for explaining the processing of the bit plane swap embedding unit 43.

FIG. 37 is a block diagram showing a configuration example of a bit plane swap embedding unit 43.

FIG. 38 is a flowchart illustrating a process of a bit plane swap embedding unit 43.

[Fig. 39] Fig. 39 is a block diagram illustrating a configuration example of the restoration unit 32.

FIG. 40 is a flowchart illustrating the processing of the restoration unit 32.

FIG. 41 is a diagram for explaining the processing of the bit plane swap return unit 61.

[Fig. 42] Fig. 42 is a block diagram illustrating a configuration example of a bit plane swap returning unit 61.

[Fig. 43] Fig. 43 is a flowchart illustrating a process of a bit plane swap return unit 61.

FIG. 44 is a diagram showing a correspondence relationship between a new code and a pixel value.

FIG. 45 is a diagram showing a correspondence relationship between a new code and a pixel value.

FIG. 46 is a diagram showing a correspondence relationship between a new code and a pixel value.

FIG. 47 is a block diagram showing a configuration example of an embodiment of a computer to which the present invention has been applied.

[Explanation of symbols]

1 embedded database, 2 embedded database, 3 embedded encoder, 4 recording medium,
5 transmission medium, 6 decompressor, 11 coding device,
12 decoding device, 21 frame memory, 22
Embedded section, 31 frame memory, 32 restoration section,
41 horizontal line rotation embedded part, 42
Vertical line rotation embedding unit, 43-bit plane swap embedding unit, 51 input data holding unit, 52 target line selection unit, 53 embedded data holding unit, 54 swap information acquisition unit, 55-bit plane swap unit, 61-bit plane swap returning unit , 62 vertical line rotation return part,
63 horizontal line rotation return unit, 71 input data holding unit, 72 reference line selection unit, 73 line of interest selection unit, 74 bit plane swap unit,
75 line rotation unit, 76 vertical line correlation value calculation unit, 77 correlation value holding unit, 78 correlation value function shape feature amount calculation unit, 79 correlation value shape feature amount holding unit, 80 optimum swap information selection unit, 81 temporary swap information Deciding unit, 82 embedded coded data holding unit, 83 embedded data holding unit, 101 bus, 102 CPU, 103 ROM, 104 RAM,
105 hard disk, 106 output unit, 107
Input section, 108 Communication section, 109 drive, 1
10 I / O interface, 111 Removable recording medium

─────────────────────────────────────────────────── ─── Continued Front Page (51) Int.Cl. ⁷ Identification Code FI Theme Coat (Reference) H04N 7/081 F Term (Reference) 5B057 CB19 CE08 CG01 CH01 CH11 5C063 AB07 DA01 DA13 DB09 5C076 AA14 AA24 BA06 BA09 5D044 AB07 DE02 DE50 EF05 EF10 GK12 GK17 5J064 AA01 BA09 BA16 BB01 BC01 BC27 BD02 BD03

Claims (24)

[Claims] 1. The embedded data to be embedded in the embedding target data is embedded in the embedding target data by manipulating the embedding target data that is the target for embedding data. A data processing apparatus for outputting embedded coded data, the first operation performing an operation so that the information having the other information embedded therein is restored to the original information by utilizing the energy bias of the information. According to a rule, the embedding target data is operated corresponding to arbitrary embedded data, and embedded coded data according to a first operation rule in which the arbitrary embedding data is embedded in the embedding target data is generated. 1 embedding means and a second operation rule different from the first operation rule , Operating the embedded coded data according to the first operation rule corresponding to any other embedded data, and embedding the other arbitrary embedded data into the embedded coded data according to the first operation rule Second, the embedded coded data is generated according to the second operation rule.
A data processing device comprising: 2. The data processing apparatus according to claim 1, wherein the first and second operation rules are operation rules in which there is no overlapping operation. 3. The data processing apparatus according to claim 1, wherein the embedding target data is image data. 4. The first operation rule is an operation rule in which the embedded coded data according to the first operation rule is restored to the image data by utilizing the correlation of the image data. The data processing device according to claim 3. 5. One of the first and second operation rules is an operation rule for operating a position of a pixel forming image data in a space or a time direction, and the one of the first and second operation rules. 4. The data processing apparatus according to claim 3, wherein the other one is an operation rule for operating a pixel forming image data in a level direction. 6. The first embedding means rotates the pixel rows arranged in a predetermined direction in the image data in the predetermined direction by an amount corresponding to the arbitrary embedded data. 4. The data processing apparatus according to claim 3, wherein as the operation rule, embedded coded data according to the first operation rule in which the arbitrary embedded data is embedded in the image data is generated. 7. The second embedding means uses, as the second operation rule, an operation rule for performing an operation that does not rotate a pixel row arranged in a predetermined direction in the image data in the predetermined direction. 7. The data processing apparatus according to claim 6, wherein the embedded coded data according to the second operation rule is generated by embedding the other arbitrary embedded data in the embedded coded data according to the first operation rule. 8. The second embedding means defines a bit plane for each bit of a bit string representing pixel values of a plurality of pixels of the image data, which is embedded coded data according to the first operation rule, As the second operation rule, performing bit-plane swapping for replacement with other arbitrary embedded data is performed by embedding the other arbitrary embedded data in the embedded coded data according to the first operation rule. The data processing device according to claim 7, wherein the embedded coded data is generated according to the second operation rule. 9. The first embedding means rotates the image data in either one of a horizontal direction and a vertical direction in correspondence with the first embedding data to thereby generate the image data. Against the first
First embedded coded data in which the embedded data is embedded, and the first embedded coded data is generated in the horizontal or vertical direction corresponding to the second embedded data. By rotating the second embedded data into the first embedded encoded data by rotating the second embedded data into the second embedded data.
Embedded code data is generated, and the second embedding means performs bit plane swapping of the second embedded coded data corresponding to third embedded data to thereby generate the second embedded code. 9. The data processing device according to claim 8, wherein third embedded coded data in which the third embedded data is embedded is generated for the encoded data. 10. The embedded data to be embedded in the embedding target data is embedded in the embedding target data by manipulating the embedding target data that is a target for embedding data, and the embedded data is embedded in the embedding target data. A data processing method for outputting embedded coded data, the first operation performing an operation such that the information in which other information is embedded is restored to the original information by utilizing a bias of energy contained in the information. According to a rule, the embedding target data is operated corresponding to arbitrary embedded data, and embedded coded data according to a first operation rule in which the arbitrary embedding data is embedded in the embedding target data is generated. 1 embedding step and a second operation rule different from the first operation rule. Therefore, the embedded coded data according to the first operation rule is manipulated in correspondence with any other embedded data, and the embedded coded data according to the first operation rule is subjected to the other arbitrary embedded data. Second generation of embedded coded data according to the embedded second operation rule
And a step of embedding the data. 11. The embedded data to be embedded in the embedding target data is embedded in the embedding target data by manipulating the embedding target data that is the target for embedding data. A program for causing a computer to perform data processing for outputting embedded coded data, which is operated so that the information embedded with other information is restored to the original information by utilizing the energy bias of the information. Embedded code according to a first operation rule in which the embedding target data is manipulated corresponding to arbitrary embedded data according to a first operation rule for performing, and the arbitrary embedding data is embedded in the embedding target data. Embedding data to generate a first embedding step, and the first operation rule. According to a second operation rule different from that of the first operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, and the embedded coded data according to the first operation rule is operated. A second operation for generating embedded coded data according to a second operation rule in which the other arbitrary embedded data is embedded
And a step of embedding the program. 12. The embedded data to be embedded in the embedding target data is embedded in the embedding target data by manipulating the embedding target data that is a target for embedding data. A recording medium in which a program for causing a computer to perform data processing for outputting embedded coded data is recorded, wherein the information is the original information in which other information is embedded by utilizing the energy bias of the information. According to the first operation rule for performing the operation so as to be restored to, the embedding target data is operated corresponding to any embedded data, and the arbitrary embedding data is embedded in the embedding target data. The first embedding step for generating embedded coded data according to the first operation rule. And a second operation rule different from the first operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, Second, embedded embedded data according to the second operation rule in which the other arbitrary embedded data is embedded in embedded encoded data according to the second operation rule
And a program having a step of embedding the recording medium. 13. The embedding target is embedding coded data, which is generated by manipulating embedding target data that is a target for embedding data, and which embeds embedded data to be embedded in the embedding target data in the embedding target data. A first data processing device that restores data and embedded data, wherein the first information is restored so that the information in which other information is embedded is restored to the original information by utilizing the energy bias of the information. According to the operation rule, the embedding target data is operated corresponding to any embedded data, and embedded coded data according to the first operation rule in which the arbitrary embedding data is embedded in the embedding target data is generated. According to a second operation rule different from the first operation rule, the first operation rule is According to the second operation rule in which the embedded coded data according to (1) is operated corresponding to other arbitrary embedded data, and the embedded coded data according to the first operation rule is embedded with the other arbitrary embedded data. The embedded coded data according to the second operation rule obtained by generating the embedded coded data is operated according to the second operation rule and the first operation rule, and the operation is performed. Utilizing the bias of the energy of the data obtained by the above, a first restoring means for restoring the embedded coded data according to the first operation rule and the other arbitrary embedded data, and the first operation rule. Embedded coded data,
A second restoring unit that restores the embedding target data and the arbitrary embedded data by operating according to the first operation rule and utilizing the energy bias of the data obtained by the operation. Characteristic data processing device. 14. The data processing apparatus according to claim 13, wherein the first and second operation rules are operation rules in which there are no overlapping operations. 15. The first operation rule is an operation rule for restoring the embedded coded data according to the first operation rule to the embedding target data by utilizing the correlation of the embedding target data, The restoration means of No. 1 stores the embedded coded data according to the second operation rule,
First operating means for operating according to the second operating rule; second operating means for operating data obtained by the operation by the first operating means according to the first operating rule; On the basis of the correlation calculated by the correlation calculating means for obtaining the correlation of the data obtained by the operation by the second operating means,
An operation determining unit that determines an operation of the embedded coded data according to the second operation rule in the first operating unit and outputs operation information representing the operation, and the operation information output by the operation determining unit 14. The embedded coded data according to the second operation rule is restored to the embedded coded data according to the first operation rule and the other arbitrary embedded data based on the above. Data processing equipment. 16. The first restoring means further comprises a feature quantity computing means for obtaining a feature quantity of a function representing the correlation obtained by the correlation computing means, and the operation determining means is the feature computing means. Based on the feature quantity of the function representing the obtained correlation, the second
16. The data processing apparatus according to claim 15, wherein the operation of the embedded coded data is determined according to the operation rule. 17. The second restoring means operates the embedded coded data according to the first operation rule according to the first operation rule, and based on the correlation of data obtained by the operation, The data processing device according to claim 15, wherein the embedding target data and the arbitrary embedding data are restored. 18. The data processing apparatus according to claim 13, wherein the embedding target data is image data. 19. One of the first and second operation rules is to operate the position of a pixel forming image data in the space or time direction, and the one of the first and second operation rules. 2. The other one is for operating pixels constituting image data in the level direction.
8. The data processing device according to 8. 20. The embedded coded data according to the second operation rule is configured such that pixel rows arranged in a predetermined direction in the image data as the embedding target data correspond to the arbitrary embedded data in the predetermined direction. Rotation is performed as the first operation rule to generate embedded coded data according to the first operation rule in which the arbitrary embedded data is embedded in the image data, and to embed according to the first operation rule. Performing a bit plane swap for exchanging a bit plane for each bit of a bit string representing pixel values of a plurality of pixels of the image data, which is encoded data, corresponding to the other arbitrary embedded data, As the second operation rule,
It is obtained by generating embedded coded data according to the second operation rule in which the other arbitrary embedded data is embedded in the embedded coded data according to the first operation rule. The restoring means is a bit plane swap means for bit plane swapping the image data that is the embedded coded data according to the second operation rule, and image data as data obtained by the bit plane swap by the bit plane swap means. On the basis of the correlation obtained by the rotation calculating means for rotating the pixel rows arranged in the predetermined direction, the correlation calculating means for obtaining the correlation of the image data as the data obtained by the rotation by the rotation means, ,
The bit plane swap means includes an operation deciding means for deciding an operation of the image data, which is the embedded coded data according to the second operation rule, and outputting operation information representing the operation. Based on the operation information output by the means, the image data that is the embedded coded data according to the second operation rule, the embedded coded data according to the first operation rule, and the other arbitrary embedded data. The data processing device according to claim 18, which is restored. 21. The embedded coded data according to the second operation rule is configured to rotate the image data in one of a horizontal direction and a vertical direction corresponding to the first embedded data. Thereby, the first embedded coded data in which the first embedded data is embedded is generated in the image data, and the first embedded coded data is generated in correspondence with the second embedded data. , The second embedded coded data in which the second embedded data is embedded in the first embedded coded data by rotating in another direction of the horizontal or vertical direction, By bit-plane swapping the second embedded coded data corresponding to the third embedded data, the second embedded code The third embedded coded data obtained by generating the third embedded coded data in which the third embedded data is embedded in the encoded data, wherein the rotation means is the bit. Rotate the pixel rows arranged in the other direction in the image data as the data obtained by the bit plane swap by the plane swap means, the correlation operation means, of the image data as the data obtained by the rotation by the rotation means 21. The data processing apparatus according to claim 20, wherein a correlation between pixel columns adjacent to each other in the one direction is obtained. 22. The embedding target is embedded coded data generated by manipulating embedding target data that is a target for embedding data, in which embedded data to be embedded in the embedding target data is embedded in the embedding target data. A first data processing method for restoring data and data to be embedded, wherein the bias of energy contained in the information is used to restore the information in which other information is embedded to the original information. According to the operation rule, the embedding target data is operated corresponding to any embedded data, and embedded coded data according to the first operation rule in which the arbitrary embedding data is embedded in the embedding target data is generated. According to a second operation rule different from the first operation rule, the first operation rule is According to the second operation rule in which the embedded coded data according to (1) is operated corresponding to other arbitrary embedded data, and the embedded coded data according to the first operation rule is embedded with the other arbitrary embedded data. The embedded coded data according to the second operation rule obtained by generating the embedded coded data is operated according to the second operation rule and the first operation rule, and the operation is performed. A first restoration step of restoring the embedded coded data according to the first operation rule and the other arbitrary embedded data by utilizing the energy bias of the data obtained by Embedded coded data,
A second restoring step of restoring the embedding target data and the arbitrary embedded data by operating in accordance with the first operation rule and utilizing an energy bias of data obtained by the operation. Characterizing data processing method. 23. The embedding target is embedded coded data generated by manipulating embedding target data that is a target for embedding data, in which embedded data to be embedded in the embedding target data is embedded in the embedding target data. A program that causes a computer to perform data processing to restore data and embedded data, so that the information embedded with other information can be restored to the original information by utilizing the energy bias of the information. Embedding according to a first operation rule in which the embedding target data is manipulated corresponding to arbitrary embedded data according to a first operation rule for performing operation, and the arbitrary embedding data is embedded in the embedding target data. Generate encoded data, and generate a second operation rule different from the first operation rule. According to the above, the embedded coded data according to the first operation rule is operated corresponding to other arbitrary embedded data, and the embedded coded data according to the first operation rule is added to the other arbitrary embedded data. Embedded coded data according to the second operation rule obtained by generating embedded coded data according to the second operation rule in which A first restoration step of operating according to the operation rule and utilizing the energy bias of the data obtained by the operation to restore the embedded coded data according to the first operation rule and the other arbitrary embedded data. And embedded coded data according to the first operation rule,
A second restoring step of restoring the embedding target data and the arbitrary embedded data by operating in accordance with the first operation rule and utilizing an energy bias of data obtained by the operation. Characteristic program. 24. The embedded coded data, which is generated by manipulating embedding target data that is a target for embedding data, and which embeds embedded data to be embedded in the embedding target data into the embedding target data, A recording medium in which a program for causing a computer to perform data processing for restoring data and embedded data is recorded, and the information in which other information is embedded is utilized by utilizing the energy bias of information. According to the first operation rule for performing operation so as to be restored to information, the embedding target data is operated corresponding to any embedded data, and the arbitrary embedding data is embedded in the embedding target data. Generating embedded coded data according to a first operation rule, According to a different second operation rule, the embedded coded data according to the first operation rule is operated corresponding to any other embedded data, and the embedded coded data according to the first operation rule is added to the embedded coded data according to the first operation rule. The embedded coded data according to the second operation rule obtained by generating the embedded coded data according to the second operation rule in which other arbitrary embedded data is embedded is operated according to the second operation rule. At the same time, the embedded coded data according to the first operation rule and the other arbitrary embedded data are operated by utilizing the bias of the energy of the data obtained by the operation according to the first operation rule. A first restoration step of restoring and the embedded coded data according to the first operation rule are stored in the first operation rule. And a second restoration step of restoring the embedding target data and the arbitrary embedded data by utilizing the bias of the energy of the data obtained by the operation. A recording medium characterized by.