JP2004328708A - Data processing device and data processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processing device and a method thereof that can easily determine a camera that has generated image data with high reliability.
SOLUTION: There is an inherent variation in the light receiving sensitivity of each light receiving element, and a predetermined image generating device that generates image data based on the light receiving results of a plurality of light receiving elements converts the first image data to be determined into a first image data. It is determined whether or not it has been generated. The correlation detection unit 5 detects a correlation between the first image data and the second image data for reference generated using a predetermined image generation device. The CPU 6 determines whether or not the first image data has been generated using the image generation device based on the correlation.
[Selection diagram] Fig. 1

Description

  The present invention relates to a data processing apparatus and a method for determining a camera that has captured image data.

For example, digital image data captured by a digital camera may be illegally used by a third party without permission from the copyright holder of the image data.
In order to prevent such unauthorized use, there is a method of adding information specifying the camera that generated the image data to the image data as digital watermark information or a header of the image file.

However, in the above-described method of adding digital watermark information to image data, there is a problem that digital watermark embedding methods cannot be unified, and thus digital watermark information may not be detected.
In addition, the above-described method of adding a header of an image file has a problem that information for identifying a camera is easily falsified by rewriting the header.

  The present invention has been made in view of the above-described problems of the related art, and has as its object to provide a data processing apparatus and a method thereof that can easily determine a camera that has generated image data with high reliability.

  In order to solve the above-mentioned problems of the related art and achieve the above-described object, a data processing apparatus according to a first aspect of the present invention generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing device for determining whether or not the image data is generated by a predetermined image generating device, wherein the first image data and a reference image generated by using the predetermined image generating device. Correlation detection means for detecting a correlation with the second image data, and whether the first image data is generated using the image generation device based on the correlation detected by the correlation detection means Determining means for determining whether or not the determination is made.

The operation of the data processing device of the first invention is as follows.
Correlation detecting means detects a correlation between the first image data and the second image data for reference generated using a predetermined image generating device.
Then, the determination means determines whether or not the first image data has been generated using the image generation device, based on the correlation detected by the correlation detection means.

In the data processing device of the first invention, preferably, the correlation detecting means uses the second image data to display a pattern having no correlation with a pattern displayed based on the first image data. Detect correlation.
In the data processing apparatus according to the first invention, preferably, the correlation detection means performs an orthogonal transformation on the first image data and the second image data to respectively perform first frequency component data and second frequency component data. Converting means for generating data; and dividing each complex number data constituting the first frequency component data by an absolute value of each complex number data to generate first complex number data; Dividing means for dividing each complex number data forming the complex data by the absolute value of each complex number data to generate second complex number data; and forming respective ones of the first complex number data and the second complex number data Replacement means for generating third complex number data in which the complex number data is replaced with complex conjugate complex number data; and the first complex not replaced by the replacement means. Multiplying means for multiplying numerical data or the second complex number data with the third complex number data generated by the replacing means to generate fourth complex number data; And an inverse transform circuit for performing inverse orthogonal transform on the complex number data to generate the correlation data.

  A data processing method according to a second aspect of the present invention determines whether image data is generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing method, wherein a first step of detecting a correlation between the first image data and a second image data for reference generated by using the predetermined image generation device; A second step of determining whether or not the first image data has been generated using the image generation device based on the correlation detected in the step.

  A data processing device according to a third aspect of the present invention is a data processing device comprising: a plurality of image generation devices that generate image data based on a light reception result of a light receiving element having a specific variation in light reception sensitivity; A data processing device for determining whether one image data has been generated, and a holding unit for holding a plurality of second image data for reference respectively generated using the plurality of image generation devices; One of the plurality of image generation devices based on the correlation detected by the correlation detection unit, the correlation detection unit detecting a correlation between the first image data and the second image data held by the storage unit. Determining means for determining whether or not the first image data has been generated by using the first image data.

The operation of the data processing device of the third invention is as follows.
Correlation detecting means detects a correlation between the first image data and the second image data held by the holding means.
A determination unit determines whether the first image data has been generated using any of the plurality of image generation devices based on the correlation detected by the correlation detection unit.

  A data processing method according to a fourth aspect of the present invention is the data processing method according to any one of the plurality of image generating apparatuses for generating image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing method performed by a data processing device that determines whether one image data is generated and holds a plurality of second image data for reference each generated using the plurality of image generation devices, A first step of detecting a correlation between the first image data and the held second image data; and a step of detecting a correlation between the plurality of image generation apparatuses based on the correlation detected in the first step. A second step of determining whether or not the first image data has been generated.

  According to a fifth aspect of the present invention, there is provided a data processing apparatus, comprising: an image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity; A data processing device for determining a plurality of second image data for reference generated in advance using the stolen image generating device, the first image data, Correlation detection means for detecting a correlation with the second image data held by the holding means, and the first image using the stolen image generation device based on the correlation detected by the correlation detection means Determining means for determining whether or not the data is generated.

The operation of the data processing device according to the fifth invention is as follows.
Correlation detecting means detects a correlation between the first image data and the second image data held by the holding means.
A determination unit determines whether or not the first image data has been generated using the stolen image generation device based on the correlation detected by the correlation detection unit.

  A data processing method according to a sixth aspect of the present invention is the data processing method according to the first aspect, wherein the first image data to be determined is generated by an image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity A data processing method performed by a data processing device that holds a plurality of second image data for reference generated in advance using the stolen image generation device, wherein the first image data and A first step of detecting a correlation with the held second image data, and the first step of using the stolen image generating apparatus based on the correlation detected in the first step. A second step of determining whether the image data has been generated.

  The data processing device according to a seventh aspect of the present invention is configured to determine whether or not the first image data to be determined is generated by the image generation device that generates the image data based on the light receiving results of the light receiving elements having the inherent variation in the light receiving sensitivity. A licensed data source that specifies a broadcast source that is permitted to broadcast image data generated using a predetermined image generation device, and a data source generated using the predetermined image generation device. Holding means for holding the obtained second image data for reference, detecting a correlation between the first image data broadcasted by a predetermined broadcast source and the second image data held by the holding means Detecting means for determining whether or not the first image data is generated by using the image generating apparatus based on the correlation detected by the correlation detecting means. And the said When it is determined that the first image data is generated by using the image generating apparatus, the first image data is determined based on the licensed broadcast source data held by the holding unit. Second determining means for determining whether or not the permission has been granted to the broadcast source that has broadcasted the image data.

The operation of the data processing device of the seventh invention is as follows.
Correlation detecting means detects a correlation between the first image data broadcasted by a predetermined broadcast source and the second image data held by the holding means.
Then, the first determination unit determines whether the first image data has been generated using the image generation device based on the correlation detected by the correlation detection unit.
Then, when the second determination unit determines that the first image data is generated by using the image generation device in the first determination unit, the storage unit holds the first image data. Based on the licensed broadcast source data, it is determined whether the license has been granted to the broadcast source that broadcasted the first image data.

  The data processing method according to an eighth aspect of the present invention is the data processing method according to the first aspect, wherein the first image data to be determined is generated by an image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. Is determined, and licensed broadcast source data that specifies a broadcast source that is permitted to broadcast image data generated using the predetermined image generation device, and a reference broadcast source data generated using the predetermined image generation device. A data processing method performed by a data processing device holding second image data, wherein the first image data broadcasted by a predetermined broadcast source and the second image data held by the data processing device are Determining whether the first image data is generated by using the image generation apparatus based on the first step of detecting the correlation of the first step and the correlation detected in the first step. A second step to If it is determined in the second step that the first image data is generated using the image generation device, the first image data is generated based on the licensed broadcast source data held by the data processing device. A third step of determining whether or not the permission has been granted to the broadcast source that has broadcasted the first image data.

  The data processing device of the ninth invention is a data processing device for retrieving image data generated by a predetermined image generation device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A correlation detecting means for detecting a correlation between the plurality of first image data stored in the storage means and the second image data for reference generated by using the predetermined image generating apparatus; Searching means for searching, based on the correlation detected by the correlation detecting means, image data generated by using the image generating apparatus, from among the plurality of first image data stored in the storage means. Have.

The operation of the data processing device according to the ninth aspect is as follows.
Correlation detection means detects a correlation between the plurality of first image data stored in the storage means and reference second image data generated using a predetermined image generation device.
Then, a search unit, based on the correlation detected by the correlation detection unit, among the plurality of first image data stored in the storage unit, image data generated using the image generation device Search for.

  A data processing method according to a tenth aspect of the present invention is performed by a data processing apparatus that searches for image data generated by a predetermined image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing method, comprising: detecting a correlation between a plurality of first image data stored in a storage unit and reference second image data generated using the predetermined image generation device. Image data generated by using the image generation device among the plurality of first image data stored in the storage unit, based on the first step and the correlation detected in the first step And a second step of searching for

  A data processing device according to an eleventh aspect of the present invention determines whether or not image data is generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing device, wherein for each of a plurality of first block image data constituting first image data to be determined, the first block image data and a second block image data generated by the image generation device. Correlation detecting means for detecting a correlation between the second block image data corresponding to the first block image data among a plurality of second block image data constituting the image data; Determining means for determining whether or not the first image data has been generated based on the image data generated and generated using the image generating apparatus, based on the correlation. To.

  A data processing method according to a twelfth aspect of the present invention determines whether or not image data is generated by a predetermined image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing method, wherein, for each of a plurality of first block image data constituting first image data to be determined, the first block image data and a second block image data generated by the image generation device. A first step of detecting a correlation between the second block image data corresponding to the first block image data among a plurality of second block image data constituting the image data; and the first step A second step of determining whether or not the first image data has been generated based on the image data generated and generated by using the image generation apparatus based on the correlation detected in the step (a). That.

  A data processing apparatus according to a thirteenth aspect determines whether or not image data is generated by a predetermined image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing device, wherein the correlation detecting means detects a correlation between the first image data to be determined and the second image data for reference generated by the image generating apparatus; Determining means for determining whether or not the first image data is generated using the image generating apparatus, based on the correlation, wherein the correlation detecting means comprises: Conversion means for orthogonally transforming the image data and the second image data to generate first frequency component data and second frequency component data, respectively; and the first frequency component data generated by the conversion means Generating first complex data based on each complex data constituting the first frequency component data so as to be more strongly limited in amplitude than the second frequency component data; Calculating means for generating second complex number data based on each complex number data forming data, and each complex number forming one of the first complex number data and the second complex number data generated by the calculating means Replacement means for generating third complex number data in which data is replaced with complex conjugate complex number data; the first complex number data or the second complex number data not replaced by the replacement means; Multiplying means for multiplying the third complex number data generated by the above to generate fourth complex number data, and the fourth complex number generated by the multiplying means And an inverse conversion circuit for generating correlation data representing the correlation to inverse orthogonal transform the over data.

  According to a fourteenth aspect of the present invention, in the data processing apparatus, image data generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity is provided as providing image data. A data processing device for registration, wherein an interface for inputting a registration request including first image data to be determined and user identification data corresponds to the user identification data and the second image data for reference Storage means for attaching and storing, and reading the second image data from the storage means based on the user identification data included in the registration request input via the interface, A correlation detection unit that detects a correlation with the first image data included in the registration request, and a correlation detection unit that detects the correlation based on the correlation detected by the correlation detection unit. And a determination means for determining whether or not to register the first image data as image data for said providing.

  As described above, according to the present invention, it is possible to provide a data processing apparatus and method capable of easily determining an image generation apparatus that has generated image data with high reliability.

Hereinafter, a data processing device 1 according to an embodiment of the present invention will be described.
FIG. 1 is a configuration diagram of a data processing device 1 of the present embodiment.
The data processing device 1 includes a camera (digital camera), which is a predetermined image generating device that generates image data based on the light receiving results of a plurality of light receiving elements having a specific variation in light receiving sensitivity, converts the image data to be determined It is determined whether or not it has been generated.

As illustrated in FIG. 1, the data processing device 1 includes, for example, a memory 2, an interface 3, a reference image generation unit 4, a correlation detection unit 5, and a determination unit 6, which are connected via a bus 9. .
Here, the reference image generating unit 4 corresponds to the reference image generating unit of the present invention, the correlation detecting unit 5 corresponds to the correlation detecting unit of the present invention, and the determining unit 6 corresponds to the determining unit of the present invention.
The configuration of the data processing device 1 illustrated in FIG. 1 is an example, and at least one of the functions of the correlation detection unit 4 and the reference image generation unit 5 may be realized by the determination unit 6.

The memory 2 stores, as reference image data S7 (second image data of the present invention), image data generated based on a predetermined camera imaging result input via the interface 3.
Here, the camera corresponds to the image generation device of the present invention.
As the reference image data S7, for example, data that displays a pattern having no correlation with a pattern displayed based on the determination target data S31 is used. For example, reference image data S7 is generated by imaging gray paper, the sky, or the like as a picture with a camera.

The interface 3 inputs image data generated based on the imaging result of a predetermined camera, and outputs the image data to the memory 2 or the reference image generation unit 4.
Further, the interface 3 inputs the image data to be determined (first image data of the present invention) and outputs this to the memory 2 or the determination unit 6.
The above-mentioned predetermined camera has a plurality of light receiving elements arranged in a matrix, and light from a subject enters the light receiving elements via a lens.
Then, the camera generates image data based on the light receiving results of the plurality of light receiving elements.

Here, the light receiving element is, for example, a CCD (Charge Coupled Device) image pickup element, and the light receiving sensitivity has a manufacturing-specific variation. The variation is artificially difficult to reproduce.
FIG. 2 is a diagram schematically illustrating a cross section of the light receiving element 11 provided in the camera.
As shown in FIG. 2, the light receiving element 11 includes a silicon substrate 12, a light shielding layer 14 provided with an opening 13, and a condenser lens 15.
A photoelectric conversion region is formed in the silicon substrate 12 at a position facing the opening 13.
The condenser lens 15 is formed to reach the silicon substrate 12 via the light shielding layer 14 from a position opposite to the silicon substrate 12 with respect to the light shielding layer 14.

The opening 13 is formed by, for example, performing fine processing on a resist serving as an etching mask by exposure to ultraviolet light, and then removing the light-shielding layer 14 and the insulating layer 16 therebelow by etching.
The condenser lens 15 is formed into a spherical shape by subjecting the glass layer to fine processing by photolithography and then melting the glass layer at a high temperature.
In the manufacturing process of the light receiving element 11 described above, the opening 13 is controlled so as to form as uniform a shape as possible, but the shape varies due to various factors. For this reason, the shape of each opening 13 of the plurality of light receiving elements varies.
Here, the shape of the light (light flux) incident on the light receiving element varies due to the variation in the shape of the opening 13 of each light receiving element, and the amount of light incident on the photoelectric conversion region of the silicon substrate 12 varies. Therefore, the light receiving sensitivity varies among the light receiving elements.
Such a variation in light receiving sensitivity is not artificial, and is unique to each light receiving element like a fingerprint. Further, the influence of the variation in the light receiving sensitivity of the light receiving element occurs in image data generated by a camera using the light receiving element.
In the present embodiment, as described above, the image data to be determined is captured by a predetermined camera by utilizing the manufacturing variation of the light receiving sensitivity of the light receiving element that occurs in the image data generated by the camera. It is determined whether or not it is.

  The reference image generation unit 4 generates reference image data using one or a plurality of image data input from the interface 3 as necessary, and writes the generated image data in the memory 2.

The correlation detection unit 5 detects a correlation between the image data to be determined input via the interface 3 and the reference image data S7 read from the memory 2, and generates correlation data S5 indicating the detection result.
The correlation detection unit 5 outputs the correlation data to the determination unit 6.

Hereinafter, the correlation detection unit 5 employing the SPOMF (Symmetrical Phase Only Matched Filtering) method will be described.
SPOMF is described in the document "Symmetric Phase-Only Matched Filtering of Fourier-Mellin Transforms for Image Registration and Recognition" IEEE Transaction on Pattern analysis and Machine Intelligence, VOL. 16 No. 12 December 1994.
FIG. 3 is a functional block diagram of the correlation detection unit 5 shown in FIG.
As shown in FIG. 3, the correlation detection unit 5 includes, for example, an FFT circuit (Fast Fourier Transforms) 21, a whitening circuit 22, an FFT circuit 23, a whitening circuit 24, a complex conjugate circuit 25, a multiplication circuit 26, and an IFFT circuit 27. Have.
The FFT circuit 21 and the FFT circuit 23 correspond to the converting means of the present invention, the whitening circuit 22 and the whitening circuit 24 correspond to the dividing means of the present invention, and the complex conjugate circuit 25 corresponds to the replacing means of the present invention. The circuit 26 corresponds to the multiplication circuit of the present invention, and the IFFT circuit 27 corresponds to the inverse conversion circuit of the present invention.

The FFT circuit 21 performs, for example, Fourier transform on the reference image data S7 read from the memory 2 to generate second frequency component data S21, and outputs this to the whitening circuit 22.
The whitening circuit 22 divides each complex data constituting the second frequency component data S21 by the absolute value of each complex data (that is, equalizes the absolute value of each element data) to obtain the second complex data S22. Is generated and output to the multiplication circuit 26.

  For example, the FFT circuit 23 performs a Fourier transform on the image data S31 to be determined input via the interface 3 to generate first frequency component data S23, and the whitening circuit 24 converts the first frequency component data S23 into the first frequency component data S23. Each complex data forming S23 is divided by the absolute value of each complex data to generate first complex data S24, which is output to the complex conjugate circuit 25.

The complex conjugate circuit 25 generates third complex data S25 in which each complex data constituting the first complex data S24 is replaced with complex conjugate complex data, and outputs this to the multiplication circuit 26.
The multiplication circuit 26 multiplies the second complex data S22 and the third complex data S25 to generate fourth complex data S26, and outputs this to the IFFT circuit 27.
The IFFT circuit 27 performs an inverse Fourier transform on the fourth complex data S26 to generate correlation data S5, and outputs this to the determination unit 6.
Here, the correlation data indicates all values obtained by correlating the relative position between the image data S31 to be determined and the reference image data S7 cyclically in two dimensions.

By the way, as shown in FIG. 4A, in a natural image, the energy of the image is concentrated in a low band. On the other hand, since the sensitivity variation of the pixels of the light receiving element is a random phenomenon, the component superimposed on the image due to the sensitivity variation is substantially uniform over a wide frequency component such as white noise as shown in FIG. Energy.
In the above-described collation by SPOMF, the absolute value of each element is made equal after conversion into the frequency domain. For this reason, the pattern of the pixel sensitivity variation pattern can be collated without being disturbed by the low-frequency signal of the natural image.

The determination unit 6 controls the operation of each component of the data processing device 1 in a centralized manner.
In addition, the determination unit 6 determines whether or not the determination target image data S31 is generated using the predetermined camera that has generated the reference image data S7 based on the correlation data S5 generated by the correlation detection unit 5. It is determined whether or not.
By the way, as described above, as the reference image data S7, for example, one that displays a pattern having no correlation with a pattern displayed based on the determination target data S31 is used.
For this reason, when the cameras that generated the image data S31 to be determined and the image data S7 for reference differ from each other, the image data S31 to be determined and the reference image data S7 are uncorrelated. That is, a large value does not occur at the origin of the correlation data S5.
On the other hand, if the camera that generated the same is used for the image data S31 to be determined and the reference image data S7, the image data S31 to be determined and the reference image data S7 are the same It has a correlation under the influence of sensitivity.
When the correlation value indicated by the correlation data S5 exceeds a predetermined value based on the correlation data S5 detected by the correlation detection unit 5, the determination unit 6 determines the image data to be determined S31 and the reference image data S7. Are generated (imaged) by the same camera.

Hereinafter, a method of determining a value used as a criterion for determination by the determination unit 6 will be described.
As described above, the correlation data S5 indicates all values obtained by correlating the relative position between the image data S31 to be determined and the reference image data S7 cyclically two-dimensionally. .
Here, in the image data S31 to be determined and the image data for reference S7, since the pattern is uncorrelated, a value other than the origin of the correlation data S5 indicates an accidental correlation value between the uncorrelated data. .
The determination unit 6 obtains the standard deviation σ of the correlation data S5, and performs the above determination based on whether or not the value C00 of the origin of the correlation data S5 exceeds a predetermined multiple of the standard deviation.
The reason why the value C00 is used is that, when a correlation is obtained between the entire images, the unique pattern of the image sensor matches the origin, so that a peak appears in the output C00 of the correlator in that case. .

Each element data in the correlation data S5 is Cij, and the number of element data is n.
The determination unit 6 generates an average value mean of the values indicated by all the element data in the correlation data S5 based on the following equation (1).

(Equation 1)
cmean = (Σcij) / n (1)

  Further, the determination unit 6 generates a standard deviation σ based on the following equation (2) using the average value mean.

(Equation 2)
σ = {(cij-cmean) × (cij-cmean)} / n} (2)

  If the value indicated by the element data c00 at the origin in the correlation data S5 exceeds 10 times the standard deviation σ, the determination unit 6 determines that the image data S31 to be determined is based on the following equation (3). It is determined that the reference image data S7 has been generated by the same camera.

(Equation 3)
c00> 10 × σ (3)

As described above, the data processing device 1 determines whether these images are generated by the same camera having the same light receiving element by correlating the images with each other by SPOMF.
The probability of making the determination (collation) accurately can be quantified as follows.
It is considered that the result of collation between randomly distributed data follows a normal distribution. When image data captured by different light receiving elements is compared, the two data are considered to be uncorrelated. The probability that the value of correlation between uncorrelated data exceeds 10σ is 7.6 × 10 −24 power.

Hereinafter, the overall operation of the data processing device 1 shown in FIG. 1 will be described.
FIG. 5 is a flowchart for explaining the operation.
Step ST1:
For example, reference image data S7 captured by a camera to be determined is input via the interface 3 and written into the memory 2.
Step ST2:
The image data S31 to be determined is input via the interface 3 and output to the correlation detection unit 5.

Step ST3:
As described above, the correlation detection unit 5 detects the correlation between the image data to be determined input via the interface 3 and the reference image data S7 read from the memory 2, and the correlation data S5 indicating the detection result. Generate
Step ST4:
The determining unit 6 calculates the standard deviation σ of the correlation data S5 based on the above equations (1) and (2).
Step ST5:
The determining unit 6 determines whether or not the value C00 of the origin of the correlation data S5 is greater than 10 predetermined times the calculated standard deviation σ based on the above equation (3). Determines that the determination target image data S31 and the reference image data S7 have been generated by the same camera, and otherwise determines that they have been generated by different cameras.

As described above, according to the data processing device 1, information for identifying the camera that generated the image data is added to the image data without adding it as digital watermark information or a header of the image file. For all the cameras in which the image data S7 has been registered, it can be determined whether or not the image data to be determined has been generated using the camera.
Therefore, according to the data processing device 1, the camera that generated the image data can be easily specified with high reliability.
Further, according to the data processing apparatus 1, if the reference image data S7 is registered, it is determined whether or not the image data to be discriminated is generated by using the camera even if there is no camera body. it can.

Hereinafter, a method for improving the accuracy of the determination will be described.
[First method]
For example, as shown in FIG. 6, the correlation detection unit 5 cuts out image data S7a (fourth image data of the present invention) from the reference image data S7 based on the first cutout position.
In addition, the correlation detecting unit 5 converts the image data S31a to the image data S31a (the third image data of the present invention) from the image data S31 to be determined based on the second extraction position shifted by (x1, y1) from the first extraction position. Cut out).
Then, the correlation detection unit 5 detects a correlation between the image data S7a and S31a.
In this case, since the SPOMF indicates the strength of the correlation when the images are sequentially and cyclically shifted, if the determination target image data S31 and the reference image data S7 are generated by the same camera, A peak appears in the element data at the point (y1, x1) in the correlation data.
Then, the correlation detection unit 5 changes the first cutout position and the second cutout position and performs the same multiple times to generate correlation data S5, and the determination unit 6 sequentially determines using the correlation data S5. Do.
Thereby, the probability of an erroneous determination by the determination unit 6 using the correlation data S5 can be reduced. That is, the erroneous determination probability is the product of the erroneous determination probabilities in each determination. For example, if the peak value is 3σ, the probability of erroneous determination is 1 × 10 −3 in one determination, but erroneous determination up to 1 × 10 −30 by performing determination while changing the cutout position ten times. Can be reduced.

[Second method]
In the image data generated by the camera, an image obtained by forming light through a lens is superimposed with a characteristic pattern of light receiving sensitivity of a light receiving element and noise due to heat and electric factors.
Moreover, the amplitude of the image data is often quantized to 8 bits.
In order to avoid such influences of image formation, noise, and quantization by a lens, it is very useful to average a plurality of reference image data S7_1 to S7_N as shown in FIG.
In this case, N (N is an integer of 2 or more) reference image data S7_1 to S7_N generated using the camera 29 are output to the reference image generation unit 4 via the interface 3.
Then, the reference image generation unit 4 generates the reference image data S7 by averaging the reference image data S7_1 to S7_N, and writes this to the memory 2.
As a result of an experiment, the number of reference image data S7_1 to S7_N is effective to be increased to about several hundred, and the correlation value is increased to about twice as compared with the case where one reference image data is used.

[Third method]
As described above, in the camera, light enters the light receiving element via the lens.
8 to 10 show how light passing through the lens 31 is focused on the photoelectric conversion region 32 of the light receiving element.
FIGS. 8A and 8B show the case where the aperture 30 is opened at a wide angle, FIGS. 9A and 9B show the case where the aperture 30 is stopped at a wide angle, and FIG. FIG. 10B is a diagram showing a case where the aperture 30 is narrowed by telephoto when the aperture is opened.
As can be seen from these figures, the state in which the light beam 33 enters (hits) the photoelectric conversion region 32 is different.
As described above, the variation in the light receiving sensitivity of the light receiving element depends on the amount of light incident on the light receiving element.
In the present embodiment, as shown in FIG. 11, a plurality of image data a to i are generated by a camera based on the aperture ratio and the focal length of different apertures 30, and these are used as reference image data S7 to use a correlation detection unit. 5 may generate the correlation data S5.
According to an experiment, by preparing a plurality of image data a to i as the reference image data S7 in this way, the value of the correlation can be reduced to about 1.5 times as compared with a case where a single image data is prepared. I found it to go up. Thereby, the accuracy of the determination by the determination unit 6 is improved.

Hereinafter, application examples of the above-described data processing device 1 will be described.
[First application example]
This application example corresponds to the third and fourth inventions.
The data processing device 1 is applicable, for example, when determining a camera that has taken a picture from image data.
As described above, by taking the correlation between the image data, it is possible to determine whether the image is taken by the same camera or an image taken by another camera.
Accordingly, it is determined as to which of the plurality of cameras the image data has been shot as follows.
For example, consider the case where there are cameras 40, 41, and 42 as shown in FIG.
In this case, as shown in FIG. 13, a predetermined subject is imaged by the cameras 40, 41, and 42, and reference image data S7_1, S7_2, and S7_3 corresponding to the imaging result are generated, respectively.
At this time, it is preferable that the subject has a uniform brightness and no pattern like a blue sky. This makes it possible to generate reference image data in which the target image data S31 and the pattern are uncorrelated. A similar effect can be obtained by disposing a milky white filter immediately before the camera lens.
Then, as shown in FIG. 13, the reference image data S7_1, S7_2, and S7_3 input via the interface 3 of the data processing device 1 are written in the memory 2.
Then, the correlation detection unit 5 generates correlation data S5 when the reference image data S7_1, S7_2, and S7_3 are used for the image data S31 to be determined.
Then, the determination unit 6 specifies the correlation data having a large peak at the position of the origin among the correlation data S5 when the reference image data S7_1, S7_2, and S7_3 are used, and uses the correlation data to generate the correlation data. It is determined that the image is captured by the camera corresponding to the reference image data.
As a result, as shown in FIG. 12, when there is image data 51, 52, and 53, these are determined by the data processing device 1 shown in FIG. The cameras 40, 41, 42 that have captured the images 52, 53 can be specified.

[Second application example]
The data processing device 1 may be applied when determining whether the image data has been tampered with.
An image is taken by a predetermined camera in advance to generate reference image data S7, which is stored in the reference image data S7 of the data processing device 1.
When the image data generated by the camera is received, the above-described determination is performed using the received image data as the determination target image data S31 using the data processing device 1, and the received image is received. It is determined whether or not the data is valid data captured by the predetermined camera.
If only the header information of the existing image is falsified and sent without taking the image with the predetermined camera, the received image data is incorrect because the correlation with the reference image data S7 cannot be obtained. Can be determined.
In the case where the correlation is obtained by the correlation detection unit 5 as described above, the correlation is not shown if the image is scaled, deformed, or rotated. Therefore, if tampering such as cutting out a part of the image and enlarging it, enlarging half of the image and reducing it half, rotating the image several times, etc. does not show the correlation, it may not be the original image Understand.

[Third application example]
For example, when it is determined that the received image data is generated (imaged) by a valid camera according to the second application example described above, a stronger image of the received image data is added to the received image data. Digital watermark data (additional data) is embedded as an identifier.
By embedding digital watermark data in this way, it is possible to use information that is easy to detect even after performing operations such as filtering and deformation, thereby giving the image information about the image including the copyright of the image. .
As shown in FIG. 14, the computer 45 inputs image data from the cameras 40, 41, and 42 and determines whether or not these are valid image data in which the reference image data S7 is registered.
Then, when the computer 45 determines that the image data is valid, the computer 45 embeds digital watermark data in the image data and outputs it to the network.
Thereby, the computer 45 can prevent the digital watermark data from being embedded when receiving the unauthorized image downloaded via the network.

[Fourth application example]
The data processing device 1 may be applied when determining whether or not the moving image data has been tampered with.
In this case, the image data generated by a valid video camera is written in the memory 2 of the data processing device 1 as reference image data S7.
FIG. 15A is a diagram for explaining moving image data S60 generated by a valid video camera.
As shown in FIG. 15A, the moving image data S60 includes a header and scenes (image data) 1 to 6.
FIG. 15B is a diagram for explaining moving image data S61 that has been tampered with illegally.
As shown in FIG. 15 (B), moving image data S61 is obtained by replacing scene 3 of moving image data S60 shown in FIG. 15 (A) with an illegal scene 3A taken by a camera other than the above-mentioned valid video camera. It is.
The data processing device 1 can determine that the scene 3A has been tampered with, for example, by sequentially inputting each scene of the moving image data S61 and correlating it with the reference image data S7.
That is, for the scenes 1, 2, 4, and 5 of the moving image data S61, a correlation exceeding the predetermined value is obtained with the reference image data S7, but for the scene 3A, a correlation exceeding the predetermined value is not obtained.
In this case, reference image data generated by a valid video camera may be attached to a header of moving image data or the like, and the data processing device 1 may use the reference image data S7.

[Fifth application example]
In the application example, it is determined whether or not a part of the image data S31 to be determined is generated (partially falsified image) based on the image data generated by the camera that generated the reference image data S7. The case will be described.
This application example corresponds to the eleventh and twelfth inventions.
FIG. 16 is a diagram for explaining the application example.
As shown in FIG. 16, in the application example, the correlation detection unit 5a shown in FIG. 1 receives the reference image data S7 and divides it into a plurality of block image data S7a.

Further, the correlation detection unit 5a receives the image data S31 to be determined and divides it into a plurality of block image data S31a.
The division by the correlation detection unit 5a is performed based on the same boundary when the reference image data S7 and the determination target image data S31 have the same size.
Here, the reference image data S7 and the determination target image data S31 are, for example, two-dimensional images of 600 pixels vertically and 800 pixels horizontally.
The block image data S7a and S31a are, for example, two-dimensional images each having 200 pixels vertically and horizontally.
The correlation detecting unit 5a determines, for each of the plurality of block image data S31a, a correlation between the block image data S31a and the block image data S7a corresponding to the block image data S31a among the plurality of block image data S7a. To detect. The method for detecting the correlation is performed based on the above-described SPOMF. In this case, the above-mentioned correlation value at the origin is normalized by the standard deviation obtained from the values of all the outputs of the SPOMF.
Then, the correlation detection unit 5a outputs the detected correlation CO for each of the plurality of block image data S31a to the determination unit 6a.

The determination unit 6a generates the determination target image data S31 by the camera that has generated the reference image data S7 based on the distribution of the detection intensity indicated by the correlation CO based on the correlation CO input from the correlation detection unit 5a. It is determined whether or not the image data is generated by partially falsifying the image data.
Specifically, as shown in FIG. 23, the determination unit 6a generates a histogram HS indicating the correspondence between the detected intensity of the correlation CO and the number of block image data indicating the detected intensity within each detected intensity range. The above determination is performed based on the histogram HS.

The determination unit 6a performs the above determination as follows.
That is, when there is no alteration in the image data S31 to be determined, as shown in FIG. 17A, the detection strength of the correlation indicated by the histogram HS falls within a predetermined range around a certain value. That is, one group is formed.
On the other hand, when the determination target image data S31 has the falsification, as shown in FIG. 17B, the correlation detection strength indicated by the histogram HS exceeds the predetermined level. One peak P1 and another peak P2 whose correlation detection intensity does not exceed the predetermined level. That is, two groups are formed. Here, the portion corresponding to the mountain P1 is a portion using the image of the camera that generated the reference image data S7, and the portion corresponding to the mountain P2 is a portion using an image other than the camera.

When the generated histogram HS has only one peak exceeding a predetermined level, the determination unit 6a determines that the image data S31 to be determined is not an image obtained by partially modifying the image generated by the camera. I do.
On the other hand, when the generated histogram HS has one peak that exceeds a predetermined level and another peak, the determination target image data S31 partially alters the image generated by the camera. It is determined that it has been performed.

Hereinafter, an operation example of the data processing device of this application example will be described.
FIG. 18 is a diagram for explaining the operation example.
Step ST11:
1 and 16 receives the reference image data S7 and divides it into a plurality of block image data S7a.
Further, the correlation detection unit 5a receives the image data S31 to be determined and divides it into a plurality of block image data S31a.
Step ST12:
The correlation detecting unit 5a determines, for each of the plurality of block image data S31a, a correlation between the block image data S31a and the block image data S7a corresponding to the block image data S31a among the plurality of block image data S7a. To detect.

Step ST13:
As illustrated in FIG. 16, the determination unit 6a generates a histogram HS indicating the correspondence between the detected intensity of the correlation CO and the number of block image data indicating the detected intensity within each detected intensity range.
Step ST14:
When the histogram HS generated in step ST13 has only one peak exceeding a predetermined level, the determination unit 6a determines that the image data S31 to be determined is not a partially altered image generated by the camera. Is determined.
On the other hand, when the histogram HS generated in step ST13 has one peak exceeding a predetermined level and another peak, the determination target image data S31 determines that the image generated by the camera is one. It is determined that the copy has been tampered with.

  The data processing apparatus described above with reference to FIGS. 16 to 18 can be applied to, for example, a system that determines whether an image posted on a homepage on a network or an image bulletin board has been generated by falsifying another image. It is.

FIG. 19 is a configuration diagram of the system 220.
As shown in FIG. 19, the system 220 has, for example, a PC 200 and databases 210, 211, and 212.
The PC 200 uses the image data provided by the one or more servers 204 on the network 202 as the image data to be determined S31, detects the correlation between the image data for reference S7 read from the database 211, and performs the above-described determination. I do.
The database 210 stores, for example, addresses of images such as homepages and bulletin boards provided by one or more servers 204 connected on the network 202.
The database 211 stores, for example, reference image data S7 generated by one or more predetermined cameras.
The database 212 stores an address (URL) of image data on the network 202 that uses the image data generated from the same camera as the reference image data S7 stored in the database 211 after falsification.

In the system 220 shown in FIG. 19, the correlation detection unit 5a of the PC 200 uses the image data read from the database 203 of the server (s) 204 on the network 202 as the image data S31 to be determined, and reads the reference data read from the database 211. The correlation with the image data for use S7 is detected.
Then, the determining unit 6a of the PC 200 determines whether the image data read from the database 203 or the image of the camera that generated the reference image data S7 stored in the database 211 has been tampered based on the correlation. If it is determined that the address has been falsified, the address is stored in the network 212.

As described above, according to the application example described above, it is possible to automatically determine that a part of the image data S31 to be determined is suspected of tampering.
Further, according to the system 201 shown in FIG. 19, it is possible to automatically detect whether or not a part of image data such as a homepage or a bulletin board on the network 202 has been tampered, and generate a list thereof.

[First business application example]
The business application example corresponds to the fifth and sixth inventions.
FIG. 20 is a diagram for explaining a case where the data processing apparatus 1 specifies a stolen camera by a print service.
As shown in FIG. 20, image data S72 generated in advance by a stolen registered camera is registered in the data processing device 1.
The user of the camera 71 takes the memory card on which the image is recorded to a DPE store and has the printer 73 print it.
At this time, the data processing device 1 performs the above-described correlation detection and determination by the data processing device 1 between the image data read from the memory card of the camera 73 and the image of the stolen camera registered in advance. This makes it possible to determine whether or not the image was taken with a stolen camera.
If the correlation between the image of the stolen camera and the correlation exceeds a predetermined value as a result of the image comparison, the fact is notified to the criminal investigation agency 74.
As the image data S72 registered as the reference image data S7, it is desirable to photograph a plurality of flat scenes in advance, but even if the device is stolen without any preparation, the already photographed image can be referred to as the reference image data S72. It may be registered as S7.

[Second business application example]
FIG. 21 is a diagram for describing a case where the data processing device 1 is applied to an ISP (Internet Service Provider) 86.
As shown in FIG. 21, the user uploads image data S82 captured by the camera 81 from a computer 83 to an ISP (Internet Service Provider) 86.
The ISP 86 outputs the uploaded image data S82 to the data processing device 1.
Then, the data processing device 1 performs the correlation detection and determination by the data processing device 1 between the image data S82 input from the ISP 86 and the image data S84 of the stolen camera registered in advance. It is possible to determine whether or not the image was taken by a stolen camera.
When the ISP 86 determines that the image data S82 is not captured by the stolen camera, the ISP 86 uses the image data 82 as data of a homepage or the like.

[Third business application example]
FIG. 22 is a diagram for explaining a case where the data processing device 1 is applied to specify a camera that has captured a voyeur image.
Currently, video cameras are brought into movie theaters to shoot movies in an illegal manner, posted on uploaders on the Internet, etc., or exchange files between users, Significantly violates rights.
Therefore, the reference image data S7 of all the manufactured video cameras is registered in the data processing device 1, and the data processing device 1 performs correlation detection and determination with the image data S93 of the voyeur movie. It is possible to identify the camera that performed the voyeur. Reporting camera information to the copyright holder 94 helps to identify vendors that create pirated content in an illegal manner.

[Fourth business application example]
This business application example corresponds to the seventh and eighth inventions.
FIG. 23 is a diagram for explaining a case where the data processing apparatus 1 is applied to determine whether a legally contracted company is broadcasting its own content.
As shown in FIG. 23, the image data S105 captured by the television station A101 using the camera 102 is registered in the data processing device 1 as reference image data S7.
The television station A101 provides the content data to the news distribution company 103.
Then, when the news distribution company 103 provides the content data to the television station B104, the television station A101 receives the content data broadcast by a plurality of television stations including the television station B104, and uses the received data as the image data S31 to be determined. Output to the processing device 1.
Thereby, the television station A101 can specify which television station is broadcasting its own content data based on the determination result of the data processing device 1.
Then, the television station A101 can take some legal action if the broadcasting of the content data is not permitted to the television station broadcasting its own content data.

[Fifth business application example]
FIG. 24 is a diagram for explaining a case where the data processing apparatus 1 is used to determine the validity of the evidence photograph of the insurance company.
It is necessary to confirm that the evidence photo of the accident used by the insurance company is correctly taken on site.
The insurance company registers, for example, the reference image data S7 previously taken by the camera 110 registered by the insurance company in the data processing device 1.
Then, at the site, the image data S111 captured by the camera 110 is transmitted from the PC 112 to the PC 113 via the network.
Then, the PC 113 outputs the image data received from the PC 112 to the data processing device 1 as the image data S31 to be determined.
Then, in the data processing device 1, by performing the above-described processing, it can be determined whether or not the image data S111 has been photographed by the valid camera 110.

[Sixth business application example]
The data processing device 1 is also applicable to, for example, transmitting images of a surveillance camera and checking at a remote location, and determining whether or not an image has been replaced halfway.
In this case, the image data captured by the monitoring camera is registered in the data processing device 1 as reference image data S7 in advance.
Then, in the data processing device 1, by using the image data received from the monitoring camera as the image data S31 to be determined and correlating with the reference image data S7, the received image data is It is possible to appropriately determine whether the image has been captured.

[Seventh business application example]
This business application example corresponds to the ninth and tenth inventions.
FIG. 25 is a diagram for explaining a case where the data processing device 1 is applied to data management.
As shown in FIG. 25, consider a case where an image file of image data captured using a plurality of cameras 121, 122 and the like is stored in a memory of the PC 120.
In this case, the user registers the reference image data S7 captured by the cameras 121 and 122 in advance in the data processing device 1.
Then, when the user searches for an image file of image data captured by the cameras 121 and 122, the user instructs the data processing apparatus 1 to reference image data S7 of the cameras 121 and 122.
The data processing apparatus 1 inputs all image files in the memory of the PC 120 as the image data S31 to be determined based on the instruction, and determines (searches) image files of image data captured by the cameras 121 and 122. I do. In this case, the determination unit 6 shown in FIG. 1 corresponds to a search unit of the present invention.
Then, based on the determination result, the data processing device 1 compares the image files in the memory of the PC 120 with an image file group 123 captured by the camera 121, an image file group 124 captured by the camera 122, and the other camera. It is classified into a group of captured image files 125.

[Eighth business application example]
FIG. 26 is a diagram for explaining a case in which the data processing device 1 is applied to prevent use of a stolen mobile phone device.
Even if the GSM mobile phone 201 is stolen, it can be used by replacing the SIM card. As an anti-theft measure, an identifier (IMEI) provided on the mobile phone 201 is frequently rewritten. Such crime can be prevented by using a camera attached to the mobile phone 201 as the individual identifier of the mobile phone 201.
When the mobile phone 201 starts a telephone call, one still image is sent from the mobile phone 201 to the repeater 202 of the telephone company when dial-up is performed. By using the still image as the reference image data S7 in the data processing device 1 using the image data S203 of the stolen mobile phone as the reference image data S7, the stolen mobile phone is used. It can be determined whether the call is the telephone 201 or not.
If the data processing device 1 determines that the mobile phone 201 is the stolen mobile phone 201, the data processing device 1 notifies the criminal investigation period 204.

[Ninth business application example]
An image bulletin board on the Internet for the purpose of posting photos is a convenient place for easy photo announcements and information exchange with photos. With the spread of digital cameras, people with various hobbies and preferences have come to use image bulletin boards.
However, these image bulletin boards often fail due to vandalism. Examples of vandalism methods include non-photographs such as TV capture images and magazine scan images, unauthorized reproduction of other people's photographs, obscene images, and continuous posting of the same image.
As a method of preventing such vandalism, it is perfect based on personal authentication, such as issuing an ID and password after confirming the identity of the user with a credit card or the like, but such a method does not attract people.
The simple ID system using cookies works to some extent, but it becomes meaningless if cookies are deleted.
In the ninth business application example, the above-described conventional problem is solved by using the above-described data processing device 1.

FIG. 27 is a diagram illustrating a ninth business application example of the present invention.
This business application example corresponds to the fourteenth invention.
As shown in FIG. 27, the ISP 486 has, for example, the data processing device 1 and the database 450 shown in FIG.
The database 450 stores, for example, valid posted images that have been determined by the data processing device 1.
FIG. 28 is a diagram for describing an example of the bulletin board image 451.
As shown in FIG. 28, a bulletin board image 451 displays a posted image posted (uploaded) by a plurality of contributors to the ISP 486, a contributor name (user ID), and a comment by the contributor. .

A user who wants to post an image on the bulletin board image 451 first performs user registration with the ISP 486.
FIG. 29 is a diagram for describing a user registration operation performed by the data processing device 1 of the ISP 486.
Step ST21:
The user operates the computer 83 to transmit a user registration request to the ISP 486.
The determination unit 6 of the data processing device 1 illustrated in FIG. 1 of the ISP 486 transmits a request for a user ID and a reference image to the computer 83 in response to the user registration request. Step ST22:
The user operates the computer 83 to transmit to the ISP 486 the user ID such as the handle name determined by the user and the reference image data S482 captured by using the camera 81 of the user.
It is preferable that the reference image data S482 has information on all pixels of the image sensor of the camera 81. Further, it is preferable that information other than the pixel sensitivity variation is not included. An ideal image is a gray and flat image that is not focused on the subject. Such an image is obtained by taking a digital camera lens close to white paper. Further, it is desirable that the reference image be at least a certain size. An example of the lower limit size may be about VGA.
The ISP 486 requests the computer 83 for a reference image having such characteristics in step ST21, for example.

Step ST23:
The determination unit 6 of the data processing device 1 verifies whether or not the reference image data S482 received in step ST22 is a flat image, and proceeds to step ST24 if it determines that the reference image data S482 is a flat image. Returns to step ST21.
Step ST24:
The determination unit 6 of the data processing device 1 verifies whether or not the reference image data S482 received in step ST22 is a gray image, and proceeds to step ST25 if it determines that the reference image data S482 is a gray image. Returns to step ST21.
Step ST25:
The determination unit 6 of the data processing device 1 verifies whether or not the reference image data S482 received in step ST22 has a predetermined size. Otherwise, the process returns to step ST21.
Step ST25:
The determination unit 6 of the data processing device 1 writes the reference image data S482 (S7) received in step ST22 and the user ID in the memory 2 shown in FIG. 1 in association with each other.

Next, an operation when the ISP 486 receives a posted image will be described.
30 and 31 are diagrams for explaining the operation.
Step ST31:
The user operates the computer 83 to transmit an image posting request to the ISP 486.
The determination unit 6 of the data processing device 1 illustrated in FIG. 1 of the ISP 486 requests the computer 83 for the user ID and the posted image in response to the image posting request.
Step ST32:
The user operates the computer 83 to transmit to the ISP 486 a registration request including a user ID such as a handle name determined by the user and post image data S483 captured using the camera 81 of the user.
Step ST33:
The determination unit 6 of the data processing device 1 searches the memory 2 shown in FIG. 1 for reference image data S482 corresponding to the user ID based on the user ID received in Step ST32: Step ST34:
The determination unit 6 of the data processing apparatus 1 determines whether or not the reference image data S482 has been searched in step ST33. If it is determined that the search was successful, the process proceeds to step ST35. If not, the process returns to step ST31.

Step ST35:
The determination unit 6 of the data processing device 1 determines whether or not the reference image data S482 retrieved and read in step ST33 and the posting image data S483 received in step ST32 have the same size. If it is determined that there is, the process proceeds to step ST38; otherwise, the process proceeds to step ST36.
Step ST36:
The determination unit 6 of the data processing device 1 determines whether or not the reference image data S482 and the posting image data S483 have the same aspect ratio. If the determination unit 6 determines that they have the same aspect ratio, the process proceeds to step ST37. Otherwise, the process returns to step ST31.
Step ST37:
The determination unit 6 of the data processing device 1 changes the scale of the reference image data S482 so as to match the scale of the posting image data S483, and sets this as reference image data S482.

Step ST38:
The correlation detection unit 5 of the data processing device 1 generates the correlation data S5 using the reference image data S482 as the reference image data S7 and the posting image data S483 as the image data S31 to be determined.
Step ST39:
The determination unit 6 of the data processing device 1 determines whether or not the correlation data S5 generated in step ST38 is equal to or greater than a predetermined threshold, and proceeds to step ST40 when determining that the correlation data S5 is equal to or greater than the predetermined threshold. If not, the process returns to step ST31.
Step ST40:
The determination unit 6 of the data processing device 1 writes the posting image data S483 received in step ST32 into the database 450 as a posting image.
As a result, the posting image data S483 is posted on the image posting plan 451 shown in FIG.

According to the ninth business application example, when a TV capture image, an image obtained by scanning a magazine with a scanner, or the like is uploaded, since these images do not have the pixel-to-pixel sensitivity variations found in the reference image, the reference It is easy to see that the image was not taken with the same camera as the image for use. In such a case, posting on the bulletin board may be rejected.
In addition, even if the image was taken with the camera that took the reference image, there are images that are not suitable for bulletin boards, such as continuous posting of meaningless photos, rephotographs of magazine gravure, etc., images that are different from the bulletin board theme. When posted, the user ID and image are displayed first. It is common for a vandal to not want to know who is going, so that vandalism can often be naturally stopped by displaying a user ID.
If the vandalism continues even though the user ID is displayed, the user is removed from the user registration database. If an attempt is made to post falsely with another user ID, the uploaded image cannot be collated with the reference image registered with the user ID, and therefore will not be posted on the bulletin board.

[Modification of Correlation Detection Unit 5]
In the above-described correlation detection unit 5, the same processing is performed on the determination target image data S31 and the reference image data S7 in the whitening circuits 22 and 24 shown in FIG.
However, the image data S31 to be determined and the image data S7 for reference are originally images having different characteristics, and performing the same processing is not the best as a collation method.
.
FIG. 32 is a functional block diagram of the correlation detection unit 505 according to the present modification.
As shown in FIG. 32, the correlation detection unit 505 includes, for example, an FFT circuit 21, a whitening circuit 522, an FFT circuit 23, a whitening circuit 524, a complex conjugate circuit 25, a multiplication circuit 26, and an IFFT circuit 27.
The present invention corresponds to the thirteenth invention.
Here, the FFT circuit 21 and the FFT circuit 23 correspond to the converting means of the present invention, the whitening circuit 522 and the whitening circuit 524 correspond to the dividing means of the present invention, and the complex conjugate circuit 25 corresponds to the replacing means of the present invention. The multiplication circuit 26 corresponds to the multiplication circuit of the present invention, and the IFFT circuit 27 corresponds to the inverse conversion circuit of the present invention.
32, the FFT circuit 21, the FFT circuit 23, the complex conjugate circuit 25, the multiplication circuit 26, and the IFFT circuit 27 denoted by the same reference numerals as in FIG. 3 are the same as those described with reference to FIG.
The whitening circuit 522 converts each complex number data A (i, j) constituting the second frequency component data S21 input from the FFT circuit 21 into an absolute value of the complex number data, as represented by the following equation (4). The second complex data Ad (i, j) (= S522) is generated by dividing | A (i, j) | by the power of k1.

(Equation 4)
Ad (i, j) = A (i, j) / (| A (i, j) | ^ k1) ... (4)

  The whitening circuit 524 converts each complex number data B (i, j) constituting the first frequency component data S23 input from the FFT circuit 23 into an absolute value of the complex number data as shown in the following equation (5). The first complex data Bd (i, j) (= S524) is generated by dividing | B (i, j) | by the power of k2.

(Equation 5)
Bd (i, j) = B (i, j) / (| B (i, j) | ^ k2) ... (5)

Here, k2> k1, and the first frequency component data S23 is subjected to a stronger amplitude limitation than the second frequency component data S21.
In the present embodiment, for example, k1 is, for example, 0.4 to 0.8, and k2 is, for example, 1.0 to 1.3.
In the experiment using the video camera A, k1 = 0.45 and k2 = 1.30 were optimal, and in the experiment using the digital still camera B, k1 = 0.6 and k2 = 1.0 were optimal.

The complex conjugate circuit 25 generates third complex data S25 in which each complex data constituting the first complex data S524 is replaced with complex conjugate complex data, and outputs this to the multiplication circuit 26.
The multiplication circuit 26 generates the fourth complex data S26 by multiplying the second complex data S522 and the third complex data S25, and outputs this to the IFFT circuit 27.
The IFFT circuit 27 performs inverse Fourier transform on the fourth complex data S26 to generate correlation data S505, and outputs this to the determination unit 6.
Here, the correlation data indicates all values obtained by correlating the relative position between the image data S31 to be determined and the reference image data S7 cyclically in two dimensions.

  The determination unit 6 determines whether or not the determination target image data S31 is generated using the predetermined camera that has generated the reference image data S7 based on the correlation data S505 generated by the correlation detection unit 505. Is determined.

  FIG. 33 is a diagram in which the values of the correlation are plotted in the digital still camera B while changing k1 and k2. By plotting in this manner, the optimum value can be found.

For example, when the FFT circuit 21 multiplies the reference image data S7 by a predetermined FFT coefficient (conversion coefficient) to generate the second frequency component data S21, the whitening circuit 522 illustrated in FIG. The second complex component data S22 is further subjected to a process of zeroing the FFT coefficient corresponding to a predetermined lower frequency band with respect to a predetermined frequency to the second frequency component data S21 defined based on May be generated.
Further, the whitening circuit 522 may further perform, on the second frequency component data S21, a process of setting the FFT coefficient corresponding to the predetermined frequency in the horizontal direction on the wide band side to the predetermined frequency to zero. Good.
More specifically, the whitening circuit 522 sets the low-frequency signals indicated by oblique lines and the high-frequency signals in the horizontal direction to zero in the output matrix 550 of the two-dimensional FFT illustrated in FIG. As a customary way of arranging the two-dimensional FFT coefficients, there are a case where the four corners are low and the center is high, and a case where the four corners are high and the center is low. FIG. 34 shows the former (the four corners are low). In this case, the center is the high band.

In the output matrix of the FFT shown in FIG. 34, H and V indicate the numbers of horizontal and vertical FFT coefficients, respectively. The number of these coefficients is equal to the number of pixels in the image before performing a two-dimensional FFT. lh and lv indicate ranges where the low-frequency coefficients at the four corners are set to zero. hv indicates a range in which the high frequency coefficient in the vertical direction is zero.
In the present embodiment, for example, “lh = h * k1”, k1 is 0.1 to 0.15,
"Lv = v * k2", where k2 is 0.1 to 0.15,
“Hv = v * k3”, where k3 is 0 to 0.2.

FIG. 35 shows the value of the correlation data S5 when the frames of the scenes 1 to 5 captured by the cameras 1 to 3 are collated on a frame basis using the correlation detection unit 5 shown in FIG. (Vertical axis).
FIG. 36 shows the values of the correlation data S505 when the frames of the scenes 1 to 5 captured by the cameras 1 to 3 are collated on a frame basis using the correlation detection unit 505 shown in FIG. (Vertical axis).
Scene 1 is an image captured under the same shooting conditions as the reference image.
Scene 2 is an image captured with the camera stationary under normal shooting conditions.
Scene 3 is an image captured under normal shooting conditions with the camera panned.
Scene 4 is an image captured in an overexposed state.
Scene 5 is an image captured in a completely dark state.
Among scenes 1 to 5, scene 1 has a very high correlation. Here, the collation results in scene 2 and scene 3 are practically important. Since scenes 4 and 5 are not under normal shooting conditions, there is no problem if collation cannot be performed.
As shown in FIGS. 35 and 36, by using the correlation detection unit 505 in the data processing device 1, the matching accuracy of the camera 1 in the scene 2 can be improved as compared with the case where the correlation detection unit 5 is used. .

  The present invention is applicable to a data processing system that determines a camera that has captured image data.

FIG. 1 is a configuration diagram of a data processing device according to an embodiment of the present invention. FIG. 2 is a diagram schematically illustrating a cross section of a light receiving element provided in the camera. FIG. 3 is a functional block diagram of the correlation detection unit shown in FIG. FIG. 4 is a diagram for explaining a natural image and a unique pattern of a light receiving element. FIG. 5 is a flowchart for explaining the operation of the data processing device shown in FIG. FIG. 6 is a diagram for describing a first technique for improving the accuracy of the determination of the data processing device shown in FIG. FIG. 7 is a diagram for describing a second technique for improving the accuracy of the determination of the data processing device shown in FIG. FIG. 8 is a diagram for explaining a third technique for improving the accuracy of the determination of the data processing device shown in FIG. FIG. 9 is a diagram for explaining a third technique for improving the accuracy of the determination of the data processing device shown in FIG. FIG. 10 is a diagram for describing a third technique for improving the accuracy of the determination of the data processing device shown in FIG. FIG. 11 is a diagram for explaining a third technique for improving the accuracy of the determination of the data processing device shown in FIG. FIG. 12 is a diagram for describing a first application example of the data processing device shown in FIG. FIG. 13 is a diagram for describing a first application example of the data processing device shown in FIG. FIG. 14 is a diagram for explaining a third applied example of the data processing device shown in FIG. FIG. 15 is a diagram for describing a fourth applied example of the data processing device shown in FIG. FIG. 16 is a diagram for describing a fifth applied example of the data processing device shown in FIG. FIG. 17 is a diagram for explaining the processing of the determination unit shown in FIG. FIG. 18 is a flowchart for explaining an operation example of the data processing device shown in FIG. 17 is a configuration diagram of a system that performs tampering determination to which the data processing device shown in FIG. 16 is applied. FIG. 20 is a diagram for describing a first business application example of the data processing device shown in FIG. FIG. 21 is a diagram for describing a second business application example of the data processing device shown in FIG. FIG. 22 is a diagram illustrating a third business application example of the data processing device illustrated in FIG. 1. FIG. 23 is a diagram for describing a fourth business application example of the data processing device shown in FIG. FIG. 24 is a diagram for describing a fifth business application example of the data processing device shown in FIG. FIG. 25 is a diagram for describing a seventh business application example of the data processing device shown in FIG. FIG. 26 is a diagram for describing an eighth business application example of the data processing device shown in FIG. FIG. 27 is a diagram for describing a ninth business application example of the data processing device shown in FIG. FIG. 28 is a diagram for describing a ninth business application example of the data processing device shown in FIG. FIG. 29 is a diagram for describing a ninth business application example of the data processing device shown in FIG. FIG. 30 is a diagram for describing a ninth business application example of the data processing device shown in FIG. FIG. 31 is a diagram for describing a ninth business application example of the data processing device shown in FIG. FIG. 32 is a diagram for explaining a modification of the correlation detection unit shown in FIG. FIG. 33 is a diagram for explaining a modification of the correlation detection unit shown in FIG. FIG. 34 is a diagram for describing a modification of the correlation detection unit shown in FIG. FIG. 35 is a diagram for describing a modification of the correlation detection unit shown in FIG. FIG. 36 is a diagram for explaining a modification of the correlation detection unit shown in FIG.

Explanation of reference numerals

DESCRIPTION OF SYMBOLS 1 ... Data processing apparatus, 2 ... Memory, 3 ... Interface, 4 ... Reference image generation part, 5, 5a, 505 ... Correlation detection part, 6, 6a ... Judgment part, S7 ... Reference image data, 9 ... Bus, 11 ... Light receiving element, 12 ... silicon substrate, 13 ... opening, 14 ... light shielding layer, 15 ... condensing lens, 16 ... photosensitive drum, 16 ... insulating layer, 21 ... FFT circuit, 22 ... whitening circuit, 23 ... FFT circuit, 24 whitening circuit, 25 complex conjugate circuit, 26 multiplication circuit, 27 IFFT circuit

Claims (28)

A data processing device that determines whether or not the image data is generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity,
Correlation detection means for detecting a correlation between the first image data to be determined and the second image data for reference generated by the image generation device;
A data processing device comprising: a determination unit configured to determine whether the first image data is generated using the image generation device based on the correlation detected by the correlation detection unit. 2. The data processing device according to claim 1, wherein the correlation detection unit detects the correlation by using the second image data that displays a pattern having no correlation with a pattern displayed based on the first image data. 3. . The correlation detecting means cuts out third image data from the first image data based on a first cutout position, and converts fourth image data from the second image data based on a second cutout position. The data processing according to claim 1, wherein the extraction and the detection of the correlation between the third image data and the fourth image data are performed plural times by changing the first extraction position and the second extraction position. apparatus. Reference image generation means for generating a plurality of image data using the predetermined image generation device, averaging the plurality of image data to generate the second image data,
The data processing device according to claim 1, wherein the correlation detection unit detects the correlation using the second image data generated by the reference image generation unit. When the image generating apparatus causes light to enter the plurality of light receiving elements via a lens,
Reference image generating means for generating a plurality of second image data by causing light to enter the light receiving element in each of a plurality of states in which the aperture of the lens is different,
The data processing device according to claim 1, wherein the correlation detection unit detects the correlation using the plurality of second image data generated by the reference image generation unit. The correlation detecting means calculates the correlation between the first image data and the second image data by using a pixel data corresponding to the light receiving element as a unit and the image of the first image data and the second image data. The data processing device according to claim 1, wherein the data processing device generates correlation data indicating the correlation detected by sequentially shifting a relative position of the second image data and the image in two dimensions. The correlation detecting means,
Conversion means for orthogonally transforming the first image data and the second image data to generate first frequency component data and second frequency component data, respectively;
Each complex number data forming the first frequency component data is divided by an absolute value of the complex number data to generate first complex number data, and each complex number data forming the second frequency component data is divided into Division means for generating second complex data by dividing by the absolute value of the complex data;
Replacement means for generating third complex data in which each complex data constituting one of the first complex data and the second complex data is replaced by complex conjugate complex data;
Fourth complex data is generated by multiplying the first complex data or the second complex data that has not been replaced by the replacement means and the third complex data generated by the replacement means. Multiplying means;
The data processing apparatus according to claim 6, further comprising: an inverse transform circuit that performs an inverse orthogonal transform on the fourth complex number data generated by the multiplying unit to generate correlation data indicating the correlation. Adding the additional data for preventing copyright infringement to the first image data when the determining means determines that the first image data is generated using the image generating device; The data processing device according to claim 1, further comprising: means. The data processing device according to claim 1, wherein the inherent variation in the light receiving sensitivity of the light receiving element occurs during a manufacturing process of the light receiving element and is artificially difficult to reproduce. A data processing method for determining whether or not image data is generated by a predetermined image generating apparatus that generates image data based on a light receiving result of a light receiving element having a variation inherent in light receiving sensitivity,
A first step of detecting a correlation between the first image data to be determined and the second image data for reference generated by the image generation device;
A second step of determining whether or not the first image data has been generated by using the image generation device based on the correlation detected in the first step. . It is determined which of the plurality of image generating apparatuses generates image data based on a light receiving result of a light receiving element having a unique variation in light receiving sensitivity, and which of the plurality of image generating apparatuses has generated the first image data to be determined. Data processing device,
Holding means for holding a plurality of second image data for reference each generated using the plurality of image generating devices;
Correlation detection means for detecting a correlation between the first image data and the second image data held by the holding means;
A determination unit configured to determine whether the first image data is generated using any of the plurality of image generation devices based on the correlation detected by the correlation detection unit. apparatus. Which of the plurality of image generating devices generates image data based on the light receiving result of the light receiving element having a specific variation in light receiving sensitivity, and which of the plurality of image generating devices has generated the first image data to be determined is A data processing method performed by a data processing device that determines and stores a plurality of second image data for reference each generated using the plurality of image generation devices,
A first step of detecting a correlation between the first image data and the held second image data;
A second step of determining whether or not the first image data has been generated using any of the plurality of image generation devices based on the correlation detected in the first step. Data processing method. A data processing device that determines whether or not first image data of a subject is generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. So,
Holding means for holding a plurality of second image data for reference generated in advance using the stolen image generation device;
Correlation detection means for detecting a correlation between the first image data and the second image data held by the holding means;
A determination unit that determines whether or not the first image data has been generated using the stolen image generation device based on the correlation detected by the correlation detection unit. . It is determined whether or not first image data to be determined has been generated by an image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. A data processing method performed by a data processing device that holds a plurality of second image data for reference generated in advance using the device,
A first step of detecting a correlation between the first image data and the held second image data;
A second step of determining whether or not the first image data has been generated using the stolen image generation device based on the correlation detected in the first step. Data processing method. A data processing device that determines whether or not first image data to be determined has been generated by an image generating device that generates image data based on a light receiving result of a light receiving element that has a unique variation in light receiving sensitivity,
Licensed broadcast source data that specifies a broadcast source that is permitted to broadcast image data generated using the predetermined image generation device, and second image data for reference generated using the predetermined image generation device Holding means for holding
Correlation detection means for detecting a correlation between the first image data broadcasted by a predetermined broadcast source and the second image data held by the holding means;
A first determination unit configured to determine whether the first image data is generated using the image generation device based on the correlation detected by the correlation detection unit;
When the first determination unit determines that the first image data is generated using the image generation device, the first determination unit determines the first image data based on the licensed broadcast source data held by the holding unit. A second determination unit that determines whether the license has been issued to the broadcast source that broadcasts the first image data. It is determined whether or not the first image data to be determined is generated by an image generating apparatus that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity. Data processing that holds licensed broadcast source data for specifying a broadcast source that is permitted to broadcast image data generated using the same, and reference second image data generated by using the predetermined image generating device A data processing method performed by the device,
A first step of detecting a correlation between the first image data broadcasted by a predetermined broadcast source and the second image data held by the data processing device;
A second step of determining whether or not the first image data is generated using the image generation device, based on the correlation detected in the first step;
If it is determined in the second step that the first image data is generated using the image generation device, the first image data is generated based on the licensed broadcast source data held by the data processing device. A third step of determining whether or not the license has been granted to the broadcast source that has broadcasted the first image data. A data processing device that searches for image data generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity,
Correlation detection means for detecting a correlation between the plurality of first image data stored in the storage means and reference second image data generated using the predetermined image generation device;
Searching means for searching for the image data generated by using the image generating apparatus among the plurality of first image data stored in the storage means, based on the correlation detected by the correlation detecting means; and A data processing device having: A data processing method performed by a data processing device that searches for image data generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity,
A first step of detecting a correlation between the plurality of first image data stored in the storage unit and the second image data for reference generated using the predetermined image generation device;
A second step of searching the plurality of first image data stored in the storage unit for image data generated using the image generation device, based on the correlation detected in the first step. And a data processing method comprising: A data processing device that determines whether or not the image data is generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity,
For each of the plurality of first block image data constituting the first image data to be determined, the first block image data and the plurality of second block data constituting the second image data generated by the image generating apparatus are provided. Correlation detection means for detecting a correlation between the second block image data and the second block image data corresponding to the first block image data;
Determining means for determining whether or not the first image data has been generated based on image data generated and generated by using the image generation device based on the correlation detected by the correlation detection means. Data processing device. 20. The data processing device according to claim 19, wherein the determination unit performs the determination based on the distribution of the correlation detected by the correlation detection unit for the plurality of first block image data. The determination unit determines the number of the first block image data in which the correlation detected by the correlation detection unit exceeds a predetermined level among the plurality of first block image data, and the predetermined level. 20. The data processing device according to claim 19, wherein the determination is performed based on a number of the first block image data that does not exceed the number. A data processing method for determining whether or not image data is generated by a predetermined image generating apparatus that generates image data based on a light receiving result of a light receiving element having a variation inherent in light receiving sensitivity,
For each of the plurality of first block image data constituting the first image data to be determined, the first block image data and the plurality of second block data constituting the second image data generated by the image generating apparatus are provided. A first step of detecting a correlation between the second block image data and the second block image data corresponding to the first block image data;
A second step of determining whether or not the first image data has been generated based on the image data generated and generated by using the image generation device based on the correlation detected in the first step A data processing method comprising: A data processing device that determines whether or not the image data is generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity,
Correlation detection means for detecting a correlation between the first image data to be determined and the second image data for reference generated by the image generation device;
A determination unit configured to determine whether the first image data is generated using the image generation device based on the correlation detected by the correlation detection unit,
The correlation detecting means,
Conversion means for orthogonally transforming the first image data and the second image data to generate first frequency component data and second frequency component data, respectively;
The first frequency component data is generated based on each complex number data constituting the first frequency component data such that the first frequency component data generated by the conversion means is subjected to a stronger amplitude limitation than the second frequency component data. Computing means for generating one complex data and generating second complex data based on each complex data constituting the second frequency component data;
Replacement means for generating third complex number data in which each complex number data constituting one of the first complex number data and the second complex number data generated by the arithmetic means is replaced with complex conjugate complex number data;
Fourth complex data is generated by multiplying the first complex data or the second complex data that has not been replaced by the replacement means and the third complex data generated by the replacement means. Multiplying means,
An inverse transform circuit for inversely transforming the fourth complex data generated by the multiplying means to generate correlation data indicating the correlation. The arithmetic means divides each complex number data forming the second frequency component data by an absolute value of the complex number data to generate second complex number data, and generates each of the first frequency component data. Of the complex number data is a value obtained by performing a predetermined arithmetic processing so that the first frequency component data is strongly subjected to an amplitude limitation with respect to the absolute value of the complex number data as compared with the second frequency component data. The data processing apparatus according to claim 23, wherein the first complex data is generated by performing division. The arithmetic means generates the second complex data by dividing each complex data constituting the second frequency component data by a value obtained by raising the absolute value of the complex data to the k1 power. 24. The data processing device according to claim 23, wherein the first complex data is generated by dividing each complex data constituting the frequency component data by a value obtained by multiplying the absolute value of the complex data by k2 (k2> k1). . The conversion means multiplies the first image data and the second image data by a predetermined conversion coefficient to generate the first frequency component data and the second frequency component data, respectively,
The calculation means may further include a process of setting the second frequency component data defined based on the conversion coefficient to zero the conversion coefficient corresponding to a predetermined frequency band on a low frequency side with respect to a predetermined frequency. The data processing device according to claim 23. The said calculation means further performs the process which makes the said conversion coefficient corresponding to the predetermined frequency of the horizontal direction a high frequency side with respect to a predetermined frequency to the said 2nd frequency component data zero. Data processing equipment. A data processing device for registering image data generated by a predetermined image generating device that generates image data based on a light receiving result of a light receiving element having a specific variation in light receiving sensitivity as image data for provision,
An interface for inputting a registration request including first image data to be determined and user identification data;
Storage means for storing the user identification data and the second image data for reference in association with each other;
The second image data is read from the storage unit based on the user identification data included in the registration request input via the interface, and the read second image data and the second image data included in the registration request are read out. Correlation detection means for detecting a correlation with the first image data;
A determination unit configured to determine whether to register the first image data as the providing image data based on the correlation detected by the correlation detection unit.

Images