JP2012010170A - Imaging apparatus, image processing device, and, image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce luminance moire.SOLUTION: An imaging device 101 has a color pattern of a periodical array with a pixel block of a predetermined size as a unit array. A pixel adding section 102 defines photoelectric conversion information outputted from the imaging device 101 as a source image and adds pixel data of M×N ( wherein M, N are natural numbers) pixels in the same color constituting the source image to produce one piece of pixel data, thereby producing a decremental image having 1/(M×N) number of pixels of the source image. M×N pixels to which pixel data are added are changed for each frame in such a way as to be the same pixels, respectively, each time they are changed L times (wherein L is a natural number being equal to or larger than 2).

Description

  The present invention relates to an imaging apparatus, an image processing apparatus, and an image processing method capable of reducing luminance moire.

  At present, due to improvements in electronic device technology, high-density imaging devices such as CMOS image sensors and CCD image sensors have been realized. For this reason, in an imaging device using such an imaging device, it is a problem to suppress an increase in the burden of digital signal processing. Here, when a reduced image is generated by simply decimating pixels from the original image output from the image sensor, image quality degradation due to luminance moire occurs. Therefore, a technique for suppressing an increase in the burden of digital signal processing while suppressing luminance moiré is desired.

For example, Patent Document 1 discloses an imaging device that generates a reduced image (where N is a natural number) having a pixel density of 1 / (2N + 1) ² of an original image output from an imaging device. Yes. In the imaging device disclosed in Patent Document 1, (2N + 1) ^two pieces of information of the same color pixel in the original image are added so that the generation positions of the pixels constituting the reduced image are equally spaced, and the reduced image Information of one pixel that constitutes is generated.

Japanese Patent Laid-Open No. 2003-230054

However, the imaging device disclosed in Patent Document 1 generates a reduced image (where N is a natural number) having a pixel density of 1 / (2N + 1) ² of the original image from the original image output from the image sensor. Only applicable. There is a demand for an imaging apparatus capable of suppressing luminance moiré even when generating reduced images with various pixel densities.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging device, an image processing device, and an image processing method capable of reducing luminance moire.

In order to achieve the above object, an imaging apparatus according to the first aspect of the present invention provides:
An image sensor having a periodic pattern of color patterns in which pixel blocks of a predetermined size are arranged in units;
Using the photoelectric conversion information output from the image sensor as an original image, the pixel data of M × N (M and N are natural numbers) pixels of the same color constituting the original image are added to obtain one pixel data. A reduced image generating means for generating a reduced image having a 1 / (M × N) number of pixels of the original image by generating,
Each of the M × N pixels to which the pixel data is added is changed for each frame so as to be the same pixel every time L times (where L is a natural number of 2 or more),
It is characterized by that.

In order to achieve the above object, an imaging apparatus according to the second aspect of the present invention provides:
An image sensor having a periodic pattern of color patterns in which pixel blocks of a predetermined size are arranged in units;
By using photoelectric conversion information output from the image sensor as an original image, among all the pixels constituting the original image, 1 / (M × N) pixels of the number of the pixels are sampled and thinned out. A reduced image generating means for generating a reduced image having 1 / (M × N) pixels of the original image,
Each of the sampled pixels is changed for each frame so as to be the same pixel every time L is changed (where L is a natural number of 2 or more).
It is characterized by that.

In order to achieve the above object, an image processing apparatus according to the third aspect of the present invention provides:
Photoelectric conversion information output from an image sensor having a color pattern of a periodic arrangement having pixel blocks of a predetermined size as a unit arrangement is used as an original image, and M × N of the same color constituting the original image (however, M, N A subtractive image generating means for generating a subtracted image having a 1 / (M × N) number of pixels of the original image by adding pixel data of pixels of a natural number) to generate one pixel data. ,
Each of the M × N pixels to which the pixel data is added is changed for each frame so as to be the same pixel every time L times (where L is a natural number of 2 or more),
It is characterized by that.

In order to achieve the above object, an image processing method according to a fourth aspect of the present invention includes:
Photoelectric conversion information output from an image sensor having a color pattern of a periodic arrangement having pixel blocks of a predetermined size as a unit arrangement is used as an original image, and M × N of the same color constituting the original image (however, M, N A subtractive image generating step of generating a subtracted image having a 1 / (M × N) number of pixels of the original image by adding pixel data of pixels of (natural number) to generate one pixel data. ,
Each of the M × N pixels to which the pixel data is added is changed for each frame so as to be the same pixel every time L times (where L is a natural number of 2 or more),
It is characterized by that.

  According to the present invention, it is possible to provide an imaging device, an image processing device, and an image processing method capable of reducing luminance moire.

It is a block diagram of the imaging device which concerns on embodiment of this invention. It is a block diagram of the principal part of the imaging device shown in FIG. It is a figure which shows a Bayer arrangement | sequence. It is a figure which shows arrangement | positioning of a pixel. (A) is a figure which shows the position of the pixel used as a sample. (B) is a diagram schematically showing a reduced image. (A) is a figure which shows the position of the pixel used as a sample. (B) is a diagram schematically showing a reduced image. It is a figure for demonstrating the process performed in each frame period. It is a figure which shows the weighted average process in an even-numbered frame. It is a flowchart which shows the image display process which the imaging device which concerns on embodiment of this invention performs. (A), (B) is a figure which shows the position of the pixel used as a sample. (A)-(H) are figures which show the position of the pixel used as a sample.

  FIG. 1 is a configuration diagram of an imaging apparatus 1000 according to the present embodiment. The imaging apparatus 1000 includes an image sensor 100, a color processing unit 103, an image buffer 104, a weighted average processing unit 105, a motion amount detection unit 106, a low-pass filter 107, a recording medium 108, and an LCD (Liquid Crystal Display) 109.

  The image sensor 100 forms an image of light emitted from an object to be imaged on a light receiving plane of the image sensor 101 by an optical system such as a lens, photoelectrically converts light and darkness of the image light into an amount of electric charge, and sequentially reads out the light. Convert to electrical signal. The image sensor 100 includes an image sensor 101 and a pixel adder 102.

  The imaging element 101 is a semiconductor element such as a CCD image sensor (Charge Coupled Device Image Sensor) or a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor). The imaging element 101 receives light emitted from an imaging target through three color filters of R, G, and B arranged in a periodic array, and performs photoelectric conversion. The image sensor 101 supplies an original image represented by an electrical signal obtained by photoelectric conversion to the pixel adder 102.

  The pixel addition unit 102 generates a reduced image based on the original image supplied from the image sensor 101. Processing executed by the pixel addition unit 102 will be described later.

  The color processing unit 103 performs color processing on the subtracted image generated by the pixel adding unit 102, and converts the subtracted image into a color processed image in which information on luminance and color difference is present in all pixels. The color processing unit 103 stores the generated color processed image in the image buffer 105.

  Every time a color processing image for one frame is generated by the color processing unit 103, the generated color processing image is accumulated in the image buffer 104.

  The weighted average processing unit 105 stores the pixel data (luminance data) of the pixels constituting the color-processed image of the previous frame stored in the image buffer 104 and the color-processed image of the current frame supplied from the color processing unit 103. A weighted average image is generated by taking a weighted average of pixel data of constituent pixels. Details of the method of generating the weighted average image will be described later.

  The motion amount detection unit 106 detects the motion amount based on the color-processed image of the previous frame and the color-processed image of the current frame stored in the image buffer 104.

  The low-pass filter 107 removes high frequency components from the weighted average image supplied from the weighted average processing unit 105, and generates a weighted average image from which the high frequency components have been removed. Note that filter characteristics such as a cut-off frequency are determined based on the amount of motion detected by the motion amount detector 106.

  A weighted average image supplied from the low-pass filter 107 is recorded on the recording medium 108.

  On the LCD 109, the weighted average image supplied from the low-pass filter 107 is displayed in real time.

  Here, the configuration of the main part of the imaging apparatus 1000 will be described. As shown in FIG. 2, the main unit 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a flash memory 204, a display control unit 205, and an operation unit 206. . Each component included in the main part 200 is connected to each other via a bus 210.

  The CPU 201 controls the overall operation of the imaging apparatus 1000 according to a program stored in the ROM 202. The CPU 201 exchanges control signals and data with each component connected via the bus.

  The ROM 202 stores programs executed by the CPU 201 and various data.

  The RAM 203 temporarily stores data and programs. The RAM 13 temporarily stores programs and data read from the ROM 202.

  The flash memory 204 stores an image generated by image processing executed by the CPU 201.

  The display control unit 205 displays an image generated by image processing on the LCD 109 in cooperation with the CPU 201.

  The operation unit 206 includes a keyboard, cursor keys, numeric keys, and the like. The operation unit 206 receives an operation from the user and supplies an operation signal to the CPU 201.

  The bus 210 connects the components included in the imaging apparatus 1000 to each other, and mediates data exchange between the components.

  The color processing unit 103, the weighted average processing unit 105, the motion detection unit 106, and the low-pass filter 107 are configured by, for example, a CPU 201, a ROM 202, and a RAM 203. Further, the image buffer 104 is configured by, for example, the RAM 203. The recording medium 108 is configured by a flash memory 204, for example.

  Here, the operation of the imaging apparatus 1000 will be described with reference to the drawings.

  First, the Bayer array will be described with reference to FIG.

  As shown in FIG. 3, the color filters included in the image sensor 101 are arranged in a color pattern of a periodic array having 2 × 2 = 4 pixels as a unit array. The arrangement shown in FIG. 3 is called a Bayer arrangement. In FIG. 3, a pixel group composed of four pixels constituting one arrangement unit is shown surrounded by a frame. In the present embodiment, description will be made assuming that the original image is composed of 10 columns × 10 rows = 100 pixels.

  In the general image sensor 100, when reading out electric charges from the image sensor 101, a method of reading out each pixel as it is from each image sensor 101 (hereinafter referred to as “all pixel readout”) and a method of reading out by reducing the number of pixels. (Hereinafter referred to as “decrease reading”) can be selected.

  For example, when capturing a still image, it is effective to capture a high-quality still image by selecting all pixel readout. On the other hand, when capturing moving images in power-limited environments such as mobile applications, it is effective to operate the image capturing apparatus 1000 with low power consumption by reducing the load of digital signal processing by selecting reduced reading. It is. Here, thinning readout and pixel addition readout are known as subtractive readout methods. The thinning readout is a method of reading out only predetermined pixels, that is, a method of thinning out and reading out pixels. In the present embodiment, description will be made assuming that pixel addition reading of 2 × 2 = 4 pixels is employed.

  Hereinafter, pixel addition reading will be described with reference to the drawings.

  FIG. 4 schematically shows pixel data (pixel information and luminance data) obtained by each image sensor 101. Note that the original image is represented by the pixel data. In FIG. 4, the pixel data of the pixel in the nth row and mth column in the original image is denoted as Xnm. Here, in addition reading of 2 × 2 = 4 pixels, four pixels of the same color are added and output as one pixel data, so pixel data having a number of pixels that is 1/4 of the total number of pixels of the original image is read. It is.

  Here, it is assumed that pixel data obtained by adding pixel data of four pixels of the same color adjacent in the oblique direction of the sampled pixel in the original image is read as pixel data of the sampled pixel. For example, if the pixel in the n-th row and m-th column is sampled in the original image and the pixel data corresponding to the pixel is Anm, Anm = X (n−1) (m−1) + X (n−1) (m + 1) ) + X (n + 1) (m−1) + X (n + 1) (m + 1). For example, A11 = X00 + X02 + X20 + X22. FIG. 5A shows the position of the sampled pixel. On the other hand, FIG. 5B shows a reduced image generated by reducing the interval between sampled pixels.

  On the other hand, in addition to generating a reduced image at the phase shown in FIG. 5A (hereinafter referred to as “phase A”) (generating a reduced image using the pixel at the position shown in FIG. 5A as a sample), FIG. A reduced image can be generated with the phase shown in FIG. 6A (hereinafter referred to as “phase B”) (a reduced image is generated using the pixel at the position shown in FIG. 6A as a sample).

  Note that, even when a pixel is sampled as shown in FIG. 6A, if the pixel in the n-th row and m-th column is sampled in the original image and the pixel data corresponding to the pixel is Anm, Anm = X (n -1) (m-1) + X (n-1) (m + 1) + X (n + 1) (m-1) + X (n + 1) (m + 1). For example, A33 = X22 + X24 + X42 + X44. FIG. 6B shows a reduced image generated when the pixel shown in FIG. 6A is used as a sample.

  Comparing FIG. 5A and FIG. 6A, the position of the pixel to be sampled is shifted in an oblique direction, and the phase is exactly reversed (hereinafter referred to as “reverse phase”).

  Here, when a zone plate chart (CZP) is photographed, an image is displayed on the LCD 109 and image quality evaluation is performed, a false signal is generated by pixel addition reading, and the false signal appears as a luminance moire. Specifically, a reduced image obtained when the sample position is set to the sample position shown in FIG. 5A, and a reduced image obtained when the sample position is set to the sample position shown in FIG. , The true signal component appears at a position where the black and white patterns coincide with each other (same phase), and the false signal component appears at a position where the black and white patterns reverse to each other (reverse phase).

  Thus, in phase A and phase B, the positions where the false signal components appear on the generated reduced image are in opposite phases. Therefore, it is possible to reduce the false signal component by reversing the phase when generating the reduced image between the even frame and the odd frame and superimposing the reduced image of the previous frame on the current reduced image.

  With reference to FIG. 7, how the imaging apparatus 1000 takes a weighted average at each timing will be described. FIG. 7 shows data output from the image sensor 100 in each frame period, data written to the image buffer 104 in each frame period, data read from the image buffer 104 in each frame period, and recording in each frame period. Data recorded on the medium 108 or data displayed on the LCD 109 is shown.

  Note that pixel addition reading is used for reading from the image sensor 100, and sampling is performed at phase A for even frames and phase B for odd frames. The output from the image sensor 100 is written into the image buffer 104 for one frame, and is read from the image buffer 104 in the next frame period. As shown in FIG. 7, the phase A image and the phase B image are weighted averaged and output in any frame by the weighted average processing.

  The processing of the imaging apparatus 1000 will be described by taking as an example the case where the frame number is 2 (hereinafter, “the frame with the frame number t” is appropriately referred to as “frame t”). In the frame period of frame 2, A (2) is output from the image sensor 100, and A (2) is written into the image buffer 104. In this frame period, B (1) recorded in the image buffer 104 in the frame period of frame 1 is read from the image buffer 104. Then, αA (2) + βB (1) is recorded on the recording medium 108 or displayed on the LCD 109. As described above, the data obtained for the current frame and the data obtained for the previous frame are weighted and added together, thereby reducing the luminance moire.

  Here, a calculation method of the weighted average for each pixel in the even frame will be described with reference to FIG.

The weighted average is calculated by the following equation (1).
Weighted average image (i, j) = α × color-processed image of current frame (i, j) + β × color-processed image of previous frame (i, j) (1)
i: horizontal coordinate in color processed image j: vertical coordinate in color processed image α: weight for current frame β: weight for previous frame

  The weight value is calculated when the weighted average is not taken (α = 1.0, β = 0.0), or when the average value is taken (α = 0.5, β = 0.5). There are cases (α = 0.8, β = 0.2).

  When the weighted average is not taken, a color processed image whose phase is reversed every frame is displayed and recorded as it is. In this case, two color-processed images having different phases depending on the afterimage of the eyes are averaged and observed, so that observation with good image quality with less luminance moire can be realized.

  When taking the average value, it is possible to realize observation with good image quality with less luminance moire without using afterimages of eyes.

  In the case of taking a weighted average, for example, it is easy to follow the motion by placing weight on the current frame (for example, α = 0.8, β = 0.2). Further, in this case, as in the case where the weighted average is not taken, the luminance moire can be reduced using the afterimage of the eyes.

  FIG. 8 shows a calculation method of the weighted average for each pixel in the even frame. In the case of an odd-numbered frame, the values of α and β are kept as they are, and the calculation is performed using a formula in which α and β in the calculation formula for each pixel value in FIG. In this way, the false signal of phase A and the false signal of phase B are canceled out by the weighted average, so that luminance moire can be reduced and good image quality with less jaggies can be obtained.

  Here, the role of the low-pass filter 107 will be described. When the zone plate chart is imaged, the ratio of the false signal component is high near the Nyquist frequency, and the luminance moire is conspicuous. In pixel addition readout, since the weighted average process serves as a low-pass filter, luminance moire appearing in the vicinity of the Nyquist frequency can be reduced to some extent, but it is not sufficient. Therefore, it is preferable to reduce the luminance moire by introducing a low-pass filter 107 whose cutoff frequency is a frequency slightly lower than the Nyquist frequency.

  Further, when a moving image is captured, if the movement of the imaging target is large, the weighted average image is blurred by weighted average processing or a low-pass filter. Therefore, it is preferable to detect the motion amount between frames for each region, and weaken the weighted average processing and the low-pass filter in a region where the motion is small. For example, the motion amount detection unit 106 performs SAD (Sum of Absolute Difference) for each region of the same size and position from the color processed image of the current frame and the color processed image recorded in the previous frame. Calculate On the other hand, the weighted average processing unit 105 changes the weights α and β according to the value of SAD. For example, the weighted average processing unit 105 increases the weight α and decreases the weight β as the SAD value increases. The low-pass filter 107 changes the cutoff frequency according to the SAD value. For example, the low-pass filter 107 sets the cutoff frequency higher as the SAD value is larger.

  Next, an image display process executed by the imaging apparatus 1000 according to the present embodiment will be described with reference to FIG. Note that the image capturing apparatus 1000 repeatedly executes the image display process illustrated in FIG. 9 while capturing a moving image.

  First, the CPU 201 images a subject (step S101). For example, the CPU 201 gives the image sensor 100 a trigger to start moving image capturing. On the other hand, in response to the trigger, the image sensor 101 starts imaging of a subject and acquires pixel data of all pixels constituting the original image.

  When completing the process in step S101, the CPU 201 determines whether or not the current frame is an even frame (step S102). The frame is the original image acquired in step S101.

  If the CPU 201 determines that the current frame is an even frame (step S102: YES), the CPU 201 executes pixel addition for the even frame (step S103). The pixel addition in the even-numbered frame is, for example, pixel addition for obtaining a reduced image of phase A shown in FIG.

  On the other hand, when the CPU 201 determines that the current frame is not an even frame (step S102: NO), the CPU 201 executes pixel addition for an odd frame (step S104). The pixel addition in the odd-numbered frame is, for example, pixel addition for obtaining a reduced image having a phase B shown in FIG.

  When the CPU 201 completes the process of step S103 or step S104, the CPU 201 generates a color processed image (step S105). The color processed image is an image generated based on the reduced image obtained by the process of step S103 or step S104, and is an image in which information on luminance and color difference is present in all pixels. The generated color processed image is stored in the flash memory 204, for example.

  When the CPU 201 completes the process of step S105, it detects the amount of movement (step S106). The amount of motion can be the sum of the differences obtained by taking the luminance difference between the previous frame and the current frame for all the pixels constituting the color processed image. The amount of motion may be obtained for each block constituting the color processed image. In the present embodiment, description will be made assuming that one motion amount is obtained from the entire color processed image.

  When completing the process in step S106, the CPU 201 calculates a weight based on the amount of motion detected in step S106 (step S107). For example, as the amount of motion is larger, the weight α is set to a larger value and the weight β is set to a smaller value. Note that α + β = 1.

  When completing the process in step S107, the CPU 201 generates a weighted average image based on the weight calculated in step S107 (step S108). The weighted average image is obtained by multiplying the pixel data of the pixels constituting the color processed image of the current frame by the weight α, and multiplying the pixel data of the pixels constituting the color processed image of the previous frame by the weight β. It is an image composed of pixels having pixel data obtained by adding things. The generated weighted average image is stored in the flash memory 204, for example.

  When completing the process in step S108, the CPU 201 sets a filter coefficient based on the amount of motion detected in step S106 (step S109). For example, the filter coefficient is set so that the cut-off frequency increases as the amount of motion increases.

  When the CPU 201 finishes the process of step S109, the CPU 201 executes the filter process on the weighted average image generated in step S108 based on the filter coefficient set in step S108 (step S110).

  CPU201 displays the weighted average image to which the filter process was performed in step S110, after complete | finishing the process of step S110 (step S111). For example, the weighted average image is displayed on the LCD 109.

  When the CPU 201 ends the process of step S111, the process returns to step S101.

  As described above, according to the imaging apparatus 1000 according to the present embodiment, it is possible to reduce luminance moiré.

  Further, in the present embodiment, a reduced image having an opposite phase is generated in the even frame and the odd frame. For this reason, luminance moiré is reduced by canceling the false signal.

  In this embodiment, the high-frequency component of the color processed image is attenuated by the low-pass filter 107. For this reason, it is possible to reduce luminance moiré due to high frequency components.

  In the present embodiment, the filter characteristic of the low-pass filter 107 is set based on the motion amount detected by the motion amount detection unit 106. For this reason, appropriate filtering according to the amount of change in luminance of an image obtained by imaging can be performed.

  In this embodiment, a weighted average image is generated by taking a weighted average of the color processed image of the frame acquired in the past and the color processed image of the current frame for each pixel. For this reason, luminance moiré is reduced in one weighted average image without using an afterimage of the eye.

  In the present embodiment, a weighted average weight is set based on the motion amount detected by the motion amount detection unit 106. Therefore, an appropriate weighted average image corresponding to the amount of change in luminance of the image obtained by imaging can be obtained.

  In this embodiment, the pixel block is composed of 2 × 2 4 pixels. As described above, even when the image sensor 101 has an arrangement pattern in which 4 × 2 × 2 pixels are arranged as a unit, appropriate pixels arranged periodically are sampled, so that luminance moire can be reduced. .

(Modification)
The present invention is not limited to the one disclosed in the above embodiment.

  In the above embodiment, an example in which a reduced image is generated from an original image by pixel addition reading has been described. However, a reduced image may be generated from the original image by pixel thinning readout. In this case, for example, pixel data of the pixel at the position shown in FIG. 5A or FIG. 6A is sampled, and a subtracted image composed of the sampled pixel data is generated. Thus, even when a reduced image is generated from the original image by pixel thinning-out readout, the false signal is canceled out, so that the luminance moiré can be reduced.

  In the above-described embodiment, the present invention has been described by taking the reduction reading of 2 × 2 pixels as an example. However, in the present invention, subtractive readout of M × N pixels (where M and N are natural numbers) can be employed. Here, when M = 1 or N = 1, it means that the number of output pixels from the image sensor is reduced only in the vertical direction or the horizontal direction, respectively. In subtractive readout of M × N pixels, only pixel data corresponding to the center 2 × 2 pixels are sampled from the M × N pixels, remain, and are output as a subtracted image with the pixel intervals reduced. When obtaining an inverse phase reduced image, the sample position is shifted by M pixels horizontally and N pixels vertically. In this way, by obtaining subtracted images that have opposite phases, and applying the above-described method, even with subtractive readout of M × N pixels, the sensitivity improvement effect, the digital signal processing load reduction effect, and the luminance moiré reduction effect Can be obtained.

  For example, in the case of 4 × 2 pixel (M = 4, N = 2) subtractive readout, the position of the sample pixel is the position shown in FIGS. 10 (A) and 10 (B). Here, the phase shown in FIG. 10A is opposite to the phase shown in FIG. 10B. In the case of thinning readout, for example, A13 = X13, A14 = X14, A23 = X23, and A24 = X24. On the other hand, in the case of addition reading, for example, A13 = X00 + X02 + X04 + X06 + X20 + X22 + X24 + X26, A14 = X01 + X03 + X05 + X07 + X21 + X23 + X25 + X27, A23 = X10 + X12 + X14 + X16 + X30 + X32 + X34 + X36 + X + That is, in addition reading, pixel data of 4 × 2 pixels (M × N pixels) of the same color in the vicinity of the sample pixel are added, and the addition result is handled as pixel data of the sample pixel. When addition reading is executed in this way, pixel data of any pixel is used for addition only once.

  Further, instead of reversing the phase (shifting the phase by 180 °) as the frame is switched, it may be shifted by a predetermined angle as the frame is switched. Further, both the horizontal and vertical phases may be shifted independently without shifting simultaneously. If the horizontal and vertical phases are shifted independently by 90 ° with reference to the image shown in FIG. 10A, the generated image is, for example, the image shown in FIG. Image shown in (C) → Image shown in FIG. 11 (D) → Image shown in FIG. 11 (E) → Image shown in FIG. 11 (F) → Image shown in FIG. 10 (B) → Shown in FIG. 11 (G) The image changes to the image shown in FIG. Here, shifting the horizontal and vertical phases by φx and φy, respectively, is represented as (φx, φy). In this case, the horizontal and vertical phases are (0 °, 0 °) → (90 °, 0 ° ) → (180 °, 0 °) → (270 °, 0 °) → (270 °, 180 °) → (180 °, 180 °) → (90 °, 180 °) → (0 °, 180 °) Has changed. Further, when the horizontal and vertical phases are shifted independently by 180 ° with reference to the image shown in FIG. 10A, the generated image is, for example, the image shown in FIG. The image changes as follows: image shown in (A) → image shown in FIG. 10 (B) → image shown in FIG. 11 (B). In this case, the horizontal / vertical phase changes in the order of (0 °, 0 °) → (180 °, 0 °) → (180 °, 180 °) → (0 °, 180 °). As described above, when the frame number increases, the phase of the reduced image to be generated changes by a predetermined angle, thereby reducing the luminance moire.

  Note that the imaging apparatus according to the present invention can also be realized using a normal computer system, not a dedicated system. For example, a computer readable recording such as a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), or an MO (Magneto Optical Disk) is recorded on a computer. An image pickup apparatus that performs the above-described processing may be configured by storing and distributing in a medium and installing it in a computer system.

  Furthermore, the program may be stored in a disk device or the like included in a server device on the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

  (Supplementary note 1) An image sensor having a periodic pattern of color patterns having a pixel block of a predetermined size as a unit array, and photoelectric conversion information output from the image sensor as an original image, the same color M constituting the original image By adding pixel data of × N pixels (where M and N are natural numbers) to generate one pixel data, a reduced image having 1 / (M × N) number of pixels of the original image is obtained. The M × N pixels to which the pixel data is added are each the same pixel every time they are changed L times (where L is a natural number of 2 or more). In addition, the imaging apparatus is changed for each frame.

  (Appendix 2) An image sensor having a periodic pattern of color patterns having a pixel block of a predetermined size as a unit array, and photoelectric conversion information output from the image sensor as an original image, all pixels constituting the original image Of these, a reduced image that generates a reduced image having 1 / (M × N) pixels of the original image by sampling and thinning out 1 / (M × N) pixels of the total number of pixels. Generating means, wherein each of the sampled pixels is changed for each frame so as to become the same pixel every time L is changed (where L is a natural number of 2 or more). An imaging device.

  (Supplementary note 3) The imaging apparatus according to supplementary note 1 or 2, wherein the L is 2, and the reduced image generation unit generates a reduced image having a phase opposite to that of the previous frame.

  (Supplementary note 4) The imaging according to any one of supplementary notes 1 to 3, further comprising low-pass filter means for attenuating a high-frequency component of a color processed image generated based on the reduced image. apparatus.

  (Additional remark 5) It further has a movement amount detection means for detecting a movement amount of a subject based on a reduced image generated for the current frame and a reduced image generated for a past frame, and the low-pass filter means includes: The imaging apparatus according to appendix 4, wherein a filter characteristic of a low-pass filter is set based on the detected amount of motion.

  (Appendix 6) Image buffer means for storing a color processed image generated based on the reduced image, the latest (L-1) color processed images stored in the image buffer means, and the current frame Any one of appendices 1 to 5, further comprising weighted average image generation means for generating a weighted average image by taking a weighted average for each pixel of the color processed image generated for The imaging device according to item.

  (Additional remark 7) It further has a motion amount detection means for detecting a motion amount of a subject based on a reduced image generated for the current frame and a reduced image generated for a past frame, and the weighted average image generating means includes The imaging apparatus according to appendix 6, wherein the weighted average weight is set based on the detected amount of motion.

  (Supplementary note 8) The imaging device according to any one of supplementary notes 1 to 7, wherein the pixel block includes 2 × 2 pixels.

  (Supplementary Note 9) Using photoelectric conversion information output from an image sensor having a color pattern with a periodic arrangement having a pixel block of a predetermined size as a unit arrangement as an original image, M × N pixels of the same color constituting the original image (however, , M and N are natural numbers). By generating pixel data by generating pixel data, a reduced image generation that generates a reduced image having 1 / (M × N) pixels of the original image The M × N pixels to which the pixel data is added are changed for each frame so that the same pixel is obtained every L times (where L is a natural number of 2 or more). An image processing apparatus characterized by that.

  (Supplementary Note 10) Using photoelectric conversion information output from an image pickup element having a periodic pattern of color patterns having a pixel block of a predetermined size as a unit array as an original image, M × N of the same color constituting the original image (however, , M and N are natural numbers). By generating pixel data by generating pixel data, a reduced image generation that generates a reduced image having 1 / (M × N) pixels of the original image Each of the M × N pixels to which the pixel data is added is changed for each frame so as to become the same pixel every time L times (where L is a natural number of 2 or more). An image processing method characterized by that.

DESCRIPTION OF SYMBOLS 100 ... Image sensor, 101 ... Image sensor, 102 ... Pixel addition part, 103 ... Color processing part, 104 ... Image buffer, 105 ... Weighted average processing part, 106 ... Motion amount detection unit 107... Low-pass filter 108... Recording medium 109. LCD 200 200 main part 201 CPU CPU 202 ROM 203 203 RAM, 204... Flash memory, 205... Display control unit, 206... Operation unit, 210.

Claims (10)

An image sensor having a periodic pattern of color patterns in which pixel blocks of a predetermined size are arranged in units;
Using the photoelectric conversion information output from the image sensor as an original image, the pixel data of M × N (M and N are natural numbers) pixels of the same color constituting the original image are added to obtain one pixel data. A reduced image generating means for generating a reduced image having a 1 / (M × N) number of pixels of the original image by generating,
Each of the M × N pixels to which the pixel data is added is changed for each frame so as to be the same pixel every time L times (where L is a natural number of 2 or more),
An imaging apparatus characterized by that. An image sensor having a periodic pattern of color patterns in which pixel blocks of a predetermined size are arranged in units;
By using photoelectric conversion information output from the image sensor as an original image, among all the pixels constituting the original image, 1 / (M × N) pixels of the number of the pixels are sampled and thinned out. A reduced image generating means for generating a reduced image having 1 / (M × N) pixels of the original image,
Each of the sampled pixels is changed for each frame so as to be the same pixel every time L is changed (where L is a natural number of 2 or more).
An imaging apparatus characterized by that. L is 2;
The reduced image generating means generates a reduced image having a phase opposite to that of the previous frame.
The imaging apparatus according to claim 1 or 2, wherein Low-pass filter means for attenuating high-frequency components of the color processed image generated based on the reduced image,
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. A motion amount detecting means for detecting a motion amount of the subject based on the reduced image generated for the current frame and the reduced image generated for the past frame;
The low-pass filter means sets a filter characteristic of the low-pass filter based on the detected amount of motion;
The imaging apparatus according to claim 4. Image buffer means for storing a color processed image generated based on the reduced image;
The weighted average image is obtained by taking the weighted average of the latest (L-1) color processed images stored in the image buffer means and the color processed image generated for the current frame for each pixel. A weighted average image generating means for generating,
The imaging apparatus according to any one of claims 1 to 5, wherein A motion amount detecting means for detecting a motion amount of the subject based on the reduced image generated for the current frame and the reduced image generated for the past frame;
The weighted average image generating means sets the weighted average weight based on the detected amount of motion;
The imaging apparatus according to claim 6. The pixel block is composed of 2 × 2 4 pixels.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. Photoelectric conversion information output from an image sensor having a color pattern of a periodic arrangement having pixel blocks of a predetermined size as a unit arrangement is used as an original image, and M × N of the same color constituting the original image (however, M, N A subtractive image generating means for generating a subtracted image having a 1 / (M × N) number of pixels of the original image by adding pixel data of pixels of a natural number) to generate one pixel data. ,
Each of the M × N pixels to which the pixel data is added is changed for each frame so as to be the same pixel every time L times (where L is a natural number of 2 or more),
An image processing apparatus. Photoelectric conversion information output from an image sensor having a color pattern of a periodic arrangement having pixel blocks of a predetermined size as a unit arrangement is used as an original image, and M × N of the same color constituting the original image (however, M, N A subtractive image generating step of generating a subtracted image having a 1 / (M × N) number of pixels of the original image by adding pixel data of pixels of (natural number) to generate one pixel data. ,
Each of the M × N pixels to which the pixel data is added is changed for each frame so as to be the same pixel every time L times (where L is a natural number of 2 or more),
An image processing method.

Images