JP2008160566A - Image processing apparatus, image processing method, and imaging system - Google Patents

Abstract

Data processing related to image processing performed for each small area is simplified by dividing a screen into small areas having a predetermined size and using linear interpolation data obtained by interpolation processing between reference points.
Data complementing is performed by linear interpolation based on reference point data between a plurality of reference points to generate linear interpolation data, and an image related to image data obtained from an image sensor using the generated linear interpolation data In an image processing apparatus that performs processing for each small region by dividing the processing into a plurality of small regions, the end point of the small region is used as a reference point for linear interpolation, and the data value obtained by calculating the image data of the small region is By using the data value of the reference point, the data processing can be simplified by making the small area for image processing equal to the complementary processing area.
[Selection] Figure 1

Description

  The present invention relates to an image processing technique for processing image data obtained from an image sensor composed of a plurality of pixels by dividing it into a plurality of small regions.

  Conventionally, as an imaging device that captures, records, and reproduces still images and moving images, a memory card having a solid-state memory element is used as a recording medium, and an electronic camera that records and reproduces still images and moving images captured by the solid-state imaging element An imaging device is known. A CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or the like is used as a solid-state imaging device of such an imaging apparatus.

  Further, when imaging using a solid-state imaging device, dark noise correction processing can be performed by performing arithmetic processing using dark image data and actual captured image data. Here, the dark image data is image data read after charge accumulation is performed in the same manner as in the main shooting without exposing the image sensor, and the main image data is stored in the state where the image sensor is exposed. This is image data read out later.

  By performing dark noise correction processing, it is possible to obtain high-quality images by correcting captured image data against image quality degradation such as dark current noise generated by the image sensor and pixel defects due to minute flaws inherent to the image sensor. Can do. In particular, since dark current noise increases with the charge accumulation time and the temperature rise of the image sensor, a large image quality improvement effect can be obtained when performing exposure at long seconds or at high temperatures.

  On the other hand, when performing arithmetic processing using dark image data, a large-capacity memory is required to perform arithmetic processing as it is for the entire screen area, which increases the cost of the imaging apparatus. Therefore, for the purpose of reducing the amount of data (memory amount) used for arithmetic processing, a method of dividing the screen into small areas having a predetermined size and performing arithmetic processing for each small area has been proposed (for example, patents). Reference 1).

  However, in the method of dividing the screen into small areas having a predetermined size and performing image processing for each small area, the step of the image data is performed at the boundary of the area in order to perform the same image processing independently for each small area. This causes a problem that image quality is likely to deteriorate. The step of the image data is particularly problematic because it appears linearly on the screen and is visually noticeable.

  In addition, for the purpose of obtaining continuity of correction data, as shown in an example in FIG. 7, the screen is divided into small areas having a predetermined size, and a process of complementing each reference point is performed using the center of gravity of the small area as a reference point. There is a way to do it. Here, FIG. 7A shows a screen 604 of the image sensor, and the screen 604 includes a horizontal optical black (OB) region 600, a vertical OB region 601, and an effective region 603. . The horizontal OB region 600 and the vertical OB region 601 are pixel regions that are shielded from light by the image sensor, and the effective region 603 is a region where light relating to the subject image is incident. On the horizontal reference axis H1 that passes through the predetermined coordinates of the vertical OB area 601, reference points A605, B606, and D corresponding to the divided subareas A, B, C, and D, respectively. C607 and a reference point D608.

However, in the case where the screen is divided into predetermined small areas, and each of the reference points is complemented with the center of gravity of the small area as a reference point, as shown in FIG. The complementary processing area is different. Therefore, there is a problem that data processing becomes complicated. In FIG. 7B, a solid line 609 indicates correction data, and a dotted line 610 indicates pixel data from the image sensor.
In addition, an imaging apparatus has been proposed in which correction data corresponding to each pixel position is generated using a central position of a small area as a reference point and an average value in the area as data of the reference point (for example, Patent Documents). 2).

JP-A-7-250259 JP 2006-222689 A

  The present invention divides a screen into small areas having a predetermined size, and simplifies data processing related to image processing performed for each small area using linear interpolation data obtained by interpolation processing between reference points. Objective.

An image processing apparatus according to the present invention is an image processing apparatus that processes image data obtained from an imaging device including a plurality of pixels by dividing the image data into a plurality of small regions, and includes a plurality of reference points between reference points. Data processing is performed for each small region using data complementing means for generating data by linear interpolation based on data and generating linearly complemented data, and linear complementation data generated by the data complementing means. Image processing means, using the end point of the small area as a reference point for linear interpolation performed by the data complementing means, and a data value obtained by calculating the image data of the small area as a data value of the reference point It is characterized by.
An image processing apparatus according to the present invention is an image processing apparatus that processes image data obtained from an imaging device including a plurality of pixels by dividing the image data into a plurality of small regions, and includes a plurality of reference points between reference points. Data processing is performed for each small region using data complementing means for generating data by linear interpolation based on data and generating linearly complemented data, and linear complementation data generated by the data complementing means. And an image processing unit, wherein the reference point is set so that a region between the reference points for which linear interpolation is performed by the data complementing unit and the small region coincide with each other.
An image processing method according to the present invention is an image processing method for processing image data obtained from an image sensor composed of a plurality of pixels by dividing the image data into a plurality of small regions. A data complementing process for complementing data by linear complementation based on data to generate linear complemented data, and using the linear complementing data generated in the data complementing process, image processing relating to the image data is performed for each small region An image processing step to be performed, wherein an end point of the small area is used as a reference point for linear interpolation, and a data value obtained by performing arithmetic processing on the image data of the small area is used as a data value of the reference point. Features.
A program according to the present invention is a program for causing a computer to execute image processing for processing image data obtained from an imaging device including a plurality of pixels by dividing the image data into a plurality of small regions, and at the end points of the small regions. A reference point setting step for setting a reference point and setting a data value obtained by calculating the image data of the small area as a data value of the reference point, and generating difference data between adjacent reference points, Differential data is generated by dividing the difference data by the number of complementary points, and the differential data is added to the data value of the reference point sequentially for each complementary point to perform data interpolation between the reference points and generate linearly complementary data. A data complementing step, and an image processing step for performing image processing on the image data for each of the small regions using the linear complementation data generated in the data complementing step. Characterized in that to execute the data.
The computer-readable recording medium of the present invention is characterized by recording a program.
The imaging system of the present invention performs preprocessing including digitization on the image processing device, an imaging device composed of a plurality of pixels, and an image signal output from the imaging device, and converts the obtained digital data into the image Preprocessing means for supplying the data to the image processing apparatus as data.

According to the present invention, the image data obtained from the image sensor composed of a plurality of pixels is divided and processed into a plurality of small regions, so that it is not necessary to hold the image data of the entire screen, and a plurality of images in the screen Since it is only necessary to hold the reference point data, the amount of memory can be reduced.
In addition, by using the end point of the small area as a reference point for linear interpolation, the small area where image processing is performed and the complementary processing area can be made equal, and data processing can be simplified compared to the conventional case. Therefore, the cost of the image processing apparatus can be reduced.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 2 is a block diagram showing a configuration example of the image processing apparatus according to the first embodiment of the present invention. The image processing apparatus 10 shown in FIG. 2 divides image data obtained from an image sensor composed of a plurality of pixels into a plurality of small areas, and calculates from data of reference points obtained based on the image data of the small areas. Image processing is performed for each small region using the linear interpolation data.

  In FIG. 2, reference numeral 200 denotes an image sensor composed of a plurality of pixels, for example, a CCD image sensor or a CMOS image sensor is used. A pixel signal output from the image sensor 200 is converted from an analog signal to digital data by an A / D conversion circuit (ADC) 201. The pixel data digitized by the ADC 201 is transferred to the image processing apparatus 10 (specifically, the pixel data storage register 202 and the reference data generation circuit 203).

  The image processing apparatus 10 includes a pixel data storage register 202, a reference data generation circuit 203, a RAM 204, a reference point data storage register B205, a reference point data storage register A206, a subtractor 207, a shifter 208, and an offset data generation circuit 209. The image processing apparatus 10 also includes a differential data storage register 210, a selector 211, an adder 212, a linear interpolation data storage register 213, and subtracters 214 and 215.

  The reference data generation circuit 203 generates reference point data relating to a predetermined small area using the input pixel data, and writes the reference point data in a RAM (Random Access Memory) 204. In this embodiment, the reference point data is a small area calculated by integrating the input pixel data after removing defective pixel data such as black scratches and white scratches due to pixel defects in the image sensor 200 from the input pixel data. The average value of Note that the reference point data is not limited to the average value of the small area, but is calculated based on the pixel data of the small area such as the maximum value, the minimum value of the small area, and pixel data of arbitrary coordinates in the small area. Any data can be used as long as it is obtained in the same manner.

  The first reference point data read from the RAM 204 is stored in the reference point data storage register A206, and the next read second reference point data is stored in the reference point data storage register B205. The subtracter 207 calculates a difference (difference data) between two adjacent reference point data stored in the reference point data storage register A 206 and the reference point data storage register B 205.

  The difference between two adjacent reference point data calculated by the subtracter 207 is right-shifted by the bit corresponding to the number of complementary points by the shifter 208 and stored in the differential data storage register 210 as differential data. For example, if the number of complementary points is 32, it is shifted by 5 bits, and if it is 256, it is shifted by 8 bits. In the present embodiment, the shifter 208 is used for the purpose of simplifying the circuit for calculating differential data, but a divider may be used instead. When a divider is used instead of the shifter 208, the number of complementary points is not limited to a power of 2, and the differential data can be obtained by dividing the difference of the reference point data by the number of complementary points with respect to an arbitrary number of complementary points. Obtainable.

  The selector 211 is supplied with the reference point data stored in the reference point data storage register A 206 and the linear interpolation data stored in the linear interpolation data storage register 213, and selectively selects the reference point data or the linear interpolation data to the adder 212. Output. The output data from the selector 211 and the differential data stored in the differential data storage register 210 are added by the adder 212 and stored in the linear interpolation data storage register 213 as linear interpolation data.

  At the starting point of linear interpolation, the adder 212 adds the reference point data stored in the reference point data storage register A206 supplied via the selector 211 and the differential data stored in the differential data storage register 210. . Except for the starting point of linear interpolation, the linear interpolation data stored in the linear interpolation data storage register 213 supplied via the selector 211 and the differential data stored in the differential data storage register 210 are added by the adder 212. The In other words, data complementation is performed by sequentially adding the differential data stored in the differential data storage register 210 to the reference point data stored in the reference point data storage register A 206 according to the number of complementary points to generate linear complementary data. Is done.

  Here, the reference point data storage register A206, the reference point data storage register B205, the subtracter 207, the shifter 208, the differential data storage register 210, the selector 211, the adder 212, and the linear interpolation data storage register 213 provide data complementing means. Composed.

  The linear interpolation data stored in the linear interpolation data storage register 213 and the pixel data stored in the pixel data storage register 202 are input to a subtracter 214 serving as an image processing unit. The subtractor 214 performs arithmetic processing for each small area obtained by dividing the linear interpolation data and the pixel data. For example, when the reference point data to be stored in the RAM 204 is generated using the pixel data of the vertical OB pixel area, the subtracter 214 can correct the vertical OB clamp.

  Further, the difference between the two adjacent reference point data calculated by the subtracter 207 is input to the offset data generation circuit 209. The offset data generation circuit 209 calculates the offset amount of the linear interpolation data by placing the reference point at the end point of the reference data calculation area (small area). Offset correction is performed by subtracting the offset amount obtained by the offset data generation circuit 209 from the output of the subtractor 214 by the subtractor 215.

  FIG. 1 is a diagram for explaining reference point data and a region for calculating the reference point data in the first embodiment.

  FIG. 1A shows a screen 104 of the image sensor. The screen 104 is composed of a horizontal OB pixel region 100, a vertical OB pixel region 101, and an effective pixel region 103. The horizontal OB pixel region 100 and the vertical OB pixel region 101 are pixel regions (light-shielding regions) that are shielded from light by the image sensor, and the effective pixel region 103 is a region where light relating to the subject image is incident.

  A reference point A105, a reference point B106, a reference point C107, and a reference point D108 are located on a horizontal reference axis H1 passing through predetermined coordinates of the vertical OB region 101. The reference points A105 to D108 are set to the end points of the areas A, B, C, and D, which are the divided small areas.

  In FIG. 1B, the pixel signal level of the horizontal reference axis H1 shown in FIG. Normally, the dark correction process in the horizontal direction can be performed by subtracting the pixel signal level of the horizontal reference axis H1 from the pixel signal level on the horizontal axis passing through the arbitrary coordinates of the effective pixel region 103.

  In the memory in the image processing apparatus according to the present embodiment, in order to reduce the amount of memory, not all of the pixel data on the horizontal reference axis H1 but calculation processing is performed from the pixel data of each of the areas A to D. The stored data value is stored as the data value of the reference point. In this embodiment, the data value stored as the data value of the reference point is the average value of the small areas (areas A to D). However, the present invention is not limited to this, and any data obtained by performing arithmetic processing from pixel data of a small area, such as the maximum value, minimum value of a small area, and pixel data of arbitrary coordinates in the small area may be used ( The same applies to the following embodiments).

  In FIG. 1B, the average value data of each region A to D is plotted as data values of the reference point A105 to the reference point D108. Each reference point is located at the left end of the corresponding region. A broken line 111 is obtained by linearly connecting the reference points to complement the data, that is, linearly approximating the reference points.

  The pixel data indicated by the dotted line 110 (or data obtained by vertically projecting the area) and the linear interpolation data indicated by the broken line 111 have offset values corresponding to difference data between adjacent reference points. Since this offset value differs depending on the difference data between adjacent reference points, it cannot be matched between all reference points. However, if the difference data between adjacent reference points is small relative to the width of the region, it is roughly Can be matched. A solid line 109 indicates correction data obtained by linearly interpolating between reference points and adding offset correction, that is, correction data obtained by performing offset correction on the linear interpolation data indicated by the broken line 111.

  In the first embodiment, since the reference point is placed at the left end of each region, correction data cannot be obtained in the data generation region of the reference point D108. Therefore, in the first embodiment, extrapolation data 112 obtained by extending the correction data between the reference points C107 and D108 to the region D is generated and added to the correction data. The extrapolated data 112 is not necessary if the region D is small enough to be unnecessary as a captured image.

As described above, according to the first embodiment, the coordinates of the reference points A105 to D108 are set to the left end of each of the areas A to D that are data generation areas of the corresponding reference points, thereby complementing the small area that performs image processing. The area to be processed becomes equal. Therefore, a process for calculating the coordinates of the reference point is not necessary, and data processing can be simplified. Further, by adding an offset correction, an error between the actual pixel data and the calculated correction data can be reduced, and the accuracy of image correction can be improved.
In addition, by processing image data by dividing it into multiple small areas, it is not necessary to hold the image data of the entire screen in the image sensor, and it is only necessary to hold multiple reference point data, thus reducing the amount of memory can do.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
In the second embodiment described below, the reference point is placed at the left end of each region in the first embodiment, and the reference point is placed at the right end of each region. Since the configuration of the image processing apparatus in the second embodiment is the same as that of the image processing apparatus in the first embodiment, the description thereof is omitted.

  FIG. 3 is a diagram for explaining reference point data and a region where the reference point data is arithmetically processed in the second embodiment.

  FIG. 3A shows a screen 304 of the image sensor. The screen 304 includes a horizontal OB pixel region 300, a vertical OB pixel region 301, and an effective pixel region 303. A reference point A305, a reference point B306, a reference point C307, and a reference point D308 are located on a horizontal reference axis H1 passing through predetermined coordinates of the vertical OB region 301. The reference points A305 to D308 are set at the right end of the areas A, B, C, and D, which are divided small areas.

  In FIG. 3B, the pixel signal level of the horizontal reference axis H1 shown in FIG. Normally, the dark correction process in the horizontal direction can be performed by subtracting the pixel signal level of the horizontal reference axis H1 from the pixel signal level on the horizontal axis passing through the arbitrary coordinates of the effective pixel region 303.

  In the memory in the image processing apparatus according to the present embodiment, in order to reduce the amount of memory, not all of the pixel data on the horizontal reference axis H1 but calculation processing is performed from the pixel data of each of the areas A to D. The stored data value is stored as the data value of the reference point.

  In FIG. 3B, the average value data of each area A to D is plotted as data values of the reference point A305 to the reference point D308. Each reference point is located at the right end of the corresponding region. A broken line 311 is obtained by linearly connecting the reference points to complement the data, that is, linearly approximating the reference points.

  The pixel data indicated by the dotted line 310 (or data obtained by vertically projecting the area) and the linear interpolation data indicated by the broken line 311 have an offset value corresponding to difference data between adjacent reference points. Since this offset value differs depending on the difference data between adjacent reference points, it cannot be matched between all reference points. However, if the difference data between adjacent reference points is small relative to the width of the region, it is roughly Can be matched. A solid line 309 indicates correction data obtained by linearly complementing the reference points and adding offset correction.

  In the second embodiment, since the reference point is placed at the right end of each region, correction data cannot be obtained in the data generation region of the reference point A305. Therefore, in the second embodiment, extrapolation data 312 is generated by extending the correction data between the reference points A305 and B306 to the area A and added to the correction data. The extrapolated data 312 is not necessary if the area A is small enough to be unnecessary as a captured image.

  As described above, according to the second embodiment, similarly to the first embodiment, the process of calculating the coordinates of the reference point is not necessary, data processing can be simplified, and image correction can be performed by adding offset correction. The accuracy of correction can be improved. Further, by dividing the image data into a plurality of small areas and processing the image data, the amount of memory can be reduced.

(Third embodiment)
Next, a third embodiment of the present invention will be described.
In the third embodiment described below, the widths of the regions A to D in the first embodiment are changed depending on the difference data of adjacent reference points. In this embodiment, the width of the region is changed. Is determined so that the difference data between adjacent reference points is approximately equal. Since the configuration of the image processing apparatus in the third embodiment is the same as that of the image processing apparatus in the first embodiment, the description thereof is omitted.

  FIG. 4 is a diagram for explaining the reference point data and the region for calculating the reference point data in the third embodiment.

  FIG. 4A shows a screen 404 of the image sensor. The screen 404 includes a horizontal OB pixel region 400, a vertical OB pixel region 401, and an effective pixel region 403. A reference point A405, a reference point B406, a reference point C407, and a reference point D408 are on the horizontal reference axis H1 passing through predetermined coordinates of the vertical OB region 401. Reference points A405 to D408 are set at the left end of region A, region B, region C, and region D, which are divided small regions. In the present embodiment, the areas A to D are divided so that difference data between adjacent reference points is approximately equal.

  In FIG. 4B, the pixel signal level of the horizontal reference axis H1 shown in FIG. Normally, the dark correction process in the horizontal direction can be performed by subtracting the pixel signal level of the horizontal reference axis H1 from the pixel signal level on the horizontal axis passing through the arbitrary coordinates of the effective pixel region 403. In the memory in the image processing apparatus according to the present embodiment, in order to reduce the amount of memory, not all of the pixel data on the horizontal reference axis H1 but calculation processing is performed from the pixel data of each of the areas A to D. The stored data value is stored as the data value of the reference point.

  In FIG. 4B, the average value data of each region A to D is plotted as data values of the reference point A405 to the reference point D408. Each reference point is located at the left end of the corresponding region. A broken line 411 is obtained by linearly connecting the reference points to complement the data, that is, linearly approximating the reference points.

  The pixel data indicated by the dotted line 410 (or data obtained by vertically projecting the area) and the linear interpolation data indicated by the broken line 411 have an offset value corresponding to difference data between adjacent reference points. In the present embodiment, each region is divided so that difference data between adjacent reference points is approximately equal. Therefore, in a region where pixel data is monotonously decreasing or monotonically increasing, the same offset value is used at all reference points. Can be taken. A solid line 409 indicates correction data obtained by linearly complementing the reference points and adding offset correction.

  In the third embodiment, since the reference point is placed at the left end of each region, correction data cannot be obtained in the data generation region of the reference point D408. In order to obtain the correction data of the area D, the correction data generated in the area C may be extended and extrapolated data may be added.

  As described above, according to the third embodiment, the process of calculating the coordinates of the reference point is not required, data processing can be simplified, and the accuracy of image correction can be improved by adding offset correction. . Further, by dividing the image data into a plurality of small areas and processing the image data, the amount of memory can be reduced.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.
In the fourth embodiment described below, the widths of the areas A to D in the first embodiment are changed depending on the difference data of adjacent reference points, and a reference point is placed at the right end of each area. It is. In the present embodiment, when determining the width of the region, the difference data between adjacent reference points is set to be approximately equal. Since the configuration of the image processing apparatus in the fourth embodiment is the same as that of the image processing apparatus in the first embodiment, the description thereof is omitted.

  FIG. 5 is a diagram for explaining the reference point data and the region for calculating the reference point data in the fourth embodiment.

  FIG. 5A shows a screen 504 of the image sensor. The screen 504 includes a horizontal OB pixel region 500, a vertical OB pixel region 501, and an effective pixel region 503. A reference point A505, a reference point B506, a reference point C507, and a reference point D508 are on the horizontal reference axis H1 passing through predetermined coordinates of the vertical OB region 501. The reference points A505 to D508 are set at the right end of the area A, the area B, the area C, and the area D, which are divided small areas. In the present embodiment, the areas A to D are divided so that difference data between adjacent reference points is approximately equal.

  In FIG. 5B, the pixel signal level of the horizontal reference axis H1 shown in FIG. Usually, the dark correction process in the horizontal direction can be performed by subtracting the pixel signal level of the horizontal reference axis H1 from the pixel signal level on the horizontal axis passing through the arbitrary coordinates of the effective pixel region 503. In the memory in the image processing apparatus according to the present embodiment, in order to reduce the amount of memory, not all of the pixel data on the horizontal reference axis H1 but calculation processing is performed from the pixel data of each of the areas A to D. The stored data value is stored as the data value of the reference point.

  In FIG. 5B, the average value data of each region A to D is plotted as data values of the reference point A505 to the reference point D508. Each reference point is located at the right end of the corresponding region. A broken line 511 is obtained by linearly connecting the reference points to complement the data, that is, linearly approximating the reference points.

  Pixel data indicated by a dotted line 510 (or data obtained by vertically projecting an area) and linear interpolation data indicated by a broken line 511 have offset values corresponding to difference data between adjacent reference points. In the present embodiment, each region is divided so that difference data between adjacent reference points is approximately equal. Therefore, in a region where pixel data is monotonously decreasing or monotonically increasing, the same offset value is used at all reference points. Can be taken. A solid line 509 indicates correction data obtained by linearly complementing the reference points and adding offset correction.

  In the fourth embodiment, since the reference point is placed at the right end of each area, correction data cannot be obtained in the data generation area of the reference point A505. In order to obtain the correction data of the area A, the correction data generated in the area B may be extended and extrapolated data may be added.

  As described above, according to the fourth embodiment, the process of calculating the coordinates of the reference point is not required, data processing can be simplified, and the accuracy of image correction can be improved by adding offset correction. . Further, by dividing the image data into a plurality of small areas and processing the image data, the amount of memory can be reduced.

Next, an imaging system to which the above-described image processing apparatus 10 is applied will be described.
FIG. 8 is a block diagram illustrating a configuration example of an imaging system to which the image processing apparatus 10 according to the above-described embodiment is applied.

  In FIG. 8, 10 is an image processing apparatus, 20 is an image sensor composed of a plurality of pixels, and 30 is an analog front end (AFE) that performs preprocessing including digitization on an image signal output from the image sensor. The AFE 30 includes a correlated double sampling (CDS) circuit 31, a programmable gain amplifier (PGA) 32, and an AD conversion circuit (A / D converter: ADC) 33, and is an optimal image for an image signal from the image sensor 20. Signal processing is performed to obtain a signal.

Reference numeral 40 denotes a timing generator (TG) that generates a control signal (drive signal) for driving the image sensor 20 and outputs the control signal to the image sensor 20. Further, the TG 40 generates and outputs a control signal (drive signal) for performing operation control of the AFE 30 and the image processing apparatus 10. The TG 40 generates these control signals (drive signals) based on the oscillation signal from the oscillator 41.
50 is a CPU, and 60 is a memory. The image processing apparatus 10, CPU 50, and memory 60 can communicate with each other via a system bus 70.

  The analog image signal output from the image sensor 20 driven based on the control signal (drive signal) from the TG 40 is subjected to preprocessing including digitization by the AFE 30. Specifically, the analog image signal from the image sensor 20 is subjected to noise removal by the CDS circuit 31, gain adjustment by the PGA 32, and analog-digital conversion (digitization) by the ADC 33.

  Image data obtained by processing the analog image signal by the AFE 30 is input to the image processing apparatus 10. Based on a command from the CPU 50, the image processing apparatus 10 performs image processing such as calculation processing such as shading correction and flaw correction and image data compression / expansion processing on the input image data. As described above, the image processing in the image processing apparatus 10 is performed by appropriately dividing input image data into a plurality of small regions. The image data that has been subjected to image processing by the image processing apparatus 10 is supplied to and stored in the memory 60 via the system bus 70.

(Other embodiments of the present invention)
For realizing the functions of the above-described embodiment for a computer (CPU or MPU) in an apparatus or system connected to the various devices so as to operate various devices to realize the functions of the above-described embodiments. Supply software programs. And what was implemented by operating the said various devices according to the program stored in the computer of the system or apparatus is also contained under the category of this invention.
In this case, the software program itself realizes the functions of the above-described embodiments, and the program itself constitutes the present invention. Means for supplying the program to the computer, for example, a recording medium storing the program constitutes the present invention. As a recording medium for storing such a program, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
In addition, such a program is also included in the embodiment of the present invention when the function of the above-described embodiment is realized in cooperation with an operating system running on a computer or other application software. Needless to say.
Further, after the supplied program is stored in a memory provided in a function expansion board or a function expansion unit related to the computer, a CPU or the like provided in the function expansion board or the like based on an instruction of the program may be a part of actual processing or Do everything. Needless to say, the present invention includes the case where the functions of the above-described embodiments are realized by the processing.

For example, the image processing apparatus shown in the first to fourth embodiments has a computer function 700 as shown in FIG. 6, and the CPU 701 performs the operation in this embodiment.
As shown in FIG. 6, the computer function 700 includes a CPU 701, a ROM 702, and a RAM 703. Further, a controller (CONSC) 705 of the operation unit (CONS) 709 and a display controller (DISPC) 706 of a display (DISP) 710 as a display unit such as a CRT or LCD are provided. Furthermore, a hard disk (HD) 711, a controller (DCONT) 707 of a storage device (STD) 712 such as a flexible disk, and a network interface card (NIC) 708 are provided. These functional units 701, 702, 703, 705, 706, 707, and 708 are configured to be communicably connected to each other via a system bus 704.
The CPU 701 comprehensively controls each component connected to the system bus 704 by executing software stored in the ROM 702 or the HD 711 or software supplied from the STD 712. That is, the CPU 701 reads out a processing program for performing the above-described operation from the ROM 702, HD 711, or STD 712 and executes it, thereby performing control for realizing the operations in the first to fourth embodiments. Do. The RAM 703 functions as a main memory or work area for the CPU 701.
The CONSC 705 controls an instruction input from the CONS 709. The DISPC 705 controls the display of the DISP 710. The DCONT 707 controls access to the HD 711 and the STD 712 that store a boot program, various applications, user files, a network management program, a processing program in the present embodiment, and the like. The NIC 708 exchanges data with other devices on the network 713 in both directions.

  It should be noted that each of the above-described embodiments is merely an example of a specific example for carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. . That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a figure for demonstrating the area | region which calculates the reference point data and reference point data in 1st Embodiment. It is a block diagram which shows the structural example of the image processing apparatus by embodiment of this invention. It is a figure for demonstrating the area | region which calculates the reference point data and reference point data in 2nd Embodiment. It is a figure for demonstrating the area | region which calculates the reference point data and reference point data in 3rd Embodiment. It is a figure for demonstrating the area | region which calculates the reference point data and reference point data in 4th Embodiment. It is a block diagram which shows the computer function which can implement | achieve the image processing apparatus in a present Example. It is a figure which shows the conventional image processing area | region and a complementary process area | region. It is a figure which shows the structural example of the imaging system to which the image processing apparatus in embodiment of this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 20 Image sensor 30 Analog front end (AFE)
40 Timing generator (TG)
200 Image sensor 202 Pixel data storage register 203 Reference point data generation circuit 204 RAM
205, 206 Reference point data storage register 207 Subtractor 208 Shifter 209 Offset data generation circuit 210 Differential data storage register 211 Selector 212 Adder 213 Linear interpolation data storage register 214 Subtractor 215 Subtractor

Claims (17)

An image processing apparatus that processes image data obtained from an imaging device including a plurality of pixels by dividing the image data into a plurality of small regions,
A data complementing means for generating linear complemented data by complementing data between a plurality of reference points by linear interpolation based on the data of the reference points;
Image processing means for performing image processing on the image data for each of the small regions using the linear interpolation data generated by the data complementing means,
The end point of the small region is used as a reference point for linear interpolation performed by the data complementing means, and a data value obtained by performing arithmetic processing on the image data of the small region is used as a data value of the reference point. Image processing device.   The image processing apparatus according to claim 1, further comprising an offset correction unit that performs offset correction of the linear interpolation data.   The image processing apparatus according to claim 1, wherein the reference point data is an average value of an area obtained by dividing image data obtained from an imaging device including a plurality of pixels into a plurality of small areas.   The image processing apparatus according to claim 1, wherein the width of the small area is set so that a difference in data value between adjacent reference points is equal. An image processing apparatus that processes image data obtained from an imaging device including a plurality of pixels by dividing the image data into a plurality of small regions,
A data complementing means for generating linear complemented data by complementing data between a plurality of reference points by linear interpolation based on the data of the reference points;
Image processing means for performing image processing on the image data for each of the small regions using the linear interpolation data generated by the data complementing means,
An image processing apparatus, wherein the reference point is set so that an area between reference points for which linear interpolation is performed by the data complementing unit and the small area are matched.   6. The image processing apparatus according to claim 5, wherein a data value obtained by performing arithmetic processing on the image data of the small area is used as the data value of the reference point. Data generating means for computing image data of the small area obtained from the image sensor;
The image processing apparatus according to claim 6, further comprising a data storage unit that stores a data value obtained by the data generation unit as a data value of the reference point. An image processing method for processing image data obtained from an image sensor composed of a plurality of pixels by dividing it into a plurality of small regions,
A data complementing process for generating linearly complemented data by complementing data by linear interpolation based on the data of the reference points between a plurality of reference points;
Using the linear interpolation data generated in the data complementing step, and performing an image processing on the image data for each small region,
An image processing method, wherein an end point of the small area is used as a reference point for linear interpolation, and a data value obtained by performing arithmetic processing on the image data of the small area is used as a data value of the reference point.   9. The image processing method according to claim 8, further comprising an offset correction step for performing offset correction of the linear interpolation data.   10. The image processing method according to claim 8, wherein the reference point data is an average value of a region obtained by dividing image data obtained from an imaging device including a plurality of pixels into a plurality of small regions.   The image processing method according to any one of claims 8 to 10, wherein the width of the small region is set so that a difference in data value between adjacent reference points is equal. A program for causing a computer to execute image processing for processing image data obtained from an image sensor composed of a plurality of pixels by dividing the image data into a plurality of small regions,
A reference point setting step for setting a reference point at an end point of the small area and setting a data value obtained by calculating the image data of the small area as a data value of the reference point;
Generate difference data between adjacent reference points, divide the difference data by the number of complementary points, generate differential data, add the differential data sequentially to the reference point data value for each complementary point, A data completion step for generating data for linear interpolation by performing data interpolation between points;
A program for causing a computer to execute an image processing step for performing image processing on the image data for each of the small regions using the linear interpolation data generated in the data complementing step.   13. The program according to claim 12, further comprising an offset correction step for performing offset correction of the linear interpolation data.   The program according to claim 12 or 13, wherein the reference point data is an average value of a region obtained by dividing image data obtained from an imaging device including a plurality of pixels into a plurality of small regions.   14. The program according to claim 12, wherein the reference point data is an average value of a region obtained by dividing image data obtained from a light-shielding region of an imaging element including a plurality of pixels into a plurality of small regions. .   13. A computer-readable recording medium on which the program according to claim 12 is recorded. The image processing apparatus according to any one of claims 1 to 7,
An image sensor composed of a plurality of pixels;
An imaging system comprising: preprocessing means for performing preprocessing including digitization on an image signal output from the imaging device, and supplying the obtained digital data as the image data to the image processing apparatus.

Images