JP2001034747A - Image conversion processor - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To correct distortion of a document image caused when a curved document is read by an image input device. A detects the distance data to the original S _f corresponding to each pixel of the CCD. Detecting the image data regarding a document S _f in line pixels P ₀ P ₉ on the light receiving surface. Pixel P _i
Expand straight distance data D _i the original S _f curved based up the corresponding original S _f in (the line segment Q ₀ 'Q _9').
A pixel (pixel on the line segment P ₀ 'P ₉ ') corresponding to the developed document S _f (line segment Q ₀ 'Q ₉ ') is obtained. Line segment P ₀ P
The image data detected on ₉ segment P ₀ to correspond to the pixels on 'P _9' performs image change processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image input device such as a scanner or a copier.

[0002]

2. Description of the Related Art In a conventional image input device such as a scanner or a copying machine, an original is read as a two-dimensional image projected on a plane where an image is taken. Therefore, unless the surface of the document to be read is curved or the like and the surface of the document to be read is not parallel to the plane to be imaged, the image or character drawn there was read as a distorted image (hereinafter referred to as an image including the character). . For example, when a thick book or the like is copied by a conventional copying machine, the vicinity of the binding margin is largely curved, and the image in the vicinity is copied with a large distortion.

[0003]

SUMMARY OF THE INVENTION The present invention relates to an image input apparatus for capturing an image of a document or the like, which suppresses the distortion of the read image caused by the document being not arranged parallel to the light receiving surface due to the curve of the document. It is an object of the present invention to provide an image conversion processing device that generates an image without correction and without distortion.

[0004]

According to the present invention, there is provided an image conversion processing apparatus comprising: an image data detecting means for detecting first image data by capturing an image of a subject; and a distance detecting means for detecting distance data representing the shape of the surface of the subject. Data detecting means, flattening processing means for associating a point or area on the object surface with a point or area on a plane based on the distance data, and first flattening processing means for associating the object surface with the plane in the flattening processing means. And image conversion processing means for generating second image data from the image data.

In the distance data detecting means, preferably,
Distance data is detected for the pixel. When the surface of the subject forms a curved surface, it is preferable that the ratio between the distance between any two points on the curved surface and the corresponding distance between the two points on the plane be constant in the flattening processing means. A point on the curved surface is associated with a point on the plane. In the flattening processing means, for example, the curved surface is divided along the bending direction, the curved surface is approximated by a straight line for each divided element using distance data corresponding to each division point in the division, and the curved surface approximated by the straight line Is developed into a plane, so that points on the curved surface correspond to points on the plane.

[0006] The image conversion processing means, for example, according to the correspondence between the curved surface and the plane obtained by the flattening processing means, each pixel of the image of the curved surface and each of the images to be obtained when the curved surface is developed into a plane. A correspondence relationship with the pixel is obtained, and the first image data related to the image of the curved surface is converted into the second image data according to the correspondence relationship between the pixels.

Preferably, the distance data detecting means includes a light source for irradiating the subject with distance measuring light, a plurality of photoelectric conversion elements for receiving reflected light from the subject and accumulating charges corresponding to the amount of received light, and A signal charge holding unit provided adjacent to the photoelectric conversion element, and an accumulated charge sweeping means for starting a signal charge accumulation operation in the photoelectric conversion element by sweeping unnecessary charge accumulated in the photoelectric conversion element; A signal charge transfer means for transferring the signal charge accumulated in the conversion element to the signal charge holding unit; and a signal for integrating the signal charge in the signal charge holding unit by alternately driving the accumulated charge sweeping means and the signal charge transfer means. And charge integration means.

The photoelectric conversion element constituting the distance data detecting means is preferably formed along the substrate, the accumulated charge sweeping means sweeps out unnecessary charges to the substrate side, and the signal charge holding section outputs the signal charges to the outside. Vertical transfer unit. More preferably, the photoelectric conversion element and the signal charge holding unit are configured as an interline CCD having a vertical overflow drain structure.

The image conversion processing device includes, for example, an image pickup unit for detecting image data of a subject and distance data to the subject, a stage for arranging the subject, and an image pickup unit for determining a distance from the subject arranged on the stage. A support member that is arranged at a distance and supports and fixes the support member, and the imaging section is mounted on the support member via an adjustment mechanism for adjusting the distance between the subject and the imaging section.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view of a camera-type image input device according to an embodiment of the present invention.

A photographing lens 1 is provided on a lower surface of the camera body 10.
1 is provided vertically downward, and an annular light emitting device 14 (light source) is provided near the opening of the imaging lens 11 so as to surround the opening. A switch 45 for performing imaging is provided on the front surface of the camera body 10, and a video output terminal 20 and an interface connector 21 are provided on the upper surface. A video output cable 22 is connected to the video output terminal, and is electrically connected to a television monitor (not shown). Also interface connector 2
1 is connected to an interface cable 23 and is electrically connected to a computer (not shown). The camera body 10 is supported from behind by a support member 13, and the support member 13 is mounted on the stage 12 and supported vertically by a support member 15 that stands upright in the vertical direction. The vertical movement of the support member 13 is performed by, for example, a rack (not shown) provided on the vertical support member 15 and a pinion (not shown) provided on a shaft of a dial 16 provided on the rear portion of the support member 13. Of the dial 1
This can be done by rotating 6. The original F is placed on the stage 12 with the side to be read facing the imaging lens 11.

FIG. 2 is a block diagram showing a circuit configuration of the camera type image input device shown in FIG. Shooting lens 11
Is provided with an aperture 25. The opening of the aperture 25 is adjusted by an iris drive circuit 26. The focus adjustment operation and the zooming operation of the taking lens 11 are controlled by a lens drive circuit 27.

An image pickup device (C
CD) 28 is provided. A subject image is formed on the CCD 28 by the photographing lens 11, and charges corresponding to the subject image are generated. Operations such as a charge accumulation operation and a charge read operation in the CCD 28 are controlled by the CCD drive circuit 30. The charge signal, that is, the image signal, read from the CCD 28 is amplified by the amplifier 31 and the A / A
The analog signal is converted into a digital signal in the D converter 32. The digital image signal is output from the imaging signal processing circuit 3
In the image memory 3, processing such as gamma correction is performed.
4 is temporarily stored. The iris drive circuit 26, lens drive circuit 27, CCD drive circuit 30, and image signal processing circuit 33 are controlled by a system control circuit 35.

The image signal read from the image memory 34 is sent to a TV signal encoder 38,
0 can be transmitted to a monitor device 39 provided outside the camera body 10. The system control circuit 35 is connected to the interface circuit 40, and the interface circuit 40 is connected to the interface connector 21. Therefore, the image signal read from the image memory 34 can be transmitted to the computer 41 connected to the interface connector 21, and can be output to the printer 42 connected thereto via the computer 41.

A light emitting element control circuit 44 is connected to the system control circuit 35. The light emitting device 14 is provided with a light emitting element 14a and an illumination lens 14b arranged in a ring, and the light emitting operation of the light emitting element 14a is controlled by a light emitting element control circuit 44. The light emitting element 14a irradiates a laser beam, which is a distance measuring light, and the laser beam irradiates the entire subject through the illumination lens 14b. The light reflected by the subject enters the photographic lens 11, and the light is detected by the CCD 28, whereby the distance to the subject is measured as described later. Also, the light emitting element 14
“a” is also used as a flash at the time of capturing an image, and a dark portion such as the vicinity of a binding margin of a curved document can be captured with proper exposure. In this measurement, CCD
The control of the transfer operation timing and the like in 28 is performed by the system control circuit 35 and the CCD drive circuit 30. A switch 45 is connected to the system control circuit 35.

Next, a flattened image detecting process performed in the present embodiment will be described with reference to FIGS.

In a standby state before the flattened image detecting process is executed, the camera performs a normal CCD video control, whereby an image signal read from the CCD 28 is transmitted through, for example, a television signal encoder 38. It is output to the monitor TV39. Original F on Stage 12
And adjust the position of the camera and the height of the camera while watching the image on the monitor TV. Thereafter, by pressing the switch 45, the flattened image detection processing shown in FIG. 3 is executed.

In step 100, distance data to the document is detected for each pixel of the CCD 28 (distance detection operation) by a method described later in detail, and stored in the image memory 34.
Thereafter, in step 105, a conventionally known CCD photographing operation is performed on the same original, and image data of the original is detected. The detected image data is stored in the image memory 34. In step 110, the distance data and the image data for each pixel stored in the image memory 34 are read out to the system control circuit 35, and the distance data and the image data are correlated for each pixel. The image data is stored in the image memory 34.

In step 115, the read image is divided into a predetermined number of pixels in the bending direction, and the flattening process is performed by obtaining the inclination and length of the document surface corresponding to each of the divided image areas. . That is, a correspondence relationship between a point on the original surface that is curved when the original is curved to be a flat surface and a corresponding point on the plane is obtained. In the image conversion process in step 120, the image data read in a distorted state in step 105 is corrected using the correspondence relationship in step 115, and the image data is converted into a flat image without distortion. In step 125, the corrected image data of the subject without distortion is displayed on the TV.
The signal is output to the monitor TV 39 via the signal encoder 38 and to the computer 41 via the interface circuit 40, and is printed by a printer 42 connected to the computer 41 in accordance with a user operation. Thus, the flattened image detection processing according to the present embodiment ends.

Next, taking the case where the image data is divided into nine parts as an example, the flattening process in step 115 and the image conversion process in step 120 will be described. FIG. 4 schematically illustrates the concept of the flattening process. A straight line AB represents a cross section of the light receiving surface of the CCD 28 on which the document is imaged. Curve S _f
Represents a cross section of a curved document. In the figure, the length in the horizontal direction is reduced so as to be the same as the light receiving surface of the CCD 28. The straight line A'B 'is the original S _f
Represents a cross section when extended in a plane. Point P
_i (i = 0, 1,..., 9) are photodiodes (pixels) on the CCD 28 corresponding to division points in image division.
, And the points P _i are arranged at a distance d from each other with a fixed number of pixels interposed therebetween. Document S point on _{f Q i (i = 0,1,} ..., 9) is a point on the original S _f corresponding to P _i (a point on the straight line AB) the center point of the pixel on the CCD 28, the pixel P _0, P ₉ correspond to both ends of the document S _f. The _{D i (i = 0,1, ...} , 9) is the distance from the pixel P _i obtained in step 100 to a point Q _i on the original S _f.

In reading the image in step 105, the original
S_fUpper arc Q_iQ_{i + 1}Is the line segment P_iP _{i + 1}Projected on
You. Arc Q_iQ_{i + 1}Is approximately a line segment Q_iQ_{i + 1}Thinking, Hara
Draft S _fTo the line segment Q_iQ_{i + 1}Continuous collection of straight lines
Approximate by At this time, the document approximated by a continuous straight line
S_fIs developed linearly, the line segment Q_iQ_{i + 1}Is this line
A line segment Q on a straight line A'B 'having the same length as the segment_i'Q_{i + 1}’
(Flattening process). In addition, line segment Q_i'Q_{i + 1}'of
The image is a line segment P on a straight line AB (CCD 28)._i’
P_{i + 1}Arc Q to correspond to_iQ_{i + 1}On the straight line AB
Line segment P_i'P_{i + 1}’. Therefore, the line segment P_iP
_{i + 1}Arc Q obtained above_iQ_{i + 1}Each pixel corresponding to the image of
Position and line segment P_i'P_{i + 1}’And the position of each pixel
Find the relationship and convert the image data according to this correspondence
If the original is curved, it is spread out flat, that is, flat
Image without distortion when extended to
processing).

The angle formed by the line segment Q _i Q _{i + 1} and the straight line AB is θ _i
And the scale of the original image formed on the CCD 28 is 1 /
_Let α be a line segment Q _i Q _{i + 1} and a line segment Q on the original Sf.
The length X _i of _i 'Q _{i + 1} ' is represented by the following equation. X _i = αd / cos θ _i where θ _i is

(Equation 1)It is. Also, the distance between the centers of adjacent pixels (pixel
D)_pAnd each pixel P _iAre arranged for every a pixels.
The pixel P_i, P_{i + 1}The distance d between is d = a
× d_p(A is an integer of 1 or more). Then the line segment
P_i'P_{i + 1}’Length d_iIs d_i= X_i/ Α = d / cos θ_i = A × d_p/ Cosθ_i Becomes As shown in FIG. 5, pixels arranged on a straight line AB are drawn.
Element P₀From pixel P₉When numbered in the direction, P₀Counting from
The M-th pixel is M = m × a + b (m is 0 ≦ m ≦ 9)
An integer, b is an integer of 0 ≦ b ≦ a−1). On CCD
And the center of the M-th pixel is the pixel P₀M × d from the center of_pof
Distance, which is the distance between the flattened image (line segment P₀’
P₉′), The distance λ represented by the following equation:_MCorresponding to

(Equation 2)

The segment P ₀ P ₉ distorted imaged document S _f on the image, the line segment P ₀ distance on _{P 9} M × d _p segment P ₀ the image distance on 'P _9' It is converted to an undistorted flattened image by stretching to L _M of the image. That segment P ₀ segment the image data of each pixel on the 'P _9' P ₀ P
What is necessary is just to correspond to the image data of the pixel on ₉ according to the above expansion. As this association, for example, the line segment P ₀ 'P
When N satisfies the following condition, the image data of the N-th pixel on ₉ ′ is regarded as the M-th image data on the line segment P ₀ P ₉ . N _M-1 <N ≦ N _M N _M-1 = round (L _M-1 / d _p ) N _M = round (L _M / d _p ) Here, round (x) is a value obtained by rounding off the decimal part of the real number x. Function. As a result, the image read on the line segment P ₀ P ₉ is converted into a flattened image on the line segment P ₀ 'P ₉ '.

Next, the principle of distance measurement in the present embodiment will be described with reference to FIGS. In FIG. 7, the horizontal axis is time t.

The distance measuring light output from the distance measuring device B is reflected by the subject S and received by a CCD (not shown). The distance measuring light is a pulsed light having a predetermined pulse width H, and therefore, the reflected light from the subject S is also a pulsed light having the same pulse width H. The rise of the reflected light pulse is delayed by a time δ · t (δ is a delay coefficient) from the rise of the distance measuring light pulse. Since the ranging light and the reflected light have traveled twice the distance r between the distance measuring device B and the subject S, the distance r is given by r = δ · t · C / 2 (1) can get. Where C is the speed of light.

For example, a state in which reflected light can be detected from the rise of a pulse of distance measuring light is determined, and the state is switched to an undetectable state before the reflected light pulse falls, that is, a reflected light detection period T is provided. And the received light amount A during the reflected light detection period T is a function of the distance r. That is, the light receiving amount A decreases as the distance r increases (the time δ · t increases).

In the present embodiment, utilizing the above-described principle,
The distance from the camera body 10 to each point on the surface of the subject S is detected by detecting the amount of received light A at each of a plurality of photodiodes (photoelectric conversion elements) provided in the CCD 28 and arranged two-dimensionally. , Subject S
The data of the three-dimensional image relating to the surface shape is input collectively.

FIG. 8 is a diagram showing the arrangement of the photodiode 51 and the vertical transfer unit 52 provided in the CCD 28.
FIG. 9 is a cross-sectional view of the CCD 28 cut along a plane perpendicular to the substrate 53. This CCD 28 is a conventionally known interline type CCD, and uses a VOD for sweeping out unnecessary charges.
(Vertical overflow drain) method.

The photodiode 51 and the vertical transfer section (signal charge holding section) 52 are formed along the surface of the n-type substrate 53. The photodiodes 51 are two-dimensionally arranged in a lattice, and the vertical transfer units 52 are provided adjacent to the photodiodes 51 arranged in a line in a predetermined direction (vertical direction in FIG. 8). The vertical transfer unit 52 includes four vertical transfer electrodes 52 a and 52 for one photodiode 51.
b, 52c and 52d. Therefore, in the vertical transfer section 52, four potential wells can be formed, and signal charges can be output from the CCD 28 by controlling the depths of these wells as conventionally known. The number of vertical transfer electrodes can be freely changed according to the purpose.

The photodiode 51 is formed in a p-type well formed on the surface of the substrate 53, and the p-type well is completely depleted by a reverse bias voltage applied between the p-type well and the n-type substrate 53. You. In this state, charges corresponding to the amount of incident light (reflected light from the subject) are accumulated in the photodiode 51. When the substrate voltage Vsub is increased to a predetermined value or more, the electric charge accumulated in the photodiode 51 is discharged to the substrate 53 side. On the other hand, when a charge transfer signal (voltage signal) is applied to the transfer gate unit 54, the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52. That is, after the charges are discharged to the substrate 53 side by the charge discharge signal, the signal charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52 side by the charge transfer signal. By repeating such an operation, the signal charges are integrated in the vertical transfer unit 52,
A so-called electronic shutter operation is realized.

FIG. 10 is a timing chart of a distance detecting operation for detecting data relating to the distance to each point on the surface of the subject. The distance detection operation will be described with reference to FIGS. 1, 2, and 8 to 10.

A charge discharge signal (pulse signal) S1 is output in synchronization with the output of the vertical synchronization signal (not shown), whereby unnecessary charges stored in the photodiode 51 are discharged in the direction of the substrate 53. Then, the accumulated charge amount in the photodiode 51 becomes zero (reference S2). After the start of output of the charge sweeping signal S1, pulse-shaped ranging light S3 having a constant pulse width is output. The period (pulse width) during which the ranging light S3 is output can be adjusted.
The adjustment is performed so that the distance measurement light S3 is turned off simultaneously with the output of the charge sweeping signal S1.

The distance measuring light S3 is reflected by the object, and
It is incident on D28. That is, although the reflected light S4 from the subject is received by the CCD 28, the charge sweeping signal S1
Is not stored in the photodiode 51 (reference S2). Charge sweep signal S1
Is stopped, the photodiode 51 starts to accumulate charges by receiving the reflected light S4, and the reflected light S4
4 and signal charges S5 caused by external light are generated. Reflected light S
When 4 disappears (reference S6), the photodiode 51 ends the charge accumulation based on the reflected light (reference S7).
Charge accumulation due to only external light continues (reference S8).

Thereafter, when the charge transfer signal S9 is output, the charges accumulated in the photodiode 51 are transferred to the vertical transfer section 52. This charge transfer is completed when the output of the charge transfer signal ends (reference S10). In other words, the charge accumulation in the photodiode 51 continues due to the presence of external light, but the signal charge S1 accumulated in the photodiode 51 until the output of the charge transfer signal ends.
1 is transferred to the vertical transfer unit 52. The charge S14 accumulated after the output of the charge transfer signal ends remains in the photodiode 51 as it is.

As described above, the period T from the end of the output of the charge sweep signal S1 to the end of the output of the charge transfer signal S9.
_{During U1} , the photodiode 51 accumulates signal charges corresponding to the distance to the subject. Then, the reflected light S4
Until the end of the light reception (reference S6), the charges accumulated in the photodiode 51 are transferred to the vertical transfer unit 52 as signal charges S12 (hatched portion) corresponding to the distance information of the subject,
Other signal charges S13 are caused only by external light.

After a predetermined time has elapsed from the output of the charge transfer signal S9, the charge sweeping signal S1 is output again, and after the transfer of the signal charge to the vertical transfer section 52, the photodiode 51
Unnecessary charges accumulated in the substrate 53 are swept toward the substrate 53.
That is, accumulation of signal charges in the photodiode 51 is newly started. Then, as described above, when the charge accumulation period T _U1 has elapsed, the signal charge is transferred to the vertical transfer unit 52.

The vertical transfer section 5 of such signal charges S11
2 is repeatedly executed until the next vertical synchronization signal is output. As a result, in the vertical transfer unit 52, the signal charge S11 is integrated, and the signal charge S11 integrated in a period of one field (a period sandwiched between two vertical synchronization signals) is regarded as a period in which the subject is stationary. If possible, it corresponds to distance information to the subject.

The detection operation of the signal charge S11 described above is for one photodiode 51, and such detection operation is performed in all the photodiodes 51 having the vertical transfer electrode 52a. As a result of the detection operation in the period of one field, the vertical transfer unit adjacent to the photodiode 51 having the vertical transfer electrode 52a holds the distance information detected by the photodiode 51. The distance information is output from the CCD 28 to the system control circuit 35 via the amplifier 31 and the like by the vertical transfer operation in the vertical transfer unit 52 and the horizontal transfer operation in the horizontal transfer unit (not shown).

However, the reflected light detected by the CCD 28 is affected by the reflectance of the surface of the subject. Therefore, the distance information obtained via the reflected light includes an error caused by the reflectance. In addition, the reflected light detected by the CCD 28 includes components such as external light in addition to the reflected light from the subject, and there is an error due to this.

Next, a method for correcting these errors will be described. FIGS. 11 to 13 are timing charts in the detection operation of the distance correction information, the reflectance information, and the reflectance correction information. 14 and 15 are flowcharts of the distance detection operation. The distance detection operation in the present embodiment will be described with reference to FIGS.

First, in step 200, a vertical synchronizing signal is output, and distance measuring light control is started. That is, the light emitting device 14 is driven, and the pulse-shaped ranging light S3 is output intermittently. Next, step 201 is executed,
The detection control by the CCD 28 is started. That is, FIG.
0, the charge detection signal S1 and the charge transfer signal S9 are output alternately, and the distance detection operation described with reference to FIG.
The signal charge S11 of the distance information is integrated in the vertical transfer unit 52.

In step 202, it is determined whether one field period has elapsed from the start of the distance detection operation, that is, whether a new vertical synchronizing signal has been output. When one field period ends, integration of the signal charges S11 over one field period is completed, and the integrated signal charges are output from the CCD 28 in step 203. This signal charge corresponds to the distance information and is temporarily stored in the image memory 34 in step 204. Step 20
At 5, the distance measuring light control is turned off, and the light emitting device 1
The light emitting operation of No. 4 stops.

In steps 206 to 209, an operation of detecting distance correction information (see FIG. 11) is performed. First, in step 206, a vertical synchronizing signal is output and CC
The detection control by D28 is started. That is, the light emission operation of the light emitting device 14 is not performed, and the charge sweeping signal S21 and the charge transfer signal S22 are turned off with the light source turned off.
Are output alternately. The charge accumulation time T _U1 is the same as the distance detection operation shown in FIG. 10, but since no distance measurement light is applied to the subject (reference S23), there is no reflected light (reference S).
24). Therefore, no signal charge of the distance information is generated, but a disturbance component such as external light is incident on the CCD 28.
The signal charge S25 corresponding to the disturbance component is generated, and the signal charge S26 that has been accumulated in the photodiode is transferred to the vertical transfer unit by the output of the charge transfer signal S22. This signal charge S26 corresponds to distance correction information for the charge accumulation time T _U1 for correcting the influence of the disturbance component on the distance information.

In step 207, it is determined whether one field period has elapsed from the start of the operation of detecting the distance correction information, that is, whether a new vertical synchronizing signal has been output. When one field period ends, in step 208, the signal charge S26 of the distance correction information is output from the CCD. The signal charge S26 of the distance correction information is temporarily stored in the image memory 34 in step 209.

In steps 210 to 214, an operation of detecting reflectance information (see FIG. 12) is performed. Step 21
In the case of 0, the vertical synchronization signal is output and the ranging light control is started, and the pulsed ranging light S33 is output intermittently. In step 211, the detection control by the CCD 28 is started, and the charge sweeping signal S31 and the charge transfer signal S3
5 are output alternately. By outputting the charge sweeping signal S31, the accumulated charge amount in the photodiode becomes zero (reference S32). Charge sweep signal S31
Is completed, the ranging light S33 is output and the CCD
Is reflected light S34. After the reflected light S34 has disappeared, a charge transfer signal S35 is output. That is, the operation of detecting the reflectance information is performed so that all of the reflected light S35 is received within the charge accumulation period T _U2 from the end of the output of the charge sweeping signal S31 to the end of the output of the charge transfer signal S35. Controlled.

As described above, in the photodiode, while the reflected light S34 is being received, the reflected light S34 and the signal charge S36 caused by the external light are accumulated, and while the reflected light S34 is not being received, the external light is accumulated. Signal charge S37 caused only by
S38 is accumulated. Then, by the output of the charge transfer signal S35, the signal charges S39 stored in the photodiode up to that time are transferred to the vertical transfer unit. This signal charge S39 corresponds to the reflectance information, and is a component S′3 based on external light.
9 is included.

In step 212, it is determined whether one field period has elapsed from the start of the reflectance information detection operation, that is, whether a new vertical synchronizing signal has been output. When one field period ends, the process proceeds to step 213, where the signal charge S39 of the reflectance information is output from the CCD. This signal charge S39 is temporarily stored in the image memory 34 in step 214. Step 215
In this case, the ranging light control is switched to the off state, and the light emitting device 14
Stops emitting light.

In steps 216 to 219, an operation of detecting reflectance correction information (see FIG. 13) is performed. In step 216, the vertical synchronizing signal is output and the CCD
28 starts detection control. That is, the light emitting device 1
In the state where the light source is turned off without performing the light emitting operation of No. 4, the charge sweeping signal S41 and the charge transfer signal S42 are output alternately. The charge accumulation time T _U2 is the same as the reflectance information detection operation shown in FIG. 12, but since no distance measurement light is applied to the subject (reference S43), there is no reflected light (reference S44). Accordingly, no signal charge of the reflectance information is generated, but a signal charge S46 corresponding to a disturbance component such as external light is generated in the CCD 28. The signal charge S46 corresponds to reflectance correction information for correcting the influence of the disturbance component on the reflectance information for the charge accumulation time T _U2 .

In step 217, it is determined whether or not one field period has elapsed from the start of the operation of detecting the reflectance correction information.
That is, it is determined whether a new vertical synchronization signal has been output. When one field period ends, the signal charges S47
Is completed over one field period of
At 18, the integrated signal charges are output from the CCD 28. This integrated signal charge corresponds to the reflectance correction information, and is temporarily stored in the image memory 34 in step 219.

In step 220, steps 200 to 2
Using the distance information, the distance correction information, the reflectance information, and the reflectance correction information obtained in step 19, the arithmetic processing of the distance data is performed. In step 221, the distance data is output, and the detection operation ends.

Next, the contents of the arithmetic processing executed in step 220 will be described with reference to FIGS. It is assumed that a subject having a reflectance R is illuminated, and the subject is regarded as a secondary light source having a luminance I and is imaged on a CCD.
At this time, the output Sn obtained by integrating the charge generated in the photodiode during the charge accumulation time t is represented by: Sn = kR.I.t (2) Here, k is a proportionality constant, which varies depending on the F number, magnification, and the like of the taking lens.

[0052] When the object is illuminated with light from a light source such as a laser, the luminance I becomes were synthesized with the luminance I _B by the luminance I _S and the background light due to the light source, I = I _S + I _B · ·・ (3)

As shown in FIG. 10, the pulse charge accumulation time is T _U1 , the pulse width of the distance measuring light S3 is T _S , and the pulse width of the signal charge S12 of the distance information is T _D , during one field period. When the charge storage time is to be repeated N times, the output SM _{10 resulting, SM 10 = Σ (k ·} R (I S · T D + I B · T U1)) = k · N · R (I S · T _D + I _B · T _U1 ) (4) The pulse width T _D can be expressed as T _D = δ · t = 2r / C (5)

As shown in FIG. 12, the pulse-shaped charge accumulation time T _U2 is controlled to be sufficiently longer than the period (pulse width) T _{S of the} distance measuring light S33, and to include the entire unit light receiving time of the reflected light. output SM ₂₀ obtained when _{the, SM 20 = Σ (k ·} R (I S · T S + I B · T U2)) = k · N · R (I S · T S + I B · T U2) ·・ ・ (6)

[0055] stopping the light emission as shown in FIG. 13, the output SM ₁₁ obtained when pulsed conducted charge accumulation as shown in FIG. _{10, SM 11 = Σ (k ·} R · I B · T _{U1) = k · N · R} · I B · T U1 ··· (7) to become. Similarly, the output SM ₂₁ obtained when performing charge accumulation as shown in FIG. _{13, SM 21 = Σ (k ·} R · I B · T U2) = k · N · R · I B · T _U2 (8)

[0056] (4), (6), (7), from equation _{(8), S D = (SM 10 -SM} 11) / (SM 20 -SM 21) = T D / T S ··· (9 ) Is obtained.

[0057] disturbance component of each outside light or the like and the distance measuring light S3 are in the reflected light S4 as described above (luminance I _B due to the background light)
It is included. T _D / T _{S in} equation (9) is the distance measuring light S3
Is obtained by normalizing the amount of reflected light S4 from the subject when illuminated by the amount of distance-measuring light S3. This is based on the amount of disturbance of distance-measuring light S3 (corresponding to signal charge S11 in FIG. 10). The value obtained by removing the component (corresponding to the signal charge S26 in FIG. 11) and the amount of reflected light S4 (the signal charge S
39 to a disturbance component (signal charge S'39 in FIG. 13).
) Is equal to the ratio of

Each output value SM ₁₀ , SM ₁₁ , SM of the equation (9)
₂₀ and SM ₂₁ correspond to steps 204, 209, 214 and 219
, Are stored as distance information, distance correction information, reflectance information, and reflectance correction information. Therefore, T _D / T _S is obtained based on these pieces of information. Pulse width Ts
Is known, the distance r is obtained from the equation (5) and T _D / T _S.

[0059] Thus (5) distance information from the camera body based on the equation (9) to each point of the surface of the object is corrected, resulting accurate distance information to the original S _f is subject Can be

In the present embodiment, the influence of external light or the like is removed from the distance information of the subject. However, if the influence of external light or the like can be neglected, the influence of the external light or the like is obtained by the equation (9). It is sufficient to omit the integrated values of the signal charges (that is, S ₁₁ and S ₂₁ ). As a result, a correction relating only to the reflectance of the surface of the subject is performed.

In the present embodiment, step 20
In 2, 207, 212, and 217, signal charges are accumulated for one field period. Alternatively, charge accumulation may be performed for a plurality of field periods.

In this embodiment, the image is equally divided for each predetermined number of pixels.
The fineness of the division may be changed for each location in accordance with the curvature by using the nth-order differential coefficient of _f (n ≧ 1) and the curvature based on them. Further, in the case of distortion due to a plane object not being parallel to the light receiving surface of the image input device, only the number of distance data necessary to detect the inclination of the object plane from the light receiving surface is detected (for example, two points). ), The image distortion due to the inclination of the subject surface may be corrected based on this.

In this embodiment, the distance data corresponding to each pixel of the captured image is detected. However, the distance data is detected only for some of the pixels, for example, only for the pixel corresponding to the division point. You may.

The optical sensor is an area sensor because the image input device of the present embodiment is a camera type image input device, but may be a scanner type image input device using a linear sensor. In this embodiment, the distance data is detected using the CCD used for image reading.
For detecting the distance data, for example, a scanning distance detecting device or the like may be separately provided and used.

In the present embodiment, the curved original is developed on a flat surface with one end fixed, but this is not necessarily the end of the original.

[0066]

As described above, according to the present invention, in an image input apparatus for imaging a document or the like, a read image generated due to the document being not arranged parallel to the light receiving surface due to the document being curved or the like. An image conversion processing device that corrects distortion and generates an image without distortion can be obtained.

[Brief description of the drawings]

FIG. 1 is a perspective view of a camera-type image input device of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of the camera type image input device shown in FIG.

FIG. 3 is a flowchart of a flattened image detection process performed in the present invention.

FIG. 4 is a diagram schematically illustrating a concept of a flattening process.

FIG. 5 is a diagram schematically illustrating the concept of image conversion processing.

FIG. 6 is a diagram for explaining the principle of distance measurement using distance measurement light.

FIG. 7: distance measuring light, reflected light, gate pulse, and CCD
FIG. 4 is a diagram showing a distribution of the amount of light received by the light source.

FIG. 8 is a diagram showing an arrangement of a photodiode and a vertical transfer unit provided in a CCD.

FIG. 9 is a cross-sectional view of the CCD cut along a plane perpendicular to the substrate.

FIG. 10 is a timing chart of a distance detection operation for detecting data relating to a distance to a subject.

FIG. 11 is a timing chart of a detection operation of distance correction information.

FIG. 12 is a timing chart of an operation of detecting reflectance information.

FIG. 13 is a timing chart of the operation of detecting the reflectance correction information.

FIG. 14 is a first half of a flowchart of a distance detection operation.

FIG. 15 is the second half of the flowchart of the distance detection operation.

[Explanation of symbols]

 10 Camera body F Original (subject)

Continued on the front page (72) Inventor Shinichi Kakiuchi 2-36-9 Maeno-cho, Itabashi-ku, Tokyo Asahi Gaku Kogyo Co., Ltd. F-term (reference) 5B047 AA01 AB02 BA02 BB02 BC05 BC16 CA11 CA13 CA21 CB09 CB23 DC09 5B057 BA02 BA15 BA17 BA19 CA12 CB12 CC03 CD12 CE05 CG09 CH18 DA08 DA17 DC02 DC09 5C022 AA13 AB28 AB51 AC27 AC42 AC69 AC74 5C072 AA01 BA17 EA05 EA08 LA12 RA06

Claims (10)

[Claims] An image data detecting unit configured to detect first image data by capturing an image of a subject; a distance data detecting unit configured to detect distance data representing a shape of a surface of the subject; Flattening processing means for associating a point or area on the subject surface with a point or area on a plane; and a step of converting the first image data based on the correspondence of the subject surface to the plane in the flattening processing means. 2. An image conversion processing device, comprising: image conversion processing means for generating the second image data. 2. The image conversion processing apparatus according to claim 1, wherein the distance data detecting means detects distance data for a pixel. 3. When the surface of the subject forms a curved surface, the flattening processing means sets a distance between any two points on the curved surface and a distance between two points on the plane corresponding to the distance.
The image conversion processing device according to claim 1, wherein a point on the curved surface is associated with a point on the plane such that a ratio with a distance between the points is constant. 4. The flattening processing means divides the curved surface along a bending direction, and approximates the curved surface with a straight line for each divided element using the distance data corresponding to each division point in the division. The point on the curved surface is associated with a point on the plane by developing the curved surface approximated by the straight line on the plane.
3. The image conversion processing device according to claim 1. 5. The image conversion processing unit develops each pixel of the image of the curved surface and the curved surface on the plane according to the correspondence between the curved surface and the plane obtained by the flattening processing unit. A first image data relating to the curved surface image is converted into a second image data in accordance with the corresponding relationship of each pixel of the image to be obtained when the image is to be obtained. Item 4. The image conversion processing device according to item 3. 6. A light source for irradiating the object with distance measurement light, a plurality of photoelectric conversion elements receiving reflected light from the object and accumulating electric charges in accordance with an amount of received light, wherein: A signal charge holding unit provided adjacent to the photoelectric conversion element, and a storage for starting a signal charge accumulation operation in the photoelectric conversion element by sweeping out unnecessary charge accumulated in the photoelectric conversion element from the photoelectric conversion element. A charge sweeping unit; a signal charge transfer unit that transfers signal charges accumulated in the photoelectric conversion element to the signal charge holding unit; and alternately driving the stored charge sweeping unit and the signal charge transfer unit. 3. The image conversion processing device according to claim 2, further comprising a signal charge integration unit that integrates the signal charge in the signal charge holding unit. 7. The image conversion processing apparatus according to claim 6, wherein said photoelectric conversion element is formed along a substrate, and said accumulated charge sweeping means sweeps out unnecessary charges toward said substrate. 8. The image conversion processing device according to claim 6, wherein the signal charge holding unit is a vertical transfer unit for outputting the signal charge to the outside. 9. The photoelectric conversion element and a signal charge holding unit,
Interline CC with vertical overflow drain structure
7. The image conversion processing device according to claim 6, wherein the image conversion processing device is configured as D. 10. An image pickup unit for detecting image data of the object and the distance data to the object, a stage for arranging the object, and the image conversion processing device, And a support member for supporting and fixing the object at a distance from the object arranged in the image pickup section, and the imaging section adjusts the distance between the object and the image pickup section to the support member. The image conversion processing device according to claim 1, wherein the image conversion processing device is mounted via an adjustment mechanism.