KR0139572B1 - Image distortion calibration method using nervous network - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for correcting image distortion of a camera using a convenient and highly accurate neural network circuit so that back projection can be efficiently used for error correction. In the image distortion correction method for preventing distortion from occurring when the image is projected onto the camera image plane, the actual coordinate values of the arbitrary correction points on the three-dimensional spatial coordinates (X, Y, Z) and the projection points of the two-dimensional image plane. Calculating a mapping function of and using the irradiation angle theta (θ) with respect to the reference line as additional information of the mapping; A second step of converting the mapping function obtained in the first step into a connection strength of a neural network circuit and storing the mapping function in a table; And obtaining a distance of the image information by reverse projecting the acquired camera image, and inputting and correcting a table value corresponding to the distance to the neural network circuit to correct the image distortion such as environment, camera type, parameter, and the like. There is an effect that can be easily and efficiently corrected regardless of the change.

Description

Image Distortion Correction Method Using Neural Network

1 is an example of hardware to which the present invention is applied.

2 is a conceptual diagram of a mapping coordinate system.

3 is a conceptual diagram of a mapping method using a neural network according to the present invention.

4 is a flowchart of a learning process for image distortion correction according to the present invention.

5 is a flowchart of an image distortion correction process according to the present invention.

6 is an exemplary diagram of a neural network circuit used in the present invention.

* Explanation of symbols for main parts of the drawings

10: laser light source 20: lens

50: camera 60: angle control device of the plane light

70: computer for image processing 80: monitor

According to the present invention, when an image acquired by a CCD (Charge Coupled Device) or a TV (TV) camera exhibits a distortion unlike the reality, the distortion is calibrated using a neural network circuit. It relates to an image distortion correction method.

CCD or television (TV) cameras and the like have manufacturing inaccuracies in optical systems, mechanical devices, and electronic devices. Accordingly, an image appearing on an image plane of the camera is distorted unlike an actual one in three-dimensional space. That is, distortion occurs when the image of the 3D space is projected onto the camera image plane.

Therefore, for the purpose of 3D object recognition, robot vision device, 3D television (TV), etc., the back coordinates of the distorted camera image are used to obtain the position coordinates in the 3D space. And many errors cause difficulty in application.

To remove this difficulty, the acquired image must be corrected to be realistic.

As a conventional correction method, a method of calibrating error components by modeling the structure and function of a camera is used.

That is, Sobel invented a method using a pin-hole model and a non-linear optimization method for camera parameters excluding lenses. Tsai invented a method of modeling and correcting distortion in four stages, including a lens, and Martin and two others invented a two plane method.

However, the conventional method by mathematical modeling and analysis should be well aware of the characteristics and specifications of the camera when modeling, and there is a problem that the mathematical analysis is complex and time consuming.

The present invention devised to solve the problems of the prior art, the image distortion by using a convenient and high accuracy neural network circuit, so that the back projection can be efficiently used for the correction of the error An object of the present invention is to provide a method for correcting image distortion.

In order to achieve the above object, the present invention, in the image distortion correction method for correcting the distortion caused when projecting the image of the three-dimensional space to the two-dimensional image plane, three-dimensional space coordinates (X, Y, Z) Calculating a mapping function between the actual coordinate values of the correction points of the image and the projection points of the two-dimensional image plane, and using the irradiation angle theta (θ) with respect to the reference line as additional information of the mapping; A second step of converting the mapping function obtained in the first step into a connection strength of a neural network circuit and storing the mapping function in a table; And obtaining a distance of the image information by reverse projecting the acquired camera image, and then inputting and correcting a table value corresponding to the distance to the neural network circuit.

Hereinafter, with reference to the accompanying drawings will be described an embodiment according to the present invention;

1 shows an example of hardware to which the present invention is applied.

When an actual three-dimensional image tries to obtain three-dimensional coordinates again from an image projected on the two-dimensional image plane of the camera, three-dimensional information cannot be obtained because the one-dimensional information is lost by the projection. As described above, the light plane is actively used in the three-dimensional space to illuminate the light plane on the object 100 (for example, the light plane 30 is formed by using the laser light source 10 and the semi-cylindrical lens 20). ) The light streaks 40 are formed at the desired position, and the streaked images are acquired by the camera 50 to obtain the spatial eye vector when the back projection of the equation of this light plane and a point of the 2D image plane is obtained. Get three-dimensional coordinates by surveying.

In other words, as in the triangulation method commonly used in surveying, the distance to the target point can be known when the angle of irradiation and the direction of the laser beam with respect to the reference line are known. In the drawings, reference numeral 60 denotes an angle adjusting device for plane light, 70 denotes an image processing computer for obtaining angle information with a reference line of plane light, and an image processing computer for outputting three-dimensional distance information, respectively.

2 is a conceptual diagram of a mapping coordinate system.

Including a projection problem that predicts the position projected onto the image plane of the camera when a point in space is given, and an inverse projection problem that calculates the spatial gaze vector when given a point on the image plane and is projected back into space. The image distortion correction process of the camera, which is a process of reducing the error by determining the mapping relationship between the dimension and the two-dimensional, is that the known points Pw (X, Y, Z) in the three-dimensional coordinate system space are two-dimensional When projected onto the image plane, the point Qi (U, V) of the image plane coordinates can be obtained so that the mapping function, which is a coordinate system correlation from the corresponding point coordinates of the two coordinates, is calculated. Correct the dimension coordinates so that there is little error.

The present invention consists of a mapping function implementation process and a process of correcting an image. First, a mapping process implementation process, which is a first step, will be described with reference to FIGS. 3 and 4.

3 is a conceptual diagram of a mapping method using a neural network according to the present invention. As shown in the drawing, three-dimensional coordinate values (X, Y, Z) of correction points Pw of various positions and various distances are set as target data. It is assigned to the value of the output terminal to the neural network circuit, and the coordinate values (U, V) of the point Qi of the camera image plane corresponding to the correction point Pw and theta (θ) of the optical plane are input data ( U, V, θ) to construct a training data set and perform training.

4 is a flowchart of a learning process for image distortion correction according to the present invention, wherein the three-dimensional coordinate values (X, Y, Z) of each of the correction points correspond to the image coordinate values and the corresponding optical plane angle values (U, The process of learning a mapping relationship with a neural network using V, θ) as a learning set is shown.

As shown in the figure, n correction points (X, Y, Z) of the three-dimensional coordinate space are selected (41), and then an objective data set is constructed (42) at the output terminal of the neural network. Next, n points (U, V) of the two-dimensional image plane corresponding to the correction point and the irradiation angle theta (θ) of the corresponding optical plane are obtained (43), and the input values (U, V, θ) of the neural network input terminal. Construct a data set (44). After constructing the data set, the neural network circuit is repeatedly trained (45) and checked whether it is trained within the set error (46). If the trained training is performed within the set error, the trained connection strength is stored in the table (47). If it is out of the set error, the process is repeatedly performed from the process 45 of performing the iterative learning on the neural network circuit.

That is, the learning is repeated so as to be learned within a desired error, and the connection strength between neurons corresponding to the mapping function is obtained, and the value of the obtained connection strength is maintained in a table.

As the number of correction points is selected and the corresponding points are selected at various distances, the correct learning is performed, but the time required increases, so the appropriate number is selected. The learning is repeatedly performed by selecting a correction point for a desired distance, and the connection strength obtained according to the distance is maintained in a table.

Next, a process of correcting an image, which is the second step, will be described with reference to FIG. 5.

5 is a flowchart of an image distortion correction process according to the present invention. The process of correcting an image first receives an image plane coordinate and an optical plane angle (51). After calculating the alternative distance (52), a table value corresponding to the distance is input as the connection strength of the neural network (53), and arbitrary image coordinate values and angle values of the optical plane (U, V, θ), the correct three-dimensional coordinate value (X, Y, Z) is output through the neural network circuit, not the distorted coordinate value, so that the corrected correct value can be obtained.

That is, the angle θ corresponds to (X, Y, Z) of the Pw point together with the U and V values of the Qi point. This expression is equal to Pw (X, Y, Z) = F (U, V, θ).

Therefore, the points Q represented by (i, j, θ) on the camera image plane can be obtained by inversely projecting the points projected to (X, Y, Z) in the three-dimensional space by the mapping function F. Can be corrected using. Here, the precision of the correction is determined by the precision of the mapping function.

6 shows an example of a neural network circuit used in the present invention, and shows a multi-layered perceptron neural network circuit using an inverse propagation learning algorithm capable of nonlinear mapping.

As shown in the figure, the number of neurons of the input and output terminals is three, respectively, and has multiple hidden layers. The input terminal inputs (U, V, θ) values for n correction points on the image plane, and the output terminal inputs (X, Y, Z) values corresponding to n correction points in three-dimensional space. , Learn by back propagation. That is, the connection strength Wij between neurons that map input / output values is obtained.

According to the present invention as described above, a mapping relationship between a correction point in three-dimensional space and a correction point projected on a two-dimensional image plane is obtained by a neural network for a desired distance and field of view, and the correction value is corrected for image distortion while having this relationship as a table. Since this is possible, the image distortion can be easily and efficiently corrected regardless of a change in external environment, camera type, or parameter.

In addition, as an example of the application, it is effective to calibrate a camera mounted on the robot vision apparatus for the purpose of 3D object recognition or distance measurement.

Claims (4)

An image distortion correction method for correcting a distortion caused when a 3D space image is projected onto a 2D image plane, the method comprising: the actual coordinate values of arbitrary correction points on the 3D space coordinates (X, Y, and Z); Obtaining a mapping function with projection points of the two-dimensional image plane, and using the irradiation angle theta (θ) with respect to the reference line as additional information of the mapping; A second step of converting the mapping function obtained in the first step into a connection strength of a neural network circuit and storing the mapping function in a table; And obtaining a distance of the image information by reverse projecting the acquired camera image, and inputting and correcting a table value corresponding to the distance to the neural network circuit. The method of claim 1, wherein the first step comprises: a fourth step of configuring an object data set at an output of the neural network after selecting n correction points (X, Y, Z) in a three-dimensional coordinate space; After n points (U, V) of the two-dimensional image plane corresponding to the respective correction points and the irradiation angle theta (θ) of the corresponding optical plane are obtained, the input values of the neural network circuit input terminals (U, V, θ) are obtained. A fifth step of constructing a data set; And performing a repeating learning on the neural network circuit until the learning is performed within a predetermined error, and performing the second step when the learning is performed within a predetermined error. 3. The method of claim 1, wherein the third step comprises: a seventh step of obtaining an alternative distance based on the image information by back-projection without correction from the obtained camera image by receiving the image plane coordinates and the optical plane angle; An eighth step of inputting a table value corresponding to the distance obtained in the seventh step as a connection strength of the neural network; And a ninth step of inputting an arbitrary image coordinate value and (U, V, θ) which is an angle value of an optical plane to the neural network circuit, and outputting a three-dimensional coordinate value (X, Y, Z). Calibration method. 4. The method of claim 3, wherein the neural network circuit is a multi-layered perceptron neural network circuit using an inverse propagation learning algorithm capable of nonlinear mapping.