CN110176020A - A kind of bird's nest impurity method for sorting merging 2D and 3D rendering - Google Patents

Abstract

The invention discloses a method for sorting bird's nest impurities by combining 2D and 3D images, comprising the following steps: S1, reconstruction and identification of bird's nest impurities; S1.1, image preprocessing; A very important link, image acquisition is often disturbed by various noises and surrounding environments, which not only affects the effect of the image, but also makes the required relevant information often submerged in it, which brings inconvenience to the subsequent feature extraction; the present invention can Improve the work efficiency of bird's nest feather impurities sorting, effectively reduce the production cost of bird's nest; bird's nest product precision is higher than manual picking, can carry out long-term stable bird's nest feather impurities picking work; after introducing the method of the present invention, the quality of bird's nest products can be improved , reduce the missed detection rate and false detection rate of bird's nest products, and obtain reliable, stable and accurate detection of bird's nest products.

Description

A method for sorting impurities in bird's nest by fusing 2D and 3D images

technical field

The invention relates to the technical field of identification of bird's nest impurities, in particular to a method for sorting bird's nest impurities by fusing 2D and 3D images.

Background technique

Removing feather impurities is an important process in the bird's nest processing industry. At present, the picking method of bird’s nest feather impurities is manual picking, which has the following problems: On the one hand, the manual picking method mainly depends on people’s subjective judgment and experience, and it is difficult to provide a reliable, stable and accurate The detection results; there is no uniform standard for labor, and the false detection rate and missed detection rate are high, which leads to uneven sorting products and damages the interests of the enterprise; on the other hand, the working environment and long-term manual work of picking bird's nest feathers have a negative There is also greater physical and mental damage.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a method for sorting bird’s nest impurities that combines 2D and 3D images. The method is based on machine vision technology to identify and locate bird’s nest impurities, which can improve work efficiency and reduce labor costs. , greatly enhance the competitiveness of enterprises in the market.

The object of the present invention is achieved through the following technical solutions:

A kind of bird's nest impurity sorting method that fuses 2D and 3D images, comprises the following steps:

S1, reconstruction and identification of impurities in bird's nest;

S1.1, image preprocessing;

Image preprocessing is a very important link in the process of bird's nest identification. Image acquisition is often disturbed by various noises and surrounding environments, which not only affects the image effect, but also makes the required relevant information often submerged in it, giving subsequent Feature extraction brings inconvenience; in order to filter out noise interference, improve image quality, and highlight the required relevant information, relevant image preprocessing is required before identifying and detecting bird's nest impurities;

S1.1.1, image filtering;

In the process of bird’s nest image transmission and processing, it is often polluted by various noises, and dark or bright spots of the image interfere, which not only reduces the image quality, but also affects the accuracy of feature extraction in image processing; therefore, it is necessary to select an effective image filter Algorithms to solve the impact of noise, commonly used image filtering algorithms are: frequency domain filtering method, mean filtering method and median filtering method in the spatial domain;

The median filtering algorithm is a neighborhood operation, and the calculation process is to arrange the corresponding values on the template in ascending order when performing median filtering on the image, and then assign the median value of this column of data to the template The pixel at the center position; among them, if the template has an odd number of points, the gray value of the middle pixel arranged in order of size is used as the middle value; if the template has an even number of points, the gray value of the middle pixel arranged in order of size The average of the two values in the middle is taken as the median;

Since the median filtering effect depends on the size of the filtering window, if it is too large, the edge will be blurred, and if it is too small, the denoising effect will be unsatisfactory, so the median filtering algorithm is improved: scan the image line by line, and judge the pixel when processing each pixel Whether it is the maximum or minimum value of the neighboring pixels covered by the filter window; if so, use the normal median filtering algorithm to process the pixel; if not, ignore it;

The improved 3×3 median filtering algorithm is used for image filtering;

S1.1.2, image enhancement;

Using the piecewise linear transformation function to enhance the image contrast is actually to enhance the contrast between the parts of the original image, that is, to enhance the gray-scale areas of interest in the input image, and relatively suppress those gray-scale areas that are not of interest;

The form of the piecewise linear transformation function is as follows:

Among them, (x ₁ , x ₂ ) and (y ₁ , y ₂ ) are the main parameters in the above formula (3.1). According to the description of the algorithm function, it can be seen that x ₁ and x ₂ are the gray scale ranges to be converted to limit the processing object , while y ₁ and y ₂ determine the slope of the linear transformation;

S1.2, image segmentation;

Image segmentation adopts the threshold segmentation method, which separates the background and the object by selecting the optimal threshold for image segmentation; by setting a reasonable threshold for image judgment, the gray value of those parts that meet the set threshold range is set to 0, otherwise set to 1 to separate the target of interest from the image and generate a binary image;

Threshold segmentation is to transform the input f of the image into the output g, and the transformation is as follows:

In the above formula, T is the set threshold, g(i,j)=0 represents the image element of the background part, and g(i,j)=1 represents the image element of the target object part; threshold segmentation is to scan the image f for its For all pixels, if f(i,j)≥T, the element g(i,j) of the segmented image is the pixel of the object; otherwise, it is the background pixel;

S1.3, feature selection and feather impurity region extraction;

After the various regions of interest are obtained through image segmentation, simple region descriptors can be used as the features representing the region, and these regional features can be combined into feature vectors for classification;

Wherein, the simple region descriptor is perimeter, area, compactness, centroid of the region, gray mean, gray median, minimum rectangle containing the region, minimum or maximum gray level, number of pixels greater or less than the mean and Euler's number;

S1.4, positioning of bird's nest feather impurity area;

When locating the identified feather impurities, it needs to be classified and corrected;

S1.4.1, Regional Classification of Feather Trash;

Using the Euclidean distance to judge the value d between the centroid of each feather impurity area and the minimum distance point of its own area, each feather impurity area can be divided into two categories; the centroid coordinates of the feather impurity area are calculated by the centroid formula (4.4) and (4.5) as (R , C), then the Euclidean distance formula is as follows:

If d=0, the feather region belongs to the first category, that is, the centroid falls within the feather region; if d≠0, the feather region belongs to the second category, that is, the centroid falls outside the feather region;

Among them, the centroid of the first type falls in the feather area is exactly what is required, while the second type of feather area needs to be corrected, and the centroid falling outside the feather area of the second type will be obtained through the correction algorithm to obtain a new point falling in the feather area (i _c , j _c );

S1.4.2, centroid correction;

In order to correct the centroid outside the feather area, a linear extension algorithm of the major semi-axis of the smallest circumscribed ellipse is introduced;

S1.5, 3D reconstruction of bird's nest impurities;

The stereoscopic registration of the 2D image of the area array camera and the 3D image of the depth camera is to use the spatial geometric coordinate transformation relationship to find the corresponding relationship between the pixel coordinates of the two images; Extract the Mark area and the centroid coordinates of the Mark area; then perform feather impurity feature extraction on two or more images of the area array camera to obtain the centroid coordinates of the bird's nest feather impurity area; then calculate the matching 3D bird's nest image according to the Mark point conversion formula The bird's nest feather impurity feature point area in the bird's nest; finally complete the image matching and generate a 3D model;

S1.5.1, Acquisition of bird's nest feather impurity point cloud data and bird's nest impurity model reconstruction;

According to the generated 3D bird's nest impurity area, the original 3D bird's nest image is segmented accordingly to generate a designated bird's nest feather impurity area image, and then the 3D bird's nest feather impurity area image is decomposed into X, Y, Z coordinate point information containing three-dimensional coordinates image, and convert the three-dimensional point X, Y, Z images into 3D bird’s nest feather impurity point clouds, and obtain the discrete three-dimensional feature points on the bird’s nest impurity surface; The surface of bird's nest feather impurities;

S1.5.2, Feature identification of bird's nest feather impurities;

Use the Mark point formula to obtain the corresponding relationship between the bird's nest images of the two area array cameras and the bird's nest image of the 3D camera, and obtain the three-dimensional representation of the points of all the bird's nest impurity regions in the 3D bird's nest image;

According to the generated 3D bird's nest impurity area, reduce the original 3D bird's nest image to the designated bird's nest feather impurity area image, and then decompose the 3D bird's nest feather impurity area image into an x, y, z coordinate image containing 3D points, where the z image is the height image; take the z image as the processing object, use the centroid formula to find the 3D bird's nest feather impurity coordinates (x, y), then each two-dimensional coordinate point in the three-dimensional image corresponds to a fixed height value Z; because the gray value of the z image It is the height value Z of the image. First, find the coordinates of the center of mass in the z image as the center of the circle, and respectively take the smallest ring in the bird's nest feather impurity area, that is, the smallest ring + 30 pixels and the smallest ring + 60 pixels. The gray value mean and deviation Deviation between circular differences can be described by the following formula:

Among them, the height Z is:

Z=Mean-Mean ₁ (3.6)

Thereby, the three-dimensional coordinates (Row3m, Colm3m, Z) of each feather area can be obtained from the 3D camera;

S2, experiment;

S2.1, camera calibration;

S2.1.1, distortion correction;

Usually, the camera lens used in the vision system has different degrees of distortion. The distance between the pixel and the center will affect the degree of image distortion. The closer to the image center, the smaller the distortion. This kind of distortion belongs to the nonlinear type, which can be described by the following formula:

In the above formula, Represents the ideal coordinates of a pixel point that conforms to the linear imaging model without distortion, (x, y) represents the coordinates of the actual image point, δ _x and δ _y are nonlinear distortion values, which are related to the position of the image point in the image, and can be expressed by the following formula:

Among them, the first item of δ _x or δ _y is Radial distortion, the second item is Centrifugal distortion, and the third item is Thin prism distortion. The coefficients in the formula are called nonlinear distortion parameters; When too many distortion parameters are introduced, the solution will be unstable and affect the accuracy improvement; therefore, the formula (4.2) can be simplified as:

It can be clearly seen that the distortion increases with the increase of the radial radius, that is, the part far from the image center is more severely distorted;

S2.1.2, Mark point selection;

Convert the coordinates of the detected object in the two-dimensional image to the three-dimensional image through the Mark point, and judge whether the converted three-dimensional coordinates are the position of the detected object to achieve the camera calibration target;

Use the combination of area array camera and 3D camera for image acquisition, and select a circle or rectangle with a certain height, uniform color and regular shape as the Mark point;

The Mark point and the detected object have the characteristics of obvious differences in grayscale and shape. First, use the grayscale threshold method to segment the area where the Mark point is located, and then use the feature extraction method to identify the Mark point;

S2.1.3, Mark point centroid coordinate calculation;

Use the above-mentioned Mark point identification method to identify the Mark points in the two-dimensional image and the three-dimensional image respectively, and then calculate the centroid coordinates of the respective Mark points; for a 2D discretized digital image, f(x,y)≥0, p+q The order moment M _pq and central moment μ _pq are defined as:

In the above formula, (i _c , j _c ) are the coordinates of the center of mass, and

Therefore, the centroid coordinates of the Mark point of the two-dimensional image and the three-dimensional image are obtained from the above formula, which are (i ₁ , j ₁ ) and (i ₂ , j ₂ );

S2.1.4, image coordinate transformation Mark point conversion formula;

According to the coordinates of the Mark point, after coordinate conversion, the coordinates of the detected object in the two-dimensional image are converted into the three-dimensional image. If the position of the coordinate point of the converted three-dimensional image is the position of the detected object, the calibration between the plane camera and the 3D camera is completed;

S2.2, Experimental results;

S2.2.1, Mark point centroid coordinates;

Use the centroid formula to calculate the centroid of the two-dimensional image Mark point and the three-dimensional image Mark point;

S2.2.2, the centroid coordinates of the bird's nest impurity area;

Create two empty arrays R and C, which are used to store the row coordinates and column coordinates of the centroid of the feather area respectively; 20 bird's nest feather impurity areas are detected in the 2D bird's nest image, and the centroid coordinates of the area are (R, C);

S2.2.3, coordinates of bird's nest impurity area;

Create two empty arrays Rows and Columns, which are used to store the row coordinates and column coordinates of the centroid of the feather area respectively; 20 bird's nest feather impurity areas are detected in the 2D bird's nest image, and the area coordinates are (Rows, Columns);

Among them, the coordinates of the 3D bird's nest feather impurity area can be calculated according to the Mark point conversion formula, and two empty arrays Rows3m and Columns3m are created to store the row coordinates and column coordinates of the centroid of the feather area in the 3D bird's nest image;

S2.2.4, 3D model of bird's nest impurities;

Use the acquired 3D coordinates (Row3m, Colm3m, Z) of the bird's nest feather impurity area to generate a 3D model map of the bird's nest impurity;

You can see the specific location and size of the bird’s nest impurities, which is convenient for picking out the bird’s nest impurities; but this is only the impurities on the surface of the bird’s nest, and the feather impurities that are not exposed on the surface of the bird’s nest are not detected; so the bird’s nest that has been sorted on the surface can be repeated The method of combining the area array camera and the 3D camera is used to collect the image of the impurities inside the bird's nest, and then the image is processed by the same method as above to sort out the impurities in the bird's nest feathers.

Preferably, in the step S1.4.2, the linear extension algorithm process of the major semiaxis of the smallest circumscribed ellipse is as follows:

Step1: Obtain the minimum circumscribed ellipse of the feather area belonging to the second category, so as to obtain the major semiaxis a of the minimum circumscribed ellipse of each area;

Step2: Starting from the center of mass (R, C) outside the area, and ending at the minimum distance point (i ₁ , j ₁ ) from the center of mass to the area, connect the two points to obtain a line segment L;

Step3: Take the center of mass to the minimum distance point (i ₁ , j ₁ ) as the starting point, draw the extension line of straight line L, and the extension length is a to obtain a new straight line M;

Step4: Calculate the intersection point (m, n) of the new straight line M and each feather area, calculate the point (m, n) and the center point (p, q) of the centroid to the minimum distance point (i ₁ , j ₁ ) of the area, the point It is the point after correction.

Preferably, the calibration process in the step S2.1.4 is as follows:

Step1: Find out the coordinates of the 2D image Mark point (i ₁ , j ₁ ) and the coordinates of the 2D image impurity area (Rows, Columns);

Step2: Calculate the line and column distances between the Mark point coordinates in the 2D image and the coordinates of the 2D bird's nest impurity area:

Row=Rows-i1

Col=Columns-j1, (4.6)

Step3: Calculate the image length ratio and width ratio between the 2D image and the 3D image:

(L _2D is the length of 2D image, L _3D is the length of 3D image)

Step4: Use the Mark points to calculate the row and column coordinates of the 3D bird's nest impurity area:

Row3m=KL*i2+Row

Colm3m=KW*j2+Col, (4.8)

Steps5: The obtained Row3m and Colm3m are the two-dimensional coordinates of each feather area in the three-dimensional image;

Steps6: Each two-dimensional coordinate point in the three-dimensional image corresponds to a fixed height value z, and the three-dimensional coordinates (Row3m, Colm3m, Z) of each feather area can be obtained from the 3D camera;

Here, set an allowable maximum deviation σ=0.5mm, if the three-dimensional coordinates (Row3m, Colm3m, Z) are found to be within the allowable maximum deviation, the camera calibration is completed; otherwise, re-correct the distortion of the camera lens, and then return to Step1 Restart the calculation, and since the accuracy of the allowable maximum deviation is 0.1mm, in order to reduce the calculation error, the accuracy of the recalculation result is retained at 0.01mm.

Compared with the prior art, the present invention has the following beneficial effects:

The invention can improve the work efficiency of bird’s nest feather impurities sorting, and effectively reduce the production cost of bird’s nest; it has higher precision than manual picking of bird’s nest products, and can carry out long-term and stable bird’s nest feather and impurity sorting work; after introducing the method of the invention, the bird’s nest The quality of the product can reduce the missed detection rate and false detection rate of bird’s nest products, and obtain reliable, stable and accurate detection of bird’s nest products; at the same time, it can reduce the labor intensity of workers and reduce the damage to workers’ eyes, cervical spine and physical and mental health; it can also improve Improve work efficiency, reduce labor costs, and greatly enhance the competitiveness of enterprises in the market.

Description of drawings

Fig. 1 is bird's nest feather impurity identification and location flowchart of the present invention;

Fig. 2 is the original image of 2D bird's nest of the present invention;

Fig. 3 is the original image of 3D bird's nest of the present invention;

Fig. 4 is a schematic diagram of the median filtering algorithm of the present invention;

Fig. 5 is a schematic diagram of the gray scale segmented linear transformation of the present invention;

Fig. 6 is the 2D bird's nest figure after the threshold segmentation background of the present invention;

Fig. 7 is the improved median filtering bird's nest figure of the present invention;

Fig. 8 is the schematic diagram of bird's nest impurity region of the present invention;

Fig. 9 is the schematic diagram of centroid point of 2D bird's nest feather impurity of the present invention;

Fig. 10 is a schematic diagram of the 3D bird's nest impurity area of the present invention;

Fig. 11 is the three-dimensional bird's nest impurity depth image processing flowchart of the present invention;

Fig. 12 is a schematic diagram of the three-dimensional height value of the bird's nest impurity centroid of the present invention;

Figure 13 is a schematic diagram of the 2D Bird's Nest Mark point area of the present invention;

Figure 14 is a schematic diagram of the 3D Bird's Nest Mark point area of the present invention;

Fig. 15 is a schematic diagram of the two-dimensional Mark point centroid of the present invention;

Fig. 16 is a schematic diagram of the center of mass of the three-dimensional Mark point of the present invention;

Fig. 17 is a 3D model diagram of bird's nest impurities generated by the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

One, a kind of bird's nest impurity sorting method that merges 2D and 3D images of the present invention is aimed at the sorting problem of bird's nest feather impurity, takes bird's nest as detection object, proposes the working mode of the combination of area array camera and 3D camera, utilizes the area array camera The advantage is to sort out the impurities in the bird's nest feather area, and then use the 3D camera to obtain the height value of the bird's nest impurity map in the 3D image to form the three-dimensional information of the impurity area, and then combine the obtained three-dimensional information of the bird's nest feather impurities to generate a three-dimensional model of the bird's nest feather impurity area, which is convenient for processing Picking of impurities in bird's nest.

As shown in Figures 1 to 17, specifically:

2. Fusion method of 2D image and 3D image.

Since the bird's nest and its feather impurities are irregular in shape and uneven, it is necessary to use three-dimensional coordinates to describe the position of the impurities. According to the shape and grayscale characteristics of the bird's nest and its feather impurities, after repeated experiments, a working method of combining an area array camera and a 3D camera is proposed, and the advantages of the area array camera are used to sort the impurities in the bird's nest feather area, and then obtained by the 3D camera The height value of the bird's nest impurity map in the 3D image forms the three-dimensional information of the impurity area. As shown in Figure 1, it is the flow chart of processing the bird's nest feather impurities of the invention.

The present invention takes bird's nest as the detection object, and identifies and locates bird's nest feather impurities based on machine vision technology, and mainly completes the following tasks:

(1) Aiming at the problem that the bird’s nest and its feather impurities are irregular in shape and uneven, it is necessary to use three-dimensional coordinates to describe the position of the impurities. The present invention proposes a working method combining an area array camera and a 3D camera. Identify the impurities in the bird's nest feather area, and then obtain the image height value through the 3D camera to obtain the three-dimensional information of the impurity area;

(2) For the identification of feather impurities, after preprocessing the collected images, a gray level-based secondary iterative selection threshold method is proposed to highlight the feather impurity area, and then the feather impurities are further accurately identified through shape feature extraction. Experimental results show that its recognition accuracy is high, and the missed detection rate is low, which is far better than the manual detection method, and meets the design requirements of the present invention;

(3) For the positioning and detection of the impurity area, the identified feather area is classified by the Euclidean distance method, and the linear extension method of the minimum circumscribed ellipse major axis of the area is proposed to correct the centroid of the area falling outside the feather area, and a Mark point-based The positioning matching method, combined with the 3D camera, can obtain the height value of each point in the image, and obtain the three-dimensional information of the feather impurity area. This method is simple in calculation, high in efficiency, and the positioning error is within 0.5mm;

(4) Finally, experiment and analyze the algorithms proposed in the present invention. Experimental results show that the method proposed in the present invention has high detection accuracy, low missed detection rate, false detection rate is lower than 4%, time consumption is far less than manual detection method, and is not interfered by human factors, etc., and all aspects of performance have reached the expected effect And the actual production and processing requirements.

3. Reconstruction and identification method of impurities in bird's nest.

The present invention first carries out image preprocessing, as can be seen from Fig. 2 and Fig. 3, the background of the original image of bird's nest is more complicated, so first remove the image background, filter out the image noise by median filtering, utilize the difference in gray scale of bird's nest and its feather impurities, Use linear segment transformation to enhance the contrast between bird's nest area and background, and then propose an automatic threshold method based on gray level secondary iteration selection to separate the background; extract the Mark point area from the image after background removal, and then calculate the centroid coordinates of the Mark point ; Due to the preliminary segmentation of the feather impurity area mixed with a small amount of false detection area, then, the feather impurity feature selection and feature matching method are proposed to remove the false detection area and the feather impurity area is sorted out again; for feather impurity positioning, the present invention proposes Mark points The positioning method converts the coordinates of the feather impurity area in the two-dimensional image to the three-dimensional image, and uses the 3D camera to obtain the height value of each point in the image to obtain the three-dimensional information of the feather impurity area, and finally uses the three-dimensional coordinates of the feather impurity area.

Generate a 3D bird's nest impurity model, which is convenient for picking out bird's nest impurities.

3.1, image preprocessing;

Image preprocessing is a very important link in the bird's nest recognition process. Image acquisition is often disturbed by various noises and the surrounding environment, which not only affects the effect of the image, but also makes the required relevant information often submerged in it, which brings inconvenience to the subsequent feature extraction. In order to filter out noise interference, improve image quality, and highlight required relevant information, relevant image preprocessing is required before identification and detection of bird's nest impurities. Among them, commonly used image preprocessing algorithms include: image filtering, image enhancement, image segmentation, morphological processing, etc. The image preprocessing algorithm is adopted, which not only makes the post-processing of the object easier, but also achieves better results.

3.1.1, image filtering;

The function of image filtering is to preserve the detailed features of the collected images as much as possible, and eliminate or weaken the useless information mixed into the target image. Image filtering is a processing method that can enrich the amount of image information, improve image quality, and enhance the effect of image recognition. Its processing effect directly affects the subsequent image processing process, and is closely related to the effectiveness and reliability of the feature recognition link. An essential step in the preprocessing process.

In the process of bird's nest image transmission and processing, it is often polluted by various noises, and dark spots or bright spots of the image appear, which not only reduces the image quality, but also affects the accuracy of feature extraction in image processing. Therefore, an effective image filtering algorithm is usually selected to solve the impact of noise. Commonly used image filtering algorithms include frequency domain filtering, mean filtering in the spatial domain, and median filtering.

Median filtering is a nonlinear signal processing method. Essentially, median filtering is a statistical sorting filter. For the point (i, j) in the original image, the median filter takes the statistical ranking median of all pixels in the area centered on this point as the response of point (i, j). Compared with frequency domain filtering and mean filtering algorithms, median filtering not only has a fast operation speed, but also has a very good filtering effect on isolated noise pixels (such as impulse noise, salt and pepper noise), and the processed image is relatively clear, and it can effectively preserve the image quality. Useful marginal information.

The median filtering algorithm is a neighborhood operation. The calculation process is to arrange the corresponding values on the template in ascending order when performing median filtering on the image, and then assign the median value of this column of data to the center position of the template. of pixels. Among them, if the module has an odd number of points, the gray value of the middle pixel arranged in order of size is used as the middle value; when the template has an even number of points, the gray value of the middle pixel arranged in order of size is placed in the middle The average value of the value is used as the intermediate value, and its realization method is shown in Figure 4. In practical applications, it is necessary to select the shape and size of the template based on the actual situation.

Since the median filtering effect depends on the size of the filtering window, if it is too large, the edge will be blurred, and if it is too small, the denoising effect will be unsatisfactory. The present invention improves the median filtering algorithm as follows: scan the image line by line, and judge whether the pixel is the maximum or minimum value of the neighborhood pixels covered by the filter window when processing each pixel; if so, use the normal The median filter algorithm processes the pixel; if not, it ignores it. The present invention adopts an improved 3×3 median filtering algorithm for image filtering, and the effect after utilizing the improved median filtering is shown in FIG. 7 .

3.1.2, image enhancement;

The method of using the piecewise linear transformation function to enhance the contrast of the image is actually to enhance the contrast between the parts of the original image, that is, to enhance the gray-scale areas of interest in the input image, and relatively suppress those gray-scale areas that are not of interest. The main advantage of piecewise linear transformation is that its form can be synthesized arbitrarily.

The form of the piecewise linear transformation function is as follows:

(x ₁ , x ₂ ) and (y ₁ , y ₂ ) are the main parameters in formula (3.1). According to the description of the algorithm function, it can be seen that x ₁ and x ₂ limit the range of gray levels that the processing object needs to convert, while y ₁ and y ₂ determine the slope of the linear transformation.

When x ₁ , x ₂ , y ₁ , and y ₂ take different combinations of values, the obtained transformation effects are different. The graph of its piecewise linear transformation function is shown in Figure 5.

3.2, image segmentation;

Image segmentation is to divide an image into several specific regions with unique characteristics, and extract objects of interest from them, usually with similarity and discontinuity.

The present invention selects a threshold value segmentation method, and separates the background and the object by selecting the optimal threshold value to perform image segmentation. The threshold segmentation method is to judge the image by setting a reasonable threshold, and set the gray value of those parts that meet the set threshold range to 0, otherwise set to 1, so as to separate the target of interest from the image, and Generate a binary image. Image threshold segmentation is an indispensable and important link in image processing.

Threshold segmentation is to transform the input f of the image into the output g, and the transformation is as follows:

In the above formula, T is the set threshold, g(i, j)=0 represents the image element of the background part, and g(i, j)=1 represents the image element of the target object part (and vice versa). Threshold segmentation is to scan all the pixels of the image f, if f(i,j)≥T, then the element g(i,j) of the segmented image is the pixel of the object, otherwise it is the background pixel.

The effect of threshold segmentation is shown in Figure 6.

3.3, feature selection and feather impurity area extraction;

After the various regions of interest are obtained through image segmentation, some simple region descriptors can be used as the features representing the region. These regional features are usually combined into feature vectors for classification. Common simple area descriptors such as perimeter, area, compactness, centroid of the area, gray mean, gray median, minimum rectangle containing the area, minimum or maximum gray level, number of pixels greater or less than the mean, and Pull number and so on.

Carry out bird's nest feather area by feature selection, the present invention selects the empirical value of feature and sets LW value as 4, S is 3.8, and area Area is set at (800,10000). The recognition results are shown in Figure 8, where a is the original 2D bird's nest image, and b is the bird's nest impurity area.

3.4, Positioning of bird's nest feather impurity area;

(1) Classification of feather impurity areas. When locating the identified feather impurities, it needs to be classified and corrected. Using the Euclidean distance to judge the value d between the centroid of each feather impurity area and the minimum distance point of its own area, each feather impurity area can be divided into two categories. The centroid coordinates of the feather impurity area are calculated as (R, C) through the centroid formulas (4.4) and (4.5), and the Euclidean distance formula is as follows:

If d=0, the feather region belongs to the first category, that is, the centroid falls inside the feather region; if d≠0, the feather region belongs to the second category, and the centroid falls outside the feather region. Wherein the first type of centroid falls in the feather area is exactly what the present invention seeks, and the second type of feather area needs to be corrected, and the centroid that falls outside the second type of feather area will be passed through the correction algorithm to obtain the new falling in the feather area. point (i _c , j _c ).

(2) Center of mass correction. In order to correct the outer centroid of the feather region, the present invention introduces the linear extension method of the major semiaxis of the smallest circumscribed ellipse. The algorithm process is as follows:

Step1: Obtain the minimum circumscribed ellipse of the feather area belonging to the second category, so as to obtain the major semiaxis a of the minimum circumscribed ellipse of each area;

Step2: Starting from the center of mass (R, C) outside the area, and ending at the minimum distance point (i ₁ , j ₁ ) from the center of mass to the area, connect the two points to obtain a line segment L;

Step3: Take the center of mass to the minimum distance point (i ₁ , j ₁ ) as the starting point, draw the extension line of straight line L, and the extension length is a to obtain a new straight line M;

Step4: Calculate the intersection point (m, n) of the new straight line M and each feather area, calculate the point (m, n) and the center point (p, q) of the centroid to the minimum distance point (i ₁ , j ₁ ) of the area, the point It is the point after correction.

The centroid points of each feather impurity area are shown in Figure 9.

Select the bird’s nest impurity area, and then generate 2D bird’s nest impurity area coordinates, as shown in Figure 8(b), and then convert the 2D impurity area coordinates into 3D bird’s nest impurity area coordinates according to the Mark point conversion formula to generate 3D bird’s nest impurity coordinates, Generate a 3D bird's nest impurity area based on these coordinate points. The generation process is shown in Figure 10, where the 2D black frame in a is the original bird’s nest impurity area, b is the 2D gray frame is the bird’s nest impurity area, and c is the 3D bird’s nest impurity area.

3.5, 3D bird's nest impurities reconstruction;

The stereoscopic registration of the 2D image of the area array camera and the 3D image of the depth camera is to use the spatial geometric coordinate transformation relationship to find the corresponding relationship of pixel coordinates between the two images. First, extract the Mark area and the centroid coordinates of the Mark area from the 2D bird’s nest image of the area array camera and the bird’s nest image of the 3D camera; Coordinates of the center of mass; then calculate the bird's nest feather impurity feature point area in the matched 3D bird's nest image according to the Mark point conversion formula; finally complete the image matching and generate a 3D model.

3.5.1, Acquisition of bird's nest feather impurity point cloud data and bird's nest impurity model reconstruction

According to the 3D bird's nest impurity area generated in Figure 10, the original image of the 3D bird's nest is segmented accordingly to generate an image of the designated bird's nest feather impurity area, and then the 3D bird's nest feather impurity area image is decomposed into X, Y, Z containing three-dimensional coordinates For the image of the coordinate point information, by converting the three-dimensional point X, Y, Z images into a 3D bird’s nest feather impurity point cloud, only the discrete three-dimensional feature points on the bird’s nest impurity surface are obtained. In order to reconstruct the bird’s nest impurity surface, it is necessary to Perform triangulation, and finally reconstruct the surface of the bird's nest feather impurities. The reconstruction process of bird's nest feather impurities is shown in Figure 11.

3.5.2, Feature identification of bird's nest feather impurities;

Use the Mark point formula to obtain the corresponding relationship between the bird's nest images of the two area array cameras and the bird's nest image of the 3D camera, and obtain the three-dimensional representation of all points in the bird's nest impurity area in the 3D bird's nest image.

According to the 3D bird's nest impurity area generated above, the original 3D bird's nest image is reduced to the designated bird's nest feather impurity area image, and then its 3D bird's nest feather impurity area image is decomposed into x, y, z coordinate images containing 3D points, where z The image is a height image. Taking the z image as the processing object, the centroid formula is used to obtain the 3D bird's nest feather impurity coordinates (x, y). Each two-dimensional coordinate point in the three-dimensional image corresponds to a fixed height value Z. Since the gray value of the z image is the height value Z of the image, we can first find the coordinates of the center of mass in the z image as the center of the circle to take the smallest ring, the smallest ring + 30 pixels, and the smallest ring + The gray value mean and deviation Deviation between the circular difference values of 60 pixels can be described by the following formula:

where, height Z:

Z=Mean-Mean ₁ (3.6)

Thus, the three-dimensional coordinates (Row3m, Colm3m, Z) of each feather area can be obtained from the 3D camera. Figure 12 shows the height values calculated according to formulas (3.4) (3.5).

The main idea of the recognition method based on 3D features is: use the 3D camera to obtain the depth information of the bird’s nest image, and then obtain the 3D point cloud model from the depth information, and then extract the 3D feature descriptors such as target size, shape, boundary from the 3D point cloud model etc., and finally use these 3D features for target recognition. The method based on 3D features has high recognition accuracy and robustness, and can realize simultaneous recognition of multiple targets.

4. Experiment.

4.1, camera calibration;

The present invention uses a combined method of an area array camera and a 3D camera to collect images, and traditional camera calibration methods cannot meet the calibration requirements of the area array camera and the 3D camera. After repeated experiments, the present invention proposes a Mark point method for calculation between two-dimensional image coordinates and three-dimensional image coordinates, which can realize positioning requirements without traditional camera calibration methods. Since most camera lenses have certain camera distortion, this section first analyzes the camera distortion correction.

4.1.1, distortion correction;

Generally, the camera lens used in the vision system has different degrees of distortion. The distance between the pixel and the center will affect the degree of image distortion. The closer the pixel is to the center of the image, the smaller the distortion will be. This distortion is of the nonlinear type and can be described by the following formula:

In the formula, Represents the ideal coordinates of pixels that conform to the linear imaging model without distortion, (x, y) represents the actual image point coordinates, δ _x and δ _y are nonlinear distortion values, which are related to the position of the image point in the image, and can be expressed by the following formula:

Among them, the first item of δ _x or δ _y is radial distortion (Radial distortion), the second item is centrifugal distortion (Centrifugal distortion), and the third item is thin prism distortion (Thin prism). The coefficients in the formula are called nonlinear distortion parameters. According to the research, introducing too many nonlinear parameters will sometimes cause the instability of the solution and affect the improvement of accuracy. Usually, non-linear distortion is sufficient to be described by the first term radial distortion in the above equation. Therefore, formula (4.2) can be simplified as:

It can be clearly seen that the distortion increases as the radial radius increases, that is, the part far from the center of the image is more severely distorted.

4.1.2, Mark point selection;

After repeated experiments, the present invention converts the coordinates of the detected object in the two-dimensional image into the three-dimensional image through the Mark point, and judges whether the converted three-dimensional coordinates are the position of the detected object to realize the camera calibration target.

The present invention uses a new way of combining an area array camera and a 3D camera to collect images, and the selection of Mark points is particularly important, which directly affects the positioning accuracy of the system. In order to improve the recognition accuracy and speed, the mark point should be selected as a circle or rectangle with uniform color and regular shape, and it should have a certain height (flush with the surface of the bird’s nest) for easy recognition. The present invention selects Mark points from two aspects of geometric shape and color, and the circle and ellipse areas in Fig. 13 and Fig. 14 are Mark points of the present invention.

The Mark point and the detected object have the characteristics of obvious differences in grayscale and shape. The present invention first uses the grayscale threshold method to segment the area where the Mark point is located, and then uses the feature extraction method to identify the Mark point.

4.1.3, Mark point centroid coordinate calculation;

The mark points in the two-dimensional image and the three-dimensional image are recognized respectively by using the above-mentioned mark point recognition method, and then the centroid coordinates of the respective mark points are calculated. For a 2D discretized digital image, f(x,y)≥0, p+q order moment M _pq and central moment μ _pq are defined as:

where (i _c , j _c ) are the coordinates of the center of mass, and

Calculate the centroid coordinates of the Mark point of the 2D image and the 3D image from the above formula, which are (i ₁ , j ₁ ) and (i ₂ , j ₂ ), respectively.

Among them, the center of mass of the two-dimensional Mark point and the center of mass of the three-dimensional Mark point are respectively shown as the circle center in Figure 15 and the black point at the center of the ellipse in Figure 16 .

4.1.4, image coordinate transformation Mark point conversion formula;

According to the coordinates of the Mark point, the coordinates of the detected object in the two-dimensional image are transformed into the three-dimensional image. If the coordinate point of the converted three-dimensional image is the position of the detected object, the calibration between the plane camera and the 3D camera is completed. The calibration process is as follows:

Step1: Find out the coordinates of the 2D image Mark point (i ₁ , j ₁ ) and the coordinates of the 2D image impurity area (Rows, Columns);

Step2: Calculate the line and column distances between the Mark point coordinates in the 2D image and the coordinates of the 2D bird's nest impurity area:

Row=Rows-i1

Col＝Columns-j1 (4.6)

Step3: Calculate the image length ratio and width ratio between the 2D image and the 3D image:

(L _2D is the length of 2D image, L _3D is the length of 3D image)

Step4: Use the Mark points to calculate the row and column coordinates of the 3D bird's nest impurity area:

Row3m=KL*i2+Row

Colm3m=KW*j2+Col (4.8)

Steps5: The obtained Row3m and Colm3m are the two-dimensional coordinates of each feather area in the three-dimensional image;

Steps6: Each two-dimensional coordinate point in the three-dimensional image corresponds to a fixed height value z, so that the three-dimensional coordinates (Row3m, Colm3m, Z) of each feather area can be obtained from the 3D camera.

Here, set a maximum allowable deviation σ=0.5mm. If the calculated three-dimensional coordinates (Row3m, Colm3m, Z) are within the allowable maximum deviation, the camera calibration is completed. Otherwise, re-correct the distortion of the camera lens, and then return to step 1 to restart the calculation, and since the accuracy of the maximum deviation allowed is 0.1mm, in order to reduce the calculation error, the accuracy of the recalculation result is retained at 0.01mm.

4.2, Experimental results;

4.2.1, Mark point centroid coordinates;

Use the centroid formula to calculate the centroids of the two-dimensional image Mark point and the three-dimensional image Mark point respectively (see Table 4-1 below):

Table 4-1, Mark point centroid coordinates

Mark point centroid coordinates 2D(i1,j1) (469.539, 976.531) 3D(i2, j2) (172.196, 1352.08)

4.2.2, the centroid coordinates of the bird's nest impurity area;

Create two empty arrays R and C, which are used to store the row coordinates and column coordinates of the centroid of the feather area respectively.

In the 2D bird's nest image, 20 bird's nest feather impurity areas are detected, and the coordinates of the center of mass of the area are as follows (R, C), where:

R＝[1159.88,1179.29,1407.11,1712.31,1708.69,1752.07,2205.47,2275.48,2215.88,2304.67,970.077,986.445,1005.75,1397.07,1445.52,1494.47,1534.48,1568.59,1669.79,2103.62,2224.48]；

C＝[930.488,836.064,1724.77,857.697,1532.5,1157.49,1510.98,1126.13,1429.93,840.633,1156.68,1250.89,1079.05,707.473,417.415,557.804,274.456,359.435,265.062,755.561,599.084]；

4.2.3, coordinates of bird’s nest impurity area;

Create two empty arrays Rows and Columns, which are used to store the row coordinates and column coordinates of the centroid of the feather area respectively.

In the 2D bird's nest map, 20 bird's nest feather impurity areas are detected, and the area coordinates are greater than 70,000. Therefore, 50 coordinates of one of the areas are extracted below, as shown below (Rows, Columns), where:

Rows＝[949,949,949,949,949,949,950,950,950,950,950,950,950,950,950,950,950,950,950,950,950,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,951,952,952,952,952,952,952,952,952]；

Columns＝[1138,1139,1171,1172,1173,1174,1135,1136,1137,1138,1139,1140,1141,1170,1171,1172,1173,1174,1175,1176,1177,1135,1136,1137 ,1138,1139,1140,1141,1142,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1134,1135,1136,1137,1138,1139,1140 ,1141]

Among them, the coordinates of the 3D bird’s nest feather impurity area can be calculated according to the Mark point conversion formula, and two empty arrays Rows3m and Columns3m are created to store the row coordinates and column coordinates of the centroid of the feather area in the 3D bird’s nest image respectively.

4.2.4, 3D model of bird's nest impurities;

Using the acquired 3D coordinates (Row3m, Colm3m, Z) of the bird’s nest feather impurity area, generate the bird’s nest impurity 3D model diagram shown in Figure 17, where a is the original image of the bird’s nest impurity 3D model, and b is the right side of the bird’s nest impurity 3D model Turn 90°, c is the 3D model of bird’s nest impurities turned 90° to the left.

From Figure 17, we can see the specific location and size of the impurities in the bird’s nest, which is convenient for picking out the impurities in the bird’s nest. But this is only the impurities on the surface of the bird's nest, and the feather impurities that are not exposed on the surface of the bird's nest are not detected. So this can reuse the method of combining the surface array camera and 3D camera to collect the image of the bird's nest internal impurities, and then process the image in the same way as above to sort out the bird's nest feather impurity area.

4.3, Experimental analysis;

The present invention has carried out relevant experiments on the various processes of bird's nest feather impurity identification and detection algorithms. Through the verification and analysis of the experiment, it is proved that the method of fusion of 2D and 3D image bird's nest impurities sorting has high detection accuracy, and the performance can achieve the expected effect and the actual bird's nest Processing requirements.

The invention can improve the work efficiency of bird’s nest feather impurities sorting, and effectively reduce the production cost of bird’s nest; it has higher precision than manual picking of bird’s nest products, and can carry out long-term and stable bird’s nest feather and impurity sorting work; after introducing the method of the invention, the bird’s nest The quality of the product can reduce the missed detection rate and false detection rate of bird’s nest products, and obtain reliable, stable and accurate detection of bird’s nest products; at the same time, it can reduce the labor intensity of workers and reduce the damage to workers’ eyes, cervical spine and physical and mental health; it can also improve Improve work efficiency, reduce labor costs, and greatly enhance the competitiveness of enterprises in the market.

The above is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above content, and any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention are all Replacement methods that should be equivalent are all included within the protection scope of the present invention.

Claims (3)

1. a kind of bird's nest impurity method for sorting for merging 2D and 3D rendering, which is characterized in that include the following steps:

S1, the reconstruct and identification of bird's nest impurity；

S1.1, image preprocessing；

Image preprocessing, is a very important link in bird's nest identification process, when Image Acquisition be subjected to various noises and The interference of surrounding enviroment not only affects the effect of image, but also required relevant information is often submerged wherein, to subsequent Feature extraction make troubles；For filter out noise jamming, improving image quality, it is prominent needed for relevant information, to bird's nest impurity It needs to do relevant image preprocessing before identification and detection；

S1.1.1, image filtering；

In bird's nest image transmitting and treatment process, often by various noise pollutions, there is the dim spot of image or bright spot interference, dropping While low image quality, the accuracy of feature extraction in image procossing is had an effect on；Therefore, effective Image filter arithmetic is chosen Solving noise bring influences, and common Image filter arithmetic has: mean filter method and intermediate value in frequency domain filtering method, spatial domain Filter method；

The median filtering algorithm is a kind of neighborhood operation, and calculating process is when carrying out median filter process to image, by template On respective value by ascending sequence arrange, then the median of this column data is assigned to the pixel of template center position； Wherein, if template has odd number point, the gray value of the intermediary image vegetarian refreshments after sequence by size is arranged is as median；If template There is even number point, the gray value after sequence arranges by size is located in the middle the average value of two values as median；

It is too big to make edge blurry since median filtering effect depends on the size of filter window, it is too small, it denoises effect and pays no attention to Think, then improve to median filtering algorithm: progressive scanning picture judges whether the pixel is filter when handling each pixel Wave window covers the maximum value or minimum value of lower neighborhood territory pixel；If so, just handling the pixel using normal median filtering algorithm； If it is not, then not managing it；

Image filtering is carried out using improved 3 × 3 median filtering algorithm；

S1.1.2, image enhancement；

Enhance picture contrast using piecewise linear transform function, actually contrast between enhancing original image each section, that is, enhances Interested gray areas in input picture, it is opposite to inhibit those uninterested gray areas；

The form of the piecewise linear transform function is as follows:

Wherein, (x₁,x₂) and (y₁,y₂) it is major parameter in above formula (3.1), it is described according to algorithmic function, it is known that x₁And x₂It is It limits and deals with objects the grey level range that needs are converted, and y₁And y₂Then determine the slope of linear transformation；

S1.2, image segmentation；

Image segmentation uses thresholding method, is separated background and object by choosing optimal threshold, to carry out image point It cuts；Image judgement is carried out by one reasonable threshold value of setting, the gray value for meeting those of given threshold range part is set It is 0, otherwise is set as 1, so that interesting target be separated from image, and generates bianry image；

Threshold segmentation is that the input f of image is transformed to output g, is converted as follows:

In above formula, T is given threshold, and g (i, j)=0 indicates that the pictorial element of background parts, g (i, j)=1 indicate target object Partial pictorial element；Threshold segmentation is that is, scan its all pixels to image f, if f (i, j) >=T, the member of image after segmentation Plain g (i, j) is exactly the pixel of object；It otherwise is exactly background pixel；

S1.3, feature selecting and feather extrinsic region extract；

After obtaining various interested regions by image segmentation, it can use sub be used as of simple region description and represent the region Feature, and by these provincial characteristics be combined into feature vector for classification use；

Wherein, it is perimeter, area, compactness, the mass center in region, gray average, gray scale intermediate value, packet that the simple region, which describes son, Minimum rectangle, minimum or maximum gray scale containing region, pixel number and Euler's numbers more than or less than mean value；

S1.4, the positioning of bird's nest feather extrinsic region；

When to the feather impurity positioning identified, need to classify to it and correct；

S1.4.1, the classification of feather extrinsic region；

Judge that each feather extrinsic region mass center, can be miscellaneous by each feather to value d between self zone minimum range point using Euclidean distance Matter region is divided into two classes；Finding out feather extrinsic region center-of-mass coordinate by mass center formula (4.4), (4.5) is (R, C), then European Range formula is as follows:

If d=0, which belongs to the first kind, i.e. mass center is fallen in feather region；If d ≠ 0, which belongs to In the second class, i.e. mass center is fallen in outside feather region；

Wherein, it is exactly required to fall in feather region for first kind mass center, and the second class feather region then needs to be modified, and will fall in the Mass center obtains the new point (i fallen in feather region by correction algorithm outside two class feather regions_c,j_c)；

S1.4.2, mass center amendment；

In order to correct mass center outside feather region, the straight line for introducing minimum external oval major semiaxis extends algorithm；

S1.5,3D bird's nest impurity are rebuild；

It is to find two using space geometry coordinate transformation relation that area array cameras 2D image is registrated with the solid of depth camera 3D rendering The corresponding relationship of pixel coordinate between width image；First respectively to the bird's nest of the 2D bird's nest image of area array cameras and 3D camera The center-of-mass coordinate in the region image zooming-out Mark and the region Mark；Then feather is carried out to two width or multiple image of area array cameras again Impurity characteristics are extracted, and the center-of-mass coordinate of bird's nest feather extrinsic region is obtained；Then it is found out according to Mark point reduction formula matched Bird's nest feather impurity characteristics point region in 3D bird's nest image；The matching of image is finally completed, 3D model is generated；

S1.5.1, the acquisition and bird's nest impurity Model Reconstruction of bird's nest feather impurity point cloud data；

It is miscellaneous to generate specified bird's nest feather to 3D bird's nest original image by dividing accordingly according to the 3D bird's nest extrinsic region of generation Then its 3D bird's nest feather extrinsic region picture breakdown is X, Y comprising three-dimensional coordinate, Z coordinate point information by matter area image Image, and convert 3D bird's nest feather impure point cloud for three-dimensional point X, Y, Z-image, obtain the three of discrete bird's nest contaminant surface Dimensional feature point；In order to rebuild bird's nest contaminant surface, also trigonometric ratio is carried out to it, finally reconstruct the table of bird's nest feather impurity Face；

S1.5.2, the identification of bird's nest feather impurity characteristics；

The corresponding relationship between two breadth array camera bird's nest images and 3D camera bird's nest image is acquired using Mark point formula, is acquired All bird's nest extrinsic regions three dimensional representation of the point in 3D bird's nest image；

According to the 3D bird's nest extrinsic region of generation, 3D bird's nest original image is reduced to specified bird's nest feather extrinsic region image, so Afterwards again by 3D bird's nest feather extrinsic region picture breakdown be the x comprising 3D point, y, z coordinate image, wherein z image be height map Picture；Using z image as process object, find out 3D bird's nest feather impurity coordinate (x, y) with mass center formula, then it is every in 3-D image One two-dimensional coordinate point all corresponds to a fixed height value Z；Since the gray value of z image is exactly the height value Z of its image, first ask Out using center-of-mass coordinate in z image as the center of circle, taken respectively with bird's nest feather extrinsic region minimum annulus, i.e.+30 pictures of minimum annulus Gray value mean value Mean and deviation D eviation between the circle difference of+60 pixels of vegetarian refreshments and minimum annulus, can be used to lower public affairs Formula description:

Wherein, height Z are as follows:

Z=Mean-Mean₁ (3.6)

So as to obtain the three-dimensional coordinate (Row3m, Colm3m, Z) in each feather region from 3D camera；

S2, experiment；

S2.1, camera calibration；

S2.1.1, distortion correction；

Camera lens used in usual vision system all exist to distort in various degree, and the excentric distance of pixel will affect image Distortion degree, range image center is closer, and distortion is just smaller, and this distortion belongs to non-linear type, can be described with following formula:

In above formula,The distortionless pixel ideal coordinates for meeting linear imaging model are represented, (x, y) represents practical figure Picpointed coordinate, δ_xAnd δ_yIt is nonlinear distortion value, position is related in the picture to picture point for it, it can be indicated with following formula:

Wherein, δ_xOr δ_yFirst item be Radial distortion radial distortion, Section 2 Centrifugal Distortion centrifugal distortion, Section 3 are the distortion of Thin prism thin prism, and the coefficient in formula is known as nonlinear distortion ginseng Number；And nonlinear distortion parameter can cause the unstable of solution when introducing excessive, influence precision raising；It therefore, can be by formula (4.2) Abbreviation are as follows:

Thus it is apparent that, distortion increases as radial radius increases, i.e., fractional distortion remote from picture centre is tighter Weight；

S2.1.2, Mark point are chosen；

By Mark point, detection object coordinate in two dimensional image is transformed into 3-D image, the three-dimensional coordinate after judging conversion It whether is detection object position to realize camera calibration target；

Image Acquisition is carried out using the mode that area array cameras and 3D camera combine, Mark point, which is chosen, has certain altitude, color Uniformly, the circle or rectangle of regular shape；

Mark point and detection object have the characteristics that gray scale and shape difference are obvious, first will be where Mark point with gray threshold method Region segmentation comes out, then is identified Mark point with feature extracting method；

S2.1.3, Mark point center-of-mass coordinate calculate；

The Mark point in two dimensional image and 3-D image is identified respectively using above-mentioned Mark point recognition methods, is then calculated respective Mark point center-of-mass coordinate；To a width 2D discretization digital picture, f (x, y) >=0, p+q rank square M_pqWith central moment μ_pqIs defined as:

In above formula, (i_c,j_c) it is center-of-mass coordinate, and

Therefore, the Mark point center-of-mass coordinate of two dimensional image and 3-D image, respectively (i are found out by above-mentioned formula₁,j₁) and (i₂, j₂)；

S2.1.4, image coordinate convert Mark point reduction formula；

According to Mark point coordinate, to be converted by coordinate, the coordinate of detection object is transformed into 3-D image in two dimensional image, if through The coordinate points position for the 3-D image being converted to is exactly detection object position, then has demarcated between plane camera and 3D camera At；

S2.2, experimental result；

S2.2.1, Mark point center-of-mass coordinate；

Two dimensional image Mark point and 3-D image Mark point mass center are calculated with mass center formula；

S2.2.2, bird's nest extrinsic region center-of-mass coordinate；

Two empty array R and C are created, the row coordinate and column coordinate of storage feather region mass center are respectively intended to；In 2D bird's nest figure Detect 20 bird's nest feather extrinsic regions, region center-of-mass coordinate is (R, C)；

S2.2.3, bird's nest extrinsic region coordinate；

Two empty array Rows and Columns are created, the row coordinate and column coordinate of storage feather region mass center are respectively intended to；In 2D Detect that 20 bird's nest feather extrinsic regions, area coordinate are (Rows, Columns) in bird's nest figure；

Wherein 3D bird's nest feather extrinsic region coordinate can be calculated according to Mark point reduction formula, create two empty arrays Rows3m and Columns3m is respectively intended to the row coordinate and column coordinate of the feather region mass center in storage 3D bird's nest figure；

S2.2.4, bird's nest impurity 3D model；

Using the bird's nest feather extrinsic region 3D coordinate (Row3m, Colm3m, Z) obtained, bird's nest impurity 3D model is generated Figure；

It can be seen that the specific location and size of bird's nest impurity, facilitate bird's nest impurity to pick；But this is bird's nest surface layer Impurity, not detected without the exposed feather impurity on surface inside bird's nest；So can be by sorted table The method that the bird's nest recycling area array cameras and 3D camera of layer combine carries out impurity Image Acquisition inside bird's nest, then to image Above-mentioned same method processing is carried out, bird's nest feather extrinsic region is sorted out.

2. the bird's nest impurity method for sorting of fusion 2D and 3D rendering according to claim 1, which is characterized in that the step It is specific as follows to extend algorithmic procedure for the straight line of minimum external oval major semiaxis in S1.4.2:

Step1: seeking belonging to the minimum external ellipse in the second class feather region, so that it is minimum external oval long by half to obtain each region Axis a；

Step2: with mass center outside region (R, C) for starting point, mass center to region minimum range point (i₁, j₁) it is terminal, connection two o'clock obtains To line segment L；

Step3: with mass center to region minimum range point (i₁, j₁) it is starting point, make straight line L extended line, extending length is that a is obtained newly Straight line M；

Step4: seeking new straight line M and each feather region intersection point (m, n), calculates point (m, n) and mass center to region minimum range point (i₁, j₁) central point (p, q), the point be correct after required point.

3. the bird's nest impurity method for sorting of fusion 2D and 3D rendering according to claim 1, which is characterized in that the step Calibration process is specific as follows in S2.1.4:

Step1: 2D image Mark point coordinate (i is found out₁,j₁) and 2D image extrinsic region coordinate (Rows, Columns)；

Step2: find out in 2D image the line-spacing of Mark point coordinate and 2D bird's nest extrinsic region coordinate and column away from:

Row=Rows-i1

Col=Columns-j1, (4.6)

Step3: the ratio between the ratio between image length between 2D image and 3D rendering and width are calculated:

(L_2DLong, the L for 2D image_3DIt is long for 3D rendering)

Step4: 3D bird's nest extrinsic region row coordinate and column coordinate are calculated using Mark point:

Row3m=KL*i2+Row

Colm3m=KW*j2+Col, (4.8)

Steps5: obtained Row3m, Colm3m are exactly each feather region two-dimensional coordinate in 3-D image；

Steps6: each two-dimensional coordinate point corresponds to a fixed height value z in 3-D image, so that it may from 3D camera To the three-dimensional coordinate (Row3m, Colm3m, Z) in each feather region；

Here, one permission maximum deviation σ=0.5mm of setting, if finding out three-dimensional coordinate (Row3m, Colm3m, Z) is allowing most In large deviation, then camera calibration is completed；Otherwise, distortion correction is carried out to camera lens again, then returns to Step1 and restarts It calculates, and due to allowing the precision of maximum deviation to be 0.1mm, error is calculated to reduce, retains the essence for recalculating result Degree is 0.01mm.