[go: up one dir, main page]

CN113033247B - Image recognition method and device, and computer-readable storage medium - Google Patents

Image recognition method and device, and computer-readable storage medium Download PDF

Info

Publication number
CN113033247B
CN113033247B CN201911252710.2A CN201911252710A CN113033247B CN 113033247 B CN113033247 B CN 113033247B CN 201911252710 A CN201911252710 A CN 201911252710A CN 113033247 B CN113033247 B CN 113033247B
Authority
CN
China
Prior art keywords
cylinder
ith
point
normal
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911252710.2A
Other languages
Chinese (zh)
Other versions
CN113033247A (en
Inventor
张洪伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority to CN201911252710.2A priority Critical patent/CN113033247B/en
Publication of CN113033247A publication Critical patent/CN113033247A/en
Application granted granted Critical
Publication of CN113033247B publication Critical patent/CN113033247B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/422—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

本申请实施例公开了一种图像识别方法及装置、计算机可读存储介质，能够提高图像识别的准确率。该方法包括：采集目标图像，并获取目标图像的点云数据；对点云数据进行法线计算，得到点云数据中的每个点的法线；基于点云数据和每个点的法线，计算第i个圆柱候选参数；其中，i为大于等于1的正整数；对第i个圆柱候选参数进行内外侧检测，得到第i个检测结果；基于第i个检测结果、点云数据和每个点的法线进行迭代搜索处理，确定出目标圆柱参数；基于目标圆柱参数，将目标图像的表面进行平面展开，得到圆柱表面的识别结果。

The embodiment of the present application discloses an image recognition method and device, and a computer-readable storage medium, which can improve the accuracy of image recognition. The method includes: collecting a target image and obtaining point cloud data of the target image; performing normal calculation on the point cloud data to obtain the normal of each point in the point cloud data; calculating the i-th cylinder candidate parameter based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1; performing inside and outside detection on the i-th cylinder candidate parameter to obtain the i-th detection result; performing iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameter; based on the target cylinder parameter, performing a planar unfolding of the surface of the target image to obtain the recognition result of the cylinder surface.

Description

Image recognition method and device and computer readable storage medium
Technical Field
The present application relates to the field of image recognition, and in particular, to an image recognition method and apparatus, and a computer readable storage medium.
Background
At present, a conventional random sampling consensus algorithm (RANSAC, random Sample Consesus) is generally adopted to detect and identify a surface image of a cylindrical object surface, when more objects except for a target cylindrical object exist in the image, as shown in fig. 1, the positions and surface information of other adjacent cylindrical objects around the target cylinder form a cylindrical curved surface of a false detection area in the shape of a cylindrical curved surface together, which influences the surface identification of the target cylinder, or as shown in fig. 2, when the target cylinder is in a background in another cylindrical curved surface shape, the target cylinder is easy to interfere in the image detection, so that false detection is caused, and the image identification accuracy is low.
Disclosure of Invention
The embodiment of the application provides an image recognition method and device and a computer readable storage medium, which can improve the accuracy of image recognition.
The technical scheme of the application is realized as follows:
the embodiment of the application provides an image identification method, which comprises the following steps:
acquiring a target image and acquiring point cloud data of the target image;
performing normal calculation on the point cloud data to obtain a normal of each point in the point cloud data;
Calculating an ith cylinder candidate parameter based on the point cloud data and the normal line of each point, wherein i is a positive integer greater than or equal to 1;
detecting the inner side and the outer side of the ith cylinder candidate parameter to obtain an ith detection result;
Performing iterative search processing based on the ith detection result, the point cloud data and the normal line of each point to determine a target cylinder parameter;
and carrying out plane expansion on the surface of the target image based on the target cylinder parameters to obtain a recognition result of the cylinder surface.
The embodiment of the application provides an image recognition device which comprises an acquisition unit, a calculation unit, a detection unit, a determination unit and a recognition unit, wherein,
The acquisition unit is used for acquiring a target image and acquiring point cloud data of the target image;
the computing unit is used for performing normal calculation on the point cloud data to obtain the normal of each point in the point cloud data;
the calculation unit is further used for calculating an ith cylinder candidate parameter based on the point cloud data and the normal line of each point, wherein i is a positive integer greater than or equal to 1;
the detection unit is used for detecting the inner side and the outer side of the ith cylinder candidate parameter to obtain an ith detection result;
The determining unit is used for carrying out iterative search processing based on the ith detection result, the point cloud data and the normal line of each point to determine a target cylinder parameter;
The recognition unit is used for carrying out plane expansion on the surface of the target image based on the target cylinder parameters to obtain a recognition result of the cylinder surface.
An embodiment of the present application provides an image recognition apparatus, including a processor, a memory, and a communication bus, where the memory communicates with the processor through the communication bus, the memory stores one or more programs executable by the processor, and when the one or more programs are executed, the processor performs the image recognition method as set forth in any one of the above.
Embodiments of the present application provide a computer-readable storage medium storing one or more programs executable by one or more processors to implement the image recognition method as set forth in any one of the above.
The embodiment of the application provides an image identification method, an image identification device and a computer readable storage medium, wherein the method comprises the steps of collecting a target image and acquiring point cloud data of the target image; the method comprises the steps of carrying out normal calculation on point cloud data to obtain normal of each point in the point cloud data, calculating an ith cylinder candidate parameter based on the point cloud data and the normal of each point, carrying out inner and outer side detection on the ith cylinder candidate parameter to obtain an ith detection result, carrying out iterative search processing based on the ith detection result, the point cloud data and the normal of each point to determine a target cylinder parameter, and carrying out plane expansion on the surface of a target image based on the target cylinder parameter to obtain a recognition result of the cylinder surface. By adopting the method, the image recognition device detects the inner side surface and the outer side surface of the candidate cylinder parameters when searching the optimal target cylinder parameters by using the iterative algorithm, thereby realizing further screening of the image recognition result and improving the accuracy of image recognition.
Drawings
FIG. 1 is a schematic diagram of a false detection situation in the prior art;
FIG. 2 is a schematic diagram of a false detection case in the prior art;
FIG. 3 is a schematic diagram of an image recognition system according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 6-1 is a schematic view of a camera optical axis direction vector according to an embodiment of the present application;
FIG. 6-2 is a schematic diagram of a normal constraint provided by an embodiment of the present application;
Fig. 6-3 are a second schematic diagram of normal constraint provided in an embodiment of the present application;
FIG. 7 is a schematic diagram of a point cloud normal vector provided by an embodiment of the present application;
FIG. 8 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 9 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 10 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 11 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 12 is a schematic diagram of exemplary centroid vectors from the center point of the ith cylinder to the center of gravity provided by an embodiment of the present application;
FIG. 13 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 14 is a schematic diagram of exemplary first and second direction values provided by an embodiment of the present application;
FIG. 15 is a schematic flow chart of an alternative image recognition method according to an embodiment of the present application;
FIG. 16-1 is a schematic illustration of a cylindrical object for surface image and text retrieval and recognition provided by an embodiment of the present application;
FIG. 16-2 is a schematic diagram II of a cylindrical object for surface image and text retrieval and recognition provided by an embodiment of the present application;
fig. 17 is a schematic diagram of an image recognition apparatus according to an embodiment of the present application;
fig. 18 is a schematic diagram of a second image recognition device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.
An image recognition system provided in an embodiment of the present application is shown in fig. 3, and includes an image acquisition module 10, a cylinder detection module 20, a validity discriminating module 30, a graphic transformation module 40 and a display module 50. The image capturing module 10 is configured to capture point cloud data Of a target image, and includes an image capturing device capable Of capturing three-dimensional data information, and the image capturing device may be a camera with a Time Of Flight (TOF) sensor, a structured light camera, a binocular camera, etc., and an RGB image capturing device. The cylinder detection module 20 is configured to detect parameters of the included cylinder pattern from the point cloud data of the target image as cylinder candidate parameters by using an image detection algorithm. The validity judging module 30 is configured to perform validity detection on the cylinder candidate parameter detected by the cylinder detecting module 20, where the validity detection may include detecting whether a volume of a cylinder shape corresponding to the cylinder candidate parameter exceeds a threshold, whether a point cloud density covered by the cylinder shape formed by the cylinder candidate parameter is higher than the threshold, whether the cylinder shape formed by the cylinder candidate parameter is a convex cylinder, and so on. The cylinder detection module 20 is further configured to iterate the cylinder candidate parameters that pass the validity detection, and, illustratively, perform optimization iteration from the cylinder candidate parameters that pass the validity detection using a random sampling coincidence algorithm, to finally obtain the target cylinder parameters. The graphic transformation module 40 is configured to fit a cylindrical shape corresponding to the target cylindrical parameter according to the target cylindrical parameter, and spread a side surface in the cylindrical shape corresponding to the target cylindrical parameter from a curved surface to a plane, so as to obtain a recognition result of the target object. The display module 50 is used for displaying the identification result on the display interface.
Based on the image recognition system provided in fig. 3, fig. 4 is a schematic flowchart of an alternative image recognition method provided in an embodiment of the present application, and the steps shown in fig. 4 will be described.
S101, acquiring a target image and acquiring point cloud data of the target image.
The image recognition method provided by the embodiment of the application is suitable for a scene for carrying out surface image recognition and extraction on the target geometric shape in the image, and exemplary, as shown in fig. 3, the notice or description information posted on the surface of a building or commodity belongs to the image and text information on a three-dimensional curved surface. The image recognition method provided by the embodiment of the application can also be used for detecting the scene of the target obstacle.
In the embodiment of the application, an image recognition device firstly collects a target image through image collection equipment and acquires point cloud data of the target image.
In the embodiment of the application, the point cloud data is a three-dimensional coordinate data set of a large number of pixel points forming a target image. The point cloud data may be obtained by using an image acquisition device capable of image three-dimensional information, and the exemplary image recognition device performs image acquisition or scanning on the target image through a camera integrated with a TOF sensor, so as to obtain the point cloud data of the target image, or may be obtained by using a structured light device or a depth camera based on a triangulation ranging technology, which is not limited in the embodiment of the present application.
In the embodiment of the application, the point cloud data of the acquired target image can be the point cloud data stored in advance, or can be the point cloud data acquired from a distance sensor such as a TOF sensor or a laser radar sensor in real time.
In the embodiment of the application, the point cloud data contains the position information of the three-dimensional space of each point in the target image, and the image recognition device can detect the three-dimensional object in the target image by utilizing the point cloud data.
In the embodiment of the application, the image recognition device can perform preliminary filtering on the acquired point cloud data to filter unnecessary information, and particularly, the image recognition device can filter the point cloud data by using a background plane filtering method so as to filter the plane of the background such as the ground and the like to reduce interference items, thereby facilitating subsequent processing.
In the embodiment of the present application, the background plane filtering may include RANSAC plane detection, background plane determination, background plane rejection, etc., which is not limited by the embodiment of the present application.
It should be noted that, in the embodiment of the present application, when the TOF sensor integrated in the camera is used to obtain the point cloud data, the method has the characteristics of being not affected by illumination variation and object textures, and on the premise of meeting the precision requirement, the acquisition cost of the point cloud data can be reduced.
S102, performing normal calculation on the point cloud data to obtain the normal of each point in the point cloud data.
In the embodiment of the application, after the point cloud data of the target image, the image recognition device calculates the normal corresponding to each point in the point cloud data, thereby obtaining the normal of each point in the point cloud data.
In the embodiment of the application, the surface of the three-dimensional object in the three-dimensional space can be regarded as being composed of a plurality of tiny planes, and for a point in the point cloud data of the target image, the normal line of the point is a vector which is always perpendicular to the plane in which the point is located.
In the embodiment of the application, for each point in the point cloud data, the normal line of each point jointly determines the shape of the three-dimensional image corresponding to the point cloud data. Therefore, in order to obtain the degree and direction of curvature of the curved surface of the cylindrical surface, the image recognition apparatus needs to calculate the normal line of each point in the point cloud data to obtain the normal line of each point in the point cloud data.
S103, calculating an ith cylinder candidate parameter based on the point cloud data and the normal line of each point, wherein i is a positive integer greater than or equal to 1.
In the embodiment of the application, after the image recognition device acquires the point cloud data and calculates the normal line of each point in the point cloud data, the image recognition device calculates the ith cylinder candidate parameter based on the point cloud data and the normal line of each point.
In the embodiment of the application, the target image contains a plurality of images with different shapes, and the image recognition device detects the images with the cylindrical shapes possibly contained in the target image by calculating the ith cylindrical candidate parameter based on the point cloud data and the normal line of each point.
In the embodiment of the application, the cylinder candidate parameters may include a cylinder axis vector d, a cylinder center point c and a cylinder radius r, and the image recognition device may use the cylinder candidate parameters to fit a corresponding candidate cylinder shape.
In the embodiment of the application, the three-dimensional coordinate information of each point in the point cloud data represents the position information of the point in space, and the normal line of each point represents the angle information of the plane where the point is located, so that the image recognition device can calculate the axial quantity d of the cylinder, the cylinder center point c and the cylinder radius r as cylinder candidate parameters according to the point cloud data of the surface of the cylinder based on the position information and the angle information.
In the embodiment of the application, i is a positive integer greater than or equal to 1, in order to obtain more accurate cylinder candidate parameters, the image recognition device can randomly select a preset number of points from the point cloud data for iterative computation each time by using a random sampling coincidence algorithm, different cylinder candidate parameters are output in each iterative computation, and for the ith iteration, the image recognition device can calculate the ith cylinder candidate parameters.
S104, detecting the inner side and the outer side of the ith cylinder candidate parameter to obtain an ith detection result.
In the embodiment of the application, for the ith iteration, after the image recognition device obtains the ith cylinder candidate parameter, the inner side and the outer side of the cylinder candidate parameter can be detected, so that the ith detection result is obtained.
In the embodiment of the application, as in the actual scene of shooting the surface of the cylinder, most of the targets for restoring the surface information of the cylinder are convex cylindrical surfaces relative to a photographer, such as the surface trademark of a bottle and a can, propaganda posters on the cylinder and the like. Therefore, the image recognition device can perform inside and outside detection on the ith cylinder candidate parameter calculated in the ith iteration, and judge whether the side surface of the cylinder fitted according to the ith cylinder candidate parameter is convex or concave.
In the embodiment of the application, when the image recognition device judges that the side surface of the cylinder fitted by the ith cylinder candidate parameter is convex, the image recognition device determines that the ith cylinder candidate parameter forms an outer cylinder as an ith detection result.
In the embodiment of the application, when the image recognition device judges that the side surface of the cylinder fitted by the ith cylinder candidate parameter is the concave surface, the image recognition device determines that the inner cylinder formed by the ith cylinder candidate parameter is the ith detection result.
In the embodiment of the application, the image recognition device can also perform validity judgment in other modes on the obtained ith cylinder candidate parameter, and exemplarily judge whether the cylinder volume corresponding to the ith cylinder candidate parameter exceeds a volume threshold value, if so, the validity judgment is not passed, and judge whether the number of points in the point cloud covered in the cylinder fitted according to the ith cylinder candidate parameter is relative to a quantity threshold value, and if so, the validity judgment is not passed.
In the embodiment of the application, for each iteration process, the image recognition device uses the same method to detect the inner side and the outer side of each cylinder candidate parameter to obtain the detection result of each iteration until the iteration condition is met, and the iteration is ended.
S105, performing iterative search processing based on the ith detection result, the point cloud data and the normal line of each point, and determining a target cylinder parameter.
In the embodiment of the application, for the ith iteration, the image recognition device can perform iterative search processing based on the ith detection result, the point cloud data and the normal line of each point to determine the target cylinder parameter.
In the embodiment of the application, the iterative search processing of the image recognition device can be specifically performed by indicating that the ith cylinder candidate parameter is applicable to the actual scene of the restored cylinder surface information when the ith detection result is an outer cylinder, and the image recognition device can fit the cylinder shape according to the ith cylinder candidate parameter to serve as a target object and take the point contained in the three-dimensional space range occupied by the target object as an inner point. For the first iteration, the image recognition device can take the first cylindrical candidate parameter as the optimal candidate parameter, and continue iteration, if the interior points in the target object corresponding to the second cylindrical candidate parameter are more than the interior points in the optimal object, the second cylindrical candidate parameter is used for replacing the first cylindrical candidate parameter as the optimal candidate parameter, otherwise, the next cylindrical candidate parameter is calculated continuously based on the point cloud data and the normal line of each point, and the like, when the iteration condition is met, the iterative search process is ended, and the cylindrical candidate parameter corresponding to the optimal candidate parameter finally obtained is taken as the target cylindrical parameter.
In the embodiment of the application, when the i-th detection result is the inner cylinder, since most of the targets for restoring the cylinder surface information are convex cylinder surfaces relative to the photographer, which indicates that the concave cylinder surface corresponding to the i-th cylinder candidate parameter is likely to be an error target for restoring the surface information at the moment, the image recognition device can consider the i-th cylinder candidate parameter invalid, and calculate the next cylinder candidate parameter based on the point cloud data and the normal line of each point again instead of comparing the optimal candidate parameter by using the i-th cylinder candidate parameter.
In the embodiment of the present application, the iteration condition may be that the iteration number reaches a preset number threshold, or that the increase of the best candidate parameter substitution is slowed down, etc., which is not limited by the embodiment of the present application.
In the embodiment of the application, after the ith iteration is finished, the image recognition device judges whether the ith iteration meets the iteration condition, if so, the image recognition device finishes the iteration, takes the best candidate parameter obtained after the ith iteration as the target cylinder parameter, and if not, the image recognition device returns to S103 to start the next iteration.
S106, based on the target cylinder parameters, carrying out plane expansion on the surface of the target image to obtain a recognition result of the cylinder surface.
In the embodiment of the application, after the image recognition device obtains the target cylinder parameters, the surface of the target image is subjected to plane expansion based on the target cylinder parameters, so that the recognition result of the cylinder surface is obtained.
In the embodiment of the application, the target cylinder parameter is a curved surface parameter of a target geometric shape corresponding to the target image, the image recognition device can expand the curved surface side surface represented by the target cylinder parameter into a plane based on the target cylinder parameter, and the image after the plane expansion is used as a recognition result.
In the embodiment of the application, the image recognition device can utilize the existing method to realize that the curved surface in the space is unfolded to be flat, and the embodiment of the application is not repeated.
It can be understood that in the embodiment of the application, when the image recognition device searches the optimal target cylinder parameter by using the iterative algorithm, the detection of the inner side surface and the outer side surface of the candidate cylinder parameter is added, so that the further screening of the image recognition result is realized, and the accuracy of image recognition is improved.
In some embodiments of the present application, after determining the target cylinder parameter in S105, parameter optimization may be further performed on the target cylinder parameter to obtain an optimized target cylinder parameter. The image recognition means may design the optimization objective function according to a specific optimization objective.
In some embodiments of the present application, when the optimization objective is to make the cylinder surface fitted according to the optimized objective cylinder parameters contain more point cloud data, the image recognition device may optimize the objective cylinder parameters with the formula (1) as an optimization objective function, where the formula (1) is as follows:
θ*＝argminθ(Σp∈Pd2(C(θ),p)) (1)
Wherein θ * is an optimized target cylinder parameter, θ is a target cylinder parameter, p is any coordinate point included in a cylinder shape range fitted by the target cylinder parameter in a three-dimensional space of the point cloud data, d 2 (C (θ), p) is a euclidean distance from the p point to the cylinder surface, argmin is used to calculate a value of θ that can make d 2 (C (θ), p) minimum, that is, a value of the target cylinder parameter θ that makes d 2 (C (θ), p) minimum is regarded as an optimized target cylinder parameter θ *.
The image recognition device optimizes and adjusts the target cylinder parameters through the formula (1), calculates the target cylinder parameters which can minimize the Euclidean distance from p points to the cylinder surface, and completes the optimization of the target cylinder parameters, so that the cylinder surface fitted according to the optimized target cylinder parameters can contain more points, the number of the point cloud data on the cylinder surface is increased, and the subsequent image recognition can be more accurately carried out.
In the embodiment of the application, the image recognition device can optimize the target cylinder parameters by adopting algorithms such as a trust domain method, a Lei Wen Beige-Marquardt method and the like, and can also use other similar optimization algorithms, and the embodiment of the application is not limited.
In the embodiment of the application, based on the optimized target cylinder parameters, S106 can be that the surface of the target image is subjected to plane expansion based on the optimized target cylinder parameters to obtain the recognition result of the cylinder surface.
It can be understood that in the embodiment of the application, the image recognition device can design the optimization objective function to further optimize the obtained target cylinder parameters, so as to obtain more accurate optimized target cylinder parameters for surface recognition, thereby improving the accuracy of image recognition.
In the embodiment of the present application, based on the normal calculation of the point cloud data by the image recognition device in fig. 4 and S102, the method for obtaining the normal of each point in the point cloud data may be as shown in fig. 5, and includes S1021-S1025 as follows:
s1021, each point in the point cloud data is used for determining a point cloud subset comprising at least one adjacent point in a preset adjacent range of each point.
In the embodiment of the application, for each point in the point cloud data, the image recognition device may select at least one neighboring point in a preset neighboring range near the point, and the set of at least one neighboring point forms a point cloud subset in the preset neighboring range.
For example, for each point in the point cloud data, the image recognition device selects 50 points in a preset proximity to form a point cloud P K, where k=50.
And S1022, performing principal component analysis on the point cloud subset of each point to obtain a first principal component vector of each point.
In the embodiment of the application, after the image recognition device obtains the point cloud subset in the preset adjacent range of each point, principal component analysis (PCA, PRINCIPAL COMPONENT ANALYSIS) is carried out on the point cloud subset of each point, and a first principal component vector of each point is obtained from a PCA analysis result.
In the embodiment of the present application, for the ith point in the point cloud data, the image recognition device may perform principal component analysis on the point cloud subset of the ith point through formula (2) to obtain a first principal component vector of the point, where the first principal component vector is as follows:
v=PVCA(PK)(0) (2)
where v is the first principal component vector of the ith point, P K is the point cloud subset of the ith point, and (0) represents the first principal component.
The image recognition device performs principal component analysis processing on the point cloud subset P K of the ith point, and selects a vector corresponding to the first principal component (0) from the principal component analysis result as a first principal component vector v of the ith point.
In the embodiment of the application, the image recognition device processes each point in the point cloud data in the same method, so as to obtain a first principal component vector of each point.
S1023, acquiring coordinate information of each point in the point cloud data.
In the embodiment of the application, the image recognition device acquires the three-dimensional coordinate information of each point in the point cloud data in addition to the first principal component vector of each point.
S1024, determining the normal vector of each point based on the coordinate information of each point and the first principal component vector of each point.
In the embodiment of the application, for the ith point in the point cloud data, the image recognition device can determine the normal vector of the ith point based on the coordinate information of the ith point and the first principal component vector of the ith point through a formula (3), and the normal vector is as follows:
v*＝argmin(Var(vTpi)) (3)
Wherein v * is a normal vector of the i-th point, p i represents a three-dimensional coordinate of the i-th point, v T represents a transposition of the first principal component vector v obtained in the formula (1), and Var represents a variance calculation.
The image recognition apparatus finds a minimum parameter for a variance function of a product of the transposed first principal component vector v T and the three-dimensional coordinate p i of the i-th point, and uses the found minimum parameter as a normal vector v * of the i-th point.
In the embodiment of the application, the image recognition device processes each point in the point cloud data in the same method, so as to obtain the normal vector of each point.
S1025, restraining the normal vector of each point to obtain the normal line of each point.
In the embodiment of the present application, after the image recognition device obtains the normal vector of each point, since the calculation of the normal vector has ambiguity, the normal vector calculated in S1024 needs to be constrained.
In the embodiment of the application, the image recognition device can restrict the normal vector of each point by calculating the included angle between the normal vector of each point and the camera optical axis direction vector, and specifically can comprise S10251-S10252, as follows:
S10251, determining an included angle between a normal vector of each point and an optical axis direction vector, wherein the optical axis direction vector represents the optical axis direction of the acquisition equipment for acquiring the target image.
In the embodiment of the application, the optical axis direction vector represents the optical axis direction of the acquisition equipment for acquiring the target image, and for the ith point in the point cloud data, the image recognition device can determine the included angle between the normal vector of each point and the optical axis direction vector through the formula (4), as follows:
θ=Z·v* (4)
wherein θ is an angle between a normal vector of the ith point and an optical axis direction vector, Z is a camera optical axis direction vector, and v * is a normal vector of the ith point.
The image recognition device multiplies Z and v * to obtain an included angle theta between the normal vector of the ith point and the optical axis direction vector.
In the embodiment of the application, for each point in the point cloud data, the image recognition device processes the point cloud data in the same method, so that an included angle between a normal vector and an optical axis direction vector of each point is obtained.
S10252, respectively restraining normal vectors of all points corresponding to the included angles based on the included angles and preset normal constraint values to obtain normal lines of all points, wherein the normal vectors of all points correspond to one included angle.
In the embodiment of the application, after the image recognition device obtains the included angle between the normal vector of each point and the optical axis direction vector, each included angle is compared with a preset normal constraint value, and the normal vector of each point corresponding to each included angle is respectively constrained according to the comparison result to obtain the normal of each point;
In the embodiment of the application, the normal vector of each point corresponds to an included angle.
In the embodiment of the application, as shown in fig. 6-1, for a point in the point cloud data, when the included angle between the normal vector of the point and the optical axis direction vector is smaller than 0, the normal vector of the point is characterized by meeting the constraint, the normal vector of the point can be used as the normal of the point as shown in fig. 6-2, when the included angle between the normal vector of the point and the optical axis direction vector is larger than 0, the normal vector of the point is characterized by not meeting the normal constraint, and the normal vector of the point needs to be turned, namely the normal vector of the point is reversely measured, and the turned normal vector is used as the normal of the point as shown in fig. 6-3.
In the embodiment of the present application, for each point in the point cloud data, after the image recognition device calculates the normal line of each point, the schematic diagram of the normal vector of the point cloud may be as shown in fig. 7, where in fig. 7, the left side is an image formed by the point cloud data, and the right side is an image formed by the normal vector of each point in the point cloud data.
It can be understood that in the embodiment of the application, the image recognition device obtains the surface shape of the image corresponding to the point cloud data by obtaining the normal line of each point in the point cloud data.
In the embodiment of the present application, based on fig. 4, the method for calculating the ith cylinder candidate parameter by the image recognition device in S103 based on the point cloud data and the normal line of each point may be as shown in fig. 8, and includes S1031-S1033:
s1031, selecting the ith sampling point data from the point cloud data, wherein the ith sampling point data comprises first sampling point data and second sampling point data.
In the embodiment of the application, in order to calculate the ith cylinder candidate parameter corresponding to the ith iteration, the image recognition device firstly selects the ith sampling point data required by the ith iteration from the point cloud data.
In the embodiment of the application, the sampling point data is three-dimensional coordinate data of the randomly selected sampling points in the point cloud data, and for each iteration, the image recognition device can randomly select two sampling points as two points on the surface of one cylinder side, and calculate cylinder candidate parameters corresponding to the two sampling points, so that the ith sampling point data can comprise first sampling point data and second sampling point data.
In some embodiments, the image recognition device may select two sampling points from a plurality of three-dimensional coordinate points of the point cloud data, and use positions under homogeneous coordinates of the two sampling points as the first sampling point data p 1 and the second sampling point data p 2, respectively.
S1032, a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data are obtained from the normals of each point.
In the embodiment of the application, after the image recognition device obtains the first sampling point data and the second sampling point data, the image recognition device may obtain a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data from the normals of each point calculated in S102.
In some embodiments, for a first sample point data p 1 selected by the image recognition device, the image recognition device may obtain a corresponding first normal n 1, and for a second sample point data p 2 selected by the image recognition device, the image recognition device may obtain a corresponding second normal n 2.
S1033, determining an ith cylinder axis vector, an ith cylinder center point and an ith cylinder radius based on the first sampling point data, the first normal line, the second sampling point data and the second normal line.
In the embodiment of the application, the image recognition device can calculate the ith candidate cylinder parameter corresponding to the ith iteration based on the first sampling point data, the first normal line, the second sampling point data and the second normal line acquired in the ith iteration.
In the embodiment of the application, the ith candidate cylinder parameter comprises the ith candidate cylinder parameter.
In the embodiment of the present application, based on fig. 8, the image recognition device determines, based on the first sampling point data, the first normal line, the second sampling point data, and the second normal line, that the ith cylinder axis vector, the ith cylinder center point, and the ith cylinder radius may specifically be as shown in fig. 9, including S10331-S10335, as follows:
S10331, calculating an i-th cylindrical axis vector based on the first normal line and the second normal line.
In the embodiment of the application, the image recognition device can calculate the ith cylindrical axis vector based on the first normal line and the second normal line.
In the embodiment of the application, since the first normal represents the normal vector of the plane in which the first sampling point data is located, and the second normal represents the normal vector of the plane in which the second sampling point data is located, the image recognition device can calculate the normal vector perpendicular to the plane in which the first sampling point data is located and the plane in which the second sampling point data is located at the same time, that is, the axial vector of the cylinder, based on the first normal and the second normal.
In the embodiment of the application, the image recognition device can calculate the ith cylindrical axis vector through the formula (5), and the formula (5) is as follows:
d=n1×n2 (5)
Wherein n 1 is a first normal line corresponding to the first sampling point data obtained by the ith iteration, n 2 is a second normal line corresponding to the second sampling point data obtained by the ith iteration, and d is a cylindrical axis vector.
The image recognition device carries out cross multiplication on the first normal n 1 and the second normal n 2 to obtain an ith cylindrical axial quantity d.
S10332, determining three-dimensional projection space matrix parameters of the straight line where the first normal line is located according to the first sampling point data and the first normal line.
In the embodiment of the application, the straight line where the first normal line is located can be determined by two different points contained in the first normal line or two different planes containing the first normal line, and the image recognition device determines two different points contained in the first normal line or an expression of an intersection line of the two different planes containing the first normal line as a three-dimensional projection space matrix parameter of the straight line where the first normal line is located according to the first sampling point data and the first normal line.
In the embodiment of the application, the image recognition device can determine the plztec matrix representation of two different points contained in the first normal through the formula (6), and the formula (6) is as follows:
Wherein p 1 is first sampling point data, L 1 is a pllurk matrix of a straight line where a first normal n 1 corresponding to p 1 is located, and L 1 is a2×2 matrix including 4 degrees of freedom parameters.
And the image recognition device calculates a Pluker matrix representation of the straight line where the first normal line is located in the three-dimensional Euclidean space according to the first sampling point data and the first normal line.
S10333, determining the plane where the second sampling point data is located according to the second sampling point data and the second normal line.
In the embodiment of the application, since the plane in the three-dimensional space can be uniquely determined by a point and a line on the plane, the image recognition device can determine the plane in which the second sampling point data is located according to the second sampling point data and the second normal line.
In the embodiment of the application, the image recognition device can determine the plane where the second sampling point data is located through the formula (7), and the formula (7) is as follows:
π2＝(n2×d;-p2·(n2×d)) (7)
Where p 2 is the second sample point data, pi 2 is the plane in which the sample point data p 2 is located, and pi 2 is the 4*1 matrix including 3 coordinate components and 1 normal vector component.
The image recognition device calculates three coordinate components of a plane pi 2 where the second sampling point data is located according to the cylindrical axis vector d and the second normal n 2, calculates a direction vector of the plane where the second sampling point data is located according to the second sampling point data, the cylindrical axis vector d and the second normal n 2, and finally obtains a plane pi 2 where the second sampling point data containing 3 coordinate components and 1 normal vector component is located.
S10334, determining an ith cylindrical center point where the straight line intersects with the plane according to the three-dimensional projection space matrix parameter of the straight line where the first normal line is located and the plane where the second sampling point data is located.
In the embodiment of the application, the image recognition device can determine the ith cylindrical center point where the straight line intersects with the plane through the obtained three-dimensional projection space matrix parameter of the straight line where the first normal line is located and the plane where the second sampling point data is located.
In the embodiment of the application, after obtaining L 1 and pi 2, the image recognition device may calculate the ith cylindrical center point through formula (8):
c=L1π2 (8)
Where c is the cylindrical center point at which the straight line L 1 intersects the plane pi 2.
S10335, determining the radius of the ith cylinder according to the first sampling point data and the center point of the ith cylinder, or determining the radius of the ith cylinder according to the second sampling point data and the center point of the ith cylinder.
In the embodiment of the application, after the image recognition device obtains the ith cylindrical center point where the straight line where the first normal line is located and the plane where the second sampling point data is located through the above process, as for the point on the cylindrical side surface, the distance from the point to the cylindrical center is the radius of the cylinder. The image recognition device can determine the radius of the ith cylinder according to the data of the first sampling point and the center point of the ith cylinder, or can determine the radius of the ith cylinder according to the data of the second sampling point and the center point of the ith cylinder.
In the embodiment of the application, the image recognition device can calculate the ith cylinder radius through the formula (9), and the formula (9) is as follows:
r=|p1-c| (9)
and r is the radius of the cylinder, and the image recognition device determines the radius of the ith cylinder by calculating the distance from the data of the first sampling point to the center point of the ith cylinder.
In the embodiment of the application, the image recognition device can also determine the three-dimensional projection space matrix parameter of the straight line where the second normal line is located according to the second sampling point data and the second normal line by the same calculation principle, determine the plane where the first sampling point data is located according to the first sampling point data and the first normal line, and determine the ith cylindrical center point where the straight line intersects with the plane according to the three-dimensional projection space matrix parameter of the straight line where the second normal line is located and the plane where the first sampling point data is located. And will not be described in detail herein.
S1034, an ith cylinder axis vector, an ith cylinder center point and an ith cylinder radius form an ith cylinder candidate parameter.
In the embodiment of the application, the image recognition device can fit a unique corresponding cylindrical shape through the ith cylindrical axis vector, the ith cylindrical center point and the ith cylindrical radius, and takes the ith cylindrical axis vector, the ith cylindrical center point and the ith cylindrical radius as the ith cylindrical candidate parameters.
It can be understood that, in the embodiment of the present application, for the ith iteration, the image recognition device randomly selects two sampling points from the point cloud data, and calculates the corresponding ith cylinder candidate parameter on the premise that the two sampling points are taken as two points on the surface of the cylinder side, so that further detection and verification of the validity are required for the calculated ith cylinder candidate parameter.
In the embodiment of the present application, based on the inner and outer detection of the ith cylinder candidate parameter in fig. 8 and S104, a method for obtaining the ith detection result may be as shown in fig. 10, and includes S201-S204, as follows:
s201, determining a target object formed by the ith cylindrical candidate parameter from the target image, wherein the target object is an image with a cylindrical shape.
In the embodiment of the application, after the image recognition device calculates the ith cylinder candidate parameter, the image recognition device determines a target object formed by the ith cylinder candidate parameter from the target image.
In the embodiment of the application, the target object is an image with a cylindrical shape.
In the embodiment of the application, the image recognition device fits a cylindrical shape image uniquely corresponding to the three-dimensional space of the point cloud data according to the ith cylindrical axis vector, the ith cylindrical center point and the ith cylindrical radius in the ith cylindrical candidate parameter, and the cylindrical shape image is used as a target object formed by the ith cylindrical candidate parameter
S202, acquiring internal point cloud data belonging to a target object from the point cloud data.
In the embodiment of the application, after the image recognition device acquires the target object, the point cloud data in the space range occupied by the target object in the point cloud data is used as the internal point cloud data.
S203, calculating the center of gravity of the corresponding internal point cloud according to the internal point cloud data.
In the embodiment of the application, the internal point cloud data is also the center of gravity of the corresponding internal point cloud calculated by the image recognition device formed by gathering a plurality of points according to the internal point cloud data.
In the embodiment of the application, the gravity center of the internal point cloud is a three-dimensional coordinate value, and is the average value of all coordinates in the internal point cloud data.
In the embodiment of the application, the image recognition device can calculate the center of gravity of the internal point cloud by calculating the average value of the coordinates of the internal point cloud, and can also be other methods.
S204, detecting the inner side and the outer side of the ith cylinder candidate parameter based on the center of gravity, the ith cylinder candidate parameter and an optical axis direction vector to obtain an ith detection result, wherein the optical axis direction vector represents the optical axis direction of the acquisition equipment for acquiring the target image.
In the embodiment of the application, after the image recognition device obtains the center of gravity, the i-th cylinder candidate parameter can be subjected to inner and outer side detection based on the center of gravity, the i-th cylinder candidate parameter and the optical axis direction vector, so that the i-th detection result is obtained.
In the embodiment of the present application, based on fig. 10, the image recognition device performs inner and outer detection on the ith cylindrical candidate parameter based on the center of gravity, the ith cylindrical candidate parameter and the optical axis direction vector, and the method for obtaining the ith detection result may be as shown in fig. 11, and includes S2041-S2043, as follows:
s2041, based on the i-th cylindrical center point and the gravity center, constitutes a gravity center vector.
In the embodiment of the present application, the camera optical axis direction vector is shown in fig. 6-1, and the image recognition device may use the vector from the i-th cylindrical center point to the center of gravity as the center of gravity vector, as shown in fig. 12.
In some embodiments of the present application, when the center of gravity is G and the i-th cylindrical center point is c, the image recognition apparatus may calculate the value of (G-c) as the center of gravity vector.
S2042, when the gravity center vector is consistent with the optical axis direction vector, obtaining an ith detection result of an ith cylinder candidate parameter forming an outer cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter.
In the embodiment of the application, when the gravity center vector is consistent with the optical axis direction vector, namely (G-C). Z >0, the gravity center vector is consistent with the optical axis direction vector, the image recognition device obtains the ith detection result of the ith cylinder candidate parameter forming the outer cylinder, and the inner side and outer side detection of the ith cylinder candidate parameter is completed.
S2043, when the gravity center vector is inconsistent with the optical axis direction vector, obtaining an ith detection result of the ith cylinder candidate parameter forming an inner cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter.
In the embodiment of the application, when the gravity center vector is inconsistent with the optical axis direction vector, namely (G-C). Z <0, the gravity center vector is inconsistent with the optical axis direction vector, the image recognition device obtains the ith detection result of the ith cylinder candidate parameter forming the inner cylinder, and the inner and outer side detection of the ith cylinder candidate parameter is completed.
In the embodiment of the present application, based on the inner and outer detection of the ith cylinder candidate parameter in fig. 8 and S104, another method for obtaining the ith detection result may also be as shown in fig. 13, including S301-S304, as follows:
s301, obtaining a first direction value according to the first sampling point data, the first normal line and the ith cylindrical center point.
In the embodiment of the present application, as shown in fig. 14, the image recognition apparatus calculates a direction vector from the i-th cylindrical center point to the first sampling point data as a first direction value according to the first sampling point data, that is, the three-dimensional coordinates of the first sampling point, the first normal line, and the i-th cylindrical center point.
In some embodiments of the present application, the image recognition apparatus may calculate (p 1-c)·n1's value as the first direction value.
S302, obtaining a second direction value according to the second sampling point data, the second normal line and the ith cylindrical center point.
In the embodiment of the present application, as shown in fig. 14, the image recognition device calculates, as the second direction value, a direction vector from the i-th cylindrical center point to the second sampling point data, based on the second sampling point data, that is, the three-dimensional coordinates of the second sampling point, the second normal line, and the i-th cylindrical center point.
In some embodiments of the application, the image recognition means may calculate (p 2-c)·n2's value as the second direction value).
And S303, when the first direction value and the second direction value are both greater than 0, obtaining an ith detection result of an ith cylinder candidate parameter forming an outer cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter.
In the embodiment of the application, when the first direction value and the second direction value are both greater than 0, the cylinder side surface with the convex cylinder shape fitted by the ith cylinder candidate parameter is indicated, the image recognition device obtains the ith detection result of the outer cylinder formed by the ith cylinder candidate parameter, and the inner side and outer side detection of the ith cylinder candidate parameter is completed.
S304, when any one of the first direction value and the second direction value is smaller than 0, an ith detection result of the ith cylinder candidate parameter forming an inner cylinder is obtained, and inner and outer side detection of the ith cylinder candidate parameter is completed.
In the embodiment of the application, when at least one of the first direction value and the second direction value is smaller than 0, the cylinder shape fitted by the ith cylinder candidate parameter is the concave cylinder side surface, the image recognition device obtains the ith detection result of the ith cylinder candidate parameter forming the inner cylinder, and the inner side and outer side detection of the ith cylinder candidate parameter is completed.
In the embodiment of the present application, based on the iterative search processing performed in fig. 11 or fig. 13 and based on the i-th detection result, the point cloud data and the normal line of each point in S105, it is determined that the target cylinder parameters may be as shown in fig. 15, including S1051-S1056, as follows:
s1051, when the ith detection result is that the ith cylinder candidate parameter forms an inner cylinder, calculating the (i+1) th cylinder candidate parameter based on the point cloud data and the normal line of each point until the target cylinder parameter meeting the iteration condition is obtained.
In the embodiment of the present application, when the ith detection result is that the ith cylinder candidate parameter forms an inner cylinder, since the object for surface image and text retrieval and recognition is usually a convex surface of a photographer, such as a cylindrical object surface in fig. 16-1 or a cylindrical building surface in fig. 16-2, the ith cylinder candidate parameter is identified as an invalid parameter for the image, the image recognition device discards the ith cylinder candidate parameter obtained by the ith iteration, and resumes the next iteration, that is, the sampling point and the normal corresponding thereto are randomly selected again from the point cloud data and the normal of each point, and calculation of the ith+1 cylinder candidate parameter is performed until the target cylinder parameter satisfying the iteration condition is obtained.
S1052, when the ith detection result is that the ith cylinder candidate parameter forms an outer cylinder, determining a target object formed by the ith cylinder candidate parameter from a target image, wherein the target object is an image with a cylinder shape.
In the embodiment of the application, when the ith detection result is that the ith cylinder candidate parameter forms an outer cylinder, the ith cylinder candidate parameter can be used for image recognition, and the image recognition device determines a cylinder-shaped image fitted by the ith cylinder candidate parameter from the target image as a target object.
S1053, counting the ith interior point quantity of the interior point cloud data belonging to the target object from the point cloud data.
In the embodiment of the application, the image recognition device counts the number of the interior points in the space range occupied by the target object from the point cloud data, and the number is used as the ith interior point number of the interior point cloud data.
S1054, when the number of the ith inner points is larger than or equal to the corresponding maximum number of the inner points of the cylinder candidate parameters before the ith cylinder candidate parameter and the iteration condition is met, taking the ith cylinder candidate parameter as a target cylinder parameter.
In the embodiment of the application, when the number of the ith interior points is greater than or equal to the corresponding maximum number of interior points of the cylinder candidate parameters before the ith cylinder candidate parameter and the iteration condition is met, the image recognition device takes the ith cylinder candidate parameter as the target cylinder parameter.
In the embodiment of the application, for the first iteration, the image recognition device takes the 1 st cylinder candidate parameter as the optimal cylinder candidate parameter, for the second iteration, when the 2 nd interior point number corresponding to the 2 nd cylinder candidate parameter is larger than the 1 st interior point number corresponding to the 1 st cylinder candidate parameter, the fitting degree of the cylinder shape fitted by the 2 nd cylinder candidate parameter is higher than the fitting degree of the cylinder shape fitted by the 1 st cylinder candidate parameter, the image recognition device takes the 2 nd cylinder candidate parameter as the optimal cylinder candidate parameter, and for the ith iteration, when the i th interior point number is larger than or equal to the corresponding maximum interior point number of the cylinder candidate parameter before the i th cylinder candidate parameter, the image recognition device takes the i th cylinder candidate parameter as the optimal cylinder candidate parameter, and so on until the iteration condition is met, the image recognition device takes the optimal cylinder candidate parameter obtained by the last iteration as the target cylinder parameter.
S1055, when the number of the ith inner points is smaller than the corresponding maximum number of the inner points of the cylinder candidate parameters before the ith cylinder candidate parameter and the iteration condition is met, taking the cylinder candidate parameter corresponding to the maximum number of the inner points as the target cylinder parameter.
In the embodiment of the application, when the number of the ith interior points is smaller than the corresponding maximum number of interior points of the cylinder candidate parameters before the ith cylinder candidate parameter and the iteration condition is met, the image recognition device takes the cylinder candidate parameter corresponding to the maximum number of interior points as the target cylinder parameter.
In the embodiment of the application, for the first iteration, the image recognition device takes the 1 st cylinder candidate parameter as the optimal cylinder candidate parameter, for the second iteration, when the 2 nd interior point number corresponding to the 2 nd cylinder candidate parameter is smaller than the 1 st interior point number corresponding to the 1 st cylinder candidate parameter, the fitting degree of the cylinder shape fitted by the 1 st cylinder candidate parameter is higher than the fitting degree of the cylinder shape fitted by the 2 nd cylinder candidate parameter, the image recognition device continues to take the 1 st cylinder candidate parameter as the optimal cylinder candidate parameter, and for the ith iteration, when the i th interior point number is smaller than the corresponding maximum interior point number of the cylinder candidate parameter before the i th cylinder candidate parameter, the image recognition device takes the cylinder candidate parameter corresponding to the maximum interior point number as the optimal cylinder candidate parameter. And the like, until the iteration condition is met, the image recognition device takes the optimal cylinder candidate parameter obtained in the last iteration as the target cylinder parameter.
S1056, when the iteration condition is not met, calculating the (i+1) th cylinder candidate parameter based on the point cloud data and the normal line of each point until the target cylinder parameter meeting the iteration condition is obtained.
In the embodiment of the application, when the iteration condition is not satisfied, and the iteration frequency does not reach the preset threshold, the image recognition device performs the next iteration, randomly selects the sampling point and the normal corresponding to the sampling point from the point cloud data and the normal of each point again, and performs calculation of the (i+1) th cylinder candidate parameter until the target cylinder parameter satisfying the iteration condition is obtained.
It can be understood that in the embodiment of the application, the image recognition device continuously optimizes the cylinder candidate parameters in the iteration process, so that the cylinder shape fitted by the target cylinder candidate parameters is more approximate to the target object in the target image, thereby improving the accuracy of image recognition.
An embodiment of the present application provides an image recognition apparatus 5, as shown in fig. 17, the image recognition apparatus 5 includes an acquisition unit 100, a calculation unit 200, a detection unit 300, a determination unit 400, and a recognition unit 500, wherein,
The acquisition unit 100 is configured to acquire a target image, and acquire point cloud data of the target image;
The computing unit 200 is configured to perform normal calculation on the point cloud data to obtain a normal of each point in the point cloud data;
The calculating unit 200 is further configured to calculate an ith cylinder candidate parameter based on the point cloud data and the normal line of each point, where i is a positive integer greater than or equal to 1;
The detecting unit 300 is configured to perform inside and outside detection on the ith cylindrical candidate parameter to obtain an ith detection result;
the determining and calculating unit 400 is configured to perform iterative search processing based on the i-th detection result, the point cloud data, and the normal line of each point, and determine a target cylinder parameter;
the identifying unit 500 is configured to perform planar expansion on the surface of the target image based on the target cylinder parameter, so as to obtain a result of identifying the cylinder surface.
In some embodiments of the present application, the computing unit 200 is further configured to determine, for each point in the point cloud data, a point cloud subset including at least one neighboring point within a preset neighboring range of the each point, perform principal component analysis on the point cloud subset of the each point to obtain a first principal component vector of the each point, obtain coordinate information of each point in the point cloud data, determine a normal vector of the each point based on the coordinate information of the each point and the first principal component vector of the each point, and constrain the normal vector of the each point to obtain a normal line of the each point.
In some embodiments of the present application, the calculating unit 200 is further configured to determine the included angle between the normal vector of each point and an optical axis direction vector, where the optical axis direction vector characterizes an optical axis direction of an acquisition device that acquires the target image, and constraint the normal vector of each point corresponding to the included angle based on the included angle and a preset normal constraint value, so as to obtain a normal line of each point, where the normal vector of each point corresponds to an included angle.
In some embodiments of the present application, the detection unit 300 is further configured to determine a target object formed by the ith cylinder candidate parameter from the target image, where the target object is an image with a cylinder shape, obtain interior point cloud data of the target object from the point cloud data, calculate a center of gravity of a corresponding interior point cloud according to the interior point cloud data, and perform inner and outer detection on the ith cylinder candidate parameter based on the center of gravity, the ith cylinder candidate parameter, and an optical axis direction vector to obtain the ith detection result, where the optical axis direction vector represents an optical axis direction of an acquisition device that acquires the target image.
In some embodiments of the present application, the detecting unit 300 is further configured to form a gravity center vector based on the i-th cylinder center point and the gravity center, obtain the i-th detection result that the i-th cylinder candidate parameter forms an outer cylinder when the gravity center vector is consistent with the optical axis direction vector, complete the inner and outer detection of the i-th cylinder candidate parameter, and obtain the i-th detection result that the i-th cylinder candidate parameter forms an inner cylinder when the gravity center vector is inconsistent with the optical axis direction vector, complete the inner and outer detection of the i-th cylinder candidate parameter.
In some embodiments of the present application, the computing unit 200 is further configured to select an ith sampling point data from the point cloud data, where the ith sampling point data includes a first sampling point data and a second sampling point data, obtain a first normal line corresponding to the first sampling point data and a second normal line determined by the first sampling point data from normal lines of each point, and determine the ith cylinder axis vector, the ith cylinder center point, and the ith cylinder radius based on the first sampling point number, the first normal line, the first sampling point, and the second normal line, and the ith cylinder axis vector, the ith cylinder center point, and the ith cylinder radius form the ith cylinder candidate parameter.
In some embodiments of the present application, the computing unit 200 is further configured to calculate the ith cylinder axis vector based on the first normal line and the second normal line, determine a three-dimensional projection space matrix parameter of a straight line where the first normal line is located according to the first sampling point data and the first normal line, determine a plane where the second sampling point data is located according to the second sampling point data and the second normal line, determine the ith cylinder center point where the straight line intersects the plane according to the three-dimensional projection space matrix parameter of the straight line where the first normal line is located and the plane where the second sampling point data is located, and determine the ith cylinder radius according to the first sampling point data and the ith cylinder center point, or determine the ith cylinder radius according to the second sampling point data and the ith cylinder center point.
In some embodiments of the present application, the computing unit 200 is further configured to determine, according to the second sampling point data and the second normal line, a three-dimensional projection space matrix parameter of a straight line where the second normal line is located, determine, according to the first sampling point data and the first normal line, a plane where the first sampling point data is located, and determine, according to the three-dimensional projection space matrix parameter of the straight line where the second normal line is located and the plane where the first sampling point data is located, the i-th cylinder center point where the straight line intersects with the plane.
In some embodiments of the present application, the detecting unit 300 is further configured to obtain a first direction value according to the first sampling point data, the first normal line, and the i-th cylinder center point, obtain a second direction value according to the second sampling point data, the second normal line, and the i-th cylinder center point, obtain the i-th detection result that the i-th cylinder candidate parameter constitutes an outer cylinder when the first direction value and the second direction value are both greater than 0, complete the inner and outer detection of the i-th cylinder candidate parameter, and obtain the i-th detection result that the i-th cylinder candidate parameter constitutes an inner cylinder when any one of the first direction value and the second direction value is less than 0, and complete the inner and outer detection of the i-th cylinder candidate parameter.
In some embodiments of the present application, the determining unit 400 is further configured to, when the ith detection result is that the ith cylinder candidate parameter forms an inner cylinder, calculate an ith+1th cylinder candidate parameter based on the point cloud data and a normal line of each point until a target cylinder parameter satisfying an iteration condition is obtained, determine, from the target image, a target object formed by the ith cylinder candidate parameter when the ith detection result is that the ith cylinder candidate parameter forms an outer cylinder, wherein the target object is an image having a cylindrical shape, and count an ith inner point number of the inner point cloud data belonging to the target object from the point cloud data, and, when the ith inner point number is greater than or equal to a corresponding maximum inner point number of the cylinder candidate parameter before the ith cylinder candidate parameter and satisfies an iteration condition, take the ith cylinder candidate parameter as the target cylinder parameter, and, when the ith inner point number is less than the ith inner point number and the corresponding maximum inner point number is not equal to the ith cylinder candidate parameter, calculate the iteration condition until the maximum inner point number is not equal to the corresponding maximum inner point number of the ith cylinder candidate parameter, and when the iteration condition is satisfied, and the maximum inner point number is not equal to the maximum inner point number is equal to the maximum inner point number of the cylinder candidate parameter, and the iteration condition is satisfied.
An embodiment of the present application provides an image recognition apparatus 6, as shown in fig. 18, where the image recognition apparatus 6 includes a processor 125, a memory 126, and a communication bus 127, where the memory 126 communicates with the processor 125 through the communication bus 127, and where the memory 126 stores one or more programs executable by the processor 125, and where the processor 125 performs the image recognition method according to any one of the above when the one or more programs are executed.
Embodiments of the present application provide a computer-readable storage medium storing one or more programs executable by one or more processors to implement the image recognition method as set forth in any one of the above.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the present application.

Claims (11)

1. An image recognition method, comprising:
acquiring a target image and acquiring point cloud data of the target image;
performing normal calculation on the point cloud data to obtain a normal of each point in the point cloud data;
Calculating an ith cylinder candidate parameter based on the point cloud data and the normal line of each point, wherein i is a positive integer greater than or equal to 1;
detecting the inner side and the outer side of the ith cylinder candidate parameter to obtain an ith detection result;
Performing iterative search processing based on the ith detection result, the point cloud data and the normal line of each point to determine a target cylinder parameter;
Based on the target cylinder parameters, carrying out plane expansion on the surface of the target image to obtain a recognition result of the cylinder surface;
Wherein the calculating the ith cylinder candidate parameter based on the point cloud data and the normal line of each point comprises:
selecting the ith sampling point data from the point cloud data, wherein the ith sampling point data comprises first sampling point data and second sampling point data;
Acquiring a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data from the normals of each point;
determining the ith cylinder axis vector, the ith cylinder center point and the ith cylinder radius based on the first sampling point data, the first normal, the second sampling point data and the second normal;
the ith cylinder axis vector, the ith cylinder center point and the ith cylinder radius form the ith cylinder candidate parameter;
The detecting the inner side and the outer side of the ith cylinder candidate parameter to obtain an ith detection result comprises the following steps:
determining a target object formed by the ith cylindrical candidate parameter from the target image, wherein the target object is an image with a cylindrical shape;
Acquiring inner point cloud data belonging to the target object from the point cloud data;
Calculating the center of gravity of the corresponding internal point cloud according to the internal point cloud data;
and detecting the inner side and the outer side of the ith cylinder candidate parameter based on the center of gravity, the ith cylinder candidate parameter and an optical axis direction vector to obtain the ith detection result, wherein the optical axis direction vector represents the optical axis direction of the acquisition equipment for acquiring the target image.
2. The method of claim 1, wherein the performing a normal calculation on the point cloud data to obtain a normal for each point in the point cloud data comprises:
For each point in the point cloud data, determining a point cloud subset including at least one neighboring point within a preset neighboring range of the each point;
Performing principal component analysis on the point cloud subset of each point to obtain a first principal component vector of each point;
acquiring coordinate information of each point in the point cloud data;
Determining a normal vector of each point based on the coordinate information of each point and the first principal component vector of each point;
and constraining the normal vector of each point to obtain the normal line of each point.
3. The method of claim 2, wherein said constraining the normal vector of each point to the normal of each point comprises:
Determining an included angle between the normal vector of each point and an optical axis direction vector, wherein the optical axis direction vector represents the optical axis direction of acquisition equipment for acquiring the target image;
And respectively restraining the normal vector of each point corresponding to the included angle based on the included angle and a preset normal constraint value to obtain the normal of each point, wherein the normal vector of each point corresponds to an included angle.
4. The method according to claim 1, wherein the performing inside-outside detection on the i-th cylinder candidate parameter based on the center of gravity, the i-th cylinder candidate parameter, and an optical axis direction vector, to obtain the i-th detection result, includes:
constructing a gravity center vector based on the ith cylindrical center point and the gravity center;
When the gravity center vector is consistent with the optical axis direction vector, obtaining the ith detection result of the ith cylinder candidate parameter forming an outer cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter;
And when the gravity center vector is inconsistent with the optical axis direction vector, obtaining the ith detection result of the ith cylinder candidate parameter forming the inner cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter.
5. The method of claim 1, wherein the determining the i-th cylinder axis vector, i-th cylinder center point, and i-th cylinder radius based on the first sample point data, the first normal, the second sample point data, and the second normal comprises:
calculating the ith cylindrical axis vector based on the first normal and the second normal;
according to the first sampling point data and the first normal, determining a three-dimensional projection space matrix parameter of a straight line where the first normal is located;
Determining a plane where the second sampling point data is located according to the second sampling point data and the second normal;
determining the ith cylindrical center point where the straight line intersects with the plane according to the three-dimensional projection space matrix parameter of the straight line where the first normal line is located and the plane where the second sampling point data is located;
and determining the radius of the ith cylinder according to the first sampling point data and the center point of the ith cylinder, or determining the radius of the ith cylinder according to the second sampling point data and the center point of the ith cylinder.
6. The method of claim 1, wherein after calculating the ith cylinder axis amount based on the first normal and the second normal and determining the ith cylinder radius from the first sample point data and the ith cylinder center point, or wherein before determining the ith cylinder radius from the second sample point data and the ith cylinder center point, the method further comprises:
Determining matrix parameters of a straight line where the second normal line is located according to the second sampling point data and the second normal line;
determining a plane where the first sampling point data is located according to the first sampling point data and the first normal;
And determining the ith cylindrical center point where the straight line intersects with the plane according to the three-dimensional projection space matrix parameter of the straight line where the second normal line is located and the plane where the first sampling point data is located.
7. The method according to any one of claims 1, 5 and 6, wherein the ith cylinder candidate parameter includes the ith cylinder center point, and the performing inside and outside detection on the ith cylinder candidate parameter to obtain an ith detection result includes:
obtaining a first direction value according to the first sampling point data, the first normal line and the ith cylindrical center point;
Obtaining a second direction value according to the second sampling point data, the second normal line and the ith cylindrical center point;
when the first direction value and the second direction value are both larger than 0, obtaining the ith detection result of the ith cylinder candidate parameter forming an outer cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter;
And when any one of the first direction value and the second direction value is smaller than 0, obtaining the ith detection result of the ith cylinder candidate parameter forming the inner cylinder, and finishing inner and outer side detection of the ith cylinder candidate parameter.
8. The method according to claim 1, wherein the determining the target cylinder parameter based on the i-th detection result, the point cloud data, and the normal line of each point by performing iterative search processing includes:
when the ith detection result is that the ith cylinder candidate parameter forms an inner cylinder, calculating the (i+1) th cylinder candidate parameter based on the point cloud data and the normal line of each point until a target cylinder parameter meeting an iteration condition is obtained;
When the ith detection result is that the ith cylinder candidate parameter forms an outer cylinder, determining a target object formed by the ith cylinder candidate parameter from the target image, wherein the target object is an image with a cylinder shape;
Counting the number of the ith inner points of the inner point cloud data belonging to the target object from the point cloud data;
When the number of the ith interior points is larger than or equal to the corresponding maximum number of interior points of the cylinder candidate parameters before the ith cylinder candidate parameter and the iteration condition is met, taking the ith cylinder candidate parameter as the target cylinder parameter;
When the number of the ith inner points is smaller than the corresponding maximum number of inner points of the cylinder candidate parameters before the ith cylinder candidate parameter and the iteration condition is met, taking the cylinder candidate parameter corresponding to the maximum number of inner points as the target cylinder parameter;
And when the iteration condition is not met, calculating the (i+1) th cylinder candidate parameter based on the point cloud data and the normal line of each point until the target cylinder parameter meeting the iteration condition is obtained.
9. An image recognition device is characterized by comprising an acquisition unit, a calculation unit, a detection unit, a determination unit and a recognition unit, wherein,
The acquisition unit is used for acquiring a target image and acquiring point cloud data of the target image;
the computing unit is used for performing normal calculation on the point cloud data to obtain the normal of each point in the point cloud data;
the calculation unit is further used for calculating an ith cylinder candidate parameter based on the point cloud data and the normal line of each point, wherein i is a positive integer greater than or equal to 1;
the detection unit is used for detecting the inner side and the outer side of the ith cylinder candidate parameter to obtain an ith detection result;
The determining unit is used for carrying out iterative search processing based on the ith detection result, the point cloud data and the normal line of each point to determine a target cylinder parameter;
the identification unit is used for carrying out plane expansion on the surface of the target image based on the target cylinder parameters to obtain a cylinder surface identification result;
The computing unit is also used for selecting an ith sampling point data from the point cloud data, wherein the ith sampling point data comprises first sampling point data and second sampling point data, a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data are obtained from the normal of each point, and the ith cylinder axis vector, the ith cylinder center point and the ith cylinder radius are determined based on the first sampling point data, the first normal, the second sampling point data and the second normal;
The detection unit is further used for determining a target object formed by the ith cylindrical candidate parameter from the target image, wherein the target object is an image with a cylindrical shape, acquiring interior point cloud data belonging to the target object from the point cloud data, calculating the gravity center of a corresponding interior point cloud according to the interior point cloud data, and detecting the inner side and the outer side of the ith cylindrical candidate parameter based on the gravity center, the ith cylindrical candidate parameter and an optical axis direction vector to obtain the ith detection result, wherein the optical axis direction vector represents the optical axis direction of acquisition equipment for acquiring the target image.
10. An image recognition device comprising a processor, a memory and a communication bus, the memory being in communication with the processor via the communication bus, the memory storing one or more programs executable by the processor, the processor performing the method of any of claims 1-8 when the one or more programs are executed.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, the one or more programs may be executed by one or more processors to implement the method of any of claims 1-8.
CN201911252710.2A 2019-12-09 2019-12-09 Image recognition method and device, and computer-readable storage medium Active CN113033247B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911252710.2A CN113033247B (en) 2019-12-09 2019-12-09 Image recognition method and device, and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911252710.2A CN113033247B (en) 2019-12-09 2019-12-09 Image recognition method and device, and computer-readable storage medium

Publications (2)

Publication Number Publication Date
CN113033247A CN113033247A (en) 2021-06-25
CN113033247B true CN113033247B (en) 2025-04-01

Family

ID=76450998

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911252710.2A Active CN113033247B (en) 2019-12-09 2019-12-09 Image recognition method and device, and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN113033247B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113657375B (en) * 2021-07-07 2024-04-19 西安理工大学 Bottled object text detection method based on 3D point cloud
CN113628166B (en) * 2021-07-12 2025-03-18 广东电网有限责任公司广州供电局 A method and device for detecting geometric dimensions of a circuit breaker chamber of a switch cabinet

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102713671A (en) * 2009-12-11 2012-10-03 株式会社拓普康 Point cloud data processing device, point cloud data processing method, and point cloud data processing program
CN106408581A (en) * 2016-09-12 2017-02-15 厦门大学 Rapid three-dimensional point cloud linear extraction method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110304619A1 (en) * 2010-06-10 2011-12-15 Autodesk, Inc. Primitive quadric surface extraction from unorganized point cloud data
CN108010036B (en) * 2017-11-21 2020-01-21 江南大学 Object symmetry axis detection method based on RGB-D camera
CN109147038B (en) * 2018-08-21 2023-02-07 北京工业大学 3D Pipeline Modeling Method Based on 3D Point Cloud Processing

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102713671A (en) * 2009-12-11 2012-10-03 株式会社拓普康 Point cloud data processing device, point cloud data processing method, and point cloud data processing program
CN106408581A (en) * 2016-09-12 2017-02-15 厦门大学 Rapid three-dimensional point cloud linear extraction method

Also Published As

Publication number Publication date
CN113033247A (en) 2021-06-25

Similar Documents

Publication Publication Date Title
JP5705147B2 (en) Representing 3D objects or objects using descriptors
US9378431B2 (en) Method of matching image features with reference features and integrated circuit therefor
JP5677798B2 (en) 3D object recognition and position and orientation determination method in 3D scene
US9064171B2 (en) Detection device and method for transition area in space
EP2915138B1 (en) Systems and methods of merging multiple maps for computer vision based tracking
JP5385105B2 (en) Image search method and system
CN110663060B (en) Method, device, system and vehicle/robot for representing environmental elements
CN110942515A (en) Point cloud-based target object three-dimensional computer modeling method and target identification method
CN102722887A (en) Image registration method and device
CN110766758A (en) Calibration method, device, system and storage device
CN112528781B (en) Obstacle detection method, device, equipment and computer readable storage medium
CN113111513B (en) Sensor configuration scheme determining method and device, computer equipment and storage medium
JP6119225B2 (en) Intersection road boundary detection device and program
CN113033247B (en) Image recognition method and device, and computer-readable storage medium
US20150254527A1 (en) Methods for 3d object recognition and registration
CN105447869B (en) Camera self-calibration method and device based on particle swarm optimization algorithm
CN104992431A (en) Method and device for multispectral image registration
CN115656991A (en) Vehicle external parameter calibration method, device, equipment and storage medium
CN110069131B (en) Multi-fingertip positioning method based on near-infrared light circular spot detection
CN117288120A (en) Three-dimensional imaging measurement system based on multi-viewpoints and its calibration method
CN115546785A (en) Three-dimensional target detection method and device
CN115909485A (en) A human behavior recognition method and storage medium based on radar point cloud imaging
CN111368624A (en) Loop detection method and device based on generation of countermeasure network
CN119380356B (en) Pointer type instrument reading identification method based on template semantic key point matching
CN117710702B (en) Visual positioning method, device, storage medium and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant