CN115100271A - Detection method, device, computer equipment and storage medium for picking height - Google Patents

Abstract

The application relates to a method and a device for detecting the goods picking height, computer equipment and a storage medium. The method comprises the following steps: acquiring original point cloud data of a target goods taking area; the target goods taking area comprises a tray for storing and taking goods; carrying out picture conversion based on the original point cloud data to obtain a target picture; the target picture comprises a tray object used for representing the tray; determining a positioning contour edge of the tray object from the target picture; the positioning contour edge is an edge used for positioning the goods taking height in the contour of the tray object; determining target point cloud data matched with the positioning contour edge in the original point cloud data; and determining the target goods taking height based on the target point cloud data. According to the method and the device, the target goods taking height can still be normally calculated under the condition that original point cloud data are incomplete, and therefore the accuracy of the target goods taking height is improved.

Description

Detection method, device, computer equipment and storage medium for picking height

technical field

The present application relates to the technical field of logistics applications, and in particular, to a method, device, computer equipment and storage medium for detecting the height of pickup.

Background technique

With the development of the logistics industry, the number of goods continues to increase, and the demand for picking up goods is also increasing. Currently, the pickup is mainly carried out by means of handling equipment, for example by an unmanned forklift, according to a predetermined pickup height.

Under normal circumstances, the pickup height is directly calculated from the point cloud data on the pallet. However, when the pallet is defective or occluded, the point cloud data on the pallet will be incomplete, thus affecting the accuracy of the pickup height. Therefore, how to improve the accuracy of the pickup height has become a technical problem that needs to be solved urgently by those skilled in the art.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a pickup height detection method, device, computer equipment and storage medium that can improve the pickup height accuracy in response to the above technical problems.

In a first aspect, the present application provides a method for detecting a pickup height. The method includes:

Obtain the original point cloud data of the target pickup area; the target pickup area includes pallets for accessing goods;

Perform image conversion based on the original point cloud data to obtain a target image; the target image includes a tray object used to represent the tray;

Determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is an edge in the contour of the pallet object used for positioning the pickup height;

Determine the target point cloud data that matches the edge of the positioning contour in the original point cloud data;

The target pickup height is determined based on the target point cloud data.

In a second aspect, the present application also provides a device for detecting the height of a pickup. The device includes:

a data acquisition module for acquiring original point cloud data of a target pickup area; the target pickup area includes a pallet for accessing goods;

A picture conversion module, configured to perform picture conversion based on the original point cloud data to obtain a target picture; the target picture includes a tray object used to represent the tray;

an edge determination module, configured to determine a positioning contour edge of the pallet object from the target image; the positioning contour edge is an edge in the contour of the pallet object used for positioning the pickup height;

a data matching module for determining the target point cloud data in the original point cloud data that matches the edge of the positioning contour;

A height determination module, configured to determine a target pickup height based on the target point cloud data.

In some embodiments, the picture conversion module includes a screening unit and a projection unit. The screening unit is used for screening point cloud data on a vertical section under the laser coordinate system from the original point cloud data to obtain reference point cloud data. The projection unit is configured to perform projection processing on the reference point cloud data to obtain the target picture. The data matching module is further configured to determine, from the reference point cloud data, target point cloud data matching the edge of the positioning contour.

In some embodiments, the screening unit is further configured to preliminarily screen preliminary point cloud data that meets preset conditions from the original point cloud data; divide the preliminary point cloud data into a voxel grid; The target points on the vertical section under the laser coordinate system are screened in the grid to obtain the reference point cloud data.

In some embodiments, the edge determination module is further configured to preprocess the target image to obtain a preprocessed image; perform contour recognition on the preprocessed image to obtain an identified contour; determine the identified contour from the identified contour. the positioning contour edge of the pallet object.

In some embodiments, each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to represent whether the corresponding voxel grid has point cloud data, the edge The determining module is further configured to, for each pixel coordinate point in the pixel coordinate system of the target picture, determine a voxel index corresponding to the pixel coordinate point; the voxel index corresponding to the pixel coordinate point is The voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system; according to the voxel index corresponding to the pixel coordinate point, grayscale processing is performed on the target image to obtain grayscale degree image; perform dilation and erosion operation on the grayscale image to obtain the preprocessed image.

In some embodiments, the data matching module is further configured to determine the target voxel index corresponding to the pixel coordinate point of the edge of the positioning contour; Point to the point in the target voxel grid to obtain the target point cloud data.

In some embodiments, the height determination module is further configured to perform straight line fitting on the target point cloud data to obtain a fitted line segment; and determine the target pickup height based on the midpoint value of the fitted line segment .

In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the above method for detecting a pickup height are implemented.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by the processor, implements the steps in the above-mentioned method for detecting a pickup height.

In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the above method for detecting a pickup height.

The above-mentioned detection method, device, computer equipment and storage medium for the height of the pickup are to obtain the original point cloud data of the target pickup area; the target pickup area includes a pallet for accessing the goods; image conversion is performed based on the original point cloud data, Obtain the target image; the target image includes the pallet object used to characterize the pallet; determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is the edge used to locate the pickup height in the contour of the pallet object; determine the original point cloud The target point cloud data in the data that matches the edge of the positioning contour; the target pickup height is determined based on the target point cloud data. In this application, the original point cloud data of the target pickup area is obtained, and the original point cloud data is image-converted to obtain the target image, and only the positioning contour edges in the target image are determined to match the positioning contour edges in the original point cloud data. Target point cloud data, and determine the target pickup height based on the target point cloud data, can still calculate the target pickup height normally when the original point cloud data is not complete, thereby improving the accuracy of the target pickup height.

Description of drawings

1 is a schematic flowchart of a method for detecting a pickup height in some embodiments;

Figure 2 is a schematic diagram of a tray in some embodiments;

3 is a schematic diagram of a laser coordinate system in some embodiments;

4 is a schematic diagram of a voxel coordinate system in some embodiments;

5 is a schematic diagram of a pixel coordinate system in some embodiments;

6 is a structural block diagram of a device for detecting a pickup height in some embodiments;

Figure 7 is a diagram of the internal structure of a computer device in some embodiments.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

In some embodiments, as shown in FIG. 1 , a method for detecting the height of picking up goods is provided. In this embodiment, the method is applied to a server for illustration. It can be understood that this method can also be applied to handling equipment. It can also be applied to a system including a handling device and a server, and is realized through the interaction of the handling device and the server. Wherein, the handling equipment refers to the transportation equipment for handling goods, and the handling equipment may be, but is not limited to, at least one of an Automated Guided Vehicle (AGV) and a forklift, wherein the forklift may be, but is not limited to, an unmanned forklift. . In this embodiment, the method includes the following steps:

Step 102: Obtain original point cloud data of the target pickup area.

Wherein, the target pickup area includes one or more pallets for accessing goods.

A pallet is a cargo tool for transporting goods in groups.

Point cloud data refers to a set of vectors in a three-dimensional coordinate system. These vectors are usually expressed in the form of X, Y, and Z three-dimensional coordinates, which are generally used to represent the outer surface shape of an object.

Specifically, laser scanning is performed on the target pickup area by a laser device to obtain corresponding original point cloud data, where the laser device refers to a device capable of emitting laser light. The server obtains the original point cloud data obtained by laser scanning of the laser equipment, which is used for subsequent calculation of the target pickup height of the handling equipment.

In some embodiments, the laser device can be fixed on the handling device, and the specific fixing position of the laser device is not limited, as long as the fixed laser device can accurately scan the point cloud data.

In some embodiments, the laser device can be fixed at the midpoint between the roots of the two gripping arms used for gripping or picking up goods in the handling device, so as to ensure that the laser device does not perform laser scanning during laser scanning. Obstructed by cargo, thereby improving laser scanning accuracy. The root of the clamp arm refers to the end of the clamp arm in the handling equipment that faces the body of the handling equipment.

In some embodiments, after the laser equipment is fixed on the handling equipment, the server controls the handling equipment to drive to the position to be picked up, and the laser equipment performs laser scanning on the target pickup area to obtain original point cloud data. Wherein, the position to be picked up is a pre-set approximate pick-up position of the handling equipment.

In other embodiments, after the server controls the handling equipment to travel to the position to be picked up, the handling equipment feeds back the pre-target position of the pallet to be picked up to the sensing algorithm module. The perception algorithm module obtains the point cloud data in front of the handling equipment through the laser equipment fixed on the handling equipment, and obtains the point cloud data associated with the pre-target position from the point cloud data in front of the handling equipment according to the pre-target position sent by the handling equipment. Part of the point cloud data, as the original point cloud data.

Wherein, the pallet to be picked up refers to the pallet designated by the handling equipment to pick up the goods in the target picking area. The perception algorithm module is a module on the server for extracting point cloud data.

The pre-target position of the pallet to be taken refers to the coordinate pose of the pallet to be taken relative to the coordinate system where the handling equipment is located, specifically including but not limited to the longitudinal width of the pallet to be taken relative to the coordinate system where the handling equipment is located in the X-axis direction , the lateral length of the pallet to be taken in the Y-axis direction relative to the coordinate system where the handling equipment is located, the vertical height of the pallet to be taken relative to the coordinate system of the handling equipment in the Z-axis direction, and the pallet plane of the pallet to be taken compared to the handling equipment. At least one of the included angles in the Y-axis direction of the coordinate system where the device is located.

The point cloud data in front of the handling equipment includes point cloud data of the pallet to be taken, and point cloud data of the surrounding environment of the pallet to be taken.

In some embodiments, the process of extracting the point cloud data of interest from the point cloud data in front of the handling equipment by the perception algorithm module according to the predetermined target position is as follows: the perception algorithm module is based on the pre-calibrated laser external parameters and pre-configured waiting Take the pallet size parameters of the pallet, and extract part of the point cloud data that matches the pre-target position of the pallet to be taken from the point cloud data in front of the handling equipment, as the original point cloud data.

Among them, the laser external parameter refers to the coordinate system conversion relationship between the laser equipment and other coordinate systems, such as the coordinate system where the handling equipment is located. Tray size parameters, including but not limited to at least one of tray length, tray width, and tray height.

In some embodiments, the perception algorithm module may further extract, according to the tray size parameter, point cloud data located in a preset size range and matching the tray to be taken, as the original point cloud data. The preset size range is set according to the pallet size parameter and the preset value. Specifically, the difference between the tray size parameter and the preset value is used as the minimum value of the preset size range, and the sum of the tray size parameter and the preset value is used as the maximum value of the preset size range. The preset value can be set according to actual needs, for example, the preset value can be set to 0.3 cm.

In other embodiments, the laser device can also be set separately from the handling device. Specifically, the laser device can be fixedly installed in a position where laser scanning can be performed for the target pickup area. When the original point of the target pickup area needs to be collected When cloud data is used, laser scanning is performed directly through laser equipment.

Step 104: Perform image conversion based on the original point cloud data to obtain a target image.

Wherein, the target picture includes a tray object used to represent the tray.

In some embodiments, the server may directly convert the original point cloud data into a picture to obtain a target picture. In other embodiments, the server may also filter the original point cloud data, and convert the filtered point cloud data into a picture to obtain the target picture. The present application can improve the accuracy of the target image by filtering out a part of the original point cloud data before image conversion.

Step 106: Determine the positioning contour edge of the pallet object from the target image.

Wherein, the edge of the positioning contour is the edge in the positioning contour of the pallet object used to locate the pickup height. In practical applications, the contour of the pallet object includes an inner contour and an outer contour, the positioning contour of the pallet object is the inner contour of the pallet object, and the edge of the positioning contour of the pallet object includes but is not limited to the upper edge of the inner contour of the pallet object and the inner contour of the pallet object at least one of the lower edge lines.

The positions of the inner contour of the pallet object, the outer contour of the pallet object, the upper edge of the inner contour of the pallet object, and the lower edge of the inner contour of the pallet object can be referred to as shown in 2.

In some embodiments, the server may directly extract the positioning contour of the tray object, that is, the inner contour, from the target image, and extract the corresponding positioning contour edge based on the inner contour of the tray object, that is, the upper edge of the inner contour of the tray object or the tray object. The lower edge of the inner contour.

In other embodiments, the server may also first extract the outer contour of the pallet object from the target image, identify the positioning contour within the contour range of the outer contour, that is, the inner contour, and then extract the corresponding positioning based on the inner contour of the pallet object The contour edge, that is, the upper edge of the inner contour of the pallet object or the lower edge of the inner contour of the pallet object.

Step 108: Determine the target point cloud data matching the edge of the positioning contour in the original point cloud data.

Specifically, the server determines, from the original point cloud data, point cloud data that matches the edge of the positioning contour according to the positioning contour edge determined in the target image, as the target point cloud data.

In some embodiments, the server may also filter the original point cloud data, and determine the point cloud data that matches the edge of the positioning contour from the filtered point cloud data, as the target point cloud data.

Step 110: Determine the target pickup height based on the target point cloud data.

The target picking height refers to the height at which the handling equipment picks up the pallets to be picked up.

Specifically, after acquiring the target point cloud data matching the edge of the positioning contour, the server performs fitting on the target point cloud data, and determines the target pickup height according to the fitting result.

In some embodiments, the server may further determine the pickup plane of the pallet to be taken according to the target point cloud data, and determine the target pickup height according to the pickup plane. The picking plane refers to the plane on which the handling equipment needs to pick up the pallets to be taken.

In the above method for detecting the pickup height, the original point cloud data of the target pickup area is obtained; the target pickup area includes a pallet for accessing the goods; image conversion is performed based on the original point cloud data to obtain the target image; the target image including the pallet object used to characterize the pallet; determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is the edge in the contour of the pallet object used to locate the pickup height; determine the original point cloud data and the positioning contour edge Match the target point cloud data; determine the target pickup height based on the target point cloud data. In this application, the original point cloud data of the target pickup area is obtained, and the original point cloud data is image-converted to obtain the target image, and only the positioning contour edges in the target image are determined to match the positioning contour edges in the original point cloud data. Target point cloud data, and determine the target pickup height based on the target point cloud data, can still calculate the target pickup height normally when the original point cloud data is not complete, thereby improving the accuracy of the target pickup height.

In some embodiments, step 104 specifically includes, but is not limited to: screening point cloud data on a vertical section under the laser coordinate system where the laser device is located from the original point cloud data to obtain reference point cloud data; The data is subjected to projection processing to obtain the target image.

The laser coordinate system refers to a three-dimensional coordinate system constructed based on laser equipment, and the vertical section located under the laser coordinate system refers to a two-dimensional plane located in the vertical direction of the laser coordinate system, specifically the xoy plane, that is The plane of z=0 in the laser coordinate system. The schematic diagram of the laser coordinate system may refer to FIG. 3 .

Specifically, the server selects the two-dimensional plane of the vertical section under the laser coordinate system from the original point cloud data, that is, the point cloud data of the XOY plane with Z=0 under the laser coordinate system, and obtains the reference point cloud data. Project the reference point cloud data into a picture to get the target picture.

In some embodiments, the server may also select the point cloud data of the two-dimensional plane located in the vertical direction under the laser coordinate system from the original point cloud data, and then select the point cloud data of the two-dimensional plane located in the vertical direction under the laser coordinate system. The point cloud data is filtered again to obtain the reference point cloud data. Project the reference point cloud data into a picture to get the target picture.

In some embodiments, step 108 specifically includes, but is not limited to, determining, from the reference point cloud data, target point cloud data that matches the edge of the positioning contour.

Specifically, the server determines, from the reference point cloud data, point cloud data that matches the edge of the positioning contour as the target point cloud data.

In some embodiments, the step of "screening point cloud data on a vertical section under the laser coordinate system from the original point cloud data to obtain reference point cloud data" specifically includes but is not limited to: preliminary screening from the original point cloud data Preliminary point cloud data that meets the preset conditions; divide the preliminary point cloud data into a voxel grid; screen the target points on the vertical section under the laser coordinate system from each voxel grid to obtain the reference point cloud data.

The preset condition refers to each point after plane fitting is performed on the original point cloud data of the XOY plane under the laser coordinate system where the laser equipment is located (that is, the plane with Z=0 under the laser coordinate system). The preset distance range of the combined plane, where the fitted plane is the plane obtained after plane fitting the original point cloud data, and the preset distance range can be set to 2 cm to 3 cm.

In practical applications, the Random Sample Consensus (RANSAC) algorithm can be used to perform plane fitting on the original point cloud data. Among them, the random sampling consensus algorithm is an algorithm that calculates the mathematical model parameters of the data according to a set of sample data sets containing abnormal data, and obtains valid sample data.

Specifically, after the server performs plane fitting on the original point cloud data and obtains the fitted plane, it screens out the point cloud data that meets the preset distance range, for example, screens out points within a range of 2 cm to 3 cm from the fitted plane, to form preliminary point cloud data. Next, the server performs voxel filtering on the preliminary point cloud data, specifically: dividing the preliminary point cloud data into voxel grids, and filtering the target points on the vertical section under the laser coordinate system from each voxel grid respectively. , and replace all the points in the corresponding voxel grid with the target points screened out in each voxel grid to obtain the reference point cloud data.

The voxel grid may be a cubic grid, specifically a cube, and the edge length of the voxel grid may be 1 cm. The voxel grid on the vertical section in the laser coordinate system may also be a plane grid, specifically a square, and the corresponding side length may also be 1 cm.

Among them, the target points are points selected from each voxel grid according to preset criteria. Generally, in each voxel grid, the point with the largest Y coordinate in the laser coordinate system (that is, the ordinate in the laser coordinate system) can be used as the target point. For ease of description, the ordinate in the laser coordinate system is simply referred to as the Y coordinate below.

In other embodiments, the server may also first screen the point cloud data on the vertical section under the laser coordinate system from the preliminary point cloud data, and divide the point cloud data on the vertical section under the laser coordinate system After the voxel grid is formed, the target points are directly screened from each voxel grid, and the target points screened in each voxel grid are replaced by all points in the corresponding voxel grid to obtain the reference point cloud data.

In some embodiments, step 106 specifically includes but is not limited to: preprocessing the target image to obtain a preprocessed image; performing contour recognition on the preprocessed image to obtain a recognized contour; and determining the positioning contour edge of the pallet object from the recognized contour.

Specifically, the server needs to preprocess the target image, for example, the target image needs to be subjected to grayscale processing and erosion and expansion operations, so as to improve the image accuracy. Next, the server identifies the contour of the pallet object from the preprocessed image, and compares the contour of the pallet object with the size of the actual pallet, and extracts the contour that conforms to the actual size of the pallet as an alternative contour, and identifies the contour from the alternative contour. The positioning contour edge of the pallet object.

Wherein, the size of the actual tray includes, but is not limited to, at least one of the length and width of the actual tray.

In some embodiments, the specific process of identifying the edge of the positioning contour of the pallet object from the candidate contours is as follows: for each candidate contour, search for pixels whose gray value is a preset value from bottom to bottom or from bottom to top For example, find the pixel point with a gray value of 255, so as to find all the positioning contour edges on the pallet object.

In some embodiments, after the step "replace all points in the corresponding voxel grid with the target points selected in each voxel grid to obtain reference point cloud data", a volume can also be established according to the reference point cloud data The index array corresponding to the reference point cloud data is set according to the established voxel coordinate system. For the convenience of description, the index array corresponding to the reference point cloud data is referred to as the voxel grid index array for short.

Among them, each pair of voxel grid index arrays is an index array of target points in each voxel grid. That is, [the voxel index of each voxel grid] = (the voxel index of each target point with the largest Y coordinate in the voxel grid), if there is no point in the voxel grid, then [the voxel Grid's voxel index] array value is set to a fixed number, such as negative 1.

The specific form of the voxel coordinate system may refer to FIG. 4 , and 0, 1, 2, etc. displayed in the coordinate system of FIG. 4 are all voxel indices. (x_min, y_min) in FIG. 4 represents the coordinate point with the smallest X coordinate and the smallest Y coordinate in the laser coordinate system, and is also the origin of the voxel coordinate system. (x_max, y_min) in Figure 4 represents the coordinate point with the largest X coordinate and the smallest Y coordinate in the laser coordinate system, (x_min, y_max) represents the coordinate point with the smallest X coordinate and the largest Y coordinate in the laser coordinate system, (x_max , y_max) represents the coordinate point with the largest X coordinate and the largest Y coordinate in the laser coordinate system. Among them, the X coordinate under the laser coordinate system is the abscissa obtained under the laser coordinate system, and the Y coordinate under the laser coordinate system is the ordinate obtained under the laser coordinate system. For the convenience of description, the following also refers to the laser coordinate system. The abscissa is abbreviated as the X coordinate, and the ordinate in the laser coordinate system is abbreviated as the Y coordinate.

It should be noted that taking (x_min, y_min) as the origin of the voxel coordinate system is for the convenience of calculation. In practical applications, points at other positions in the reference point cloud data may also be taken as the origin of the voxel coordinate system, which is not limited in this application.

In some embodiments, each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to represent whether the corresponding voxel grid has point cloud data. Specifically, according to the value corresponding to the voxel index Determine whether the corresponding voxel grid has point cloud data. If the array value corresponding to the voxel index is negative 1, it means that the corresponding voxel grid has no point cloud data. If the array value corresponding to the voxel index is not negative 1, It means that the corresponding voxel grid is point cloud data.

In some embodiments, points in the voxel grid corresponding to all voxel indices in the voxel coordinate system may also be projected to obtain the target picture. Specifically, the points in the voxel grid are all located on the XOY plane of the laser coordinate system where the laser device is located. Therefore, it is necessary to project the points in the voxel grid parallel to the XOY plane to obtain the target image.

In some embodiments, after the target picture is obtained, it can also be determined according to the position of the point in the voxel grid corresponding to the voxel index in the voxel coordinate system, in the pixel coordinate system established based on the target picture, and the voxel index The corresponding pixel coordinate point.

In some embodiments, the step of "preprocessing the target picture to obtain a preprocessed picture" specifically includes but is not limited to: for each pixel coordinate point in the pixel coordinate system of the target image, determining the volume corresponding to the pixel coordinate point Pixel index; according to the voxel index in the voxel coordinate system corresponding to the pixel coordinate point, grayscale processing is performed on the target image to obtain a grayscale image;

The pixel coordinate system refers to a pixel coordinate system established based on the target image, and the origin of the pixel coordinate system needs to be consistent with the origin of the voxel coordinate system. The voxel index corresponding to the pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system.

In some embodiments, the specific form of the pixel coordinate system can refer to FIG. 5 , the origin of the voxel coordinate system (x_min, y_min) is the origin of the pixel coordinate system where the target picture is located, and (x_min, y_min) represents the X in the pixel coordinate system The pixel with the smallest coordinate and the smallest Y coordinate. (x_max, y_min) in Figure 5 represents the pixel with the largest X coordinate and the smallest Y coordinate in the pixel coordinate system, (x_min, y_max) represents the pixel with the smallest X coordinate and the largest Y coordinate in the pixel coordinate system, (x_max , y_max) represents the pixel with the largest X coordinate and the largest Y coordinate in the pixel coordinate system.

Specifically, the server determines, for each pixel coordinate point in the pixel coordinate system of the target image, a voxel index in the voxel coordinate system that corresponds to the pixel coordinate point one-to-one. The server obtains the corresponding array value from the voxel grid index array according to the voxel index, and sets the pixel coordinate points corresponding to the voxel index of different array values to different grayscale values, thereby obtaining a grayscale image. Next, the server performs an expansion and erosion operation on the grayscale image, specifically, a closing operation on the grayscale image, so as to fill in the holes in the grayscale image, and fit the cracks in the grayscale image to ensure the grayscale image. The position and shape of the center remains unchanged. Closing the grayscale image can prevent voids in the point cloud data scanned for the pallet to be taken due to obstructions attached to the pallet to be taken or other factors.

In practical applications, the step "setting pixel coordinate points corresponding to voxel indices of different array values to different grayscale values to obtain a grayscale image" specifically includes but is not limited to: if the array corresponding to the voxel index If the value is greater than a fixed value (the fixed value can be set to minus 1), the gray value of the pixel coordinate point corresponding to the voxel index whose array value is greater than the fixed value is set to a preset value, for example, 255. If the array value corresponding to the voxel index is equal to or less than the fixed value, the gray value of the pixel coordinate point corresponding to the voxel index whose array value is equal to or less than minus 1 is set to another preset value, for example, set to 0.

In some embodiments, the step "from the reference point cloud data, determine the target point cloud data that matches the edge of the positioning contour" specifically includes but is not limited to: determining the target voxel index corresponding to the pixel coordinate point of the edge of the positioning contour; From the reference point cloud data, determine the point located in the target voxel grid pointed to by the target voxel index, and obtain the target point cloud data.

Specifically, the server determines the pixel coordinate point of the edge of the positioning contour in the pixel coordinate system in the target picture, and determines the voxel index corresponding to the pixel coordinate point of the edge of the positioning contour from the voxel index as the target voxel index. Then, the server determines the voxel grid pointed to by the target voxel index from the reference point cloud data, that is, the target voxel grid, and extracts the corresponding target point from the target voxel grid to obtain the target point cloud data.

In some embodiments, step 110 specifically includes but is not limited to: performing straight line fitting on the target point cloud data to obtain a fitted line segment; and determining the target pickup height based on the midpoint value of the line segment of the fitted line segment.

Specifically, the server performs straight line fitting on the target point cloud data, for example, performs least squares straight line fitting on the target point cloud data to obtain the fitted straight line. The server obtains the fitted line segment formed by the target point cloud data on the fitted straight line, and determines the target pickup height of the handling equipment according to the midpoint value of the line segment corresponding to the midpoint of the fitted line segment. The midpoint value of the line segment refers to the Y value of the midpoint of the fitted line segment in the laser coordinate system.

It should be noted that, for any straight line, it can be expressed as a straight line equation, that is, in the form of y=bx+a, where b is the slope and a is the intercept. For N points in the target point cloud data, countless straight lines can be used to fit, and the least squares method is used to fit the objective function to obtain the optimal solution of the objective function, that is, the best slope and the best slope in the straight line equation. Intercept, according to the best slope and best intercept in the equation of the straight line, the fitting straight line can be determined.

In some embodiments, the steps of using least squares straight line fitting to obtain the optimal slope and optimal intercept in the straight line equation are as follows, and the involved formulas include formulas (1) to (7).

The first step: establish the objective function based on the straight line equation.

The second step: derivation of the objective function of formula (1) to obtain the expression of the straight line equation.

The third step: after sorting out the straight line equation expressions obtained by the formula (2) and the formula (3), the sorted straight line equation expressions are obtained.

aN+b∑x _i =∑y _i (4)

a∑x _i +b∑x _i ² =∑x _i y _i (5)

Step 4: Solve the straight line equation expression after sorting, and obtain the best estimated values of the straight line parameters a and b, so as to obtain the fitted straight line.

和

表示直线方程中最佳的斜率和最佳的截距，N表示目标点云数据中点的数量。Among them, x _i and y _i are the X coordinate value of each point of the target point cloud data in the laser coordinate system (that is, the abscissa value of each point in the laser coordinate system) and the Y coordinate value (that is, each point in the laser coordinate system). The ordinate value of each point), a and b represent the slope and intercept in the equation of the line, respectively,

and

Represents the best slope and best intercept in the line equation, and N represents the number of points in the target point cloud data.

In other embodiments, in order to prevent the handling equipment from colliding during the pickup process, the target pickup height of the present application may also be based on the Y value (ie, the midpoint value of the fitted line segment) and the pallet foot set at the bottom of the pallet. The height of the pier is determined. Specifically, the target pickup height = Y value + half of the height of the pallet foot pier. Among them, the pallet foot pier is set at the bottom of the pallet, and the pallet foot pier is also called pallet foot, pallet block and pallet square pier, etc., which are used to protect the goods from being squeezed and collided by the handling equipment during transportation.

In some embodiments, the detection method of the pickup height of the present application further includes, but is not limited to, the following steps:

First, fix the laser equipment on the handling equipment, and control the handling equipment to drive to the position to be picked up, then obtain the original point cloud data of the target area through the laser equipment.

Secondly, the server obtains the original point cloud data collected by the laser device, preliminarily screens the preliminary point cloud data that meets the preset conditions from the original point cloud data, divides the preliminary point cloud data into voxel grids, and selects the preliminary point cloud data from each voxel network. The target points on the vertical section under the laser coordinate system are screened in the grid to obtain the reference point cloud data.

Next, the server establishes a voxel coordinate system for the reference point cloud data, and determines a voxel index corresponding to the voxel coordinate system. The server performs projection processing on the reference point cloud data to obtain the target image, and preprocesses the target image to obtain the preprocessed image.

In some embodiments, the process of preprocessing the target image is: establishing a pixel coordinate system for the target image, determining a voxel index corresponding to the pixel coordinate point for each pixel coordinate point in the pixel coordinate system of the target image, According to the voxel index corresponding to the pixel coordinate point, grayscale processing is performed on the target image to obtain a grayscale image; the grayscale image is subjected to dilation and erosion operation to obtain a preprocessed image. Among them, each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to indicate whether the corresponding voxel grid is point cloud data, and the voxel index corresponding to the pixel coordinate point is the pixel coordinate point. The voxel index of the corresponding voxel grid in the voxel coordinate system.

Then, the server performs contour recognition on the preprocessed image to obtain the recognized contour; the positioning contour edge of the pallet object is determined from the recognized contour. Specifically, the server first identifies all the contours in the preprocessed picture to obtain the identified contour, and then determines the positioning contour edge of the pallet object from the identified contour.

Next, the server determines the target voxel index corresponding to the pixel coordinate point of the edge of the positioning contour, and from the reference point cloud data, determines the point located in the target voxel grid pointed to by the target voxel index, and obtains the target point cloud data .

Finally, the server performs straight line fitting on the target point cloud data to obtain a fitted line segment, and determines the target pickup height based on the midpoint value of the fitted line segment.

It should be understood that, although the steps in the flowcharts involved in the above embodiments are sequentially displayed according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but may be performed at different times The execution order of these steps or phases is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or phases in the other steps.

Based on the same inventive concept, an embodiment of the present application also provides a pickup height detection device for implementing the above-mentioned pickup height detection method. The solution to the problem provided by the device is similar to the solution described in the above method, so the specific limitations in the embodiments of the one or more pickup height detection devices provided below can refer to the above for the pickup height The limitation of the detection method is not repeated here.

In some embodiments, as shown in FIG. 6 , a detection device for picking height is provided, including: a data acquisition module 602, a picture conversion module 604, an edge determination module 606, a data matching module 608, and a height determination module 610, in:

The data acquisition module 602 is used for acquiring the original point cloud data of the target pickup area; the target pickup area includes pallets for accessing goods;

The picture conversion module 604 is configured to perform picture conversion based on the original point cloud data to obtain a target picture; the target picture includes a tray object used to represent the tray;

The edge determination module 606 is used to determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is the edge used for positioning the pickup height in the contour of the pallet object;

A data matching module 608, configured to determine the target point cloud data that matches the edge of the positioning contour in the original point cloud data;

The height determination module 610 is configured to determine the target pickup height based on the target point cloud data.

The present application obtains the original point cloud data of the target pickup area through the above-mentioned detection device for the pickup height; the target pickup area includes a pallet for accessing the goods; image conversion is performed based on the original point cloud data to obtain the target picture; The target image includes the pallet object used to characterize the pallet; the positioning contour edge of the pallet object is determined from the target image; the positioning contour edge is the edge in the contour of the pallet object used to locate the pickup height; determine the difference between the original point cloud data and the positioning The target point cloud data that matches the contour edge; the target pickup height is determined based on the target point cloud data. By obtaining the original point cloud data of the target pickup area, converting the original point cloud data to a picture to obtain the target picture, and only for the positioning contour edge in the target picture, determine the target point in the original point cloud data that matches the positioning contour edge Cloud data, and the target pickup height is determined based on the target point cloud data, so that the target pickup height can still be calculated normally when the original point cloud data is not complete, thereby improving the accuracy of the target pickup height.

In some embodiments, the picture conversion module 604 includes a screening unit and a projection unit. The screening unit is used for screening the point cloud data on the vertical section under the laser coordinate system from the original point cloud data to obtain the reference point cloud data. The projection unit is used for projecting the reference point cloud data to obtain the target picture. The data matching module 608 is further configured to determine, from the reference point cloud data, the target point cloud data that matches the edge of the positioning contour.

In some embodiments, the screening unit is further configured to preliminarily screen preliminary point cloud data that meets preset conditions from the original point cloud data; divide the preliminary point cloud data into a voxel grid; and screen the laser light from each voxel grid respectively The target point on the vertical section in the coordinate system is obtained to obtain the reference point cloud data.

In some embodiments, the edge determination module 606 is further configured to preprocess the target image to obtain a preprocessed image; perform contour recognition on the preprocessed image to obtain a recognized contour; and determine the positioning contour edge of the pallet object from the recognized contour.

In some embodiments, each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to represent whether the corresponding voxel grid has point cloud data, and the edge determination module 606 is further configured to target the target image For each pixel coordinate point in the pixel coordinate system of , determine the voxel index corresponding to the pixel coordinate point; the voxel index corresponding to the pixel coordinate point is the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system. According to the voxel index corresponding to the pixel coordinate point, gray-scale processing is performed on the target image to obtain a gray-scale image; the gray-scale image is subjected to dilation and erosion operation to obtain a pre-processed image.

In some embodiments, the data matching module 608 is further configured to determine the target voxel index corresponding to the pixel coordinate point of the edge of the positioning contour; from the reference point cloud data, determine the target voxel grid pointed to by the target voxel index to get the target point cloud data.

In some embodiments, the height determination module 610 is further configured to perform straight line fitting on the target point cloud data to obtain a fitted line segment; and determine the target pickup height based on the midpoint value of the line segment of the fitted line segment.

Each module in the above-mentioned detection device for picking height can be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In some embodiments, a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 7 . The computer device includes a processor, memory, and a network interface connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The nonvolatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The computer equipment's database is used to store point cloud data and pictures. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting a pickup height is realized.

Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In some embodiments, a computer device is provided, including a memory and a processor, where a computer program is stored in the memory, and the processor implements the steps in the foregoing method embodiments when the processor executes the computer program.

In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in the foregoing method embodiments.

In some embodiments, a computer program product is provided, including a computer program, which implements the steps in each of the foregoing method embodiments when the computer program is executed by a processor.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to a memory, a database or other media used in the various embodiments provided in this application may include at least one of a non-volatile memory and a volatile memory. Non-volatile memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory (Magnetoresistive Random Memory) Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration and not limitation, the RAM may be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM). The database involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a blockchain-based distributed database, etc., but is not limited thereto. The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., and are not limited to this.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the present application should be determined by the appended claims.

Claims (10)

1. A method for detecting pickup height, the method comprising:

acquiring original point cloud data of a target goods taking area; the target goods taking area comprises a tray for storing and taking goods;

performing picture conversion based on the original point cloud data to obtain a target picture; the target picture comprises a tray object used for representing the tray;

determining a positioning contour edge of the tray object from the target picture; the locating profile edge is an edge in the profile of the pallet object for locating a pickup height;

determining target point cloud data matched with the positioning contour edge in the original point cloud data;

and determining a target goods taking height based on the target point cloud data.

2. The method of claim 1, wherein the performing a picture conversion based on the original point cloud data to obtain a target picture comprises:

screening point cloud data on a vertical section under a laser coordinate system from the original point cloud data to obtain reference point cloud data;

performing projection processing on the reference point cloud data to obtain the target picture;

the determining the target point cloud data matched with the positioning contour edge in the original point cloud data comprises:

and determining target point cloud data matched with the positioning contour edge from the reference point cloud data.

3. The method of claim 2, wherein the step of filtering the point cloud data on a vertical section under a laser coordinate system from the original point cloud data to obtain reference point cloud data comprises:

preliminarily screening preliminary point cloud data which accord with preset conditions from the original point cloud data;

dividing a voxel grid for the preliminary point cloud data;

and respectively screening target points on the vertical section under the laser coordinate system from each voxel grid to obtain the reference point cloud data.

4. The method of claim 3, wherein said determining a locating profile edge of said tray object from said target picture comprises:

preprocessing the target picture to obtain a preprocessed picture;

carrying out contour recognition on the preprocessed picture to obtain a recognition contour;

determining the locating profile edge of the tray object from the identified profile.

5. The method according to claim 4, wherein each voxel grid corresponds to a voxel index in a voxel coordinate system, and the voxel index is used for characterizing whether the corresponding voxel grid has some cloud data;

the preprocessing the target picture to obtain a preprocessed picture comprises the following steps:

determining a voxel index corresponding to each pixel coordinate point in a pixel coordinate system of the target picture; the voxel index corresponding to the pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system;

performing gray processing on the target picture according to the voxel index corresponding to the pixel coordinate point to obtain a gray image;

and performing expansion corrosion operation on the gray level image to obtain the preprocessed picture.

6. The method of claim 3, wherein determining target point cloud data from the reference point cloud data that matches the locating profile edges comprises:

determining a target voxel index corresponding to a pixel coordinate point of the edge of the positioning contour;

and determining points in a target voxel grid pointed by the target voxel index from the reference point cloud data to obtain the target point cloud data.

7. The method of any one of claims 1 to 6, wherein determining a target pickup height based on the target point cloud data comprises:

performing linear fitting on the target point cloud data to obtain a fitting line segment;

and determining the target goods taking height based on the line segment midpoint value of the fitted line segment.

8. A device for detecting the height of a pick, the device comprising:

the data acquisition module is used for acquiring original point cloud data of a target goods taking area; the target goods taking area comprises a tray for storing and taking goods;

the picture conversion module is used for carrying out picture conversion based on the original point cloud data to obtain a target picture; the target picture comprises a tray object used for representing the tray;

an edge determination module for determining a positioning contour edge of the tray object from the target picture; the locating profile edge is an edge in the profile of the pallet object for locating a pickup height;

the data matching module is used for determining target point cloud data matched with the positioning contour edge in the original point cloud data;

and the height determining module is used for determining the target goods picking height based on the target point cloud data.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.

Images