CN102096927A - Target tracking method of independent forestry robot - Google Patents

Abstract

The invention discloses a target tracking method of an autonomous forestry robot, which is applied to the target tracking of an autonomous forestry robot including computer vision, a digitally controlled pan-tilt and a central control computer, including a moving target detection part and a pan-tilt real-time motion control part, The moving target detection part includes an image preprocessing module, a multi-frame difference method motion information acquisition module, a moving target discrimination module, and a target center coordinate calculation module. The moving target detection part detects moving target information, and records the circumscribed rectangle of the target and the target center coordinates ; The center of the target controlled by the real-time motion control part of the PTZ is located in the vicinity of the center of the image. During the operation process of the autonomous forestry robot, the operation target can always be located at the center of the image of the forestry robot vision system or in the vicinity.

Description

Target Tracking Method for Autonomous Forestry Robot

technical field

The invention relates to a robot target tracking technology, in particular to an autonomous forestry robot target tracking method.

Background technique

The autonomous forestry robot is a kind of special robot, and its research is attracting more and more attention, which is mainly due to the specific environment of the forest area. During the operation process, the autonomous forestry robot hopes that the operation target will always be located in the center of the image of the forestry robot's vision system or in the vicinity. Since both the target and the forestry robot may be in motion, the target of the forestry robot is in motion relative to the robot, and the shape and size of the target are uncertain. In order to ensure that the target is always located in the image center of the vision system of the forestry robot Or in the nearby area, it is necessary to detect the moving target in real time, calculate the center coordinates of the target, and adjust the direction and angle of the gimbal in real time according to the difference between the target center coordinates and the image center position.

In the prior art, due to the uncertainty of the detected target, especially when the moving target is more complex, it is not easy to describe the image color and shape, the real-time detection of the target in the image and video sequence has certain difficulties; at the same time, the pan/tilt The real-time adaptive control algorithm is rarely reported.

Contents of the invention

The object of the present invention is to provide an autonomous forestry robot target tracking method that enables the autonomous forestry robot to always make the operating target located at the center of the image of the forestry robot vision system or in the vicinity during the operation.

The purpose of the present invention is achieved through the following technical solutions:

The target tracking method of the autonomous forestry robot of the present invention is applied to the target tracking of the autonomous forestry robot including computer vision, digital control pan-tilt and central control computer, including the moving target detection part and the real-time motion control part of the pan-tilt;

The moving target detection part includes an image preprocessing module, a multi-frame difference method motion information acquisition module, a moving target discrimination module, and a target center coordinate calculation module;

The image preprocessing is used to perform image filtering, image grayscale and image segment linear segment transformation on multi-frame images;

The motion information acquisition module of the multi-frame difference method first performs pairwise difference on all images, and performs image segmentation on each difference image, and then accumulates and sums the divided difference images to obtain moving object information;

The moving target discrimination module judges whether there is a moving target according to a set threshold;

If there is a moving target, the target center coordinate calculation module records the circumscribed rectangle of the target and the target center coordinates;

The real-time motion control part of the pan-tilt controls the center of the target to be located in the vicinity of the center of the image.

As can be seen from the technical solutions provided by the present invention above, the autonomous forestry robot target tracking method provided by the embodiments of the present invention includes a moving target detection part and a cloud platform real-time motion control part, and the moving target detection part detects the moving target information and records The circumscribed rectangle of the target and the coordinates of the target center; the center of the target controlled by the real-time motion control part of the pan/tilt is located in the vicinity of the image center. During the operation process, the autonomous forestry robot can always keep the operation target at the center of the image of the vision system of the forestry robot or the nearby area.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings on the premise of not paying creative work.

1 is a schematic diagram of an autonomous forestry robot target tracking task provided by an embodiment of the present invention;

Fig. 2 is the structural representation of the cloud platform real-time control system based on computer vision in the embodiment of the present invention;

Fig. 3 is the overall flowchart of target tracking of autonomous forestry robot in the embodiment of the present invention;

Fig. 4 is the flow chart of real-time detection of moving target in the embodiment of the present invention;

Fig. 5 is a flow chart of the real-time motion control of the pan/tilt in the embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

The target tracking method of the autonomous forestry robot of the present invention, the method is applied to the target tracking of the autonomous forestry robot comprising computer vision, digital control platform and central control computer, and its preferred specific implementation is:

Including moving target detection part and PTZ real-time motion control part;

The moving target detection part includes an image preprocessing module, a multi-frame difference method motion information acquisition module, a moving target discrimination module, and a target center coordinate calculation module;

The image preprocessing is used to perform image filtering, image grayscale and image segment linear segment transformation on multi-frame images;

The motion information acquisition module of the multi-frame difference method first performs pairwise difference on all images, and performs image segmentation on each difference image, and then accumulates and sums the divided difference images to obtain moving object information;

The moving target discrimination module judges whether there is a moving target according to a set threshold;

If there is a moving target, the target center coordinate calculation module records the circumscribed rectangle of the target and the target center coordinates;

The real-time motion control part of the pan-tilt controls the center of the target to be located in the vicinity of the center of the image.

The motion information acquisition module of the multi-frame difference method obtains the motion target information including steps:

A. Image difference calculation: set the video image sequence and sampling rate to be suitable for accurate extraction of targets with different motion rates;

B. Threshold segmentation of differential images: perform threshold segmentation for each differential image to obtain a binary image;

C. Summing all the binary images, locating the area of continuous motion, and obtaining the boundary information of the moving target.

In the step A, the default setting is: n=3 grayscale pictures for each group of image sequences, and the sampling rate is 0.2s. That is, one frame of pictures is collected every 200 ms, and a total of 3 frames of pictures are continuously acquired as a group, and three images are used to perform pairwise difference calculations to obtain a total of n _Δf = 3*(3-1)/2 difference images;

In the step B, the segmentation threshold is calculated by using the maximum inter-class variance threshold method, specifically using a random threshold to divide the grayscale of pixels into two categories, and the threshold with the farthest distance between the two types of grayscale mean values is the optimal segmentation threshold;

In the step C, for the summed image, noise and interference points are removed by using a mathematical morphology processing method of first erosion and then expansion, so as to obtain boundary information of the moving target.

The set threshold is the area threshold of the moving target;

The target center coordinates are the ratio of the first-order moment and the zero-order moment of the target image.

The real-time motion control part of the pan-tilt controls the digital control pan-tilt's horizontal and vertical angle adjustment measurements according to the target center coordinates, and constitutes a closed-loop control system.

The control algorithm of the real-time motion control part of the cloud platform includes:

Let the horizontal length of the circumscribed rectangle of the target be W, the vertical length be H, the coordinates of the center of the target be (x, y), and the coordinates of the center of the image be (x ₀ , y ₀ ). The adjustment steps of the digital control pan/tilt are as follows:

D. Calculate the position difference (Δx, Δy) between the target center and the image center, Δx=x ₀ -x, Δy=y ₀ -y;

则目标已经出于图像中心或附近区域，云台不需要调整，转向步骤D；若

云台需要进行调整，转向步骤F；E. If

Then the target is already in the center of the image or the nearby area, the gimbal does not need to be adjusted, turn to step D; if

The gimbal needs to be adjusted, turn to step F;

当Δθ_x＞0云台在水平方向应该沿着逆时针方向调整，当Δθ_x＜0云台在水平方向应该沿着顺时针方向调整；若|Δy|＞δ，云台垂直方向调整角度Δθ_y＝k_yΔy，其中

当Δθ_y＞0云台在水平方向应该向下调整，当Δθ_y＜0云台在水平方向应该向上调整；云台水平、垂直两个方向调整结束后转入步骤D；F. If |Δx|>δ, adjust the pan-tilt horizontal direction angle Δθ _x = k _x Δx, where

When Δθ _x > 0, the pan/tilt should be adjusted counterclockwise in the horizontal direction, and when Δθ _x < 0, the pan/tilt should be adjusted clockwise in the horizontal direction; if |Δy|>δ, the vertical angle of the pan/tilt is adjusted by Δθ _y = k _y Δy, where

When Δθ _y > 0, the gimbal should be adjusted downward in the horizontal direction, and when Δθ _y < 0, the gimbal should be adjusted upward in the horizontal direction; after the horizontal and vertical adjustments of the gimbal are completed, go to step D;

In the above formula, δ is the set threshold.

The invention is mainly used for autonomous forestry robots to automatically detect targets when operating in forest areas, and to adjust the angle of the platform in real time so that the central coordinates of the forestry robot's operating targets are always in the center of the image or in the vicinity, belonging to the field of forestry intelligent equipment.

The autonomous forestry robot of the present invention comprises the hardware support of the cloud platform real-time control system based on computer vision that camera, digital control cloud platform and control computer etc. are formed, comprises three parts such as camera, digital control cloud platform and control computer. The camera is set up on the digital control pan-tilt, and the camera is connected to the monitoring computer through the IEEE1394 interface, and the image obtained by the camera is read in real time through the monitoring computer, and the image is processed and analyzed according to the relevant algorithm proposed by this law. The digital control pan/tilt is connected to the computer through the RS232 interface and communicates. The computer sends the pan/tilt real-time attitude control command to the numerical control pan/tilt through the RS232 interface. The pan/tilt changes its own attitude according to the control command to realize the target tracking of the autonomous forestry robot. The type of hardware in the system and the interface connected between the hardware are not limited thereto.

The present invention has the following advantages:

The autonomous forestry robot target tracking method of the present invention does not depend on specific hardware system, is used in the real-time control system of cloud platform based on computer vision, as long as this system comprises monitoring computer, digital control cloud platform and video camera, the concrete model of these hardwares and interface The type is not limited to the IEEE1394 interface and RS232 interface described above, and the different types of interfaces are only driven by different methods;

The target automatically tracked in real time in the present invention is obtained through automatic detection of moving targets, and does not require prior knowledge related to the target, such as the size, shape and type of the target. Autonomous forestry robots of the invention that can be applied to different types of operations;

The self-adaptive real-time control of cloud platform among the present invention is a closed-loop control, and control algorithm can self-adaptively adjust the size of cloud platform level, the size of vertical two direction angle adjustments according to the distance of target position and image center, at target position and image center position When it is larger, the gimbal direction angle adjustment is large, and when the target position and the image center position are small, the gimbal direction angle adjustment amount is small. At the same time, a tolerance margin δ is designed, which does not require the center of the target to coincide completely with the center of the image, but only needs the target to be in the area near the center of the image, which can prevent the gimbal from being in an adjustment state when the target is already very close to the center of the image Case.

Concrete embodiment, as shown in Figure 1 to Figure 5:

The forestry robot target tracking task schematic diagram that Fig. 1 provides has reflected target tracking concrete and tracking process; Fig. 2 has provided the real-time control system structural diagram of cloud platform based on computer vision, just realizes a kind of autonomous forestry robot target that the present invention proposes The hardware system of the tracking method; Fig. 3, Fig. 4 and Fig. 5 are software flowcharts of a method for target tracking of an autonomous forestry robot.

As shown in Figure 1, the target tracking of the autonomous forestry robot is to automatically control the horizontal and vertical angles of the pan/tilt when the autonomous forestry robot is walking or stationary, so that the target is always located in the center of the image or in the nearby area;

As shown in Figure 2, it is a computer vision-based real-time control system for PTZ, in which there are no special requirements for cameras, PTZ monitoring calculations and the interfaces between them;

Referring to Fig. 1 again, after the camera captures the image and detects the target, calculate the difference (Δx, Δy) between the target center coordinates and the image center position, and the monitoring computer adjusts the posture of the cloud platform according to the cloud platform real-time motion control method proposed by the present invention, Keep the target always in the center or nearby area of the image.

As shown in Figure 3, it is the overall flow chart of the target tracking of the autonomous forestry robot. The initialization of the camera and the pan/tilt needs to use the relevant functions in the API function library of the camera and the pan/tilt provided by the manufacturer of the camera and the pan/tilt to perform the initialization of the camera and the pan/tilt respectively. Initialize, and then the relevant functions in the API function library of the camera carry out image acquisition. Process and analyze the collected image, first detect whether there is a target in the image, if there is a target, calculate the center position of the target and judge whether the target is in the image center and its vicinity, if the target center is not in the image center and its vicinity, according to the target center Coordinates and the real-time control algorithm of the gimbal adjust the gimbal attitude so that the target center is always in the center of the image and its vicinity.

As shown in Figure 4, it is the flow chart of the moving target detection program. The detailed implementation steps of the algorithm include: ① Image difference calculation, which can set the video image sequence and sampling rate to be suitable for accurate extraction of targets with different moving rates. The default setting is: n=3 grayscale pictures for each group of image sequences, and the sampling rate is 0.2s. That is, a frame of pictures is collected every 200ms, and a total of 3 frames of pictures are continuously acquired as a group. Three images are used to perform pairwise difference operations, and then the total number of difference images is n _Δf =3*(3-1)/2. ②Threshold segmentation of the difference image, threshold segmentation is performed on each difference image to obtain a binary image. Using the criterion based on the maximum between-class variance to calculate the threshold of difference image segmentation, the basic idea is to use the random threshold Z' to divide the grayscale of pixels into two categories, so that Z, which is the farthest distance between the mean values of the two categories of grayscale, is the optimal segmentation threshold. ③Sum all the difference images after image thresholding processing, locate the area of continuous motion, and obtain the boundary information of the moving target. For the summed image, the noise and interference points are removed by using the mathematical morphology first erosion and then expansion processing method, and finally a more ideal boundary of the moving target can be obtained.

则目标已经出于图像中心或附近区域，云台不需要调整，转向步骤(1)；若云台需要进行调整，转向步骤(3)。(3)若|Δx|＞δ，云台水平方向调整角度Δθ_x＝k_xΔx，其中

当Δθ_x＞0云台在水平方向应该沿着逆时针方向调整，当Δθ_x＜0云台在水平方向应该沿着顺时针方向调整；若|Δy|＞δ，云台垂直方向调整角度Δθ_y＝k_yΔy，其

当Δθ_y＞0云台在水平方向应该向下调整，当Δθ_y＜0云台在水平方向应该向上调整；云台水平、垂直两个方向调整结束后转入步骤(1)。As shown in Figure 5, it is the flow chart of the real-time motion control of the pan/tilt, and the specific process of the attitude adjustment of the pan/tilt is given. Let the horizontal length of the bounding rectangle of the target be W, the vertical length be H, the coordinates of the center of the target be (x, y), and the coordinates of the center of the image be (x ₀ , y ₀ ). The specific implementation steps of the adjustment algorithm of the digital control pan/tilt are as follows: (1) Calculate the position difference (Δx, Δy) between the target center and the image center, Δx=x ₀ -x, Δy=y ₀ -y. (2) If

Then the target is already in the center of the image or the nearby area, the gimbal does not need to be adjusted, turn to step (1); if The gimbal needs to be adjusted, go to step (3). (3) If |Δx|>δ, the horizontal adjustment angle of the gimbal is Δθ _x = k _x Δx, where

When Δθ _x > 0, the pan/tilt should be adjusted counterclockwise in the horizontal direction, and when Δθ _x < 0, the pan/tilt should be adjusted clockwise in the horizontal direction; if |Δy|>δ, the vertical angle of the pan/tilt is adjusted by Δθ _y = k _y Δy, which

When Δθ _y > 0, the gimbal should be adjusted downward in the horizontal direction, and when Δθ _y < 0, the gimbal should be adjusted upward in the horizontal direction; after the horizontal and vertical adjustments of the gimbal are completed, turn to step (1).

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person familiar with the technical field can easily conceive of changes or changes within the technical scope disclosed in the present invention. Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (6)

1. autonomous forestry robot target tracking, this method is applied to comprise the target following of the autonomous forestry robot of computer vision, digital control The Cloud Terrace and central control computer, it is characterized in that, comprise moving object detection part and The Cloud Terrace real time kinematics control section;

Described moving object detection partly comprises movable information acquisition module, moving target discrimination module, target's center's coordinate Calculation module of image pretreatment module, Multi Frame Difference point-score;

Described image pre-service is used for multiple image is carried out image filtering, image gray processing and the conversion of image segmentation linear segmented;

The movable information acquisition module of described Multi Frame Difference point-score at first carries out difference in twos to all images, and each difference image is carried out image segmentation, then the difference image after cutting apart is accumulated summation, obtains moving target information;

Described moving target discrimination module judges whether to exist moving target according to preset threshold;

If have moving target, boundary rectangle and target's center's coordinate of then described target's center coordinate Calculation module records target;

Described The Cloud Terrace real time kinematics control section is controlled the near zone that is centered close to picture centre of described target.

2. autonomous forestry robot target tracking according to claim 1 is characterized in that, the movable information acquisition module of described Multi Frame Difference point-score obtains moving target information and comprises step:

A, image difference calculate: sequence of video images and sampling rate are provided with, to be applicable to the target of accurate extraction different motion speed;

The Threshold Segmentation of B, difference image: carry out Threshold Segmentation at each difference image, obtain bianry image;

C, all bianry images are sued for peace, the zone of location persistent movement obtains moving object boundary information.

3. autonomous forestry robot target tracking according to claim 2 is characterized in that:

In the described steps A, default setting is: every group of image sequence n=3 width of cloth gray scale picture, sampling rate is 0.2s.Be that every 200ms gathers a frame picture, obtaining 3 frame pictures continuously altogether is one group, utilizes 3 width of cloth images to carry out calculus of differences in twos, obtains difference image n altogether _{Δ f}=3* (3-1)/2 width of cloth;

Among the described step B, utilizing the maximum between-cluster variance threshold method to calculate segmentation threshold, specifically is to utilize at random that threshold value is divided into two classes with pixel grey scale, and this two classes gray average distance threshold value farthest is an optimal segmenting threshold;

Among the described step C,, utilize the disposal route of the after expansion of corrosion earlier of mathematical morphology to remove noise and noise spot, obtain the boundary information of moving target for the image after the summation.

4. autonomous forestry robot target tracking according to claim 1 is characterized in that described preset threshold is the area threshold of moving target;

Described target's center coordinate is the first moment of target image and the ratio of zeroth order square.

5. according to each described autonomous forestry robot target tracking of claim 1 to 4, it is characterized in that, described The Cloud Terrace real time kinematics control section is according to the level of described target's center coordinate control figure control The Cloud Terrace, the angular setting tolerance of vertical both direction, and the formation closed-loop control system.

6. autonomous forestry robot target tracking according to claim 5 is characterized in that the control algolithm of described The Cloud Terrace real time kinematics control section comprises:

The boundary rectangle cross-directional length that makes target is W, and vertical-direction length is H, and target's center's coordinate is that (x, y), the picture centre coordinate is (x ₀, y ₀), the set-up procedure of digital control The Cloud Terrace is as follows:

D, calculating target's center and picture centre position difference (Δ x, Δ y), Δ x=x ₀-x, Δ y=y ₀-y;

E, if

Then target is for picture centre or near zone, and The Cloud Terrace does not need to adjust, and turns to step D; If

The Cloud Terrace need be adjusted, and turns to step F;

F, if | Δ x|＞δ, the The Cloud Terrace horizontal direction is adjusted angle delta θ _x=k _xΔ x, wherein

As Δ θ _x＞0 The Cloud Terrace in the horizontal direction should be along counterclockwise adjusting, as Δ θ _x＜0 The Cloud Terrace in the horizontal direction should be along the clockwise direction adjustment; If | Δ y|＞δ, the The Cloud Terrace vertical direction is adjusted angle delta θ _y=k _yΔ y, wherein

As Δ θ _y＞0 The Cloud Terrace should be adjusted downwards in the horizontal direction, as Δ θ _y＜0 The Cloud Terrace should adjust upward in the horizontal direction; The Cloud Terrace level, vertical both direction adjustment change step D over to after finishing;

In the following formula, δ is a preset threshold.

Images