CN111506082A - A system and method for automatically following and shooting obstacle avoidance - Google Patents

Abstract

The invention discloses an automatic following shooting obstacle avoidance system and method. In the system, a camera is set up on a three-axis PTZ, the camera shoots images or videos of a following target, and after the images are processed, a relationship between the following target and the three-axis PTZ is obtained. Distance; the first ultrasonic ranging module is set on the heading axis of the three-axis gimbal, the first ultrasonic ranging module moves with the heading axis of the three-axis gimbal, and always faces the following target, and measures the following target and the three-axis gimbal The gimbal camera is placed on the shooting mobile device, and the motor controls the forward and backward of the smart car; the steering gear controls the steering of the smart car; the second ultrasonic ranging module is set at the front end of the smart car to measure the distance between the obstacle and the smart car . The invention jointly controls the shooting mobile device loaded with the AI camera to stop moving through the sampling obtained following distance, obstacle avoidance events, and control instructions sent by the App, so as to ensure that the system realizes stable and reliable obstacle avoidance and parking functions.

Description

A system and method for automatically following and shooting obstacle avoidance

technical field

The invention belongs to the field of artificial intelligence, and relates to an automatic follow-up shooting obstacle avoidance system and method.

Background technique

With the popularization of smart devices, consumer users are increasingly using various smart electronic devices with increasing intensity and popularity. Among them, photography and video recording are the most popular functions in current smart devices, and consumers use them most frequently.

In the prior art, when people create video material with more flexible targets and more flexible scene switching, the method used is nothing more than manual follow-up photography, that is, aerial photography with remote control drones or hand-held cameras and stabilizers. The hand-held stabilizer requires the photographer to put a lot of energy to follow, including stabilizing the camera, which is laborious and laborious; while the tracking intensity of aerial photography is not enough, the function for targeted video shooting is not strong enough, and the battery life is poor. The follow-up method requires two or more followers, which is not in line with the original intention of fully intelligent automation. Automatic follow-up shooting can effectively solve the shortcomings of the above methods, but the distance measured by the AI visual ranging algorithm during the automatic follow-up shooting process has a large error at close range, and when the obstacle or target is too close, it is not enough. Achieving the ideal obstacle avoidance effect may cause the device to collide with the following target or obstacles. When the automatic follow-up shooting system with the gimbal is connected with different mobile carriers, facing different motors, it is necessary to ensure the safe and reliable realization of the parking function. The automatic follow-up shooting system with App control involves a variety of follow-up control modes, and it is necessary to ensure reliable and effective obstacle avoidance and parking functions in different modes.

SUMMARY OF THE INVENTION

In order to solve the above problems, the purpose of the present invention is to provide an automatic follow-up shooting obstacle avoidance system, including a pan-tilt camera and a shooting mobile device, wherein,

The pan-tilt camera includes a three-axis pan-tilt, a camera and a first ultrasonic ranging module, the camera is erected on the three-axis pan-tilt, and the camera shoots images or videos of the following target; the first ultrasonic ranging module is set On the heading axis of the three-axis gimbal, the first ultrasonic ranging module moves with the heading axis of the three-axis gimbal, and always faces the following target, and measures the distance between the following target and the three-axis gimbal;

The pan-tilt camera is placed on the photographing mobile device. The photographing mobile device includes a motor, a steering gear, an intelligent car and a second ultrasonic ranging module. The motor controls the intelligent car to move forward and backward; the steering gear controls the intelligent car. The second ultrasonic ranging module is arranged at the front end of the smart car to measure the distance between the obstacle and the smart car.

Preferably, a suspension type shock-absorbing structure is provided on the chassis of the smart car.

Preferably, the motor is a high-speed brushless DC type.

Preferably, a controller is also included, and the controller includes an MCU based on the ARM-M3/M4 core architecture.

Preferably, the controller is an STM32 series or GD32 series microcontroller chip.

Preferably, the camera ranging is based on a monocular ranging algorithm.

Based on the above purpose, the present invention also provides a method for automatically following and shooting obstacles, using the above system, including the following steps:

S10, the system starts, obtains the instant follow distance, sets the preset follow distance dis_follow and sends it to the PTZ camera;

S20, ultrasonic ranging, to determine whether obstacle avoidance is required;

S21, yes, set the obstacle avoidance flag;

S30, if no, judge the system distance dis_sys according to the gimbal camera ranging to select ultrasonic ranging or gimbal camera ranging;

S40, determine whether the system distance dis_sys is less than the parking distance;

S41, yes, set the speed zero setting flag;

S50, otherwise, follow the target slowly and return to S20;

Among them, after setting the obstacle avoidance sign or setting the speed zero sign, stop the motor running and stop the smart car. In S50, before following the target at a slow speed, when the system distance is measured by a camera, it will automatically follow at a variable speed; otherwise, when the system distance is measured by ultrasonic distance, it will follow at a slow speed.

Preferably, the ultrasonic ranging, judging whether obstacle avoidance is required, includes making the obstacle avoidance distance dis_obstacle, the distance dis_us1 measured by the first ultrasonic ranging module, and the distance dis_us2 measured by the second ultrasonic ranging module, when dis_us1 < dis_obstacle or dis_us2 When <dis_obstacle, obstacle avoidance is required.

Preferably, the judging system distance dis_sys according to the gimbal camera ranging to select ultrasonic ranging or gimbal camera ranging is to make the system sample the switching value of the following distance dis_cam_us1, and the gimbal camera measures the distance dis_cam, when dis_cam≤dis_cam_us1 , the system distance dis_sys=dis_us1, otherwise dis_sys=dis_cam.

Preferably, the judging whether the system distance dis_sys is smaller than the parking distance is to make the maximum following deviation dis_bias_max, and when dis_sys _< dis_follow-dis_bias_max, it is smaller than the parking distance.

Compared with the prior art, the present invention has at least the following beneficial effects: by combining ultrasonic ranging and AI camera (camera) ranging, ultrasonic ranging is the criterion for short distances, and AI camera ranging is the criterion for non-close distances. Ensure accurate ranging. The system follows and shoots the target according to the following distance set by the App. When the distance between the target and the AI camera is less than the following distance, it controls the shooting mobile device to stop. When the following distance is set to a small value, due to the large deviation of the AI ranging, the ultrasonic measurement is performed. The distance is used as the system distance. When the system distance is measured by ultrasonic and the obstacle avoidance distance is not reached, the vehicle speed is slowed down to a slow speed. When the measured distance is less than the obstacle avoidance distance, the vehicle stops to achieve effective obstacle avoidance. When the app sets the following distance to be not close, the AI distance measurement is less than the set following distance, and the car will stop directly. During the following process, use the App to send a stop command to stop directly. When the automatic mode is switched to the manual mode, the default is to stop. From the above, the present invention adopts different ranging methods by following the distance between the target and the shooting device, adds obstacle avoidance judgment at the same time, and combines App control to form a parking association, so as to ensure accurate and reliable obstacle avoidance and parking.

Description of drawings

Fig. 1 is the structural block diagram of the automatic following shooting obstacle avoidance system of the specific embodiment of the system of the present invention;

FIG. 2 is a flow chart of steps of a method for automatically following and photographing obstacle avoidance according to a method embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

On the contrary, the present invention covers any alternatives, modifications, equivalents and arrangements within the spirit and scope of the present invention as defined by the appended claims. Further, in order to give the public a better understanding of the present invention, some specific details are described in detail in the following detailed description of the present invention. The present invention can be fully understood by those skilled in the art without the description of these detailed parts.

System Example 1

Referring to FIG. 1, an automatic follow-up shooting obstacle avoidance system according to an embodiment of the present invention includes a pan-tilt camera 10 and a shooting mobile device 20, wherein,

The pan-tilt camera 10 includes a three-axis pan-tilt 12, a camera 11 and a first ultrasonic ranging module 13. The camera 11 is erected on the three-axis pan-tilt 12. The camera 11 shoots images or videos of the following targets, and processes the images to obtain Follow the distance between the target and the three-axis gimbal 12; the first ultrasonic ranging module 13 is arranged on the heading axis of the three-axis gimbal 12, and the first ultrasonic ranging module 13 moves with the heading axis of the three-axis gimbal 12, and always Facing the following target, measure the distance between the following target and the three-axis gimbal 12;

The pan-tilt camera 10 is placed on the shooting mobile device 20. The shooting mobile device 20 includes a motor 21, a steering gear 22, a smart car 23 and a second ultrasonic ranging module 24. The motor 21 controls the smart car 23 to move forward and backward; the steering gear 22 controls Steering of the smart car 23 ; the second ultrasonic ranging module 24 is arranged at the front end of the smart car 23 to measure the distance between the obstacle and the smart car 23 .

In a specific embodiment

The chassis of the smart car 23 is provided with a suspension shock-absorbing structure. The motor 21 is a high-speed brushless DC type. The steering control part of the steering gear 22 in the photographing mobile device 20 is nested with a layer of control loops for the angle of the yaw (course) axis.

System Example 2

The system can be controlled by APP, and the APP control includes mode switching, which switches the system operating mode between automatic and manual mode. The controller can select all MCUs based on ARM-M3/M4 core architecture, such as STM32 series, GD32 series or 32-bit microcontroller chips of other platforms. Preferably, a microcontroller chip with a model of GD32F330 is selected as the controller of this embodiment. , the controller includes a speed automatic control unit and an automatic direction control unit, and the two control units are program modules that can be executed by a micro-control chip with a model of GD32F330.

The controller in automatic control is used to receive control information such as following distance, mode switching, and parking sent by the App. When in the automatic following mode, it is determined whether the measurement distance of the first ultrasonic ranging module 13 is used as the system distance dis_sys according to the distance measured by the camera 11, and whether there is an obstacle in front of the second ultrasonic ranging module 24 is judged. When the target distance is followed When the system is too close, the first ultrasonic ranging module 13 regards the following target as an obstacle, and the first ultrasonic ranging module 13 and the second ultrasonic ranging module 24 realize stable obstacle avoidance and slow following. When there is no obstacle, the controller performs corresponding amplitude limiting, filtering, amplification and other processing after processing the speed and direction respectively, and then drives the three-axis gimbal 12, the rudder through the drive circuit in the three-axis gimbal 12, the steering gear 22 and the motor 21. The machine 22 and the motor 21 work. When the target is in motion and the position is constantly changing, the system can measure the rotation angle of the 12yaw axis of the three-axis gimbal, continuously obtain the angle deviation between the camera 11 and the front of the target, and control the 12yaw axis of the three-axis gimbal to continuously correct the direction to ensure the camera. 11 is always facing the target. Based on this, the distance obtained by the camera 11 ranging algorithm and the distance collected by the first ultrasonic ranging module 13 fixed on the yaw axis are processed jointly to determine whether it is necessary to slow down and avoid obstacles and stop. The above method is used to ensure that the target and the system are close to the system to stop in time, and when the front encounters an obstacle, the system is stopped in time.

In the case of no obstacle, according to the distance information obtained by the distance measurement of the camera 11, it is determined whether the distance reference of the control system is based on the image information of the camera 11 or the distance measured by the first ultrasonic ranging module 13. When the system distance is less than the following distance, That is, when dis_sys < dis _{_} follow, stop directly, the system distance is greater than the following distance, less than the switching standard and greater than the obstacle avoidance distance, that is, dis _obstacle ≤dis _sys ≤dis _cam2us , and dis _sys >dis _follow , set the speed to slow , that is, the speed of the smart car 23 is set to speed_low, the speed output is smoothed, and each speed change is limited to not exceed speed_delta_max; if it is less than the obstacle avoidance distance, that is, diS _sys <dis_obstacle, it indicates that the distance between the target and the shooting devices is too close , the speed zero setting flag is set in the speed control link, and the flag state is returned to the speed output limiting processing link to control direct parking; at the same time, when the second ultrasonic ranging module 24 encounters an obstacle, that is, dis _us2 <dis _obstacle , has reached the obstacle avoidance range, and directly avoids the obstacle and stops.

This system involves a variety of parking needs. In the automatic follow mode, the following distance set by the App is used to control the dis _follow . When the distance between the target and the device is less than the set following distance, encountering an obstacle, and the App controls the parking. The smart car 23 stops moving immediately; in manual mode, the obstacle avoidance function is shielded, and the App controls the movement and parking manually.

Method Example 1

Referring to Fig. 2, a method for automatically following and photographing obstacle avoidance, using the above system, includes the following steps:

S10, the system starts, obtains the instant follow distance, sets the preset follow distance _{dis_follow} and sends it to the PTZ camera;

S20, ultrasonic ranging, to determine whether obstacle avoidance is required;

S21, yes, set the obstacle avoidance flag;

S30, if no, judge the system distance dis_sys according to the gimbal camera ranging to select ultrasonic ranging or gimbal camera ranging;

S40, determine whether the system distance dis_sys is less than the parking distance;

S41, yes, set the speed zero setting flag;

S50, otherwise, follow the target slowly and return to S20;

Among them, after setting the obstacle avoidance sign or setting the speed zero sign, stop the motor running and stop the smart car. In S50, before following the target at a slow speed, when the system distance is measured by a camera, it will automatically follow at a variable speed; otherwise, when the system distance is measured by ultrasonic distance, it will follow at a slow speed.

In a specific embodiment, S20, ultrasonic ranging, judging whether obstacle avoidance is required, including making the obstacle avoidance distance dis_obstacle, the distance dis_us1 measured by the first ultrasonic ranging module, and the distance dis_us2 measured by the second ultrasonic ranging module, when dis_us1<dis_obstacle Or when dis_us2 < dis_obstacle, obstacle avoidance is required.

The S30 judges the system distance dis_sys according to the gimbal camera ranging to select ultrasonic ranging or gimbal camera ranging. In order to make the system sample the switching value of the following distance dis_cam_us1, the gimbal camera measures the distance dis_cam. When dis_cam≤dis_cam_us1, the system distance dis_sys =dis_us1, otherwise dis_sys=dis_cam.

S40, determine whether the system distance dis_sys is less than the parking distance, in order to make the maximum follow deviation dis_bias_max, when dis_sys _< dis_follow- _{dis_bias_max} , it is less than the parking distance.

After the system is started, the real-time following distance is obtained. The app sets the following distance dis _{_follow} and sends it to the PTZ camera. When the following distance is not set on the App side, the following distance is set as the default following distance; The second ultrasonic ranging module judges whether obstacle avoidance is required. When there is an obstacle, the obstacle avoidance sign is set. When there is no obstacle, the system distance is determined according to the camera AI ranging. dis_sys Select the distance measured by the first ultrasonic ranging module dis _us1 Or the camera AI ranging dis _cam , according to the set maximum follow deviation dis_bias_max, when the condition dis_sys < dis_follow-dis_bias_max is satisfied, the speed zero flag is set, and if it is not satisfied, the corresponding follow control is performed to control the motor and steering gear. Action, during the following control process, when the App controls the stop or switches to manual mode, the speed zero setting flag is set; when the control mode is switched to manual mode in the app, the default speed zero setting flag is set, and when the manually set speed value is not When it is 0, the speed zero setting flag in the manual mode is released. When the manual setting speed is 0, the speed zero setting flag is activated; when the speed zero setting flag or the obstacle avoidance flag is set, the control system sets the speed of the smart car to 0. 0, realize the parking control during the follow-up shooting process, and restore the normal operation of the system when the flag is reset. Further, in order to ensure that the mobile device will not overshoot during the traveling process, under the condition of ensuring the system response speed, smooth output processing is set for the speed output to ensure that each speed change will not be too large.

To sum up, the present invention takes the automatic shooting following system as a platform, combines the AI algorithm ranging of the pan-tilt camera with the ultrasonic ranging, and obtains the distance between the following target and the device, and selects different system distance references according to the following distance. Standard, after entering the buffer obstacle avoidance area, the speed reduction control is performed, and after entering the obstacle avoidance area, the mobile device is controlled to stop to realize the obstacle avoidance effect. In the process of automatic following, it receives the control signal of the App, and comprehensively controls the parking of the mobile device through the combination of the App, ultrasonic waves, and camera AI ranging to ensure that the system can stably realize the parking function.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

1. An automatic following shooting obstacle avoidance system is characterized by comprising a pan-tilt camera and a shooting moving device, wherein,

the holder camera comprises a three-axis holder, a camera and a first ultrasonic ranging module, the camera is arranged on the three-axis holder, and the camera shoots an image or a video of a following target; the first ultrasonic ranging module is arranged on a course axis of the three-axis holder, moves along with the course axis of the three-axis holder and is always opposite to the following target, and the distance between the following target and the three-axis holder is measured;

the pan-tilt camera is placed on the shooting moving device, the shooting moving device comprises a motor, a steering engine, an intelligent trolley and a second ultrasonic ranging module, and the motor controls the intelligent trolley to move forwards and backwards; the steering engine controls the steering of the intelligent trolley; the second ultrasonic ranging module is arranged at the front end of the intelligent trolley and used for measuring the distance between the obstacle and the intelligent trolley.

2. The automatic follow-up shooting obstacle avoidance system according to claim 1, wherein a suspension type shock absorption structure is arranged on a chassis of the intelligent trolley.

3. The automatic follow-up shooting obstacle avoidance system according to claim 1, wherein the motor is of a high-speed brushless dc type.

4. The automatic follow-up shooting obstacle avoidance system of claim 1, further comprising a controller, wherein the controller comprises an MCU based on an ARM-M3/M4 kernel architecture.

5. The automatic follow-up shooting obstacle avoidance system according to claim 4, wherein the controller is an STM32 series or GD32 series micro control chip.

6. The automatic follow-up shooting obstacle avoidance system of claim 1, wherein the camera ranging is based on a monocular ranging algorithm.

7. An automatic follow-up shooting obstacle avoidance method, characterized in that the system of any one of claims 1 to 6 is adopted, and the method comprises the following steps:

s10, starting the system, acquiring the instant following distance, setting the preset following distance dis _ follow, and sending the preset following distance dis _ follow to the pan-tilt camera;

s20, ultrasonic ranging is carried out, and whether obstacle avoidance is needed or not is judged;

s21, if yes, setting an obstacle avoidance mark;

s30, if not, selecting ultrasonic ranging or pan-tilt camera ranging according to the pan-tilt camera ranging judgment system distance dis _ sys;

s40, judging whether the system distance dis _ sys is smaller than the parking distance;

s41, if yes, setting a speed zero setting mark;

s50, if not, slowly following the target, and returning to S20;

after the obstacle avoidance mark is set or the speed zero mark is set, the motor is stopped to run, and the intelligent trolley is stopped.

8. The method as claimed in claim 7, wherein the ultrasonic ranging determines whether an obstacle avoidance is required, including enabling the obstacle avoidance distance dis _ obstacle, the first ultrasonic ranging module measures the distance dis _ us1, the second ultrasonic ranging module measures the distance dis _ us2, and when dis _ us1 < dis _ obstacle or dis _ us2 < dis _ obstacle, the obstacle avoidance is required.

9. The method according to claim 8, wherein the selecting of the ultrasonic ranging or pan/tilt/zoom camera ranging is determined according to pan/tilt/zoom camera ranging distance dis _ sys, wherein in order to make the system sample the switching value dis _ cam _ us1 following the distance, the pan/tilt camera measures the distance dis _ cam, and when dis _ cam is not greater than dis _ cam _ us1, the system distance dis _ sys is not greater than dis _ us1, otherwise, the distance is not greater than dis _ cam.

10. The method according to claim 9, wherein the system distance dis _ sys is determined to be less than the stopping distance, such that the maximum following deviation dis _ bias _ max is smaller than the stopping distance when dis _ sys < dis _ follow-dis _ bias _ max.

Images