CN118311976A - Multi-UAV obstacle avoidance method, system, device and medium based on CFS - Google Patents

Abstract

The present invention relates to the field of machine learning technology, and specifically to a multi-UAV obstacle avoidance method, system, device and medium based on CFS. The method comprises: obtaining a depth image collected by a UAV of a current environment, encoding the depth image to obtain a visual representation, integrating the current speed, target position and the visual representation of the UAV to obtain an observation vector; inputting the observation vector into a pre-trained obstacle avoidance model to obtain the flight action of the UAV; the obstacle avoidance model is constructed based on a deep reinforcement learning network, the obstacle avoidance model comprises a parallel visual feature extraction network, a policy network and a value network, the policy network comprises a plurality of linear layers, a CFS module is embedded between adjacent linear layers, and the CFS module is used to select causal features directly related to the obstacle avoidance task from the visual representation; controlling the flight of the UAV based on the flight action of the UAV until it reaches the target position; the present invention can provide a more generalized obstacle avoidance strategy in a complex and unknown environment.

Description

Multi-UAV obstacle avoidance method, system, device and medium based on CFS

Technical Field

The present invention relates to the field of machine learning technology, and in particular to a multi-UAV obstacle avoidance method, system, device and medium based on CFS.

Background technique

With the rapid development of unmanned aerial vehicle (UAV) systems, they have been widely used in many fields, such as agriculture, search and rescue, mining, and patrol inspection. In order to achieve effective collaboration between multiple UAVs, it is particularly important to find an optimal path to avoid obstacles and reach the target location, especially in large-scale UAV groups. Traditional multi-UAV obstacle avoidance methods mainly rely on real-time simultaneous localization and mapping (SLAM) technology, using sensors such as laser radar (LiDAR) to perceive the surrounding environment and generate trajectories through path planning. In addition, in order to improve the performance of the SLAM system, prior map information is usually introduced. However, these traditional methods usually require a lot of computing resources and are limited by the existing prior map information, making it difficult to adapt to unknown environments.

Summary of the invention

In view of this, the purpose of the embodiments of the present invention is to provide a multi-UAV obstacle avoidance method, system, device and medium based on CFS to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

On the one hand, an embodiment of the present invention provides a multi-UAV obstacle avoidance method based on CFS, the method comprising the following steps:

Obtain a depth image of the current environment collected by the drone, encode the depth image through an encoder to obtain a visual representation, and integrate the current speed of the drone, the target position, and the visual representation to obtain an observation vector;

The observation vector is input into a pre-trained obstacle avoidance model to obtain the flight action of the UAV; wherein the obstacle avoidance model is constructed based on a deep reinforcement learning network, the obstacle avoidance model includes a parallel visual feature extraction network, a policy network and a value network, the policy network includes a plurality of linear layers, and a CFS module is embedded between adjacent linear layers, and the CFS module is used to select causal features directly related to the obstacle avoidance task from the visual representation;

The UAV is controlled to fly based on the flight action of the UAV until it reaches a target position.

Optionally, selecting causal features directly related to the obstacle avoidance task from the visual representation comprises:

Get visual representations;

A differentiable binary mask is generated through a trainable weight in the CFS module and a multi-layer perceptron, which is embedded into the policy network to activate the causal feature channel and suppress the non-causal feature channel to eliminate the non-causal features in the visual representation and retain the causal features.

Optionally, the obstacle avoidance model is trained in the following manner:

Acquire a sample observation vector; the sample observation vector includes a sample velocity, a sample target position, and a sample depth image;

Iteratively training the visual feature extraction network based on the sample depth image to obtain a trained visual feature extraction network;

Iteratively train the policy network based on the sample depth image, sample speed and sample target position, perform value evaluation on the drone action iteratively output by the policy network through the value network, and iterate the policy network according to the value evaluation result; guide the training and parameter optimization of the policy network by maximizing the obstacle avoidance reward function to obtain a trained policy network and value network;

The trained visual feature extraction network, the trained strategy network and the value network are used as obstacle avoidance models.

Optionally, the iteratively training the visual feature extraction network based on the sample depth image to obtain a trained visual feature extraction network includes:

Inputting the sample observation vector into a visual feature extraction network for iterative training;

During the iterative training process, a reconstructed image obtained by the visual feature extraction network is obtained, and a first loss value of the visual feature extraction network is calculated based on the depth image and the corresponding reconstructed image;

When the first loss value drops to a first threshold, the iterative training is stopped to obtain a trained visual feature extraction network.

Optionally, obtaining a reconstructed image reconstructed by a visual feature extraction network includes:

The depth image is converted into multi-dimensional latent features through a four-layer convolutional network and a fully connected layer, and the reconstructed image is reconstructed by the decoder.

Optionally, the iterative training of the policy network based on the sample depth image, the sample speed and the sample target position comprises:

The target reaching reward function is used to evaluate the action of the drone output by the policy network until the second loss value calculated according to the target reaching reward function is lower than the second loss threshold.

Optionally, the performing value evaluation on the drone action iteratively output by the strategy network through the value network includes:

The expected cumulative reward output by the value network is evaluated using the obstacle avoidance reward function until the third loss value calculated according to the obstacle avoidance reward function is lower than the third loss threshold.

On the other hand, an embodiment of the present invention provides a multi-UAV obstacle avoidance system based on CFS, including:

The first module is used to obtain a depth image of the current environment collected by the drone, encode the depth image through an encoder to obtain a visual representation, and integrate the current speed, target position, and the visual representation of the drone to obtain an observation vector;

The second module is used to input the observation vector into a pre-trained obstacle avoidance model to obtain the flight action of the UAV; wherein the obstacle avoidance model is constructed based on a deep reinforcement learning network, and the obstacle avoidance model includes a parallel visual feature extraction network, a policy network and a value network, and the policy network includes multiple linear layers, and a CFS module is embedded between adjacent linear layers, and the CFS module is used to select causal features directly related to the obstacle avoidance task from the visual representation;

The third module is used to control the flight of the drone based on the flight action of the drone until it reaches the target position.

On the other hand, an embodiment of the present invention provides a multi-UAV obstacle avoidance device based on CFS, comprising:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

On the other hand, an embodiment of the present invention provides a computer-readable storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to perform the above method.

The embodiments of the present invention include the following beneficial effects: By introducing the CFS module, the policy network can better eliminate the influence of non-causal factors in the input features, thereby providing a more generalized obstacle avoidance strategy in complex and unknown environments.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a schematic diagram of a process flow of a multi-UAV obstacle avoidance method based on CFS provided by an embodiment of the present invention;

FIG2 is an architecture diagram of a multi-UAV obstacle avoidance method based on CFS provided by an embodiment of the present invention;

FIG3 is an architecture diagram of a multi-UAV obstacle avoidance system based on CFS provided by an embodiment of the present invention;

FIG4 is a structural block diagram of a CFS-based multi-UAV obstacle avoidance device provided in an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations/steps, nor must they be executed in the order described. For example, some operations/steps can be decomposed, and some operations/steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

First, several names involved in the present invention are explained below:

Unmanned Aerial Vehicle (UAV) is an aircraft that does not require direct human control and can fly autonomously or be operated by remote control.

Deep Reinforcement Learning (DRL) is a branch of machine learning that combines the concepts of deep learning and reinforcement learning. Deep learning is a machine learning technique based on neural networks, while reinforcement learning is a decision-making process in which an agent learns how to make decisions to maximize cumulative rewards by interacting with the environment.

The Soft Actor-Critic (SAC) algorithm is a method for solving reinforcement learning problems in discrete action spaces and continuous action spaces. The SAC algorithm is an off-policy reinforcement learning algorithm, which means that it can use previous experience to improve the strategy, not just the current decision.

Regularized Autoencoder (RAE) is a variant of autoencoder that enhances its performance and generalization ability by adding a regularization term to the loss function of the autoencoder. Autoencoder is an unsupervised learning algorithm used to learn an effective representation of data, namely feature dimensionality reduction or feature learning.

The Actor-Critic algorithm framework is a commonly used strategy learning method in reinforcement learning. It combines the update mechanism of the value module (Critic) and the policy module (Actor). The policy network is the core component of the reinforcement learning algorithm to implement the decision-making process of the intelligent agent. Its design and training directly affect the performance of the intelligent agent on specific tasks. In reinforcement learning (RL), the policy module (Actor) is used in combination with the value module (Critic). The value module evaluates the performance of the current strategy, while the policy module updates the strategy based on these evaluations. The policy module is one of the core components, which defines how the intelligent agent (agent) chooses actions in a given state.

The Causal Feature Selection (CFS) module is a method used in machine learning models to identify and select input features that have a causal relationship with the target variable. This method is different from traditional feature selection techniques, which usually select features based on the correlation between the features and the target variable. The goal of the CFS module is to identify features that are not only correlated with the target variable, but may also have a direct impact on it.

Success-Weighted Path Length (SPL) is a metric used in reinforcement learning to measure policy performance, especially in tasks that require the agent to reach a specific goal. SPL combines the binary feedback of whether the task was successful or not and the number of steps the agent took to reach the goal.

The existing technology mainly uses deep reinforcement learning (DRL) for multi-UAV obstacle avoidance navigation, and realizes end-to-end obstacle avoidance navigation without environmental maps through sensor input. Among them, the SAC+RAE method can train and learn collision avoidance strategies in a known environment in multi-UAV obstacle avoidance navigation, so as to successfully avoid obstacles and reach the target point in a known environment. However, when facing an unknown obstacle environment, the success rate of the obstacle avoidance navigation of the UAV will drop significantly, revealing the problem of poor generalization ability of the deep reinforcement learning model. In order to improve the generalization ability of the DRL model, the researchers introduced causal representation learning, and identified key features by constructing a causal structure to cope with OOD scenarios. The causal feature selection (CFS) module proposed in the present invention can effectively extract the causal factors in the visual representation by integrating it into the policy network, reduce the interference of non-causal factors, and significantly improve the obstacle avoidance navigation performance of the UAV in unknown scenes. Compared with the existing technology, it shows stronger generalization and robustness.

The disadvantages of the prior art are:

Insufficient generalization ability: Although the obstacle avoidance navigation method based on DRL, especially SAC+RAE, performs well in known environments, its navigation success rate drops significantly when facing unseen obstacles. This shows that the generalization ability of existing technologies in unknown environments is limited, and it is difficult to effectively adapt to obstacles that have not been seen during training.

The SAC+RAE method may falsely associate the shape of the obstacle with the obstacle avoidance strategy during the training process, causing the obstacle avoidance strategy of the drone to fail when encountering obstacles of different shapes.

Existing techniques may fail to distinguish causal factors (such as obstacle distance and drone speed) from non-causal factors (such as obstacle shape and background texture) in visual information. As a result, non-causal factors may negatively affect policy predictions through spurious correlations when the test environment changes.

Cause:

DRL methods usually assume that training and test data are independent and identically distributed, but in real applications, the actual deployment environment is often quite different from the training environment, which violates the independent and identically distributed assumption and thus causes the generalization problem of deep reinforcement learning.

The regularized autoencoder (RAE) used by SAC+RAE may implicitly encode all visual information, including non-causal factors irrelevant to the task, which may cause the policy network to be affected by spurious correlations.

Instead of using a causal inference framework to identify and exploit causal relationships in data, existing techniques rely on associative learning, which can lead to degraded generalization performance in OOD scenarios.

This invention aims to solve the obstacle avoidance and navigation problem of drone swarms in complex outdoor environments, especially how to achieve robust navigation performance and effective obstacle avoidance through deep reinforcement learning methods based on causal feature selection when facing unknown environments and obstacles. This technical challenge belongs to the field of robotics, specifically involving autonomous navigation and obstacle avoidance strategies for drones in multi-agent systems. In addition, it also covers the application of deep reinforcement learning in the field of artificial intelligence, and the role of computer vision in environmental perception and path planning. Ultimately, this technology can provide support for multiple practical application areas such as agriculture, search and rescue, mining, and patrol inspections, and improve the operational efficiency and safety of drones through advanced control strategies.

As shown in FIG. 1 and FIG. 2 , FIG. 1 is a multi-UAV obstacle avoidance method based on CFS provided by an embodiment of the present invention, and the method comprises the following steps:

S100, obtaining a depth image of the current environment collected by the drone, encoding the depth image through an encoder to obtain a visual representation, and integrating the current speed, target position, and the visual representation of the drone to obtain an observation vector;

The drone captures depth images through its front central camera, which contain distance information from the drone's viewpoint to the surfaces of various objects in the environment. At the same time, the drone's current speed and target position information are also integrated into the observation vector ,in, Indicates the current speed. Indicates the target location, Indicates a visual representation.

S200, inputting the observation vector into a pre-trained obstacle avoidance model to obtain a flight action of the drone; wherein the obstacle avoidance model is constructed based on a deep reinforcement learning network, the obstacle avoidance model includes a parallel visual feature extraction network, a policy network and a value network, the policy network includes a plurality of linear layers, a CFS module is embedded between adjacent linear layers, and the CFS module is used to select causal features directly related to the obstacle avoidance task from the visual representation;

The CFS module introduces a causal feature selection mechanism during the training process to automatically identify and select visual features that are causally related to the obstacle avoidance task, while filtering out non-causal features that may lead to incorrect decisions, thereby enabling the DRL model to have higher generalization capabilities in unknown environments.

S300: Control the flight of the drone based on the flight action of the drone until the drone reaches a target position.

The non-sparse reward function adopted by the present invention provides more learning signals for the UAV, which not only speeds up the learning process but also improves the robustness and adaptability of the obstacle avoidance strategy.

The present invention enables multiple drones to maintain a high obstacle avoidance success rate and stable behavior performance when facing obstacles and environmental backgrounds that they have never seen before. By introducing the CFS module, the policy network of the present invention can better resist the influence of non-causal factors in the input features, thereby providing a more reliable obstacle avoidance strategy in complex and unknown environments.

In some improved embodiments, the step of selecting causal features directly related to the obstacle avoidance task from the visual representation includes:

Get visual representations;

A differentiable binary mask is generated through a trainable weight in the CFS module and a multi-layer perceptron, which is embedded into the policy network to activate the causal feature channel and suppress the non-causal feature channel to eliminate the non-causal features in the visual representation and retain the causal features.

The CFS module is embedded in the policy network, which is responsible for selecting causal features directly related to the obstacle avoidance task from the extracted visual representation. The CFS module generates a differentiable binary mask through a trainable weight and a small multi-layer perceptron (MLP) to activate the causal feature channel and suppress the non-causal feature channel. The present invention relies on the differentiable binary mask generated by the CFS module to select causal features, which is more generalizable and robust than the method of directly using all input features in the traditional DRL model. By embedding the CFS module in the policy network, using a small multi-layer perceptron (MLP) and ReLU activation function to convert the trainable weights into binary masks, end-to-end learning of the feature selection process is achieved.

Since the binary mask generated by the CFS module is differentiable, this allows it to be optimized end-to-end in the DRL framework through the back-propagation algorithm. This optimization method ensures that the CFS module can work effectively with the policy network and ensures the adaptability and robustness of the learned policy for obstacle avoidance tasks.

In some improved embodiments, the obstacle avoidance model is trained in the following manner:

S210, obtaining a sample observation vector; the sample observation vector includes a sample velocity, a sample target position, and a sample depth image;

S220, iteratively training the visual feature extraction network based on the sample depth image to obtain a trained visual feature extraction network;

S230, iteratively training the policy network based on the sample depth image, sample speed, and sample target position, performing value evaluation on the drone action iteratively output by the policy network through the value network, and iterating the policy network according to the value evaluation result; guiding the training and parameter optimization of the policy network by maximizing the obstacle avoidance reward function, and obtaining a trained policy network and value network;

S240, using the trained visual feature extraction network, the trained strategy network and the value network as an obstacle avoidance model.

Specifically, the policy network outputs the action of the drone, and the value network evaluates the expected cumulative reward of the action; the improved Actor-Critic algorithm is trained using the soft actor-critic algorithm, which includes a policy network and a value network composed of a three-layer multilayer perceptron (MLP). The present invention also designs a non-sparse reward function, which combines the target arrival reward function and the obstacle avoidance reward function to promote the drone to learn an effective obstacle avoidance strategy, and verifies the effectiveness of the proposed method through testing in a simulated environment. The reward function designed by the present invention not only considers the reward for the drone to reach the target, but also carefully considers obstacle avoidance and path efficiency. This design provides the drone with the effective feedback it needs when exploring the environment and learning obstacle avoidance strategies, thereby speeding up the learning process and ultimately improving the quality of the strategy.

In some improved embodiments, the iterative training of the visual feature extraction network based on the sample depth image to obtain a trained visual feature extraction network includes:

Inputting the sample observation vector into a visual feature extraction network for iterative training;

During the iterative training process, a reconstructed image obtained by the visual feature extraction network is obtained, and a first loss value of the visual feature extraction network is calculated based on the depth image and the corresponding reconstructed image;

When the first loss value drops to a first threshold, the iterative training is stopped to obtain a trained visual feature extraction network.

Specifically, the regularized autoencoder (RAE) receives the observation information of the drone and encodes it into a 50-dimensional latent feature. This step involves converting the depth image into a latent feature through a four-layer convolutional network and a fully connected layer, and then reconstructing the reconstructed image corresponding to the depth image through the decoder to capture the key visual information in the environment. The first loss value is calculated by the first loss function J(RAE).

In some improved embodiments, the step of obtaining a reconstructed image obtained by reconstructing a visual feature extraction network includes:

The depth image is converted into multi-dimensional latent features through a four-layer convolutional network and a fully connected layer, and the reconstructed image is reconstructed by the decoder.

In some improved embodiments, the iterative training of the policy network based on the sample depth image, the sample speed and the sample target position includes:

The target reaching reward function is used to evaluate the action of the drone output by the policy network until the second loss value calculated according to the target reaching reward function is lower than the second loss threshold.

In some improved embodiments, the value evaluation of the drone action iteratively output by the policy network through the value network includes:

The expected cumulative reward output by the value network is evaluated using the obstacle avoidance reward function until the third loss value calculated according to the obstacle avoidance reward function is lower than the third loss threshold.

The second loss value Z _π is calculated by the second loss function J(π). The third loss value Z _Q is calculated by the third loss function J(Q).

This paper proposes a robust policy learning method for multi-UAV obstacle avoidance, which improves the generalization ability of deep reinforcement learning (DRL) in unknown environments by integrating a causal feature selection (CFS) module.

For the obstacle avoidance and navigation tasks of multiple UAVs in complex environments, the existing deep reinforcement learning (DRL) method has limited generalization ability in unknown environments, resulting in poor obstacle avoidance strategies in unknown environments. In practical applications, obstacle avoidance and navigation of UAVs in unknown environments is a common task. The present invention provides a DRL method integrated with a causal feature selection (CFS) module, which can improve the obstacle avoidance and navigation performance of UAVs in unknown complex environments.

The present invention innovatively applies causal feature selection technology to the obstacle avoidance navigation strategy learning of UAVs, which is a DRL method based on causal representation learning. The present invention improves the DRL model and proposes a policy network with an integrated CFS module. By performing causal feature selection on the visual input of the UAV, non-causal factors irrelevant to the obstacle avoidance task are filtered out, thereby reducing the impact of false correlation and improving the generalization ability of the policy network.

Compared with the existing technology, the method of the present invention shows superior performance in terms of success rate, SPL, average speed and other indicators in scenes with unknown backgrounds and obstacles, which proves the effectiveness and robustness of the obstacle avoidance navigation strategy learned by this method in unknown environments.

Compared with the prior art, the advantages of the present invention are:

By designing a causal feature selection (CFS) module, the present invention can better adapt to unknown environments and improve the navigation success rate of UAVs in unseen obstacles and backgrounds.

The CFS module extracts causal factors from visual representation and eliminates spurious correlations between non-causal factors and action prediction, thereby avoiding the degradation of drone generalization performance caused by spurious correlations.

The present invention reduces the impact of non-causal factors on strategy learning by clearly distinguishing causal and non-causal factors in visual information, thereby enhancing the generalization performance of the DRL model for unknown environmental changes.

By integrating the CFS module into the policy network, the present invention is able to more effectively learn the optimal obstacle avoidance navigation strategy and maintain a high success rate even in the face of complex and unknown environmental conditions.

Referring to FIG. 3 , an embodiment of the present invention provides a multi-UAV obstacle avoidance system based on CFS, including:

The first module is used to obtain a depth image of the current environment collected by the drone, encode the depth image through an encoder to obtain a visual representation, and integrate the current speed, target position, and the visual representation of the drone to obtain an observation vector;

The second module is used to input the observation vector into a pre-trained obstacle avoidance model to obtain the flight action of the UAV; wherein the obstacle avoidance model is constructed based on a deep reinforcement learning network, and the obstacle avoidance model includes a parallel visual feature extraction network, a policy network and a value network, and the policy network includes multiple linear layers, and a CFS module is embedded between adjacent linear layers, and the CFS module is used to select causal features directly related to the obstacle avoidance task from the visual representation;

The third module is used to control the flight of the drone based on the flight action of the drone until it reaches the target position.

It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

Referring to FIG. 4 , an embodiment of the present invention provides a multi-UAV obstacle avoidance device based on CFS, comprising:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

It can be seen that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

In addition, the embodiment of the present application also discloses a computer program product or a computer program, and the computer program product or the computer program is stored in a computer-readable storage medium. The processor of the computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above method. Similarly, the contents in the above method embodiment are all applicable to the storage medium embodiment, and the functions specifically implemented by the storage medium embodiment are the same as those in the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method embodiment.

The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules/units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And/or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and/or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character "/" generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims (10)

1. A CFS-based multi-unmanned aerial vehicle obstacle avoidance method, the method comprising the steps of:

acquiring a depth image acquired by an unmanned aerial vehicle for a current environment, encoding the depth image through an encoder to obtain a visual representation, and integrating the current speed, the target position and the visual representation of the unmanned aerial vehicle to obtain an observation vector;

Inputting the observation vector into a pre-trained obstacle avoidance model to obtain the flight action of the unmanned aerial vehicle; the obstacle avoidance model is constructed based on a deep reinforcement learning network, and comprises a parallel visual feature extraction network, a strategy network and a value network, wherein the strategy network comprises a plurality of linear layers, CFS modules are embedded between adjacent linear layers, and the CFS modules are used for selecting causal features directly related to an obstacle avoidance task from visual representation;

and controlling the unmanned aerial vehicle to fly based on the flying action of the unmanned aerial vehicle until reaching a target position.

2. The method of claim 1, wherein selecting causal features from the visual representation that are directly related to obstacle avoidance tasks comprises:

Acquiring a visual representation;

A differentiable binary mask is generated by a trainable weight and a multi-layer perceptron in the CFS module, the binary mask is embedded in the strategy network, the causal feature channel is activated and the non-causal feature channel is suppressed to eliminate non-causal features in the visual representation and preserve causal features.

3. The method of claim 1, wherein the obstacle avoidance model is trained by:

obtaining a sample observation vector; the sample observation vector includes a sample velocity, a sample target position, and a sample depth image;

Performing iterative training on the visual feature extraction network based on the sample depth image to obtain a trained visual feature extraction network;

Performing iterative training on the strategy network based on the sample depth image, the sample speed and the sample target position, performing value evaluation on unmanned aerial vehicle operation which is iteratively output by the strategy network through a value network, and iterating the strategy network according to a value evaluation result; training and parameter optimization of the strategy network are guided by a mode of maximizing the obstacle avoidance reward function, and a trained strategy network and a trained value network are obtained;

and taking the trained visual feature extraction network, the trained strategy network and the value network as an obstacle avoidance model.

4. A method according to claim 3, wherein the iteratively training the visual feature extraction network based on the sample depth image to obtain a trained visual feature extraction network comprises:

Inputting the sample observation vector into a visual feature extraction network for iterative training;

In the iterative training process, a reconstructed image obtained by reconstructing a visual feature extraction network is obtained, and a first loss value of the visual feature extraction network is calculated based on the depth image and the corresponding reconstructed image;

and stopping iterative training when the first loss value is reduced to a first threshold value, and obtaining a trained visual feature extraction network.

5. The method of claim 4, wherein the acquiring the reconstructed image of the reconstruction of the visual feature extraction network comprises:

the depth image is converted into multi-dimensional latent features through a four-layer convolution network and a full connection layer, and the obtained reconstructed image is reconstructed through a decoder.

6. The method of claim 3, wherein the iteratively training the policy network based on the sample depth image, sample velocity, and sample target location comprises:

and evaluating the action of the unmanned aerial vehicle output by adopting the target arrival reward function to the strategy network until a second loss value calculated according to the target arrival reward function is lower than a second loss threshold value.

7. A method according to claim 3, wherein said evaluating the value of unmanned aerial vehicle actions iteratively output by said policy network over a value network comprises:

And evaluating the expected accumulated rewards output by the value network by adopting the obstacle avoidance rewards function until a third loss value calculated according to the obstacle avoidance rewards function is lower than a third loss threshold value.

8. A CFS-based multi-unmanned aerial vehicle obstacle avoidance system, the system comprising:

the first module is used for acquiring a depth image acquired by the unmanned aerial vehicle for the current environment, encoding the depth image through an encoder to obtain a visual representation, and integrating the current speed, the target position and the visual representation of the unmanned aerial vehicle to obtain an observation vector;

the second module is used for inputting the observation vector into a pre-trained obstacle avoidance model to obtain the flight action of the unmanned aerial vehicle; the obstacle avoidance model is constructed based on a deep reinforcement learning network, and comprises a parallel visual feature extraction network, a strategy network and a value network, wherein the strategy network comprises a plurality of linear layers, CFS modules are embedded between adjacent linear layers, and the CFS modules are used for selecting causal features directly related to an obstacle avoidance task from visual representation;

and the third module is used for controlling the unmanned aerial vehicle to fly based on the flying action of the unmanned aerial vehicle until reaching the target position.

9. Many unmanned aerial vehicle keep away barrier device based on CFS, its characterized in that includes:

at least one processor;

At least one memory for storing at least one program;

the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method of any one of claims 1 to 7.

10. A computer readable storage medium, in which a processor executable program is stored, characterized in that the processor executable program is for performing the method according to any one of claims 1 to 7 when being executed by a processor.