CN114024639B - Distributed channel allocation method in wireless multi-hop network - Google Patents

Abstract

The invention relates to the field of wireless network communications, and specifically to a distributed channel allocation method in a wireless multi-hop network, which includes adopting a physical architecture that at least includes a physical device layer, a computing layer and a network service layer. The physical device layer is randomly deployed in the network. n wireless nodes form a multi-hop wireless communication network. Each node acts as an autonomous intelligent agent and interacts with the uncertain network environment through the local decision-making module; the convergence node in the computing layer is responsible for all other sites in the network. The collected data is aggregated, analyzed and processed, and the node has edge computing function, and can train an asynchronous DRL model based on the empirical information collected distributedly by the node, model the multi-channel allocation problem as a POMDP problem, and use the trained asynchronous DRL Model performs channel allocation; the invention solves the problems of hidden terminals and exposed terminals in high-density multi-hop wireless networks, and effectively avoids data conflicts and waste of channel resources.

Description

A distributed channel allocation method in wireless multi-hop networks

Technical field

The invention relates to the field of wireless network communications, and in particular to a distributed channel allocation method in a wireless multi-hop network.

Background technique

Multi-channel media access control (MMAC) technology can enable communication links that interfere with each other in single-channel communication to achieve interference-free data transmission in multiple orthogonal channels. MMAC can effectively avoid single-channel interference problems and improve the throughput of the entire network. Therefore, it is considered to be a technology with great potential to alleviate the shortage of channel resources in wireless networks. Although multi-channel communication has many advantages over single-channel communication, it brings many new problems:

Channel allocation and negotiation: The most basic and important issue in multi-channel MAC communication technology is how to reasonably allocate channel resources to ensure that each node can maximize the network capacity of the entire network under the premise of normal communication. In addition, before communication, nodes need to negotiate the use of the channel to ensure that the two communication nodes work on the same channel during data transmission.

Multi-channel broadcast: A wireless network based on a single-channel model can easily implement broadcast because each sensor node is on the same channel; however, in a multi-channel environment, when a node broadcasts, since the nodes are distributed in multiple channels As a result, some nodes cannot receive broadcast content. The broadcast function plays an important role in network applications. Therefore, how to implement the broadcast function is another difficult problem faced by multi-channel communication.

Multi-hop hidden terminals and exposed terminals: As shown in Figure 1, multi-hop hidden terminals are nodes within the communication range of the receiving node but outside the communication range of the sending node. Since these nodes cannot receive the data sent by the sending node, they may send data to the same receiving node, causing data transmission conflicts. In high-density situations, the hidden terminal problem can cause unnecessary data conflicts and greatly reduce network performance. The multi-hop exposed terminal problem refers to a node that is within the coverage of the sending node but outside the coverage of the receiving node. The exposed terminal delays sending because it hears the sending of the sending node. Exposing the presence of terminals will lead to unnecessary waste of channel resources.

Contents of the invention

In order to effectively reduce interference and data conflicts in the network, improve channel utilization and system throughput, and ensure the reliability of data service transmission between nodes, the present invention proposes a distributed channel allocation method in a wireless multi-hop network, which adopts at least It includes the physical architecture of the physical device layer, computing layer and network service layer. The physical device layer consists of n wireless nodes randomly deployed in the network to form a multi-hop wireless communication network. Each node acts as an autonomous intelligent agent. The local decision-making module interacts with the uncertain network environment; the aggregation node of the computing layer is responsible for aggregating, analyzing and processing data collected by other sites in the network, and the node has edge computing capabilities or uses dedicated edge server nodes. Offload the computing tasks of the node, and train the asynchronous DRL model based on the empirical information collected distributedly by the node. Model the multi-channel allocation problem as a POMDP problem, and use the asynchronous DRL model trained by the centralized node or edge server to perform distributed channel allocation. distribute.

Furthermore, the multi-channel allocation problem is modeled as a POMDP problem, that is, the Agent observes the current network state s and performs action a in time period t, and after executing action a, it transitions to the network state of the next time period with state transition probability P s′, and obtain the corresponding reward R from the environment, then the POMDP problem is expressed as:

M＝<S,A,P,R,γ>;

Among them, M represents the POMDP problem model; S is the state set representing the state space; A is the action set representing the action space, where the action a∈A represents the channel number that the node wants to switch; R is the reward function; γ is the discount factor. That is, in a given environmental state s∈S, if the Agent performs action a∈A, the environmental state will migrate from s to s′, that is, s→s′, and at the same time obtain the corresponding reward R from the environment.

Further, the environmental state observed by node i in the t-th time period Expressed as:

in, It represents the occupancy of each wireless channel by the neighbor nodes of node i, that is, the potential interference degree of each channel; K is the number of available channels, and N refers to the number of nodes;/> Indicates the occupation of channel j by neighbor nodes of node i in the tth time period,/> Indicates that there is a neighbor node of node i using channel j,/> Indicates that there is a neighbor node of node i using channel j;/> n _i,o is the total number of neighbor nodes of node i.

Furthermore, when a node performs action a and moves from state s to the next state s′, the reward R obtained from the environment can be expressed as:

Among them, R(s,a) is the reward R after node i switches the channel to channel k in the t-th data period, that is, R=R(s,a); Indicates whether there is a neighbor node of node i using channel k in the current cycle: if there is no neighbor node of node i using channel k, then/> On the contrary,/> is the probability of successful transmission by the neighbor of node i in the tth time period.

Further, the asynchronous DRL model deployed at the computing layer includes the current network, target network, error calculation module and experience pool, as well as a decision-making module deployed locally on the wireless node. The network structure of the local decision-making module is the same as the current network. Parameters are obtained periodically from edge nodes; where:

The target network fixes the network parameters and obtains the target value function, The current network is used to evaluate policy update parameters and approximate the value function;

The parameters θ of the current network are updated every time period; the parameters θ of the target network ^are updated every fixed number of time periods and remain unchanged during the period;

The experience e=<s,a,r,s′>,s,s′∈S,a∈A in the experience pool is collected asynchronously from the wireless multi-hop network environment by the nodes in the network;

The error calculation module updates the parameters of the current network through the TD deviation calculated between the target network and the current network; in addition, the parameters of the current network are copied to the target network at regular intervals.

Furthermore, the objective value function The calculation includes:

Among them, R(s _t , a _t ) is the node i∈[1,N] (N is the number of nodes). After executing the action a _t ∈A in the t-th time period state s _t ∈S, in the t-th time period The reward obtained; Q(s _t+1 ,a _t+1 ; θ ^- ), (s _t+1 ∈S, a _t+1 ∈A) represents a network, that is, the t+1th time period is based on the target network , that is, the parameter is θ ^- , and node i performs action a _t+1 in state s t ₊₁ ; s _t+1 is the state of node i in the t+1th time period; a _t+1 is the state of node i in Action performed in the t+1th time period; max _at+1 ∈AQ(s _t+1 ,a _t+1 ; θ ^- ) indicates that node i is in state s _t + based on the target network (parameter is θ ^- ) ₁ select action a _t+1 to maximize the corresponding Q value.

Further, the error calculation module calculates the current network Q (s _t , a _t ; θ) and the target value Error between:

Use gradient descent to update neural network parameters:

Among them, L(θ) is the TD error function of the model; Represents the expectation for the selected mini-batch empirical data; θ parameters of the current network updated in real time; α learning rate;/> is the corresponding gradient; Q(s _t , a _t ; θ) represents a network, that is, a network in which node i performs action a _t in state s _t under the network parameter θ in the t-th time period.

Further, the entire system time is divided into multiple consecutive superframe times. One superframe time is a time period. Each superframe includes a beacon frame, a control period and a data transmission period. The control period adopts a fixed control channel to transmit relevant control information and channel allocation decisions; the data transmission cycle uses K non-overlapping channels to support interference-free parallel data transmission; and during the control cycle, all nodes in the network switch to the control channel to listen Control information related to sending; in the data transmission period, the node with data to be sent switches to the channel where its parent node is located to transmit data based on the channel access mechanism.

Further, when performing action a, the node adopts a channel access mechanism based on RTS/DCTS, including:

If node d is located at the mth hop, its next hop is the m+1th hop node, that is, node d is the parent node of node i; if node e is located at the mth hop, its next hop is the m+1th hop node. is node j, that is, node e is the parent node of node j; the four nodes all work on the same channel, and the backoff values of node i and node j are 0;

When node i sends an RTS frame to node d, node d waits for a CIFS time and then returns a CTS frame;

After receiving the RTS frame of node i or the CTS frame of node d, the child node of node d will set the corresponding NAV based on the information in the Duration field;

When node e receives an RTS frame from node i, it waits for a SIFS and returns a CTS frame to notify its child nodes that its child nodes delay data transmission during the transmission of node i;

Among them, RTS refers to request to send; CTS refers to clear to send; CIFS is the inter-frame interval used for the destination node to return CTS; SIFS refers to the frame used to separate each frame belonging to a conversation, and CIFS is slightly larger than SIFS.

Furthermore, if node j is within the communication range of node i, and its parent node is not within the communication range of node i, then after node j receives the RTS frame, after waiting for a RIFS, node j sends the RTS frame to parent node e. .

The invention solves the problems of hidden terminals and exposed terminals in high-density multi-hop wireless networks, effectively avoids data conflicts and waste of channel resources, and improves overall network performance. In addition, based on the node's channel access performance and channel occupancy during the data transmission cycle, an asynchronous DRL model is proposed to dynamically optimize the node's channel allocation strategy for wireless multi-hop multi-channel networks. A new wireless mode based on mobile edge computing (MEC) is proposed to solve the computing and storage pressure of terminal nodes, and a distributed interaction (micro-learning) and centralized training (macro-learning) framework is designed to train asynchronous DRL models. Therefore, the asynchronous DRL model proposed by the present invention can be implemented even on a resource-limited terminal. In addition, the present invention considers the non-stationary problem in the multi-agent scenario (MAS), and only uses local local information of neighbors to avoid drastic dynamic changes in the network, while further accelerating network convergence.

Description of drawings

Figure 1 is an example diagram of hidden and exposed terminals in multiple channels provided in the prior art;

Figure 2 is a system architecture diagram of edge computing empowerment provided by an embodiment of the present invention;

Figure 3 is a superframe structure diagram used in the present invention;

Figure 4 is an asynchronous DRL model based on distributed decision-making architecture in the present invention;

Figure 5 is a centralized training process for asynchronous DRL models provided by an embodiment of the present invention.

Figure 6 is one of the working principle diagrams of RTS/DCTS provided by the embodiment of the present invention;

Figure 7 is the second working principle diagram of RTS/DCTS provided by the embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

The present invention proposes a distributed channel allocation method in a wireless multi-hop network, adopting a physical architecture that at least includes a physical device layer, a computing layer and a network service layer. The physical device layer is composed of n wireless nodes randomly deployed in the network. In a hopping wireless communication network, each node acts as an autonomous intelligent agent and interacts with the uncertain network environment through local decision-making modules; the convergence node in the computing layer is responsible for aggregating, analyzing and analyzing the data collected by other sites in the network. Processing, and the node has edge computing function, it can offload the computing tasks of the node, and train the asynchronous DRL model based on the empirical information collected distributedly by the node, model the multi-channel allocation problem as a POMDP problem, and use the trained asynchronous DRL The model performs channel allocation.

Example 1

This embodiment provides a system architecture diagram, as shown in Figure 2. The system architecture includes a physical device layer, a computing layer, and a network service layer. Among them, the physical device layer consists of n wireless nodes randomly deployed in the network to form a multi-hop wireless communication network. Each node acts as an autonomous intelligent agent and interacts with the uncertain network environment through the local decision-making module; calculation The aggregation node of the layer is responsible for aggregating, analyzing and processing data collected by other sites in the network, and the node has an edge computing function that can offload the computing tasks of the node, and can train an asynchronous DRL model based on the empirical information collected distributedly by the node.

During the data transmission process, this embodiment chooses to transmit data in a superframe structure. The superframe structure is shown in Figure 3. The system time is divided into multiple consecutive superframe times. Each superframe includes a beacon frame. control cycle and data transmission cycle. Among them, the control period uses a fixed control channel to transmit relevant control information and channel allocation decisions; the data transmission period uses K non-overlapping channels to support interference-free parallel data transmission. Therefore, during the control period, all nodes in the network must switch to the control channel to listen and send related control information (routing, time synchronization, channel switching, etc.); during the data transmission period, nodes with data to be sent switch to their parent Data transmission is performed based on the channel access mechanism on the channel where the node is located.

The asynchronous DRL model used in this embodiment is shown in Figure 4. DRL is used to solve the dynamic multi-channel allocation problem in multi-hop wireless networks. The embodiment of the present invention combines the DQN function approximation capability and the A3C asynchronous experience sampling architecture, and proposes an asynchronous DRL model, aiming to reasonably allocate channels to nodes to maximize the reliability of data transmission. Among them, the DRL model deployed on the edge server adopts the DQN architecture. DNN is introduced to extract features from the original data to approximate the behavior value function. At the same time, it is combined with the asynchronous training framework of A3C to solve the problem that DQN is not suitable for high-dimensional action spaces and MAS, breaking the It improves the correlation between experiences, significantly improves the convergence speed of the network, and solves the problem of being unable to implement the A3C algorithm on resource-limited wireless nodes.

This embodiment considers that the computing power, energy, and memory capacity of wireless nodes in certain scenarios are limited, resulting in computing bottlenecks and low performance, limiting support for advanced applications, and running computationally intensive tasks, that is, training DRL models. Therefore, embodiments of the present invention adopt a wireless network architecture based on edge computing empowerment to transfer the computing tasks of node training asynchronous DRL models to resource-rich edge nodes (aggregation nodes). As shown in Figure 2, the asynchronous DRL model deployed at the computing layer consists of the current network (main), the target network (target) and the experience pool (experience replay). Therefore, the convergence node empowered by edge computing completes the training and update tasks of the model.

When using the asynchronous DRL model for channel allocation, the present invention combines the function approximation capability of DQN and the asynchronous interaction architecture of A3C. In the asynchronous DRL model given in Figure 4, the distributed interaction module (micro-learning) allows terminal nodes to use local Observation information asynchronously selects channel resources. Furthermore, a centralized training module (macro learning) trains the asynchronous DRL model by adjusting operating parameters, thus guiding the system toward application-specific global optimization goals (e.g., maximizing the reliability of data transmission). Among them, each terminal node maintains a DRL prediction model to independently allocate channels. Specifically, the embodiment of the present invention models the multi-channel allocation problem as a POMDP problem. The POMDP problem consists of five tuples: M = <S, A, P, R, γ>, state s, action a, and state transition. Probability P, reward function R and discount factor γ. The agent observes the current network state s and executes action a in the control cycle of each time step t. Then it moves to the next state with the state transition probability and obtains reward R _t+1 from the environment.

State space, S={S ₁ , S ₂ ,..., S _2K+N }. Among them, K is the number of available channels, and N refers to the number of nodes. For a specific node i, in the tth period, its state vector,

in, Indicates the occupation of channel j by neighbor nodes of node i,/> It means that there is a neighbor channel of node i occupying channel j; otherwise, S _i,t,j =0. /> is the total number of neighbor nodes of node i.

Action space, A={a ₁ ,a ₂ ...,a _K }, a _k ∈A. Among them, it is used to indicate the channel number that node i wants to switch in the next data transmission cycle, a _k =ch _i,t,k , ch _i,t,k =k∈[1,K].

Reward function, R. When node i is in the t-th data period, the local observation state perform action When switching to channel ch _i,t,k , after the data transmission period ends, the environment will return an immediate reward value to the node, R=R(s,a). This value can be solved by the following function:

in, In the current data period, it means that there is no neighbor node of node i using channel ch _i,t,k ; otherwise,/> is the number of neighbor nodes of node i using channel ch _i,t,k =k. /> is the successful transmission probability of node transmitting data on ch _i,t,k .

The convergence node empowered by edge computing centrally trains the DRL model based on the experience information collected asynchronously distributed by each node in the network, and sends the updated network model parameters to the node. Each node can obtain the latest data from its parent node. network parameters.

The centralized training process of the DRL model is shown in Figure 5. There are two networks with the same structure but different parameters in the asynchronous DRL model. The current value of the Q estimate is predicted using the latest parameters; while the realistic prediction of Q Neural network target value parameter, which uses the old parameters from before. In this embodiment, the status of the node is used as the input of the neural network, and the different actions performed by each node are used as the category of the node. The probability of the node executing each action is predicted through the neural network, and this probability is used as the output of the neural network. , that is, the value of Q, for example, Q(s,a; θ) represents the probability of the node executing action a when the input node state s is θ when the parameters of the neural network are θ.

When training the model, some (mini-batch) experiences are randomly taken from the experience pool for training to break the correlation between experiences. In addition, since the experience information in the experience pool in the present invention is asynchronously sampled and provided by the agent, the correlation between experiences can be further broken and a richer experience can be provided.

It can be seen from Figure 5 that <s, a> information is used as the input of the current value network to obtain Q (s, a; θ), which is used to evaluate the current state behavior value function; s′∈S information is used for the target value network input to obtain the corresponding maxQ(s′,a′; θ ^- ); calculate include:

Therefore, based on value, using the DQN error function module, the error value can be further calculated:

The current network updates the parameters of the current value network based on the gradient of the error function:

Among them, s∈S,a∈A. After a certain number of iterations, the parameters of the current value network are copied to the target value network;

θ ^- ←θ

Repeat the above process until the network reaches a stable state.

Although the asynchronous DRL-based channel allocation model improves network performance by applying multiple parallel data transmissions, the problem of hidden terminals and exposed terminals on specific channels will be further exacerbated in high-density wireless multi-hop network scenarios. Figure 1 shows the hidden terminal and exposure problems in wireless multi-hop networks. When node D is transmitting data to node C, node B is located outside the communication range of node D. Therefore, node B mistakenly believes that the channel is in an idle state, so when node B sends data to nodes C and A at this time, a data conflict occurs at node C, resulting in unnecessary data retransmission, further exacerbating network congestion; in addition, When node B1 transmits data to node A1, because node B2 is in the communication range of node B1, and nodes B2 and A2 are not in the communication range of node A1 and B1 respectively, node B2 mistakenly thinks that the channel is in an idle state and delays data transmission. This will lead to unnecessary waste of channel resources. Therefore, embodiments of the present invention propose to solve the above problems of hidden terminals and exposed terminals in the wireless multi-hop network based on the RTS/DCTS mechanism. The RTS/DCTS mechanism is further described below through examples.

Figure 6 is a schematic diagram of solving the problem of hidden terminals in wireless multi-hop networks based on RTS/DCTS according to a preferred embodiment of the present invention. Among them, nodes i and j, nodes d and e are respectively located at m and m+1 hops (referring to different and adjacent hop numbers) and work on the same channel. Node d is the parent node of node i, and node e is the parent node of node j. Node e is also a neighbor node of node i. Assume that the backoff values of nodes i and j are both 0 at this time.

When node i sends an RTS frame to node d, node d waits for a CIFS time and then returns a CTS frame;

After receiving the RTS frame of node i or the CTS frame of node d, the child node of node d will set the corresponding NAV based on the information in the Duration field;

When node e receives an RTS frame from node i, it waits for a SIFS and returns a CTS frame to notify its child nodes that during the transmission of node i, its child nodes delay data transmission to avoid the hidden terminal problem.

In the channel access mechanism in the multi-hop environment, the hidden terminal problem is inevitable, so the successful transmission probability of node i on a specific channel k, The following formula can be used to calculate:

where τ is the transmission probability in the channel access slot. specifically, ( _ns is the total number of child nodes of the node's parent node). n _a represents the number of neighbor nodes of node i, and n _f represents the number of neighbor nodes of the parent node of node i (excluding the child nodes of the parent node).

Regarding the reserved terminal problem, please refer to FIG. 7 , which is a schematic diagram of an example of solving the problem of exposed terminals in a wireless multi-hop network based on RTS/DCTS according to a preferred embodiment of the present invention. Among them, nodes i and j, nodes d and e are respectively located at m and m+1 hops (referring to different and adjacent hop numbers) and work on the same channel. Node d is the parent node of node i, and node e is the parent node of node j. Node j is also a neighbor node of node i. Assume that the backoff values of nodes i and j are both 0 at this time.

When node i sends RTS to node d, node d waits for a CIFS time and then returns a CTS frame; because node j is within the communication range of node i. Therefore, node j will also receive an RTS frame, but since the destination node of the RTS frame is not the destination node of node j, node j will not set NAV based on the Duration field information of the RTS;

When node j receives the RTS frame, it waits for a RIFS to determine whether it has received the CTS frame; because its parent node e is not within the communication range of node i, node e will not return a CTS after SIFS; therefore, node j No CTS frame is received after RIFS; node j sends RTS frame to parent node e;

When nodes in the network perform the above process, they can effectively solve the data conflict and channel resource waste problems caused by hidden terminals and exposed terminals in the network; therefore, the successful transmission probability can be rewritten as:

Based on the RTS/DCTS mechanism, data conflicts between data links under adjacent parent nodes located on the same channel can be effectively avoided through SIFS and CTS; in addition, the channel access mechanism introduces RIFS inter-frame spacing It solves the problem of violent terminals in the network, thus improving the probability of successful transmission of nodes, that is, Therefore, the channel access mechanism can improve the successful transmission probability of nodes in the network;

In addition, it can be seen from the above formula that P _s is related to the parameter n _a and n _f are directly related, and the parameters n _s , n _a and n _f can be further optimized by optimizing the channel allocation strategy; therefore, the embodiment of the present invention takes the node’s successful transmission probability on the channel/> Used as part of the channel allocation model reward function to further optimize network performance.

The channel allocation and channel access mechanism proposed by the embodiment of the present invention first optimizes channel resources from different levels. Channel allocation optimizes channel resources from the frequency domain, and channel access optimizes channel resources from the time domain. In addition, a reasonable channel allocation mechanism will further alleviate the interference problem during the channel access process, and the node's channel access performance will further optimize its channel allocation strategy.

Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art will understand that various changes, modifications, and substitutions can be made to these embodiments without departing from the principles and spirit of the invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (4)

1. A distributed channel allocation method in a wireless multi-hop network is characterized in that a physical architecture at least comprising a physical device layer, a computing layer and a network service layer is adopted, the physical device layer comprises n wireless nodes which are randomly deployed in the network to form a multi-hop wireless communication network, the multi-channel allocation problem is modeled as a POMDP problem, an asynchronous DRL model is utilized to realize distributed channel allocation, each node is used as an autonomous Agent, interaction is carried out with an uncertain network environment through a local decision module, a gathering node of the computing layer is responsible for gathering, analyzing and processing data collected by other stations in the network, the node has an edge computing function, the computing task of the node can be unloaded, the asynchronous DRL model can be trained based on experience information collected by the node in a distributed mode, and the wireless node periodically updates parameters of the local decision module from the gathering node, and the method concretely comprises the following steps:

the POMDP problem consists of five tuples, m= < S, a, P, R, γ >, state S, action a, state transition probability P, reward function R, and discount factor γ;

the Agent observes the current network state s and executes the action a in the control period of each time step t; then transition to the next state with state transition probability, obtaining rewards R from the environment _t+1 ；

State spaceWhere K is the number of available channels and N is the number of nodes; for a specific node i, at the t-th period, its state vector, +.>

Wherein,representing the occupancy of channel j by the neighbor node of node i,/>Indicating that the neighbor channel with node i occupies channel j; on the contrary, S _i,t,j ＝0；/>Is the total number of neighbor nodes of node i;

motion space a= { a ₁ ,a ₂ ...,a _K }，a _k E A, wherein a is a channel number for indicating that node i is to switch in the next data transmission period _k ＝ch _i,t,k ,ch _i,t,k ＝k∈[1,K]；

Reward function R, when node i is in the t data period, locally observing stateExecuting an actionSwitching to channel ch _i,t,k At the end of the data transmission period, the environment returns to the node an immediate prize value, r=r (s, a), which can be solved by the following function:

wherein,in the current data period, the neighbor node without the node i uses the channel ch _i,t,k The method comprises the steps of carrying out a first treatment on the surface of the On the contrary, let(s)> Is to use channel ch _i,t,k Number of neighbor nodes of node i of=k; />Is node ch _i,t,k Successful transmission probability of data transmission is carried out on the data;

the aggregation node energized by edge calculation is used for intensively training a DRL model based on the experience information acquired by each node in the network in a distributed and asynchronous way, and sending updated network model parameters to the nodes, wherein each node can acquire the latest network parameters from a father node;

taking the states of the nodes as inputs of the neural network, and executing each node differentlyThe action is taken as the category of the node, the probability of each action executed by the node is predicted by the neural network, and the probability is taken as the output of the neural network, namely<s,a>Information is used as input of a current value network to acquire Q (s, a; theta) for evaluating a current state behavior value function; the S ' S information is used for the input of the target value network to obtain the corresponding maxQ (S ', a '; θ) ^- ) The method comprises the steps of carrying out a first treatment on the surface of the CalculatingComprising the following steps:

thus, based onThe value, adopting the DQN error function module, can further calculate the error value:

the current network updates parameters of the current value network based on the error function gradient:

wherein S epsilon S and a epsilon A; copying parameters of the current value network to the target value network every time a certain number of iterations are performed;

θ ^- ←θ

repeating the above process to make the network reach a stable state.

2. The method of claim 1, wherein the entire system time is divided into a plurality of consecutive super-frame times, one super-frame time being a time period, each super-frame including a beacon frame, a control period and a data transmission period, the control period employing a fixed control channel to transmit the associated control information and channel allocation decisions; k non-overlapping channels are adopted in the data transmission period to support interference-free parallel data transmission; and in the control period, all nodes in the network switch to the control channel to intercept and send the relevant control information; and switching the node with data to be sent to a channel where a parent node is located in the data transmission period to perform data transmission based on a channel access mechanism.

3. The method for distributed channel allocation in a wireless multi-hop network according to claim 1, wherein the node uses an RTS/DCTS-based channel access mechanism in performing act a, comprising:

if the node d is located in the m-th hop and the m+1st hop node of the next hop is the node i, namely the node d is a father node of the node i; if the node e is located in the m-th hop and the m+1st hop node of the next hop is the node j, namely the node e is the father node of the node j; the four nodes all work on the same channel, and the back-off value of the node i and the node j is 0;

when the node i sends an RTS frame to the node d, the node d waits for a CIFS time and returns a CTS frame;

after receiving the RTS frame of the node i or the CTS frame of the node d, the child node of the node d sets a corresponding NAV based on the information in the Duration field;

when node e receives the RTS frame from node i, waiting for a SIFS, returning a CTS frame to inform the child node thereof that the child node delays data transmission during the transmission of node i;

wherein, RTS refers to request sending; CTS refers to clear to send; CIFS is the interframe space for the destination node to return CTS; SIFS refers to a technique for separating frames belonging to a session, and CIFS is slightly larger than SIFS.

4. A method for distributing channels in a wireless multi-hop network according to claim 3, wherein if node j is located in the communication range of node i and its parent node is not located in the communication range of node i, when node j receives the RTS frame, after waiting for a RIFS, node j sends the RTS frame to parent node e.