KR102720564B1 - Resource management apparatus and method for slice - Google Patents

Abstract

A slice resource management device according to the present invention includes a learning unit which collects first information regarding a relationship between a slice customer and a slice tenant, collects second information regarding a relationship between a slice provider and the slice tenant, and learns to find an optimal slice resource allocation amount by referring to the first information and the second information, wherein the first information includes at least one of a characteristic of an application service provided by the slice tenant, a traffic status performed in the slice, and QoS of the slice, and the second information includes price information provided by the slice provider, and the learning unit proposes an optimal slice resource suitable for a current state according to a result of the learning.

Description

{Resource management apparatus and method for slice}

The present invention relates to a device and method for managing resources of a slice.

In a 5G network, multiple logically separated end-to-end networks can be created on a single physical network to support dedicated networks for tenants providing different services. The unit referring to the dedicated network or end-to-end network is called a slice.

It is important that each of the above slices is guaranteed virtualized resources and does not affect or are affected by the communication of other slices. The virtualized resources may include spectrum, transmission, storage, and computational power.

The above slide tenant rents slice resources from the slice provider and provides the rented slice resources to the slice customer. It is difficult to find out how much resources the slice tenant will rent from the slice provider in order to provide services to the slice customer. No prior art has been presented yet for this purpose.

However, as a related technology, a slice brokerage platform is proposed in 'O. U. Akguel, I Malanchini, and A Capone, "Dynamic Resource Trading in Sliced Mobile Networks," IEEE TNSM, January 2019.' The slice brokerage platform provides a method for dynamically allocating resources that can provide the best quality of service (QoS) to the slice tenant.

However, satisfying only the quality of service cannot satisfy the various needs of the above slide customers. For example, there are problems in that the revenue of the above slide tenant, the quality of service, and the dynamic and smooth resource allocation cannot be performed.

O. U. Akguel, I Malanchini, and A Capone, “Dynamic Resource Trading in Sliced Mobile Networks,” IEEE TNSM, January 2019.

The present invention is proposed against the background mentioned above, and proposes a resource management device and method for a slice that maximizes the benefits of a tenant while satisfying the QoS of user traffic in a network slicing environment with limited physical bandwidth resources.

The present invention proposes a slice resource management device and method capable of responding to the needs of various slice customers by adaptively changing the amount of bandwidth allocated to each slice according to the ratio characteristics of elastic traffic and inelastic traffic that are different for each application service.

A slice resource management device according to the present invention includes a learning unit which collects first information regarding a relationship between a slice customer and a slice tenant, collects second information regarding a relationship between a slice provider and the slice tenant, and learns to find an optimal slice resource allocation amount by referring to the first information and the second information, wherein the first information includes at least one of a characteristic of an application service provided by the slice tenant, a traffic status performed in the slice, and QoS of the slice, and the second information includes price information provided by the slice provider, and the learning unit proposes an optimal slice resource suitable for a current state according to a result of the learning.

A slice resource management method according to another aspect of the present invention includes: collecting information related to a slice; performing reinforcement learning using at least some of the collected information as state information to determine how much slice resources a slice tenant will purchase from a slice provider; and providing the determined slice resource purchase amount to a slice customer by the slice tenant from a slice provider.

According to the present invention, by learning to adjust bandwidth according to the traffic status of each slice using reinforcement learning, and dynamically adjusting the resources of the slice, particularly the bandwidth, accordingly, the resources of the slice can be managed more adaptively according to the current status.

According to the present invention, efficient bandwidth control is possible on an application service basis, thereby greatly increasing the profits of slice tenants.

According to the present invention, since complex calculations are not required for calculating the amount of slice resources, there is an advantage of less overhead in the system.

According to the present invention, since optimal actions are reported for various episodes, more accurate slice resources can be found at the current time without having to refer to information about past times.

Figure 1 is a block diagram of the entire system providing slices.
FIG. 2 is a diagram illustrating the operation of a slice resource management device, including a slice resource management device that determines how much of the slice's resources a slice tenant will be allocated from a slice provider.
Figure 3 is a drawing explaining a slice resource management method according to an embodiment.
Figure 4 is a graph comparing the correlation between slice resource demand and QoS for elastic and inelastic traffic.
FIGS. 5 to 10 are drawings explaining the effects of a slice resource management device and method according to an embodiment.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the embodiments presented below, and those skilled in the art who understand the spirit of the present invention will be able to easily propose other embodiments included within the scope of the same spirit by adding, changing, deleting, and adding components, but this will also be considered to be included within the scope of the spirit of the present invention.

Figure 1 is a configuration diagram of the entire system providing the above slice.

Referring to Fig. 1, the resources of the slice provider (3) can be leased to the slice tenant (1). The slice tenant (1) can provide a dedicated application service to the slice customer (2) by using the slice.

The physical resources possessed by the above slice provider (3) may include wireless network resources (21), wired network resources (22), and cloud resources (23).

If the above slice customer (2) is more or uses more traffic, the slice tenant (1) should be allocated more resources from the slice provider (3). Of course, the opposite case can work in the opposite way. On the other hand, if the slice tenant (1) is allocated resources from the slice provider (3), more costs are incurred. Therefore, it is an important task for the slice tenant (1) to be allocated appropriate slice resources.

FIG. 2 is a diagram illustrating the operation of a slice resource management device, including a slice resource management device that determines how much of the slice's resources the slice tenant will be allocated from the slice provider.

Referring to Fig. 2, the slice resource management device (30) is a system provided for slice resource management of the slice tenant (1). The slide resource management device (30) may be included in a system server on the slice tenant (1) side, but may also be provided to a slice provider (2) or other third parties.

The above slice resource management device (30) collects first information necessary for the relationship between the slice customer (2) and the slice tenant (1). The first information may include a traffic monitoring unit (11) that monitors traffic status within the slice, and a QoS calculation unit (13) that calculates QoS (Quality of Service) of the traffic. The traffic monitoring unit (11) may monitor real-time resource utilization. The traffic monitoring unit (11) may include a service feature unit (12) that may index and classify application service features of the slice into elastic/inelastic.

The above slice resource management device (30) collects second information necessary for the relationship between the slice supplier (3) and the slice tenant (1). The second information may include price information, which is information on the price paid by the slice supplier for the supply of the slice. The above slice resource management device (30) may include a price monitoring unit (14) that monitors the price information.

The above slice resource management device (30) may include a learning unit (15) that learns to find out the optimal slice resource allocation amount by referring to the first information and the second information. The learning unit (15) may perform learning to find out the optimal slice resource allocation amount by referring to the first information and the second information. Based on the result learned by the learning unit (15), the optimal slice resource allocation amount corresponding to the current state may be provided at the current point in time.

The above learning unit (15) can apply reinforcement learning. Various reinforcement learning methods such as Q-learning, policy gradient, actor-critic, and deterministic policy gradient can be applied to the above reinforcement learning. In the embodiment, Q-learning is applied.

In the above learning unit (15), learning can be performed to propose an optimal slice resource allocation amount using the application service characteristics, the traffic status, the QoS, and the price information.

According to the Q-table obtained from the above learning unit (15), the resource allocation unit (16) can allocate optimal slice resources in real time in the current state.

The above resource allocation unit (16) can allocate slice resources to a specific slice tenant through a connection with the slice provider (3), and the slice resources can include the bandwidth of the slice.

Figure 3 is a drawing explaining a slice resource management method according to an embodiment.

Referring to FIG. 3, first, the slice resource management device collects various information related to the slice (S1). The information collected here may include the application service characteristics, the traffic status, the QoS, and the pricing information.

The reinforcement learning can be performed using at least some of the collected information as state information, so that slice resource management can be performed. For example, the slice resource management can be determining how much slice resource to purchase (S2).

After the above-mentioned slice resource purchase amount is determined, the slice tenant can purchase slice resources from the slice provider according to the determined purchase amount (S3). Of course, the purchased slice resources can be provided to the slice customer.

Below, the above slice resource management method is described in more detail.

First, the Q-learning of the embodiment can be performed by continuously updating the Q value according to the rule of the following mathematical expression 1.

Here, s represents state information, a represents action information, α represents the learning rate, γ represents a discount factor, and v represents a reward.

The above reward (v) can be expressed by the following mathematical formula 2.

Here, p represents the price, R represents the slice resource, n represents the index of the slice, t represents time, sell represents a sale to a slice customer, buy represents a purchase from a slice provider, violate represents not satisfying the QoS, ω represents the QoS weight, and π represents the policy.

In the above mathematical expression 2, the QoS information can be referenced in calculating the penalty.

According to the above mathematical expression 2, the Q-learning of the above mathematical expression 1 is The Q value is updated to move toward the policy that prioritizes the best. In addition, the above mathematical expression 1 can be repeated to update the Q-table whenever a new event occurs.

In the update of the above mathematical expression 2, the state information has a great influence. The state information is explained in detail.

The above status information may include the application service characteristics, the traffic status, time information, and the price information.

In the above application service features, the application service is divided into elastic traffic and inelastic traffic. The elastic traffic and inelastic traffic are divided, and the ratio information of the two traffics can be used as status information of reinforcement learning.

The above elastic traffic and the above inelastic traffic are described.

First, the above elastic traffic refers to traffic that has a wide adjustment range for throughput and delay transmitted through the network. Therefore, application services corresponding to elastic traffic can still satisfy application services even if throughput and delay adjustments occur to some extent. P2P services can be considered as an example of elastic traffic.

On the other hand, the above inelastic traffic refers to traffic that is sensitive to requirements for throughput and delay and cannot satisfy application services when throughput and delay adjustments occur in the network. Audio streaming and VoIP services are examples of inelastic traffic.

Figure 4 is a graph comparing the correlation between slice resource demand and QoS for the elastic traffic and the inelastic traffic.

Referring to Figure 4, the elastic traffic is an average resource demand ( ) but to satisfy the above inelastic traffic, a certain minimum demand ( ) can be seen to be demanding.

As we have already seen, this may be due to the characteristics of the application service that each traffic needs to achieve satisfactory performance. For example, in the case of the above inelastic traffic, it is difficult to operate without a certain level of slice resources, so there is a minimum demand.

The ratio of the above elastic traffic and the above inelastic traffic is called the flow ratio (φ: flow ratio). can be defined as follows.

The above traffic status is the demand for the entire slice resource (D _n,t ). can be defined as, where I represents inelastic traffic, E represents elastic traffic, and D represents demand.

In the above elastic traffic and the above inelastic traffic, the average resource demand ( ), a certain minimum demand ( ) is requested as seen in the following.

The above price information is the current market price ratio (ρ). can be expressed as

The above t is time information, and updates can be performed with a constant time period (τ) in which an event occurs each time among the total time (T), as a time unit.

Learning is performed by implementing the above mathematical expressions 1 and 2, and depending on the result, more or less of the slice resource can be purchased in a predetermined unit (σ).

In the above slice resource management device and method, the result of Q-learning performed through the above process is stored, and when the current state is input, the optimal slice resource allocation amount can be notified.

The effects of the slice resource management device and method according to the above embodiment are described with reference to FIGS. 5 to 10.

Figure 5 shows different average rewards for an episode. In Figure 5, α is 0.1, γ is 0.9, ω is 1.6, the purchase price is $40, the sale price is $200, the total traffic is 200, and φ is randomly selected within the range greater than 0 and less than 1.

Referring to Figure 5, it can be confirmed that the Q-value converges well to a given value.

FIG. 6 is a graph comparing how well slice resources were managed according to time slots. FIG. 6(a) shows the result according to an embodiment, FIG. 6(b) shows the result of dynamically allocating resources in response to the demand of the previous time slot (previous method), FIG. 6(c) shows the result of allocating half of the total resource demand of the previous episode (half of demand), and FIG. 6(d) shows the result of fixed resources (fixed resources).

Referring to FIG. 6, it can be seen that in the case of the embodiment, the supply of slice resources changes most smoothly in response to the demand for slice resources.

Figure 7 is a graph comparing the profit (Figure 7(a)) and QoS satisfaction rate (Figure 7(b)) of the slice tenant while changing the trading interval in slice resource management.

Referring to Fig. 7(a), it can be seen that in the case of the embodiment, there is little effect even if the transaction interval is changed. In contrast, in the case of allocating half of the total resource demand of the previous episode or allocating fixed resources, there is no effect even if the transaction interval is changed, and in the case of dynamically allocating resources in response to the demand of the previous time slot, lower profits and QoS satisfaction rates were shown compared to the embodiment.

According to this, it can be seen that the example shows the best effect even if the transaction interval changes.

Figure 8 compares the QoS satisfaction rate by changing the flow ratio for the ratio of the elastic traffic and the inelastic traffic.

Referring to Fig. 8, it can be seen that the embodiment shows a slightly lower QoS satisfaction rate compared to when allocating fixed resources when the inelastic traffic is considerably large. However, considering that the revenue of the slice tenant is high, the embodiment may be superior.

In other cases, it can be seen that good QoS satisfaction is achieved at any flow rate.

Figure 9 is a diagram showing the effect according to the above QoS weight.

Referring to FIG. 9, it can be seen that as the QoS weight (ω) in the mathematical expression 2 increases, the QoS violation (penalty) decreases, but the revenue of the slice tenant decreases.

Figure 10 is a graph comparing the performance of purchasing slice resources differently depending on the supply price of slice resources by the slice supplier.

Referring to FIG. 10, in the case of the embodiment, it can be seen that when it exceeds $200, the purchase of the slice resource is lowered, and accordingly, it can be seen that it responds appropriately to the supply price of the slice supplier.

According to the present invention, the slice resource amount can be dynamically and smoothly purchased at each resource transaction interval by learning based on reinforcement learning according to the characteristics (elastic traffic or inelastic traffic) of the application service and the traffic status.

According to the present invention, there is an advantage in that complex calculations are not required to determine whether to increase or decrease the amount of slice resources even in a short resource transaction interval, and the best action in the current state can be simply selected.

According to the present invention, since the optimal action for multiple state sets is determined over multiple episodes, there is no need to refer to information about past times.

According to the present invention, tenants can smoothly manage slice resources to increase their profits, and when viewed from the network as a whole, there is an effect of not wasting network resources.

1: Slice Tenant
2: Slice Customer
3: Slice Provider
30: Slice Resource Management Device
11: Traffic Monitoring Department
12: Service Features
13: QoS calculation section
14: Price Monitoring Department
15: Learning Department
16: Resource Allocation Department

Claims (5)

Collect first-hand information about your relationship with Slice Customers and Slice Tenants;
Collect secondary information regarding the relationship between the Slice Provider and the Slice Tenant;
A learning unit is included that learns to find the optimal slice resource allocation amount by referring to the first information and the second information.
The first information includes the QoS of the slice, the characteristics of the application service provided by the slice tenant, and the traffic status performed in the slice.
The above second information includes price information provided by the slice supplier,
The above learning unit proposes the optimal slice resource suitable for the current state based on the learned results.
Slice resource management device. Collecting information related to the slice;
Reinforcement learning is performed using at least some of the collected information as state information to determine how much of the slice's resources the slice tenant will purchase from the slice provider; and
The above slice tenant includes supplying the determined slice resource purchase amount from the slice provider and providing it to the slice customer,
A slice resource management method, wherein the information collected from collecting the above information includes characteristics of application services provided by the slice tenant, traffic status performed in the slice, QoS of the slice, and price information which is compensation information for the provision of the slice. delete In the second paragraph,
The above application service features include:
A slice resource management method that includes ratio information between elastic and inelastic traffic. In the second paragraph,
The above reinforcement learning is Q-learning,
The above Q-learning is,
<Mathematical formula 2>

Here, p represents the price, R represents the slice resource, n represents the index of the slice, t represents the time, sell represents a sale to the slice customer, buy represents a purchase from the slice provider, viloate represents not satisfying the QoS, ω represents the QoS weight, and π represents the policy.
In the above mathematical expression 2, To make the best of it,
<Mathematical formula 1>

Here, s represents state information, a represents action information, α represents the learning rate, γ represents a discount factor, and v represents a reward.
A slice resource management method that updates the Q value of mathematical expression 1.

Images