CN111970719B - A method and device for quality assessment of anchor stations under 5G NSA network - Google Patents

Abstract

The invention relates to the field of communication, in particular to a method and a device for evaluating the quality of an anchor point station in a 5G NSA network. Mainly comprises the following steps: acquiring anchor point station quality index data; according to the evaluation dimension, selecting an evaluation factor to be evaluated from the quality index data; different weights are configured for each evaluation factor according to the evaluation service requirement; and calculating the weighted total score of each evaluation factor of the preset evaluation area according to the value and the weight of the evaluation factor, and taking the weighted total score as the quality score of the preset evaluation area. The invention can conveniently and accurately acquire the quality condition of each anchor point station, provides support for 5G NSA network optimization, and improves the user perception of 5G service users.

Description

A method and device for evaluating anchor station quality in 5G NSA network

[Technical field]

The present invention relates to the field of communications, and in particular to a method and device for evaluating the quality of anchor stations in a 5G NSA network.

[Background technology]

The 5G standard is divided into two networking modes: non-stand alone (NSA) and standalone (SA). In order to realize the idea of rapid network construction, telecom operators choose NSA architecture in the early stage of 5G network construction and evolve to SA architecture in the later stage. In the NSA architecture, the anchor station is the channel for 5G base stations to access the network. The continuous coverage performance of the anchor network affects the user's mobility experience of the 5G network. Especially for 5G voice, it is implemented through VoLTE and carried by anchor points. If the continuous coverage performance of the anchor point is poor, the voice experience of 5G users may be worse than that of the existing VoLTE, affecting the user experience.

At present, the commonly used anchor station quality assessment methods include: road test, single station verification, network management monitoring, perception quality assessment system (Deep Packet Inspection, abbreviated as: DPI), etc. Road test and fixed-point test can only cover some trunk roads. It is impossible to implement road test in special scenes or areas such as residential areas, parks, and buildings where test vehicles are difficult to enter. In addition, the manpower and vehicle costs required for road test are high, and it is generally only used in the early stage of network construction. Network management monitoring focuses more on the operation quality of network equipment, and provides network service quality from equipment parameter configuration and equipment performance evaluation. The evaluation focus is KPI indicators. The lack of KQI indicator evaluation cannot reflect the user's service perception. Moreover, network management is mainly oriented to the equipment side and cannot fully monitor the user level; on the other hand, the current network management monitoring is mainly used for operation and maintenance indicators. However, as the operator's business gradually shifts from operation and maintenance to operation, operators begin to pay more attention to service perception, and network management monitoring cannot obtain the corresponding indicators of service perception. DPI only covers core network data, and service perception evaluation is calculated through core network data. The results cannot fully reflect or represent the quality of wireless networks and transmission networks.

In view of this, how to overcome the defects of the existing technology and solve the problem that the existing anchor station quality assessment cannot easily obtain comprehensive and accurate anchor station quality indicators is a problem to be solved in this technical field.

[Summary of the invention]

In view of the above defects or improvement needs of the prior art, the present invention solves the problems that the road test method in the current anchor station quality assessment method has incomplete coverage and high cost, and the network management monitoring method cannot obtain service perception indicators.

The embodiment of the present invention adopts the following technical solution:

In a first aspect, the present invention provides a method for anchor station quality assessment in a 5G NSA network, specifically: obtaining anchor station quality index data; selecting evaluation factors to be evaluated from the quality index data according to evaluation dimensions; configuring different weights for each evaluation factor according to evaluation business requirements; and calculating the weighted total score of each evaluation factor in a preset evaluation area according to the value and weight of the evaluation factor, as the quality score of the preset evaluation area.

Preferably, obtaining the anchor station quality index data includes: obtaining the anchor station quality index data through one or more methods of road testing, fixed-point testing, integrated network management system, MR and/or perception index evaluation system.

Preferably, the evaluation dimensions include a network service quality dimension and/or a network service perception dimension.

Preferably, different weights are configured for each evaluation factor, including: selecting evaluation factors to be evaluated according to business needs, and setting weights for each evaluation factor according to business characteristics; selecting evaluation indicators to be evaluated for each evaluation factor, and setting weights for each evaluation indicator according to business characteristics.

Preferably, the weighted total score of each evaluation factor in the preset evaluation area is calculated, including: according to the value and weight of each evaluation indicator, the weighted total score of all evaluation indicators in each evaluation factor is calculated, and the weighted total score of all evaluation indicators is used as the value of the evaluation factor; according to the value and weight of each evaluation factor, the weighted total score of all evaluation factors is calculated.

Preferably, the weighted total score of all evaluation factors in a preset evaluation area is calculated, including: presetting a baseline value and a challenge value for each evaluation indicator in each evaluation factor; when the value of a first category of evaluation indicators is less than the baseline value, calculating the evaluation indicator according to the preset lower limit value of the evaluation indicator; when the value of the first category of evaluation indicators is greater than or equal to the challenge value, calculating the evaluation indicator according to the preset upper limit value of the evaluation indicator; when the value of a second category of evaluation indicators is greater than the baseline value, calculating the evaluation indicator according to the preset lower limit value of the evaluation indicator; when the value of the second category of evaluation indicators is less than or equal to the challenge value, calculating the evaluation indicator according to the preset upper limit value of the evaluation indicator.

Preferably, when the evaluation dimension is the network service quality dimension, one or more of terminal capability, coverage level, anchor point capacity, interference avoidance and anchor point performance are used as evaluation factors.

Preferably, when the evaluation dimension is the network service perception dimension, one or more of the terminal, network, pipeline and service are used as evaluation factors.

Preferably, when the terminal is used as an evaluation factor, the terminal is a 5G NSA terminal that supports dual connection capabilities and has the same frequency as all anchor stations and 5G base stations in the area where the anchor priority scheme has been implemented.

On the other hand, the present invention provides a device for evaluating the quality of anchor stations in a 5G NSA network, specifically: comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for evaluating the quality of anchor stations in a 5G NSA network in the first aspect.

Compared with the prior art, the beneficial effect of the embodiment of the present invention is that: data of multiple dimensions are obtained as evaluation factors, and corresponding weights are set for the evaluation factors according to different services, and the weighted total score of the evaluation factors of multiple dimensions is used as the standard for anchor station quality evaluation. Through this quality evaluation method, the quality status of each anchor station can be obtained conveniently and accurately, providing support for 5G NSA network optimization and improving the user perception of 5G service users.

【Brief Description of the Drawings】

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a method for evaluating anchor station quality in a 5G NSA network provided by an embodiment of the present invention;

FIG2 is a flow chart of another method for evaluating anchor station quality in a 5G NSA network provided by an embodiment of the present invention;

FIG3 is a flow chart of another method for evaluating anchor station quality in a 5G NSA network provided by an embodiment of the present invention;

FIG4 is an evaluation factor model used in a method for evaluating anchor station quality in a 5G NSA network provided by an embodiment of the present invention;

FIG5 is another evaluation factor model used in a method for evaluating anchor station quality in a 5G NSA network provided by an embodiment of the present invention;

Figure 6 is a schematic diagram of the structure of a device for evaluating the quality of anchor stations in a 5G NSA network provided by an embodiment of the present invention.

[Specific implementation method]

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

The Option3 family has great advantages in rapid 5G deployment. At the same time, the commercial driving force for upgrading LTE to eLTE to connect NGC needs to be further clarified. In the Option3 family, Option3x introduces NR with the least impact on the existing LTE network and the best performance. 5GNR is a global 5G standard based on a new air interface design of OFDM. It is also a very important foundation for the next generation of cellular mobile technology. 5G technology will achieve ultra-low latency and high reliability. Option3x is the common choice for NSA, and Option2 is the common choice for SA, and has become the first choice for most operators.

On the one hand, 5G NSA meets the construction requirements of the first batch of 5G networks, and on the other hand, it saves time for deploying NGC core networks, virtualization, and cloud transformation. Option3x LTE eNB and NR gNB adopt dual connections to significantly improve the user's downlink peak rate. NSA Option3x can formulate flexible diversion strategies based on the 4G network load, while fully utilizing network resources and avoiding congestion of the existing 4G network. In the NSA network, users are carried on the LTE network, and mobility management reuses the existing mobility management of LTE. When the UE moves out of the NR coverage, the data plane is not interrupted and the LTE connection is maintained. Option3x needs to rely on the X2/Xn interface to interconnect with LTE. Network upgrades and transformations require corresponding LTE capacity matching and version matching, including product implementation that complies with the latest standard technical specifications.

The main difference between NSA and SA networking is that NSA anchors 5G control signaling on 4G base stations, and the anchored base stations are called anchor stations; the SA solution is that 5G base stations directly access the 5G core network, and the control signaling is completely independent of the 4G network. Under the NSA standard, there is no need to build a new core network. New 5G base stations access the 4G core network or the 5G core network through 4G base stations. 5G and 4G do not need to interoperate, and services can be initiated on 4G and 5G networks at the same time. 5G terminals need to access the 4G network at the same time and support 4G/5G dual links.

In the NSA solution, users access 5G and 4G base stations simultaneously through dual connections. NAS signaling is transmitted by anchor stations, and user data flows are split between 5G base stations and 4G base stations. Users enjoy 5G services through collaboration between 5G and 4G base stations. The terms used are explained as follows:

(1) Dual connection: The mobile terminal can communicate with both 4G and 5G at the same time and download data at the same time. Generally, there will be one primary connection and at least one secondary connection.

(2) Control plane anchor point: The point responsible for the control plane in the dual connection is called the control plane anchor point.

(3) Diversion control point: User data needs to be divided into two paths of dual connection for independent transmission. The location where the diversion is performed is called the diversion control point.

Embodiment 1:

In the NSA networking scenario, due to the frequency range supported by the NSA terminal or other special factors, it is difficult for all existing LTE base stations in the existing network to be used as NSA anchor stations. Once an NSA user accesses a non-anchor LTE base station, it will be impossible to add an NR secondary carrier, and thus cannot enjoy 5G services. The core network to which the Option 3X base station is connected is the 4G core network, and the control plane anchor point is at the 4G base station, which places the data diversion control point on the 5G base station. Since the anchor station is used as the signaling channel for the NSA terminal to access the network, a 4G frequency with good network continuous coverage performance should be selected as the anchor point. In order to select a suitable NSA anchor point, it is necessary to combine the end-to-end support of the anchor industry chain and the estimated 4G network coverage, and select a 4G frequency with good continuous coverage performance and high edge rate as the anchor point. The method for evaluating the quality of anchor stations under the 5G NSA network provided in the embodiment of the present invention evaluates the quality of anchor stations in different dimensions, obtains the quality status of anchor stations, and provides data support for the optimization of anchor stations.

As shown in FIG1 , the method for evaluating anchor station quality in a 5G NSA network provided by an embodiment of the present invention includes the following specific steps:

Step 101: Obtain anchor station quality indicator data.

Traditional quality assessment methods generally use only a single quality assessment method, and the main quality assessment methods used include road testing, single-station verification, network management monitoring, and DPI. Road testing can directly reflect the communication quality of different areas, but it can only cover some trunk roads. It cannot be implemented in ordinary streets or roads that are difficult for test vehicles to enter, such as residential areas, parks, buildings, special scenes, etc. Road testing is generally only used to optimize the location of base stations in the early stage of network construction, and the manpower and vehicle costs required for road testing are relatively high. Network management focuses more on the operating quality of network equipment, and evaluates the service quality provided by the network from the perspective of equipment parameter configuration and equipment performance. The evaluation focuses on KPI indicators and is mainly used in equipment operation and maintenance. However, as operators gradually shift from operation and maintenance to operation, operators begin to pay more attention to service perception. The network management evaluation method lacks KQI indicator evaluation and cannot reflect user service perception. In addition, network management is mainly oriented to the equipment side and cannot fully monitor user perception. Measurement Report (abbreviated as: MR) refers to information sent every 480ms on the service channel (470ms on the signaling channel). This data can be used for network evaluation and optimization. The DPI evaluation method uses DPI to collect data for service perception evaluation, mainly for key indicators in services such as users' Internet access to browse web pages, watch videos, send/receive instant messages, and play games. Different weights and indicator weights are set according to the service type to comprehensively evaluate service perception. DPI has advantages in signaling and service surface data. The data contains user information, terminal type, area, network element, base station, access service type, etc. Therefore, the service perception evaluation can be performed according to different dimensions, but DPI only covers core network data. The service perception evaluation is calculated through core network data. The results cannot fully reflect or represent the quality of wireless networks and transmission networks, and an end-to-end service perception evaluation system cannot be realized.

In the anchor station quality assessment method provided in this embodiment, multiple anchor station quality indicator data are obtained for quality assessment according to the assessment dimensions and business requirements. In a specific embodiment of the present invention, according to different quality indicator data to be used, the method for obtaining anchor station quality indicator data includes: one or more of drive testing, single station verification, integrated network management system, MR and DPI.

The quality indicator data obtained by the drive test and single-station verification methods include: (1) Basic coverage indicators: SS-RSRP, SINR, CSI-RSRP, SINR, CQI, coverage, CQI distribution statistics, beam-level statistical measurements, etc. (2) Throughput indicators: uplink and downlink average rate, uplink and downlink peak rate, PHY/MAC/RLC/PDCP layer rate distribution, etc. (3) Scheduling capability indicators: MCS statistics, RANK statistics, Blert statistics, MIMO statistics, PDCCH statistics, etc.

The integrated network management system includes wireless network management, transmission network, core network management, etc. The quality indicator data obtained include: (1) Basic network indicators: uplink interference (average uplink interference level of cell RB, maximum uplink interference level of cell RB), transmission quality (transmission delay, transmission packet loss rate), channel quality (number of RBs used for uplink MCS level 0-28, number of RBs used for downlink MCS level 0-28), CQI, etc. (2) Network performance indicators: access category (SN addition success rate, average/maximum SN addition duration), retention category (SN abnormal release rate), mobility (SN leg handover success rate), integrity category (RLC layer packet loss rate, MAC layer block error rate), etc. (3) Network traffic indicators: resource category (number of PRBs occupied by uplink traffic channels, number of PRBs occupied by downlink traffic channels), traffic category (number of uplink PDCP PDU bytes of cell user plane, number of downlink PDCP PDU bytes of cell user plane), number of users (average number of terminal RRC connections, maximum number of terminal RRC connections), etc.

The quality indicator data obtained by the MR method includes the terminal and the eNodeB. The terminal performs and reports the downlink level strength, quality and TA of the eNodeB cell, and the eNodeB performs and reports the measurement of the uplink terminal reception level strength and quality.

The quality indicator data obtained by DPI includes: (1) Terminal: The core network obtains the terminal access location, terminal capability, number of terminal users, terminal traffic, terminal signaling indicators, terminal service indicators, etc. (2) Access: NSA terminal access anchor station attachment success rate, attachment delay, etc. (3) Residency: 5G service usage time residence ratio, 5G service traffic residence ratio, etc. (4) Switching: Directional switching, anchor station switching, etc. (5) Load: Anchor station load, non-anchor station load, etc. (5) Auxiliary station: NR site dual connection establishment success rate, etc. (6) Traffic: Anchor station traffic, NR station traffic, etc. (7) Perception: Service quality rate, etc.

Step 102: According to the evaluation dimension, the evaluation factor data to be evaluated is selected from the quality indicator data.

Since the existing quality assessment methods have their own advantages and disadvantages, in order to better and more comprehensively assess the quality of anchor stations, it is necessary to use quality indicator data obtained in a variety of ways to jointly establish an end-to-end perception assessment system that includes wireless networks, transmission networks, and core networks. In the specific implementation process, multiple groups of evaluation indicators can be selected to be combined into multiple evaluation factors to evaluate different aspects of service perception, and each evaluation factor contains one or more evaluation indicators.

In the specific implementation scenario of this embodiment, the evaluation dimensions generally include the network service quality dimension and the network service perception dimension. In actual use, one of the dimensions can be selected for evaluation based on factors such as service characteristics and available data, or both evaluation dimensions can be used for comprehensive perception evaluation to obtain more comprehensive perception data.

When network service quality is used as the evaluation dimension, one or more of the coverage level, anchor point capacity, interference avoidance and anchor point performance are used as evaluation factors. Since network service quality is generally positively correlated with the anchor point service quality perceived by the end user, the quality perceived by the end user can be indirectly evaluated by the quality of network service. In actual use, the evaluation index data may specifically include: coverage rate, uplink PRB utilization rate, downlink PRB utilization rate, RRC reconstruction success rate, call drop rate, etc. obtained by the integrated network management system; same frequency, anchor point attachment success rate, switching success rate, etc. obtained by DPI.

When network service perception is used as the evaluation dimension, one or more of the terminals, networks, pipelines and services are used as evaluation factors. The network service perception dimension can directly obtain the user's perception of the anchor station service quality from the three aspects of occupancy, residence and experience. In actual use, the evaluation indicator data can specifically include: Terminal: the number of 5G NSA terminals under the anchor station, the terminal function switch status and other conditions of attachment to the anchor station; Network: Analysis of network quality based on attachment, dual connection, number of cell users, performance, etc.; Pipeline: TCP quality analysis of service pipelines, etc.; Service: Analysis of high-bandwidth video service quality, etc.

In the specific implementation scenario of this embodiment, only 5G NSA terminals with the same frequency as the 5G NSA base station and supporting dual connectivity can access the 5G NSA network, and only the data generated by this type of terminal is valuable for evaluation. Therefore, when the terminal is used as an evaluation factor for evaluation, only the data of this type of terminal is used for evaluation.

Furthermore, when using the terminal as an evaluation factor, the terminal is a 5G NSA terminal with the same frequency as the anchor station and supporting dual connectivity. NSA terminals must first occupy the anchor cell before using 5G services to improve user perception. Therefore, it is necessary to migrate NSA terminals to the anchor cell and ensure stable occupancy, which is an important issue that needs to be solved at present.

Four problems faced by anchor point priority occupation;

1. If the terminal occupies a non-anchor point, how to migrate from the non-anchor point to the anchor point in the connected state;

2. If the terminal occupies a non-anchor point, how to migrate from the non-anchor point to the anchor point in the idle state;

3. If the terminal has occupied an anchor point, how can the connection state occupy the anchor point stably to prevent premature switching to a non-anchor point?

4. If the terminal has occupied an anchor point, how to stably occupy the anchor point in the idle state to prevent premature reselection to a non-anchor point. In the implementation scenario of this embodiment, the following anchor point priority schemes are generally used, mainly including: directional switching + directional reselection + NSA independent mobility strategy + prohibition of NSA load balancing.

(1) Directional switching:

1. When an NSA terminal accesses a non-anchor cell, a directional handover to the anchor cell is initiated.

2. When the NSA terminal switches in a non-anchor cell, a directional handover to the anchor cell is initiated.

3. When an NSA terminal is performing VoLTE services at a non-anchor point, it is prohibited to initiate directional switching.

4. When the NSA terminal terminates the VoLTE service at the non-anchor point, it initiates a directional handover to the anchor point cell.

(2) Directed reselection:

When an NSA terminal is released in a non-anchor cell, the rrcConnectionRelease message carries the anchor frequency and is set to the highest priority, which can guide the terminal to the anchor frequency.

When an NSA terminal is released in an anchor cell, the rrcConnectionRelease message carries the anchor frequency and is set to the highest priority, so that the terminal can be guided to the anchor frequency.

(3) Mobility strategy:

The NSA terminal connection strategy is that it is easier to switch from a non-anchor cell to an anchor cell, and it is more difficult to switch from an anchor cell to a non-anchor cell. The anchor cell ensures smooth switching.

(4) Prohibit NSA terminal load balancing configuration:

When an NSA terminal occupies an anchor cell, it is prohibited to load balance the NSA terminal to other frequencies.

Step 103: According to the evaluation business requirements, different weights are configured for each evaluation factor data.

In the anchor station quality assessment method provided in this embodiment, multiple quality indicator data are used to assess the quality. Depending on the device scenario, business scenario and assessment dimension, different quality indicator data have different impacts on the service quality perceived by the terminal. Therefore, when performing quality assessment, different weights need to be configured for the assessment factors in the quality indicator data.

Furthermore, since each evaluation factor may include more than one evaluation indicator, and each evaluation indicator has a different degree of influence on the evaluation factor. Therefore, in a specific usage scenario, not only can weights be set for different evaluation factors to improve the accuracy of quality assessment, but different weights can also be set for the evaluation indicators in each evaluation factor to further distinguish the degree of influence of different evaluation indicators on quality perception. Specifically, as shown in Figure 2, in actual use, different weights can be configured for each evaluation factor through the following steps:

Step 201: Select the evaluation factors to be evaluated according to the evaluation dimensions, and set weights for each evaluation factor according to business characteristics.

Step 202: Select an evaluation indicator to be evaluated for each evaluation factor, and set a weight for each evaluation indicator according to business characteristics.

Through the bipolar weight allocation of step 201 and step 202, the influence of different evaluation indicators and evaluation factors on business perception can be more accurately distinguished, and the accuracy of the evaluation can be improved. In the specific implementation scenario of this embodiment, the number of levels of weight allocation is not limited to two levels, and can be adjusted according to actual needs. When there are fewer evaluation indicators, only one level can be used. When there are more evaluation indicators and evaluation factors, the evaluation indicators can be further grouped into more levels such as evaluation sub-factors for weight setting, so as to further improve the accuracy of the evaluation.

Step 104: Calculate the weighted total score of each evaluation factor data of the preset evaluation area according to the value and weight of the evaluation factor data as the quality score of the preset evaluation area.

After setting weights for each evaluation factor, the weighted sum of the acquired values of each evaluation factor can be calculated as the quality score of the preset evaluation area, and the quality of the anchor station can be evaluated according to the quality score.

Furthermore, in certain specific usage scenarios of this embodiment, weights are also set for the evaluation indicators in each evaluation factor. When calculating the value of the evaluation factor, it is necessary to calculate the weighted total score of all evaluation indicators in each evaluation factor as the value of the evaluation factor.

In some specific implementation scenarios of this embodiment, the values of some evaluation factors or evaluation indicators may be far lower than normal values or far higher than normal values due to factors such as measurement limits or errors. In order to avoid the numerical deviation of abnormal values from normal values being too large and affecting the calculation results of the quality score, the lower limit of the normal value can be set as the reference value, and the upper limit of the normal value can be set as the challenge value. According to the different properties of evaluation indicators, the evaluation indicators are divided into two categories: (1) The first category of evaluation indicators: such as success rate, download rate indicators, etc., all values below the reference value are calculated according to the preset lower limit value, and all values above the challenge value are calculated according to the preset upper limit value; (2) The second category of evaluation indicators: such as delay, retransmission rate, fragmentation rate indicators, etc., all values above the reference value are calculated according to the preset lower limit value, and all values below the challenge value are calculated according to the preset upper limit value. Through the above processing, the influence of the specific numerical value of the abnormal value on the quality score is reduced. As shown in Figure 3, the calculation can be performed through the following steps.

Step 301: Preset a baseline value and a challenge value for each evaluation indicator data in each evaluation factor.

Step 302: Determine whether the type of the evaluation indicator is a first-type evaluation indicator or a second-type evaluation indicator.

Step 303: When the value of the first type of evaluation indicator is less than the reference value, the evaluation indicator is calculated according to the preset lower limit value of the evaluation indicator.

Step 304: When the value of the first type of evaluation indicator is greater than or equal to the challenge value, the evaluation indicator is calculated according to a preset upper limit value of the evaluation indicator.

Step 305: When the value of the second type of evaluation indicator is greater than the reference value, the evaluation indicator is calculated according to the preset lower limit value of the evaluation indicator.

Step 306: When the value of the second type of evaluation indicator is less than or equal to the challenge value, the evaluation indicator is calculated according to a preset upper limit value of the evaluation indicator.

Through steps 301 to 306, according to the characteristics of different evaluation indicators, the influence of abnormal values on the perception evaluation results is eliminated through the baseline value and the challenge value, thereby avoiding the deviation of the evaluation results caused by the absolute value of the evaluation indicator being too large or too small.

After steps 101 to 104 provided in this embodiment, the quality indicators of the anchor station can be evaluated through the quality indicator data of multiple evaluation dimensions. This avoids the problems of the difficulty in implementing various quality evaluation methods, the difficulty in obtaining data, and the one-sided evaluation results. It can comprehensively and accurately evaluate the perceived quality of the anchor station, which can be used as the basis for selecting 5G NSA anchor stations or the basis for evaluating other service quality.

Embodiment 2:

In some specific implementations, the method for evaluating the quality of anchor stations in a 5G NSA network provided in Example 1 can be implemented using the evaluation system model provided in this embodiment.

The evaluation system model used in this embodiment mainly uses evaluation factors such as terminal, coverage, capacity, interference, and performance to perform a perceptual evaluation of the anchor station quality in the network service quality dimension.

According to step 102, an evaluation factor to be evaluated is selected, and the evaluation principle of each evaluation factor is as follows:

Terminal capability: obtained through DPI. NSA terminals may not support all LTE frequencies in the existing network. Therefore, it is necessary to select mainstream NSA terminal capability information when planning NSA anchor points.

Coverage level: obtained through wireless network management data and network management system. 1) NSA anchor points must have continuous coverage, otherwise NR cannot be added in areas without anchor point coverage. 2) The NSA anchor points supported by commercial terminals are limited. For anchor points supported by mainstream terminals, even if the current coverage is poor, they must be used as high-priority anchor points, and capacity expansion is required to fill in blind spots when necessary.

Anchor point capacity: obtained through DPI. Including uplink capacity and downlink capacity. 1) Uplink capacity: It is recommended to give priority to the uplink capacity of the anchor point, because 5G uplink is easily limited and the demand for uplink diversion is greater. 2) If downlink diversion needs to be enabled, it is necessary to give priority to the frequency with large downlink capacity as the high-priority anchor point. If DL CA is enabled in the existing LTE network, the terminal's NSA DC and LTE CA combined capabilities need to be considered. However, there is no specific terminal capability information at this stage, so this factor is not considered for the time being. If DL CA is not enabled in the existing LTE network, it is necessary to give priority to LTE frequencies with higher single-frequency downlink capacity as NSA high-priority anchor points.

Interference avoidance: obtained through wireless network management data. Harmonic/intermodulation interference needs to be avoided. Certain LTE+NR frequency band combinations have harmonic/intermodulation interference risks, such as (LTE1.8G+NR3.5G) (LTE2.6G+NR4.9G). When planning NSA anchor points, it is recommended to lower the priority of LTE frequencies with interference risks. Whether there is interference risk is subject to the actual formula calculation results.

Coupling of mobility policies: obtained through network management data. 1) The non-essential mobility policy of SRAN 15.0 version LTE is not decoupled from NSA. Common mobility policies include CA PCC anchor policy/MLB. If the preferred frequency points of the non-essential mobility policy are not configured as NSA anchor points, there is a risk that NR cannot be added. Therefore, it is recommended to use the preferred frequency points of the non-essential mobility policy of the existing LTE network as NSA anchor points, such as the target frequency points of MLB. 2) For LTE's necessary switching policy, NSA users are required to comply, otherwise there is a risk of dropped calls.

Anchor point basic performance: obtained through the network management system. If the basic performance of each frequency point in the existing LTE network (including access, handover success rate, call drop rate, RRC re-establishment ratio, and ping-pong handover times) varies greatly, and the frequency points with poor basic performance are difficult to optimize and improve, it is necessary to give priority to selecting the frequency points with good basic performance as the NSA high-priority anchor points.

The evaluation indicators used in each evaluation factor include:

1. Same frequency: The proportion of terminals of all terminal types in a cell that support the same frequency as the base station cell frequency within a certain period of time.

2. Coverage: Coverage is defined as the percentage of test points with F value 1 in all test points in the test area. Among them: RSRP represents the received power of the downlink pilot signal; SINR represents the signal quality of the received pilot signal; RSRP ≥ R and RSRQ ≥ S represent whether the conditions are met, and R and S are the thresholds of RSRP and RSRQ in the calculation. If RSRP ≥ R and RSRQ ≥ S are both met, F takes the value of 1. If one of them is not met or neither is met, F takes the value of 0.

3. Uplink PRB utilization, downlink PRB utilization:

TTI PUSCH PRB utilization rate = number of PUSCH PRBs used per TTI/total number of PUSCH PRBs per TTI.

TTI PDSCH PRB utilization rate = number of PDSCH PRBs used per TTI/total number of PDSCH PRBs per TTI.

4. Interference rate: The sampling ratio of MR RIP>=-105dBm.

5. Anchor station attachment success rate: number of anchor station attachment successes/number of anchor station attachment requests × 100%.

6. Switching success rate (X2 cut-in, X2 cut-out, S1 cut-in, S1 cut-out):

X2 cut-in success rate = X2 cut-in success times/X2 cut-in attempt times × 100%.

X2 cut-out success rate = X2 cut-out success times/X2 cut-out attempt times×100%.

S1 cut-in success rate = S1 cut-in success times/S1 cut-in attempt times × 100%.

S1 cut-out success rate = S1 cut-out success times/S1 cut-out attempt times×100%.

7. Drop rate: E-RAB drop rate = (number of E-RABs released by the eNodeB due to abnormal reasons + number of E-RABs included in the UE context released by the eNodeB due to abnormal reasons) / number of successful E-RAB establishments × 100%.

8. RRC re-establishment success rate: RRC re-establishment success rate = number of RRC re-establishment successes/number of RRC re-establishment attempts (service related) × 100%.

According to step 103, as shown in FIG4, weights are set for the evaluation factors and evaluation indicators. The weights of the evaluation factors and the evaluation indicators are shown in Table 1, where the weights, benchmark values and challenge values can be adjusted according to actual experience.

Table 1

According to step 104, the weighted total score of the evaluation factors is calculated as the quality score of the evaluation area.

In order to obtain the quality score more accurately, instead of using the value of the evaluation indicator directly, the score of each evaluation indicator can be calculated using the following formula.

Use formula 1 to calculate the scores of the same frequency, coverage, and success rate indicators:

Use formula 2 to calculate the utilization rate, interference rate, and drop rate indicator scores:

The above formula is used to calculate the value of the evaluation index, and the weighting of the evaluation index and the evaluation factor is calculated to obtain the quality score of the anchor station in the evaluation area.

Embodiment 3:

In some specific implementations, the method for evaluating the quality of anchor stations in a 5G NSA network provided in Example 1 may be implemented using another evaluation system model provided in this embodiment.

The evaluation system model used in this embodiment mainly evaluates from three aspects: occupancy, residence, and experience. It uses four evaluation factors, namely, terminal, network, pipeline, and service, to perform a perceptual evaluation, and conducts a comprehensive perceptual evaluation of the anchor station quality in the network service quality dimension and the network service perception dimension.

According to step 102, an evaluation factor to be evaluated is selected, and the evaluation principle of each evaluation factor is as follows:

Terminal: number of 5G NSA terminals under the anchor station, terminal function switch status, and other information about the anchor station.

Network: Analyze network quality based on attachment, dual connection, number of cell users, performance, etc.

Pipeline: Based on analysis of service pipeline TCP quality, etc.

Service: Based on analysis of high-bandwidth video service quality, etc.

The evaluation indicators and calculation methods of the indicators used in each evaluation factor are:

Number of 5G terminals: obtained through DPI. 4G and 5G signaling statistics all online 5G terminal users.

Number of 5G terminals attached to 5G networks: obtained through DPI. Number of 5G terminal users attached to 5G networks.

Proportion of 5G terminals attached to 5G networks: obtained through DPI. [Number of 5G terminals attached to 5G networks]/[Number of 5G terminals].

Total number of 5G terminals reporting 5G on/off status: obtained through DPI. Number of terminals with dual connectivity capabilities reported.

Number of 5G terminals with 5G switched on: obtained through DPI. The number of terminals with dual connectivity capabilities reported with 5G switched on.

The percentage of 5G terminals with 5G switch on: [Number of 5G terminals with 5G switch on]/[Total number of 5G terminals reporting 5G switch status].

Anchor station 5G terminal attachment success rate: obtained through DPI. [ATTACH success number]/[ATTACH request number], the statistical scope is: 1. 5G terminal, based on 5G terminal library identification; 2. 4G anchor station, based on the anchor station configuration table.

ERAB modification indication success rate: obtained through DPI. [E-RAB modification success number]/[E-RAB modification request number], E-RAB modification indication success number: counts the number of E-RAB MODIFICATION CONFIRM messages; E-RAB modification indication request number: counts the cumulative number of E-RAB MODIFICATION INDICATION messages.

SgNB addition success rate: obtained through the integrated network management system. [Number of successful SgNB additions]/[Number of SgNB addition requests].

Anchor point RRC connection establishment success rate: obtained through the integrated network management system. [RRC connection establishment success number]/[RRC connection establishment request number].

Regional 5G access number residence ratio: obtained through DPI. Calculated in the 5G coverage area, 5G access number residence ratio = [5G terminal access times in 5G network services/[5G terminal access times in 5G network services + 5G terminal access times in 4G network services]; The statistical scope of the 5G coverage area is all anchor stations and 5G stations in the area where the anchor priority scheme has been implemented.

Regional 5G traffic retention ratio: obtained through DPI. Calculated in the 5G coverage area, 5G traffic retention ratio = [5G terminal traffic in 5G network] / [5G terminal traffic in 5G network + 5G terminal traffic in 4G network]; the statistical scope of the 5G coverage area is all anchor stations and 5G stations in the area where the anchor priority scheme has been implemented.

Regional 5G duration residence ratio: obtained through DPI. Calculated in the 5G coverage area, 5G traffic residence ratio = [5G terminal in 5G network service time] / [5G terminal in 5G network service time + 5G terminal in 4G network service time]; the statistical scope of the 5G coverage area is all anchor stations and 5G stations in the area where the anchor priority scheme has been implemented.

SN abnormal release rate (drop rate): obtained through the integrated network management system. 1-[Number of SN normal release requests]/[Number of SN release requests].

SN switching success rate: obtained through the integrated network management system. [SN switching success times]/[SN switching request times].

Anchor wireless drop rate: obtained through the integrated network management system. [Number of abnormal eNodeB release requests]/[Number of eNodeB release requests].

Video download rate: obtained through DPI. Streaming media package download volume/streaming media download duration; Streaming media package download volume: the amount of data downloaded from the time when the SP sends the "HTTP Reply 200OK" message to the terminal to the time when the SP sends the "finish TCP data ack" message to the terminal; Streaming media download duration: the time from the time when the SP sends the "HTTP Reply 200OK" message to the terminal to the time when the SP sends the "finish TCP data ack" message to the terminal.

Rate-to-bitrate ratio: obtained through DPI. Effective video download rate/bitrate; Effective download rate = streaming application layer download volume of valid streaming documents/(total streaming download time-data transmission time without data packets).

Initial buffering success rate: obtained through DPI. Initial buffering success times/initial buffering request times*100.

Initial buffering delay: obtained through DPI. Total initial buffering delay/number of initial buffering successes.

Video freeze rate: obtained through DPI. Number of freezes/number of initial buffering successes*100.

Video freeze duration ratio: obtained through DPI. Total buffering duration during playback/number of initial buffering successes*100.

Backhaul uplink effective rate: obtained through DPI. [Streaming load]/[Streaming download duration].

Downstream streaming rate (kbps): obtained through DPI. [Downstream streaming traffic]/[Downstream transmission duration].

TCP 12-way handshake success rate: obtained through DPI. TCP SYN ACK times/TCP SYN times*100%.

TCP 23-way handshake success rate: obtained through DPI. TCP ACK times/TCP SYN ACK times*100%.

TCP 12-way handshake delay: obtained through DPI. TCP SYN ACK time-TCP SYN time sum/TCP SYNACK times,

TCP 23-way handshake delay: obtained through DPI. TCP ACK time-TCP SYN ACK time/TCP ACK times.

TCP uplink retransmission rate: obtained through DPI. Number of TCP uplink retransmission packets/number of uplink packets*100%.

TCP downlink retransmission rate: obtained through DPI. Number of TCP downlink retransmission packets/number of downlink packets*100%.

TCP uplink out-of-order rate: obtained through DPI. Number of TCP uplink out-of-order packets/number of uplink packets*100%.

TCP downstream out-of-order rate: obtained through DPI. TC;P number of downstream out-of-order packets/number of downstream packets*100%.

Average uplink PRB utilization: obtained through the integrated network management system. Average uplink PRB utilization of the cell during the statistical period.

Downlink PRB average utilization: obtained through the integrated network management system. The average downlink PRB utilization of the cell during the statistical period.

Uplink interference noise (average): obtained through the integrated network management system. Uplink average interference level.

Average number of users (RRC connection): obtained through the integrated network management system. The average number of RRC connected users during the statistical period.

Anchor point paging congestion rate: obtained through the integrated network management system. Number of discarded paging/number of paging requests.

Anchor point coverage: obtained through the integrated MR system. Coverage is defined as the percentage of test points with an F value of 1 among all test points in the test area.

According to step 103, as shown in FIG5, weights are set for the evaluation factors and evaluation indicators. The weights of the evaluation factors and the evaluation indicators are shown in Table 2, where the weights, reference values, and challenge values can be adjusted according to actual experience. Since there are many evaluation indicator values in the model provided in this embodiment, when setting the weights, not only the two levels of evaluation factors and evaluation indicators provided in Embodiments 1 and 2 are set, but also the level of evaluation sub-factors is added to some evaluation factors. In the specific use of the anchor station quality evaluation method of the present invention, the appropriate evaluation level can be selected according to actual needs, and the evaluation indicators can be organized.

Table 2

According to step 104, the weighted total score of the evaluation factors is calculated as the quality score of the evaluation area.

In order to obtain the quality score more accurately, instead of using the value of the evaluation indicator directly, the score of each evaluation indicator can be calculated using the following formula.

Use formula 3 to calculate the scores of quantity, proportion, coverage, success rate, and speed indicators:

Use Formula 4 to calculate the scores of utilization, retransmission rate, drop rate, congestion rate, jam rate, jam duration ratio, latency, and interference noise indicators:

The above formula is used to calculate the value of the evaluation index, and the weighting of the evaluation index and the evaluation factor is calculated to obtain the quality score of the anchor station in the evaluation area.

A comprehensive comparison between the evaluation system model provided in this embodiment and the evaluation system model in Example 2 shows that the evaluation system model in this embodiment is superior to the evaluation system model in Example 2. The evaluation system model in this embodiment has finer dimensions and wider coverage, and is more valuable for end-to-end quality evaluation.

Embodiment 4:

On the basis of the method for evaluating the quality of anchor stations in a 5G NSA network provided in the above-mentioned embodiments 1 to 3, the present invention further provides a device for evaluating the quality of anchor stations in a 5G NSA network that can be used to implement the above-mentioned method, as shown in FIG6 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The device for evaluating the quality of anchor stations in a 5G NSA network in this embodiment includes one or more processors 21 and a memory 22. Among them, FIG6 takes one processor 21 as an example.

The processor 21 and the memory 22 may be connected via a bus or other means, and FIG6 takes the connection via a bus as an example.

The memory 22 is a non-volatile computer-readable storage medium for the method for evaluating the quality of anchor stations in a 5G NSA network, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the method for evaluating the quality of anchor stations in a 5G NSA network in Embodiments 1 to 3. The processor 21 executes various functional applications and data processing of the device for evaluating the quality of anchor stations in a 5G NSA network by running the non-volatile software programs, instructions and modules stored in the memory 22, that is, implementing the method for evaluating the quality of anchor stations in a 5G NSA network in Embodiments 1 to 3.

The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The program instructions/modules are stored in the memory 22. When executed by one or more processors 21, the method for evaluating the quality of anchor stations in a 5G network in the above-mentioned embodiments 1 to 3 is executed, for example, the various steps shown in Figures 1, 2 and 3 described above are executed.

A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (4)

1. A method for evaluating the quality of an anchor station in a 5G NSA network is characterized by comprising the following steps:

acquiring anchor point station quality index data by one or more modes of drive test, single station verification, comprehensive network management system, MR and DPI;

the network service quality dimension and/or the network service perception dimension are used as evaluation dimensions, and evaluation factors needing to be evaluated are selected from the quality index data; when the network service quality is taken as an evaluation dimension, one or more of coverage level, anchor point capacity, interference avoidance and anchor point performance are taken as evaluation factors; when network service perception is taken as an evaluation dimension, one or more of a terminal, a network, a pipeline and a service are taken as evaluation factors;

Selecting at least one level of evaluation factors to be evaluated according to service requirements, and setting weights for each evaluation factor according to service characteristics; selecting an evaluation index to be evaluated for each evaluation factor, and setting a weight for each evaluation index according to the service characteristics;

calculating a weighted total score of each evaluation factor of a preset evaluation area according to the values and weights of the evaluation factors and the evaluation indexes, and taking the weighted total score as a quality score of the preset evaluation area;

when a terminal is used as the evaluation factor, determining whether the terminal initiates directional switching to an anchor point cell according to the access state of the terminal in a non-anchor point cell or the VoLTE service state, setting an anchor frequency point carried in a rrcConnectionRelease message when the terminal is released in the non-anchor point cell as the highest priority for directional reselection, ensuring that the non-anchor point cell is easier to switch to the anchor point cell but the anchor point cell is harder to switch to the non-anchor point cell according to an NSA independent mobility policy, prohibiting NSA terminal load balancing to other frequency points by using a policy prohibiting NSA load balancing, selecting an anchor point station based on the policies of directional switching, directional reselection, NSA independent mobility policy and prohibiting NSA load balancing, and using a 5G NSA terminal which has the same frequency as the anchor point station and supports dual-connection capability as the evaluation factor;

Acquiring the evaluation factor of the current pipeline based on a service pipeline TCP quality analysis when the pipeline is used as the evaluation factor;

when using the network as the evaluation factor, the network quality is analyzed based on one or more of the attach, dual connectivity, cell user volume and performance, with the analysis result being the evaluation factor of the current network.

2. The method for quality assessment of anchor stations in a 5G NSA network according to claim 1, wherein the calculating a weighted sum of the assessment factors of the preset assessment area includes:

calculating the weighted total score of all the evaluation indexes in each evaluation factor according to the value and the weight of each evaluation index, and taking the weighted total score of all the evaluation indexes as the value of the evaluation factor;

and calculating the weighted total score of all the evaluation factors according to the value and the weight of each evaluation factor.

3. The method for quality assessment of anchor stations under a 5G NSA network according to claim 1, wherein the calculating a weighted sum of all assessment factors of a preset assessment area includes:

presetting a reference value and a challenge value for each evaluation index in each evaluation factor;

when the value of the first type of evaluation index is smaller than the reference value, calculating the evaluation index according to the preset lower limit value of the evaluation index;

When the value of the first type of evaluation index is larger than or equal to the challenge value, calculating the evaluation index according to the preset upper limit value of the evaluation index;

when the value of the second type of evaluation index is larger than the reference value, calculating the evaluation index according to the preset lower limit value of the evaluation index;

and when the value of the second type of evaluation index is smaller than or equal to the challenge value, calculating the evaluation index according to the preset upper limit value of the evaluation index.

4. An apparatus for evaluating quality of an anchor station in a 5G NSA network, wherein the apparatus is characterized by:

comprising at least one processor and a memory connected by a data bus, the memory storing instructions for execution by the at least one processor, the instructions, when executed by the processor, for performing the method of anchor station quality assessment under a 5G NSA network according to any one of claims 1-3.