CN120034819A

CN120034819A - Method, device and network side equipment for obtaining positioning model performance information

Info

Publication number: CN120034819A
Application number: CN202311567614.3A
Authority: CN
Inventors: 于航; 崇卫微
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2023-11-22
Filing date: 2023-11-22
Publication date: 2025-05-23
Also published as: WO2025108287A1

Abstract

The present application discloses a method, an apparatus and a network-side device for obtaining positioning model performance information, and belongs to the field of wireless communication technology. The method for obtaining positioning model performance information in an embodiment of the present application includes: a first network element sends a positioning message to at least one first access network device, where the positioning message is used to request a report of one of the following: AI measurement information generated by reasoning using a target positioning model, AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information; the first network element receives a positioning result message sent by at least one first access network device, where the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: AI measurement information, AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information; the first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

Description

Method and device for acquiring performance information of positioning model and network side equipment

Technical Field

The application belongs to the technical field of wireless communication, and particularly relates to a method and a device for acquiring performance information of a positioning model and network side equipment.

Background

Artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) localization aims at predicting measurement information using AI localization models, thus giving a position estimate of the target, so that a lot of measurement resources can be saved and the computational power consumption can be reduced.

In the related art, an AI positioning model may be set in the access network device, and used for performing model reasoning to obtain AI measurement information, and reporting the AI measurement information to a positioning control function network element, for example, a location management function (Location Management Function, LMF), where the positioning control function network element may perform location estimation based on the received AI measurement information.

However, in practical applications, the performance of the AI-positioning model is unknown, so that the accuracy and precision of an estimation result obtained by performing position estimation using AI measurement information obtained by reasoning of the AI-positioning model cannot be determined, and whether the obtained estimation result is accurate cannot be obtained.

Disclosure of Invention

The embodiment of the application provides a method, a device and network side equipment for acquiring performance information of a positioning model, which can solve the problem that the accuracy and precision of an estimation result obtained by performing position estimation by using AI measurement information obtained by reasoning of the AI positioning model cannot be determined due to unknown performance of the AI positioning model.

The first aspect provides a method for acquiring performance information of a positioning model, which comprises the steps that a first network element sends positioning information to at least one first access network device, wherein the positioning information is used for requesting reporting of one of AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information which are generated by reasoning through the target positioning model, the first network element receives positioning result information sent by the at least one first access network device, the positioning result information carries target positioning measurement information, and the target positioning measurement information comprises one of the AI measurement information, the real measurement information and difference information between the AI measurement information and the real measurement information, and the first network element acquires first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

The positioning monitoring information sending method of the positioning model comprises the steps that first access network equipment receives positioning information sent by a first network element, wherein the positioning information is used for requesting reporting of one of AI measurement information, AI measurement information and real measurement information which are generated by reasoning through the target positioning model, difference information between the AI measurement information and the real measurement information, the first access network equipment obtains the real measurement information and the AI measurement information which is deduced through the target positioning model, and the first access network equipment sends positioning result information to the first network element on the basis of the real measurement information and the AI measurement information, wherein the positioning result information carries target positioning measurement information, and the target positioning measurement information comprises one of the AI measurement information, the real measurement information, the difference information between the AI measurement information and the real measurement information.

The positioning model performance information acquisition device comprises a first sending module and a first receiving module, wherein the first sending module is used for sending positioning information to at least one first access point first access network device, the positioning information is used for requesting reporting of one of AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information which are generated by reasoning through the target positioning model, the first receiving module is used for receiving positioning result information sent by the at least one first access network device, the positioning result information carries target positioning measurement information, and the target positioning measurement information comprises one of the AI measurement information, the real measurement information and the difference information between the AI measurement information and the real measurement information, and the first acquisition module is used for acquiring first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

In a fourth aspect, a positioning monitoring information sending device of a positioning model is provided, and the positioning monitoring information sending device comprises a second receiving module and a second sending module, wherein the second receiving module is used for receiving a positioning message sent by a first network element, the positioning message is used for requesting to report one of AI measurement information, AI measurement information and real measurement information which are generated by reasoning through the target positioning model, difference information between the AI measurement information and the real measurement information, the second obtaining module is used for obtaining the real measurement information and the AI measurement information which is deduced through the target positioning model, and the second sending module is used for sending a positioning result message to the first network element based on the real measurement information and the AI measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information comprises one of the AI measurement information, the real measurement information, and the difference information between the AI measurement information and the real measurement information.

In a fifth aspect, a network side device is provided, comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method as described in the first aspect, or implement the steps of the method as described in the second aspect.

In a sixth aspect, a network side device is provided, which includes a processor and a communication interface, where the processor is configured to implement the steps of the method according to the first aspect, or implement the steps of the method according to the second aspect, and the communication interface is configured to be coupled to the processor.

In a seventh aspect, there is provided a readable storage medium having stored thereon a program or instructions which when executed by a processor, performs the steps of the method according to the first aspect or performs the steps of the method according to the second aspect.

In an eighth aspect, a communication system is provided, comprising a first network element operable to perform the steps of the method according to the first aspect, and an access network device operable to perform the steps of the method according to the second aspect.

In a ninth aspect, there is provided a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being for running a program or instructions to implement the method according to the first aspect or to implement the method according to the second aspect.

In a tenth aspect, there is provided a computer program/program product stored in a storage medium, the program/program product being executable by at least one processor to implement the method according to the first aspect or to implement the method according to the second aspect.

In the embodiment of the application, a first network element sends a positioning message to at least one first access network device, wherein the positioning message is used for requesting reporting one of AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information which are generated by reasoning by using a target positioning model, the first network element receives a positioning result message sent by the at least one first access network device, the positioning result message carries the target positioning measurement information, the target positioning measurement information comprises one of the AI measurement information, the real measurement information and the difference information between the AI measurement information and the real measurement information, and the first network element acquires first positioning performance monitoring information of the target positioning model based on the target positioning measurement information. Therefore, the first network element can monitor the positioning performance of the target positioning model based on the target positioning measurement information reported by the first access network equipment, and the problem that the accuracy and precision of an estimation result obtained by using the AI measurement information obtained by reasoning of the AI positioning model for position estimation cannot be determined due to the unknown performance of the AI positioning model is solved.

Drawings

Fig. 1 shows a block diagram of a wireless communication system to which embodiments of the present application are applicable;

FIG. 2 is a schematic diagram of a layout of an AI positioning model in accordance with an embodiment of the application;

fig. 3 is a schematic flow chart of a method for obtaining performance information of a positioning model according to an embodiment of the present application;

Fig. 4 is a schematic flow chart of sending positioning monitoring information of a positioning model according to an embodiment of the present application;

Fig. 5 is a schematic flow chart of another method for obtaining performance information of a positioning model according to an embodiment of the present application;

fig. 6 is a schematic flow chart of a method for obtaining performance information of a positioning model according to an embodiment of the present application;

Fig. 7 is a schematic structural diagram of an apparatus for acquiring performance information of a positioning model according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a structure of a positioning monitoring information device of a positioning model according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a communication device according to an embodiment of the present application;

fig. 10 shows a schematic hardware structure of a network side device according to an embodiment of the present application;

Fig. 11 shows a schematic hardware structure of another network side device according to an embodiment of the present application.

Detailed Description

The technical solutions of the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the application, fall within the scope of protection of the application.

The terms "first," "second," and the like, herein, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the "first" and "second" distinguishing between objects generally are not limited in number to the extent that the first object may, for example, be one or more. Furthermore, the "or" in the present application means at least one of the connected objects. For example, "A or B" encompasses three schemes, namely scheme one including A and excluding B, scheme two including B and excluding A, scheme three including both A and B. The character "/" generally indicates that the context-dependent object is an "or" relationship.

The term "indication" according to the application may be either a direct indication (or an explicit indication) or an indirect indication (or an implicit indication). The direct indication may be understood that the sender explicitly informs the specific information of the receiver, the operation to be executed, the request result, and the like in the sent indication, and the indirect indication may be understood that the receiver determines the corresponding information according to the indication sent by the sender, or determines the operation to be executed, the request result, and the like according to the determination result.

It should be noted that the techniques described in the embodiments of the present application are not limited to long term evolution (Long Term Evolution, LTE)/LTE evolution (LTE-Advanced, LTE-a) systems, but may also be used in other wireless communication systems, such as code division multiple access (Code Division Multiple Access, CDMA), time division multiple access (Time Division Multiple Access, TDMA), frequency division multiple access (Frequency Division Multiple Access, FDMA), orthogonal frequency division multiple access (Orthogonal Frequency Division Multiple Access, OFDMA), single-carrier frequency division multiple access (Single-carrier Frequency-Division Multiple Access, SC-FDMA), or other systems. The terms "system" and "network" in embodiments of the application are often used interchangeably, and the techniques described may be used for both the above-mentioned systems and radio technologies, as well as other systems and radio technologies. The following description describes a New Radio (NR) system for exemplary purposes and NR terminology is used in much of the following description, but the techniques may also be applied to systems other than NR systems, such as the 6 th Generation (6G) communication system.

Fig. 1 shows a block diagram of a wireless communication system to which an embodiment of the present application is applicable. The wireless communication system includes a terminal 11 and a network device 12. The terminal 11 may be a Mobile phone, a tablet Computer (Tablet Personal Computer), a Laptop (Laptop Computer), a notebook (Personal DIGITAL ASSISTANT, PDA), a palm Computer, a netbook, an Ultra-Mobile Personal Computer (Ultra-Mobile Personal Computer, UMPC), a Mobile internet device (Mobile INTERNET DEVICE, MID), a Personal Digital Assistant (PDA), Augmented Reality (Augmented Reality, AR), virtual Reality (VR) devices, robots, wearable devices (Wearable Device), aircraft (FLIGHT VEHICLE), in-vehicle devices (Vehicle User Equipment, VUE), on-board equipment, pedestrian terminals (PEDESTRIAN USER EQUIPMENT, PUE), smart home (home appliances having wireless communication function, such as refrigerator, television, Washing machine or furniture, etc.), game machine, personal computer (Personal Computer, PC), teller machine or self-service machine, etc. The wearable device comprises an intelligent watch, an intelligent bracelet, an intelligent earphone, intelligent glasses, intelligent jewelry (intelligent bracelets, intelligent rings, intelligent necklaces, intelligent anklets, intelligent footchains and the like), an intelligent wristband, intelligent clothing and the like. The in-vehicle apparatus may also be referred to as an in-vehicle terminal, an in-vehicle controller, an in-vehicle module, an in-vehicle component, an in-vehicle chip, an in-vehicle unit, or the like. It should be noted that the specific type of the terminal 11 is not limited in the embodiment of the present application. The network-side device 12 may include an access network device or core network device, where the access network device may also be referred to as a radio access network (Radio Access Network, RAN) device, a radio access network function, or a radio access network element. The Access network device may include a base station, a wireless local area network (Wireless Local Area Network, WLAN) Access Point (AS), or a wireless fidelity (WIRELESS FIDELITY, WIFI) node, etc. wherein the base station may be referred to as Node B (NB), evolved Node B (eNB), next generation Node B (the next generation Node B, gNB), new air interface Node B (NR Node B), access point, relay station (Relay Base Station, RBS), serving base station (Serving Base Station, SBS), base transceiver station (Base Transceiver Station, BTS), A radio base station, a radio transceiver, a Basic service set (Basic SERVICE SET, BSS), an Extended service set (Extended SERVICE SET, ESS), a Home Node B (HNB), a home evolved Node B (home evolved Node B), a transmission and reception point (Transmission Reception Point, TRP), or some other suitable terminology in the field, so long as the same technical effect is achieved, the base station is not limited to a specific technical vocabulary, and it should be noted that, in the embodiment of the present application, only a base station in an NR system is described by way of example, and the specific type of the base station is not limited.

The core Network device may include, but is not limited to, at least one of a core Network node, a core Network Function, a Mobility management entity (Mobility MANAGEMENT ENTITY, MME), an access Mobility management Function (ACCESS AND Mobility Management Function, AMF), a session management Function (Session Management Function, SMF), a user plane Function (User Plane Function, UPF), a Policy control Function (Policy Control Function, PCF), a Policy AND CHARGING Rules Function (PCRF), an edge application service discovery Function (Edge Application Server Discovery Function, EASDF), a Unified data management (Unified DATA MANAGEMENT, UDM), a Unified data repository (Unified Data Repository, UDR), a home subscriber server (Home Subscriber Server, HSS), a centralized Network configuration (Centralized Network configuration, CNC), a Network storage Function (Network Repository Function, NRF), a Network opening Function (Network Exposure Function, NEF), a Local NEF (or L-NEF), a binding support Function (Binding Support Function, BSF), an application Function (Application Function, AF), a location management Function (Location Management Function, LMF), a gateway's mobile location center (Gateway Mobile Location Centre, GMLC), a Network data analysis Function (NWDAF), and the like. It should be noted that, in the embodiment of the present application, only the core network device in the NR system is described as an example, and the specific type of the core network device is not limited.

In the related art, as shown in fig. 2, an AI-positioning model is provided at a base station, and a UE transmits measurement feedback (e.g., sounding reference signal (Sounding REFERENCE SIGNAL, SRS) to the base station, and the base station calculates AI-measurement information using the AI-positioning model, and then transmits the AI-measurement information to an LMF to implement position calculation.

However, the accuracy of the model of the AI positioning model may not be high in actual use, the positioning accuracy may be poor, and the performance of the AI positioning model is unknown, so that the accuracy and precision of an estimation result obtained by performing position estimation by using AI measurement information obtained by reasoning of the AI positioning model cannot be determined, and whether the obtained estimation result is accurate cannot be obtained.

In order to solve the problem, the embodiment of the application provides a method, a device and network side equipment for acquiring performance information of a positioning model, which can be used for monitoring information such as positioning precision, model accuracy and the like of a specific AI model by triggering a network by a trigger, and reporting the performance result information of the AI model to the trigger under the condition that a reporting condition is met.

The technical scheme provided by the embodiment of the application is described in detail through some embodiments and application scenes thereof by combining the attached drawings.

Fig. 3 is a flowchart of a method for obtaining performance information of a positioning model according to an embodiment of the present application, where the method 300 may be performed by a first network element. In other words, the method may be performed by software or hardware installed on the communication device. In an embodiment of the present application, the first network element may be a positioning control function network element, for example, a location management function (Location Management Function, LMF).

As shown in fig. 3, the method may include the following steps.

S310, the first network element sends a positioning message to at least one first access network device.

In an embodiment of the present application, the positioning message is used to request reporting of one of:

(1) And the first network element can request at least one first access network device to report the AI measurement information which is generated by using the target positioning model for reasoning through the positioning message, and the first access network device can use the target positioning model for reasoning after receiving the positioning message so as to acquire the AI measurement information.

(2) The first network element can request at least one first access network device to report the AI measurement information and the real measurement information which are generated by reasoning by using a target positioning model through a positioning message, and the first access network device can use the target positioning model to conduct reasoning after receiving the positioning message so as to acquire the AI measurement information and acquire the real measurement information in a non-AI mode, for example, by measuring a positioning reference signal.

(3) And difference information between the AI measurement information and the real measurement information. The first network element can request at least one first access network device to report difference information between AI measurement information and real measurement information generated by reasoning by using a target positioning model through a positioning message, the first access network device can use the target positioning model to perform reasoning after receiving the positioning message so as to acquire the AI measurement information, and acquire the real measurement information by adopting a non-AI mode, for example, by measuring a positioning reference signal, and then acquire the difference information between the AI measurement information and the real measurement information through comparison.

Optionally, the positioning message may carry at least one of the following:

(1) The identification information is used for indicating the target positioning model to be monitored, wherein the number of the target positioning models to be monitored can be one or a plurality of the target positioning models, and the embodiment of the application is not limited.

(2) The first indication information is used for indicating the first access network equipment to locate by using the AI measurement information inferred by the target location model, and the first indication information can also indicate the first access network equipment to report the AI measurement information inferred by the first access network equipment by using the target location model. For example, the first indication information may be an AI-positioning indication (AI positioning ind).

(3) And the second indication information is used for indicating that the first network element is to monitor the performance of the target positioning model. The second indication information indicates that the first network element will monitor the performance of the target positioning model, and the first access network device can determine, according to the second indication information, that the AI measurement information, the AI measurement information and the real measurement information, and the difference information between the AI measurement information and the real measurement information need to be reported. For example, the second indication information may be a Monitoring indication (Monitoring ind).

In an alternative implementation, the first network element may be triggered by the target device to at least one first access network device. Therefore, in the alternative implementation manner, before S310, the method may further include the first network element receiving a positioning performance monitoring message sent by a target device, where the positioning performance monitoring message is used to request performance measurement on the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model, and the target device includes one of the first access network device, the second access network device, and the second network element. That is, in this alternative implementation, the first network element may perform performance monitoring of the object location model under the triggering of the first access network device, the second access network device, or the second network element.

For example, the first access network device may train itself to generate the target positioning model, and then request the first network element to monitor the performance of the target positioning model, or the second access network device may train itself to generate the target positioning model and send the target positioning model to the first access network device for use, and the second access network device may request the first network element to monitor the performance of the target positioning model, or the second network element trains to generate the target positioning model and send the target positioning model to the first access network device for use, and the second network element may request the first network element to monitor the performance of the target positioning model.

Or the first network element can trigger to monitor the performance of the target positioning model. For example, the first network element trains and generates the target positioning model, and sends the target positioning model to at least one first access network device for use. To obtain the positioning performance of the target positioning model, the first access network device may trigger performance monitoring of the target positioning model, e.g. performing S310 described above.

In the above implementation manner, the second Network element may be a Network data analysis Function (Network DATA ANALYTICS Function, NWDAF), or may be another Network element for training the object location model, which is not limited in the embodiment of the present application.

In the above implementation manner, the identification information of the target positioning Model carried in the positioning performance monitoring message may be an identification of the target positioning Model, that is, a Model ID, or may be a function identification of the target positioning Model (functionality ID), or an analysis identification of the target positioning Model (ANALYTICS ID).

In an optional implementation manner, the positioning performance monitoring message may further carry at least one of the following:

(1) The identification information of the target device, for example, the identification information of the first access network device, the identification information of the second access network device or the identification information of the second network element, is used for indicating the identity information of the target device sending the positioning performance monitoring message. For example, if the access network device (including the first access network device and the second access network device) is an identification of the access network device, for example, a global cell identifier (cell global identity, CGI), or the identification information may also be an operation administration maintenance (Operation Administration AND MAINTENANCE, OAM) identification of the access network device (including the first access network device and the second access network device).

(2) Monitoring time period information (monitoring period) for indicating a time of performance monitoring of the target positioning model, and a time of performance monitoring of the model, for example, a future day, a future month, etc., can be indicated by the monitoring time period information. Alternatively, the monitoring period information may include at least one of a monitoring start time, a monitoring end time, a monitoring repetition pattern, a monitoring repetition start time, a monitoring repetition end time, and the like

(3) A first threshold (notify threshold) for indicating to notify the target device if the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold. The first threshold may characterize reporting positioning performance monitoring information of the target positioning model to the target device if the performance parameter reaches a certain condition. For example, if a timed report, the first threshold may include one or more of a report start time, a report interval time, and a number of reports. If a condition report, the first threshold may be at least one of a positioning accuracy threshold, a model accuracy threshold.

In the above alternative implementation, the positioning performance monitoring message may be used to request performance monitoring data of a single target positioning model, or may request performance monitoring data of multiple target positioning models. For example, the positioning performance monitoring message may be a model performance monitoring request message for which the first network element may reply to the performance monitoring data of the target positioning model once. Or the positioning performance monitoring message may also be a model performance monitoring subscription message, for which the first network element may need to reply to the performance monitoring data of the target positioning model multiple times, for example, reply to the target device once each time the performance monitoring data of the target positioning model changes, or reply to the target device periodically according to a certain period.

In the above optional implementation manner, optionally, after the first network element receives the positioning performance monitoring message sent by the target device, the method may further include sending, by the first network element, a positioning performance monitoring response message to the target device, where the positioning performance monitoring response message carries identification information of the target positioning model. The first network element of the target device can be informed of the received positioning performance monitoring message through the positioning performance monitoring response.

In an optional implementation manner, after the first network element receives the positioning performance monitoring message, if the first network element has known second positioning performance monitoring information of the target positioning model and the known second positioning performance monitoring information meets a second threshold, the positioning performance monitoring response message may further carry a performance index corresponding to the second positioning performance monitoring information, for example, positioning accuracy information, model accuracy information and the like. The second threshold may be the same as the first threshold, for example, in a case where the first threshold is a positioning accuracy threshold or a model accuracy threshold, or the second threshold may be different from the first threshold, for example, in a case where the first threshold includes one or more of a reporting start time, a reporting interval time, and a reporting number, the second threshold is different from the first threshold, for example, the second threshold may be a preset value or a value indicated by the network side device, or the like. In this optional implementation manner, if the first network element has completed performance monitoring on the target positioning model and the performance meets the requirement (for example, if the positioning performance monitoring message carries a first threshold, and the first threshold is a positioning accuracy threshold or a model accuracy threshold), the first network element may directly inform the target device of the performance index corresponding to the second positioning performance monitoring information in the response message, so that it is not necessary to initiate the performance monitoring process again, thereby saving the process.

In the above optional implementation manner, the positioning performance monitoring response message may further carry at least one of the following:

(1) And fourth indication information, configured to indicate one of accepting performance monitoring on the target positioning model and rejecting performance monitoring on the target positioning model, for example, when the first network element has completed performance monitoring on the target positioning model, and a performance index corresponding to second positioning performance monitoring information obtained by monitoring does not meet a requirement (for example, a performance index corresponding to the acquired second positioning performance monitoring information does not meet a second threshold), where the fourth indication information carried in the positioning performance monitoring response message may indicate rejection of performance monitoring on the target positioning model.

(2) And the reason value is used for indicating the reason for refusing to monitor the performance of the target positioning model. For example, reasons for refusing performance monitoring of the target positioning model include at least one of network congestion, load of the first network element being greater than a threshold, model performance of the target positioning model not meeting requirements, and positioning performance monitoring of the target positioning model being performed.

After receiving the positioning performance monitoring response message, the target device can learn whether the first network element accepts performance monitoring on the target positioning model, and if not, can further learn the reject reason.

In an alternative implementation manner, after receiving the positioning performance monitoring message, the first network element may execute S310, then execute S312 and S314 below, obtain first positioning performance monitoring information of the target positioning model, and then send a positioning performance monitoring notification message to the target device, where the positioning performance monitoring notification message carries identification information of the target positioning model and a performance index corresponding to the first positioning performance monitoring information, for example, at least one of positioning accuracy information and model accuracy information corresponding to the first positioning performance monitoring information. Therefore, the target equipment can acquire the performance index of the target positioning model, and further can determine whether to use the target positioning model for reasoning or not based on the performance index.

S312, the first network element receives the positioning result message sent by the at least one first access network device.

The positioning result message carries target positioning measurement information, wherein the target positioning measurement information comprises one of the AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information.

After receiving the positioning message, the first access network device may send a positioning result message to the first network element, and report at least one of the AI measurement information, the real measurement information, and the difference information between the AI measurement information and the real measurement information to the first network element through the positioning result message.

In an alternative implementation manner, the positioning result message may further carry the following information:

(1) The identification information of the target positioning model can indicate the target positioning model associated with the first network element and the target positioning measurement information.

(2) And the third indication information is used for indicating that the target positioning measurement information is carried in the positioning result message. The third indication information can indicate that the positioning result message carries information related to AI measurement information which is inferred by using the target positioning model. For example, the third indication information may be an AI information indication (AI info ind).

Optionally, the positioning result message may further carry first timestamp information, which is used to indicate a time when the AI measurement information is inferred by using the target positioning model. Corresponding AI measurement information and conventional measurement information can be found in the subsequent steps through the first timestamp information, so that performance comparison is convenient.

And S314, the first network element acquires first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

In an alternative implementation, in the case that the target positioning measurement information includes the AI measurement information and the real measurement information, in S314, the first network element may compare a difference between the AI measurement information and the real measurement information, and then acquire first positioning performance monitoring information of the target positioning model based on the difference between the AI measurement information and the real measurement information. For example, in the case that the number of the first access network devices is multiple, the first network element may obtain a difference between AI measurement information corresponding to each first access network device and real measurement information based on the AI measurement information and the real measurement information reported by each first access network device, and then calculate the difference corresponding to each first access network device based on a predetermined algorithm, so as to obtain the first positioning performance monitoring information, where the predetermined algorithm may be a variance, a Mean Square Error (MSE), an average absolute error (MAE), or the like, and the specific algorithm is not limited in the embodiment of the present application.

In an alternative implementation, in the case that the target positioning measurement information includes the difference information, in S314, the first network element may directly obtain the first positioning performance monitoring information of the target positioning model based on the difference information.

In an alternative implementation manner, in the case that the target positioning measurement information includes the AI measurement information, in S314, the first network element may acquire terminal location information, where the terminal location information includes terminal location estimation and second timestamp information, then acquire model location estimation based on the AI measurement information, and finally acquire first positioning performance monitoring information of the target positioning model based on a difference between the terminal location estimation and the model location estimation.

In the above alternative implementation manner, the first network element may acquire the terminal location information from the UE, or the first network element may also be terminal location information obtained by calculation, for example, terminal location information obtained according to real measurement information, or may also be terminal location information obtained by the first network element based on another positioning task at the same time (the positioning task uses a positioning method other than AI) or a new positioning task initiated in the time period, which is not limited in the embodiment of the present application.

In the above alternative implementation manner, the first network element may match the second timestamp information in the obtained terminal location information with the time information of the AI measurement information obtained by using the target positioning model inference, perform location estimation using the AI measurement information obtained at the same time to obtain a model location estimate, and compare the model location estimate with the terminal location estimate in the terminal location information to obtain a difference between the two location estimates.

In an alternative implementation manner, the first network element may reply to the target device with a model performance detection reply message or a model performance detection notification message, where the message may carry at least one of the following model performance information:

model identification information, model ID;

performance (performance) key performance indicators (Key Performance Indicators, KPI) for characterizing performance parameters of the target positioning model reasoning, which may include positioning accuracy information, model accuracy information, etc., for example.

In the above implementation manner, optionally, the first network element may send the model performance information to the target device if a local reporting policy or a condition of the first threshold is met.

For example, the first network element may send the model performance information to the first access network device, which may send the model performance monitoring information to a model trainer (e.g., LMF or NWDAF).

According to the technical scheme provided by the embodiment of the application, a first network element sends a positioning message to at least one first access network device, wherein the positioning message is used for requesting reporting one of AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information which are generated by reasoning by using the target positioning model, the first network element receives a positioning result message sent by the at least one first access network device, the positioning result message carries the target positioning measurement information, the target positioning measurement information comprises one of the AI measurement information, the real measurement information and the difference information between the AI measurement information and the real measurement information, and the first network element acquires first positioning performance monitoring information of the target positioning model based on the target positioning measurement information. Therefore, the first network element can monitor the positioning performance of the target positioning model based on the target positioning measurement information reported by the first access network equipment, and the problem that the accuracy and precision of an estimation result obtained by using the AI measurement information obtained by reasoning of the AI positioning model for position estimation cannot be determined due to the unknown performance of the AI positioning model is solved.

Based on the same technical conception, the embodiment of the application also provides a positioning monitoring information sending method of the positioning model.

Fig. 4 is a schematic flow chart of a method for sending positioning monitoring information of a positioning model according to an embodiment of the present application, where the method 400 may be performed by the first access network device, as shown in fig. 4, and the method mainly includes the following steps.

S410, the first access network device receives the positioning message sent by the first network element.

In the embodiment of the application, the positioning message is used for requesting to report one of AI measurement information, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information which are generated by reasoning by using the target positioning model.

Wherein the positioning message is the same as the positioning message in the method 300, the first network element may send the positioning message to the first access network device according to the related description in the method 300, and specific reference may be made to the related description in the method 300.

In an alternative implementation, before S410, the method may further include the first access network device sending a positioning performance monitoring message to the first network element, where the positioning performance monitoring message is used to request performance measurement on the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model. In this alternative implementation manner, the first access network device may trigger the first network element to perform performance monitoring on the target positioning model, for example, the first access network device may trigger the first network element to perform performance monitoring on the target positioning model after receiving the target positioning model distributed by the network side device, or the first access network device may trigger the first network element to perform performance monitoring on the target positioning model after training by itself to generate the target positioning model, so as to determine positioning performance of the target positioning model.

In the optional implementation manner, optionally, the positioning performance monitoring message may further carry at least one of the following:

(1) The identification information of the first access network device may be, for example, an Identification (ID) of the first access network device, for example, a global cell identifier (cell global identity, CGI), or the identification information may also be an operation administration maintenance (Operation Administration AND MAINTENANCE, OAM) identification of the first access network device (including the first access network device and the second access network device).

In the above alternative implementation, the positioning performance monitoring message may be used to request performance monitoring data of a single target positioning model, or may request performance monitoring data of multiple target positioning models. For example, the positioning performance monitoring message may be a model performance monitoring request message for which the first access network device may receive performance monitoring data of the primary target positioning model replied by the first network element. Or the positioning performance monitoring message may also be a model performance monitoring subscription message, for which the first access network device may receive performance monitoring data of the multiple target positioning models returned by the first network element, for example, when the performance monitoring data of the target positioning model changes each time, the first network element may return the performance monitoring data of the target positioning model to the first access network device once, or periodically return the performance monitoring data of the target positioning model to the first access network device according to a certain period.

In an optional implementation manner, after the first access network device sends the positioning performance monitoring message to the first network element, the method may further include the first access network device receiving a positioning performance monitoring response message sent by the first network element, where the positioning performance monitoring response message carries identification information of the target positioning model. Through the positioning performance monitoring response, the first access network device can know that the first network element has received the positioning performance monitoring message.

Optionally, the positioning performance monitoring response message may further carry a performance index corresponding to the second positioning performance monitoring information, for example, positioning accuracy information, model accuracy information, and the like. In this alternative implementation manner, if the first network element has completed performance monitoring on the target positioning model and the performance meets the requirement (for example, the performance index corresponding to the acquired second positioning performance monitoring information meets the second threshold), the first network element may directly inform the target device of the performance index corresponding to the second positioning performance monitoring information in the response message, so that it is not necessary to initiate the performance monitoring procedure again, thereby saving the procedure.

(1) And fourth indication information, configured to indicate one of accepting performance monitoring on the target positioning model and rejecting performance monitoring on the target positioning model, for example, when the first network element has completed performance monitoring on the target positioning model and the performance does not meet the requirement (for example, the performance corresponding to the acquired second positioning performance monitoring information indicates that the performance does not meet the second threshold value), where the fourth indication information carried in the positioning performance monitoring response message may indicate rejection of performance monitoring on the target positioning model.

After receiving the positioning performance monitoring response message, the first access network device can learn whether the first network element accepts performance monitoring on the target positioning model, and if not, can further learn the reject reason.

In an alternative implementation, the positioning message may carry at least one of the following:

(1) Identification information of the target positioning model;

(2) The first indication information is used for indicating the first access network equipment to locate by using the AI measurement information inferred by the target location model;

(3) And the second indication information is used for indicating that the first network element is to monitor the performance of the target positioning model.

Through the positioning message, the first access network device can acquire information that the first network element will perform performance monitoring on the target positioning model or that the first network element needs to acquire.

In a specific application, the positioning message may be a new radio positioning protocol a (NRPPa) positioning message.

And S412, the first access network equipment acquires real measurement information and AI measurement information deduced by using a target positioning model.

After receiving the positioning message, the first access network device knows at least one item of AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information, which are generated by using the target positioning model to carry out reasoning, and therefore the first access network device can use the target positioning model to carry out reasoning to obtain the AI measurement information.

And S414, the first access network equipment sends a positioning result message to the first network element based on the real measurement information and the AI measurement information.

In the embodiment of the application, the positioning result message carries target positioning measurement information, wherein the target positioning measurement information comprises one of the AI measurement information, real measurement information and difference information between the AI measurement information and the real measurement information.

The first access network equipment sends a positioning result message carrying target positioning measurement information to the first network element, so that the first network element can acquire positioning performance monitoring information of the target positioning model based on the target positioning measurement information, and monitoring of the positioning performance of the target positioning model is realized.

In an alternative implementation, the positioning result message carries the following information:

(1) Identification information of the target positioning model;

(2) And the third indication information is used for indicating that the target positioning measurement information is carried in the positioning result message.

Optionally, the positioning result message further carries first timestamp information, which is used for indicating the time when the AI measurement information is inferred by using the target positioning model.

In an alternative implementation manner, in the case that the first access network device sends the positioning performance monitoring message to the first network element, after S414, the method may further include the first access network device receiving a positioning performance monitoring notification message sent by the first network element, where the positioning performance monitoring notification message carries identification information of the target positioning model and a performance index corresponding to the first positioning performance monitoring information, for example, at least one of positioning accuracy information and model accuracy information corresponding to the first positioning performance monitoring information. Therefore, the first access network equipment can acquire the performance index of the target positioning model, and further can determine whether to use the target positioning model for positioning reasoning or not based on the performance index.

According to the technical scheme provided by the embodiment of the application, after receiving the positioning information, the first access network equipment can send the positioning result information to the first network element, wherein the positioning result information carries the target positioning measurement information, and the target positioning measurement information comprises one of the following AI measurement information, the real measurement information and the difference information between the AI measurement information and the real measurement information, so that the first network element can acquire the first positioning performance monitoring information of the target positioning model based on the target positioning measurement information, the monitoring of the positioning performance of the target positioning model is realized, and the problems of accuracy and precision of an estimation result obtained by position estimation by using the AI measurement information obtained by AI positioning model reasoning cannot be determined due to unknown performance of the AI positioning model are solved.

In the embodiment of the application, after the target equipment acquires the performance parameters of the target positioning model, the target positioning model can be optimized under the condition of poor model performance so as to improve the positioning precision of the target positioning model and optimize the service experience of AI positioning.

The technical solution provided by the embodiment of the present application is described below by taking the first network element as an LMF and the second network element as NWDAF as an example.

Fig. 5 is a schematic flow chart of another method for obtaining performance information of a positioning model according to an embodiment of the present application, as shown in fig. 5, the method mainly includes the following steps:

In step 501, the base station a acquires an AI-positioning model.

In fig. 5, base station a is represented by an access network device (RAN).

As shown in fig. 5, this step includes, but is not limited to, the following three embodiments:

step 501a, the base station a trains and generates an AI positioning model;

Step 501b, the AI positioning model generated by LMF training is then sent to the base station A;

step 501c, nwdaf trains the generative model and transmits to base station a.

It should be noted that, before step 501b or step 501c, optionally, the base station a may send a request or subscribe message to the LMF or NWDAF, requesting that the LMF or NWDAF provide the AI location model to the base station a.

In step 502, the base station a determines to perform performance monitoring on the target AI-positioning model, and triggers a performance monitoring procedure.

In step 503, the base station a sends a model performance monitoring request or subscription message to the LMF, requesting performance monitoring of the target AI-location model and feeding back performance monitoring information.

The model performance monitoring request or subscription message may carry at least one of the following:

Model identification information, information characterizing the target AI positioning Model, which may be, for example, one of Model IDs, functionalityID, ANALYTICS ID;

base station identification information, which characterizes identity information of the base station a requesting model detection, for example, CGI;

A monitoring period (monitoring period) characterizing a time at which the monitoring of the model performance is performed (e.g., a future day, a future month), e.g., may include at least one of a monitoring start time, a monitoring end time, a monitoring repetition pattern, a monitoring repetition start time, a detection repetition end time;

A first threshold (threshold) characterizing the reporting of the performance parameter to the base station a if a certain condition is reached, e.g. comprising a timing report, reporting start time, reporting interval time, reporting number of times, or comprising a condition report, a positioning accuracy threshold and a model accuracy threshold.

It should be noted that, the above model performance monitoring request or subscription message may be encapsulated in NRPPa messages and sent to the LMF, where NRPPa is a point-to-point protocol stack between NG-RAN and LMF, and is used to transfer signaling related to positioning, or a message may be added to transfer the request or subscription message.

It should be noted that, the reply corresponding to the model performance monitoring request message is one-time, and the reply corresponding to the model performance monitoring subscription message is one or more times.

It should be noted that, the monitoring period and the first threshold may be optional parameters, and when one or both of them are not provided, the LMF may detect and report according to a local policy or a default monitoring parameter, for example, the local configuration defaults to a future day when the detection period is a future day, and the first threshold is reported when the positioning accuracy is lower than 1 m.

Note that, the base station identification information may be a base station ID, or an OAM may be identified using an OAM ID corresponding to the base station.

Step 504, the LMF decides whether to execute the model detection according to the current service condition, the load condition and the known model detection performance.

If not, for example, the model ID requesting detection has been detected and has poor performance or is being detected, then step 505 is performed and the subsequent steps are ignored, and if so, then step 504 is performed and the subsequent steps are performed.

Alternatively, step 505 is not required and the LMF does not need to reply, and the LMF performs steps 506-513 after deciding to perform the monitoring and the LMF performs step 513 directly after deciding not to perform the monitoring.

Step 505, the LMF sends a model performance detection reply message to the base station a.

The model performance detection reply message may carry at least one of the following:

Model identification information characterizing the monitored target model ID;

Performance KPI, performance parameters characterizing model reasoning, including positioning accuracy information, model accuracy information and the like;

Fourth indication information characterizing acceptance or rejection of performance monitoring of the target model, including acceptance and rejection;

A cause value characterizing a cause of refusal of performance monitoring;

It should be noted that, when the LMF has known that the monitored performance or the known performance of the target model meets the first threshold in step 503, the performance KPI is carried in the model performance detection reply message, and the model performance information is reported to the base station a. The fourth indication information is optional parameters, and can be implicit indication information, for example, the indication information carrying the cause value indicates rejection, the indication information not carrying the cause value indicates acceptance, or the indication information is explicit indication information, and the acceptance or rejection of performance monitoring of the target model is explicitly characterized, wherein the cause value is optional parameters, and comprises one of network congestion, overload, unsatisfied requirements of model performance and monitoring.

In step 506, in the mobile originated location information request (Mobile Original Location Request, MO-LR), the mobile terminated location request (Mobile Terminated Location Request, MT-LR), the delayed (deferred) MT-LR, etc., the LMF determines that the UE needs to be located, selects a location method, and determines the base station participating in the location.

In this step, the LMF may locate the UE by interacting with the access and mobility management function (ACCESS AND Mobility Management Function, AMF) and the gateway's mobile location center (Gateway Mobile Location Centre, GMLC).

It should be noted that the participating positioning base station may not be the aforementioned base station a, but may be another base station. The illustration in the subsequent steps is not differentiated for simplicity and does not mean that the RAN here always has only one base station.

In step 507, the lmf sends a positioning message (enhanced or newly added NRPPa message) to one or more base stations B (not limited to base station a, where base station B may be base station a), requesting one or more base stations B to report measurement information of the UE, where the positioning message carries at least one of the following:

Model identification information, e.g., model ID, analytics ID;

The first indication information, i.e. AI positioning indication, characterizes AI measurement information which needs to be reported by the base station and is deduced by using a target model;

It should be noted that, in the related art, the LMF sends NRPPa a positioning message to one or more base stations B, and the one or more base stations B returns conventional measurement information (i.e. non-AI measurement information) to the LMF, and in the embodiment of the present application, the base station is required to report AI measurement information through the second indication information.

In step 508, one or more base stations B obtain measurement information (i.e., the actual measurement information described above) and AI measurement information for conventional positioning.

In step 509, the one or more base stations B reply to the LMF with a measurement result, where at least one of the following may be carried in the measurement result:

Third indication information, AI indication, indicating that this measurement carries AI measurement information inferred using the target model;

Model identification information, e.g., model ID, analytics ID;

AI measurement information characterizing AI measurement information inferred using the target model;

AI measurement and conventional measurement difference information, which characterizes the difference information between AI measurement information and conventional measurement information, may be a matrix information;

conventional measurement information (i.e. the above-mentioned real measurement information);

Time stamp.

In the case of carrying the third indication, one of the two parameters of the AI measurement information, and the conventional measurement information difference information may be carried.

In step 510, the lmf obtains UE location information from the UE, that is, UE location estimation calculated as a label location result or as a non-AI location, or performs one-time location on the UE using a location method different from the location method selected in S506, to obtain a conventional location result of the UE.

This step is an optional step.

One implementation way is that S506 selects to use an uplink positioning method, and in this step, a new positioning process is initiated to the UE by using a downlink positioning method, so as to require the UE to report the UE location. It should be noted that, in order for the LMF to compare the positioning estimates (AI positioning and conventional positioning) of the two positioning, the UE may carry a timestamp when reporting the positioning estimate, and the LMF binds according to the timestamp, or the LMF allocates the same task identification information (e.g. LCS correlation ID) for the two positioning.

And 511, performing performance comparison by the LMF, and calculating performance KPI information such as positioning accuracy, model accuracy and the like.

In a specific application, the positioning accuracy and model accuracy may be calculated by selecting one of the following methods:

(1) Comparing the positioning estimation, calculating the AI positioning estimation by using AI measurement, and comparing with the conventional positioning estimation obtained in the step 510, wherein the comparison is the AI position estimation and the label positioning result;

(2) Indirect comparison 1, comparing measurement information, namely directly comparing AI measurement information with conventional measurement information;

(3) And 2, directly judging the information of the AI measurement and the conventional measurement difference, and determining the performance of the model by judging whether the measurement difference meets a threshold value or not by the LMF.

Step 512, LMF replies a model performance detection reply or notification message to base station a.

Wherein the model performance detection reply or notification message may carry at least one of:

model identification information, model ID;

Performance KPI, performance parameters characterizing model reasoning, including positioning accuracy information, model accuracy information, etc.

Optionally, when the LMF local reporting policy or the condition of the first threshold in step 503 is met, the LMF returns the above-mentioned model performance detection reply or notification message to the base station a, and sends the model performance information to the base station a.

In step 513, the base station a sends the model performance monitoring information to the model trainer (LMF or NWDAF), including the information in step 512 above.

This step is an optional step.

Fig. 6 is a schematic flow chart of a method for obtaining performance information of a positioning model according to an embodiment of the present application, which is different from fig. 6 in that the method is triggered by NWDAF to perform model performance monitoring. As shown in fig. 6, the method mainly comprises the following steps:

step 601, nwdaf decides to perform performance monitoring on the target AI-positioning model.

For example, NWDAF may generate a target AI-positioning model in a training process, issue the target AI-positioning model to at least one base station a for use, and then, NWDAF decides to monitor performance of the target AI-positioning model.

In step 602, the nwdaf sends a model performance monitoring request or subscription message to the LMF, and it should be noted that this step may occur through a direct interface between NWDAF and the LMF, or may be forwarded through the GMLC, and the following step 611 is similar.

Step 603-step 610, similar to the above steps 504-511, the lmf performs performance monitoring of the target AI location model, and calculates performance KPI information such as location accuracy and model accuracy.

In step 611, the lmf sends a reply model performance detection reply or notification message to NWDAF.

model identification information, model ID;

According to the technical scheme provided by the embodiment of the application, the problem that the accuracy and precision of the estimation result obtained by carrying out position estimation on the AI measurement information obtained by reasoning the AI positioning model cannot be determined due to unknown performance of the AI positioning model is solved, so that the model provider can optimize the AI positioning model based on the monitoring result, and the positioning precision of the AI positioning model is improved.

According to the method for acquiring the performance information of the positioning model provided by the embodiment of the application, the execution subject can be an acquisition device of the performance information of the positioning model. In the embodiment of the present application, an example of a method for acquiring performance information of a positioning model by using an acquiring device for performance information of a positioning model is described.

Fig. 7 is a schematic structural diagram of an apparatus for acquiring performance information of a positioning model according to an embodiment of the present application, and as shown in fig. 7, the apparatus 700 includes a first sending module 701, a first receiving module 702, and a first acquiring module 703.

In the embodiment of the present application, a first sending module 701 is configured to send a positioning message to at least one first access point first access network device, where the positioning message is used to request to report one of AI measurement information, AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information that are generated by using the target positioning model for reasoning, and a first receiving module 702 is configured to receive a positioning result message sent by the at least one first access network device, where the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the AI measurement information, the AI measurement information and the real measurement information, and difference information between the AI measurement information and the real measurement information, and a first obtaining module 703 is configured to obtain first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

In an alternative implementation, the positioning message carries at least one of the following:

Identification information of the target positioning model;

the first indication information is used for indicating the first access network equipment to locate by using the AI measurement information inferred by the target location model;

And the second indication information is used for indicating that the first network element is to monitor the performance of the target positioning model.

In an alternative implementation manner, the positioning result message further carries the following information:

Identification information of the target positioning model;

And the third indication information is used for indicating that the target positioning measurement information is carried in the positioning result message.

In an alternative implementation, the positioning result message further carries first timestamp information, which is used to indicate a time when the AI measurement information is inferred using the target positioning model.

In an alternative implementation, based on the target positioning measurement information, first positioning performance monitoring information of the target positioning model is obtained, including one of:

comparing the difference between the AI measurement information and the real measurement information under the condition that the target positioning measurement information comprises the AI measurement information and the real measurement information, and acquiring first positioning performance monitoring information of the target positioning model based on the difference between the AI measurement information and the real measurement information;

acquiring first positioning performance monitoring information of the target positioning model based on the difference information under the condition that the target positioning measurement information comprises the difference information;

And acquiring terminal position information under the condition that the target positioning measurement information comprises the AI measurement information, wherein the terminal position information comprises terminal position estimation and second timestamp information, acquiring model position estimation based on the AI measurement information, and acquiring first positioning performance monitoring information of the target positioning model based on the difference between the terminal position estimation and the model position estimation.

In an optional implementation manner, the first receiving module 702 is further configured to receive a positioning performance monitoring message sent by a target device, where the positioning performance monitoring message is used to request performance measurement on the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model, and the target device includes one of the first access network device, the second access network device, and the second network element.

In an optional implementation manner, the positioning performance monitoring message further carries at least one of the following:

Identification information of a target device of the second network element;

Monitoring time period information for indicating time for performance monitoring of the target positioning model;

And the first threshold is used for indicating to notify the target equipment when the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold.

In an alternative implementation manner, the identification information of the target device includes one of an identification ID of the first access network device or the second access network device, and an OAM ID corresponding to the first access network device or the second access network device.

In an optional implementation manner, the first sending module 701 is further configured to send a positioning performance monitoring response message to the target device, where the positioning performance monitoring response message carries identification information of the target positioning model.

In an optional implementation manner, in a case that second positioning performance monitoring information of the target positioning model is known and the second positioning performance monitoring information obtained meets a second threshold, the positioning performance monitoring response message further carries a performance index corresponding to the second positioning performance monitoring information.

In an optional implementation manner, the positioning performance monitoring response message further carries at least one of the following:

Four indicating information for indicating one of accepting performance monitoring of the target positioning model and rejecting performance monitoring of the target positioning model, the four indicating information including

And the reason value is used for indicating the reason for refusing to monitor the performance of the target positioning model.

In an alternative implementation, the reasons for refusing performance monitoring of the target positioning model include at least one of:

Network congestion;

The load of the first network element is greater than a threshold;

The model performance of the target positioning model does not meet the requirements;

Positioning performance monitoring is being performed on the target positioning model.

In an alternative implementation, the positioning performance monitoring message includes one of:

a positioning performance monitoring request message, configured to request to acquire positioning performance monitoring data of the target positioning model once;

and the positioning performance monitoring subscription message is used for requesting to acquire the positioning performance monitoring data of the target positioning model at least once.

In an optional implementation manner, the first sending module 701 is further configured to send a positioning performance monitoring notification message to the target device, where the positioning performance monitoring notification message carries identification information of the target positioning model and a performance index corresponding to the first positioning performance monitoring information.

In an alternative implementation, the performance metrics include at least one of:

Positioning accuracy information;

Model accuracy information.

The device for acquiring the performance information of the positioning model provided by the embodiment of the application can realize each process realized by the embodiment of the method of fig. 3 and achieve the same technical effect, and in order to avoid repetition, the description is omitted here.

Fig. 8 shows a schematic structural diagram of a positioning monitoring information sending device of a positioning model according to an embodiment of the present application, and as shown in fig. 8, the device 800 includes a second receiving module 801, a second obtaining module 802, and a second sending module 803.

In the embodiment of the application, a second receiving module 801 is configured to receive a positioning message sent by a first network element, where the positioning message is used to request to report one of AI measurement information, AI measurement information and real measurement information generated by using the target positioning model for reasoning, and difference information between the AI measurement information and the real measurement information, a second obtaining module 802 is configured to obtain the real measurement information and AI measurement information inferred by using the target positioning model, and a second sending module 803 is configured to send a positioning result message to the first network element based on the real measurement information and the AI measurement information, where the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the AI measurement information, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information.

Identification information of the target positioning model;

The positioning monitoring information sending device of the positioning model provided by the embodiment of the application can realize each process realized by the method embodiment of fig. 4 and achieve the same technical effect, and in order to avoid repetition, the description is omitted here.

As shown in fig. 9, the embodiment of the present application further provides a communication device 900, which includes a processor 901 and a memory 902, where the memory 902 stores a program or an instruction that can be executed on the processor 901, for example, when the communication device 900 is a first network element, the program or the instruction is executed by the processor 901 to implement each step of the above-mentioned method embodiment for obtaining performance information of a positioning model, and the same technical effects can be achieved. When the communication device 900 is an access network device, the program or the instruction, when executed by the processor 901, implements the steps of the positioning monitoring information sending method embodiment of the positioning model, and can achieve the same technical effects, so that repetition is avoided, and no further description is given here.

The embodiment of the application also provides network side equipment, which comprises a processor and a communication interface, wherein the communication interface is coupled with the processor, and the processor is used for running programs or instructions to realize the steps of the method embodiment shown in fig. 3 or 4. The network side device embodiment corresponds to the network side device method embodiment, and each implementation process and implementation manner of the method embodiment can be applied to the network side device embodiment, and the same technical effects can be achieved.

Specifically, the embodiment of the application also provides network side equipment. As shown in fig. 10, the network-side apparatus 1000 includes an antenna 1001, a radio frequency device 1002, a baseband device 1003, a processor 1004, and a memory 1005. The antenna 1001 is connected to a radio frequency device 1002. In the uplink direction, the radio frequency device 1002 receives information via the antenna 1001, and transmits the received information to the baseband device 1003 for processing. In the downlink direction, the baseband device 1003 processes information to be transmitted, and transmits the processed information to the radio frequency device 1002, and the radio frequency device 1002 processes the received information and transmits the processed information through the antenna 1001.

The method performed by the network side device in the above embodiment may be implemented in a baseband apparatus 1003, where the baseband apparatus 1003 includes a baseband processor.

The baseband apparatus 1003 may, for example, include at least one baseband board, where a plurality of chips are disposed on the baseband board, as shown in fig. 10, where one chip, for example, a baseband processor, is connected to the memory 1005 through a bus interface, so as to call a program in the memory 1005 to perform the network device operation shown in the above method embodiment.

The network-side device may also include a network interface 1006, such as a common public radio interface (Common Public Radio Interface, CPRI).

Specifically, the network side device 1000 of the embodiment of the present application further includes instructions or programs stored in the memory 1005 and capable of running on the processor 1004, and the processor 1004 calls the instructions or programs in the memory 1005 to execute the method executed by each module shown in fig. 8, and achieves the same technical effects, so that repetition is avoided and therefore, the description is omitted herein.

Specifically, the embodiment of the application also provides network side equipment. As shown in fig. 11, the network-side device 1100 includes a processor 1101, a network interface 1102, and a memory 1103. The network interface 1102 is, for example, a common public radio interface (common public radio interface, CPRI).

Specifically, the network side device 1100 of the embodiment of the present application further includes instructions or programs stored in the memory 1103 and capable of running on the processor 1101, and the processor 1101 calls the instructions or programs in the memory 1103 to execute the method executed by each module shown in fig. 7, so as to achieve the same technical effect, and thus, for avoiding repetition, the description is omitted herein.

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, where the program or the instruction, when executed by a processor, implements each process of the above-mentioned positioning model performance information obtaining method embodiment, or implements each process of the above-mentioned positioning model positioning monitoring information sending method embodiment, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.

Wherein the processor is a processor in the terminal described in the above embodiment. The readable storage medium includes computer readable storage medium such as computer readable memory ROM, random access memory RAM, magnetic or optical disk, etc. In some examples, the readable storage medium may be a non-transitory readable storage medium.

The embodiment of the application further provides a chip, the chip comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or instructions, each process of the positioning model performance information obtaining method embodiment is realized, or each process of the positioning model positioning monitoring information sending method embodiment is realized, the same technical effect can be achieved, and the repetition is avoided, and the description is omitted here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, or the like.

The embodiment of the present application further provides a computer program/program product, where the computer program/program product is stored in a storage medium, and the computer program/program product is executed by at least one processor to perform each process of the above-described embodiment of the method for obtaining performance information of a positioning model, or implement each process of the above-described embodiment of the method for sending positioning monitoring information of a positioning model, and achieve the same technical effects, so that repetition is avoided and no further description is given here.

The embodiment of the application also provides a communication system, which comprises a first network element and first access network equipment, wherein the first network element can be used for executing the steps of the method for acquiring the performance information of the positioning model, and the first access network equipment can be used for executing the steps of the method for transmitting the positioning monitoring information of the positioning model.

In an embodiment, the communication system further comprises a target device for performing the steps performed by the target device in the method for obtaining positioning model performance information as described above.

In one embodiment, the target device comprises one of the first access network device, the second access network device and the second network element.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the description of the embodiments above, it will be apparent to those skilled in the art that the above-described example methods may be implemented by means of a computer software product plus a necessary general purpose hardware platform, but may also be implemented by hardware. The computer software product is stored on a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes instructions for causing a terminal or network side device to perform the methods according to the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms of embodiments may be made by those of ordinary skill in the art without departing from the spirit of the application and the scope of the claims, which fall within the protection of the present application.

Claims

1. A method for obtaining positioning model performance information, characterized by comprising:

The first network element sends a positioning message to at least one first access network device, wherein the positioning message is used to request reporting of one of the following: artificial intelligence AI measurement information generated by reasoning using the target positioning model, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information;

The first network element receives a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI measurement information, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information;

The first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

2. The method according to claim 1, wherein the positioning message carries at least one of the following:

identification information of the target positioning model;

The first indication information is used to instruct the first access network device to use the AI measurement information inferred by the target positioning model to perform positioning;

The second indication information is used to indicate that the first network element will perform performance monitoring on the target positioning model.

3. The method according to claim 1 or 2, characterized in that the positioning result message also carries the following information:

identification information of the target positioning model;

The third indication information is used to indicate that the positioning result message carries the target positioning measurement information.

4. The method according to any one of claims 1 to 3 is characterized in that the positioning result message also carries a first timestamp information for indicating the time when the AI measurement information is inferred using the target positioning model.

5. The method according to any one of claims 1 to 4, characterized in that the first network element obtains the first positioning performance monitoring information of the target positioning model based on the target positioning measurement information, comprising one of the following:

In a case where the target positioning measurement information includes the AI measurement information and the real measurement information, the first network element compares a difference between the AI measurement information and the real measurement information, and acquires first positioning performance monitoring information of the target positioning model based on the difference between the AI measurement information and the real measurement information;

In a case where the target positioning measurement information includes the difference information, the first network element acquires first positioning performance monitoring information of the target positioning model based on the difference information;

In the case where the target positioning measurement information includes the AI measurement information, the first network element obtains terminal location information, wherein the terminal location information includes: a terminal location estimate and second timestamp information; based on the AI measurement information, a model location estimate is obtained; based on a difference between the terminal location estimate and the model location estimate, first positioning performance monitoring information of the target positioning model is obtained.

6. The method according to any one of claims 1 to 5, characterized in that before the first network element sends a positioning message to at least one first access network device, the method further comprises:

The first network element receives a positioning performance monitoring message sent by a target device, wherein the positioning performance monitoring message is used to request performance measurement of the target positioning model, and the positioning performance monitoring message carries identification information of the target positioning model; the target device includes one of the following: the first access network device, the second access network device, and the second network element.

7. The method according to claim 6, characterized in that the positioning performance monitoring message also carries at least one of the following:

identification information of the target device;

Monitoring time period information, used to indicate the time for performance monitoring of the target positioning model;

The first threshold is used to indicate that when the performance parameter corresponding to the first positioning performance monitoring information reaches the first threshold, the target device is notified.

8. The method according to claim 7 is characterized in that the identification information of the target device includes one of the following: an identification ID of the first access network device or the second access network device, and an operation management and maintenance OAM ID corresponding to the first access network device or the second access network device.

9. The method according to claim 7, characterized in that after the first network element receives the positioning performance monitoring message sent by the target device, the method further comprises:

The first network element sends a positioning performance monitoring response message to the target device, wherein the positioning performance monitoring response message carries identification information of the target positioning model.

10. The method according to claim 9 is characterized in that, when the first network element has obtained the second positioning performance monitoring information of the target positioning model, and the performance indicator corresponding to the obtained second positioning performance monitoring information meets the second threshold, the positioning performance monitoring response message also carries the performance indicator corresponding to the second positioning performance monitoring information.

11. The method according to claim 9 or 10, characterized in that the positioning performance monitoring response message also carries at least one of the following:

The fourth indication information is used to indicate one of the following: accepting the performance monitoring of the target positioning model or refusing the performance monitoring of the target positioning model;

A reason value is used to indicate the reason for rejecting the performance monitoring of the target positioning model.

12. The method according to claim 11, wherein the reason for refusing to perform performance monitoring on the target positioning model comprises at least one of the following:

Network congestion;

The load of the first network element is greater than a threshold;

The positioning performance of the target positioning model is being monitored.

13. The method according to any one of claims 6 to 12, wherein the positioning performance monitoring message comprises one of the following:

A positioning performance monitoring request message, used to request to obtain positioning performance monitoring data of the target positioning model once;

The positioning performance monitoring signing message is used to request to obtain the positioning performance monitoring data of the target positioning model at least once.

14. The method according to any one of claims 6 to 12, characterized in that after the first network element obtains first positioning performance monitoring information of the target positioning model based on the target positioning information, the method further comprises:

The first network element sends a positioning performance monitoring notification message to the target device, wherein the positioning performance monitoring notification message carries identification information of the target positioning model and a performance indicator corresponding to the first positioning performance monitoring information.

15. The method according to claim 10 or 14, characterized in that the performance indicator comprises at least one of the following:

Positioning accuracy information;

Model accuracy information.

16. A method for sending positioning monitoring information of a positioning model, characterized by comprising:

The first access network device receives a positioning message sent by the first network element, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information;

The first access network device obtains real measurement information and AI measurement information inferred using the target positioning model;

The first access network device sends a positioning result message to the first network element based on the real measurement information and the AI measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI measurement information, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information.

17. The method according to claim 16, wherein the positioning message carries at least one of the following:

identification information of the target positioning model;

18. The method according to claim 17, wherein the positioning result message carries the following information:

identification information of the target positioning model;

19. The method according to any one of claims 16 to 18 is characterized in that the positioning result message also carries a first timestamp information for indicating the time when the AI measurement information is inferred using the target positioning model.

20. A device for acquiring positioning model performance information, characterized by comprising:

A first sending module is configured to send a positioning message to at least one first access point and a first access network device, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information;

A first receiving module is configured to receive a positioning result message sent by the at least one first access network device, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI measurement information, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information;

The first acquisition module is used to acquire first positioning performance monitoring information of the target positioning model based on the target positioning measurement information.

21. A positioning monitoring information sending device for a positioning model, characterized by comprising:

A second receiving module is configured to receive a positioning message sent by the first network element, wherein the positioning message is used to request reporting of one of the following: AI measurement information generated by reasoning using the target positioning model, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information;

A second acquisition module is used to acquire real measurement information and AI measurement information inferred using the target positioning model;

A second sending module is used to send a positioning result message to the first network element based on the real measurement information and the AI measurement information, wherein the positioning result message carries target positioning measurement information, and the target positioning measurement information includes one of the following: the AI measurement information, the AI measurement information and real measurement information, and difference information between the AI measurement information and the real measurement information.

22. A network side device, characterized in that it comprises a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method according to any one of claims 1 to 19 are implemented.

23. A readable storage medium, characterized in that the readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 19 are implemented.