GB2577861A

GB2577861A - Measuring a network performance characteristic

Info

Publication number: GB2577861A
Application number: GB1814218.2A
Authority: GB
Inventors: Dominic Shaw Ryan; Robert Blake Andrew
Original assignee: Spatialbuzz Ltd
Current assignee: Spatialbuzz Ltd
Priority date: 2018-08-31
Filing date: 2018-08-31
Publication date: 2020-04-15
Anticipated expiration: 2038-08-31
Also published as: GB2577861B; GB201814218D0

Abstract

A method of operating a user equipment (UE) configured for use with a communications network is provided. The method comprises measuring a network performance characteristic associated with a measurement period, and determining whether a predetermined change in a reporting characteristic has occurred. If the predetermined change has occurred, the network performance characteristic is reported to the communications network. The characteristic may be a distance moved by the UE, RSSI, data rate or throughput or change of state of an application.

Description

MEASURING A NETWORK PERFORMACE CHARACTERISTIC

Field of the invention

This invention relates to measuring performance characteristics of a network used by a computing device such as a battery-powered device such as a smart phone.

Background

Commonly, computing devices are capable of wireless communication with networks. It is useful to analyse such wireless interactions, typically for the purpose of adjusting or repairing the network that supports the interactions, in order to enhance the quality of the wireless communications enabled by the network. It is also useful to associate with an analysis of a wireless interaction between a computing device and a network an assessment of the status of the computing device, in order to give context to the analysis. For example, when the strength of a radio signal reaching a computing device from a network is measured, it is valuable to have information about the status of the computing device, such as its geographical location (so that the measurement can be associated with the part of the network serving that location), whether it is indoors or outdoors (which will have an impact on the how the measurement is interpreted) and, in the case where the computing device is mobile, whether or not the device is moving (which again will have an impact on how the measurement is interpreted). A common scenario for such analysis is the wireless interactions between a smart phone and cellular radio communications network for the purpose of allowing the network operator to enhance the performance of the network for the benefit of the user of the smart phone and its other customers besides.

The reporting of such analysis of wireless interaction typically requires energy consumption on the part of the computing device. It is generally desirable to reduce energy consumption associated with such reporting, particularly in the case where the computing device is a battery-powered mobile device such as a smart phone.

Summary of the invention

According to a first aspect of the invention there is provided a method of operating a user equipment, UE, configured for use with a communications network, the method comprising: measuring a network performance characteristic; determining whether a predetermined change in a reporting characteristic has occurred; and if so, reporting the network performance characteristic to the communications network.

In conventional systems, network performance characteristics are typically reported to the network at predetermined time intervals. For relatively stable network connections, these intervals may be too short, meaning that more reports than necessary are sent, each of which requires energy to send. On the other hand, in situations where the network performance characteristic varies more often than reports are sent to the network, useful information about the performance of the network may be lost or reported too late to the network.

In contrast, in the present invention, the network performance characteristic is sent when a predetermined change in a reporting characteristic occurs. This ensures that useful information is sent in a timely manner to the network, without sending unnecessary reports. Reducing the number of unnecessary reports saves energy, which is especially useful when the device implementing the method is battery-powered.

The reporting characteristic may be a characteristic of the UE, the network, or the connection between the UE and the network. In some embodiments the network performance characteristic may only be reported to the communications network when it is determined that a predetermined change in the reporting characteristic has occurred. In alternative embodiments, the network performance characteristic may additionally be reported at predetermined time intervals. The predetermined time intervals may for example be less frequent than in conventional methods, as the reporting characteristic trigger ensures reports are sent to the network when useful.

In some embodiments, the predetermined change may comprise a change in a value of a reporting characteristic greater than a predetermined amount.

In some such embodiments, the reporting characteristic may be a distance moved by the UE. In embodiments where the communications network is a cellular communications network, the reporting characteristic may be a distance moved by the UE relative to a cell radius of a cell to which the UE is connected. For example, the predetermined change may comprise a movement of the UE greater than 10% of the cell radius, or the predetermined change may comprise a distance moved by the UE greater than a value selected from the range 5%-15% of the cell radius. It may be advantageous to use the distance moved relative to the cell radius, rather than absolute distance, for the reporting characteristic as cells with smaller radii are typically placed in dense usage environments (e.g. urban centres), where small changes in distance have a greater impact on network performance than in less dense, larger cell radii, areas.

In some embodiments, the reporting characteristic may be a received signal strength indication. The predetermined change may be a sustained increase or decrease of at least 10 dB, or of at least a value selected from the range 5 dB to 15 dB. As used herein, a sustained change may be a change which is sustained over a predetermined time period. The predetermined time period may for example be 10 seconds, 30 seconds, 1 minute, or 5 minutes, or the time period may be a value selected from the range 1 minute to 30 minutes.

In some embodiments, the reporting characteristic may be a data rate or data throughput. The predetermined change may for example be a factor of two (i.e. a halving or a doubling) or more change in the data rate/data throughput, or a factor of three or more, or factor of five or more.

In some embodiments, the predetermined change may comprise a change of state of an application stored on the UE.

In some such embodiments, the change of state may comprise an application running on the UE closing, or the change of state may comprise a predetermined application closing. Determining the predetermined change in a reporting characteristic may comprise determining that a predetermined application has been recently active on the UE. Determining that an application has closed or has been recently active may comprise using historical data such as a list of recently closed applications, as set out in more detail below.

In some embodiments, the predetermined change may comprise a change of state of a network connection.

In some such embodiments, the change of state may comprise a change of network connection type. For example the change of network connection may be between a 3G cellular connection and a 4G cellular connection. The change of network connection may be between a cellular connection and a wireless local area network, WLAN, connection.

In some embodiments, the predetermined change may comprise a predetermined user interaction. The predetermined user interaction may comprise a user performing a test of a network connection. For example, this may indicate that the user considers the network performance to be unsatisfactory, indicating a potential issue with the network connection. Reporting the network performance characteristic in response to the user running such a test may enable a network operator to diagnose and correct any problems on the network.

In some embodiments, the network performance characteristic may comprise the reporting characteristic. For example, if the reporting characteristic is a distance moved by the UE, the distance moved may be reported as the network performance characteristic. Alternatively, the network performance characteristic may be different to the reporting characteristic. For example, if the reporting characteristic is distance moved by the UE, a data rate may be reported as the network performance characteristic.

In some embodiments, a plurality of network performance characteristics may be measured and reported when the predetermined change in reporting characteristic is determined.

In some embodiments, the method may comprise determining whether a respective predetermined change has occurred for a plurality of reporting characteristics. The step of reporting the network performance characteristic may be performed if the respective predetermined change has occurred for any of the plurality of reporting characteristics. In alternative embodiments, the step of reporting the network performance characteristics may only be performed if the respective predetermined change has occurred for each of the plurality of reporting characteristics.

In some embodiments, the measured network performance characteristic may be associated with a measurement period. For example, the measured (and subsequently reported) network performance characteristic may be an average of the network performance characteristic over the measurement period. The predetermined change in the reporting characteristic may occur during the measurement period.

In some embodiments, the network performance characteristic may comprise at least one of: average transmitted data rate, average received data rate, peak transmitted data rate, peak received data rate, received signal strength, packet latency, and packet jitter.

In some embodiments wherein the network performance characteristic is associated with a measurement period, the method may further comprise: identifying, using historical data, an application on the UE responsible for transmitting data to, and/or receiving data from, the communication network in the measurement period; and associating the identity of the identified application with the network performance characteristic.

In conventional methods, network performance characteristics such as data rate or throughput are reported to the network without an indication of how a UE has been using the network. Different applications executed on the UE use the network in different ways. For example, some applications may transmit or receive large amounts of data (e.g. video player applications); whereas other applications may be light data users (e.g. messaging applications). It has been realised that an understanding of which application/s have been making use of the network allows network performance characteristics to be put in context, and so increases the usefulness and accuracy of measured network performance characteristics.

As one example, if multiple applications are installed on a UE and the user (or the UE itself) switches between these applications, the data rate measured could fluctuate significantly (due to the differing data demands of particular applications). In conventional reporting methods, this fluctuation could be misinterpreted as a poor quality of service or a network problem and not as a natural consequence of user (or device) behaviour. The present method, however, provides the ability to discern real network performance and separate this from actual user or application behaviour.

As another example, the customer satisfaction with the communications network is likely to depend upon the application/s on the UE actually being used be a user. A user may be satisfied by a comparatively low data rate if a light data application (e.g. a messaging application) is being used, but may be dissatisfied by the same data rate if a heavy data application is being used (e.g. a video player application). The present invention therefore allows a network provider to better understand the experience of their customers.

The user equipment may for example be a mobile device such as a mobile telephone. The communications network may be a cellular communications network or a WiFig network. The method may be performed on the UE (i.e. the method may be a method of characterising the performance of a communications network by a UE), or may be performed by a combination of the UE and the network. For example, the step of measuring the network performance characteristic and the step of associating the identity of the identified application with the network performance characteristic may be performed either by the UE or by the network.

Using historical data allows the method to identify applications that may be responsible for data usage, without requiring access to currently running application information, which may not be available. Historical data is data recorded before the step of identifying the application is performed. For example, the historical data may be stored in a memory of the UE.

The step of identifying an application may comprise identifying an application likely (or most likely) to be responsible for transmitting data to, and/or receiving data from, the communication network in the measurement period.

In some embodiments, identifying the application may comprise identifying a recently active application, for example from a list of recently active applications. In this case, the list of recently active applications forms the historical data. A recently active application may be an application which was active at any point between the beginning of the measurement period and the present time, or between the beginning of the measurement period and the end of the measurement period. The recently active application may be a currently active application, or an application which has been closed. An active application may be one which has been in the foreground of a user interface of the UE.

It may not be possible to determine for certain which application/s was responsible for data use during the measurement period. However, a recently active application may be the most likely candidate for using data in the measurement period, and so provides a best guess of the actual application. If there are multiple recently active applications, the most recently active application may be taken to be the identified application; or it may be determined which application from a plurality of recently active applications is most likely to have used data (or a majority of the data) in the measurement period.

In some embodiments, identifying the recently active application may comprise identifying a recently closed application from a list of recently closed applications. In particular, identifying the application may comprise identifying an application from the list of recently closed applications which closed during or after the measurement period. In this case, the list of recently closed applications forms the historical data.

The UE operating systems (e.g. Android @) may limit access to information about which application/s are currently running on the UE. However, it may be possible to access a list of applications which have recently closed. By identifying a recently closed application, the method may provide a 'best guess' of the application/s responsible for data use in the measurement period. Thus, for example, it may be possible to ascertain that, on the balance of probabilities, a specific application was running and responsible for the last (or a prior) N packets transmitted and/or received from or by the communications network, during (or over) which a certain set of network performance characteristics measurements were made. It is consequently then possible to be able to directly associate these measurement reports and their associated measurement data with an application which was, in all probability, running in the foreground at the time at which the measurements were made.

In some embodiments, identifying the application may comprise identifying the most recently closed application.

In some embodiments, identifying the application may comprise: determining a data usage level for the measurement period; and identifying a recently active application having a data usage profile corresponding to the data usage level.

The data usage level for the measurement period may be an indication of the amount of data used (transmitted and/or received) by the UE in the measurement period. The data usage level may be a level selected from a data usage classification scheme; for example the data usage level may be classified as 'heavy data use' if an amount of data over a predetermined threshold has be used in the measurement period, or as 'light data use' if the amount of data used is less than the or a second predetermined threshold.

The data usage profile of an application may be an indication of the likely amount of data used (transmitted and/or received) by an application in a given period. The data usage profile may be based on an expected data usage from previous uses of the application, either on the UE or externally. The data usage profile may be a level selected from a data usage profile classification scheme; for example an application may be classified as a 'heavy data user' or a 'light data user'.

Thus an application in the recently closed list may be identified as the most likely candidate for being responsible for data usage in the measurement period by matching the data usage level to the data usage profile of the recently closed applications. This allows the method to more accurately predict which application was responsible.

Furthermore, there may be multiple windows of activity within the measurement period with different data usage levels, corresponding to multiple applications being used during the measurement period. The method may comprise determining a data usage level of each window of activity. Each data usage level may then be matched to a recently closed application having a corresponding data usage profile. Thus the method may enable identification of multiple applications responsible for transmitting/receiving data, and allowing each application to be associated with the network performance characteristic for a corresponding time window of the measurement period.

In some embodiments, determining the data usage level for the measurement period may comprise determining the number of data packets transmitted/received by the UE during the measurement period.

In some embodiments, determining the data usage level for the measurement period may comprise determining the total amount of data transmitted/received by the UE during the measurement period.

In some embodiments, associating the identified application with the network performance characteristic comprises qualifying the network characteristic based on a data usage profile of the identified application. Qualifying the network characteristic may comprise modifying an action that would otherwise have been performed. For example, the step of qualifying may comprise recognising that the network performance characteristic is affected by a heavy data usage application. The network performance characteristic may then be disregarded. The step of qualifying may comprise recognising that a worse than expected network performance characteristic is acceptable for the identified application. For example, low network speeds may be acceptable for low data use applications.

Some embodiments may further comprise the step of reporting, by the UE, the network performance characteristic and identified application to the communications network.

In some embodiments, the network performance characteristic may be a data rate at which data is received from or transmitted to the communications network by the UE, and wherein the method further comprises: determining a data usage profile of the identified application; determining if the data usage profile matches a predetermined data usage profile; and if so, identifying the measured data rate as the maximum data rate available to the UE during the measurement period.

When a heavy data usage application is run, for example a video player, it is likely that the UE will be demanding the maximum data rate which the network is capable of providing at that time (based upon the amount of congestion on the network and the user's location relative to a base-station). Thus the measured data rate for measurement periods where such an application is running is the maximum speed at which the UE can receive/transmit data -in other words the measured data rate provides a broadband speed test of the network. Advantageously, this 'speed test' is provided without any additional data having to be downloaded to the UE, as the measurement uses data that would be downloaded anyway.

In some embodiments, the method further comprises: determining a data usage metric for the measurement period; and associating the one or more data usage metrics with the network performance characteristic and identified application. The data usage metric may for example comprise at least one of: an amount of data sent by the UE, an amount of data received by the UE, and network latency. The data usage metric may be determined for the whole measurement period, or may be determined for a plurality of points or windows within the measurement period. The step of associating may be performed on the UE, or may be performed by the network after receipt of the network performance characteristic from the UE.

Such embodiments may provide further context to the network performance characteristic, and additional evidence for determining the application responsible for transmitting/receiving data in the measurement period. In embodiments where the data usage metric is determined for a plurality of points/windows in the measurement periods, the data usage metric may also assist in identification of windows of activity corresponding to different applications being used during the measurement period.

In some embodiments, the measurement period may be a period between 0.5s and 2s, or between is and 10s, or between 105 and 1 minute, or between 1 minute and 10 minutes, or between 10 minutes and 1 hour, or between 1-2 hours, or between 2-24 hours, or more. The network performance characteristic may be sampled at one or more points during the measurement period. The network performance characteristic may be an average of a plurality of measurements in the measurement period, or a single point sample may be used to represent the network performance characteristic in the measurement period.

According to a second aspect of the invention there is provided a user equipment (UE) for use with a communications network, the UE comprising: a transmitter for communicating with the communications network; and a processor configured to perform the method of any embodiment of the first aspect.

According to a third aspect of the invention there is provided program code, which when executed by compatible data processing hardware, will cause the hardware to perform a method according to any embodiment of the first aspect.

The invention also relates to apparatus, program code and data carriers non-transiently storing program code for carrying out methods of the kinds described above.

Brief description of the drawings

By way of example only, certain embodiments of the invention shall now be described by reference to the accompanying drawings, in which: Figure 1 is a block diagram schematically illustrating a 4G telecommunications network and a user equipment (UE) that can interact with the network; Figure 2 is a flow chart of a process performed by the UE of Figure 1 for making measurements on its interactions with the network shown in Figure 1; Figure 3 is a flow chart of an alternative process performed by the UE of Figure 1 for making measurements on its interactions with the network shown in Figure 1; Figure 4 is a flow chart of a further alternative process performed by the UE of Figure 1 for making measurements on its interactions with the network shown in Figure 1; and Figure 5 is a flow chart of a further alternative process performed by the UE of Figure 1 for making measurements on its interactions with the network shown in Figure 1.

Detailed description

Figure 1 illustrates a user equipment (UE) 100 that is a mobile subscriber device -such as a smartphone, for example -that is operating in a 4G cellular telecommunications network 110. The UE 100 comprises a processor 112, a memory 114, a GPS receiver 116, a 4G transceiver 118, a WiFigi transceiver 120 and three antennas 122-126. In practice, of course, the UE comprises many other elements besides, though description of these elements is not necessary to understand the invention as embodied by the devices and processes that are described by reference to the drawings. As shown, the 4G cellular network 110 comprises three base stations 128, 130 and 132 that are interconnected by a backhaul infrastructure 134. Like the UE 100, in practice, the network 110 comprises many other elements besides those shown, though description of these elements -further base stations, for example -is not necessary to understand the invention as embodied by the devices and processes that are described by reference to the drawings.

The processor 112 has overall control of the UE 100 and executes instructions, with the aid of the storage provided by memory 114 to which the processor 112 is operatively coupled, in order to perform such tasks as may be required of the UE 100 by its user and by the operator of the network 110. The processor 112 is operatively coupled to the GPS receiver 116, the 4G transceiver 118, and the WiFi0 transceiver 120 in order to control and operate those subsystems. In order to perform their respective functions, the GPS receiver 116, the 4G transceiver 118, and the WiFii® transceiver 120 are operatively coupled to antennas 122-126 respectively.

The GPS receiver 116 receives signals from overhead GPS satellites (not shown), mathematically estimates the location of the UE 100 from those signals, and reports the estimated location to the processor 112.

The 4G transceiver 118 both transmits radio communications to, and receives radio communications from, the 4G cellular communications network 110. The signals transmitted by the 4G transceiver 118 may contain data generated by the user of the UE 100 and also signalling data -control information, measurement information and the like -that is utilised by the 4G network 110 for the purpose of enabling the network 110 to at least adequately conduct radio communications with its users' UEs. Similarly, the signals received by the 4G transceiver 118 may contain data intended for user consumption -voice-call data or video data, for example -as well as signalling data for enabling the UE 100 to interact efficiently with the 4G network 110.

The WiFi@ transceiver 120 both transmits radio communications to, and receives radio communications from, WiFi® access points (not shown) in the vicinity of the UE 100. The signals transmitted by the WiFi@ transceiver 120 may contain data generated by the user of the UE 100 and also signalling data that enables or facilitates communication between the WiFig transceiver 120 and one or more WiFi0 access points. Similarly, the signals received by the WiFi@ transceiver 120 may contain data intended for user consumption -voice-call data or video data, for example -as well as signalling data that enables or facilitates communication between the WiFiC) transceiver 120 and one or more WiFiC) access points.

The UE 100 is powered by a battery (not shown) and hence the processor 112 is configured to manage the UE's power consumption. The UE 100 has a selection of operating modes, each with a different power consumption. The processor 112 stops and starts -or, as it is commonly said, wakes and puts to sleep -various subsystems and processes within the UE in order to move between these modes so as to manage overall power consumption and defer the moment at which the user next deems it necessary to charge the battery. Some subsystems and processes are relatively power hungry. The GPS receiver 116, the 4G transceiver 118 and the WiFii® transceiver 120 are relatively power-hungry subsystems, and the processor 112 is arranged to deactivate them under certain conditions in order conserve battery energy. For example, the 4G transceiver 118 can be deactivated or put to sleep upon a predetermined period expiring following the last time that the 4G transceiver 124 was required for communication with the 4G network 110. Even if the user places no demands on the 4G transceiver 124 while 4G transceiver 124 is in sleep mode, the processor 112 will periodically wake the 4G transceiver 124 in order for signalling data to be sent between the UE 100 and the 4G network 110.

Similarly, the WiFi® transceiver 126 can be put to sleep, in the sense that the frequency with which it searches for WiFiC) access points in its vicinity is substantially reduced if the number of WiFiC) access points that it can detect has been zero for a predetermined threshold period of time or if the UE 100 is placed in a stand-by mode (whether by the UE 100 autonomously or by a command from its user). The operation of the GPS receiver 116 to estimate the UE's location is typically ad hoc upon user request or is on a scheduled basis dictated by software (such as an exercise app, in the case where the UE 100 is a smart phone). In order to control power consumption, scheduled use of the location estimation function can be reduced by reducing the frequency of making location estimates and ad hoc use can be temporarily disabled.

The memory 114 contains inter alia programme code for execution by the processor 112 that constitute one or more applications (apps) 138a-n. Apps 138-n are intended to be used be the user of the UE, such as a messaging app or a video player app. When executed by the processor 112, these apps may appear in the foreground of a graphical user interface of the UE, and may be responsible for transmitting and receiving user-type data (often called payload data) to and from the 4G network 110.

The memory 114 also contains programme code for execution by the processor 112 that constitutes a monitoring app 136 that, at least in part, performs measurements of network performance characteristics on the 4G network 110 for the benefit of the operator of the 4G network 110. Such measurements may, for example, be of the following types.

* The time it takes for round-trip communication from the UE 100 to a predetermined network or internet resource (not shown) via the 4G network 110 -a so-called latency measurement.

* The rate at which user-type data can be transmitted to the 4G network 110 from the UE 100.

* The rate at which user-type data can be received from the 4G network 110 by the UE 100.

* The error rate in user-type data transmitted between the UE 100 and the 4G network 110.

* RSSI of signals received from a 4G transmitter located at a cell site with which the UE is in communication.

The monitoring app 136 communicates the results of such measurements to the operator of the 4G network 110 over the 4G network 110 via the 4G transceiver 118. It is useful for the network operator to know the location in its network where a measurement was made, so that the operator can understand how different areas of its network are performing, and diagnose and remedy any underperformance. Therefore, the monitoring app 136 is configured to transmit each measurement with an indication of the location where the measurement was made, as well as an indication of the time each measurement was made.

The results of such network performance measurements may be reported to the operator of the network 110 at predetermined time intervals, and/or when a predetermined change occurs in a reporting characteristics, as described in more detail below.

In addition, it is useful if the network operator can understand what the dominant (foreground) app was which was running when network performance measurements were taken, as this is useful diagnostic information which can help to identify the source of any problems on the network. It may be that there is no specific problem and the network operator simply wishes to understand more about how his network performs in particular locations, at particular times and when running particular applications. For example a video streaming app, such as YouTube@, will typically generate significant traffic and 'stretch' a network at times of heavy usage. Such apps may provide useful network stress testing, as described in more detail below.

Figure 2 shows a flow chart of a routine 200 that can be performed by the monitoring app 136 (or more correctly, performed by the processor 112 executing the monitoring app 136) to make the network performance characteristic measurements for the operator of the 4G network 110.

At step 201, a network performance characteristic is measured, as described above. The network performance characteristic is associated with a particular measurement period (e.g. it may comprise an average value for the measurement period, or may be a point measurement used to represent the measurement characteristic for the measurement period). The measurement may be triggered for example if a certain amount of time has elapsed since a previous measurement, or based on a particular location or change of location of the UE, as would be understood by the person skilled in the art.

At step 202, an app 138 from the apps 138a-n is identified as being the dominant app responsible for transmitting data to or receiving data from the 4G network 110 in the measurement period, using historical data. This step may comprise directly identifying an active or recently active app, or identifying a recently closed app, as described in more detail below.

At step 203, the identity of the identified app 138 is associated with the network performance characteristic for the measurement period. The operator of the 4G network 110 may qualify the network performance characteristic based on the identity of the app 138, for example modifying an action that would be taken to correct a perceived network problem based on the identity of the app 138.

Finally, at optional step 204, the network performance characteristic and identity of the identified app are reported to the operator of the 4G network 110. Alternatively, the network performance characteristic and identity of the app 138 may be stored for later analysis or communication to the operator of the 4G network 110.

In some embodiments, the application 138 is identified in step 202 by directly determining whether an app has been recently active (an active app being an app in the foreground of the GUI), for example by comparing the time an app was active with the time of the measurement period. However, such a direct approach is not always possible. Some UE operating systems, such as Android (RIM), prevent the monitoring app 136 from accessing information about which apps 138a-n are currently active. However, when an app 138a-n closes (or is no longer the foreground app), it is possible to interrogate the operating system, (e.g. interrogate the "app database," via UsageStatsManager on Android®) to find out which app it was which has just terminated. It is also possible to ascertain when (at what time) the previous foreground app closed.

Using the list of recently closed apps, and the time at which an app closed, it is reasonable to assume that for the bulk or all of the intervening measurement period, the app 138 which has just terminated was responsible for the bulk of the data sent/received over the mobile network. Thus even though direct access to the identity of active apps 138a-n may be prohibited, an informed guess may be used to identify, at step 202 of the routine 200, an app 138 responsible for the majority of data usage during the measurement period.

It is also possible to analyse the characteristics of the packets of data transmitted/received during the measurement period themselves to ascertain whether each was likely to have resulted from that particular app 138a-n. Figure 3 shows an alternative routine 300 performed by the monitoring app 136, in which such an analysis is used.

In figure 3, steps 301, 303, and 304 are identical to the corresponding steps 201, 203, and 204 of routine 200 shown in figure 2. At step 302 an app 138 responsible for data use during the measurement period is identified, similarly to step 202. However, in this embodiment identifying the app 138 comprises the further steps 305 and 306.

At step 305, a data usage level for the measurement period is determined. The data usage level classifies the data usage during the measurement period based on how many packets were sent and/or received during the measurement period, and the total number of bytes sent and/or received over the cellular (and, optionally, WiFiC) interfaces) of the UE. For example the data usage level is classified as 'heavy data use' if an amount of data over a predetermined threshold has been used in the measurement period, or as 'light data use' if the amount of data used is less than the predetermined threshold or a second predetermined threshold. Other classifications may be used.

At step 306, an app 138 having a data usage profile corresponding to the determined data usage level is selected from a list of recently closed apps. The data usage profile of an app classifies its expected data usage. For example an application may be classified as a 'heavy data user' (such as YouTubeCD) or a 'light data user' (such as a messaging application). If the data usage level during the measurement period was 'heavy data use', then a 'heavy data user' app is selected from the list of recently closed apps. If the average (typical) behaviour of the app approximately matches the actual data use during the measurement period, it is likely that the selected app was responsible for the majority of data use in the measurement period.

Once the app 138 is selected using the steps 305, 306, the process 300 continues to step 303, where the identity of the identified app 138 is associated with the network performance characteristic; and then optionally to step 304, where the collected data is transmitted to the operator of the 4G network 110.

It is possible that multiple apps were used during the measurement period. By dividing the measurement period into a plurality of windows, and performing steps 305 and 306 for each window of the measurement period, it may be possible to identify the different apps that were used. For example, a period of heavy data usage may be followed by a period of light data usage, which may be followed by another period of heavy data usage. If the list of recently closed apps comprises a heavy data user such as a video player, and a light data user such as a messaging app, it can be assumed that the user was initially watching a video on the video player, then switched to the messaging app, then switched back to watching a video. Thus by analysing the data usage level, detailed information can be gathered about how the UE was being used during the measurement period, and so provide a fuller context to the measured network characteristic.

The data usage profile of an app may be determined based on an average data use of the app, either on the UE or externally, or based on the genre of app, e.g. any video player app may be classed as a heavy data user. As well as being useful for identifying the foreground app in step 302 above, the data usage profile can also be used to yield additional information from the network performance characteristic. For example, the fact that the identified app 138 is a heavy data user may be used to estimate a likely user satisfaction with a given network speed/reliability.

As a further example, if an app is classified as a heavy data user, it may be assumed that the UE will be demanding data at the maximum rate which the network 110 is capable of providing at that time (based upon the amount of congestion on the network and the user's location relative to the base-station). A measurement of this data rate (as the network characteristic) effectively provides a 'broadband speed check' style measurement on the network 110, without incurring any additional overhead of 'wasted' data (i.e. data purely generated to measure network speed).

Figures 4 and 5 provide specific examples of how the above processes may be implemented on a UE.

Figure 4 shows a flowchart of the basic operation of a process 400 which may be performed by the monitoring app 136. The process begins at step 405 and then moves on to steps 410 and 420, in parallel. Considering, first, the path beginning with step 410: step 410 is the first step in a continuously-running loop (operational whilst the monitoring app 136 is operational). In this step, the user device is awoken from a deep-sleep (ultra-low power) state, if it was in such a state at the time, into a low-power state. The process then gathers measurement data from the device recording a network performance characteristic. Additional data, such as location data, e.g. from the UE's GPS receiver, may also be recorded. These data are recorded together with a timestamp for the time at which the data was gathered, in a database 412 stored within the memory 114 of the UE.

Examples of the network characteristics which may be measured and stored include: - Data-rate (or throughput) at which the UE is loading data from the network 110 - Data rate (or throughput) at which the UE is sending data to the network 110 - Latency and Jitter experienced by packets travelling from and to the UE.

Examples of types of additional data which may be gathered and stored are: Device data: - Device identity (e.g. media access control (MAC) address, international mobile subscriber identity (IMSI), international mobile equipment identity (IMEI) or similar identifier(s)) - Brand, model, type allocation code (TAC), memory used/free, storage used/free, CPU usage - UTC Timestamp, Local device time Connection data: - Received signal strength indication (RSSI), reference signals received power (RSRP), reference signal received quality (RSRQ), reference signal signal-to-noise ratio (RSSNR), signal-to-noise-plus-interference ratio (SINR), channel quality indicator (CQI), timing advance (TA) - Cell ID (and/or SSID, if WiFi®) of the cell (or access point) to which the device is connected - Cell IDs (and/or SSIDs) of any other cells/access points from which the device can receive signals - Frequency band, absolute radio frequency channel number (ARFCN), location area code (LAC), type allocation code (TAG), mobile country code (MCC), mobile network code (MNC) Location data: - Device location (geographic location) - Speed of movement of the device - Direction of movement - Location accuracy - Whether the UE is (or is likely to be) located indoors or outdoors The process then moves on to step 415 in which the UE returns to its 'deep sleep' state (or other prior state it was in when the current iteration of the continuous loop involving steps 410 and 415 was initiated), and waits for a predetermined period. This period could be short, for example 1-2 seconds, allowing rapid and regular sampling of the activity taking place on the UE and gathering of its associated measurement data, or it could be a longer period of minutes (e.g. 1-2 minutes, 1-5 minutes) or tens of minutes (e.g. 10-20 minutes), perhaps driven by the inclusion of GPS or other location data (without unduly impacting upon battery life), or even longer (hours, days, e.g. 1-2 hours, 1-3 hours, 5 hours, 24 hours, 1-2 days) providing an aggregated user experience. The process then loops back to re-run the loop and, in parallel, re-consider the decision of step 420, which will be discussed below.

At step 420, the process queries whether one or more measurement reports should now be sent to the network. It bases this decision upon whether a measurement criterion has been met (e.g. whether there has been a predetermined change in a reporting characteristic). In particular, it bases this decision on two stimuli: timer 417 and event 418. Timer 417 is a timer which outputs a positive stimulus or changes a logic level from logic '0' to logic '1' once a pre-set time interval has elapsed, for example 10 minutes or, in an alternative embodiment, a much shorter period such that a measurement report is sent as soon as it is generated; it then resets to zero and resumes timing another (typically identical) interval. Event stimulus 418 outputs a stimulus or changes a logic level when a given event has taken place (i.e. a predetermined change in a reporting characteristic). Examples of events any or all of which may be used are: - The UE having moved greater than a predetermined distance - The UE having experienced greater than a predetermined change in RSSI - The UE having experienced greater than a predetermined change in data rate or throughput - A particular app having been recognised as having been in use (e.g. is actively in use or is identified in the list of recently closed apps) - The UE having moved into or within a place of interest defined as a polygon boundary around a location - The network connection having changed e.g. a handover to another cell, the change of radio bearer technology from 4G to 3G, or switching from cellular to WiFi®.

- The user performing a test using a coverage or network status checking app (or any other similar form of app) - A long-running app having been closed (and hence its identity becoming available, in operating systems which limit access to the active application list).

The outputs of timer 417 and event stimulus 418 feed a logical OR step 419 which yields a positive (logic '1') output in the event that the output from timer 117 is logic '1' or the output from event stimulus 118 is logic '1' or both outputs are logic '1'. The output of logical OR 419, in turn, feeds decision step 420; in the event that logical OR 419 has a positive or logic '1' output, then decision step 420 will conclude that a measurement report should be sent and the method will move on to step 425. Otherwise, if the output of logical OR 419 is negative or zero, then decision step 420 will conclude that a measurement report should not be sent and the method moves to step 415, which has been discussed above.

At step 425, the app 138 which was running in the foreground of the user interface of the UE during the measurement period is identified (as described with respect to steps 202 and 302 in processes 200 and 300 above). In particular, the process interrogates a list of recently closed applications, which lists applications that have closed within a particular time period. If this time period is sufficiently short (for example 10 minutes), then it is possible to make an informed association between the app or apps which were running within the time period and network performance measurements taken during or within that same time period. Such measurements may be aggregated over the time period, as will be discussed in particular in relation to data rate, in association with Figure 5 below, or may be 'snapshot' measurements of the instantaneous performance taken at a time, or at multiple times, within the measurement period.

The process then moves on to step 430 in which the timestamp and associated measurements are retrieved from the memory 114 and these are combined with data pertaining to the app which was running at the time indicated by the timestamp, which was ascertained in step 425, to form a complete measurement report. Note that it is also possible to 'tag' the identity of the app with the measurement data at a number of points within the process and not just at step 430; for example it could be undertaken within step 410 or step 415. In a further alternative embodiment, it is possible to send the data twice (in, for example, step 435); the first time without an associated app 'tag', with the aim of getting data to the network as quickly as possible, and then, later, re-sending the same data, but this time with app 'tags' attached (once these have been identified). This scenario may be useful when a single app has been in use for a long period (e.g. when the user is watching a video on a video player app); the identity of the running app will not be available until the app has been closed down and, in the video player example, this could be many minutes or tens of minutes and it may be beneficial to send network performance characteristic updates more frequently than this.

Finally, the method moves on to step 435 in which the measurement report formed in step 430 is transmitted to the network 110 by the UE. The method then returns to steps 410 and 420 (in parallel) as before, and repeats until either the measurement app 136 is stopped or the UE turned off.

Figure 5 shows an alternative flowchart showing a process 500 for calculating the throughput during particular (typically short) time periods, as obtained by a UE when in communication with a network 110, and associating each period with the foreground app which was likely to have been running during the time period in question (i.e. data throughput is the measured network performance characteristic).

The method begins at step 505 and then moves on to steps 510 and 520, in parallel. Considering, first, the path beginning with step 510: step 510 is again the first step in a continuously-running loop (operational whilst the measurement app is operational). In this step, the UE is awoken from a deep-sleep (ultra-low power) state, if it was in such a state at the time, into a low-power state and then gathers measurement data from the device including data on the number of bytes received and sent since the last time measurement data was gathered. It records this data, together with a timestamp for the time at which the data was gathered, in the memory 114 of the UE. Examples of the additional types of data which may be gathered and stored are similar to those discussed above in relation to Figure 4, above.

The method then moves on to step 515 in which the user device returns to its 'deep sleep' state (or whatever prior state it was in when the current iteration of the continuous loop involving steps 510, 515 was initiated). The method then loops back to re-run the loop and, in parallel, re-consider the decision of step 520, which will be discussed below.

In step 520, the method queries whether one or more measurement reports should now be sent to the network 110. It bases this decision upon two stimuli: timer 517 and event 518. Timer 517 is a timer which outputs a positive stimulus or changes a logic level from logic '0' to logic '1' once a pre-set time interval has elapsed, for example 10 minutes, it then resets to zero and resumes timing another (typically identical) interval. Event stimulus 218 outputs a stimulus or changes a logic level when a given event has taken place. Examples of events are similar to those listed above in relation to figure 4.

The outputs of timer 517 and event stimulus 518 feed a logical OR step 519 which will yield a positive (logic '1') output in the event that the output from timer 517 is logic '1' or the output from event stimulus 518 is logic '1' or both outputs are logic '1'. The output of logical OR 519, in turn, feeds decision step 520; in the event that logical OR 519 has a positive or logic '1' output, then decision step 520 will conclude that a measurement report should be sent and the method will move on to step 521. Otherwise, if the output of logical OR 519 is negative or zero, then decision step 520 will conclude that a measurement report should not be sent and the method moves to step 515, which has been discussed above.

At step 521, the data throughput is calculated for each of a plurality of time windows (which may be considered windows within a measurement period, or a plurality of measurement periods), based upon the number of bytes received and/or sent, as recorded by the UE (e.g. derived from AndroidC)'s TrafficStats or NetworkStatsManager). Alternatively or additionally, network latency may be measured for each of the plurality of time windows. Network latency could be measured, for example, using a standard (device-initiated) 'ping' test (either: internet control message protocol (ICMP), transmission control protocol (TCP) or user datagram protocol (UDP)).

The results of the calculations are then stored in a database within the memory 114, and are associated, within the database, with the timestamp of the original measurement upon which the calculation was based. Note that this calculation could alternatively be undertaken by the network 110 once it is in receipt of measurement data and timestamp information from the user device and does not necessarily need to be undertaken on the UE itself.

The process then moves on to step 525 in which the identity of the app which was running at the time indicated by a timestamp retrieved from database 522, is retrieved from the list of recently closed apps, similar to the process discussed above in relation to Figure 4.

The process then moves on to step 530 in which the timestamp and associated measurements, including the calculated data rate or throughput, are retrieved from the memory 114 and are combined with data pertaining to the app which was running at the time indicated by the timestamp (e.g. the identity of the app, or the data usage profile of the identified app), which was ascertained in step 525, to form a complete measurement report.

Finally, the method moves on to step 535 in which the measurement report formed in step 530 is transmitted to the network by the UE. The method then returns to steps 510 and 520 (in parallel) as before, and repeats until either the measurement app is stopped or the user device turned off.

It is noted that in processes 400 and 500, the active app is only identified when a reporting criterion is met, in advance of reporting the measurements to the network 110. In alternative embodiments, the app may be identified prior to the reporting criterion being met (e.g. each time the network performance characteristic is measured), and the identity of the app stored in the memory 114.

Figure 6 illustrates an alternative method 600. In method 600, an application is not necessarily identified and associated with the measured network performance characteristic/s.

At step 601, a network performance characteristic is measured, similarly to step 201 of method 200. The network performance characteristic may be associated with a particular measurement period (e.g. it may comprise an average value for the measurement period, or may be a point measurement used to represent the measurement characteristic for the measurement period). The measurement may be triggered for example if a certain amount of time has elapsed since a previous measurement, or based on a particular location or change of location of the UE 100, as would be understood by the person skilled in the art.

At step 602, it is determined whether a predetermined change has occurred in a reporting characteristic. Determining whether a predetermined change has occurred may comprise measuring a reporting characteristic; comparing the reporting characteristic to a previous measurement of the reporting characteristic; and determining whether a predetermined change has occurred in the reporting characteristic since the previous measurement. For example, the predetermined change may be a change in a value of the reporting characteristic that exceeds a predetermined change threshold. The reporting characteristic may be a property of the UE 100, of the network 110, or of the connection between the UE and the network.

If the predetermined change in the reporting characteristic has occurred, the method proceeds to step 603. At step 603 the network performance characteristic (and, optionally, the identity of an identified app associated with the network performance characteristic) is reported to the operator of the 4G network 110.

If the predetermined change in the reporting characteristic has not occurred, the method proceeds to step 604. At step 604 the reporting of the network performance characteristic to the network 110 is deferred, for example for a predetermined period of time, or until a predetermined change in a or the reporting characteristic occurs. The network performance characteristic may be stored in a memory of the UE until it is reported. Alternatively or additionally, the network performance characteristic may be re-measured for the later reporting.

Figure 7 illustrates a specific example of how the above process may be implemented on a UE.

Figure 7 shows a flowchart of the basic operation of a process 700 which may be performed by the monitoring app 136. Process 700 is similar to process 400, but does not include the step of identifying an application.

Process 700 begins at step 705 and then moves on to steps 710 and 720, in parallel. Considering, first, the path beginning with step 710: step 710 is the first step in a continuously-running loop (operational whilst the monitoring app 136 is operational). In this step, the user device is awoken from a deep-sleep (ultra-low power) state, if it was in such a state at the time, into a low-power state. The process then gathers measurement data from the device recording a network performance characteristic. Additional data, such as location data, e.g. from the UE's GPS receiver, may also be recorded. These data are recorded together with a timestamp for the time at which the data was gathered, in a database 712 stored within the memory 114 of the UE.

Examples of types of additional data which may be gathered and stored are: Device data: - Device identity (e.g. media access control (MAC) address, international mobile subscriber identity (IMSI), international mobile equipment identity (IMEI) or similar identifier(s)) - Brand, model, type allocation code (TAC), memory used/free, storage used/free, CPU usage - UTC Timestamp, Local device time Connection data: - Received signal strength indication (RSSI), reference signals received power (RSRP), reference signal received quality (RSRQ), reference signal signal-to-noise ratio (RSSNR), signal-to-noise-plus-interference ratio (SINR), channel quality indicator (CQI), timing advance (TA) - Cell ID (and/or SSID, if WiFi®) of the cell (or access point) to which the device is connected - Cell IDs (and/or SSIDs) of any other cells/access points from which the device can receive signals - Frequency band, absolute radio frequency channel number (ARFCN), location area code (LAC), type allocation code (TAG), mobile country code (MCC), mobile network code (MNC) Location data: - Device location (geographic location) - Speed of movement of the device - Direction of movement - Location accuracy - Whether the UE is (or is likely to be) located indoors or outdoors The process then moves on to step 715 in which the UE returns to its 'deep sleep' state (or other prior state it was in when the current iteration of the continuous loop involving steps 710 and 715 was initiated), and waits for a predetermined period. This period could be short, for example 1-2 seconds, allowing rapid and regular sampling of the activity taking place on the UE and gathering of its associated measurement data, or it could be a longer period of minutes (e.g. 1-2 minutes, 1-5 minutes) or tens of minutes (e.g. 10-20 minutes), perhaps driven by the inclusion of GPS or other location data (without unduly impacting upon battery life), or even longer (hours, days, e.g. 1-2 hours, 1-3 hours, 5 hours, 24 hours, 1-2 days) providing an aggregated user experience. The process then loops back to re-run the loop and, in parallel, re-consider the decision of step 720, which will be discussed below.

At step 720, the process queries whether one or more measurement reports should now be sent to the network. It bases this decision upon whether there has been a predetermined change in a reporting characteristic. In particular, it bases this decision on two stimuli: timer 717 and event 718. Timer 717 is a timer which outputs a positive stimulus or changes a logic level from logic '0' to logic '1' once a pre-set time interval has elapsed, for example 10 minutes or, in an alternative embodiment, a much shorter period such that a measurement report is sent as soon as it is generated; it then resets to zero and resumes timing another (typically identical) interval. Event stimulus 718 outputs a stimulus or changes a logic level when a given event has taken place (i.e. a predetermined change in a reporting characteristic). Examples of events any or all of which may be used are: - The UE having moved greater than a predetermined distance - The UE having experienced greater than a predetermined change in RSSI - The UE having experienced greater than a predetermined change in data rate or throughput - A particular app having been recognised as having been in use (e.g. is actively in use or is identified in the list of recently closed apps) - The UE having moved into or within a place of interest defined as a polygon boundary around a location - The network connection having changed e.g. a ha ndover to another cell, the change of radio bearer technology from 4G to 3G, or switching from cellular to WiFi@.

The outputs of timer 717 and event stimulus 718 feed a logical OR step 719 which yields a positive (logic '1') output in the event that the output from timer 717 is logic '1' or the output from event stimulus 718 is logic '1' or both outputs are logic '1'. The output of logical OR 719, in turn, feeds decision step 720; in the event that logical OR 719 has a positive or logic '1' output, then decision step 720 will conclude that a measurement report should be sent and the method will move on to step 730. Otherwise, if the output of logical OR 719 is negative or zero, then decision step 720 will conclude that a measurement report should not be sent and the method moves to step 715, which has been discussed above.

The process then moves on to step 730 in which the timestamp and associated measurements are retrieved from the memory 114 to form a complete measurement report.

Finally, the method moves on to step 735 in which the measurement report formed in step 730 is transmitted to the network 110 by the UE. The method then returns to steps 710 and 720 (in parallel) as before, and repeats until either the measurement app 136 is stopped or the UE turned off.

It is noted that in alternative example routines, the timer 717 and logical OR 719 may be omitted, so that the determination of whether to send a report to the network 110 is based solely on whether the predetermined change in the network performance characteristic has occurred (i.e. whether the output of event stimulus 718 is logic '1' or logic '0').

Example embodiments of the invention have been described for the purposes of illustration only, and not to limit the scope of the invention as defined in the appended claims.

Claims

CLAIMS1. A method of operating a user equipment, UE, configured for use with a communications network, the method comprising: measuring a network performance characteristic; determining whether a predetermined change in a reporting characteristic has occurred; and if so, reporting the network performance characteristic to the communications network.
2. The method of claim 1, wherein the predetermined change comprises a change in a value of a reporting characteristic greater than a predetermined amount.
3. The method of claim 2, wherein the reporting characteristic is a distance moved by the UE.
4. The method of claim 2, wherein the reporting characteristic is a received signal strength indication.
5. The method of claim 4, wherein the reporting characteristic is a data rate or data throughput.
6. The method of claim 1, wherein the predetermined change comprises a change of state of an application stored on the UE.
7. The method of claim 6, wherein the change of state comprises an application running on the UE closing.
8. The method of claim 7, wherein the change of state comprises a predetermined application closing.
9. The method of any of claims 6 to 8, wherein determining the predetermined change in a reporting characteristic comprises determining that a predetermined application has been recently active on the UE.
10. The method of claim 1, wherein the predetermined change comprises a change of state of a network connection.
11. The method of claim 10, wherein the change of state comprises a change of network connection type.
12. The method of claim 11, wherein the change of network connection is between a 3G cellular connection and a 4G cellular connection.
13. The method of claim 10, wherein the change of network connection is between a cellular connection and a wireless local area network, WLAN, connection.
14. The method of claim 1, wherein the predetermined change comprises a predetermined user interaction.
15. The method of claim 14, wherein the predetermined user interaction comprises a user performing a test of a network connection.
16. The method of any preceding claim, wherein the network performance characteristic comprises the reporting characteristic.
17. The method of any of claims 1 to 15, wherein the network performance characteristic is different to the reporting characteristic.
18. The method of any preceding claim, wherein the measured network performance characteristic is associated with a measurement period, and wherein the predetermined change in the reporting characteristic occurs during the measurement period.
19. A user equipment UE for use with a communications network, the UE comprising: a transmitter for communicating with the communications network; and a processor configured to perform the method of any of claims 1 to 18.
20. Program code, which when executed by compatible data processing hardware, will cause the hardware to perform a method according to any one of claims 1 to 18.