GB2637360A - Method and system for predicting user sentiment - Google Patents

Abstract

A method and system for mitigating a predicted user sentiment (such as frustration) for a user device using a trained machine learning model, the predicted user sentiment occurring at a given time. The mitigation process comprises obtaining telemetry data 110 representing operating characteristics of the user device within a predetermined time period preceding the given time. The telemetry data is analysed 120 using the trained machine learning model to determine a correlation between the telemetry data and one or more potential user sentiments. Based on the correlation, one or more proposed actions are determined 130 to mitigate against the potential user sentiments. The one or more proposed actions are provided to an operator 140, and based on a selection by the operator, a selected action is initiated 150 on the user device. A method for training a machine learning model to predict the user sentiment and mitigating action uses data related to the operating characteristics of a plurality of user devices within a predetermined time period preceding the given time, data related to one or more mitigating actions taken, and a relationship between some of the operating characteristics data and some of the mitigating actions taken.

Description

METHOD AND SYSTEM FOR PREDICTING USER SENTIMENT

Technical Field

The present invention relates to a method and system for predicting user sentiment, more particularly, predicting user sentiment of a user using a device.

Background

A user's experience/sentiment plays a pivotal role in determining the success and acceptance of new and existing computer systems, applications, and services.

Understanding and predicting user sentiment is important for developers and service providers aiming to enhance their user experience, satisfaction, and overall system performance.

Various online platforms and applications incorporate sentiment analysis tools to gauge user opinions and emotions expressed in textual content, reviews, and feedback. For example, many platforms offer surveys and/or feedback forms which are completed after a user has interacted with their product enabling them to implement new features and/or refinements to their product to improve future users' overall satisfaction and sentiment. However, these methods provide no improvement for the current user's experiences nor do they enable improvements to be implemented in such a way to prevent negative feedback.

Summary

According to aspects of the present disclosure, there is provided a first method as set out in the appended claims, a computer program product such as a non-transitory computer-readable storage medium carrying instructions for carrying out the first method, and a system comprising at least a user device and an operator device configured to perform the first method.

The first method is one of mitigating a predicted user sentiment for a user device using a trained machine learning model. The predicted user sentiment occurs at a given time. The method comprises obtaining at least telemetry data associated with the user device, the telemetry data representing operating characteristics of the user device within a predetermined time period preceding the given time. The machine learning model is then used to analyse at least the telemetry data to determine a correlation between at least the telemetry data and one or more potential user sentiments. Based on this correlation, one or more proposed actions are determined, where the proposed actions mitigate against the one or more potential user sentiments. These proposed actions are presented to an operator, and based on a selection by the operator, at least one selected action, of the proposed actions, is initiated on the user device.

By pre-empting a user sentiment, and determining one or more actions to be initiated/ implemented on the user device, improvements to the user experience may be achieved. Furthermore, by basing the actions proposed on technical data obtained from a user device, limitations and/or resource availability may be used to indicate whether a particular user sentiment is likely to occur, such that improvements to the user experience can be implemented. Improvements to the user experience will, therefore, result in a reduction in the likelihood of user frustration, whilst also potentially minimizing resource usage on the user device by implementing improvement actions before a given event likely to impact user sentiment occurs.

Optionally, the method comprises obtaining user input characteristics, the user input characteristics associated with one or more inputs provided by a user to the user device; wherein, determining the one or more proposed actions comprises analysing using the trained machine learning model the user input characteristics to determine the correlation between the user input characteristics and one or more of the potential user sentiments. This enables user interactions with the user device to be considered when predicting user sentiment at a given time. The user interactions may be indicative of a negative user experience, which can then be used to suggest, by the machine learning model, one or more potential actions to prevent the upcoming negative user experience.

The method may also comprise obtaining application telemetry data associated with at least one application operating on the user device, wherein determining the one or more proposed actions comprises analysing using the trained machine learning model the application telemetry data to determine the correlation between the application telemetry data and one or more of the potential user sentiment. This enables additional data to be used, specifically referring to the application itself, to determine whether the user sentiment is likely to change. By considering aspects of the application, a more accurate overview of the experience by the user can be evaluated and any potential user sentiment determined more accurately. As such, any actions proposed can also be tailored to mitigate against that particular user sentiment. Optionally, the one or more proposed actions are based on preferences and/or characteristics associated with a user of the user device. This enables the proposed actions to be based on the unique preferences and/or characteristics of the user, resulting in more customized actions to mitigate the user sentiment. It also enables thresholds associated with the other users to be considered to provide the most suitable rem edi ati on actions.

The predicted user sentiment for a user device may be a predicted degradation of a user experience associated with a user of the user device. This enables the prediction of degradation in the user experience and subsequent remedial/mitigation action to be taken to improve the user experience and/or prevent the predicted degradation from occurring.

The predicted degradation is indicative of at least one of an increase in an average processing time for at least one operation on the user device; an average transmission time of a message being sent from the user device; and an average interaction time with one or more hardware resources of the user device. Such information can be used to indicate a degradation in the user experience.

Optionally, the telemetry data, associated with the user device, is stored in a storage. The storage may also be remote from the user device. This enables future determinations and/or predictions to be made based on the previous data, not just real-time data. It also enables the machine learning model to be trained based on this data, to improve the accuracy of the model.

The analysis, using the machine learning model, may be performed by a remote device, the remote device being communicably coupled to the user device. This enables data from other devices to be considered when predicting a user sentiment, and also when training the machine learning model. Furthermore, it results in a reduction in the processing power and/or storage needed at the user device, which may be better utilised improving the user experience and/or preventing the predicted user sentiment.

According to other aspects of the present disclosure, there is provided a second method as set out in the appended claims, a computer program product such as a non-transitory computer-readable storage medium carrying instructions for carrying out the second method, and a system comprising at least a user device and an operator device configured to perform the second method.

The second method is one of of training a machine learning model to mitigate a predicted user insight sentiment at a given time for a user device, the method comprising obtaining training data associated with a plurality of user devices, different to the user device, the training data comprising at least first data representing operating characteristics of the plurality of user devices within a predetermined time period, the predetermined time period preceding the given time second data representing one or more mitigating actions undertaken; and a relationship between at least some of the first data and at least some of the second data; and training the machine learning model using the training data the machine learning model configured to predict the occurrence of a user insight sentiment at the given time and determine at least one mitigating action to be implemented by an operator.

By training a machine learning model to predict user sentiment based on at least data representing operating characteristics of user devices, enables accurate prediction of an alteration in the user sentiment to be accurately predicted. Furthermore, by training the machine learning model to determine rectification actions to mitigate the predicted user sentiment, based at least on the operating characteristics of the user device, improvements in the user experience can be achieved, as the likelihood of frustration is reduced. Furthermore, this can minimize resource usage on the user device.

Optionally, training the machine learning model may comprise at least one of user input characteristics associated with one or more inputs provided by a user to at least one of the plurality of user devices, the user input characteristics having been obtained over the predetermined time period and application telemetry data associated with at least one application operating on at least one of the plurality of user devices, the application telemetry data having been obtained over the predetermined time period. This enables the machine learning model to be trained using data representative of user interactions with one or more user devices as well as application information to determine whether a negative user sentiment is likely. As such, this enables a more accurate indication of the likelihood that a particular user sentiment is likely to occur.

Training the machine learning model may further comprise periodically obtaining at least additional telemetry data from the user device, storing at least the telemetry data in storage, and updating, the machine learning model based on at least the additional telemetry data. This enables the machine learning model to be periodically updated and/or retrained based on telemetry data from the user device, thereby providing the most accurate prediction mechanism for mitigating predicted user insights.

Optionally, the training of the machine learning model is undertaken on a remote device communicably coupled to the user device. This enables the machine learning model to be trained on a remote device, and therefore obtain data from multiple sources (e.g., multiple user devices), as well as be provided to multiple user devices when trained. It also reduces the processing/ memory requirements at the user device freeing up said resources for other more important user device tasks.

The training data may be obtained from storage associated with the user device, and/or from storage remote from the user device. This enables the machine learning model to be trained based on data associated with a given user device itself, and/or based on data associated with other user devices which may or may not be related to the given user device.

Brief Description of the Drawings

Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.

Figure 1 is a flowchart showing a method of mitigating a predicted user sentiment for a user device using a trained machine learning model according to an

example;

Figure 2 is a flowchart showing a method of training a machine learning model to mitigate a predicted user sentiment according to an example; and Figure 3 is a schematic representation of a system for mitigating a predicted user sentiment according to an example.

Detailed Description

The importance of monitoring and managing user sentiment is becoming more and more important as users interact with increasing numbers of digital products and services. However, the existing methods and systems often lack the precision and real-time capabilities required to effectively predict negative user sentiment during a user' s interactions with a computer system, and therefore it is not possible to address any problems before, or as, they arrive. Analysing diverse data sources, including user input, system responses and performance, and contextual information, to accurately anticipate negative sentiment before it escalates enables action to be taken so that the user never experiences or develops negative sentiment towards the product. Addressing this need is vital for pre-emptive measures, to mitigate the impact of negative sentiment and enhance overall user satisfaction. Such measures may include system improvements, customer support interventions, or personalized user interactions.

Figure 1 is a flowchart 100 showing a method of mitigating a predicted user sentiment for a user device using a trained machine learning model. By using a trained machine learning model, user sentiment can be predicted and then suitable remediation actions are determined to prevent the likelihood of a given user sentiment from occurring. By predicting the likelihood of a user sentiment occurring at a future time, situational awareness regarding a particular system and/or user device can be determined providing valuable information to operators of the system without the need for the traditional surveying of users which is, in and of itself, a retrospective task. Such sentiments can provide the operators with an understanding of problems and/or issues which are affecting users, enabling them to prioritise remediation actions, to prevent the predicted user sentiments from occurring in the first place, and/or occurring for other users of the system. For example, by prioritising the remediation actions, the operator can ensure that a negative user insight, such as an application slow down/ crash, occurring is mitigated, and as such the negative user insight never occurs or the likelihood of it occurring is reduced. It will be appreciated that, whilst the examples below mainly relate to the mitigation of negative user insights, other, more positive, user insights associated with a given user interaction with a system or device may be mitigated. For example, where resources are limited, the given user's share of those resources can be curtailed to prevent other users from experiencing a negative user insight. This is despite the fact that it would result in a positive experience for the given user.

At step 110, telemetry data associated with a user device, such as the user device 310 described below with reference to Figure 3, is obtained. The telemetry data may include a large amount of data associated with the user device such as data associated with a processor or memory of the user device, and/or data relating to the consumption of such user device resources. The telemetry data is data for a given time period which precedes the time of the predicted user sentiment. It will be appreciated that any relevant data associated with any resource the user device may be obtained alongside data associated with the consumption of those resources. For example, data such as processor utilization, memory utilization, average processing time, transmission times over a network interface, disk space utilization, number of dropped packets during transmission, and the time that a disk system is busy, amongst others may be obtained. The telemetry data may be used, as set out below, to determine a potential/upcoming change in user experience or sentiment. Potential changes to the user experience or sentiment may be indicated by the telemetry data, and include the detection of increased processor and memory utilization, for example, which may be indicative of a degradation in user experience.

In some examples, other data may also be obtained. In such examples, at step 111, user input characteristics are obtained. These user input characteristics represent a user's interaction with the user device, such as user device 310 described below in relation to Figure 3. As with the telemetry data described above, the user input characteristics are based on inputs made to the user device at a time preceding the time of the predicted user sentiment. The user input characteristics may be based on data obtained by an input module of the user device, such as inputs made using a mouse, keyboard, touch screen, or any other suitable input device. The user input characteristics may represent a potential change in the user experience and/or sentiment, for example, where a user is becoming increasingly frustrated. In such examples, detected inputs, such as a rapid increase in the inputs received by the user device, such as rapid inputs to an external input device, like a mouse or keyboard, may be indicative of increased user frustration. Obtaining such data can also feed into the prediction of user sentiment, as will be described below.

Similarly, at step 112, in some examples, application telemetry data may be obtained. The application telemetry data may be status information associated with the execution of an application on a given user device at a time preceding the time of the predicted user sentiment. Characteristics associated with the execution of the application, such as the time taken to perform a given operation or task, and/or delays in the execution can also be used to predict a change in the user sentiment.

It will be appreciated that whilst certain examples of telemetry data, user input characteristics, and application telemetry data are described above, other data and examples of such data may also be used in the analysis described below.

Upon obtaining the relevant data, it may be stored in a storage, as will be described below, which may, in some examples be associated with the user device or in other examples, may be remote from the user device.

Once the relevant data has been obtained, the data is analysed at step 120. Analysing the data comprises a number of different analysis techniques to monitor for outliers and trends in the data which may in turn be used to predict one or more changes in user sentiment, at a given time. The analysis techniques used at step 120 may comprise obtaining additional data from one or more third-party sources, and or other correlating data from previous examples of the user sentiment changing. The obtained data, and any other additional data, is then provided to a trained machine learning model, such as the machine learning model trained in accordance with method 200 described below in relation to Figure 2. The machine learning model is trained to analyse the data provided to it and to predict a likely change in user sentiment. For example, the predicted change in user sentiment may result in a detected degradation in the user experience, such as a slowing down of the processing of data, an indication of frustration by the user based on their inputs, and/or an application crashing. It will be appreciated that there are any number of different indications used to predict that there will be a change in user sentiment. The predicted user sentiment may be a negative or positive user sentiment and may be dependent on the characteristics of the particular user. The predicted user sentiment may result in a degradation of the user's experience with the user device, such as there being an increase in the average processing time for operations on the user device, an increase in the average transmission time of a message being sent from the user device, and/or an increase in the average interaction time with one or more resources of the user device. It will also be appreciated that other indications of a degradation in the user experience may also be used.

In certain use cases, a first user may be less frustrated by a particular situation and/or performance degradation in comparison to a second user. As such, a predicted negative user sentiment is less likely for the first user than the second user in that situation.

Once a change in the user sentiment has been predicted, the trained machine learning model determines a correlation between at least the telemetry data, possibly the user input characteristics, and the application telemetry data, and one or more potential user sentiments. The potential user sentiments represent user sentiments which can be mitigated by performing one or more actions. As such, the analysis and the trained machine learning model are used to predict that a particular user sentiment is likely to occur, and as such determine which of a catalogue of potential user sentiments the predicted user sentiment is most likely. The catalogue of potential user sentiments and their associated characteristics may be obtained from a remote storage solution, such as a data lake, stored within a network.

Following the analysis, at step 130 one or more proposed actions are determined. The one or more proposed actions are based on the correlation between the telemetry (and other) data, and the one or more potential user sentiments. As such, the proposed actions are used to mitigate against the predicted user sentiment, to improve and/or change the overall user experience and to prevent the predicted user sentiment from occurring. In some examples, it will be appreciated that when the predicted user sentiment is a positive one, the proposed actions may be actions which emphasise, or make it more likely, that the predicted user sentiment actually occurs. Furthermore, the one or more proposed actions may be determined based on the characteristics and/or preferences of the given user, to provide a personalised mitigation strategy. This enables the proposed actions to be more suitable for the given user, preventing unnecessary and/or inappropriate actions from being provided to the operator as set out below.

When the one or more proposed actions have been determined, at step 140, the proposed actions are provided to an operator. The operator may be a system administrator associated with a company or other entity of the user for example. The operator may have suitable administrative privileges such that they can instruct the given user' s device to implement one or more of the proposed actions as will be set out below. The one or more proposed actions may be provided to another user device, such as one used by the operator, or to an administration console. The one or more proposed actions may be transmitted over a network, such as the Internet, or other communication methodology, to the operator' s device. The proposed actions may be provided in such a way that they are presented within a graphical user interface to the operator, however, it will be appreciated that there are other means of providing the one or more proposed actions to the operator. The one or more proposed actions are provided to the operator in such a manner that the operator can select at least one of the proposed actions. For example, where the one or more proposed actions are provided to the operator in a graphical user interface, the operator may be able to select a desired action using an input device such as a mouse, keyboard, and/or touchscreen display.

Following the selection of proposed action(s) by the operator, the one or more selected actions, if required, may be transmitted back to the user device over a network or via another communication means. At step 150, the selected action is implemented on the user device, so as to mitigate the predicted user sentiment from occurring. By implementing one or more the selected actions on the user device, the user experience is improved and/or maintained, such that any negative sentiment is prevented from occurring. In some examples, user preferences stored on the user's device may be used to indicate whether the selected action is appropriate. By performing this check on the user device, more accurate and/or appropriate actions can be implemented, without the need to send user characteristics and/or personally identifying information to a remote operator device, thereby increasing privacy and security. This may be particularly beneficial when multiple selected actions are provided by the operator.

Figure 2 is a flowchart showing a method 200 of training a machine learning model to mitigate a predicted user sentiment, such as the machine learning model described above in relation to Figure 1. The machine learning model may comprise one or more neural networks, such as a convolutional neural network, capable of being trained to predict a user sentiment at a future time and provide one or more actions to mitigate against the predicted user sentiment.

At step 210, training data for training the machine learning model is obtained. The training data comprises at least operating characteristics 211, mitigating actions 212, and relationship data 213. As described above, the operating characteristics 211 represent the performance characteristics of a user device over a period of time. In some examples, the operating characteristics may represent the performance characteristics of a plurality of user devices, of the same and/or different types to that of the given user device. The operating characteristics 211 may include resource usage data and characteristics representing at least the processor, memory, communication, and disk space of the user device. It will be appreciated that any number of other resource usage data and characteristics may be obtained and used in the training of the machine learning model.

In some examples, as well as the operating characteristics of the user device(s), user input characteristics 214 may be obtained representing previous examples of how users, which may or may not include the given user implementing the method 100 described above, interact with the user device(s). For example, the user input characteristics 214 may include data representing the types, frequency, and length of inputs provided to a user device over a period of time. Such user input characteristics 214 may be indicative of user frustration (e.g., the quick repetition of clicks or the erratic movement of a cursor), and as such can be used as an indication of user sentiment. Similarly, other data may also be obtained for use in the training of the machine learning model. Data such as application telemetry data 215 may be used to indicate whether a user's sentiment is likely to change. Data relating to applications may include data representing delays in the performing of operations and application crash data. It will be appreciated that other telemetry data associated with the applications may also be used. Such data can indicate, either alone or in combination with other data whether there is likely to be a change in user sentiment.

The application telemetry data, user input data, and operating characteristics may be obtained in relation to different applications running on a plurality of different user devices, not just the same, or the same type of, user device used by the user described above in relation to method 100 of Figure 1 In addition to the operating characteristics, and in some examples, the user input characteristics and application telemetry data, obtaining the training data at step 210 may also comprise obtaining one or more mitigating actions 212. The mitigating actions represent actions which may be undertaken by a device to improve and/or mitigate a predicted user sentiment. The mitigating actions may include any number of different actions such as restarting applications, clearing caches, closing unused applications, and reconnecting to a network, among others. It will be appreciated that any number of mitigating actions may be obtained and used in the training of the machine learning model as will be described in further detail below.

As part of the training data that is obtained at step 210, relationship data 213 is also obtained. The relationship data represents a relationship between at least some of the operating characteristics 211 and at least some of the mitigating actions 212 which may be undertaken to mitigate a predicted user sentiment. In other examples, where user input characteristic data 214 and/or application telemetry data 215 is also obtained, the relationship data 213 may also comprise data representing relationships between the user input characteristic data 214 and one or more mitigating actions 212, and the application telemetry data 215 and one or more mitigating actions 212. By associating the data 211, 214, 215 with a mitigating action 212 the training data may be used to indicate which mitigating actions are to be undertaken when a particular user sentiment is predicted.

The training data 211 -215 may be obtained from storage associated with the user device and/or a remote device communicably coupled, such as via a network, to the user device. Furthermore, in some examples, the training data may be periodically updated by obtaining additional training data (e.g., telemetry data/operating characteristics, mitigating actions, relationship data, user input characteristics, and application telemetry data) associated with the user device. The additional training data may be stored in the storage (either remote or local to the user device), and the machine learning model may be periodically retrained based on the additional training data.

Once the training data 211 -215 has been obtained at step 210, the machine learning model is trained at step 220. The machine learning model may be trained on the user device itself, or in some examples may be trained on a remote device, thereby freeing up resources on the user device for other more important processing tasks. The machine learning model is trained based on the training data and may be trained using specialized hardware, such as a neural processor. The machine learning model is trained to predict changes in user sentiment, such as a potential degradation in a user experience, and then suggest one or more mitigating actions to prevent or mitigate against that degradation. The prediction of the changes in user sentiment may be customizable based on one or more preferences and/or characteristics of the user to provide more personalized suggestions for mitigating actions. In addition to being trained to predict the change in user sentiment, the machine learning model is also trained to output one or more suggested mitigating actions based on the prediction, to be implemented on the user device. As with the prediction, the mitigating action may also be customizable based on user preferences provided to the machine learning model to provide more appropriate mitigations. In some examples, the machine learning model may be trained to output a single mitigating action, and in other examples, the machine learning model may be trained to output a plurality of mitigating actions.

It will be appreciated that whilst method 200 described above relates to the training of the machine learning model based on the obtained training data, it will be appreciated that the machine learning model may also be trained based on other data in addition to the training data described above.

Figure 3 is a schematic representation of a system 300 for mitigating a predicted user sentiment by implementing a method, such as method 100 described above. The system 300 comprises at least one user device 310, and at least one operator device 320. Each component may comprise a network adapter or networking module 316, 322 that is arranged to facilitate communication with any number of components via a network 330 such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet). The network adapter or networking module 316, 322 may be configured to communicate using either a wired or wireless communication method, such as cellular connectivity (LTE, 3G, 4G, or 5G), ethernet, or over a Wi-Fi network.

In some examples, the system 300 comprises a remote storage system 340 having storage for storing training data, such as the training data 211 -215 obtained in method 200 described above, and/or other data for use in the training of and implementation of the machine learning model for use in mitigating a predicted user sentiment. The storage of the storage system 340 may be a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, for example, a CD ROM or a semiconductor ROM, a magnetic recording medium, for example, a floppy disk or hard disk; optical memory devices in general, although it will be appreciated that other storage mediums may be used. The storage system 340 may be accessed via a local area LAN, a WAN, and/or a public network (e.g., the Internet) via the network adaptor. Whilst the storage system 340 is shown as separate from the other resources of the system 300, it will be appreciated that the storage system 340 may form part of the user device 310, or another device, such as the operator device 320, or maybe a virtual component associated with a cloud computing implementation of the system 300. In yet further examples, the storage system 330 may be located on another server in a different location than the user device 310 or operator device 320.

The system 300 comprises at least a user device 310 and an operator device 320 which may be implemented in hardware, or maybe an AWS server or other server provided by a cloud services provider. The user device 310 may be configured on the same network 330 as the operator device 320 or storage system 340 or may be accessed via an external network such as the Internet.

The user device 310 comprises a processor 312, which may be a general-purpose processor, such as a central processing unit, or any other suitable processor, as well as storage 314, for providing the relevant resources for executing any number of applications. The storage may be a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, for example, a CD ROM or a semiconductor ROM; a magnetic recording medium, for example, a floppy disk or hard disk; optical memory devices in general. Operating characteristics of the processor 312 and the storage 314 may be used, as described above to predict a user sentiment. As mentioned above, the user device 310 may be connected to a network 330 via the networking module 316 to communicate with other devices and/or the remote storage 340.

The operator device 320 also comprises a networking module 322 to communicate with other devices on the network 330. It also comprises a graphical user interface 324 to enable operators to provide inputs 327 via the operator device input module, to indicate and/or select one or more of the mitigating actions output by the machine learning model.

The machine learning model may be implemented on the user device 310, the operator device 320, or on another device (not shown). The device 310, 320 used to train and implement the machine learning model may comprise a neural processing unit (not shown) which may be part of a system on chip or other processing component of the device, such as processor 312 of the user device. By implementing the machine learning model on the user device 310 itself increases in security and performance may be achieved as there is no need to transmit any of the operating characteristics or other data to a remote device, over the network 330. Alternatively, by implementing the machine learning model on a remote device, such as the operator device 320, resource usage at the user device 310 is minimised, potentially further improving user experience/ sentiment since the implementation is not taking up resources which may be used for the execution of other applications.

As set out above, the user device 310 comprises a processor 312 which may be used to determine one or more proposed actions to mitigate against the one or more potential user sentiments based on the correlation between the telemetry data and the potential user sentiments provided by the machine learning model. The networking module 316 is used to transmit these actions, over the network 330, to the operator device320, where an operator can provide an input 327, via the operator device input module 326, on a graphical user interface 324. An indication of the operator' s selection of the one or more proposed actions is then transmitted back to the user device 310 via the networking module 322 and over the network 330. The processor 312 of the user device 310 the n implements the selected action to prevent the predicted user sentiment from occurring.

In some examples, the user device 310 also comprises a user device input module 318 to receive user inputs 319. These user inputs may also be used by the machine learning model to predict a user sentiment and to determine one or more mitigating actions. It will be appreciated that other data may also be used, such as data obtained from the user device 310 relating to characteristics of one or more applications operating on the processor 312.

At least some aspects of the embodiments described herein with reference to Figures 1 -3 comprise computer processes performed in processing systems or processors. However, in some examples, the disclosure also extends to computer programs, particularly computer programs on or in an apparatus, adapted for putting the disclosure into practice. The program may be in the form of non-transitory source code, object code, a code intermediate source and object code such as in partially compiled form, or any other non-transitory form suitable for use in the implementation of processes according to the disclosure. The apparatus may be any entity or device capable of carrying the program. For example, the apparatus may comprise a storage medium, such as a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, for example, a CD ROM or a semiconductor ROM; a magnetic recording medium, for example, a floppy disk or hard disk; optical memory devices in general; etc. It is to be understood that although some of the disclosure above relates to the use of cloud computing, the implementation described is not limited to a cloud computing environment. Rather, embodiments of the present disclosure are capable of being implemented in conjunction with any other type of computing environment.

In the preceding description, for purposes of explanation, numerous specific details of certain examples are set forth. Reference in the specification to "an example" or similar language means that a particular feature, structure, or characteristic described in connection with the example is included in at least that one example, but not necessarily in other examples.

The above embodiments are to be understood as illustrative examples of the disclosure. Further embodiments of the disclosure are envisaged. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the disclosure, which is defined in the accompanying claims.

Claims (20)

1. CLAIMS1 A method (100) of mitigating a predicted user sentiment for a user device using a trained machine learning model, the predicted user sentiment occurring at a given time, the method comprising: obtaining (110) at least telemetry data associated with the user device (310), the telemetry data representing operating characteristics of the user device within a predetermined time period, the predetermined time period preceding the given time; analysing (120), using the trained machine learning model, at least the telemetry data associated with the user device, to determine a correlation between at least the telemetry data and one or more potential user sentiments; based on the correlation, determining (130) one or more proposed actions to mitigate against the one or more potential user sentiments; provide (140), to an operator (320), the one or more proposed actions; and based on a selection by the operator, initiating (150) a selected action on the user device, wherein the selected action is one of the one or more proposed actions.
2. 2. The method of mitigating a predicted user sentiment according to claim 1, further comprising : obtaining (111) user input characteristics, the user input characteristics associated with one or more inputs provided by a user to the user device, wherein, determining the one or more proposed actions comprises analysing using the trained machine learning model the user input characteristics to determine the correlation between the user input characteristics and one or more of the potential user sentiments.
3. 3. The method of mitigating a predicted user insight according to claim 1 or claim 2, further comprising: obtaining (112) application telemetry data associated with at least one application operating on the user device, wherein determining the one or more proposed actions comprises analysing using the trained machine learning model the application telemetry data to determine the correlation between the application telemetry data and one or more of the potential user sentiments.
4. 4. The method of mitigating a predicted user sentiment according to any previous claim, wherein the one or more proposed actions are based on preferences and/or characteristics associated with a user of the user device.
5. 5. The method of mitigating a predicted user sentiment according to any previous claim, wherein the predicted user sentiment for a user device is a predicted degradation of a user experience associated with a user of the user device.
6. 6. The method of mitigating a predicted user sentiment according to claim 5, wherein the predicted degradation is indicative of at least one of an increase in: an average processing time for at least one operation on the user device; an average transmission time of a message being sent from the user device; and an average interaction time with one or more hardware resources of the user device.
7. 7. The method of mitigating a predicted user sentiment according to any previous claim, further comprising storing at least the telemetry data, associated with the user device, in a storage (314, 340).
8. 8. The method of mitigating a predicted user sentiment according to claim 7, wherein the storage (340) is remote from the user device.
9. 9. The method of mitigating a predicted user sentiment according to any previous claim, wherein analysing, using the trained machine learning model, is performed by a remote device, the remote device being communicably coupled to the user device.
10. 10. A method (200) of training a machine learning model to mitigate a predicted user sentiment at a given time for a user device, the method comprising: obtaining (210) training data associated with a plurality of user devices, different to the user device, the training data comprising at least: first data (211) representing operating characteristics of the plurality of user devices within a predetermined time period, the predetermined time period preceding the given time second data (212) representing one or more mitigating actions undertaken; and a relationship (213) between at least some of the first data and at least some of the second data; and training (220) the machine learning model using the training data, the machine learning model configured to predict an occurrence of the user sentiment at the given time and determine at least one mitigating action to be implemented on the user device (310).
11. 11. The method of training a machine learning model according to claim 10, wherein the training at a further comprises at least one of user input characteristics(214) associated with one or more inputs provided by a user to at least one of the plurality of user devices, the user input characteristics having been obtained over the predetermined time period; and application telemetry data (215) associated with at least one application operating on at least one of the plurality of user devices, the application telemetry data having been obtained over the predetermined time period.
12. 12. The method of training a machine learning model according to claim 10 or claim 11 further comprising: periodically obtaining at least additional operating characteristics associated with the user device; storing at least the telemetry data in storage; and updating the machine learning model based on at least the additional operating characteristics.
13. 13. The method of training a machine learning model according to any of claims 10 to 12, wherein the training of the machine learning model is undertaken on a remote device communicably coupled to the user device.
14. 14. The method of training a machine learning model according to any of claims 10 to 13, wherein the training data is obtained from storage associated with the user device.
15. 15. The method of training a machine learning model according to any of claims 10 to 13, wherein the training data is obtained from a storage remote from the user device. 10
16. 16. A system (300) for mitigating a predicted user sentiment using a trained machine learning model, the predicted user sentiment occurring at a given time, the system comprising: a user device (310), comprising a processor (312), configured to: obtain, from storage (314), at least telemetry data representing operating characteristics of the user device within a predetermined time period, the predetermined time period preceding the given time; analyse, using the trained machine learning model, at least the telemetry data associated with the user device, to determine a correlation between at least the telemetry data and one or more potential user sentiments; based on the correlation, determine, by the processor, one or more proposed actions to mitigate against the one or more potential user sentiments; transmit, over a network (330), and via a user device networking module (316), the one or more proposed actions to the operator device (320); receive, via the user device networking module, an indication of a selected action from the operator device, wherein the selected action is one of the one or more proposed actions; and implement, by the processor, a selected action; and an operator device (320) configured to: receive, via an operator device networking module (322), the one or more proposed actions from the user device; provide, via a graphical user interface (324), to an operator using the operator device, the one or more proposed actions; receive, via an operator device input module (326), an input from the operator indicating the selected action of the one or more proposed actions; and transmit, via the operator device networking module, the indication of the selected action to the user device.
17. 17. The system for mitigating a predicted user sentiment using a trained machine learning model according to claim 16, wherein the user device is further configured to: obtain user input characteristics (319), the user input characteristics associated with one or more inputs provided by a user, via a user device input module (318) of the user device; and determine one or more proposed actions comprises analysing using the trained machine learning model the user input characteristics to determine the correlation between the user input characteristics and one or more of the potential user sentiments.
18. 18. The system for mitigating a predicted user sentiment using a trained machine learning model according to claim 16 or claim 17, wherein the user device is further configured to: obtain application telemetry data associated with at least one application operating on the processor of the user device, and determine one or more proposed actions comprises analysing using the trained machine learning model the application telemetry data to determine the correlation between the application telemetry data and one or more of the potential user sentiments. 25
19. 19. A non-transitory computer-readable storage medium comprising a set of computer-readable instructions stored thereon, which when executed by at least one processor are arranged to mitigate a predicted user sentiments for a user device using a trained machine learning model, the predicted user sentiment occurring at a given time, the instructions being arranged to cause the processor to: obtain (110) at least telemetry data associated with the user device, the telemetry data representing operating characteristics of the user device within a predetermined time period, the predetermined time period preceding the given time; analyse (120), using the trained machine learning model, at least the telemetry data associated with the user device (310), to determine a correlation between at least the telemetry data and one or more potential user sentiments; based on the correlation, determine (130)one or more proposed actions to mitigate against the one or more potential user sentiments; provide (140), to an operator, the one or more proposed actions; and based on a selection by the operator, initiate (150) a selected action on the user device, wherein the selected action is one of the one or more proposed actions.
20. 20. A non-transitory computer-readable storage medium comprising a set of computer-readable instructions stored thereon, which when executed by at least one processor are arranged to train a machine learning model to mitigate a predicted user sentiment at a given time for a user device, the instructions being arranged to cause the processor to:: obtain (210) training data associated with a plurality of user devices, different to the user device, the training data comprising at least: first data (211) representing operating characteristics of the plurality of user devices within a predetermined time period, the predetermined time period preceding the given time second data (212) representing one or more mitigating actions undertaken; and a relationship (213) between at least some of the first data and at least some of the second data; and train (220) the machine learning model using the training data, the machine learning model configured to predict the occurrence of a user sentiment at the given time and determine at least one mitigating action to be implemented by an operator.