WO2020172035A1 - Communication system using context-dependent machine learning models - Google Patents

Abstract

A computer system reflects nuances in communications that are specific to each relationship between different pairs and groups of individuals by using a different context-dependent machine learning model associated with each relationship among the individuals. Each machine learning model is "context-dependent" by virtue of being applied to and trained based on data from the interactions within a specific relationship among a small number of users, such as between a pair of individuals, or between pairs of individuals within a group, with which the model is associated. While such models are thus individualized and relationship-specific, there generally is insufficient data available to train such models using conventional techniques. In some instances, such as in the context of a new relationship, there can be little or no data from prior interactions with which to train the model for that relationship. To address this problem, several techniques are applied in combination to gather more data and to continually train the models based that data.

Description

COMMUNICATION SYSTEM USING

CONTEXT-DEPENDENT MACHINE LEARNING MODELS

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a non-provisional application of both U.S. provisional patent application 62/807,009, filed February 18, 2019, and entitled“Communication System using Context-Dependent Machine Learning Models”, and U.S. provisional patent application 62/807,017, filed February 18, 2019, and entitled“Communication System using Context- Dependent Machine Learning Models”, both of which are hereby incorporated by reference.

BACKGROUND

[0002] There have been several attempts to create computer systems that interact

conversationally with humans. In general, the goal of such computer systems has been to make the computer system appear to be human when providing natural language responses to natural language inputs. While there are many possible implementations, such a computer system generally includes a trained model that generates a natural language response for any given natural language input. The model generally is trained using a large data set of possible inputs and responses, with the intent of providing the best possible response to any likely input. These computer systems are used in several applications, such as“chat hots” for online customer support.

[0003] Computer systems also have been used to assist humans in interacting with each other. Such applications include, but are not limited to, managing interactions through electronic mail or other messaging and communication platforms, intelligent calendaring and scheduling systems, and intelligent task management. For example, rule-based computer systems may be used to automatically process electronic mail or other messages. As another example, a computer system may have access to individuals’ calendars and identify times when the individuals are jointly available for a meeting. If a machine learning model is used in these contexts, such a model generally is trained using a large data set of possible inputs and outputs, for a large number of users, with the intent of providing the best possible output to any likely input.

SUMMARY

[0004] A problem with typical attempts to apply machine learning and natural language processing to communications such as electronic mail and messaging is the assumption that a single machine learning model can apply to all human interactions. Another problem is the assumption that the machine learning model should enable the computer to provide accurate responses to inputs from a population of users. Because of these two assumptions, the machine learning model is trained with a large data set of natural language inputs and natural language responses, which is computationally expensive. The resulting model also does not reflect nuances in communications that are specific to each relationship between different pairs and groups of individuals. Looked at another way, such a model is applicable only to those communications which are generally uniform across all individuals.

[0005] To build a computer system that can reflect nuances in communications that are specific to each relationship between different pairs and groups of individuals, a context- dependent machine learning model is associated with each such pair of individuals or pairings of each individual to a group. Each machine learning model is“context-dependent” by virtue of being applied to and trained based on data from the interactions within a specific relationship among a small number of users, such as between a pair of individuals, or between pairs of individuals within a group, with which the model is associated.

[0006] While such models are thus individualized and relationship-specific, there generally is insufficient data available to train such models using conventional techniques. In some instances, such as in the context of a new relationship, there can be little or no data from prior interactions with which to train the model for that relationship. To address this problem, several techniques are applied in combination to gather more data and to continually train the models based that data.

[0007] First, the constraint that an output of a machine learning model is accurate and prescriptive, in terms of providing a natural language response, is relaxed: the output is instead predictive and assistive in terms of predicting an action to be performed by a user. Based on the data available about the current context of a relationship, the model generates an output which is at first a set of predictions or guesses about the likely nature of the interaction and a likely action to be taken by the user. Instead of having a single best natural language output, the model provides a set of likely actions a user may take. For example, in response to an incoming message, the model may infer that the sender of the message would like to set up a meeting with the recipient, in response to which a user may suggest times to meet. The model also may surface other possible actions for the user. Initially, such other possibilities may be no suggestion from the model for an action to be performed. The model can be assistive by offering to the user a choice from among several actions, instead of prescribing a predetermined action. Initially, the model may not present any of the choices it has predicted when a confidence value for the prediction is too low.

[0008] Second, data about the interactions between individuals are processed to generate additional data to be used by the model, so that a predicted action is based on more than just the content of a message. Significantly, an individual’s actual actions performed in response to the assistive and predictive outputs to the model are feedback data that can be used to train and update the model. The interactions between the individuals also are treated as a sequence of events, such as content from messages and user actions, rather than merely message content. The additional data about the actions performed in the context of the different messages over time provides more data to the model. Other types of processing can be used to generate other types of metadata. For example, sentiment analysis can be applied to text to infer sentiment of an individual with respect to the message and such sentiment data can be an input to the model.

[0009] Third, the machine learning models employ active learning. The machine learning models generate and track predicted actions based on events in a conversation. In some instances, the computer system presents options to the user based on the predictions. The actual actions by the user, or options selected by the user, also are tracked. Because the computer system tracks the predictions and actions in the context of events, such as messages, from which the predictions are generated and to which the actions are a response, this information can be used to train the model to correlate actions with context. A user’s selections in response to presented options can be used directly to train the model to avoid false positives. If no option is presented and the user performs an action, the action and the proceeding context can be used directly to train the model to avoid false negatives. The user’s selection is a kind of feedback data which indicates how well the model predicted an action. For example, if the predicted action from the model is validated from the user’s action, the model’s prediction can be reinforced; conversely, if the predicted action is not validated by the user’s action, the model can be trained to reduce the likelihood of the same prediction in the future. In some implementations, the models can be updated in real time, i.e., in time to be applied to the next interaction between those individuals.

[0010] As a result of applying active learning to a model based on feedback generated by the user given predictive outputs from the model, the model becomes trained to the nuances of the relationship between the individuals. Also, the predictive and assistive nature of the model’s outputs evolve, becoming more prescriptive and accurate over time, but also can be adaptive to changes in the relationship over time.

[0011] Such a system can be used for scheduling events, such as meetings, on a calendar. In this context, a model can determine whether an individual is requesting a time for a meeting based on an incoming message. After it has been determined that the individual is requesting a time for a meeting, the scheduling of that meeting can be processed. A model for scheduling meetings among individuals also can be context-dependent, predictive, and evolve over time based on a user’s history of scheduling meetings with an individual or a group of individuals. While a variety of algorithms could be used to find a jointly available time, one that treats availability of time as context-dependent and predictive of a likely accepted time may provide a better user experience. Determining the intent from a message, such as an intent to schedule a meeting, can be tuned to the individual user and their relationships, since the language used by and between different people may vary.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram of an example computer system using context-dependent learning models.

[0013] Figure 2 is a block diagram of another example computer system using context- dependent learning models.

[0014] Figure 3 is a diagram illustrating example data structures in an example

implementation of such a computer system.

[0015] Figure 4 is a data flow diagram illustrating how supervised, active learning can be implemented.

[0016] Figure 5 is a flow chart describing an example operation of such a computer system.

[0017] Figure 6 is a diagram illustrating example data structures in an example

implementation of scheduling an event in such a computer system. [0018] Figure 7 is a diagram illustrating the process of scheduling an event based on calendar data.

[0019] Figure 8 is a block diagram of a computer.

DETAILED DESCRIPTION

[0020] Referring to Figure 1, a block diagram of an example implementation of a computer system using context-dependent machine learning models will now be described.

[0021] In Figure 1, a computer system 100 processes incoming and outgoing electronic messages 102. Electronic messages can be used in a text messaging system (e.g., an SMS system), electronic mail (“email”) system, direct messaging system, instant messaging system, chat system (e.g., a WHATSAPP application) or other system that transmits electronic messages between entities. The entities typically are individuals but may be groups of individuals or organizations. The electronic messages typically include text, but may include any other digital media, such as image data, sound data, etc. The computer system also may include other applications related to interactions among users, such as calendars, contacts, and calls.

[0022] As explained in more detail below, interactions between entities are treated as a sequence of events, including both content from messages and user actions, rather than merely message content. One kind of event is a message, with its content (typically unstructured data) and associated metadata. Another kind of event is an action, which is structured data representing an action performed by a user. Events can be timestamped so that they can be ordered by time to allow detection of patterns in time of actions in context of messages and other actions.

[0023] In this communication system there is a concept of a“client” and a“user”. A“user” of the computer system is an entity for which data for one or more relationships with “clients” is stored. Each one of these relationships has an associated machine learning model, which is thus context-dependent by virtue of being applied to and trained based on data from the interactions between the pair of entities with which the model is associated.

[0024] The term“user” and“client” are used differently from the terms“sender” and “receiver” of a message in the communication system - since a user (in this instance a receiver) may receive a message from the client (in this instance a sender), but in turn the user (in this instance a sender) may respond by sending a message back to the client (in this instance a receiver). Generally, the system tracks user actions in the context of the messages between the user and the client. In some implementations, client actions also may be tracked. [0025] As shown in Figure 1, for each user the computer system stores a user profile 120.

For each client with whom the user has interactions, the computer system stores a client relationship profile 122, associated with that user’s user profile 120. Thus, if the same entity has interactions with two users, that entity will have two client relationship profiles, one for each user. Similarly, if a user has interactions with two different entities as clients, that user will have two client relationship profiles, one for each client.

[0026] The user profiles and client relationship profiles are stored in a database, which can be implemented using any of a variety of database technologies. In an implementation using a no-SQL database such as a MONGODB database, these data structures can be stored as documents containing a set of pairs of field names and corresponding values, where each field name and a corresponding value are stored in the form of a respective key -value pair.

For example, a user profile or a client relationship profile can be represented as a document containing a set of key -value pairs. All users can be represented as a collection of such documents. In yet other implementations, a relational database can be used, in which the collection of documents is equivalently represented as a table of records, with each row representing a user and each column corresponding to one of the field names, and each cell in a row storing a value corresponding to the field name of the respective column. Similarly, an object-oriented database also can be used in which each user profile or client relationship profile is an instance of an object type.

[0027] In Figure 1, an incoming message 102 from a client is processed by an input processing module 104. In the example shown in Figure 1, the input processing module includes a metadata extraction module 106 and a workflow prediction module 108.

[0028] The metadata extraction module is one of the components which generate additional data about the interactions between the entities. The metadata extraction module processes an incoming message into data 110 representing a message type of event, which is added to the sequence of events representing the interactions between the entities in the client relationship profile 122. Other types of processing can be used to generate other types of metadata. For example, sentiment analysis can be applied to text to infer sentiment of an individual with respect to the message. Such metadata can be associated with the content of the message stored in the sequence of events for the client relationship. Such metadata also can be an input to the models used for the client relationship, providing more data to improve the models.

[0029] The workflow prediction module is the context-dependent machine learning model for the user and client relationship pair. While Figure 1 (and 2) illustrate one workflow prediction module, there is a different model applied for each user/client pair in the system.

To allow the system to scale more easily, there can be a single instance of a workflow prediction module that is parameterized, and the system can maintain different parameters defining the trained models for each user/client pair. The workflow prediction module receives as an input the sequence of events for the client relationship to provide a predictive output indicating an action that is likely to be performed given the current context from the sequence of events. Typically, the workflow prediction module is applied after receiving an incoming message, but also can be applied at any time to predict an action for a user. In response to an incoming message, the workflow prediction module 108 uses, as its inputs, the content and metadata of the incoming message 102, the user profile 120, the client relationship profile 122 for the entity sending the message, and the sequence of events for that relationship. As described in more detail below, the workflow prediction module outputs one or more predicted actions for the user based on these inputs. This prediction output can be stored as an event in the sequence of events for this client relationship. The output also can include a confidence value and associated context information, which can include, for example, data indicating the relevant events in sequence of events, or metadata for a message.

[0030] The predictions output from the workflow prediction module are an input to a decision module 140 (the“work flow executive”), which processes the data to determine what action the computer system will take. In some instances, if the confidence values are too low, the decision module may determine that the computer system is to take no action, and the predictions are merely stored as events. In other instances, the decision module may determine to present a proposed action to the user, in which case data about that proposed action may be sent to an application on a user’s device 180 for further action (to prompt the user and receive a response from the user). In some instances, the decision module may determine to present one or more proposed options to the user, which may be sent to the application on the user’s device which prompts the user for a response; the response in turn is sent to the computer system to be used for training the model. In yet other instances, the decision module may determine to execute a proposed action without user intervention.

[0031] The user’s device has an application that communicates with the computer system 100. This application receives data describing the proposed action and prompts the user for a response. The application receives input from the user as the user’s response, and provides data describing the response to the computer system. In some instances, the application prompts the user with respect to one or more options. The selected option is sent to the computer system, which can use such feedback for active learning of the model. The application can otherwise include the full functionality of a message application through which the user interacts with clients.

[0032] A server handler module 150 receives, as its input, data describing the decision output by the decision module. Also, data describing any action taken, or option selected, by the user through the user device also can be received as an input (152). The server handler is responsible, based on its inputs, including any from the user device and those from the decision module, to perform any selected action, to send any response (whether automatic or authored by the user) to the client, update the model implemented in the workflow prediction module, and update the user/client relationship profile with events including at least data describing the predicted actions, and events including data describing actions taken.

[0033] In Figure 2, which is similar to Figure 1 is most respects, illustrates a scenario in which one of the“clients” is also a“user” of the computer system. In this scenario, both entities have“user” profiles: the computer system maintains a first user profile for a first entity which tracks data associated with the first entity’s role as a user and maintains a second user profile for a second entity which tracks data associated with the second entity’s role as a user. The first user profile can be associated with a first client relationship profile for the second entity; the second user profile can be associated with a second, different, client relationship profile for the first entity. Such an implementation allows different models to be trained differently to reflect different behaviors of the different parties in a relationship. This implementation also maintains a uniform implementation for all users and clients.

[0034] Figure 3 illustrates example data structures in an example implementation of such a computer system.

[0035] An incoming or outgoing message is normalized into a message data structure that stores data representing the message. The message data structure has an identifier and can include: a“To” field, storing data indicating an entity to which the message is being sent; a “From” field, storing data indicating an entity from which the message is being sent; a “Body” field, storing the text content of the message; and a“Timestamp” field, storing data indicating a date and time on which the message was sent or received. Any media data in addition to the text content can be represented by data stored in a“Media” field. Examples of other data which can be stored for the message include, but which are neither required nor limiting: one or more“Tag” fields, storing one or more keywords associated with the message; one or more“Sentiment” fields, storing data indicative of sentiment associated with the message, which can be represented by, for example, a score associated with a sentiment; and one or more other“Metadata” fields which can store, for example, data indicating whether the message is spam, or other data.

[0036] A normalized calendar event is generated by importing and processing calendar data from one or more calendars for users. A calendar event data structure has an identifier and can include: a“user ID” field storing data indicative of a user identifier that identifies the user; a“time zone” field storing data indicative of a time zone associated with the event; a “start time” field and an“end time” field storing data indicative of the date and time for the event; a“title” field storing data indicative of a title or subject for the event. Examples of other data that can be included in a calendar event data structure include, but are not limited to, indications of attendees, location, and notes for the event.

[0037] An action data structure is generated for a prediction action from the model, or for an action performed by the user or computer system, such as a response to a client. Each action has an identifier and stores data indicating a type, selected from among a plurality of types, a time stamp, and context data. The plurality of types represents the set of possible types of actions that the model can predict. This set can be expanded, reduced, and modified to allow the scope of actions that the models can predict to be adaptive and dynamic. The context data can be data indicating the message, or sequence of messages and actions, to which the action is related. The user prompts presented to the user through the user’s device, if any, as generated by the workflow prediction model and selected by the server handler model, also are included in the action data. The user’s response to the prompt, if any, also is stored.

[0038] A conversation data structure is a data structure that defines a conversation between a user and a client as a sequence of events occurring between two entities, where an event is a message or an action. Each conversation has a conversation identifier. Each event can be represented by data indicating an identifier either a message or an action, and a time stamp associated with the event.

[0039] Note that actions can include activities both inside and outside of the computer system 100. As a first example, when the computer system sends a prompt to the user device, the user can choose to accept an action, or not, or select from among a set of possible actions.

The system records the action of presenting the options and also records what the user’s response is to those options. As a second example, as described in more detail below, the prompt to the user can include one or more proposed times for a meeting, for which the computer system generates and stores probabilities of acceptance. The suggested times are stored as a predicted action. The user’s choice also is recorded as an action. As a third example, when a user sends a message to a client, this act of sending a message is recorded as an action, and the content of the message also is stored as a message. As a fourth example, when a user adds a calendar entry associated with a client as an attendee, this action can be recorded.

[0040] By recording both messages and actions within a conversation data structure, the history of interaction between the user and the client can be processed by machine learning models to predict actions for the user based on context.

[0041] Turning now to Figure 4, an overview of the machine learning process, underlying the predictions made by the computer system, will now be described. First, considering the object of the machine learning model, it generates a predicted action based on the interactions within a specific relationship between the user and one or more other clients (see 400).

While the machine learning model can be applied in response to an incoming message from a client, it can be applied periodically, or in response to any other event, or even in response to the absence of an event that was otherwise predicted to occur. For example, the system may predict that the user sends a client a message on a particular day, time, or frequency.

[0042] The predicted action which is output by the model provides, when applied to an incoming message, is related to an inferred intent of the message and has a corresponding score representing the confidence in that prediction. The inferred intent can be derived using a natural language processing tool such as the Language Understanding (LUIS) tool from Microsoft Corporation. The inferred intent has a corresponding action that a user can perform in response to the incoming message. For example, the incoming message,“When are you free next?”, can be processed to correspond to an intent of“Suggest Times”. The “Suggest Times” intent can correspond to an action of scheduling a meeting.

[0043] Given the predicted action from the model, the machine learning process uses verification (see 402) of that prediction to improve learning. Such verification is

implemented by surfacing to the user one or more specific suggestions, which have a high confidence, to receive a user’s acceptance of a suggestion or selection from among a set of suggestions. In response, the user’s input provides a labeling for the content of the message as matching or not matching the predicted action corresponding to the inferred intent of the message. Thus, the system can have, for a predicted action, a corresponding prompt to verify that prediction. As an example, a prompt, e.g.,“Would you like to suggest times?”, can be presented to the user. The user’s answer is treated as a label for whether the data representing the incoming message (e.g.,“When are your free next?”) is properly associated with the inferred intent (e.g.,“Suggest Times”), and the corresponding predicted action. [0044] The combination of the data representing the message, the inferred intent, predicted action, and user’s response as a label (see 404) can be used as a data set (see 408) for implementing supervised and active learning of a machine learning model, which can be understood as a natural language processing-based classifier (410), where the classification is the predicted action. A K-fold cross-validation (450) can be applied to the trained classifier to validate the training. The updated model can be used to process further incoming messages. Notably, by having a separate model for each user/client relationship, the predictions made by each model evolve based on the nuances of the interactions between a particular user and client.

[0045] Some additional examples illustrate how such machine learning models can be used.

[0046] In the context of a real estate agent communicating with clients, the agent may frequently provide information about a newest listing, available open houses, or other information about a listing as requested in a message from a client. The machine learning model may learn to predict that the agent will send information about a first listing in response to such a message. However, at some point, a second listing will become available and the agent will start to information about the second listing. The machine learning model may prompt the agent to send the information for the first listing, in response to which the agent may initially authorize such information to be sent. The model will reinforce this prediction. However, after the second listing is available, a prediction to send the

information for the first listing will be rejected by the agent, and instead the agent will send the information for the second listing. The model, through active learning based on the user’s actions in response to the predicted action, will learn that the second listing is to be sent.

Thus, the model adapts over time to changes in the relationship between the user and the client.

[0047] When data for a model is sparse, such as during the early stages of a relationship where the conversation data structure is sparsely populated, the system can use models for other, similar user/client relationships to make some predictions. For example, a user may indicate, by way of a list, data tags, or other data structure, that certain clients are similar. As another example, the system can determine that clients are similar based on their conversation histories. As another example, the system can predict that clients are similar and suggest such a prediction to the user. User similarity also may be used. For example, if a first user has a new relationship with a client, and a second, similar user has an existing relationship with the same client, then the models for the second user’s relationship with the client may be used to generate predictions for the first user’s relationship with that client. Similarity among users can be determined based on data tags, organizational role, similarity of conversational histories, a list, or other techniques. Finally, similarity of both users and clients may be used to identify other models to make predictions a user/client relationship for which the conversation history is sparse.

[0048] Turning now to Figure 5, a flowchart describing an example implementation for operation of the computer system of Figures 1-4 will now be described.

[0049] In Figure 5, some initial data processing is performed on data (500) available for a user, such as any history of messages, call activity, call transcripts, or other data about interactions with others. Such existing message data can be analyzed (502) to identify patterns, intent, sentiment and other data to help with initial building of models for the user.

[0050] Other data that can be pre-processed includes existing calendar data (504) and contact data (506). Calendar data can be input from a calendar and normalized (508) into the calendar data structure shown in Figure 3. Similarly, contact information from various sources of contacts for the user can be imported and normalized (510) into a contact data structure. The calendar data also can be analyzed to identify patterns and reliability metrics.

[0051] The interaction data with each client for a user is processed into a combination of structured data (512) and probabilistic data (514) for that user/client relationship. The structured data can include contact information and conversation data. Other structured data can include, customer relationship management data, current context, interaction patterns, tagged interactions, etc. Probabilistic data includes data for an individual, group and a populace, such as predicted future activities, predicted availability and prediction results.

[0052] In response to an inbound message (516) from a client, the computer system generates (518) workflow predictions. A workflow prediction is one or more actions that a user may take in response to the message based on the current context-dependent model associated with this user/client relationship. The predicted actions (520) have associated context and probability or confidence.

[0053] In some instances, a user may initiate (522) their own action in response to an incoming message. The computer system uses data describing this action to update (524) the client relationship profile. The computer system also can use the predictions (520) and the user action (522) to update the prediction results.

[0054] In some instances, such as when the computer system has proposed times for scheduling a meeting, these times can be suggested (530) to a user. The user’s response to these suggestions can be used to update (526) the prediction result. If the user accepts the suggestion, a suggestion (534) can be sent to a client. The client’s response (536) to the suggestion also can be used to update (528) the prediction result.

[0055] The predicted actions can be any action that the computer system can be directed to take by the user and that the computer system can track as an event associated with a user/client relationship. The computer system can have a list or other data structure that tracks those actions that are tracked and that can be predicted. As a result, the system can be expandable, adaptive and dynamic, in the kinds of actions that it can track and predict.

[0056] By way of example, predicted actions can include, but are not limited to, generating outbound message content, tagging a client with data (likely to leave a positive review, to be a qualified lead, status or other data in a CRM tool, favored client status), setting a meeting duration, sending a reminder, setting reliability factors for scheduling, setting and updating availability preferences for scheduling, predicting likelihood of client action in response, identifying frequency of message, call, or other interaction and detecting variations from that predicted frequency, identifying other patterns and detecting variation from those patterns, proposing follow-up actions when variations from identified patterns are detected, flagging a communication for attention based on sentiment or other metadata, flagging a client for attention based on detected inaction, determining type, content, or action to take based on customer history, sending unavailability information about the user to the client based on conversation history with the client, including notification and away mode patterns, in addition to calendar data, setting a payment amount for a transaction, modifying scheduling recommendations for an event, prioritizing responses based on data tags for clients. Such actions may occur, not only in a messaging application such as a text messaging, chat, or direct messaging application, but also in a calendar application, contact application, task management application, other communication application, or other application.

[0057] Context can include, but is not limited to, data from an incoming message, calendar data, external data sources, contact data, voice call transcripts, and various metadata derived from them such as frequency of interaction, timing of interactions (e.g., dates, days of week, times of day), patterns of such metadata, inaction (e.g., time between chat messages, emails, text messages, phone calls). The content of messages, metadata derived about the messages, and data about actions can all be processed into events stored in the sequence of events representing the conversation history for the user/client relationship.

[0058] Some example use cases for these context-dependent machine learning models include, but are not limited to: a. Use conversation history to associate incoming messages with outbound message content that we can suggest to the user over time;

b. Use relationship history to learn which clients will most likely leave positive reviews to flag those clients to the user;

c. Use context from conversations, calendar, and external data sources to identify prospective qualified leads;

d. Use context from messages to learn likely workflow parameters, e.g., a quick consult is likely a different meeting duration for scheduling;

e. Use context to learn how reliable or unreliable a client is, present a measure of this reliability to the user, and take the reliability into account in workflows such as scheduling and aggressiveness of meeting reminders;

f. Use relationship context to predict what stage a client is in and suggest

appropriate adjustments in client records to user;

g. Use conversation history to assess when user is likely to respond and update availability preferences per relationship;

h. Use conversation patterns including call and text cadence as well as customer value to determine best times to follow up;

i. Use phone data and conversation timing as well as calendar data to populate when a user is most likely to be away and unable to respond to client inbound messages;

j . Use inaction such as time between opening a chat and sending a reply, to predict typical response patterns and whether an inbound message should be flagged for attention;

k. Use conversation history to determine what message or content or action to take in the next reply to the customer;

l. Use conversation context to determine when and how much money is meant to be transacted in a payment flow;

m. Use conversation context to update recommendations for scheduling

recommendations, or, similarly, use reply context to determine what times might need to be adjusted or selected;

n. Use independent relationship flags, both system-created and user-created, to prioritize certain responses for certain recipients, e.g., recommending favored times for clients determined to be more important; and o. Use notification and away mode patterns to determine preferences for response and availability that informs future responses.

[0059] In some implementations, the workflow prediction module can include a calendaring tool that provides suggested times for a meeting. In one implementation of such a calendaring tool, additional data structures, such as shown in Figure 6, are used to support generating suggestions for calendar events.

[0060] In this example implementation, for each entity who will participate in the event to be scheduled, an attendance prediction profile is generated, in a manner described below.

Generally, the data stored in the attendance prediction profile data structure provides a likelihood that, for any given time span in a range of time for which the event is to be scheduled, the entity will accept that time span for the event. The granularity of the time spans to which probabilities are associated can be a setting or derived based on the context of the conversation. The time span for the event can be a setting in the computer system or may be determined from the context of the conversation. A group scheduling profile is a data structure that represents a combination of the likelihoods generated from the attendance prediction profiles of the group. A scheduling recommendations data structure stores data representing the recommended times to be proposed for scheduling a meeting.

[0061] In one implementation, the attendance prediction profile data structure includes a user identifier or a client relationship identifier to associate it with a particular user or client relationship of that user. Notably, a different profile is generated for each client relationship a user has, because that user may have different availability depending on the person with whom an event is to be scheduled.

[0062] A set of vectors is then defined, each generally having a value, representing a probability, in the range between 0.0 and 1.0; values in the vector can be initialized to“-1” or other known value to indicate whether the value has been affected by any of the processing steps. In such an implementation, with the various availability, reliability, and other information are represented as vectors, digital signal processing wave manipulations in continuous space can be applied to the vectors, allowing efficient mathematical processing of the information. In one implementation, the probability values associated with specific terms for availability can be predetermined, such as 0.1 for“definitely cannot”, 0.3 for“usually cannot”, 0.4 for no preference, 0.7 for“usually can”, and 0.9 for“definitely can”.

[0063] An initial vector is a vector that represents the general availability of the user to the general population for the given range of time. This initial vector can be based on an interpretation of the user’s calendar. A context vector provides a probability based on context, e.g., whether the event is for a group, or other metadata about the event, such as indications of urgency. Other vectors, described in connection with the scheduling algorithm below, include a pairwise vector, filter vector, member vector, and flexibility vector.

[0064] In one implementation, the group scheduling profile data structure includes an attendance prediction profile vector generated for each of N entities being scheduled for the event. These N vectors are combined into a composite vector. A contribution vector is generated from the N flexibility vectors.

[0065] In one implementation, the scheduling recommendations data structure includes data describing a set of one or more recommendations, preferably ordered. Each recommendation is defined by data describing a time span, the attendance probability and the summarized contribution vector score for that time span.

[0066] Turning now to Figure 7, a process for generating a proposed meeting time will now be described. In this process, the scheduling is treated as a problem of predicting a likely time that will be accepted by all parties to the meeting, and not necessarily identifying a time on a calendar at which all of the parties are designated as“free”. By treating the scheduling problem in this way, the“reliability” of the calendar and user preferences can be factored into the selection. For example, the conversation history between a user and a client may indicate that one of the entities will cancel an existing meeting with another party to schedule a meeting for this relationship. Also, previous interactions, such as calendar events and the conversation history may indicate that there is a pattern in which the parties generally meet, such as weekday mornings. As noted in Figure 6, a number of probability distributions for time spans are determined and combined to identify a set of most likely times that would be accepted for the event.

[0067] In Figure 7, the process begins by determining (700) the attendance prediction profile representing a user’s predicted availability. This process includes first determining general availability, noted in Figure 6 as an initial vector. This can be determined from declared office hours, ingesting external calendars, and the like, and inferring and weighting reliability of calendars. For example, events in which a user or client is the sole participant likely represent tasks to be performed and not meetings and may be less reliable as unavailable times. Other metadata available from the calendar may indicate that some times are not reliable as“available”, such as when those times are outside of office hours. Any

communication about availability specifically around the proposed event also can be taken into account. This results in the user’s member vector. Another vector that is computed is the flexibility vector, also called a contribution vector, which indicates the user’s likelihood of changing their mind if they are otherwise unavailable.

[0068] In some implementations, the original availability vector can be generated by translating a natural language input into a mathematical representation of a time availability. The natural language input describing availability, in English for the purpose of illustration, typically takes a form such as,“I usually cannot do Tuesdays, all day”, or“I definitely can do November 1, 2018, except between 3pm and 5pm”. Such statements are first normalized, and then translated into a mathematical form. The natural language processing can implement conflict resolution by prioritizing statements such that terms which are more specific (e.g., “definitely” or“between 3pm and 5pm”) or more definitive are given precedence over terms that are more general and less definitive (e.g.,“usually” or“all day”). The availability of a time slot is set based on the precedence of the terms associated with that time slot. The probability for the time slot also can be dependent on terms used to describe the availability, such as noted above. Unresolvable conflicts can be presented to a user to resolve.

[0069] Next, the process accounts (702) for client preferences. This information can be obtained from explicit statements in messages from the client, inferred preferences from past conversation histories (both with this user and other users), patterns in availability for this client (and optionally similar clients). This data can be determined for all clients currently being scheduled for the event. The user’s availability information is combined with the client availability information into a combined prediction matrix.

[0070] At 704, given the combined prediction matrix, all possible times are found, calculating optimality for each candidate time by factoring in the availability, likely unstated flexibility, preference strength and cost of rescheduling. In some implementations, a Monte Carlo simulation of“moves” for each client still pending scheduling can be performed to determine a top three options to present to this client, which provides a globally optimal schedule. In general, the probabilities for each user that it may be able to attend any given time are averaged across all users. The flexibility scores also are averaged over time. The time spans having the highest flexibility scores and highest probabilities of availability are most likely to be accepted by most if not all of the users.

[0071] At 706, the top three options are then presented to the user for approval. The user’s approval provides feedback for the training the scheduling algorithm and can be stored as an event in the conversation history. The three options are then presented to the client, at 708, and the client either selects from among them or rejects them. Both confirmations and rejections are recorded to update a client’s preference model. The incoming message from the client in response to the proposed time also is processed to infer its intent and the inference model for this client can be updated based on the feedback received.

[0072] In one implementation, after times are suggested, the probability that the suggested times are available can be decreased within the user’s member vector. After the client chooses an option, declines all provided options, or declines all options and offers a different option, the normalized calendar entry data can be updated with any actual scheduled meeting which blocks out the time moving forward for the user. Additionally, the option chosen and the options not chosen are updated within the combined user/client matrix to reflect the actual selected times. The flexibility vector can be updated based on the general level of predicted schedule constraints versus the overall time block to be scheduled.

[0073] Having now described an example implementation, Figure 8 illustrates an example of a computer with which components of the computer system of the foregoing description can be implemented. This is only one example of a computer and is not intended to suggest any limitation as to the scope of use or functionality of such a computer. The system described above can be implemented in one or more computer programs executed on one or more such computers as shown in Figure 8.

[0074] The computer can be any of a variety of general purpose or special purpose computing hardware configurations. Some examples of types of computers that can be used include, but are not limited to, personal computers, game consoles, set top boxes, hand-held or laptop devices (for example, media players, notebook computers, tablet computers, cellular phones including but not limited to“smart” phones, personal data assistants, voice recorders), server computers, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, networked personal computers, minicomputers, mainframe computers, and distributed computing environments that include any of the above types of computers or devices, and the like.

[0075] A computer 800 includes a processing system comprising at least one processing unit 802 and at least one memory 804. The processing unit 802 can include multiple processing devices; the memory 804 can include multiple memory devices. A processing unit 802 comprises a processor which is logic circuitry which responds to and processes instructions to provide the functions of the computer. A processing device can include one or more processing cores (not shown) that are multiple processors within the same logic circuitry that can operate independently of each other. Generally, one of the processing units in the computer is designated as a primary processor, typically called the central processing unit (CPU). A computer can include coprocessors that perform specialized functions such as a graphical processing unit (GPU).

[0076] The memory 804 may include volatile computer storage devices (such as a dynamic or static random-access memory device), and non-volatile computer storage devices (such as a read-only memory or flash memory) or some combination of the two. A nonvolatile computer storage device is a computer storage device whose contents are not lost when power is removed. Other computer storage devices, such as dedicated memory or registers, also can be present in the one or more processors. The computer 800 can include additional computer storage devices (whether removable or non-removable) such as, but not limited to, magnetically-recorded or optically -recorded disks or tape. Such additional computer storage devices are illustrated by removable storage device 808 and non-removable storage device 810. Such computer storage devices 808 and 810 typically are nonvolatile storage devices. The various components are generally interconnected by an interconnection mechanism, such as one or more buses 830.

[0077] A computer storage device is any device in which data can be stored in and retrieved from addressable physical storage locations by the computer by changing state of the device at the addressable physical storage location. A computer storage device thus can be a volatile or nonvolatile memory, or a removable or non-removable storage device. Memory 804, removable storage 808 and non-removable storage 810 are all examples of computer storage devices. Computer storage devices and communication media are distinct categories, and both are distinct from signals propagating over communication media.

[0078] Computer 800 may also include communications connection(s) 812 that allow the computer to communicate with other devices over a communication medium.

Communication media typically transmit computer program instructions, data structures, program modules or other data over a wired or wireless substance by propagating a signal over the substance. By way of example, and not limitation, communication media includes wired media, such as metal or other electrically conductive wire that propagates electrical signals or optical fibers that propagate optical signals, and wireless media, such as any non- wired communication media that allows propagation of signals, such as acoustic,

electromagnetic, electrical, optical, infrared, radio frequency and other signals.

[0079] Communications connections 812 are devices, such as a wired network interface, or wireless network interface, which interface with communication media to transmit data over and receive data from signal propagated over the communication media. [0080] The computer 800 may have various input device(s) 814 such as a pointer device, keyboard, touch-based input device, pen, camera, microphone, sensors, such as

accelerometers, thermometers, light sensors and the like, and so on. The computer 800 may have various output device(s) 816 such as a display, speakers, and so on. Such devices are well known in the art and need not be discussed at length here.

[0081] The various computer storage devices 808 and 810, communication connections 812, output devices 816 and input devices 814 can be integrated within a housing with the rest of the computer, or can be connected through various input/output interface devices on the computer, in which case the reference numbers 808, 810, 812, 814 and 816 can indicate either the interface for connection to a device or the device itself as the case may be. The various modules, tools, or applications, and data structures and flowcharts implementing the methodology described above, as well as any operating system, file system and applications, can be implemented using one or more processing units of one or more computers with one or more computer programs processed by the one or more processing units. A computer program includes computer-executable instructions and/or computer-interpreted instructions, such as program modules, which instructions are processed by one or more processing units in the computer. Generally, such instructions define routines, programs, objects, components, data structures, and so on, that, when processed by a processing unit, instruct or configure the computer to perform operations on data, or configure the computer to implement various components, modules or data structures.

[0082] In one aspect, an article of manufacture includes at least one computer storage medium, and computer program instructions stored on the at least one computer storage medium. The computer program instructions, when processed by a processing system of a computer, the processing system comprising one or more processing units and storage, configures the computer as set forth in any of the foregoing aspects and/or performs a process as set forth in any of the foregoing aspects.

[0083] Any of the foregoing aspects may be embodied as a computer system, as any individual component of such a computer system, as a process performed by such a computer system or any individual component of such a computer system, or as an article of manufacture including computer storage in which computer program instructions are stored and which, when processed by one or more computers, configure the one or more computers to provide such a computer system or any individual component of such a computer system. [0084] The subject matter defined in the appended claims is not necessarily limited to the specific implementations described above. The specific implementations described above are disclosed as examples only.

[0085] What is claimed is:

Claims

1. A computer system, comprising:

a machine learning model having inputs receiving a conversation data structure corresponding to interactions between a user and a client, the conversation data structure including a sequence of events, the events including at least messages and actions, the machine learning model providing predicted actions in response to the inputs;

a server handler tracking the predicted actions of the machine learning model and actual actions of the user; and

a training module training the machine learning model based on the tracked predicted actions and actual actions.

2. The computer system of claim 1, wherein the training module trains the machine learning model further based on messages between the user and the client in the conversation data structure.

3. The computer system of claim 1, wherein the computer system supports a plurality of users and a plurality of clients, and further comprising a different machine learning model for each different relationship between a user of the plurality of users and a client of the plurality of clients.

4. The computer system of claim 1, further comprising:

a decision module having inputs receiving the predicted actions from the machine learning model and sending a prompt to a user device requesting selection from among the predicted actions;

the server handler receiving data from the user device indicating a selection from among the predicted actions and storing the selection as actual action related to the predicted actions; and

the training module training the machine learning model based on the selection related to the predicted actions.

5. The computer system of claim 1, further operative to:

given a first machine learning model for interaction between a first user and a first client, identifying a second machine learning model for interaction between a second user and a second client; and using the second machine learning model to generate a predicted action for the first user for interaction with the first client.

6. A computer system comprising:

a machine learning model having inputs receiving a plurality of calendar data structures for at least one user and at least one client, the machine learning model providing a prediction of a likely time that will be accepted by the at least one user and the at least one client in response to the plurality of calendar data structures;

a server handler tracking the predictions of the machine learning model and actual accepted times by the at least one user and the at least one client; and

a training module training the machine learning model based on the tracked predictions and actual accepted times.