HK1260681B - Computer-implemented method, system, and readable medium for security management - Google Patents

Description

Computer-implemented method, system, and computer-readable medium for security management

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and the benefit of U.S. Non-Provisional Patent Application No. 15/441,154, filed February 23, 2017, entitled “TECHNIQUES FOR DISCOVERING AND MANAGING SECURITY OF APPLICATIONS” [Attorney Docket No. 088325-1032871 (185500US)], which claims priority to and the benefit of each of the following patent applications:

1) U.S. Provisional Application No. 62/300,715, filed on February 26, 2016, entitled “Systems and Methods for Discovering and Monitoring Unsanctioned Enterprise Assets” [Attorney Docket No. 088325-1032870 (185501US)]; and

2) U.S. Provisional Application No. 62/460,716, filed on February 17, 2017, entitled “Systems and Methods for Discovering and Monitoring Unsanctioned Enterprise Assets” [Attorney Docket No.: 088325-1039197 (185502US)].

The entire contents of each of the above-identified patent applications are incorporated herein by reference for all intents and purposes.

Background Art

Organizations may rely on a variety of technical devices, software, hardware, and/or computing services to implement computing environments (e.g., enterprise computing environments). These computing environments are increasingly being implemented as or using "cloud" environments. A "cloud" environment can refer to a collection of locally and remotely hosted computing resources and systems. The term "cloud computing" refers to various aspects of distributed computing over a network. Cloud computing environments can implement a variety of service models, including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), and Network as a Service (NaaS). "Cloud" can also refer to a single service provider's data storage and client applications. Many applications can implement cloud computing environments to enable devices to gain additional functionality or capabilities beyond those available on the device itself. Such applications can be implemented using one or more service providers (also referred to herein as "providers"), each of which has one or more service provider systems (also referred to herein as "provider systems") that utilize one or more computer systems. Examples of such service providers include companies such as Box, Dropbox, Microsoft, Docusign, Salesforce, Oracle, and Amazon. Each service provider can offer many different applications or functionality, enabling access to applications and/or data as cloud-based services.

Reliance on computing environments leads organizations to extensively use both authorized and unauthorized applications. Authorized applications are applications that are registered or known to the organization. In some cases, authorized applications may be distributed by the organization. Unauthorized applications are applications that are unknown and/or not associated or registered with the organization. Unauthorized applications can include applications that operate independently of other applications as well as third-party integrated applications that are integrated as plug-ins or add-ons into approved (IT-managed) applications. Whether authorized or unauthorized, many applications pose significant security risks to an organization's computing environment. Security risks include unsecured exposure to the organization's private network or access to private and confidential data that should be restricted through security controls. Applications that pose security risks may or may not be under the organization's management. As such, these applications may operate in a "shadow" or hidden manner, unknown and/or not overseen by the organization for security controls. Furthermore, unknown application usage can lead to inefficient and overused computing resources, such as bandwidth and data storage. Undetected usage can impact performance and access to critical resources within the organization's computing environment.

Applications operating in an unregulated manner can be accessed from service providers without authorization. For example, a salesperson might use an unapproved file-sharing app on their mobile device to share a spreadsheet with their team members for collaboration, rather than sending it via email. While using such applications can improve productivity, it can also create security risks and compliance issues within the organization. For example, if the application is not secure enough, confidential documents containing business-sensitive information could be vulnerable to information leakage. Because these applications have not been evaluated by the organization, they are not prepared to take action against security vulnerabilities. Furthermore, some seemingly useful applications may intentionally or unknowingly distribute adware or even malware. Many organizations attempt to block such applications or websites, but this can cause frustration among employees due to the impact on productivity. Furthermore, employees attempt to circumvent such barriers by, for example, using external VPN services, mobile data services, and so on. However, managing an organization's computing environment requires visibility into all applications in use so that they can proactively monitor and control suspicious or malicious applications.

Summary of the Invention

The present disclosure generally relates to managing security in computing environments and, more specifically, to techniques for discovering and managing applications in an organization's computing environment. These techniques can enable an organization to monitor and manage access to applications to minimize security threats and risks in the organization's computing environment. Discovering the use of applications can enable an organization to effectively monitor and manage the efficiency and consumption of resources, thereby enhancing the performance of the organization's computing environment.

Security monitoring and control systems (also referred to as "security systems" and "security management systems") can discover the use of applications within a network or organization. Various sources, including but not limited to third-party data sources, network traffic, and service provider systems, can be utilized to identify unique applications that are accessed within the organization's network. The security monitoring and control system can be implemented in a distributed manner, including agents on the network to discover application usage. The security monitoring and control system can communicate with distributed computing systems, such as multiple service provider systems (e.g., cloud service provider systems), to access data about applications used on devices for the organization. The security monitoring and control system can obtain network data about network traffic to identify unique applications. These techniques can provide deep visibility into the activities of applications used in the organization, which can help detect anomalies or emerging threats with respect to application usage and user behavior in the organization's computing environment.

The security monitoring and control system may perform analysis and correlation, including using one or more data sources, to determine information about the application. The information may include organizational information about the application provider. The information may include security information about security risk indicators for the application. The information may include one or more indicators (e.g., values) of characteristics related to the use of the application, such as security aspects. Information about the application may be used to calculate a security measure for the application ("application risk score") and the user ("user risk score"). A measure of security may be calculated using one or more pieces of information (e.g., indicators) in combination with a weighted value attribute for each piece of information. The score may be analyzed with respect to a security policy to determine security threats posed by the application or user based on the use of the application.

In some embodiments, a graphical interface may be provided to a user (e.g., a security administrator) to view information about the use of an application. This information may provide details about the service provider of the application and/or a visualization of different types of security risks, and may indicate a measure of the severity of each security risk. The graphical interface may be interactive to configure remedial actions to be performed based on the security policy regarding each security risk. For organizations that have difficulty identifying and managing security risks through unknown application usage, the graphical interface may enable the organization to efficiently and reliably discover the use of all (if not most) applications in a computing environment in order to minimize security risks and maximize resource consumption of computing-related resources. In at least one embodiment, the security monitoring and control system may be configured to assess the risks associated with the use of an application to automatically determine the severity of the risk. Based on the severity of the risk, the security monitoring and control system may execute one or more instructions to configure the access allowed to the application, whether that access is denied or restricted.

In some embodiments, a security monitoring and control system (also referred to as a security system or security management system) may include a computer system and may be configured to implement the methods and operations disclosed herein. The computer system may include one or more processors and one or more memories accessible to the one or more processors and storing one or more instructions that, when executed by the one or more processors, cause the one or more processors to implement the methods and/or operations disclosed herein. Still other embodiments relate to systems and non-transitory machine-readable tangible storage media that employ or store instructions for the methods and operations disclosed herein. In some embodiments, a non-transitory computer-readable medium including instructions stored thereon may be implemented such that when the instructions are executed on the processor, the methods disclosed herein may be performed. In some embodiments, a system is disclosed herein comprising: one or more processors; and a memory accessible to the one or more processors, wherein the memory stores one or more instructions that, when executed by the one or more processors, cause the one or more processors to perform the methods disclosed herein. In some embodiments, a system is disclosed that includes an apparatus for performing any method disclosed herein.

In at least one embodiment, a computer-implemented method in a computer system of a security management system is disclosed. All steps may be performed by the security management system. The method may include obtaining data about a user's network activity on an organization's network. The method may include using the data about the network activity to identify applications that the user has accessed on the network. The method may include using the data about the network activity to determine access information about the network activity corresponding to the application that the user has accessed. The method may include using the access information to search for domain information about the application. The method may include determining security information about the application. The method may include using the security information to calculate a measure of security for the application that has been accessed. The method may include performing remedial actions for the application by applying a security policy based on the measure of security.

In some embodiments, the security information includes a first value that is a first indicator of a first security threat to the application and a second value that is a second indicator of a second security threat to the application. The first indicator can be obtained from a first data source. The second indicator can be obtained from a second data source.

Calculating the measure of security may include calculating a first weighted value based on multiplying the first value by the first weighted value; calculating a second weighted value based on multiplying the second value by the second weighted value; calculating a weighted sum based on the sum of the first weighted value and the second weighted value; and calculating a weighted sum based on the sum of the first weighted value and the second weighted value. The measure of security may be a value calculated by dividing the weighted sum by the weighted sum. In some embodiments, the first weighted value is different from the second weighted value. In some embodiments, the first value is different from the second value.

In some embodiments, obtaining data about network activity includes obtaining network data from one or more network devices on the network. The network can be protected within the organization's computing environment. The computing environment can be protected from public network hazards.

In some embodiments, the method may include determining organizational information for an application and generating a graphical interface displaying information about the application, wherein the information about the application is displayed based on the organizational information and a measure of security calculated for the application. The graphical interface may indicate a remediation action to be performed for the application. In some embodiments, the organizational information is about an entity providing the application, and the organizational information may indicate one or more attributes about the application.

In some embodiments, the obtained data is used for communication on a network. Identifying the application may include processing the data to identify a portion of the data corresponding to a request for access to an application by a user. The portion of the data may indicate application information about the request for the application. The application information may be used to identify the application as being accessed by the user. In some embodiments, the portion of the data may be used to determine access information about network activity corresponding to the application. The access information may indicate a timestamp of the network activity of the application, an IP address of the system providing the application, a media access control (MAC) address of a device used to access the application, and user information about the user. In some embodiments, the access information indicates the IP address of the system providing the application. Searching for domain information includes performing a query on domain information corresponding to a domain hosting the application based on the IP address of the first application. In some embodiments, the access information indicates source information of the application, the source information indicating a location of the application provided by the host. Searching for domain information may include sending a request for a certificate for the application to the host based on the source information of the application.

In some embodiments, applying the security policy based on the measure of security includes determining whether the measure of security meets a risk threshold for the application. A remedial action may be configuring the network to prevent the application from being accessed by users on the network.

In some embodiments, the obtained data further relates to a network of a plurality of users who are tenants of the organization on the network. The plurality of users may include the user. The remedial action is to prevent the plurality of users from accessing the application.

In some embodiments, the remedial action for the application includes causing a graphical interface to prompt a user to adjust a configuration operation of the application based on a security policy applied to the measure of security.

In some embodiments, a computer-implemented method at a computer system of a security management system is disclosed. All steps may be performed by the security management system. The method may include obtaining, from a first service provider system, first data regarding a first application accessed by a user from the first service provider system. The method may include obtaining, from a second service provider system, second data regarding a second application accessed by the user from the second service provider system. The method may include using the first data and the second data to determine access information for a third application accessed by the user. The method may include using the access information to search domain information regarding a provider system providing the third application. The method may include determining security information about the third application. The method may include using the security information to calculate a security measure of the accessed third application. The method may include applying a security policy based on the security measure to perform remedial actions for the third application. In some embodiments, the first application is different from the second application. The first service provider system is different from the second service provider system. The first service provider system may provide access to the first application as a first cloud service. The second service provider system may provide access to the second application as a second cloud service.

In some embodiments, the method may include determining organizational information for a third application; and generating a graphical interface displaying information about the third application. The information about the application may be displayed based on the organizational information and a measure of security calculated for the third application. The graphical interface may indicate a remediation action to be performed for the third application.

In some embodiments, the first data indicates that the user has accessed the first application through a third application. The second data may indicate that the user has accessed the second application through the third application. Determining the access information may include determining that the third application has been accessed to provide access to the first application and the second application.

In some embodiments, the security information includes a first value that is a first indicator of a first security threat to the application, and a second value that is a second indicator of a second security threat to the application. The first indicator may be obtained from a first data source. The first value is different from the second value. The second indicator is obtained from a second data source. The measure of security may be calculated by: calculating a first weighted value based on multiplying the first value by a first weighted value; calculating a second weighted value based on multiplying the second value by a second weighted value, wherein the first weighted value is different from the second weighted value; calculating a weighted sum based on the sum of the first weighted value and the second weighted value; and calculating a weighted sum based on the sum of the first weighted value and the second weighted value. The measure of security may be a value calculated based on dividing the weighted sum by the weighted sum.

The foregoing as well as other features and embodiments will become more apparent by reference to the following description, claims, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

1A , 1B, and 1C illustrate a security monitoring and control system according to an embodiment.

2 and 3 show block diagrams illustrating a security monitoring and control system according to an embodiment.

4 is a flow diagram illustrating a process for retrieving software-defined security configuration data from a cloud service, according to an embodiment.

5 is a flow diagram illustrating a process for collecting activity data from a cloud service, according to an embodiment.

FIG6 illustrates components of a security monitoring and control system for analyzing application usage, according to an embodiment.

7 and 8 illustrate block diagrams of processes for discovering and managing security for applications, according to some embodiments.

FIG9 illustrates a sequence flow diagram of a process for computing a measure of security of an application, according to some embodiments.

10-12 show flow charts illustrating processes for detecting and managing security of applications according to an embodiment.

13 illustrates a sequence flow diagram of a process for calculating a measure of safety for a user based on application usage, in accordance with some embodiments.

FIG14 illustrates a chart of measures used to assess security for a user based on application usage, in accordance with some embodiments.

15-26 illustrate interfaces for implementing a storage device as a secure device for managing access to resources, according to an embodiment.

FIG27 depicts a simplified diagram of a distributed system for implementing an embodiment.

28 illustrates a simplified block diagram of one or more components of a system environment in which services may be provided as cloud services, according to an embodiment of the present disclosure.

FIG. 29 illustrates an exemplary computer system that may be used to implement embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following description, for illustrative purposes, specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent that various embodiments can be practiced without these specific details. For example, circuits, systems, algorithms, structures, techniques, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring the embodiments with unnecessary detail. The drawings and descriptions are not intended to be limiting.

While app use can help improve productivity, it can also create security risks and compliance issues within an organization. For example, if apps aren't secure enough, confidential files containing business-sensitive information can be vulnerable to information leaks. Because these apps haven't been evaluated by organizations, they're less prepared to address security vulnerabilities. Furthermore, some seemingly useful apps can intentionally or unknowingly distribute adware or even malware.

Many organizations attempt to block such apps or websites, but this can frustrate employees due to the impact on productivity. Furthermore, employees attempt to circumvent these barriers by using external VPN services, mobile data services, and so on. The recent industry trend is to not block such services to ensure employee productivity. However, IT departments need visibility into all apps in use so they can proactively monitor and control suspicious or malicious apps.

Detecting applications, analyzing any security threats they pose, and taking corrective action can be difficult with traditional tools. A process for discovering and monitoring applications according to several embodiments of the present disclosure involves analyzing information from various data sources and correlating the data to discover application usage that can result in unauthorized disclosure of sensitive data and/or negatively impact an organization's computing environment.

Some embodiments (such as those disclosed with respect to the figures in this disclosure) can be described as processes drawn as flowcharts, flow diagrams, data flow diagrams, structure diagrams, sequence diagrams, or block diagrams. Although a sequence diagram or flow diagram may describe operations as sequential processing, many operations can be performed in parallel or concurrently. In addition, the order of the operations can be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the diagram. A process can correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, its termination can correspond to the function returning to the calling function or main function.

The processes described herein (such as those described with reference to the drawings in this disclosure) can be implemented with software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processor cores), hardware, or a combination thereof. The software can be stored in a memory (e.g., stored on a storage device, stored on a non-transitory computer-readable storage medium). In some embodiments, the processes depicted herein with sequence diagrams and flow charts can be implemented by any system disclosed herein. The specific series of processing steps in this disclosure are not intended to be limiting. Other orders of steps can also be performed according to alternative embodiments. For example, alternative embodiments of the present disclosure can perform the steps outlined above in different orders. Moreover, the individual steps shown in the drawings can include multiple sub-steps that can be performed in various orders appropriate to the individual steps. In addition, additional steps can be added or removed depending on the specific application. Those of ordinary skill in the art will recognize many variations, modifications, and substitutions.

In one aspect of some embodiments, each process in this figure of the present disclosure may be performed by one or more processing units. A processing unit may include one or more processors, including a single-core or multi-core processor, one or more cores of a processor, or a combination thereof. In some embodiments, a processing unit may include one or more dedicated coprocessors, such as a graphics processor, a digital signal processor (DSP), or the like. In some embodiments, custom circuits may be used to implement some or all of the processing units, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

I. Computing Environment for Discovery and Analysis Applications

Turning now to the figures, technology is disclosed that includes a system 100 of a security monitoring and control system 102 (also referred to herein as a "security management system" and "security system"). The security monitoring and control system 102 can be implemented within a computing environment having an organization's communication network 104. The network 104 can be a private network that can communicate with a public network (e.g., the Internet) to access application services 110. Examples of communication networks can include mobile networks, wireless networks, cellular networks, local area networks (LANs), wide area networks (WANs), other wireless communication networks, or combinations thereof. The security monitoring and control system 102 can be managed by a service provider, such as a security service provider (sometimes referred to as a cloud access security broker (CASB)), which uses the security monitoring and control system 102 to configure and manage the organization's security.

A tenant can be an organization or group whose members include users of services provided by a service provider (e.g., a cloud service provider). Users can have individual accounts with the provider, and tenants can have enterprise accounts with the cloud provider that encompass or aggregate multiple individual user accounts. In many embodiments of the present disclosure, the security monitoring and control system 102 can enable tenants to view information about security accounts, including controls and activity on those accounts for the various services they use, review analytical reports, and configure security controls through pre-set security classification levels.

In several embodiments, the security monitoring and control system 102 uses machine learning and other algorithms to analyze information about user activity in one or more clouds to perform threat detection and provide recommendations about appropriate responses to different categories of threats. The analysis can include determining models of normal and/or abnormal behavior in user activity, and detecting patterns of suspicious activity in one cloud or across multiple clouds. Some patterns can involve detecting the same operations or different operations in multiple clouds associated with the same user account or IP address. The analysis can also include providing alerts and recommending remedial actions in the cloud(s) where suspicious activity was detected, and/or taking remedial actions in clouds other than the cloud that displayed the suspicious activity. In many embodiments of the present disclosure, the processing for detecting and analyzing applications on devices within an organization's network involves collecting and combining information from various data sources.

The system for security monitoring and control according to an embodiment of the present disclosure includes multiple components that can be located on a single hardware platform or on multiple hardware platforms that communicate with each other. The components may include software applications and/or modules that configure a server or other computing device to perform processing for discovery and management, as will be discussed further below.

FIG1A shows a system 100 according to an embodiment of the present disclosure, which includes a security monitoring and control system 102, a client device 106 that can be used to access the security monitoring and control system 102, and an application service 110 to be monitored. As disclosed herein, a "client" (also referred to herein as a "client system" or "client device") can be a device or an application executing on a device. The system 100 includes a plurality of different types of client devices 106, each of which has the ability to communicate over a network 104. The client devices 106 communicate with the security monitoring and control system 102 and present a graphical interface for interacting with the service. The security monitoring and control system 102 can communicate with the application service 110 to retrieve security configurations, application data, and other information and to set security controls, as will be discussed further below.

FIG1B illustrates a system 150 having an implementation of a security monitoring and control system 102 implemented for an organization. Specifically, the system 150 illustrates how the security monitoring and control system 102 can be implemented to detect application usage by users of the organization on client devices 106 on a communication network, such as a private network (e.g., an intranet 170) and a non-private network (e.g., the Internet 160). Examples of communication networks can include mobile networks, wireless networks, cellular networks, local area networks (LANs), wide area networks (WANs), other wireless communication networks, or combinations thereof. Client devices 106 operating in the intranet 170 can be in an isolated computing environment protected by a firewall 142. Users (e.g., security administrators) can manage the operation of the security monitoring and control system 102. The security monitoring and control system 102 can be implemented in the organization's computing environment, outside of the computing environment, or both. The security monitoring and control system 102 can be provided as a cloud-based service over the network 160.

Each of the client devices 106 can be used to access applications that are authorized or unauthorized for use on the organization's devices. Applications can be accessed by different service providers, such as a trusted application provider 120 and an unknown application provider 122. Client devices 106 inside and outside the intranet 170 can be used to access the services of a third-party service provider 124 that enables access to applications and/or data managed by different service providers.

The security monitoring and control system 102 can monitor application activity based on the network activity of the organization's client devices through network data from one or more agents operating on network devices. The security monitoring and control system 102 can analyze and correlate data from applications to provide deep visibility into activity in the organization and help detect anomalies or emerging threats and security risks based on application usage.

Turning now to FIG. 1C , system 150 is shown as another example of how client devices 106 (such as personal BYOD ("Bring Your Own Device") and company-owned desktops/laptops) can be used to access different types of applications within and outside an organization. Applications can be provided by a service provider system 180. A service provider system may also be referred to herein as a "provider system." Each service provider system disclosed herein may be operated and managed by a service provider. Applications may include unauthorized applications 122 and third-party unauthorized applications 124. Organizational users may use many approved applications for daily work, such as Salesforce for tracking customer activity, Google Apps/Office 365 for collaboration, Box for sharing files, and so on. These applications allow third-party applications to be installed within these applications, which in turn allows these third-party applications to access approved application data on the user's behalf. These unapproved third-party applications can increase organizational risk from a security and compliance perspective because they can access business-sensitive data that may be vulnerable due to poor security or potential unauthorized data leakage. Therefore, such third-party application discovery can help IT teams gain greater visibility.

The security monitoring and control system 102 can discover application usage, including shadow applications, by collecting data from multiple sources, correlating it with threat intelligence information, and analyzing it. This provides greater depth of visibility and better compliance coverage than simply discovering shadow IT information from network devices. When an organization's users access unapproved applications from the office network, connectivity-related information such as the destination network address, the requester's network address, and timestamps is recorded by network devices such as routers and firewalls (e.g., application firewall 144 and network firewall 142). Some of these application firewalls also record the requester's identity, which allows for locating the actual user who is using the application. When an organization's users install applications on mobile devices such as smartphones and tablets, the MDM (Mobile Device Management) service 182 can discover the details of the installed applications, as long as the MDM application is installed on the device. Similarly, when users install unapproved applications on company-owned devices, such installation details can be centrally discovered using centrally managed security management tools (e.g., application usage tracking server 184) with company-owned license management. Logs from these data sources can provide visibility into unauthorized or shadow applications installed on desktop/laptop devices.

II. Architecture for Security Monitoring and Control Systems

Some embodiments, such as systems, methods, and machine-readable media, are disclosed for discovering and managing the security of applications in a computing environment. Figure 2 illustrates a system 200 in which a user can operate a client device (such as client devices 106-1, 106-2, ... 106-N (collectively referred to as client devices 106)) to access one or more applications (also referred to herein as "applications") from one or more service providers. For example, system 200 may include one or more service providers (such as service provider 210 and service provider 212). Each service provider may provide one or more services via a network 160 (e.g., the Internet). Services may include cloud-based or network-based services. For example, service provider 212 may be a "cloud" service provider. Services may include providing applications as services. Each service provider may have a service provider system including one or more computer systems. One service provider system may be different from another.

Client device 106 can be a user's personal device (e.g., BYOD) or a device managed by an organization. As shown in FIG1C , client device 106 can access a service provider's applications over a network within the organization's computing environment, a network 160 external to the computing environment, or a combination thereof. Applications can be operated to access data and/or resources within the organization's computing environment. Some client devices can use third-party applications 214 provided by the service provider to access applications and/or data for applications. Applications can be registered with the organization to act as users within that organization. Some applications may not be registered and therefore may be unauthorized or unknown to the organization. Each application can utilize and/or access resources within the organization's computing environment. Security monitoring and control system 102 can discover applications and their use with respect to the organization's computing environment. Client device 106 can be operated by a user of the organization (such as an administrator) to utilize services provided by security monitoring and control system 102. Users of client device 106 can be part of one or more tenants, or groups within the organization. As such, security monitoring and control system 102 can provide services for discovering and managing applications on a per-user and/or per-tenant basis.

Resources may include, but are not limited to, files, web pages, documents, web content, computing resources, or applications. For example, system 200 may include resources such as applications and/or content accessible through those applications. Applications may be used to request and access resources. For example, an application may request access to a web page from a resource server based on a URL identifying the requested resource. Resources may be provided by one or more computing systems (e.g., resource servers that provide access to one or more resources).

An organization may have one or more computing environments (such as computing environment 240 and computing environment 260). Each of the computing environments may be a cloud computing environment or an enterprise computing environment. Each of the computing environments may provide access to the organization's computing resources to the client devices of the organization's users. Each computing environment may include one or more computers and/or servers (e.g., one or more access manager servers), which may be general-purpose computers, dedicated server computers (e.g., including PC servers, UNIX servers, mid-range servers, mainframe computers, rack servers, etc.), server farms, server clusters, distributed servers, or any other appropriate arrangement and/or combination. The computing environment may run any operating system or various additional server applications and/or middle-tier applications, including HTTP servers, FTP servers, CGI servers, Java servers, database servers, etc. Exemplary database servers include, but are not limited to, commercially available database servers such as Oracle and Microsoft. The computing environment may be implemented using hardware, firmware, software, or a combination thereof.

Each of the computing environments can be implemented as a secure environment for an organization. For example, the organization's computing environment 240 and computing environment 260 can be implemented as a secure environment (e.g., an intranet) behind a computing firewall 230. One or more firewalls can be implemented to protect the computing environments. Each of the computing environments can be implemented using one or more network devices. For example, computing environment 240 can be implemented using one or more network devices 242, and computing environment 260 can be implemented using one or more network devices 262. Each of the network devices can facilitate communication within the computing environment and with external networks (e.g., network 160). Network devices can include, but are not limited to, routers, gateways, access points, bridges, and the like. Network data can be collected at each network device in the computing environment. The data can be collected in log files.

The security monitoring and control system 102 may provide a web-based client interface, a dedicated application, an application program interface (API), a graphical interface, a communication interface, and/or other tools for facilitating communication between the client device 106 and the security monitoring and control system 102. For example, the security monitoring and control system 102 may include an interface 220 for exposing services of the security monitoring and control system 102. The interface 220 may generate and/or provide an interface that enables the client device 106 to access the security monitoring and control system 102. The security monitoring and control system 102 may be implemented to perform the operations disclosed herein, including the processes disclosed with reference to Figures 1A-1C and 7-14.

The security monitoring and control system 102 can be implemented by a computing system. The computing system can include one or more computers and/or servers (e.g., one or more access manager servers), which can be general-purpose computers, dedicated server computers (including, by way of example, PC servers, UNIX servers, mid-range servers, mainframe computers, rack-mount servers, etc.), server farms, server clusters, distributed servers, or any other suitable arrangement and/or combination. The security monitoring and control system 102 can run any operating system or various additional server applications and/or middle-tier applications, including HTTP servers, FTP servers, CGI servers, Java servers, database servers, etc. Exemplary database servers include, but are not limited to, commercially available database servers from Oracle, Microsoft, etc. The security monitoring and control system 102 can be implemented using hardware, firmware, software, or a combination thereof.

The security monitoring and control system 102 may include at least one memory, one or more processing units (or processor(s)), and memory. The processing unit(s) may be implemented in hardware, computer-executable instructions, firmware, or a combination thereof, as appropriate. In some embodiments, the security monitoring and control system 102 may include several subsystems and/or modules. Each of these subsystems and/or modules in the security monitoring and control system 102 may be implemented in hardware, software (e.g., program code, instructions executable by a processor) executed on hardware, or a combination thereof. In some embodiments, the software may be stored in memory (e.g., a non-transitory computer-readable medium), on a storage device, or on some other physical storage, and may be executed by one or more processing units (e.g., one or more processors, one or more processor cores, one or more GPUs, etc.). The computer-executable instructions or firmware implementation of the processing unit(s) may include computer-executable or machine-executable instructions written in any suitable programming language to perform the various operations, functions, methods, and/or processes described herein. The memory may store program instructions that may be loaded and executed on the processing unit(s), as well as data generated during the execution of these programs. The memory can be volatile (such as random access memory (RAM)) and/or non-volatile (such as read-only memory (ROM), flash memory, etc.). The memory can be implemented using any type of persistent storage device, such as a computer-readable storage medium. In some embodiments, the computer-readable storage medium can be configured to protect a computer from electronic communications containing malicious code. The computer-readable storage medium may include instructions stored thereon that, when executed on a processor, perform the operations described herein.

The security monitoring and control system 102 may also provide services or software applications that can include both non-virtualized and virtualized environments. In some embodiments, these services may be provided to users of the client as web-based or cloud-based services or under a Software as a Service (SaaS) model. The services provided by the security monitoring and control system 102 may include application services. Application services may be provided by the security monitoring and control system 102 via a SaaS platform. The SaaS platform may be configured to provide services that fall under the SaaS category. The SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. By utilizing the services provided by the SaaS platform, customers may utilize applications executed in the security monitoring and control system 102, which may be implemented as a cloud infrastructure system. Users may access application services without the need for the customer to purchase separate licenses and support. A variety of different SaaS services may be provided. Users operating the client may then utilize one or more applications to interact with the security monitoring and control system 102 to utilize services provided by the subsystems and/or modules of the security monitoring and control system 102.

The security monitoring and control system 102 may also include or be coupled to additional storage, which may be implemented using any type of persistent storage device, such as a memory storage device or other non-transitory computer-readable storage medium. In some embodiments, the local storage may include or implement one or more databases (e.g., a document database, a relational database, or other type of database), one or more file stores, one or more file systems, or a combination thereof. For example, the security monitoring and control system 102 is coupled to or includes one or more data stores for storing data, such as the storage device 222. Memory and additional storage are both examples of computer-readable storage media. For example, computer-readable storage media may include volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data.

In the example shown in FIG2 , storage 222 may include tenant configuration information ("tenant configuration information") 224, which may include configuration information for tenants and their accounts, as well as user accounts associated with each tenant account. Users belonging to a tenant organization may have user accounts for various cloud applications. Tenant configuration information may also include tenant accounts, where cloud applications exercise administrative privileges over the user accounts of users belonging to that organization. A user's user account is typically associated with a tenant account for the tenant to which the user belongs. According to embodiments of the present invention, the association of a user account with a tenant account may be used in various ways, including retrieving information about user activity of users associated with a tenant. As discussed further below, the credentials of a tenant account may be used to log into a service provider system to retrieve data about user accounts and activity for services associated with the tenant account. This configuration information may include security settings for access, logging settings, and access settings (e.g., whitelists and blacklists). Storage 222 may include information about each user registered with an organization and/or user leased by the organization. Storage 222 may include application information 232 based on events regarding application usage and log information collected for network activity within the computing environment. Application information 232 may include organizational information obtained for an application from a data source. The information in the storage 222 can be maintained and curated by the security monitoring and control system 102 based on user activity and/or user input. For example, the storage 222 can include a registry such as those disclosed herein. The storage 222 can include security information 226 related to security analysis performed by the security monitoring and control system 102. The security information 226 can include security information obtained from one or more data sources. The storage 222 can include domain information 228 for domain information about the service provider of the application.

The security monitoring and control system 102 can be coupled to or communicate with one or more data sources 280, which can be implemented using any type of persistent storage device, such as a memory storage device or other non-transitory computer-readable storage medium. In some embodiments, the local storage device may include or implement one or more databases (e.g., a document database, a relational database, or other type of database), one or more file stores, one or more file systems, or a combination thereof. For example, the data sources 280 may include a security information data source 282, an organizational information data source 284, and a domain information data source 286. Each of these data sources may be implemented and/or accessible as a service provided by a service provider system. Each data source may include an interface for requesting data about an application and/or its provider. For example, the security information data source 282 may be provided by a company that offers Security Score as a service. In another example, the organizational information data source 284 may be provided by a service. The domain information source 286 may be provided by a provider system that provides Domain Name System (DNS) lookup services.

In some embodiments, security monitoring and control system 102 may include a log collector system 234 that performs operations for collecting network data regarding activity within a computing environment. Network data may be collected from log files obtained from one or more monitored computing environments. Log collector system 234 may be configured to communicate with one or more modules and/or subsystems implemented in each computing environment to collect network data. For example, computing environment 240 and computing environment 260 may each include a log manager 246 and a log manager 266, respectively. Each log manager may collect and/or aggregate data from one or more agents implemented to collect data regarding network activity (e.g., agent 244 in computing environment 240 and agent 264 in computing environment 260). Data may be collected in the form of log files. Each log manager and/or agent may be implemented on or communicate with a network device. Log collector system 234 may communicate with log managers 246, 266 and/or agents 244, 264 to collect data regarding network activity within the computing environment.

Each of the log manager and agent can be implemented in hardware, software (e.g., program code, instructions executable by a processor) executed on hardware, or a combination thereof. In some embodiments, the software can be stored in memory (e.g., non-transitory computer-readable media), on a storage device, or some other physical storage, and can be executed by one or more processing units (e.g., one or more processors, one or more processor cores, one or more GPUs, etc.). The computer-executable instructions or firmware implementation of the processing unit(s) can include computer-executable instructions or machine-executable instructions written in any suitable programming language to perform the various operations, functions, methods, and/or processes described herein. The memory can store program instructions that can be loaded and executed on the processing unit(s), as well as data generated during the execution of these programs. The memory can be volatile (such as random access memory (RAM)) and/or non-volatile (such as read-only memory (ROM), flash memory, etc.). The memory can be implemented using any type of persistent storage device, such as a computer-readable storage medium. In some embodiments, the computer-readable storage medium can be configured to protect the computer from electronic communications containing malicious code. Computer-readable storage media may include instructions stored thereon that, when executed on a processor, perform the operations described herein.

The log collector system 234 can be configured to communicate with each service provider through an interface provided by each service provider. The log collector system 234 can obtain log files and/or event data about the use of the service by one or more users from the service provider. The log collector system 234 can be configured to communicate with a module (e.g., an agent) on the client device and/or a mobile device management service to obtain event information about application usage.

Data regarding network activity and application usage can be processed by data analysis system 236 within security monitoring and control system 102. Data analysis system 236 can implement the techniques disclosed herein to analyze network data, including log files, to determine unique applications being accessed. Data analysis system 236 can also perform operations to identify domain information regarding the domain of the service provider providing the application. Domain information can be obtained from one or more data sources, such as domain information 286. Domain information can be obtained by performing queries against the data source and/or requesting credentials from the application's service provider.

The security monitoring and control system 102 may include a processor system 238 configured to obtain information about and/or related to the use of applications. The processor 238 may communicate with one or more data sources 280 to obtain information. The processor 238 may manage and curate information stored in the storage device 222. All or some of the information stored in the storage device 222 may be based on user input and/or curation.

The security analyzer 270 in the security monitoring and control system 102 may implement the techniques disclosed herein to determine measures regarding the security of an application, a user, or a combination thereof.

The control manager 272 in the security monitoring and control system 102 can manage and control access to applications within the computing environment. The security monitoring and control system can use one or more policies (e.g., security policies) to control the access that devices are allowed to applications within the organization's computing environment. Policies can be configured by users for one or more users (or collectively, tenancies). Policies can dictate remediation actions to be performed based on a security analysis of the users' use of applications. Remediation can include sending notifications, displaying information (e.g., reports), and/or limiting or blocking access to applications. The control manager 272 can communicate with the computing environment to configure network devices and/or firewalls to prevent or limit access to applications. This control can prevent, if not reduce, security risks and/or minimize inefficient or undesirable consumption of computing resources (e.g., bandwidth and memory usage). The control manager 272 can send one or more instructions to the computing environment and/or network devices to control access to applications. In some embodiments, the security monitoring and control system 102 can implement a module (e.g., an agent) on each client device 106 that is configured to communicate with the security monitoring and control system 102. The control manager 272 may send one or more instructions to an agent on the client device 106 to modify the functionality of the device to prevent or reduce access to the application.

In many embodiments of the present disclosure, a system for security includes a management application executed on a hardware platform, a user interface component, and a data repository stored on the hardware platform. FIG3 illustrates a system for security 300 according to an embodiment of the present disclosure. System 300 can be implemented in a security monitoring and control system as disclosed herein, such as security monitoring and control system 102. The cloud management application in system 300 can include a cloud crawler 302, a cloud seeder 304, and a data loader 306. As will be discussed in further detail below, cloud crawler application 302 can retrieve information about security controls from a cloud provider, cloud seeder application 304 can modify security controls of a cloud provider's tenant accounts to reflect a desired security posture, and data loader application 306 can retrieve activity information about the cloud provider's tenant accounts and generate analytics.

In several embodiments, data retrieved by cloud crawler application 302 is entered into application catalog database 308, and data retrieved by data loader application 306 is entered into logging repository 310 and/or analytics and threat intelligence repository database 311. The data entering logging repository 310 may be in different formats and/or have different value ranges—such data may be reformatted and/or structured before being moved to analytics repository 311. Data regarding activity information in analytics repository 311 may be utilized to generate reports that can be visually presented to system administrators via a user interface, and to generate analytics for determining threat levels, detecting specific threats, and predicting potential threats.

The aggregation of activity information in the analysis repository 311 regarding access patterns and other event statistics enables the system to establish a baseline of user behavior. Machine learning techniques can then be applied to detect threats and provide recommendations on how to respond to them. Threat models can be developed to detect known or unknown or emerging threats. Threats can also be identified by comparing activity data with external threat intelligence information (such as information provided by third-party providers), as will be discussed further below.

The accounts of a particular user in different cloud applications (e.g., different user identities) can be associated together in the user identity repository 309. The user identity repository 309 and/or other storage in the cloud security system can store information about tenant accounts and user accounts associated with each tenant account. Users belonging to a tenant organization can have user accounts for various cloud applications. A tenant organization can also have a tenant account for a cloud application that exercises administrative authority over the user accounts of users belonging to the organization. A user's user account is typically associated with a tenant account of the tenant to which the user belongs. According to embodiments of the present disclosure, the association of a user account with a tenant account can be used in various ways, including retrieving information about user activities of users associated with a tenant. As will be discussed further below, the credentials of a tenant account can be used to log into a cloud application service to retrieve activity data about user accounts associated with the tenant account.

As will be discussed in more detail below, the user identity repository 309 can also be used to facilitate user tracking and profiling across multiple cloud applications. In addition, collecting information about user behavior across multiple cloud services enables the system to proactively alert system administrators about threats on other cloud services when a threat is detected based on behavior on one or more cloud services and/or proactively protect other services on which users maintain data by applying remedial measures, such as adding additional steps to authentication, changing passwords, blocking specific one or more IP addresses, blocking email messages or senders, or locking accounts.

In several embodiments of the present disclosure, system 300 includes applications or software modules to perform analytics on collected data, as discussed in further detail below. The applications or software modules can be stored in volatile or non-volatile memory and, when executed, configure processor 301 to perform certain functions or processes. These applications can include threat detection and predictive analytics applications 312 and/or descriptive analytics applications 313. Threat detection and predictive analytics applications 312 can use machine learning and other algorithms to generate analytics to identify and predict security threats from activity patterns and behavioral models. Descriptive analytics applications 313 can generate analytics, such as, but not limited to, statistics about users, user activities, and resources. The analytics can be performed using data stored in analytics and threat intelligence repository 311.

As will be discussed further below, embodiments of the present disclosure may include remediation functionality that provides manual and/or automated processing in response to threats. In some embodiments, analysis may utilize information received from tenant systems that describes threat intelligence provided by the tenant. These sources, which may be referred to as customer baselines 317, may include information such as, but not limited to, specific IP addresses to watch or block, email addresses to watch or block, vulnerable browsers or versions thereof, and vulnerable mobile devices or versions of mobile hardware or software. In additional embodiments, analysis may utilize information received from external third-party feeds 318, 320, and 321 to enhance threat intelligence by providing external information of security threats, such as, but not limited to, identification of infected nodes, malicious activity from specific source IP addresses, malware-infected email messages, vulnerable web browser versions, and known cloud attacks.

The incident remediation application 313 can be used to coordinate and/or perform remedial actions in response to detected threats. It can be invoked when a recommended remedial action is presented and selected in an alert. The incident remediation application 313 can perform the selected remedial action or instruct another application (such as the cloud seeder application 304) to perform the selected remedial action. When the selected remedial action is to be performed manually or outside the cloud security system, the incident remediation application 313 can track the status of the remedial action and whether the remediation incident is complete. The incident remediation application can be used to save the results of manual or automatic remedial actions to memory. In several embodiments, the selected remedial action will be performed by a system external to the cloud security system, such as by a third party or tenant's incident remediation system. In this case, the incident remediation application 313 can instruct or call the third party or tenant's incident remediation system to perform the action using automatic integrated processing.

The cloud seeder application 304 can be used to implement security policies by setting security controls within tenants' accounts in various cloud applications. As will be discussed in further detail below, the cloud seeder can set security controls under various conditions, such as, but not limited to, as part of threat remediation or at the behest of a system user. Examples of techniques for implementing security controls and adjusting security controls can be implemented using the techniques disclosed in U.S. patent application Ser. No. 14/523,804, filed on October 24, 2014, entitled “SYSTEMS AND METHODS FOR CLOUD SECURITY MONITORING AND THREAT INTELLIGENCE.”

In other embodiments of the present disclosure, the user interface components include a management console 314 that provides users with control management for setting security controls for one or more clouds, and an analysis visualization console 316 for viewing analysis generated by the system. As will be discussed in more detail below, data in the data warehouse can be used to generate information and reports displayed in the user interface. The following discusses the use of cloud management applications to retrieve security configuration data from cloud applications.

III. Processing for Retrieving Software-Defined Security Configuration Data from Cloud Services

In many embodiments of the present disclosure, a cloud crawler application retrieves software-defined security configuration data from a cloud service. Software-defined security configuration data describes the configuration of security controls within a specific cloud service. Security controls are mechanisms that restrict access to applications and data hosted by the cloud. Software-defined security configuration data can include data describing user-defined roles, user groups and subgroups, encryption keys, tokens, access controls, permissions, configurations, authentication policy types, mobile access policies, and many other types of security controls. The process of retrieving software-defined security configuration data from a cloud service is illustrated in Figure 4.

The process includes step 402 for connecting to the cloud. The cloud may require authorization or some other indication of consent to access systems and internal data. Authorization may be provided via a token (such as using the OAuth open standard) or via credentials (such as a username and password). Those skilled in the art will recognize that there are various other techniques for authorizing access to the cloud provider's systems and data. Connecting may also include providing a service URL (Universal Resource Locator).

The process also includes step 404, which is used to collect software-defined security configuration data about the security controls of the cloud application. The software-defined security configuration data can be collected by utilizing the API (application programming interface) provided by the cloud application. The API and API classes that can be used according to the embodiment may include REST (Representational State Transfer), J2EE (Java 2 Platform, Enterprise Edition), SOAP (Simple Object Access Protocol) and native programming methods (such as native application API for Java). Other technologies can also be used to request information, including scripting languages (such as Python and PHP), deployment descriptors, log files, database connections through JDBC (Java Database Connectivity) or REST, and resident applications (cloud beacons), which will be discussed further below. The information sent or received can be represented in various formats, including but not limited to JSON (JavaScript Object Notation), XML (Extensible Markup Language) and CSV (Comma Separated Values). Those skilled in the art will recognize that according to the embodiments of the present disclosure, any of the various formats suitable for a particular application can be used.

The received software-defined security configuration data about the security controls of the cloud application can be used in step 406 to generate security control metadata, i.e., normalized descriptors for entering information into a common database. The security control metadata is classified (mapped into categories) and indexed in step 408. The classification can conform to standards specified by the security organization and/or can be certified and/or audited by a third party. In addition, the security control metadata and/or classification of metadata can be developed around the requirements of specific regulations or standards. For example, regulations and standards such as the Health Insurance Portability and Accountability Act (HIPAA), the Sarbanes-Oxley Act, FedRAMP, and the Payment Card Industry Data Security Standard (PCI DSS) may require reporting and audit trails. The security control metadata can be formatted in a manner that displays the type of information required by the regulations and standards and facilitates the generation of the required reports.

At step 410, the security control metadata is entered into an application catalog database. In many embodiments of the present disclosure, the application catalog database is a Cassandra database. In other embodiments, the application catalog database is implemented in another type of database suitable for the application. Those skilled in the art will recognize that, according to embodiments of the present disclosure, any of a variety of databases may be used to store the application catalog for later retrieval, report generation, and analysis generation, as discussed further below.

The above discusses specific processes for discovering and storing security control metadata according to embodiments of the present disclosure. According to embodiments of the present disclosure, any of the various processes for retrieving software-defined security configuration data and generating security control metadata may be utilized. Those skilled in the art will recognize that the number and type of controls and the mechanisms for retrieving software-defined security configuration data may vary in different embodiments of the present disclosure supported by different cloud applications. For example, application-specific retrieval mechanisms may be used to support other cloud applications (such as Office 365, GitHub, Workday, and various Google applications). Furthermore, the process for retrieving software-defined security configuration data may be automated or manual based on target cloud provider support.

IV. Control Management Platform

In many embodiments of the present disclosure, a control management platform provides users with a standardized view of controls for multiple clouds. The platform may include a user interface that displays a simplified view of controls for different clouds on the same screen. Information provided to the control management platform can be retrieved from an application catalog database using metadata-based schema mapping. The platform can be used to assign consistent access policies across clouds. Controls can be displayed and/or set according to specified classifiers, such as, but not limited to: standard, strict, custom. Higher levels of classification correspond to stricter controls. In several embodiments, the classification and/or designation of security controls complies with standards set by organizations such as the National Institute of Standards and Technology (NIST), the International Organization for Standardization (ISO), and/or the Payment Card Industry Data Security Standard (PCI DSS), and/or specific certifications provided by one of such organizations. In several embodiments of the present disclosure, the control management platform may also provide a plug-in interface for integration with SaaS, PaaS, and native applications.

The control management platform user interface can display key security indicators in a gallery format with color-coded risk factors (such as red, green, yellow). Other statistics or metrics can be displayed, such as, but not limited to, user login attempts, groups with the most added users, the most deleted files, users with the most deleted files, and users who downloaded the most files. Some types of information can be specific to a particular cloud application provider, such as Salesforce.com showing who is downloading opportunity/budget data, contracts, or contacts. In several embodiments of the present disclosure, the user interface provides a unified view of security controls for a tenant's registered cloud applications. The user interface can display the values of any or all security control settings set for different cloud applications, as well as the deviation of the current values from the values associated with the predetermined policy or configuration. The collection of activity data from the cloud application provider is described next.

V. Processing of Activity Data Collected from Cloud Services

In many embodiments of the present disclosure, a cloud data loader application configures a computing device to collect activity data from cloud services regarding user activity, security configuration, and other relevant information for tenants. A process for collecting activity data from cloud services according to an embodiment of the present disclosure is illustrated in FIG5 .

The process includes a step 502 for connecting to one or more clouds and a step 504 for collecting activity data from the cloud. In many embodiments, the connection is made over an encrypted communication channel. In other embodiments, the connection must be authenticated by a token or using login credentials, as in the connection made with the cloud crawler application discussed further above. In several embodiments of the present disclosure, the collection is scheduled to occur periodically (e.g., every 4 hours or every 6 hours). In many embodiments, the schedule for collection is configurable by the tenant. In other embodiments, data is collected and retrieved in real time using a real-time computing system (such as, for example, Storm) as events occur. The system can be configured to designate certain events or activities as high-risk events so that they can be retrieved in near real time in addition to scheduled retrievals.

Activity data can include various types of information accessible from a remotely hosted cloud application to systems external to the cloud application when the external system maintains appropriate credentials (which can be issued by the cloud application or another authorized entity). Activity data associated with a user account can include information related to the use and/or actions of the user account at the cloud application. Activity data can include information sources such as user logs or audit trails. More specific types of activity data can include, but are not limited to, login and logout statistics (including attempts and successes), IP addresses used to access the application, devices used to access the application, and cloud resources accessed (including, but not limited to, files and folders in file management cloud applications such as Box, employees and contractors in human resources cloud applications such as Workday, and contacts and accounts in customer relationship management cloud applications such as Salesforce). Activity data can include the user account or other user identifier of the user associated with the event or statistic. Activity data can include information about the system state or activity of the cloud application (such as, but not limited to, server activity, server reboots, security keys used by the server, and system credentials), where such information is visible to the system or accessible using authorized credentials.

Activity data may also include information about security configurations for tenant (and associated user) accounts. Security configurations may include setting values for security controls (discussed further above) for tenants (and/or associated users).

In some embodiments, certain events are considered high risk, and activity data related to these events is retrieved in near real-time outside of scheduled intervals.

At step 506, the retrieved activity data is stored in the analysis and threat intelligence repository database 311. The analysis and threat intelligence repository database 311 can be any database or data repository with query capabilities. In several embodiments of the present disclosure, the analysis and threat intelligence repository database 311 is built on a NoSQL-based infrastructure such as Cassandra or other distributed data processing systems, but any data warehouse infrastructure can be appropriately utilized for this application. In some embodiments, data is first entered into the login repository 310 and reformatted and/or structured before being moved to the analysis repository 311.

In some embodiments of the present disclosure, data may be received in different formats used by different cloud applications. For example, the data may be formatted in JSON (JavaScript Object Notation) or other data exchange formats, or may be used as log files or database entries. In other embodiments, the process includes step 508 for normalizing and reformatting the data into a common format for storage in and retrieval from the analysis and threat intelligence repository database 311. Reformatting the data may include categorizing and structuring the data into a common format. In several embodiments of the present disclosure, the database adapts to structural changes and new values by running automated processes to check for changed data. In some embodiments, a cloud crawler application (as discussed further above) identifies differences in the structure or values of the retrieved data and implements the changes in the application catalog database 308 and/or the analysis and threat intelligence repository database 311. System reports may be pre-generated at step 510 by a job scheduled to run on the dataset. A specific process for collecting activity data using a cloud loader application is discussed above. According to embodiments of the present disclosure, any of a variety of processes may be used to collect activity data. The following discusses reports that may be generated in advance or on demand by a system user or administrator according to embodiments of the present disclosure.

VI. Report

The data stored in the application catalog database and/or the analysis and threat intelligence repository database 311 can be used to generate various reports. Report categories can include: authentication and authorization, network and device, system and change data, resource access and availability, malware activity, and failures and critical errors. Reports can be based on various attributes, such as, but not limited to, per-application, per-user, per-protected resource, and per-device access. Reports can highlight recent changes, such as updated features or newly modified policies in cloud applications. Reports can be pre-generated by a scheduled job (e.g., for performance reasons) or can be requested by a user or administrator.

In various embodiments of the present disclosure, reports include analytics generated with respect to data. The analytics can utilize Apache Software Foundation technologies such as Hadoop, Hive, Spark, and Mahout, or other features available in the data storage framework being used. Several embodiments utilize the R programming language to generate analytics. In other embodiments, the generation of analytics includes the use of machine learning algorithms, proprietary algorithms, and/or external threat intelligence from external commercial sources such as FireEye and Norse, or public threat intelligence communities such as Zeus and Tor. Techniques for generating analytics according to embodiments of the present disclosure are discussed below.

VII. Analysis and Security Intelligence

A security monitoring and control system according to an embodiment of the present disclosure can use the collected data to generate analytics. The analytics can be generated by an analytics process and/or an analytics module, referred to as an analytics engine. FIG6 illustrates an overview of generating analytics using components of a threat intelligence platform 600 according to an embodiment of the present disclosure. Platform 600 can be implemented in system 200. All or part of platform 600 can be implemented in security monitoring and control system 102.

One type of analysis that can be generated is descriptive or statistical analysis. Statistics can be generated using a predefined set of system queries, such as, but not limited to, MapReduce jobs and Spark and Apache Hive queries. Descriptive analytics can be generated for a single application or across multiple applications using correlation techniques. Examples of reports that can be generated include, but are not limited to, login statistics (e.g., users with the most failed logins, including login history based on IP address taking into account IP reputation, geolocation, and other factors), user statistics (e.g., users with the most resources (files, EC2 machines, etc.), entitlements across clouds, number of password changes), activity statistics (e.g., user activity across clouds), statistics on key rotation (e.g., whether SSH keys have been rotated in the last 30 days), and resource statistics (e.g., number of folders, files downloaded by users, files downloaded by roaming or mobile users). Trends can be identified, such as login activity over a certain time period, password-related support issues based on the past history of these issues, or identifying the types of mobile devices that saw the most activity over a certain time period. The data in the reports can be displayed in a user interface as an event viewer that displays a "wall" of events along with actions that users can take to respond to or remediate them. Alerts can be built based on predefined rules that can include specific events and thresholds.

Another type of analysis that can be generated is predictive and heuristic analysis. These can be combined with machine learning algorithms to generate threat models, such as, but not limited to, deviations from baseline expectations, rare and uncommon events, and behavioral analysis to derive suspicious user behavior. Algorithms and profiles can be trained to intelligently predict whether abnormal behavior is a security risk. Third-party feeds from providers such as, but not limited to, MaxMind, FireEye, Qualys, Mandiant, AlienVault, and Norse STIX can be integrated to enhance threat intelligence by providing external information about and related to potential security threats, such as, but not limited to, IP (Internet Protocol) address reputation, malware, identification of infected nodes, vulnerable web browser versions, user use of proxy or VPN servers, and known attacks on the cloud. In several embodiments, threat information is represented in the Structured Threat Information Expression (STIX) data format. For example, one or more services can contribute information about a specific IP address, such as its reputation (e.g., known to have software vulnerabilities, a high volume of malware, or an attack source) and/or the geographic location associated with the IP address. This information can be combined with retrieved activity data related to the IP address (such as the time of the login attempt from the IP address) and information derived from the activity data (such as the distance of the login attempt). These factors can be used to determine a "login speed" metric. Metrics can be determined for other activities, such as file access, sales transactions, or virtual machine instances.

In many embodiments of the present disclosure, various types of algorithms can be particularly useful for analyzing data. Decision trees, time series, naive Bayesian analysis, and techniques for building user behavior profiles are examples of machine learning techniques that can be used to generate predictions based on patterns of suspicious activity and/or external data feeds. Techniques such as clustering can be used to detect outliers and anomalous activity. For example, a threat can be identified based on a series of failed login attempts to an account that accessed one or more files or from an IP address that has been flagged as malicious (via a third-party feed or otherwise). In a similar manner, a threat can also be based on different activity patterns within a cloud or across multiple clouds over a period of time. As further discussed above, activity data from different clouds can be in different formats or have different possible values or value ranges. Normalizing the data in the processing discussed above can include reformatting the data so that it is comparable, has the same meaning, and/or carries the same importance and relevance across different clouds. Thus, algorithms can aggregate and compare data from different clouds in a meaningful way. For example, a series of failed login attempts to a particular user account in one cloud can be considered not to be a threat. However, a series of failed logins to user accounts associated with a user across multiple clouds can indicate a concerted effort to crack the user's password and therefore raise an alarm. Clustering and regression algorithms can be used to classify data and find common patterns. For example, a clustering algorithm can place data into clusters by aggregating all entries of a user logging in from a mobile device. Predictive analysis can also include identifying threats based on activity, such as a user not accessing a specific cloud application for several months and then showing high activity the next month, or a user downloading a file weekly over the past few weeks, thereby presenting a potential advanced persistent threat (APT) scenario. In several embodiments of the present disclosure, data collected over time is used to build a model of normal behavior (e.g., patterns of events and activities) and to mark behavior that deviates from normal as abnormal behavior. After one or more marked events or activities are characterized as true or false positives (e.g., through user feedback), this information can be provided back to one or more machine learning algorithms to automatically modify the parameters of the system. Therefore, machine learning algorithms can be used in at least the manner discussed above to make recommendations and reduce false positives. Activity data collected from various parameters over a period of time can be used together with machine learning algorithms to generate patterns known as user behavior profiles. Activity data can include contextual information such as IP address and geographic location.

Algorithms for association rule learning can be used to generate recommendations. In several embodiments of the present disclosure, a profile linking algorithm is used to link activities across multiple cloud applications by finding cross-application dependencies. A single user can be identified across multiple clouds using one or more attributes or identification elements, such as a master user identifier (ID) or a single sign-on (SSO) authentication mechanism (e.g., Active Directory, Okta) that is commonly used across clouds. Correlations of activities across applications can include finding a user with a first entitlement in a first cloud application who has a second entitlement in a second cloud application, a user logged into two cloud applications simultaneously from different IP addresses, a user who has multiple failed login attempts and then changed their password, and a user who has a large number of failed logins in common across two cloud applications.

In many embodiments of the present disclosure, a user identity repository 109 can be utilized to facilitate user tracking and profiling across multiple cloud applications. A specific user's account across different cloud applications can be linked by associating a user identifier associated with the account (e.g., jdoe, john.doe, etc.), through a master (universal) user identifier as mentioned above, or an SSO mechanism or other method. The user identity repository 109 can contain information linking the accounts of each user associated with a tenant. A user who utilizes multiple cloud application accounts under the control or ownership of a tenant can be referred to as an "enterprise user."

In several embodiments of the present disclosure, a recommendation engine tracks abnormal behavior in user activity to detect attacks and unknown threats. The recommendation engine can track user activity across multiple clouds for suspicious events. Events can include predefined risky actions (e.g., downloading a file containing credit card numbers, copying encryption keys, elevating privileges of a normal user). Alerts can be issued with detailed information about the event and remediation recommendations.

Dynamic policy-based alerts can be generated for events related to specific users/employees. The process can monitor activity data associated with specific users and generate customized alerts for specific actions taken by the user.

In many embodiments of the present disclosure, algorithms are designed to simulate normal user activity using user activity data in the analytics and threat intelligence repository database 311. The simulation can be used to train other machine learning algorithms to learn the normal behavior of users in the system. In general, specific security issues may not always be repeated and therefore may not be detected by purely supervised algorithms. However, techniques such as outlier detection establish a baseline that is useful for detecting anomalous activity. This anomalous activity, together with contextual threat intelligence, can provide more accurate predictions of threats with low prediction errors.

In other embodiments of the present disclosure, analysis can be used to detect security control drift, which can refer to changing one or more security controls in a seemingly arbitrary manner that can increase security risk. Risk events can be generated in response to changes to one or more security controls in one or more cloud applications and actionable intelligence associated with risk events (also referred to herein as "security risks," "risks," "threats," and "security threats"). Threats can include unusual or non-compliant activities, events, or security controls related to the use of an application. As with other types of events, alerts can be sent to tenants, tenant systems, or other monitoring entities. For example, the password policy for a tenant in a cloud application may have been changed to impose fewer requirements (e.g., the type and/or number of characters). This may generate a risk event and an alert to recommend changing the password policy back to the original password policy.

Alerts regarding any of the events discussed above can be displayed on a user interface, such as the control management platform discussed further above. Alerts can include information about the detected event, such as, but not limited to, the event identifier, date, time, risk level, event category, user account and/or security control associated with the event, cloud application associated with the event, a description of the event, remediation type (e.g., manual or automatic), and/or event status (e.g., open, closed). Information in the alert regarding each risk event can include an identifier (ID), affected cloud application and instance, category, priority, date and time, description, recommended remediation type, and status. Each risk event can also have user-selectable actions, such as editing, deleting, marking status complete, and/or executing a remediation action. Selecting a remediation action can invoke an application, such as incident remediation application 313 and/or cloud seeder application 304, to execute the selected remediation. Alerts and/or other information regarding identified threats can be sent to entities external to security monitoring and control system 102.

In many embodiments of the present invention, alerts can be visual and appear in a user console, such as the control management platform discussed further above. In several embodiments, alerts are delivered over a network, such as via email, short message service (SMS) or text messaging, or a web-based user console. Alerts can be delivered as secure messages (e.g., over a secure communication channel or requiring a key or login credentials to view). Alerts can include information about recommended or available remedial action(s), such as implementing stronger security controls, and requesting selection of remedial action(s) to be taken.

The above discusses a specific process for retrieving and analyzing activity data according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, any of a variety of processes for retrieving and analyzing activity can be used. The following discusses a process for remediating identified threats.

VIII. Systems for Discovering and Analyzing Applications

Figures 7 and 8 illustrate block diagrams of processes for discovering and managing the security of applications according to some embodiments. Figure 7 illustrates how the security monitoring and control system 102 can discover third-party applications and display a graphical interface with information about those applications, including measures of the security of the applications and users of those applications.

FIG7 illustrates a process 700 for application discovery and analysis of application risk and user risk associated with application usage. Process 700 can be implemented for an application that has been authorized, such as by registering with the security monitoring and control system 102. Process 700 can begin at 720 by a user operating a client device (e.g., client console 702) to provide information about an application (e.g., registering an application). Client device 702 can communicate with security monitoring and control system 102 using an interface or service (e.g., a Representational State Transfer (REST) service).

The security monitoring and control system 102 can perform processing to discover application usage and measures of security for applications and users of those applications. At 722, application events can be downloaded from a service provider system that provides registered applications. The application events can be provided in the form of data records about applications that have been accessed. At 724, the application events can be used to discover data about applications that have been accessed and/or other third-party applications or registered applications that have been used to access another application. At 726, events about third-party applications can be stored in a repository. Event information can include a timestamp about the event, user information (e.g., username or email ID), third-party application name (link->application details), approved operation and instance name, IP address, and geographic location information.

At 728, processing may be performed to determine information about the third-party applications that have been accessed. Application events may be used to determine unique information identifying each application. Using the information about the applications, the security monitoring and control system 102 may calculate application risk scores for the applications using the techniques disclosed herein. In some embodiments, an application risk score registry 704 may be maintained from which application risk scores may be obtained. The registry may be maintained and automatically updated based on new and/or updated information about application usage. The application risk score may be calculated based on application details and risk score indicators from a risk score feed 740 obtained using a third-party source, such as a third-party application registry 706. The application details may include vendor information about the service provider or vendor providing the third-party application. The vendor information may include the vendor name, vendor logo, vendor domain, vendor description, vendor category (business), and/or security indicators (e.g., a score or link to access security score assessment support data). The information about the application may be sent to the client console 702 at 730 for display in a graphical interface such as that depicted in FIG15.

At 736, the security monitoring and control system 102 may obtain user information about the user of the application identified in the application event. At 734, a user risk score may be calculated based on the user information and the application event details. In some embodiments, the user risk score may be calculated based on information obtained from a data source that maintains information about the user's application usage. Information about the user (including the user risk score and application event details) may be sent to the client console 702. At 732, the client console 702 may display the information about the user, including the user risk score and application event details, in a graphical interface. Processing may end at 730 and/or 732.

FIG8 illustrates a process 800 for application discovery and analysis of application and user risks associated with application usage. Process 800 can be implemented to discover applications based on analysis of log data. Log data can be obtained from one or more agents of security monitoring and control system 102, which can collect log information about various applications and network activities.

Process 800 in FIG8 may begin at 820 by a user operating a client device (e.g., client console 802) to specify information identifying a log file. For example, the user may provide a log file and/or information accessing a source of the log file. The log file may be ingested at 820. Client device 802 may communicate with security monitoring and control system 102 using an interface or service (e.g., a Representational State Transfer (REST) service).

The security monitoring and control system 102 can perform processing to discover application usage and measure security for applications and users of those applications. At 822, logs can be obtained through staged processing. At 824, the logs can be processed for parsing to identify unique network activity. At 826, the network activity can be analyzed to discover third-party applications. Throughout the various stages of log processing, information can be transmitted by the security monitoring and control system 102 to the client console 802 to display the log processing status at 830.

At 826, the logs can be used to determine unique information identifying each application. Event details regarding application usage can be determined based on the logs. Event details can include a timestamp, a third-party application name (link -> application details), an IP address, a geographic location, user information (e.g., a username or email ID), and/or the address of the source device requesting the application (e.g., a MAC address). At 840, using the information about the applications, the security monitoring and control system 102 can determine information about each discovered application. This information can include details about the application and an application risk score using the techniques disclosed herein.

In some embodiments, an application risk score registry 804 may be maintained from which application risk scores may be obtained. The registry may be maintained and automatically updated based on new and/or updated information about application usage. Application risk scores may be calculated based on application details and risk score indicators from a risk score feed 846 obtained using a third-party source (such as a third-party application registry 806). Application details may include vendor information about a service provider or vendor providing a third-party application. Vendor information may include vendor name, vendor logo, vendor domain, vendor description, vendor category (business), and/or security indicators (e.g., a score or link to access security score assessment support data). Information about the application may be sent to the client console 802 at 832 for display in a graphical interface such as that shown in FIG. 15 (e.g., a log discovery report about application usage).

At 844, the security monitoring and control system 102 may obtain user information about the user of the application identified in the application event. At 842, a user risk score may be calculated based on the user information and the application event details. In some embodiments, the user risk score may be calculated based on information obtained from a data source that maintains information about the user's application usage. Information about the user (including the user risk score and application event details) may be sent to the client console 802. At 834, the client console 802 may display the information about the user, including the user risk score and application event details, in a graphical interface. Processing may end at 832 and/or 834.

IX. Measuring the Security of Computing Applications

9 illustrates a sequence flow diagram 900 of a process for measuring security of a computing application according to some embodiments. The process may be implemented by the security monitoring and control system 102.

The measure of security may be calculated based on information provided by one or more third-party sources, one or more users, sources managed and curated by the security control and management system 102, or a combination thereof. The measure of security may be based on information about the application and the provider of the application ("organizational information") and/or information about the security of the application ("security information"). The measure of security may be a value on a scale that defines a measure of the security risk of the application (also referred to herein as an "application risk score"). For example, the scale may be defined as between 1 and 5, where higher values represent greater risk. In other embodiments, any of a variety of ranges and values appropriate for a particular assessment may be used. The score may be used by the security monitoring and control system 102 to provide alerts, provide reports, and/or perform remedial actions. The measure of security is a value used as a priority indicator to help users (e.g., security administrators) mitigate security threats posed by the application. In some embodiments, the measure of security may be calculated in several ways.

One technique for calculating a security measure can involve calculating a risk score that indicates a security measure regarding the severity of the security threat posed by a specific application to an organization. The risk score can be calculated based on one or more indicators (also referred to herein as "threat indicators" or "security indicators"). Each indicator can be a value or information indicating a security risk. For example, in Figure 9, a unique security indicator can be obtained from one or more data sources 902 (also referred to herein as a "risk score feed"), where each data source ("S") (collectively referred to herein as data sources " _S1 , _S2 , ..., _Sn " 902) provides a "feed" of one or more indicators for an application. Each unique indicator ("I") (collectively referred to herein as indicators " _I1 , _I2 , ..., _In " 904) can be provided separately by each data source 902. An indicator can be a specific type of indicator based on security information, organizational information, or both. An indicator can be a value that provides an indication of the security threat posed by an application. For example, a first indicator can be a first value indicating a first security threat posed by an application. A second indicator can be a second value indicating a second security threat posed by an application. The first indicator may be obtained from a first data source, and the second indicator may be obtained from a second data source. The first data source may be one of data sources 280 in FIG. 2 . The second data source may be the same as the first data source or a different data source from data sources 280 .

One type of security threat indicator is a security threat indicator that can be provided by one or more third-party sources, such as open source (e.g., abuse.ch) or commercial sources (e.g., www.SecurityScoreCard.com). The security threat indicator can be based on security information such as, but not limited to, the security status of the application (e.g., insecure network settings with weak encryption algorithms), endpoint security (e.g., outdated devices used in the vendor's organization), IP reputation in the vendor's network (e.g., malware), hacker chat (e.g., discussions about the application in a hacker network), leaked information (e.g., publicly exposed sensitive information from the vendor), application security (e.g., website vulnerabilities), and DNS settings (e.g., incorrect settings that could lead to spoofed application websites).

Another type of security indicator may be an organization-based indicator provided by one or more third-party sources, such as open source (e.g., Wikipedia.com) or commercial sources (e.g., Clearbit.com). The organization-based indicator may be based on organizational information, such as, but not limited to, the category of business, the physical address of the provider (which includes geographic location information such as country), how long the business has been in operation, the age of the application's Internet domain registration, the popularity of the application according to a website ranking list (such as Alexa Domain Ranking), or a combination thereof.

Each threat indicator may represent a security risk score, which is a value indicating a measure of security risk or threat according to a scale defined by the source of that indicator. In some embodiments, the threat indicator may be compared to the scale to determine the security risk score. The scale may be defined by or based on information provided by the source.

The security monitoring and control system 102 can calculate a measure of security as a value or score based on one or more threat indicators. In other words, the measure of security can be calculated as a combined measure of security risk. Each indicator can be processed to adjust (e.g., normalize) its value to an adjusted value based on a scale to be used for all indicators (e.g., a scale of values from 0 to 100). Once normalized, the values for each indicator can be combined to determine a combined score.

The combined security score 908, indicating a risk measure, can be based on all or some of the indicators. The number and/or type of indicators considered for determining the combined security score can be configurable. As such, the security monitoring and control system 102 can be configured to add or remove sources 902 from which indicators are obtained, and can be configured to add or remove indicators 904 obtained from sources 902. In some embodiments, a graphical interface can be presented that enables a user to configure the number and type of indicators considered for the score. In some embodiments, the combined score can be based on a security policy that defines the criteria by which the score is calculated. The criteria can be based on one or more attributes of security. These attributes can be used to select the indicators to be considered for the score.

In some embodiments, the combined score 908 can be based on a combination of indicators using a weight value for each indicator. For example, a weight ("W") (collectively referred to herein as weights " _W1 , _W2 , ..., _Wn " 904) can be selected to apply to one or more specific indicators used to calculate the combined score 908. The weight can be a value that is an integer of 1 or a fraction of 1. Different weights can be selected for different indicators. In the example above, the first weight can be a first weight value for a first indicator, and the second weight can be a second weight value for a second indicator. The weight values can be different.

The weights can be configured by the user through a graphical interface. In some embodiments, the weights can be selected based on a security policy, where the security policy is defined based on specific indicators being given specific weights. When an indicator has more importance or suggestion for a security risk, a larger weight value can be considered. When an indicator has less importance or suggestion for a security risk, a smaller weight value can be considered. In some embodiments, weights can be selected for all indicators from a particular source. For example, a weight can be applied to all indicators from that source based on the reliability or trust of the source. In some embodiments, the security monitoring and control system 102 can store data about the threat research analysis of the application. This data can be used to selectively select a weight for each indicator.

A combined score can be calculated based on the indicators and the weights assigned to those indicators. In at least one embodiment, the combined score can be calculated using equation 908 to provide: Combined score = (I ₁ (W ₁ ) + I ₂ (W ₂ ) + ..... _In (W _n )) / (W _{1 +} W ₂ + .... W _n ). In this equation, the value of each indicator ("I") 904 is calculated by multiplying each indicator by a corresponding weight ("W") 906. A first value is calculated, which is the sum of the values calculated for each indicator (I ₁ (W ₁ ) + I ₂ (W ₂ ) + ..... _In (W _n )). A second value is calculated, which is the sum of each weight value applied to obtain the first value. The combined score 908 can be calculated based on dividing the first value by the second value.

In the example continuing from above, a first weighted value may be calculated based on multiplying a first value of the first indicator by a first weighted value. A second weighted value may be calculated based on multiplying a second value of the second indicator by a second weighted value. A weighted sum value may be calculated based on the sum of the first weighted value and the second weighted value. A weighted sum value may be calculated based on the sum of the first weighted value and the second weighted value. A measure of security may be calculated as a value based on dividing the weighted sum by the weighted sum.

In some embodiments, the security monitoring and control system 102 may obtain feedback 910 from one or more users regarding the effectiveness and accuracy of the combined score 908. The feedback 910 may be obtained over a network, facilitated by a graphical interface, or manual feedback. Any of the sources, indicators, and/or weights may be adjusted based on the feedback 910. Based on the feedback 910, the combined score 908 may be adjusted. A new combined score 912 ("adjusted score") may be calculated in the same manner as the combined score 908. Except that the adjusted score 912 may be calculated based on indicators and/or weights selected based on the feedback 910. Sources, indicators, and/or weights may be added or removed from the content used to calculate the combined score 908. The weight values for the indicators may be periodically revised based on our security analysts and customer feedback to improve the risk score. The revision process for the indicator weights may be performed by automated machine learning algorithms (such as decision trees and neural networks).

A regression analysis 914 can be performed based on each indicator and/or combined score for a particular security threat. The regression analysis can include building and updating a linear regression model. The linear regression model can provide an output such as S = c ₁ (I ₁ ) + c ₂ (I ₂ ) + ... + c _n (I _n ). The coefficients c _i calculated by the regression model can be new or modified weights that replace the initial weights used to calculate the combined score 908. As more feedback and more data are collected, the model will provide greater accuracy.

The following describes an example scenario in which the security monitoring and control system 102 can use threat intelligence from open source services such as abuse.ch ( _S1 ) and commercial services such as securityscorecard.io ( _S2 ) to determine security measures. In this example, two sources, _S1 and _S2 , are used. Source _S1 provides a domain reputation service that provides information about whether a domain is used to host malware, spam programs, and the like. Domain reputation will be indicator _I11 of _S1 . Source _S2 provides information about application security ( _I21 ) (e.g., whether applications hosted on the domain have security vulnerabilities) and network security ( _I22 ) (e.g., whether weak encryption algorithms are used). Application security will be indicator _I21 , and network security will be indicator _I22 of _S2 .

In this scenario, each source can be assigned an initial weight value, depending on the data quality and reliability of the data source. For example, the issue reported by _S1 can have a 40% weight, _w11 . Since _S1 has one indicator _I11 , this indicator will receive the entire 40% weight. Source _S2 has two indicators, which can share the weight value assigned to the source. For simplicity, we assume that _I21 and _I22 share the same weight - _w21 and _w22 each account for 30%. Using the technology disclosed herein, the combined score 908 can be calculated as follows: (( _I11 * _w11 )+( _I21 * _w21 )+( _I22 * _w22 ))/( _w11 + _w21 + _w22 ). By inserting the weights, the combined score can be reflected as =((I ₁₁ *0.4)+(I ₂₁ *0.3)+(I ₂₂ *0.3))/(0.4+0.3+0.3). In this example, the threat intelligence source indicators are rated as I ₁₁ =65, I ₂₁ =91, and I ₂₂ =90, and the domain's risk score will be 80 (rounded up). In another scenario, if the customer confirms that the domain is legitimate and there are no domain reputation issues, then I ₁₁ becomes 0. Therefore, the domain risk score for this customer drops to 54 (rounded up). In yet another scenario, if the customer confirms that they want to whitelist the application because the vendor has resolved all reported issues, then I ₁₁ and I ₂₁ will be 0. Therefore, the domain risk score for the customer will be 0, indicating that there is no risk to the customer from this application. As the customer continues to adjust the combined score, the regression model can periodically learn the relationship between the weights and the risk score and adjust the weight values accordingly.

X. Processing for Detecting and Analyzing Application Security

10 illustrates a flow chart 1000 of a process for discovering and managing risk to an application, according to an embodiment. In many embodiments, one or more features of the process discussed below may be performed by the security monitoring and control system 102 of FIG.

Flowchart 1000 may begin at 1002 by collecting information about applications accessed by one or more users of an organization. Information may be collected from one or more data sources using the techniques disclosed herein. The data sources may include, but are not limited to, routers, network firewalls, application firewalls, cloud applications, cloud application mobile device management (MDM) servers, and cloud application usage tracking servers. In some embodiments, information may be retrieved from the data source by requesting the information using a "pull" type mechanism. In other embodiments, the information may be provided by the data source via a "push" type mechanism without the need for a request. The information may be monitored from data in network traffic within the organization's environment. The environment may be a secure network environment, such as one configured with one or more network security features, such as a firewall.

In various embodiments, the information transmitted from the data source can be in any of a variety of formats. Some data sources can interact with it via a specified application programming interface (API). Other data sources can store information in database tables or log files.

The information retrieved from the router and network firewall may include information about the websites visited or other connections made by the user's device. Such information may include, but is not limited to, source and destination IP, protocol, service (e.g., 443 for HTTP), query string within the HTTP request, and/or product (e.g., instance name of what platform the user is using to connect to the network). The IP address can be used to perform a reverse lookup of the geographic location and/or to assess the reputation of the IP address. In some embodiments, deep packet inspection (DPI) is utilized to access additional information within the network traffic, such as usernames and other embedded data. User devices often utilize a virtual private network (VPN) to connect to the enterprise network, so the traffic traverses the enterprise network and can be captured by a router or firewall within the network.

In some embodiments, information about an application can be obtained from a third party and/or log information can be obtained from the application that provides the application of interest or another application. For example, information about a plug-in application of an authorized application can be retrieved from a data log provided by the authorized application. In another example, a lead-generation application (e.g., a third-party application unknown to the organization) can be used in conjunction with an application (such as an application provided by a cloud service provider known to the organization). Third-party application information may include, but is not limited to, access time, username, login URL, application name, application source IP, and/or login time. In addition, supplemental information such as a reverse lookup of a URL or IP address can be generated using the techniques disclosed herein. In some embodiments, information can be obtained through an interface (e.g., an application programming interface) of the service provider that provides the application.

In some embodiments, a server can manage and store information related to the use of applications. This information can be retrieved from the server. In one example, a tracking server can store information related to the use of cloud applications by users in an organization. A server used for mobile device management (MDM) can manage the management of applications and other software on mobile devices (such as smartphones or tablets). For example, the Google application suite can be provided to customers on a subscription basis. A Microsoft server can manage Microsoft products installed on devices throughout an enterprise. Windows 10 applications can be managed by Windows system administrators. Other cross-platform software management systems can also be utilized in the enterprise.

At step 1004, the information obtained in step 1002 can be used to determine whether the application is associated with a security risk. Security risks can be used to determine whether the application is unauthorized or unapproved for use by users in the organization. For example, information about the application can be used to determine whether the organization has approved the use of the application. The information can be processed using relevant technologies such as those disclosed in Figures 9 and 11. Information about the application can be obtained from one or more sources (including security monitoring and control system 102, third-party data sources, and processing of information obtained in step 1002). The information may include organizational information about the organization that provides the application. The information may include security information about security related to the application. The security information may include measurements or scores related to indicators or security aspects of the application. In some embodiments, the information can be obtained directly from the provider of the application.

At step 1006, one or more characteristics (e.g., security indicators) may be determined for the application being evaluated. Characteristics may include, but are not limited to, whether the application is standalone or an extension or plug-in to an approved application; the data sets that the extension or plug-in application accesses from the approved application (e.g., sales data versus product code); the type or category of the application (e.g., business content versus social network); the age or registration location of the domain name associated with the application; the domain name reputation and/or internet site ranking of the internet domain associated with the application as provided by a third-party reputation or ranking service (e.g., Alexa, Google ranking or ranking factors, market capitalization, publicly listed versus private companies, etc.), and/or the IP address reputation of the website associated with the application, which indicates issues such as a history of spam or any malware-related issues (e.g., as provided by a third-party service).

Characteristics may be determined using information retrieved from other sources.The determination involves quantification by assigning a numerical value (eg, a weight) to each characteristic determined for use in assessing the security risk that an application may pose to an organization.

In some embodiments, additional features may relate to applications that integrate with authorized or secure applications. For example, when a third-party application integrates with a popular approved application (such as Salesforce, Google Apps, etc.), the approved application vendor may provide guidance on how to access data. Due to this access, the approved application may be deemed to have an increased risk and its overall risk score increased. For example, the third-party application may not be able to securely manage and clear the accessed data, or potential security vulnerabilities in the third-party application may be a source of unauthorized data leakage that is beyond the control of the approved application. Features may be determined for factors related to the security of the access granted to the third-party application and the interface of the third-party application with the approved or authorized application.

At step 1008, a security score or measure of security may be determined for each of the one or more applications based on the determined characteristics. The scores and characteristics may be stored in a database, such as an application registry. The scores may be determined using techniques disclosed herein, such as those described with reference to FIG. 9 .

The security score can be a value on a scale that defines a measure of security risk. For example, the scale can be defined as between 1 and 5, with higher values representing greater risk. In other embodiments, any of a variety of ranges and values suitable for a particular assessment can be used. The score can be used by the security monitoring and control system 102 to provide alerts, provide reports, and/or implement remedial measures. In addition, the security monitoring and control system 102 can utilize the score and other information about application security in conjunction with information about more secure (e.g., approved, authorized, or approved) applications to perform security assessments and determine threat levels.

Flowchart 1000 may end at step 1010 .

FIG11 illustrates a flowchart 1100 of a process for discovering and managing application risk according to an embodiment. In many embodiments, one or more features of the process discussed below may be performed by the security monitoring and control system 102 of FIG1. Flowchart 1100 illustrates certain embodiments of the process depicted in FIG10.

Flowchart 1100 may begin at step 1102 by obtaining information about a user's network activity. This information may be obtained from data obtained using techniques for monitoring network activity, including those disclosed with reference to FIG. 8 . Data regarding network activity may be obtained by monitoring and/or obtaining data (e.g., log data or recorded data) from network devices. In order for an organization to monitor application usage, the organization may monitor its internal or protected network (e.g., an intranet) for network activity. Network activity may be monitored by obtaining information from network resources (e.g., network devices) about network traffic both within and outside the organization's network.

In some embodiments, data about network activity can be collected from multiple data sources. Log files can be ingested and processed by the security monitoring and control system 102 to identify information about network activity, such as application usage. Log files can be obtained using the techniques disclosed in FIG8 . Data about application events can be obtained from one or more sources using the techniques disclosed in FIG7 . In some embodiments, information about network activity (such as specific services and application usage) can be obtained from one or more third-party sources (such as service providers for applications). The security monitoring and control system 102 can utilize the service provider's interface to obtain log information about the activities of one or more users.

The information may be obtained from multiple sources in a variety of different formats. The data containing the information may be processed to prepare (e.g., normalize) the information into a format for processing to determine application usage. The data may be processed to perform deduplication to identify unique instances of network activity.

At step 1104, the information obtained about the network activity is used to determine one or more applications that have been accessed. The information obtained about the network activity can be obtained from data about the network activity. The data can be used for communications on the network. The data further relates to the network of multiple users who are tenants on the organization's network. Obtaining data about the network activity can include obtaining network data from one or more network devices on the network. The network data can be obtained from one or more agents and/or log managers implemented in the computing environment using the techniques disclosed herein. The network data can be obtained by a log collector of a security monitoring and control system. The network can be protected in the organization's computing environment so that the computing environment is secure from public networks.

Accessing an application can include allowing the application's data to be accessed by different applications. For example, the application's data can be accessed by a third-party application that the user agrees to access the user's data from the application's service provider. The application can be authorized (e.g., approved) or unauthorized (e.g., unapproved) by the organization from which the user accesses the application. The information can include data about the application being used. The data can be the application name or a link to the application being accessed. The information about the application can be used to retrieve information from a third-party source (e.g., the provider of the application). This information can be included in activity data obtained from the provider's service provider system. Each unique application can be identified to determine which applications the user has accessed. In some embodiments, some applications can be of the same type or kind, but can correspond to different instances and/or different access accounts. Because each unique application can pose a security vulnerability to one or more of the features being evaluated, each application can be identified for analysis. The data corresponding to each unique application can be stored in association for further processing by the application.

At step 1106, access information is determined for each of the one or more applications that have been accessed. The access information can be determined using the information obtained in step 1102. The application can be identified by processing the data to identify a portion of the data corresponding to the request for the application accessed by the user. The portion of the data can indicate application information about the request for the application, which is used to identify the application as being accessed by the user. The data can include information about the request (such as an IP address, a source identifier, a link to the application), or other information related to the source of the request or the destination where the application is located. The access information can be determined based on the data corresponding to each application. The portion of the data can be used to determine access information about the network activity corresponding to the application. The access information can include information used to access or related to accessing the application. The access information can be obtained from the data corresponding to each unique application. Using the information about each application, access information about each application can be identified for the user. The access information can include network information about the request to access the application, including information about the source of the request (e.g., the source device) and the destination to which the request was sent. The access information may include, but is not limited to, a timestamp of the network activity, application information (e.g., application name, source location accessing the application, and/or details about the application), source IP address, local IP address, geographic location information about the source, user information (e.g., user ID or email ID), and media access control (MAC) address. The source location of the application may be a link (such as a URL or URI). The information obtained in step 1102 may be processed to identify network activity for each unique application identified in step 1104.

At step 1108, domain information about the domain of the provider system providing each application is determined based on the access information. The domain information can be determined using the techniques disclosed with reference to Figures 7 and 8. The domain of the application can include information about the provider and the provider's system providing the application (e.g., host system information). For example, the domain information can include a domain name, an IP address, system information, or other information about the host system of the domain in the provider system.

Domain information can be determined by querying one or more data sources using all or some of the access information. The security monitoring and control system 102 can maintain a data store of information about applications and providers of applications. In some embodiments, a third-party data source can be used to find or query information about the domain of an application. One third-party data source can be a database managed by a third party, such that the database stores information about the domain and system information about one or more application providers. Another third-party data source can be a Domain Name System (DNS) source or some other source that stores information about a domain or an entity (e.g., a provider) of a domain. Information about the domain can be queried from the third-party data source based on the access information. For example, an NSlookup command can be issued to determine the domain of an IP address in the access information.

In some embodiments, a request may be sent to the system of the provider of the application to obtain domain information. Based on the access information, a request may be generated and set to the system providing the application. The request may be sent based on the source location of the application indicated by the access information. The domain information obtained by querying the data source may often not provide specific information about the domain of the provider's system. The request may be a network call to an endpoint (e.g., a SQL connection endpoint) to obtain a certificate for the application. The request may include the URI of the application. The URI may be obtained from the access information for the application. In at least one example, the network call may be a browser call (e.g., HTTPS) to obtain a certificate for information about a hosted site, which is the domain of the provider of the application. By sending the call in this manner, the site may not be able to prevent the request from being blocked. The certificate may include domain information about the hosted site of the provider of the application. In addition to or as an alternative to querying the data source, a request to the provider's system may be implemented.

Steps 1110 and 1112 can be performed simultaneously or in any order based on the applications that have been identified. At step 1110, organizational information can be determined for each of the one or more applications. The organizational information can include information about each organization of the service provider that is clearly associated with the provider system that provides each of the one or more applications that have been accessed. The organizational information can include information identifying the organization associated with the provider system that provides the application. The organizational information can include business entity information (such as the organization's registration information). The organizational information can include information about the application, including details about the application (such as the source of the application and the type of application). The organizational information can include location information about the location of the provider system (such as the location where the provider system is hosted). In some embodiments, the organizational information can include statistical information about each application that has been accessed. The statistical information can include data usage of network activity related to the application, network traffic of the application (e.g., upload and download traffic), and other statistical information about the use or operation of the application. The organizational information can be determined using domain information to identify the organization that provides the application.

Organization information can be obtained from one or more sources. One source can include a third-party source that aggregates information about applications, including the organization(s) that provide these applications. The security monitoring and control system 102 can maintain its own data store of application information based on monitoring network activity. In some embodiments, users can provide information about applications. This information can be stored by the security monitoring and control system 102.

At step 1112, security information about each of the one or more applications that have been accessed may be determined. The security information may include information about one or more security-related events of the application. The security information may provide a measure of security (e.g., a security score) about one or more features of the application related to security. In at least one embodiment, the security information may be obtained via one or more feeds 902 disclosed with reference to FIG. 9 . The measure of security provided by the feed may correspond to a security feature of the application. The measure of security may be based on one or more indicators or features. The security information may be determined using domain information, organizational information, or a combination thereof. In some embodiments, the security information may be obtained based on information identifying the application.

Security information can be obtained from one or more sources. A source may include a third-party source that aggregates security-related information about applications, including the organization(s) that provide these applications. The security monitoring and control system 102 may maintain its own data store of application security information based on monitoring network activity. In some embodiments, users can provide security information about applications. This information can be stored by the security monitoring and control system 102. A measure of security can be indicated in the security information provided by any of these sources. In some embodiments, security information can be obtained from one or more sources in different formats. Each data source can provide a specific type of security information. The security information can be normalized or processed to provide a measure of security in a specific format or scale.

Step 1114 may be performed based on performing one or more of steps 1110 and/or 1112. At step 1114, a measure of security (e.g., an application risk score) is calculated as an individual measure for each application that has been accessed from the one or more applications. The measure of security may be a value or indication of the security of the application. For example, the measure of security may be a value on a scale of 1 (e.g., low security risk) to 5 (e.g., high security risk). The scale for the measure of security may be based on one or more characteristics of security, or may be based on a set of characteristics collectively.

In some embodiments, a measure of security can be calculated for multiple applications based on one or more characteristics (such as the type or security risk of the application). The technique disclosed with reference to Figures 8 and 9 can be used to calculate the measure of security. The measure of security can be calculated using any information determined in the previous steps of flowchart 1100. For example, organizational information, security information, or a combination thereof can be used to calculate the measure of security. The measure of security can be calculated for one or more indicators or characteristics of security. A weight value can be determined for each characteristic. Based on the indicators and weights, the measure of security can be calculated for each characteristic individually or collectively. In some embodiments, the measure of security can be calculated for an application based on use by multiple users (such as a group of users as tenants with access accounts). The steps in flowchart 1100 can be implemented for applications accessed by multiple users. The measure of security can be calculated for a type of application across one or more providers and/or one or more accounts.

In some embodiments, at step 1116, a display may be provided to display information about each of the one or more applications that have been accessed. The display may be a graphical interface provided within the application or a web browser. The display may be interactive to monitor and manage the security of the application. The display may be generated by the security monitoring and control system 102. Providing the display may include rendering the display at the client. Upon generation, the display may be sent to the client for presentation. Examples of various interactive displays are disclosed with reference to Figures 15-26. In some embodiments, the display may be provided as a report in a message sent to the client. The report may be a notification regarding security related to the application.

At step 1118, one or more remediation actions may be performed for each of the one or more accessed applications. A remediation action is an action performed on a remediation or correction basis to address security concerns (e.g., security risks or threats) posed by an application. Examples of remediation actions include, but are not limited to, sending a notification message regarding the security of an application, displaying information regarding the security of an application, and adjusting the operation and/or access of an application (e.g., restrictive access adjustments).

For example, controlling access to an application can include blocking or preventing a user or group of users from accessing the application. Limiting, blocking, or preventing access to an application can be accomplished in a number of ways. One or more instructions can be sent or configured to adjust access to the application. For example, one or more instructions can be configured on a network within an organization to deny any request to the application, or to prevent requests from being transmitted outside the organization, effectively denying or preventing access. One or more instructions can be configured to deny certain types of requests to the application. Users can be prompted in an interface to provide information to configure access to the application so that it is restricted according to policy.

In another example, the action may be to place information about the application on a whitelist or blacklist to allow or deny access to the application, respectively. In some embodiments, remedial actions may not be performed for each application based on evaluating the application according to a policy.

Remedial actions for an application may be performed based on a measurement of the security of the application. Remedial actions may be automatic, manual, require user or administrator participation, or a combination thereof. Actions may be configured based on input from a user (e.g., an analyst). Remedial actions may be performed based on one or more policies. Policies may be user configurable and/or adjusted based on security feedback, such as the technology disclosed with reference to FIG9 . For example, remedial actions may be performed based on a measurement of the security of an application that meets a risk threshold (e.g., high risk). In some embodiments, remedial actions may be performed based on security information for one or more features of the application. A measurement of security may be calculated based on those features. As such, remedial action may be based on a measurement of the security of those features for which the measurement of security was calculated. Examples of measurements of security and remedial actions are shown with reference to FIG15-26 .

In some embodiments, remedial actions may be performed based on a security policy. The security policy may define one or more remedial actions to be performed based on a security measurement and/or security information. The security policy may be applied based on a security measurement and/or any information disclosed herein (e.g., organizational information or security information). Operations may be performed to assess security risks or threats. The security risk or threat may be based on a security measurement and/or security information (e.g., one or more security indicators). The policy may define one or more criteria, such as a threshold (e.g., a security threshold or a risk threshold) for when remedial action is taken. The criteria may be defined by one or more values based on a scale of security measurements or a scale set by a provider of security indicators. For example, applying the security policy may include determining whether the security measurement meets an applied risk threshold. The security measurement may be compared to one or more values in the policy to assess the severity of the security risk. The values may be defined based on one or more security indicators. These values may be defined based on one or more security indicators such that the security measurement is compared to a threshold defined based on the security indicators used to calculate the security measurement. Security indicators obtained in the security information may also be compared to further assess the security risk.

In some embodiments, the remedial action may be to configure one or more aspects of the computing environment to prevent access to applications within the computing environment. One or more instructions may be sent to the computer systems and/or network devices of the organization's computing environment to specify which applications to block and to whom the applications are blocked or restricted. Access to applications may be configured through policies or other configurable information that can control access within the computing environment. In some embodiments, instructions may be sent to an agent on a client device operated by a user. The agent may be instructed to change its operation to prevent access to a particular application and/or to change the operation of the application in the particular environment in which the client device is used. The remedial action may be to prevent access to the application by multiple users. In some embodiments, the remedial action may include sending one or more instructions to a service provider system. For example, instructions may be sent to the service provider system to adjust one or more security controls and/or settings for accessing the application. The service provider system may provide an interface for accessing the security controls and/or settings. The remedial action may include using the interface (e.g., by making a call) to adjust the security controls and/or settings of the application.

Remedial actions may include alerting users. Alerts about security risks and/or changes in access to applications may be sent to administrators or other users' devices. In some embodiments, remedial actions for applications include causing a graphical interface to prompt the user to adjust configuration operations for the application.

Flowchart 1100 may end at step 1120 .

Figure 12 illustrates a flowchart 1200 of a process for discovering and managing application risks according to an embodiment. In many embodiments, one or more features of the process discussed below can be performed by the security monitoring and control system 102 of Figure 1. Flowchart 1200 can be implemented as part of or using the techniques disclosed with reference to Figures 10 and 11. The process disclosed with reference to flowchart 1200 can be implemented to assess and manage the security of an application that has been used across multiple service providers (e.g., cloud service providers). The application can be a first application that enables or facilitates access to a second application and/or data of the second application provided by a service provider that is different from or the same as the service provider that applied the first application.

Flowchart 1200 may begin at step 1202 by obtaining data (e.g., "first data") about one or more applications accessed from a service provider system (e.g., a first service provider system) on an organization's network. The first data may be obtained via a programming interface provided by the first service provider system of the service provider. The data may provide access information, such as the applications and/or data being accessed and information about how to access the applications and/or data. The access information may provide information about third-party applications accessing the applications or their data from the first service provider system.

At step 1204, data (e.g., "second data") about one or more applications accessed from a service provider system (e.g., a second service provider system) on the organization's network may be obtained. The second data may be obtained via a programming interface provided by the service provider's second service provider system. The data may provide access information, such as the applications and/or data accessed and information about how to access the applications and/or data. The access information may provide information about third-party applications accessing the applications or their data from the second service provider system.

Access information from several service provider systems can be aggregated to assess the security of applications accessed by users on an organization's network. Access information from each service provider system can be processed to identify one or more applications accessed by users on the organization's network. Applications can be distinct but share one or more common attributes. Applications can be distinct but related based on the user who allows third-party applications to access the applications and/or their data. Allowing third-party applications access may be risky and/or unacceptable to the organization.

At step 1206, access information for an application (e.g., a third application) accessed by the user may be determined. In at least one embodiment, the application may be a type of application accessed through the first service provider system and the second provider system. The access information from each service provider system may be processed to identify the application type of the application accessed through the first service provider system and the second provider system. The application may be identified as a type of application based on the application information obtained from each provider system. One or more data sources may be used to retrieve application information for each application accessed. In steps subsequent to step 1206, where the application(s) accessed by the user are of a common type, the information determined in the following steps may be determined for each application. A measure of security may be calculated for each application or combination of applications. The measure of security may be based on an average or a measure based on a combination of measures of security for each application.

In at least one embodiment, access information from each service provider system can be used to determine that the same third-party application is being used to access different applications and/or their data provided by each service provider system. Each service provider system can provide information that can indicate that an application is being accessed through a third-party application.

At step 1208, domain information for the provider system of the application(s) may be determined. Steps 1210 and 1212 may be performed simultaneously or in any order based on the identified applications. At step 1210, organizational information may be determined for the application. At step 1212, security information may be determined for the application. The information determined for flowchart 1200 may be determined using techniques disclosed herein, such as those disclosed with reference to FIG. 11 .

At step 1214, a measure of security may be calculated for the accessed applications. The measure of security may be calculated for each application or for a combination of applications identified as being accessed by the user. The measure of security may be based on an average or a combination of measures of security for each application.

At step 1216, an interactive display of information about the application(s) that have been accessed is provided. The interactive display may be included as part of or generated as a graphical interface. The graphical interface may display a measure of the security of the application.

At step 1218, remedial actions may be performed on the accessed application(s). Remedial actions may be performed as disclosed herein, such as the techniques disclosed with reference to FIG. 11 . In the event that the application is a third-party application that provides access to the application or its data at the service provider system, the third-party application may be prevented from accessing the application. In some embodiments, a request may be sent to the service provider system to limit or deny access to the application(s) and/or its data by the third-party application. In some embodiments, an interactive display may be modified to prompt the user to adjust settings for the third-party application to limit or revoke access to the application and/or its data from the service provider system.

Flowchart 1200 may end at step 1220 .

XI. Measuring the Security of Computing Users Based on Application Usage

Figures 13 and 14 illustrate techniques for calculating a measure of a user's safety based on application usage according to some embodiments. Specifically, Figure 13 illustrates an example of a sequence flow chart 1300 of processing. These techniques can be implemented by the security monitoring and control system 102. The technique disclosed with reference to Figure 9 can be used to calculate a measure of a user's safety.

A measure of a user's security (also referred to herein as a "user risk score") can provide an indication of the risk or threat to the security of an organization that the user may pose. As discussed above, a user may pose a security threat to an organization's network based on using applications in an unauthorized or unsafe manner. Such use may expose the organization to vulnerabilities in its private network and data. In some cases, an application may pose a threat to an organization based on inefficient or improper use of the organization's resources (such as corporate networks and/or computing resources). The user risk score can provide a measure that indicates the severity of security threats associated with users in the organization. The user risk score can be continuously generated by profiling the user's actions on the application.

In some embodiments, the security monitoring and control system 102 may provide a graphical interface that can be presented to display information about and/or related to risk scores. An example of a graphical interface is shown in priority application U.S. Provisional Application No. 62/460,716, filed on February 17, 2017, entitled “Systems and Methods for Discovering and Monitoring Unsanctioned Enterprise Assets” [Attorney Docket No. 088325-1039197 (185502US)]. The graphical interface can be presented in a console as a user report with risk scores and associated visualizations (KSIs). The graphical interface can serve as a single pane of glass that combines risk elements from various sources and presents them in a unified manner for both approved and unapproved applications. The graphical interface enables users (e.g., security administrators) to view associated risk indicators to understand why some users have high risk scores, including unusual actions performed in applications, access to risky unapproved applications, and so on. This helps users take or configure remedial actions such as blocking the application in the firewall, educating users to avoid the application, or suspending the user account. The graphical interface can enable users to configure customized alerts based on creating policies that match certain conditions (such as risky application score, application category, user risk score, etc.).

A measure of a user's security may be calculated based on information provided by one or more third-party sources, one or more users, sources managed and curated by the security control and management system 102, or a combination thereof. The measure of a user's security may be based on information including, but not limited to, information about applications and the providers of the applications ("organizational information"), information about the security of the applications ("security information"), and/or usage associated with the applications ("application usage information"). The organizational information and security information may include information disclosed herein, such as measures regarding determining the risk of an application. The application usage information may indicate information about the use of the application, such as the type of operations/actions performed (e.g., large exports of data or contact downloads for an application, or excessive file access), the categories of applications accessed (e.g., applications associated with malware websites, information leak websites, or applications/tools used by hackers will increase the user's risk score), or unusual deviations from the use of the application. For example, application usage information may provide an indication of malware activity based on a user who rarely uses a provider system to access files suddenly beginning to download a large number of documents.

The measure of security for a user can be a value on a scale that defines a measure of the user's security risk (e.g., a user risk score). For example, the scale can be defined as between 1 and 5, where higher values represent greater risk to the user. In other embodiments, any of a variety of ranges and values suitable for a particular assessment can be used. The score can be used by the security monitoring and control system 102 to provide reminders, provide reports, and/or perform remedial actions. The measure of security is a value used as a priority indicator to help users (e.g., security administrators) mitigate security threats posed by applications. In some embodiments, the measure of security can be calculated in several ways.

A user risk score can indicate a measure of security regarding the severity of a security threat posed to an organization by a specific application. A user risk score can be calculated based on one or more indicators (also referred to herein as "threat indicators" or "security indicators"). Each indicator can be a value or information indicating a security risk. For example, in Figure 13, a unique security indicator can be obtained from one or more data sources (also referred to herein as a "risk score feed"), providing a "feed" of one or more application indicators. Each unique indicator ("I") (collectively referred to herein as indicators "I ₁ , I ₂ , ..., _In " 1304) can be provided separately by each data source. An indicator can be an indicator of a specific type of information related to or about an application, such as the type of information indicated above. An indicator can be a value that provides an indication of a security threat posed by a user. For example, a first indicator can be a first value indicating a first security threat to a user. A second indicator can be a second value indicating a second security threat to a user. The first indicator can be obtained from a first source, and the second indicator can be obtained from a second source.

Indicators 1302 may include, for example, a user risk score for authorized applications, a risk score for unknown/unauthorized applications, or actions associated with an application. The following example illustrates how a user risk score is assessed. Note that the number of indicators used in this example is limited, but the system can be configured to use additional indicators. In one example, a first indicator _I1 may be a user risk score obtained from a data source for applications authorized (e.g., approved) for use by the organization. The user risk score may be calculated from user actions within the approved application. This risk score combines various user activities within the approved application, such as uploading/downloading documents. A second indicator _I2 may be a measure of the deviation in upload volume for unapproved applications. For example, the second indicator may be calculated as (1-day upload volume - 30-day average upload volume) / (30-day upload volume standard deviation). A third indicator _I3 may be a measure of the deviation in risk scores for visited sites. For example, the third indicator may be calculated as (1-day sum of application risk scores - 30-day daily average application risk scores) / (30-day daily application risk score standard deviation). In this example, all three indicators may be used to calculate a user risk score.In some embodiments, additional indicators such as those described with reference to FIG. 14 may be used to calculate a user risk score.

In some embodiments, the security monitoring and control system 102 can use risk scores for applications and users to generate a graph. Additional indicators can be derived using one or more graphs. Turning now to Figure 14, this figure is a graph 1402 of a first cluster of users accessing applications A and B. Figure 14 illustrates a graph 1404 of a second cluster of users accessing applications X, Y, and Z. Each graph can be generated using information about user and application usage obtained from one or more data sources (such as those disclosed herein). The graphs can be generated using one or more clustering algorithms known to those skilled in the art. Examples of clustering algorithms can include Markov clustering (MCL) algorithms. Using one or more graphs, a group of users can be identified based on one or more attributes, such as users who visit similar sites. This analysis can be performed using graph analysis, including one or more known graph analysis techniques, such as the Girvan-Newman edge clustering algorithm.

Continuing with the example of indicators _I1-3 above, a fourth indicator _I4 can be derived using graphical analysis. For example, _I4 can be calculated as (1-day sum of application risk scores - 30-day daily average application risk scores for the cluster) / (30-day daily application risk score standard deviation for the cluster). A fifth indicator _I5 can be calculated as (1-day upload volume - 30-day average upload volume for the cluster) / (30-day upload volume standard deviation for the cluster).

Each threat indicator may represent a user risk score, which is a value indicating a measure of the user's security risk or threat according to a scale defined by the source for that indicator. In some embodiments, the threat indicator may be compared to the scale to determine the user risk score. The scale may be defined by or based on information provided by the source.

The security monitoring and control system 102 can calculate a user risk score based on one or more threat indicators. In other words, a measure of security can be calculated as a combined measure of the user's security risk. Each indicator can be processed to adjust (e.g., normalize) its value to an adjusted value based on a scale to be used for all indicators (e.g., a scale of values from 0 to 100). Once normalized, the values of each indicator can be combined to determine a combined score.

The combined security score 1308, indicating a user risk measure, can be based on all or some of the indicators. The number and/or type of indicators considered to determine the combined security score can be configurable. As such, the security monitoring and control system 102 can be configured to add or remove sources from which indicators are obtained, and can be configured to add or remove indicators 1304 obtained from sources. In some embodiments, a graphical interface can be presented that enables the user to configure the number and type of indicators considered for the score. In some embodiments, the combined score can be based on a security policy that defines the criteria by which the score is calculated. The criteria can be based on one or more attributes of security. These attributes can be used to select the indicators to be considered for the score.

In some embodiments, the combined user risk score 1308 can be based on a combination of indicators using a weight value for each indicator. For example, a weight ("W") (collectively referred to herein as weights " _W1 , _W2 , ..., _Wn " 1304) can be selected to apply to one or more specific indicators used to calculate the combined score 1308. The weight can be a value that is an integer of 1 or a fraction of 1. Different weights can be selected for different indicators. In the example above, the first weight can be a first weight value for the first indicator, and the second weight can be a second weight value for the second indicator. The weight values can be different.

The weights can be configured by the user through a graphical interface. In some embodiments, the weights can be selected based on a security policy, where the security policy is defined based on specific indicators being given specific weights. When an indicator has more importance or suggestion for a security risk, a larger weight value can be considered. When an indicator has less importance or suggestion for a security risk, a smaller weight value can be considered. In some embodiments, weights can be selected for all indicators from a particular source. For example, a weight can be applied to all indicators from that source based on the reliability or trust of the source. In some embodiments, the security monitoring and control system 102 can store data about the threat research analysis of the application. This data can be used to selectively select a weight for each indicator.

A combined score can be calculated based on the indicators and the weights assigned to those indicators. In at least one embodiment, the combined score can be calculated using equation 1308 to provide a combined score = (I ₁ (W ₁ ) + I ₂ (W ₂ ) + ..... _In (W _n )) / (W _{1 +} W ₂ + .... W _n ). In this equation, the value of each indicator ("I") 1304 is calculated by multiplying each indicator by a corresponding weight ("W") 1306. A first value is calculated, which is the sum of the values calculated for each indicator (I ₁ (W ₁ ) + I ₂ (W ₂ ) + ..... _In (W _n )). A second value is calculated, which is the sum of each weight value applied to obtain the first value. The combined score 1308 can be calculated based on dividing the first value by the second value. In the example continuing from above, the user risk score may be calculated as (I ₁ (W ₁ )+I ₂ (W ₂ )+I ₃ (W ₃ )+I ₄ (W ₄ )+I ₅ (W ₅ ))/(W _{1 +} W ₂ +W _{3 +} W ₄ +W ₅ ).

In some embodiments, the security monitoring and control system 102 may obtain feedback 1310 from one or more users regarding the validity and accuracy of the combined score 1308. Feedback 1310 may be obtained via a network, facilitated by a graphical interface, or provided as manual feedback. Any of the sources, indicators, and/or weights may be adjusted based on the feedback 1310. Based on the feedback 1310, the combined score 1308 may be adjusted. A new combined score 1312 ("adjusted score") may be calculated in the same manner as the combined score 1308, except that the adjusted score 1312 may be calculated based on indicators and/or weights selected based on the feedback 1310. Sources, indicators, and/or weights may be added or removed from the content used to calculate the combined score 1308. The weight values for the indicators may be periodically revised based on our security analysts and customer feedback to improve the risk score. The revision process for the indicator weights may be performed using automated machine learning algorithms, such as decision trees and neural networks.

A regression analysis 1314 can be performed based on each indicator and/or combined score for a particular security threat. The regression analysis can include building and updating a linear regression model. The linear regression model can provide an output such as S = c ₁ (I ₁ ) + c ₂ (I ₂ ) + ... + c _n (I _n ). The coefficients c _i calculated by the regression model can be new or modified weights that replace the initial weights used to calculate the combined score 1308. As more feedback and more data are collected, the model will provide greater accuracy.

In some embodiments, calculating a user risk score may include scaling the individual scores (or Z-scores) calculated for each indicator. In one illustrative approach, negative z-scores are set to 0. This is because below-average activity levels are considered within the normal range. Z-scores above 6 may be scaled to 6 (a z-score of 3 is considered an outlier, and a z-score of 6 is considered an extreme outlier). Z-scores between 0 and 6 are scaled between 0 and 100 as follows: scaled z-score = z-score * (100/6).

Continuing with the example discussed above, indicator _I1 may be an approved application risk score of 92 (on a scale of 0-100), so this score may not be scaled. Indicator _I2 may be calculated as a raw z-score: (100-20)/10=8, where today's upload is 100MB and the average over the last 30 days is 20MB with a standard deviation of 10MB. Since the score is 8, scaling may be required. The scaled score may be 100.000. Indicator _I3 may be calculated as a raw z-score: (50-20)/5=6, where the average risk score of sites visited today is 30, the average over the last 30 days is 20, and the standard deviation is 5. _I3 may be scaled to 100.000. Indicator _I4 may be calculated as a raw z-score: (30-25)/10=0.5, where the average risk score of sites visited today is 30, the average for the peer group over the last 30 days is 25, and the standard deviation is 10. The score of I ₄ can be scaled to 8.3333. Indicator I ₅ can be calculated as the raw z score: (40-60)/30=-0.667, where today's upload is 40MB, the peer group average over the past 30 days is 60MB, and the standard deviation is 30MB. The score of I ₅ can be scaled to 0.

Continuing with this example, based on data quality and quantity, each indicator can be assigned a data source weight as follows: _W1 = 60%, _W2 = 30%, W3 ₌ 5%, W4 ₌ 2%, W5 ₌ 3%. The weights and indicators (which can be scaled) can be used to calculate a user risk score. In this example, the user risk score (scaled Z score) can be calculated as (0.60*92) + (0.30*100.0) + (0.05*100.0) + (0.02*8.333) + (0.03*0) = 90 (rounded). If the administrator can verify that the user has no other significant risks, the administrator can adjust the final score from 90 to 25. The score can be applied to a linear regression analysis 1314 using a linear regression algorithm: Features ( _I1 , _I2 , ...): 92, 100.0, 100.0, 8.333, 0, target variable (adjusted combined score): 25. Using these along with the features and target variable for all user scores, the regression algorithm will calculate new weights (w1, w2, ...) for each indicator of the customer and update the risk scores for all users (in the customer tenant) accordingly.

XII. Interface for discovering and managing application security

Figures 15-26 illustrate interfaces according to some embodiments, for example, graphical interfaces for discovering and managing the security of applications in a computing environment. Each graphical interface, such as a graphical user interface (GUI), can be displayed at a client that accesses services provided by the security monitoring and control system 102 disclosed in the figures. The graphical interface can be displayed as part of an access portal (such as a website) or displayed within an application. Additional examples of graphical interfaces are shown in priority application U.S. Provisional Application No. 62/460,716, filed on February 17, 2017, entitled "Systems and Methods for Discovering and Monitoring Unsanctioned Enterprise Assets" [Attorney Docket No.: 088325-1039197 (185502US)].

In this disclosure, an "element" may be included in a graphical interface. An element may be a displayable and/or part of a graphical interface. Examples of elements include, but are not limited to, controls, buttons, navigation bars, or other visible components, which may be part of an interface that can be perceived by sound, vision, touch, or a combination thereof. An element may receive input. For example, an interactive element may be an element that is interactive to receive input. An interactive element may receive input to enable interaction with the graphical interface. For example, an interactive element may be one of many elements in a graphical interface, such as a heat map for displaying network data.

In Figure 15, a GUI 1500 is shown in an application that enables a user to discover and manage the security of applications accessed on an organization's network. GUI 1500 is shown as an interactive interface (e.g., a dashboard) that is interactive for viewing applications that have been discovered to be accessed on an organization's network. The GUIs in Figures 15-26 can be presented as part of a dashboard in an application or based on its interaction with a dashboard in an application. The dashboard can include one or more interactive tabs for the interactive interface, such as "Summary," "Application Discovery," and "Key Security Indicators." The information and elements in the interface can be referenced in multiple GUIs of the accompanying drawings to illustrate additional examples.

Turning now to the example of FIG. 15 , GUI 1500 is shown with "Application Discovery" selected. Information about applications can be displayed based on how the information was obtained. For example, information about applications can be displayed under a tab in the "From Registered Applications" interface for applications known based on registration. In another example, information about applications can be displayed under a tab in the "From Logs" interface 1504 for applications discovered based on the techniques disclosed herein for identifying applications accessed by a user. Functionality shown for applications in any tab can be combined into a single tab or implemented in another tab.

GUI 1500 may include an interactive element 1506 to selectively provide input to filter the discovered applications. Filtering may be performed based on one or more attributes, such as time, date, application type, action, risk, or any other type of information displated for the discovered applications.

As disclosed herein, many technologies can be used to discover applications, such as checking the log files (e.g., system log files) used for network activities. GUI 1500 can include interactive elements 1510 to provide input to configure how to collect log files for analysis to discover applications. Interaction with element 1510 can cause additional or modified GUI 2000 ("system log settings") to be displayed next to or with GUI 1500. GUI 2000 can enable users to configure how to collect log files. GUI 2000 can include one or more elements that can receive input to configure the token used to access log files, the path or source used for log files, the location (e.g., endpoint) for placing collected log files, and one or more connection parameters (e.g., port) for log files. The parameters used to configure log files can include one or more parameters for a secure location and/or the connection for storing log files.

GUI 1500 can be interactive to display a grid or table view of information about each application discovered. In the example shown, GUI 1500 is displayed with a row for each application that has been discovered and accessed. Each row displays information about the discovered application, including the domain of the host system providing the application, the highest risk, (one or more) incidents, and configurable remediation actions. Each entry or row in the view can be interactive, including fields within each entry. The highest risk can display information about different categories of security risks for the application.

GUI 1500 may include an interactive element 1508 to export a data file of information for any application entry in the table. The data file may be used to manage, monitor, and follow up on actions related to application security.

Interacting with the element corresponding to "Incident" in the entry of the table may result in GUI 1900 in FIG. This element may be interactive, allowing selection of one or more previously opened incidents. GUI 1900 may be displayed on top of or alongside GUI 1500. The GUI may be presented to display information about already opened incidents. GUI 1900 may include one or more interactive elements to provide input for configuring events for an application. Interaction with the element for opening an incident may be specific to the application corresponding to the element in the entry. GUI 1900 in FIG. 19 may include elements 1902 ("Category"), 1904 ("Discovered Application Name"), 1906 ("Vendor Domain"), 1908 ("Description"), 1910 ("Remedial Action"), 1912 ("Assigned To"), 1914 ("Priority"), 1916 ("Approve"), 1918 ("New Incident"), and 1920 ("Cancel").

Element 1902 may indicate one or more security risks for which the application has been discovered. Risks may have been selected in GUI 1500 before initiating a new incident. Element 1904 may indicate the name of the application. Element 1906 may indicate the domain of the host system hosting the application. Element 1908 may be interactive to specify a description of the incident. The description may be pre-populated based on the security risk. The description may be used to provide a description of the application's usage. Element 1910 may be interactive to specify one or more remediation actions. Remediation actions may be pre-selected in element 1910 based on the security risk. Element 1912 may be interactive to specify one or more users to be notified about the security of the application. Element 1914 may be interactive to specify the priority of the application. Element 1916 may be interactive to indicate approval of incident creation. Element 1918 may be interactive to submit the request to create the incident based on input in GUI 1900. Element 1920 may be interactive to cancel the incident creation.

16. The GUI 1500 may be modified by interacting with an element corresponding to an entry in the table to display information about the application found corresponding to that entry. For example, in Figure 16, GUI 1500 is shown as an expanded GUI 1600 (e.g., a graphical interface) with entries for the application found. GUI 1600 may display organizational information about the application. The organizational information may include information about the use associated with the application, such as the number of users, network activity (e.g., uploaded data and downloaded data), the category of the application, and information about the organization providing the application. Information about the organization may include name, address, domain, website, description of the organization, and/or other information about the organization's registration of the application.

GUI 1600 can display information about each security risk displayed in an entry in GUI 1500. Each security risk can be displayed along with information describing the security risk and can be displayed with a visual appearance related to the security risk. For example, the visual appearance can have an image and color to indicate the severity of the risk. The security risk can be determined based on security indicators in the security information determined for the discovered application. Information specific to the statistics or underlying information about the security risk can be displayed. In some embodiments, the security risk can be based on information provided by a third-party source, a user, and/or curated by the security monitoring and control system 102. In some embodiments, the security risk can be based on one or more indicators of security. The security risk can be based on the application of one or more security policies. The security policy can be applied using a measure of security determined for one or more security risks.

In the example shown in FIG16 , GUI 1600 may include an element for each security risk. The elements corresponding to the security risks may be interactive so that the GUI is displayed either alongside or in addition to GUI 1500. In one example, GUI 1700 of FIG17 may be displayed based on an interaction with element 1604 for a security risk ("IP Reputation"). In another example, GUI 1800 of FIG18 may be displayed based on an interaction with element 1606 for a security risk ("Application Security").

Each of GUIs 1700 and 1800 displays information about a specific security risk. This information may correspond to one or more events related to the security risk of accessing an application. Information may be displayed based on each event or event category, including information about the actual event. Each security risk may be displayed along with information about the issue. One or more remediation actions may be provided. This information may be provided by a third-party source, a user, and/or curated by the security monitoring and control system 102. For example, this information may be provided by the provider of the application. Each security risk may be displayed with a visual appearance indicating the severity of the risk. In some embodiments, each risk may be presented to an element such that the element may be interactive, causing another GUI to be displayed to automate and/or configure remediation actions.

Returning now to FIG. 15 , for each entry corresponding to a uniquely discovered application, an element such as 1512 may be presented in GUI 1500. The element may be interactive so as to configure one or more remediation actions for the application. The element may provide one or more options that may be specific to the security risk of the application. Interaction with the element may cause another GUI to be displayed alongside GUI 1500 or in addition to GUI 1500 to configure remediation actions. A security risk may be selected so as to select remediation options for the security risk. In some embodiments, one or more remediation actions may be presented based on the security risk and one or more security policies. Executed remediation actions may also be displayed along with the entry for the discovered application.

Interacting with element 1502 may cause GUI 1500 to be updated to display GUI 2100 of FIG. 21 . Similar to GUI 1500 , GUI 2100 may display information about discovered applications. This information may include security risks. GUI 2100 may be interactive, just like GUI 1500 . Applications identified in GUI 2100 may be applications that have already been registered with the organization. Interacting with item 2102 in FIG. 21 may cause GUI 2200 to be displayed.

Similar to Figures 17 and 18, Figures 23-25 show GUIs with information about security risks displayed in GUI 2200. GUI 2200 may include element 2202 ("Endpoint Security") for endpoint security risks, element 2204 ("Network Security") for network security risks, and element 2206 ("IP Reputation") for IP reputation security risks. Each of element 2202, element 2204, and element 2206 may cause GUI 2300 of Figure 23, GUI 2400 of Figure 24, and GUI 2500 of Figure 25, respectively, to be displayed alongside or in addition to GUI 2200.

Figure 26 illustrates a GUI 2600 for "Application Discovery" showing details about the applications that were discovered. GUI 2600 may include an interactive view 2602 that displays an entry in a table view for each unique application discovered. Similar to GUI 1500 of Figure 15, each entry may include information about the application, including the application name and the provider of the application. As with GUI 1500, each entry may be interactive so that additional information about the application corresponding to the entry is displayed. An entry may indicate one or more highest risks associated with one or more indicators. Unlike GUI 1500, GUI 2600 may show an entry for each unique application discovered, regardless of the source of the discovery (e.g., log or registry).

In some embodiments, the entry may indicate other information about the event related to the discovered application.The entry may indicate information about the use of the application, such as the number of users who have accessed the application and the date the application was accessed.

In some embodiments, the entries in the table view may correspond to registered applications and/or applications discovered from the log. Each entry may include one or more elements to perform remedial actions. View 2602 may include one or more interactive elements (e.g., toolbars) to configure when to identify accessed applications and/or a time period for automatically discovering applications. The toolbar may be interactive to configure notifications, settings, and searches for accessed applications.

GUI 2600 may include an area 2604 that displays a visualization of statistical information regarding a measurement of application usage. The statistical information may be displayed based on one or more security risks. GUI 2600 may include an area 2606 that displays a visualization of application usage based on the measurement of usage. GUI 2600 may include an area 2608 that displays a visualization of groupings of applications based on information such as category, domain, service provider, security risk, or other categories of information associated with the applications.

XIII. General-purpose computer system for access management system and client system

27 depicts a simplified diagram of a distributed system 2700 for implementing an embodiment. In the illustrated embodiment, the distributed system 2700 includes one or more client computing devices 2702, 2704, 2706, and 2708 configured to execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), etc., over one or more networks 2710. A server 2712 can be communicatively coupled to the remote client computing devices 2702, 2704, 2706, and 2708 via the network 2710.

In various embodiments, server 2712 may be adapted to run one or more services or software applications. In certain embodiments, server 2712 may also provide other services, or the software application may include both non-virtualized and virtualized environments. In some embodiments, these services may be provided to users of client computing devices 2702, 2704, 2706, and/or 2708 as web-based or cloud services or under a software-as-a-service (SaaS) model. Users operating client computing devices 2702, 2704, 2706, and/or 2708 may then interact with server 2712 using one or more client applications to utilize the services provided by these components.

In the configuration depicted in Figure 27, software components 2718, 2720, and 2722 of system 2700 are shown as being implemented on server 2712. In other embodiments, one or more components of system 2700 and/or the services provided by these components may also be implemented by one or more of client computing devices 2702, 2704, 2706, and/or 2708. A user operating a client computing device can then utilize one or more client applications to use the services provided by these components. These components can be implemented in hardware, firmware, software, or a combination thereof. It should be understood that a variety of different system configurations are possible, which may differ from distributed system 2700. Therefore, the embodiment shown in Figure 27 is an example of a distributed system for implementing an embodiment system and is not intended to be limiting.

Client computing devices 2702, 2704, 2706, and/or 2708 may include various types of computing systems. For example, the client computing device may include a portable handheld device (e.g., a cellular phone, a computing tablet, a personal digital assistant (PDA)) or a wearable device (e.g., a Google head-mounted display) running software such as Microsoft Windows and/or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 10, and Palm OS. The device may support various applications, such as various Internet-related applications, email, short message service (SMS) applications, and may use various other communication protocols. The client computing device may also include a general-purpose personal computer, for example, a personal computer and/or laptop computer running various versions of Microsoft Apple and/or Linux operating systems. The client computing device may be a workstation computer running any of various commercial or UNIX-like operating systems (including but not limited to various GNU/Linux operating systems such as Google Chrome OS). Client computing devices may also include electronic devices capable of providing network(s) 2710 communications, such as thin client computers, Internet-enabled gaming systems (e.g., Microsoft game consoles with or without gesture input devices), and/or personal messaging devices.

Although the distributed system 2700 in Figure 27 is shown as having four client computing devices, any number of client computing devices can be supported. Other devices, such as devices with sensors, etc., can interact with the server 2712.

The network(s) 2710 in the distributed system 2700 may be any type of network familiar to those skilled in the art that can support data communications using any of a variety of available protocols, including, but not limited to, TCP/IP (Transmission Control Protocol/Internet Protocol), SNA (Systems Network Architecture), IPX (Internetwork Packet Exchange), AppleTalk, etc. By way of example only, the network(s) 2710 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating under any Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol suite, and/or any other wireless protocol), and/or any combination of these and/or other networks.

The server 2712 may be comprised of one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and/or combination. The server 2712 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization. One or more flexible logical storage device pools may be virtualized to maintain virtual storage devices for the servers. The virtual network may be controlled by the server 2712 using software-defined networking. In various embodiments, the server 2712 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the server 2712 may correspond to a server for performing processing as described above according to an embodiment of the present disclosure.

The server 2712 can run an operating system including any of the operating systems discussed above, as well as any commercially available server operating system. The server 2712 can also run any of a variety of additional server applications and/or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, servers, database servers, etc. Exemplary database servers include, but are not limited to, commercially available database servers from Oracle, Microsoft, Sybase, IBM (International Business Machines), etc.

In some implementations, the server 2712 may include one or more applications to analyze and integrate data feeds and/or event updates received from users of the client computing devices 2702, 2704, 2706, and 2708. By way of example, the data feeds and/or event updates may include, but are not limited to, feeds, updates, or real-time updates received from one or more third-party information sources and continuous data streams, which may include real-time events associated with sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. The server 2712 may also include one or more applications that display the data feeds and/or real-time events via one or more display devices of the client computing devices 2702, 2704, 2706, and 2708.

Distributed system 2700 may also include one or more databases 2714 and 2716. These databases can provide a mechanism for storing information, such as user interaction information, usage pattern information, adaptation rule information, and other information used by embodiments of the present disclosure. Databases 2714 and 2716 can reside in various locations. As an example, one or more of databases 2714 and 2716 can reside on a non-transient storage medium local to (and/or residing in) server 2712. Alternatively, databases 2714 and 2716 can be remote from server 2712 and communicate with server 2712 via a network-based or dedicated connection. In one set of embodiments, databases 2714 and 2716 can reside in a storage area network (SAN). Similarly, any necessary files for executing the functions possessed by server 2712 can be appropriately stored locally and/or remotely on server 2712. In one set of embodiments, databases 2714 and 2716 can include a relational database suitable for storing, updating, and retrieving data in response to SQL-formatted commands, such as a database provided by Oracle.

In some embodiments, a cloud environment can provide one or more services. Figure 28 is a simplified block diagram of one or more components of a system environment 2800 in which services can be provided as cloud services, according to an embodiment of the present disclosure. In the embodiment shown in Figure 28, system environment 2800 includes one or more client computing devices 2804, 2806, and 2808 that can be used by users to interact with a cloud infrastructure system 2802 that provides cloud services. Cloud infrastructure system 2802 can include one or more computers and/or servers, which can include those described above for server 2712.

It should be appreciated that the cloud infrastructure system 2802 depicted in FIG28 may have other components than those depicted. Furthermore, the embodiment shown in FIG28 is merely one example of a cloud infrastructure system that may incorporate embodiments of the present disclosure. In some other embodiments, the cloud infrastructure system 2802 may have more or fewer components than shown, may combine two or more components, or may have a different configuration or arrangement of components.

Client computing devices 2804, 2806, and 2808 can be devices similar to those described above for client computing devices 2702, 2704, 2706, and 2708. Client computing devices 2804, 2806, and 2808 can be configured to operate client applications, such as web browsers, proprietary client applications (e.g., Oracle Forms), or some other application that can be used by users of the client computing devices to interact with cloud infrastructure system 2802 to use services provided by cloud infrastructure system 2802. Although exemplary system environment 2800 is shown with three client computing devices, any number of client computing devices can be supported. Other devices, such as devices with sensors, can interact with cloud infrastructure system 2802.

The network(s) 2810 can facilitate communication and data exchange between the client computing devices 2804, 2806, and 2808 and the cloud infrastructure system 2802. Each network can be any type of network familiar to those skilled in the art that can support data communication using any of a variety of commercially available protocols, including the protocols described above for the network(s) 2110.

In certain embodiments, the services provided by the cloud infrastructure system 2802 may include a host of services available on-demand to users of the cloud infrastructure system. A variety of other services may also be provided, including but not limited to online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. The services provided by the cloud infrastructure system can be dynamically scalable to meet the needs of its users.

In certain embodiments, a specific instantiation of a service provided by cloud infrastructure system 2802 may be referred to herein as a "service instance." Generally speaking, any service made available to users from a cloud service provider's system via a communications network (such as the Internet) is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are distinct from the consumer's own on-premises servers and systems. For example, a cloud service provider's system may host an application, and users may subscribe to and use the application on demand via a communications network such as the Internet.

In some examples, services in a computer network cloud infrastructure may include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided by the cloud provider to users, or as otherwise known in the art. For example, a service may include password-protected access to remote storage on the cloud via the Internet. As another example, a service may include a hosted relational database and scripting language middleware engine based on a web service for private use by networked developers. As another example, a service may include access to an email software application hosted on the cloud provider's website.

In certain embodiments, the cloud infrastructure system 2802 may include application suites, middleware, and database service offerings delivered to consumers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud provided by the assignee.

The cloud infrastructure system 2802 can also provide computing and analytical services related to "big data." The term "big data" is generally used to refer to extremely large data sets that can be stored and manipulated by analysts and researchers to visualize large amounts of data, detect trends, and/or otherwise interact with the data. This big data and related applications can be hosted and/or manipulated by the infrastructure system at many levels and different scales. Dozens, hundreds, or thousands of processors linked in parallel can act on this data to render it or simulate external forces on the data or its content. These data sets can involve structured data, such as data organized in a database or otherwise organized according to a structured model, and/or unstructured data (e.g., emails, images, data blobs (binary large objects), web pages, complex event processing). By leveraging the ability of embodiments to relatively quickly focus more (or fewer) computing resources on a target, the cloud infrastructure system can be better used to perform tasks on large data sets based on needs from enterprises, government agencies, research organizations, private individuals, a group of like-minded individuals or organizations, or other entities.

In various embodiments, the cloud infrastructure system 2802 can be adapted to automatically provision, manage, and track consumer subscriptions to services provided by the cloud infrastructure system 2802. The cloud infrastructure system 2802 can provide cloud services via different deployment models. For example, services can be provided under a public cloud model, where the cloud infrastructure system 2802 is owned by an organization that sells cloud services (e.g., owned by Oracle Corporation) and makes the services available to the general public or different industrial enterprises. As another example, services can be provided under a private cloud model, where the cloud infrastructure system 2802 operates only for a single organization and can provide services to one or more entities within the organization. Cloud services can also be provided under a community cloud model, where the cloud infrastructure system 2802 and the services provided by the cloud infrastructure system 2802 are shared by several organizations in a related community. Cloud services can also be provided under a hybrid cloud model, which is a combination of two or more different models.

In some embodiments, the services provided by cloud infrastructure system 2802 may include one or more services provided under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other categories of services including hybrid services. A consumer may subscribe to one or more services provided by cloud infrastructure system 2802 via a subscription order. Cloud infrastructure system 2802 then performs processing to provide the services in the consumer's subscription order.

In some embodiments, the services provided by the cloud infrastructure system 2802 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, a SaaS platform may provide the ability to build and deliver on-demand application suites on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. By utilizing the services provided by the SaaS platform, consumers can utilize applications executed on the cloud infrastructure system. Consumers can obtain application services without the need for consumers to purchase licenses and support separately. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility to large organizations.

In some embodiments, platform services can be provided by the cloud infrastructure system 2802 via a PaaS platform. The PaaS platform can be configured to provide cloud services belonging to the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared common architecture, and the ability to build new applications using the shared services provided by the platform. The PaaS platform can manage and control the underlying software and infrastructure used to provide PaaS services. Consumers can obtain PaaS services provided by the cloud infrastructure system 2802 without having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), and others.

By utilizing the services provided by the PaaS platform, consumers can adopt programming languages and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one embodiment, the database cloud service may support a shared service deployment model that enables organizations to pool database resources and provide database as a service to consumers in the form of a database cloud. The middleware cloud service can provide consumers with a platform for developing and deploying various business applications, and the Java cloud service can provide consumers with a platform for deploying Java applications in the cloud infrastructure system.

A variety of infrastructure services can be provided by IaaS platforms in cloud infrastructure systems. Infrastructure services facilitate the management and control of underlying computing resources (such as storage, network, and other basic computing resources) so that consumers can utilize the services provided by SaaS and PaaS platforms.

In certain embodiments, cloud infrastructure system 2802 may also include infrastructure resources 2830 for providing resources used to deliver various services to consumers of the cloud infrastructure system. In one embodiment, infrastructure resources 2830 may include a pre-integrated and optimized combination of hardware (such as servers, storage devices, and networking resources) for executing services provided by PaaS and SaaS platforms, as well as other resources.

In some embodiments, resources in the cloud infrastructure system 2802 can be shared by multiple users and dynamically reallocated as needed. Furthermore, resources can be allocated to users in different time zones. For example, the cloud infrastructure system 2802 can enable a first set of users in a first time zone to utilize the cloud infrastructure system's resources for a specified number of hours, and then enable the reallocation of the same resources to another set of users in a different time zone, thereby maximizing resource utilization.

In certain embodiments, a plurality of internal shared services 2832 may be provided that are shared by different components or modules of the cloud infrastructure system 2802 to enable provisioning of services by the cloud infrastructure system 2802. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling cloud support, email services, notification services, file transfer services, and the like.

In certain embodiments, cloud infrastructure system 2802 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) within the cloud infrastructure system. In one embodiment, cloud management functionality can include the ability to provision, manage, and track subscriptions received by cloud infrastructure system 2802 and the like.

28 , cloud management functionality may be provided by one or more modules such as an order management module 2820, an order orchestration module 2828, an order provisioning module 2824, an order management and monitoring module 2826, and an identity management module 2828. These modules may include or may be provided utilizing one or more computers and/or servers, which may be general purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and/or combination.

In exemplary operation, at 2834, a consumer using a client device (such as client computing device 2804, 2806, or 2808) may interact with cloud infrastructure system 2802 by requesting one or more services provided by cloud infrastructure system 2802 and placing an order for a subscription to the one or more services provided by cloud infrastructure system 2802. In certain embodiments, the consumer may access a cloud user interface (UI), such as cloud UI 2812, cloud UI 2814, and/or cloud UI 2822, and place a subscription order via these UIs. Order information received by cloud infrastructure system 2802 in response to the consumer placing the order may include information identifying the consumer and the one or more services provided by cloud infrastructure system 2802 to which the consumer intends to subscribe.

At step 2836, the order information received from the customer can be stored in the order database 2818. If this is a new order, a new record can be created for the order. In one embodiment, the order database 2818 can be one of several databases operated by the cloud infrastructure system 2818 and in conjunction with other system elements.

At step 2838, the order information may be forwarded to the order management module 2820, which may be configured to perform billing and accounting functions related to the order, such as verifying the order and, upon verification, booking the order.

At step 2840, information about the order may be communicated to order orchestration module 2822, which is configured to orchestrate the provisioning of services and resources for the order placed by the consumer. In some cases, order orchestration module 2822 may utilize the services of order provisioning module 2824 for provisioning. In certain embodiments, order orchestration module 2822 enables management of the business processes associated with each order and applies business logic to determine whether the order should proceed with fulfillment.

As shown in the embodiment depicted in FIG28 , at 2842 , upon receiving an order for a new subscription, order orchestration module 2822 sends a request to order provisioning module 2824 to allocate resources and configure the resources needed to fulfill the subscription order. Order provisioning module 2824 enables the allocation of resources for the services ordered by the consumer. Order provisioning module 2824 provides a level of abstraction between the cloud services provided by cloud infrastructure system 2800 and the physical implementation layer used to provision the resources for providing the requested services. This enables order orchestration module 2822 to be isolated from implementation details, such as whether services and resources are actually provisioned in real time, or pre-provisioned and allocated/assigned only upon request.

Once the services and resources are provisioned, a notification may be sent to the subscribed consumer indicating that the requested service is now ready for use, at step 2844. In some cases, information (e.g., a link) may be sent to the consumer enabling the consumer to begin using the requested service.

At step 2846, the consumer's subscription order may be managed and tracked by the order management and monitoring module 2826. In some cases, the order management and monitoring module 2826 may be configured to collect usage statistics regarding the consumer's use of the subscribed service. For example, statistics may be collected regarding the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

In certain embodiments, cloud infrastructure system 2800 may include an identity management module 2828 configured to provide identity services, such as access management and authorization services within cloud infrastructure system 2800. In some embodiments, identity management module 2828 may control information about consumers who wish to utilize services provided by cloud infrastructure system 2802. This information may include information authenticating the identities of these consumers and information describing the actions those consumers are authorized to perform with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). Identity management module 2828 may also include descriptive information about each consumer and management of how and by whom the descriptive information may be accessed and modified.

FIG29 illustrates an exemplary computer system 2900 that can be used to implement an embodiment of the present disclosure. In some embodiments, computer system 2900 can be used to implement any of the various servers and computer systems described above. As shown in FIG29 , computer system 2900 includes various subsystems, including a processing unit 2904 that communicates with multiple peripheral subsystems via a bus subsystem 2902. These peripheral subsystems may include a processing acceleration unit 2906, an I/O subsystem 2908, a storage subsystem 2918, and a communication subsystem 2924. The storage subsystem 2918 may include a tangible computer-readable storage medium 2922 and a system memory 2910.

The bus subsystem 2902 provides a mechanism for making the various components and subsystems of the computer system 2900 communicate with each other as desired. Although the bus subsystem 2902 is schematically shown as a single bus, the alternative embodiment of the bus subsystem can utilize multiple buses. The bus subsystem 2902 can be any of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, and a local bus utilizing any various bus architectures. For example, such architectures can include an industry standard architecture (ISA) bus, a microchannel architecture (MCA) bus, an enhanced ISA (EISA) bus, a video electronics standards association (VESA) local bus, and a peripheral component interconnect (PCI) bus, which can be implemented as a mezzanine bus manufactured according to the IEEE P1386.1 standard, or the like.

The processing subsystem 2904 controls the operation of the computer system 2900 and may include one or more processing units 2932, 2934, etc. The processing units may include one or more processors, including single-core or multi-core processors, one or more cores of a processor, or a combination thereof. In some embodiments, the processing subsystem 2904 may include one or more specialized coprocessors, such as a graphics processor, a digital signal processor (DSP), etc. In some embodiments, some or all of the processing units of the processing subsystem 2904 may be implemented using custom circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs).

In some embodiments, the processing unit in the processing subsystem 2904 can execute instructions stored in the system memory 2910 or on the computer-readable storage medium 2922. In various embodiments, the processing unit can execute various program or code instructions and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in the system memory 2910 and/or on the computer-readable storage medium 2922, potentially including on one or more storage devices. Through appropriate programming, the processing subsystem 2904 can provide a variety of functions.

In certain embodiments, a processing acceleration unit 2906 may be provided for performing customized processing or for offloading some of the processing performed by the processing subsystem 2904 in order to speed up the overall processing performed by the computer system 2900 .

The I/O subsystem 2908 may include devices and mechanisms for inputting information into the computer system 2900 and/or for outputting information from or via the computer system 2900. In general, the use of the term "input device" is intended to include all possible types of devices and mechanisms for inputting information into the computer system 2900. User interface input devices may include, for example, a keyboard, a pointing device such as a mouse or trackball, a touchpad or touch screen incorporated into a display, a scroll wheel, a click wheel, a dial, buttons, switches, a keypad, an audio input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices may also include motion sensing and/or gesture recognition devices such as the Microsoft Motion Sensor, the Microsoft 360 game controller, and devices that provide an interface for receiving input using gestures and spoken commands that enable a user to control and interact with the input device. User interface input devices may also include eye gesture recognition devices, such as the Google Blink Detector, which detects eye activity from a user (e.g., a "wink" when taking a picture and/or making a menu selection) and converts the eye gesture into input to an input device (e.g., Google ). Additionally, the user interface input device may include a voice recognition sensing device that enables a user to interact with a voice recognition system (eg, a navigator) through voice commands.

Other examples of user interface input devices include, but are not limited to, three-dimensional (3D) mice, joysticks or indicator rods, game pads and graphic tablets, and audio/video devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye tracking devices. In addition, user interface input devices may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, position emission tomography, medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

User interface output devices may include a display subsystem, indicator lights, or non-visual displays such as audio output devices. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as one utilizing a liquid crystal display (LCD) or plasma display, a projection device, a touch screen, or the like. In general, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 2900 to a user or other computers. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio/video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

The storage subsystem 2918 provides a repository or data storage for storing information used by the computer system 2900. The storage subsystem 2918 provides a tangible, non-transitory, computer-readable storage medium for storing the basic programming and data structures that provide the functionality of some embodiments. The software (programs, code modules, instructions) that provide the above functionality when executed by the processing subsystem 2904 can be stored in the storage subsystem 2918. The software can be executed by one or more processing units of the processing subsystem 2904. The storage subsystem 2918 can also provide a repository for storing data used in accordance with the present disclosure.

The storage subsystem 2918 may include one or more non-transitory memory devices, including volatile and non-volatile memory devices. As shown in FIG29 , the storage subsystem 2918 includes system memory 2910 and computer-readable storage media 2922. The system memory 2910 may include multiple memories, including volatile main random access memory (RAM) for storing instructions and data during program execution and non-volatile read-only memory (ROM) or flash memory in which fixed instructions are stored. In some implementations, the basic input/output system (BIOS), which contains basic routines that help transfer information between elements within the computer system 2900, such as during startup, may typically be stored in ROM. RAM typically contains data and/or program modules currently being operated on and executed by the processing subsystem 2904. In some implementations, the system memory 2910 may include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

29 , system memory 2910 may store application programs 2912, which may include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), etc., program data 2914, and an operating system 2916. By way of example, operating system 2916 may include various versions of Microsoft Apple and/or Linux operating systems, various commercial or UNIX-like operating systems (including, but not limited to, various GNU/Linux operating systems, Google OS, etc.), and/or mobile operating systems such as iOS, iPhone, iPod, iPod OS, 8OS, and iPod OS.

Computer-readable storage media 2922 can store programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by processing subsystem 2904, enable the processor to provide the aforementioned functionality can be stored in storage subsystem 2918. By way of example, computer-readable storage media 2922 can include non-volatile memory, such as a hard drive, a magnetic disk drive, or an optical disk drive such as a CD ROM, DVD, (Blu-ray) disk, or other optical media. Computer-readable storage media 2922 can include, but is not limited to, a drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disk, a digital video tape, and the like. Computer-readable storage media 2922 can also include a solid-state drive (SSD) based on non-volatile memory (such as a flash memory-based SSD, an enterprise flash drive, a solid-state ROM, and the like), a volatile memory-based SSD (such as a solid-state RAM, dynamic RAM, static RAM, DRAM-based SSD, a magnetoresistive RAM (MRAM) SSD), and a hybrid SSD that uses a combination of DRAM-based and flash memory-based SSDs. Computer-readable media 2922 may provide storage of computer-readable instructions, data structures, program modules, and other data for the computer system 2900 .

In certain embodiments, the storage subsystem 2900 may also include a computer-readable storage media reader 2920, which may be further connected to a computer-readable storage medium 2922. Optionally, together and in combination with the system memory 2910, the computer-readable storage medium 2922 may comprehensively represent remote, local, fixed, and/or removable storage devices plus storage media for storing computer-readable information.

In certain embodiments, the computer system 2900 can provide support for executing one or more virtual machines. The computer system 2900 can execute programs such as a hypervisor to facilitate the configuration and management of virtual machines. Each virtual machine can be allocated memory, computing (e.g., processors, cores), I/O, and networking resources. Each virtual machine typically runs its own operating system, which can be the same as or different from the operating systems executed by other virtual machines executed by the computer system 2900. Accordingly, multiple operating systems can potentially be run concurrently by the computer system 2900. Each virtual machine generally operates independently of the other virtual machines.

The communication subsystem 2924 provides an interface to other computer systems and networks. The communication subsystem 2924 serves as an interface for receiving data from and sending data to other systems of the computer system 2900. For example, the communication subsystem 2924 can enable the computer system 2900 to establish a communication channel to one or more client computing devices via the Internet for receiving information from and sending information to the client computing devices.

The communication subsystem 2924 can support both wired and/or wireless communication protocols. For example, in some embodiments, the communication subsystem 2924 may include a radio frequency (RF) transceiver component, a global positioning system (GPS) receiver component, and/or other components for accessing a wireless voice and/or data network (e.g., using cellular telephone technology, advanced data network technology (such as 3G, 4G or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 family of standards), or other mobile communication technologies, or any combination thereof). In some embodiments, the communication subsystem 2924 may provide a wired network connection (e.g., Ethernet) in addition to or in lieu of a wireless interface.

The communication subsystem 2924 can receive and send data in various forms. For example, in some embodiments, the communication subsystem 2924 can receive incoming communications in the form of structured and/or unstructured data feeds 2926, event streams 2928, event updates 2930, etc. For example, the communication subsystem 2924 can be configured to receive (or send) data feeds 2926 in real time from users of social media networks and/or other communication services such as feeds, updates, web feeds such as Rich Site Summary (RSS) feeds, and/or real-time updates from one or more third-party information sources.

In certain embodiments, the communication subsystem 2924 may be configured to receive data in the form of a continuous data stream that may be continuous or unbounded in nature and has no clear end, wherein the continuous data stream may include an event stream 2928 of real-time events and/or event updates 2930. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

The communication subsystem 2924 can also be configured to output structured and/or unstructured data feeds 2926, event streams 2928, event updates 2930, etc. to one or more databases, wherein the one or more databases can communicate with one or more stream data source computers coupled to the computer system 2900.

Computer system 2900 can be one of various types, including a handheld portable device (e.g., a cell phone, a computing tablet, a PDA), a wearable device (e.g., a Google head-mounted display), a personal computer, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system.

Due to the ever-changing nature of computers and networks, the description of the computer system 2900 depicted in FIG29 is intended to be merely a specific example. Many other configurations are possible with more or fewer components than the system depicted in FIG29. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other ways and/or methods of implementing the various embodiments.

Although specific embodiments of the present disclosure have been described, various modifications, changes, alternative constructions, and equivalents are also included within the scope of the present disclosure. Modifications include any relevant combination of the disclosed features. The embodiments of the present disclosure are not limited to operations within certain specific data processing environments, but can be freely operated within multiple data processing environments. In addition, although the embodiments of the present disclosure have been described using a specific series of transactions and steps, it should be apparent to those skilled in the art that the scope of the present disclosure is not limited to the described series of transactions and steps. The various features and aspects of the above-described embodiments can be used alone or in combination.

In addition, although embodiments of the present disclosure have been described using specific combinations of hardware and software, it will be appreciated that other combinations of hardware and software are also within the scope of the present disclosure. Embodiments of the present disclosure may be implemented using only hardware, or only software, or a combination thereof. The various processes described herein may be implemented on the same processor or on different processors in any combination. Accordingly, where a component or module is described as being configured to perform certain operations, such configuration may be implemented, for example, by designing electronic circuits to perform the operations, by programming programmable electronic circuits (such as microprocessors) to perform the operations, or any combination thereof. Processes may communicate using various techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

Accordingly, the specification and drawings should be considered in an illustrative rather than a restrictive sense. However, it will be apparent that additions, subtractions, deletions, and other modifications and changes may be made thereto without departing from the broader spirit and scope set forth in the claims. Therefore, while specific embodiments have been described, these embodiments are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

Claims (23)

1. A computer-implemented method for security management, comprising, at a computer system of a security management system: Obtain data about network activity associated with client devices on the organization's network, wherein the network activity is generated when the client devices operate as part of the organization's network; Data about network activity is used to identify applications that have been accessed by client devices when the client devices are operating as part of an organization’s network, where the applications are provided to the client devices by a service provider’s network, and where the organization’s network and the service provider’s network are different networks; Data about network activity is used to determine access information associated with an application, where the access information includes network activity indicating access to the application from a client device; Access information is used to determine domain information about the application, which includes information about the service provider; Domain information is used to determine security information about the application, where the security information includes one or more indicators describing the security threats associated with the application; Using security information to calculate measurements of application security; and Remedial actions for the application are performed by applying security policies based on the security measures described above. 2. The computer-implemented method of claim 1, wherein the security information includes a first value and a second value, the first value being a first indicator of a first security threat applied, the second value being a second indicator of a second security threat applied, wherein the first indicator is obtained from a first data source, and wherein the second indicator is obtained from a second data source. 3. The computer-implemented method of claim 2, wherein calculating the security measurement comprises: The calculation is based on a first weighted value, which is the first value multiplied by the first weight value. The calculation is based on a second weighted value, which is the second value multiplied by the second weight value. Calculate the first sum based on the sum of the first weighted value and the second weighted value; and Calculate a second sum based on the sum of a first weight value and a second weight value, wherein the security measurement is a value calculated by dividing the first sum by the second sum. 4. The computer-implemented method of claim 3, wherein the first weight value is different from the second weight value, and wherein the first value is different from the second value. 5. The computer-implemented method of any one of claims 1 or 2, wherein obtaining data about network activity comprises obtaining network data from one or more network devices on the organization's network, wherein the organization's network is protected within the organization's computing environment, which is not vulnerable to public networks. 6. The computer-implemented method as described in any one of claims 1 or 2, further comprising: To determine organizational information for the application; and Generate a graphical interface that displays information about the application, wherein the information about the application is displayed based on organizational information and measurements of the security calculated for the application, and wherein the graphical interface indicates remedial actions to be performed for the application. 7. The computer-implemented method of any one of claims 1 or 2, wherein data concerning network activity is used for communication on an organization's network, wherein identifying an application includes processing data to identify communication corresponding to a request for the application, and wherein the communication indicates application information regarding the request for the application, the application information being used to identify the application as being accessed by a client device. 8. The computer-implemented method of claim 7, wherein the communication is used to determine access information, and wherein the access information indicates a timestamp of network activity, an Internet Protocol (IP) address of the system providing the application, a Media Access Control (MAC) address of the device for accessing the application, and user information about the user of the client device. 9. The computer-implemented method of any one of claims 1 or 2, wherein the access information indicates the Internet Protocol (IP) address of the system providing the application, and wherein determining the domain information includes performing a query on the domain information corresponding to the domain hosting the application based on the application's IP address. 10. The computer-implemented method of any one of claims 1 or 2, wherein the access information indicates source information of the application, the source information indicating the location of the application provided by the host, and wherein determining the domain information includes sending a request for a certificate for the application to the host based on the source information of the application. 11. The computer-implemented method of any one of claims 1 or 2, wherein the application security policy includes determining whether a measurement of the security meets a risk threshold for the application, and wherein a remedial action is configuring the organization's network to prevent access to the application from a client device while the client device is operating on the organization's network. 12. The computer-implemented method of any one of claims 1 or 2, wherein the data concerning network activity includes network activity associated with a plurality of users, wherein the plurality of users are tenants on an organization’s network, and wherein the remedial action is to prevent the application from being accessed by the plurality of users. 13. The computer-implemented method of any one of claims 1 or 2, wherein the remedy for the application comprises: Generate a graphical interface; and The graphical interface displays a prompt requesting adjustments to the application's configuration, wherein the adjustments are based on a security policy applied to the security measures. 14. A security management system, comprising: One or more processors; and The one or more processors can access memory, wherein the memory stores one or more instructions, which, when executed by the one or more processors, cause the one or more processors to perform operations including: Obtain data about network activity associated with client devices on the organization's network, wherein the network activity is generated when the client devices operate as part of the organization's network; Data about network activity is used to identify applications that have been accessed by client devices when the client devices are operating as part of an organization’s network, where the applications are provided to the client devices by a service provider’s network, and where the organization’s network and the service provider’s network are different networks; Data about network activity is used to determine access information associated with an application, where the access information includes network activity indicating access to the application from a client device; Use access information to perform one or more queries on domain information associated with the application; Determine security information about the application, including one or more indicators describing the security threats associated with the application; Using domain information and security information to calculate measurements of application security; and Remedial actions for the application are performed by applying security policies based on the security measures described above. 15. The security management system of claim 14, wherein the security information includes a first value and a second value, the first value being a first indicator of a first security threat applied, the second value being a second indicator of a second security threat applied, and wherein the measurement for calculating the security includes: The calculation is based on a first weighted value, which is the first value multiplied by the first weight value. The calculation is based on a second weighted value, which is the second value multiplied by the second weight value. Calculate the first sum based on the sum of the first weighted value and the second weighted value; and Calculate a second sum based on the sum of a first weight value and a second weight value, wherein the security measurement is a value calculated by dividing the first sum by the second sum. 16. A computer-implemented method for security management, comprising, at a computer system of a security management system: First data about a first application is obtained from a first service provider system, wherein the first application is accessed from the first service provider system, and wherein access to the first application is associated with a user account; Second data about a second application is obtained from a second service provider system, wherein the second application is accessed from the second service provider system, and wherein access to the second application is associated with the user account; The first and second data are used to determine the access information of the third application that the user account has accessed; Use access information to search for domain information about the provider system that provides third-party applications; Determine security information regarding third-party applications; Using domain information and security information to calculate measurements of the security of third-party applications; and Remedial actions against third-party applications are performed by applying security policies based on the security measures described above. 17. The computer-implemented method of claim 16, wherein the first service provider system is different from the second service provider system, wherein the first service provider system provides access to the first application as a first cloud service, and wherein the second service provider system provides access to the second application as a second cloud service. 18. The computer-implemented method as described in any one of claims 16 or 17, further comprising: Determine organizational information for third-party applications; and A graphical interface is generated to display information about a third application, wherein the information about the third application is displayed based on organizational information and measurements of the security calculated for the third application, and wherein the graphical interface indicates remedial actions to be performed for the third application. 19. The computer-implemented method of any one of claims 16 or 17, wherein the first data indicates that a user account has accessed the first application through the third application, wherein the second data indicates that a user account has accessed the second application through the third application, and wherein determining the access information includes determining that the third application has been accessed to provide access to the first and second applications. 20. The computer-implemented method of any one of claims 16 or 17, wherein the security information includes a first value, the first value being a first indicator of a first security threat to a third application, and the security information includes a second value, the second value being a second indicator of a second security threat to a third application, wherein the first indicator is obtained from a first source, wherein the first value is different from the second value, wherein the second indicator is obtained from a second source, and wherein the measurement for calculating said security includes: The calculation is based on a first weighted value, which is the first value multiplied by the first weight value. The calculation is based on a second weighted value, which is the second value multiplied by the second weight value, where the first weight value is different from the second weight value; Calculate the first sum based on the sum of the first weighted value and the second weighted value; and Calculate a second sum based on the sum of a first weight value and a second weight value, wherein the security measurement is a value calculated by dividing the first sum by the second sum. 21. A non-transient computer-readable medium comprising instructions stored thereon, which, when executed on a processor, perform the method as claimed in any one of claims 1-13 or 16-20. 22. A security management system, comprising: One or more processors; and The one or more processors have access to a memory that stores one or more instructions that, when executed by the one or more processors, cause the one or more processors to perform the method as described in any one of claims 1-13 or 16-20. 23. A security management system comprising means for performing the method as claimed in any one of claims 1-13 or 16-20.