JP7502386B2 - Hierarchical API for defining multi-segment applications in SDDC - Google Patents

Description

The present invention relates to a method, a machine-readable medium, a computer device and a system.

In enterprise datacenters, firewalls have been the essence of networking and security for applications running on them. It started with Access Control Lists (ACLs), which provide rules that apply to hosts or other Layer 3 available port numbers or IP addresses, each with a list of hosts and/or networks that are allowed to use a service. After ACLs, macro-segmentation came to provide IP-based enforcement for all applications running on a host. This allowed enterprise administrators a granular level of control to secure workloads based on VLANs.

Network virtualization has transformed the world of networking and security by providing the ability to enforce distributed firewall rules across hosts in the datacenter with micro-segmentation based on L4-L7 network services and attributes. There are new firewalls with the ability to perform deep packet introspection at the transport layer and include web application filtering, verb-based firewalls, and URL filtering.

Figure 1 shows the current workflow for specifying firewall controls for micro-segmented applications. As shown, an administrator must first define (at 105) the intent of which applications they want to protect. Based on that intent, the administrator must create (at 110) domains and groups to define the boundaries of each component of the application. Once groups are created, the administrator defines (at 115) how these components can communicate with each other based on communication profiles.

After the profiles and groups are created, the resulting policies are published at 120 to the Software Defined Data Center (SDDC) Network Manager 150. After publishing the policies, the administrator must log into the Network Manager for all instances in the data center that the administrator controls and create (at 125) network and security groups based on the grouping criteria specified in the domain and group creation. The criteria can be based on logical switch ports, tags, or VM/container names.

The administrator must manually manage (130) the workload VMs/containers by applying corresponding tags to match the criteria when creating network and security groups at 125. Once the tags match the criteria, the firewall rules defined in the communication profile (at 135) are applied to each VM.

This approach has several drawbacks. For example, managing grouping criteria (tags, VM names, etc.) is manual and tedious. This is especially problematic because such management must be repeated across multiple environments (e.g., development, staging, and production). Furthermore, application discovery and classification is an administrative overhead that is often error-prone. This approach is also not scalable for dynamic workload (e.g., container) environments, where the entities being protected are ephemeral in nature.

Some embodiments provide a simplified mechanism for deploying and controlling multi-segment applications by using application-based manifests that express how the application segments of a multi-segment application should be defined or modified, and how the communication profiles between these segments should be defined or modified. In some embodiments, these manifests are application-specific. Also, in some embodiments, a deployment manager in a software-defined data center (SDDC) provides these manifests as templates to an administrator, who can use these templates to express his or her intent when deploying a multi-segment application in the data center. Application-based manifests can also be used to control previously deployed multi-segment applications in the SDDC. Using such manifests, an administrator can manage fine-grained micro-segmentation rules based on endpoint and network attributes.

A multi-segment application is an application that includes multiple application segments. In some embodiments, each of the one or more application segments is a standalone application that runs in its own memory space, independent of the memory space of other application segments of the multi-segment application. In some embodiments, different application segments of a multi-segment application are implemented by different machines (e.g., different VMs or containers).

In some embodiments, the application manifest includes a syntactic representation of a multi-segment application, which may be defined after implementing the manifest or may be defined previously. The application manifest in some embodiments is a hierarchical API that includes two or more commands that define or modify (1) one or more application segments and (2) one or more policies associated therewith. The application manifest is a hierarchical API because different commands can be nested under other commands. For example, the definition of a group of applications may include the definition of a specific machine (e.g., a specific VM) for implementing a particular application in the group. In some embodiments, the application manifest is provided to the administrator as a predefined template that encapsulates known applications and their dependencies, as well as the ability to model applications based on the administrator's requirements.

In some embodiments, the manifest is defined in a declarative language. In some embodiments, a manifest processing framework in the SDDC parses the manifest into commands that (1) instruct a compute manager in the SDDC to deploy and configure application segments of a multi-segment application defined in the manifest, and (2) instruct a network manager in the SDDC to define and deploy network forwarding and service rules to implement communication profiles between application segments and between application segments and other applications specified by the manifest.

In some embodiments, application segments are deployed as VMs or containers that run on host computers in the SDDC and/or as standalone computers. Similarly, network forwarding and service rules in some embodiments are processed by software forwarding elements (e.g., software switches and routers) and software middlebox service VMs (e.g., service containers and/or service modules that run on host computers in the SDDC). These forwarding and/or service rules are also configured in some embodiments on hardware forwarding elements in the SDDC (e.g., top-of-rack switches), standalone hardware or software gateways, and/or standalone middlebox appliances.

Some embodiments of the present invention provide a method for deploying a multi-segment application in an SDDC. The method first receives a hierarchical API command that specifies, in a declarative format, multiple operation requests for defining multiple application segments of the multi-segment application. The method parses the API command to identify the application segments. Based on the parsed API command, the method deploys software-defined (SD) resources required to deploy the multiple application segments and transport and service operations between those segments.

In some embodiments, the method uses a deployment process that ensures that any first SD resource on which a second resource depends is deployed before the second resource. In some embodiments, a second SD resource depends on a first SD resource if the second SD resource is a child of the first SD resource. Alternatively, or additionally, in some embodiments, a second SD resource may depend on a first SD resource if the second SD resource has any operational dependency on the first SD resource.

In some embodiments, the method parses the API command by identifying a set of SD resources, each set having one or more SD resources at a resource level. In some embodiments, the deployment deploys the identified SD resource set at a higher resource level before deploying the SD resources at a lower resource level. Examples of SD resources that can be specified in a hierarchical API command include SD compute elements (e.g., VMs or containers) for implementing application segments, SD forwarding elements (e.g., managed software switches and routers, logical switches and routers implemented by the managed software switches and routers, etc.) for implementing forwarding rules for forwarding data messages associated with the application segments, and SD service middlebox modules (e.g., service VMs or modules that perform middlebox service operations such as firewall operations, load balancing operations, network address translation operations, encryption operations, intrusion detection operations, intrusion prevention operations, etc.) for implementing service rules for performing services on data messages associated with the application segments.

In one embodiment, an API processing system processes an API command. The command may include a set of parameters for updating a previously deployed SD resource. In this case, the API processing system deploys the multi-segment application by updating a previously deployed SD resource for the multi-segment application using the set of parameters specified in the parsed API command. In such a case, the API command includes a set of parameters that define a new SD resource. In such a case, the API processing system deploys the SD resource by deploying the SD resource based on the set of parameters specified in the parsed API command.

In some embodiments, the hierarchical API command is processed as one atomic unit. Thus, the API processing system determines whether the SD resources identified in the hierarchical API command are deployable. If so, the API processing system sends a confirmation to the source that generated the hierarchical API command that the API command was successfully processed. Otherwise, if one or more SD resources in the API command are not deployable, the API processing system sends a message to the source that generated the hierarchical API command that the API command was not successfully processed.

Some embodiments pave the way for next-generation micro-segmentation by binding context from data compute endpoints to the network. Endpoint-based context can be related to user identity as well as application-specific attributes such as file hashes, publisher information, licenses, process information, etc. In some embodiments, the context information is used by application-based firewalls that are deployed to distributors located in SDDC (e.g., via virtualized network appliances). One of the biggest challenges in commoditizing application-based firewalls is the complex consumption model of such firewalls, since there may be thousands of processes running within an endpoint and millions of processes running within a data center. However, by using application manifests, some embodiments provide a novel approach that allows administrators to create fine-grained context-based rules and manage the very difficult task of managing these rules.

The foregoing summary is intended to serve as a brief introduction to some embodiments of the present invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The following detailed description and the figures referenced in the detailed description further describe the embodiments described in the summary, as well as other embodiments. Thus, a complete review of the summary, detailed description, drawings, and claims is necessary to understand all embodiments described herein. Also, the claimed subject matter should not be limited by the illustrative details in the summary, detailed description, and drawings.

The novel features of the invention are set forth in the appended claims. However, for purposes of illustration, certain embodiments of the invention are set forth in the following drawings.

FIG. 1 illustrates the current workflow for specifying firewall controls for micro-segmented applications. FIG. 2 illustrates a manifest processing framework for processing application manifests received from a tenant administrator at the SDDC. FIG. 3 illustrates a process illustrating the operation of the components of the manifest processing framework. , , , , 4A-E show an example application manifest that defines a multi-segment application called Slack. FIG. 5 shows an example of Slack. FIG. 6 shows a data model for the application shown in FIG. FIG. 7 shows a process illustrating an example flow for defining and deploying a multi-segment application based on a manifest template. FIG. 8 illustrates the architecture of this new classification engine for some embodiments. FIG. 9 illustrates an example of an API processing system according to some embodiments of the present invention. FIG. 10 illustrates how some embodiments implement context-based firewall rules on a host computer. FIG. 11 shows an example of data collection to learn about the deployment of multi-segment applications in a data center. FIG. 12 conceptually illustrates a computer system upon which some embodiments of the present invention may be implemented.

In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are described and explained. However, it will be apparent and clear to one skilled in the art that the invention is not limited to the described embodiments, and that the invention may be practiced without some of the specific details and examples described.

Some embodiments provide a novel application-based manifest that provides a simplified mechanism for deploying and controlling a multi-segment application and for defining communication profiles between segments of a multi-segment application. A multi-segment application is an application that includes multiple application segments. In some embodiments, each application segment may be a standalone application that runs in its own memory space, separate from the memory space of any other application segments of the multi-segment application. In some embodiments, different application segments of a multi-segment application are implemented by different machines (e.g., different VMs or containers).

In some embodiments, a deployment manager in the software-defined data center (SDDC) provides these manifests as templates to administrators. Administrators can use these templates to indicate their intent in deploying multi-segment applications in the data center. Application-based manifests can also be used to control previously deployed multi-segment applications in the SDDC. Using such manifests, administrators can manage fine-grained micro-segmentation rules based on endpoint and network attributes.

In this document, a data message refers to a collection of bits in a particular format that is sent over a network. It will be understood by those skilled in the art that the term data message may be used herein to refer to a collection of bits in various formats that may be sent over a network, such as an Ethernet frame, an IP packet, a TCP segment, a UDP datagram, etc. Also, as used in this document, references to the L2, L3, L4, and L7 layers (or layers 2, 3, 4, and 7) are references to the data link layer at layer 2, the network layer at layer 3, the transport layer at layer 4, and the application layer at layer 7, respectively, of the Open System Interconnection (OSI) layer model.

FIG. 2 illustrates a manifest processing framework 200 that processes application manifests received from tenant administrators in the SDDC. Based on the processing, the framework 200 interacts with compute and network managers in the SDDC to deploy the multi-segment application, configure transport and service elements in the SDDC, and set desired communication rules between the application's segments and between those segments and other applications and devices inside and outside the SDDC. In some embodiments, the application manifest can also include requests to adjust previously configured communication profiles for previously deployed multi-segment applications and/or previously deployed segments.

As shown, the manifest framework includes a parser 205, a constraint checker 210, a sorter 220, an orchestrator 230, and several rule and policy storages 215, 225, and 235. The operation of these components is described with reference to FIG. 3, which illustrates the process 300 that the components execute for an application manifest. As shown, the process 300 begins when the manifest processing framework 200 receives (at 305) an application manifest from an administrator's machine 260 (e.g., from a VM, container, or standalone computer used by the administrator to specify the application manifest). As mentioned above and further described below, in some embodiments, the administrator can use an application manifest template provided by the manifest framework to specify the manifest 255.

In some embodiments, the application manifest 255 includes a syntactic representation of a multi-segment application. The application manifest of some embodiments is a hierarchical API that includes two or more commands that define or modify (1) one or more application segments and (2) one or more communication profiles between each application segment and another application segment or another application/machine inside or outside the SDDC.

The application manifest is a hierarchical API that allows different commands to be nested under other commands, so that, for example, a domain definition can include an application segment group definition, which in turn can include one or more definitions of one or more machines (e.g., specific VMs or containers) for implementing the application segments. In some embodiments, the manifest is defined in a declarative language. For example, in one embodiment, the manifest is written in Javascript Object Notation (JSON) format, but in other embodiments, the manifest can be expressed in other hierarchical formats, such as Extensible Markup Language (XML) format.

After receiving the application manifest 255, the parser 205 of the framework 200 identifies (at 310) several different requests (commands) contained in the manifest. For example, in a typical three-tier application, the manifest may specify the deployment of a web server, an application server, and a database server. In such a situation, the parser parses the manifest into three sets of one or more commands, each set associated with a deployment of one tier (e.g., a deployment of a web server, an app server, or a database server). In one embodiment, the parser generates an input API tree from the manifest. The input API tree represents the parent-child relationships between the different parts of the manifest.

After the parser splits the manifest into individual requests, the constraint checker 210 determines (at 315) whether any of the individual requests violate the policy constraints stored in its constraint storage 215. If so, the framework returns an error to the administrator. If not, the sorter 220 identifies (at 320) a sorted order for implementing these requests. To identify this sorted order, in some embodiments, the sorter builds a type-specific map that identifies each SD resource identified in the manifest according to its type. To do this, in some embodiments, the sorter performs a breadth-first search of the input API tree built from the manifest by the parser, classifies the inputs into different buckets based on resource type, and stores the classified inputs in the type-specific map.

Each key in the type-specific map is a resource type, and the value of each key is a list of all resources of a particular type in the input API tree. Each node element is stored with its parent. In summary, the input API tree is sorted based on resource type, e.g., all areas in one bucket, all groups in another bucket, etc. After generating the type-specific map, the sorter defines an execution order for persisting SD resources in the input API tree. In one embodiment, the execution order is a predefined ordered list of resource types that specifies the order in which resources in the input tree are persisted. When new types are introduced into the system, the execution order is dynamically updated to include the order of the new elements. For example, a sample execution order in some embodiments may be (1) areas, (2) groups, and (3) communication maps. That is, areas must be created first, then groups, then communication maps.

The sorter then uses the service provider registry to persist the SD resources in the configured API tree. The service provider registry is a map of resource types to callback handlers. A callback handler is registered for each type in the system. The responsibility of the callback handler is to persist the registered types. As described further below, the callback handler is implemented by a deployment plugin in some embodiments. The deployment plugin is a module that connects to the API processing system and handles the persistence of changes requested by received APIs and the deployment of persisted changes.

Once the sorter identifies the invocation order and invokes the callback handlers based on the invocation order to persist the deployment data to one or more configuration databases, the orchestrator 230 interacts (at 325) with one or more network, compute, and/or service managers 240 to deploy SD resources 250 based on the configuration data persisted in the configuration databases. In some embodiments, the orchestrator 230 is implemented by a deployment plugin that also implements callback handlers for persisting data to the configuration databases.

In some embodiments, SDDC resource manager 240 also deploys the multi-segment application and its associated communication profile to SDDC resources 250 using one or more SDDC resource controllers 245. Examples of such resources include host computers, VMs, containers, software and hardware forwarding elements, software and hardware middlebox service elements, etc. Resource managers and controllers 240 and 245 include (1) compute managers and controllers that deploy and configure application segments of the multi-segment application defined in the manifest, and (2) network managers and controllers that define and deploy network forwarding and service rules to implement the communication profiles of the application segments specified in the manifest.

In some embodiments, application segments are deployed in an SDDC as VMs or containers running on host computers and/or as standalone computers. Similarly, network forwarding and service rules in some embodiments are processed by software forwarding elements (e.g., software switches and routers) and software middlebox service VMs, service containers, and/or service modules running on host computers in the SDDC. These forwarding and/or service rules are also configured in hardware forwarding elements (e.g., top-of-rack switches), standalone hardware or software gateways, and/or standalone middlebox appliances in the SDDC in some embodiments.

Figure 4 shows an example application manifest that defines a multi-segment application called Slack. The application manifest in this example is a hierarchical API template 400 that can be used to define the actual application manifest to send to the manifest processing framework. Such a template provides a mechanism to specify a common set of APIs that are often called in sequence to deploy a multi-segment application. By utilizing template APIs, an administrator can deploy a common set of requirements without having to define a set of APIs from scratch.

To specify the actual multi-segment application manifest from the manifest template 400, in some embodiments, the administrator need only modify a limited number of fields, called placeholder fields, in the copy of the template that will become the manifest. From this perspective, the template API is a set of one or more requests (for one or more resources) with blank fields or placeholders. Placeholders are application name, application identifier (App_ID), process name, process hash, user identifier, or other examples of key-value pairs used in a multi-segment application template. In some embodiments, the administrator can also modify other components of the copy of the template that will become the actual manifest (e.g., adding segments, adding or removing communication rules, changing communication rule parameters, etc.).

In some embodiments, the API template is a managed resource. This is represented by a list of placeholders and a body that is an API object. The API template 400 in FIG. 4 is for deploying a Slack application. FIG. 5 shows an example deployment of a Slack application 500. This application has a data model diagrammed in FIG. 6. Both the application 500 and the data model 600 are further described below to further explain portions of the application manifest 400.

The application manifest 400 is a hierarchical JSON format that is equivalent to a tree format. Each node in the tree corresponds to an SDDC resource and has a field that describes the resource type of the node. Each node has a special property that holds all the children of the node, indicating the parent-child relationships. A child node can in turn have multiple children, and this can go to any depth. Thus, each node can be both a parent and a child at the same time (as can non-leaf nodes in the tree).

In FIG. 4, each node has a property "resource_type" that describes the type of the node. Examples of types in some embodiments include Infra, Tenant, Domain, Group, CommunicationMap, CommunicationEntry, etc. These are all different types of resources in a data center. A node can also have a property "children" that holds all the children of the node. For example, in FIG. 4, a node of type Domain 410 is a child of type Infra 405 and has two different types of children 415-450, which are "Group" and "CommunicationMap". In some embodiments, Tenant refers to a tenant in a multi-tenant SDDC, Domain is a workload under the tenant, CommunicationMap is a security policy, and CommunicationEntry is a rule under the security policy.

In some embodiments, each SD resource may be identified by a unique path from the root that includes all taxonomic parents. For example, /VMware specifies all resources associated with tenant VMware. The path /VMware/domains/Outlook specifies all Outlook workloads for tenant VMware. The path /VMware/domains/Outlook/communication-maps/web-profile specifies the web profile for Outlook workloads for tenant VMware. The path /vmware/domains/Outlook/communicationmaps/web-profile/communication-entries/open-browser-access specifies the open browser access for Outlook workloads for tenant VMware. More generally, the format of a security policy path may be specified as /<tenant-name>/domains/<workload-name>/communication-maps/<security-policyname>/communication-entries/<rule-name>.

As shown in FIG. 4, this manifest template 400 includes a template header 401 that provides a name 402 and a description 403 of the template along with a placeholder list 404. The manifest template 400 also includes eleven requirements 405-480. Request 405 is to create a configuration called Infra. This configuration has eight children defined by requirements 405-450. Request 410 defines an area called slack_app.

Requests 415-445 define seven application segments for the slack_app multi-segment application. These seven segments are illustrated in FIG. 5. As illustrated, these seven segments include slack share 515 (defined by request 415), slack base 520 (defined by request 420), slack call 525 (defined by request 425), slack edit 530 (defined by request 430), slack download 535 (defined by request 435), slack upload 540 (defined by request 440), and slack file transfer 545 (defined by request 445). Each request 415-445 for each segment specifies that the segment is implemented by a VM tagged with a key set equal to the name (identifier) of the segment.

5 and 6 show that communication between these segments passes through slack base 520. Thus, request 450 defines a communication profile (i.e., a communication map) between these segments in terms of six communication rules (i.e., communication entries) for data message exchange between slack base 520 and each of the other segments 515 and 525-545. In some embodiments, these six communication rules are implemented by firewall rules in the data plane. FIG. 5 also shows that in some embodiments, communication rules (e.g., firewall rules) can be based on any number of different context attributes, such as app ID, process name, etc.

As shown, each communication entry is represented by the source of the data message (e.g., a source group), the destination of the data message (e.g., a destination group), an action that specifies whether the communication is allowed or denied, and a set of services, ports, and/or protocols used by the data message. The communication map also defines rules 480 (i.e., communication entries) that specify how the slack base communicates with the Active Directory Service 590 shown in FIG. 5. In some embodiments, these communication entries 455-480 are translated by the manifest processing framework into firewall rules to control communication between different segments of Slack and between these segments and other machines (inside or outside the SDDC).

In some embodiments, templates can be managed through the GET, PATCH, DELETE, and POST commands. For example, in some embodiments, GET /policy/templates returns a list of template identifiers in the database. In some embodiments, GET /policy/templates/<template-id> returns the templates for a specific template identifier. Also, in some embodiments, PATCH /policy/templates following a template JSON definition creates a new template. In some embodiments, DELETE /policy/templates/<template-id> deletes the template given a specific template identifier.

In some embodiments, POST /policy/templates/<template-id>?action=deploy is defined to define and launch a hierarchical API based on a template API. This command deploys a template given a specific template identifier <template-id>. Arguments providing values for placeholders in the template are passed in the body of the POST request. In response to a POST command with placeholder arguments, in some embodiments, a template manager of the manifest processing framework fetches the identified template, applies arguments representing the placeholder values to define the hierarchical API, and creates one or more request objects to identify each requested operation in the hierarchical API. Such template managers are described further below.

In essence, the manifest template 400 specifies a set of processes that define a multi-segment application and a set of recommended communication profiles. These definitions can be modified by an administrator based on the administrator's requirements; that is, an administrator has the option to use this verbatim in their environment or make modifications as deemed necessary in their data center. This sample application manifest for Slack can be published as an industry-wide standard, making deployment and micro-segmentation of this application easier.

FIG. 7 illustrates a process 700 depicting an example flow for defining and deploying a multi-segment application based on a manifest template. As shown, process 700 begins by providing (at 705) a list of application manifest templates. The manifest templates in some embodiments are templates for the most commonly used multi-segment applications. In some embodiments, the manifest templates are open source and community driven, allowing for greater visibility and accuracy of these applications.

Next, at 710, the administrator selects a manifest template from the list of provided manifest templates. The selected template is for the multi-segment application the administrator wants to deploy. At 715, the administrator reviews the manifest based on his intent. The selected manifest indicates the compute, network, and security intents using the default configuration defined by the template publisher. The administrator can accept the default settings or modify them to match the desired intent.

At 720, the administrator submits the manifest to a manifest processing framework for processing and deployment. Once the manifest is published to the framework, the framework deploys (at 725) the compute, network, and/or service resources specified in the manifest. After 725, process 700 ends.

In some embodiments, the framework uses a new classification engine, a middlebox service that collects different types of context attributes (such as those used in existing or new workloads) from the deployment environment and makes these attributes available to consumers when defining communication entries in the manifest. Figure 8 illustrates the architecture of this new classification engine for some embodiments.

Applications running on VMs are not transient in nature. Once an administrator installs an application, these applications are typically long running. This typically applies to VMs and bare metal servers. In some embodiments, a context engine running on a hypervisor running on a host computer periodically sends a list of processes running on guest VMs to the SDDC management plane. The application discovery engine 805 receives this information and provides visualization of the application information and associated virtual machines.

In some embodiments, this information is not polled continuously and the process is not automated. Thus, in some embodiments, a new classification vertical on the management plane internally performs a periodic synchronization operation to discover the list of running/installed processes on the VMs and tag them based on the process information. The inventory vertical 810 gets the list of VMs in the system and creates security groups internally and enables application discovery for these VMs. Once the list of applications/processes running on the VMs is identified, the VMs are tagged with processes. The grouping/tagging manager 815 collects security groups and other tagging data for the classification engine 800, while the firewall manager 820 collects contextual data from and for the deployed firewalls. Based on the collected tags, the intent from the policies created at the application level determines the security groups and communication profiles of these processes.

9 illustrates an example of an API processing system 900 of some embodiments of the present invention. The API processing system 900 implements the manifest processing framework 200 of some embodiments. In this system, each tenant can create an SDDC cluster 902 that includes one or more SDDC instances 905, which can be considered separate environments. As shown, in some embodiments, each SDDC instance 905 includes an API gateway 920, an API processor 925, a compute manager 910, a network manager 915, a controller 940, a template manager 927, a policy checker 923, a configuration data storage 935, and several deployment plugins 930.

In some embodiments, two or more of these components run on two or more machines (e.g., VMs, containers, standalone servers, etc.) in one or more data centers and communicate with each other over a network. In these or other embodiments, each SDDC instance includes multiple instances of each of these components to distribute load and provide high availability.

The compute manager 910 deploys and manages workload machines (e.g., workload VMs or containers), while the network manager 915 deploys network resources (e.g., software switches and routers) and middlebox service resources (e.g., service VMs and modules) in the data center. In some embodiments, the compute and network managers 910 and 915 use one or more controllers 940 to distribute configuration data stored in one or more configuration data storages 935 to host computers, forwarding elements (e.g., software switches and routers running on host computers or standalone switches and routers), service machines (e.g., service VMs, service containers, other service modules, and standalone service appliances), and other resources in the SDDC.

The API gateway 920 redirects all API commands based on the URL pattern to the API service module 925 or possibly to the UI manager 922. The UI manager 922 processes the API commands received through the graphical user interface and directs these commands to the API processor 925. The API processor 925 executes the process shown in Figures 2 and 3 to ensure that the different requests that are part of the received application manifest are persisted in the configuration data storage 935 and deployed in the correct order. The API processor 925 owns the user's desired state that it stores in the data storage 932. In some embodiments, the API processor 925 runs as a VM or container.

As shown, in some embodiments, the API processor 925 uses a template manager 927 that has access to several manifest templates 929 that specify the multi-segment application configuration of SDDC resources. Through the template manager 927, a user can select and modify templates (e.g., via API commands) to generate a completed manifest. Based on this completed manifest, the API processor 925 can then deploy or update a set of previously deployed SDDC resources to deploy or adjust a previously deployed multi-segment application.

To deploy resources or update previously deployed resources based on the requests in a received manifest or a completed manifest by reading a manifest template with the required inputs, in some embodiments, the API processor 925 parses the manifest into one or more requests and validates each request using the policy check engine 923 (i.e., each request is stored in the policy storage 924 and specifies whether it satisfies the constraints specified in the policy applicable to the resource referenced in the request).

In some embodiments, each policy in policy storage 924 includes (1) a target that specifies a set of one or more data center resources to which the policy applies, and (2) an expression that specifies an operational constraint on the specified set of resources. In some embodiments, the policies are expressed in a declarative format. Thus, for each request in the manifest, the policy engine compares a set of attributes of the selected request's resources with the policy's target to determine whether the policy is applicable to the resource. After identifying an applicable policy, the policy checking engine determines whether the identified policy's expression specifies a constraint that requires the selected request to be denied or permitted.

Through the deployment plugins 930, the API processor 925 persists the SD resource data in the API calls to the configuration database 935. In some embodiments, the deployment plugins 930 run as VMs or containers. Each plugin 930 has the execution capability to deploy one or more SD resource types. Examples of such types include data compute nodes (e.g., compute machines such as VMs or containers), distributed firewall rules, edge firewall rules, L2 and L3 forwarding elements (software switch standby routers), security groups, VPN services, DHCP services, DNS services, load balancing services, etc.

To deploy these services, plugin 930 interacts with compute manager 910 and network manager 915, which in turn interact with one or more controllers 940. Through these managers and controllers, plugin 930 distributes configuration data from persistent database 935 to host computers and standalone network/service devices in the SDDC to instruct these computers and devices to deploy the desired SD resources.

In some embodiments, there is one desired state and orchestration service (i.e., API processing module) per SDDC instance. This is a highly available service that is deployed in the form of a container or VM in some embodiments. This service accepts the user intent and performs the orchestration between different services. This service also owns the details of the enforcement points (compute and network managers) to which policies need to be pushed down.

Deployment plugins 930 provide the fulfillment of intents. As mentioned above, in some embodiments, each of these plugins is deployed as a separate service running in a separate container or VM. In some embodiments, some services are packaged together in a single container, but run as separate services from an architecture and communication standpoint. Since orchestration is performed by a desired state service, each plugin service in some embodiments exposes a set of REST API endpoints that may be invoked. Also, in some embodiments, the desired state service acts as a common service that returns the state of realized resources across different services. This is true even though in some embodiments the realized state data is updated in a data store by the plugin service.

Thus, the execution of the manifest produces the desired state in one fell swoop. If the system can validate and persist the entire intent, a notification is sent to the source of the manifest (e.g., http status code 200 OK is returned). When an intent is created, notifications are generated. These notifications are consumed asynchronously by the deployment plugin, which then handles fulfilling that intent. The fulfillment status can be queried from the system using the status API.

In some embodiments, the API processing system 900 provides users with the ability to query intents hierarchically. For example, in some embodiments, the system provides a GET API that facilitates reading an entire intent in one go. A dedicated flag is passed in the URL parameter to request a hierarchical GET. If no parameter is passed, the GET in some embodiments functions as a normal GET and a single resource is returned. In some embodiments, a hierarchical GET can work on the entire tree or a portion of the tree, i.e., a hierarchical GET can work from any level in the tree, so the node from which the hierarchy is retrieved can be specified.

Another aspect of hierarchical GET is filtering in some embodiments. In these embodiments, an administrator can filter the intent tree to see only the types that they are interested in. Such filtering can be simple type-based filtering, for example, an administrator can say GET the intent hierarchy for the "Domain" type. In advanced filtering mechanisms, the user can choose to get intent based on function, for example, an administrator can say GET all resources in the intent hierarchy related to the firewall function.

In some embodiments, a user can perform a hierarchical GET and club it with a hierarchical POST. In some embodiments, an administrator can get the intent hierarchy, modify it, and POST it. This enables "import/export" use cases. In some embodiments, an administrator can also get the manifest and store it. The administrator can then restore the previously obtained intents.

Figure 10 illustrates how some embodiments implement context-based firewall rules on a host computer. In some embodiments, the SDDC achieves its desired segmentation communication profile by defining and implementing such context-based rules on the host computer. As described above, the manifest processing framework translates the communication profile rules defined in the manifest and converts them into rules that the network management layer (i.e., the network manager cluster) can process to achieve the micro-segmentation required for the application.

Figure 10 shows the network management cluster receiving a set of rules that use the grouping provider to map these application segments defined in the manifest to security groups based on process name/hash. The network management cluster also receives translated firewall rules that match these process-based groups based on the communication profiles defined in the manifest. Once the process-based groups and firewall rules are created, the firewall manager 1010 of the manager cluster 1005 distributes the rules to host computers and other enforcement nodes in the SDDC.

To map file, module, and other process data to network events, some embodiments use context data captured through a Guest Introspection (GI) agent 1020 running on the machine 1015 (e.g., host VM and container). In some embodiments, the GI agent 1020 captures any guest network connections and file accesses and their associated process context. In some embodiments, all network connections and file accesses are intercepted by this agent and sent to a Context Services Engine 1030 running on the host. In the ESX hypervisor provided by VMware, the context data is sent through a hypervisor component called Mux 1025 to the context engine 1030, which acts as a conduit for sending network and file events from the GI agent to the context engine running on the host computer. Using this information, the context engine updates a context table 1035 that contains information about the user, source IP address, source port, protocol, and process information started by the guest VM.

Subsequently, when the data compute machine 1015 makes a network connection, the firewall module 1040 executing on the host computer enforces the effective firewall rules at the process level by inspecting the outgoing packets, associating context information with the packet flow from the context table, and matching it with the rules in the firewall table. This level of fine-grained control of the processes running on the VMs helps administrators detect unauthorized software/applications installed in the environment and also block malicious processes running in the system. The implementation of the effective rules is sent back to the network management cluster, providing the administrator with the status of deployed applications and effective rules. Context-based firewalls and other middlebox service engines are further described in U.S. Patent Application Serial No. 15/650,251, published as 2018-0181423, which is incorporated herein by reference.

In some embodiments, the manifest processing framework has an intent-based learning engine. Using a GI agent installed on the guest machine (e.g., guest VM or container), the manifest processing framework can identify not only the processes involved in the detected network activity, but also the installed binaries and file attributes associated with these activities. The most common attributes include file hashes, publisher information, installation paths, certificates, etc. In some embodiments, these identified processes and other attributes are collected from the deployment environment (e.g., from the host computer) and analyzed by the learning engine to generate a new multi-segment application template. The collected and analyzed data indicates how some administrators typically deploy multi-segment applications (e.g., new multi-segment applications). In some embodiments, the learning engine determines common deployment attributes from the collected data and recommends a new multi-segment manifest template based on the analysis.

As mentioned above, network and file events captured by the GI agent are stored in a context table. From here, the manifest processing framework can collect data about these captured events, as shown in FIG. 11. In some embodiments, this data is collected on a separate server cluster or set of appliances, as it can become very large depending on the collection time and application workload. This information is stored at the host level for all processes found on each VM within the host.

In some embodiments, the collected network and file event data is then aggregated and modeled to generate customized application templates for proper segmentation at the process level within the VM. For each network event, the GI agent can retrieve the corresponding process, socket, connection rate, and type of connection. The framework in some embodiments has communication channels established to query the libraries used by the process and the files accessed by the process in that instance.

This data, along with round trip times, time of day, and correlations with other processes, in some embodiments is used to train unsupervised or semi-supervised machine learning models. Some embodiments use a one-class support vector machine (SVM) classifier to identify process information. The SVM module is useful in scenarios where there is a lot of "normal" data and few cases where anomalies need to be detected.

One-class SVM is an unsupervised algorithm that learns a decision function for novelty detection, classifying new data as similar or different from the training set. SVM is a supervised learning model that can be used for classification tasks. Typically, the SVM is given labeled data that can be mapped to an n-d space. Different categories are divided by clear gaps, and the SVM finds the broadest possible category. New data samples are classified as belonging to one category based on which side of the gap they fall on. Some embodiments classify or process the information based on the new data seen and recommend a set of appropriate application templates that the administrator needs to protect the network. Using this model, some embodiments can predict running applications and their corresponding processes. In this way, the model helps identify new applications and recommends appropriate application templates to protect them.

The above-described embodiments provide several advantages. They provide an intent-based API processing system for deploying multi-segment applications through a hierarchical API data model, allowing users to specify their intents (e.g., multiple application segments and communication profiles between them) without worrying about how these resources are persisted or realized. In some embodiments, the intent-based API system allows users to define hierarchical API commands by using a simple declarative language that references a simplified hierarchical data model. Each hierarchical API command can define multiple SD resources at multiple resource levels within an SDDC, without requiring that a particular SD resource be created before another in a previous API command. Indeed, in some embodiments, one hierarchical command can be used to define all SD resources for one user (e.g., one tenant) of an SDDC (e.g., a multi-tenant SDDC).

The manifest processing framework of some embodiments leverages the hierarchy of the data model to provide a process for accepting, validating, and fulfilling part or all of the hierarchy in a single API call. The system leverages inherent knowledge of the data model to identify dependencies and invoke the underlying services in the correct order to both persist and fulfill the intent. Additionally, all persistence is done in a single transaction, allowing the entire intent to be accepted as an atomic unit.

When the manifest processing framework determines that a multi-segment application defined in a received manifest is deployable, in some embodiments, the API system uses an asynchronous process to automatically deploy the resources in the appropriate order without further input from an administrator. This process coordinates with one or more network, service, or compute managers to deploy or update one or more network, service, or compute resources based on work orders appropriate for the deployed resources.

Many of the functions and applications described above are implemented as software processes specified as a series of instructions recorded on a computer-readable recording medium (also called a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the operations indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media does not include carrier waves and electrical signals that pass over wireless or wired connections.

As used herein, the term "software" is meant to include firmware resident in read-only memory or applications stored on magnetic storage that can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions may be implemented as subparts of a larger program while remaining separate software inventions. In some embodiments, multiple software inventions may also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described herein is within the scope of the invention. In some embodiments, a software program, when installed to operate on one or more electronic systems, defines one or more specific machine implementations that execute and perform the operations of the software program.

Figure 12 conceptually illustrates a computer system 1200 on which some embodiments of the present invention may be implemented. Computer system 1200 may be used to implement any of the hosts, controllers, and managers described above. As such, it may be used to execute any of the processes described above. The computer system includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. Computer system 1200 includes a bus 1205, a processing unit 1210, a system memory 1225, a read-only memory 1230, a persistent storage device 1235, input devices 1240, and output devices 1245.

Bus 1205 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of computer system 1200. For example, bus 1205 communicatively connects processing unit 1210 to read-only memory 1230, system memory 1225, and persistent storage device 1235.

From these various memory units, the processing unit 1210 obtains instructions to execute and data to process in order to execute the processes of the present invention. The processing unit may be a single processor or a multi-core processor in various embodiments. The read-only memory (ROM) 1230 stores static data and instructions required by the processing unit 1210 and other modules of the computer system. The permanent storage unit 1235, on the other hand, is a readable and writable storage device. It is a non-volatile memory unit that stores instructions and data even when the computer system 1200 is off. Some embodiments of the present invention use mass storage devices (such as magnetic or optical disks and their corresponding disk drives) as the permanent storage unit 1235.

Other embodiments use removable storage devices (e.g., floppy disks, flash drives, etc.) as persistent storage devices. Like persistent storage device 1235, system memory 1225 is a readable and writable storage device. However, unlike storage device 1235, system memory is a volatile read/write memory, such as random access memory. System memory stores some of the instructions and data that the processor needs during execution. In some embodiments, the processes of the present invention are stored in system memory 1225, persistent storage device 1235, and/or read only memory 1230. From these various memory units, processing unit 1210 obtains instructions to execute and data to process in order to execute the processes of some embodiments.

Bus 1205 also connects to input devices 1240 and output devices 1245. The input devices allow a user to communicate information and select commands to the computer system. The input devices 1240 include alphanumeric keyboards and pointing devices (also called "cursor control devices"). The output devices 1245 display images generated by the computer system. Output devices include printers and display devices such as cathode ray tube (CRT) displays or liquid crystal displays (LCDs). Some embodiments include devices such as touch screens that function as both input and output devices.

12, bus 1205 also couples computer system 1200 to a network 1265 via a network adapter (not shown). In this manner, the computer may be part of a network of computers (such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet) or a network of networks (such as the Internet). Any or all of the components of computer system 1200 may be used in conjunction with the present invention.

Some embodiments include electronic components, such as a microprocessor, storage, and memory, that store computer program instructions on a machine-readable or computer-readable medium (alternatively referred to as a computer-readable recording medium, a machine-readable medium, or a machine-readable recording medium). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-Rs), rewritable compact discs (CD-RWs), read-only digital versatile discs (e.g., DVD-ROMs, dual-layer DVD-ROMs), various recordable/rewritable DVDs (e.g., DVD-RAMs, DVD-RWs, DVD+RWs, etc.), flash memory (e.g., SD cards, miniSD cards, microSD cards, etc.), magnetic and/or solid-state hard drives, read-only and recordable Blu-ray discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program executable by at least one processing unit and including a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as produced by a compiler, and files containing high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

Although the above discussion has primarily referred to microprocessors or multi-core processors executing software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself.

As used herein, the terms "computer," "server," "processor," and "memory" all refer to electronic or other technological devices. These terms exclude humans or groups of humans. For purposes of this specification, the terms "display" or "displaying" mean displaying on an electronic device. As used herein, the terms "computer-readable medium," "computer-readable medium," and "machine-readable medium" are entirely limited to tangible, physical objects that store information in a form that can be read by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals.

Although the present invention has been described with reference to numerous specific details, those skilled in the art will appreciate that the present invention may be embodied in other specific forms without departing from the spirit of the invention. Accordingly, those skilled in the art will appreciate that the present invention should not be limited by the illustrative details set forth above, but should instead be defined by the scope of the appended claims.

Claims (23)

1. A method for defining a multi-segment application in a software-defined data center (SDDC), comprising:
creating hierarchical application programming interface (API) commands that specify a plurality of segments of the application to be implemented within the SDDC and define a plurality of rules for controlling the transfer of data messages between the segments of the application, the hierarchical API commands having a plurality of requests that, when processed, deploy the application within the SDDC and deploy the plurality of rules for controlling the transfer of data messages between the segments of the application;
storing the hierarchical API commands as specific templates in a plurality of customizable templates used to define a plurality of multi-segment applications;
A method comprising: providing a graphical user interface or API gateway that allows the particular template to be obtained and customized to define a manifest that defines a set of segments of the multi-segment application and a set of rules for specifying communication between the segments of the multi-segment application. 2. The method of claim 1, wherein when the manifest is subsequently processed by a computer, (i) a plurality of machines on a host computer in the SDDC are deployed to implement the set of segments of the multi-segment application, and (ii) the set of rules is deployed to a set of network elements in the SDDC to control the forwarding of the data messages to and from the deployed plurality of machines that implement the segments of the multi-segment application. 3. The method of claim 2, wherein the network elements include managed forwarding elements for forwarding data messages between the application segments and between the application segments and applications other than the multi-segment application based on the deployed rules. 3. The method of claim 2 , wherein the network element includes a middlebox service element for performing a middlebox service operation based on the deployed rules on data messages sent to or from the application segment. The method of claim 4, wherein the middlebox service component includes a middlebox service machine executing on a host computer. The method of claim 4, wherein the middlebox service element includes a middlebox service engine executing on a host computer. The method of claim 4, wherein the service operation includes one of a firewall operation, a load balancing operation, a network address translation operation, an encryption operation, an intrusion detection operation, and an intrusion prevention operation. 5. The method of claim 4 , wherein the middlebox service element comprises a firewall machine or device. The method of claim 1, wherein the multi-segment application has more than three application segments defined in the hierarchical API command. The method of claim 1, wherein the multi-segment application has more than five application segments defined in the hierarchical API command. 3. The method of claim 2 , wherein the plurality of deployed machines comprises virtual machines or containers. 1. A method for deploying resources in a software-defined data center (SDDC), comprising:
receiving hierarchical API commands that specify a plurality of sets of resources for deployment within the SDDC and define a plurality of sets of rules that control data message flow associated with the resources ;
Parsing the hierarchical API command into a plurality of requests specifying a plurality of sets of the resources and a plurality of sets of the rules;
defining a sort order for the plurality of requests;
providing the plurality of requests to a set of servers based on the sort order ;
Deploy the sets of resources within an SDDC based on the requests provided to the set of servers, and provide the sets of rules to a set of network elements within the SDDC to control the data message flows associated with the resources.
A method comprising: The method of claim 12, wherein the resource includes an application segment of a multi-segment application. The method of claim 13, wherein the resources further include managed forwarding elements for forwarding data messages between the application segments. The method of claim 13, wherein the resources further include a middlebox service element for performing middlebox service operations on data messages sent to or from the application segment. The method of claim 15, wherein the service operations include at least one of a firewall operation, a load balancing operation, a network address translation operation, an encryption operation, an intrusion detection operation, and an intrusion prevention operation. The method of claim 13, wherein the multi-segment application has at least three application segments defined in the hierarchical API command. The method of claim 13, wherein the multi-segment application has more than five application segments defined in the hierarchical API command. 13. The method of claim 12, wherein the multiple sets of deployed resources include at least one virtual machine or container. The API command includes a set of parameters to update a previously deployed machine;
13. The method of claim 12, wherein the deploying of the plurality of resources comprises updating the previously deployed machine based on a set of parameters specified in the parsed API command. The API command includes a set of parameters that update a previously deployed set of rules;
13. The method of claim 12, wherein deploying the plurality of resources includes updating the set of previously provided rules based on a set of parameters specified in the parsed API command. A computer program which, when executed by at least one processing unit, implements the method of any one of claims 1 to 21. A set of processing units;
A machine-readable medium storing a program which, when implemented by at least one of said processing units, implements the method of any one of claims 1 to 21;
1. A computing device comprising:

Images