HK40104187B - System configuration freeze and change management of services deployed via continuous delivery on datacenters configured in cloud platforms - Google Patents

Description

System configuration freeze and change management for services deployed via continuous delivery in data centers configured in a cloud platform

Background Technology

Technical Field

The publicly available text generally addresses the management of software releases in cloud computing platforms, the enforcement of system configuration freezes for services via continuous delivery on data centers configured in cloud computing platforms, and the management of changes to software artifacts deployed on data centers configured in cloud computing platforms.

Related technical descriptions

Organizations are increasingly turning to cloud platforms (or cloud computing platforms), such as AWS (Amazon Web Services), Google Cloud Platform, and Microsoft Azure, to meet their infrastructure needs. Cloud platforms provide organizations with servers, storage, databases, networks, software, and more via the internet. Organizations are migrating their services to cloud platforms to provide the scalability and elasticity of computing resources.

Organizations use continuous delivery platforms (CDPs) that can manage and deploy applications on cloud platforms to maintain cloud infrastructure. These CDPs allow organizations to streamline software deployment processes and manage applications, firewalls, clusters, servers, load balancers, and other computing infrastructure on the cloud platform. CDPs make it easier for developers to make changes to software artifacts that affect services running on the system and to deploy updated software artifacts. However, sometimes it is undesirable to make any changes to the services running on the system. Organizations providing the system may tend to impose system freezes (also known as feature freezes or compute interruptions) for such periods. For example, if the system expects particularly high traffic. Modifying software artifacts associated with services can reduce system stability because unforeseen circumstances such as software defects or bugs may occur as a result of the changes.

Large systems, such as multi-tenant systems, can manage services from numerous organizations representing tenants within the multi-tenant system and can interact with multiple cloud platforms. A multi-tenant system may need to maintain thousands of tenants across multiple cloud platforms. Furthermore, each cloud platform may support different software, languages, and features. Therefore, tracking changes made to services deployed and configured on data centers within cloud platforms is a tedious and error-prone process. Consequently, even if there is a computational outage affecting a service deployed in an online system, developers are likely to modify the software artifacts associated with that service, thus reducing system stability.

Furthermore, several users and user teams may be involved in software releases associated with the service. Therefore, tracking changes made to services deployed and configured on cloud platforms in data centers becomes complex because frequent changes to source code and system configurations are possible. Tracking changes to services deployed on large systems is cumbersome because relevant information is either difficult to access or unavailable. This leads to high maintenance costs for multi-tenant systems that support and track software releases and changes to software releases configured on cloud platforms in data centers.

Attached Figure Description

Figure 1 is a block diagram illustrating the system environment of a multi-tenant system configured in a data center on a cloud platform according to the implementation scheme.

Figure 2A is a block diagram illustrating the system architecture of deployment module 210 according to the implementation scheme.

Figure 2B illustrates the overall process for deploying software artifacts in a data center according to the implementation plan.

Figure 3 is a block diagram illustrating the architecture of a software release management module according to one implementation scheme.

Figure 4 illustrates an example of a data center declarative specification according to one implementation plan.

Figure 5 illustrates an exemplary data center created on a cloud platform based on a declarative specification according to one implementation scheme.

Figure 6 is a block diagram illustrating the generation of a data center on a cloud platform based on a declarative specification according to one implementation scheme.

Figure 7 illustrates the overall process, according to the implementation scheme, for generating a pipeline for configuring software artifacts for deployment on a data center on a cloud platform.

Figure 8 illustrates an exemplary mainstream waterline according to an implementation scheme.

Figure 9 illustrates the overall process of phased execution of the mainstream waterline environment on the cloud platform according to the implementation plan.

Figure 10 illustrates an exemplary mainstream waterline according to an implementation scheme.

Figure 11 shows the system architecture of the system configuration freeze module according to the implementation scheme.

Figure 12 illustrates an exemplary pipeline for forcibly enforcing system configuration freeze according to an implementation scheme.

Figure 13 illustrates the process for changing the system configuration of services deployed in a data center on a cloud platform, according to the implementation scheme.

Figure 14 illustrates the overall process for freezing the system configuration of a data center entity configured on a cloud platform, according to the implementation scheme.

Figure 15 shows the system architecture of the change processing module according to the implementation plan.

Figure 16 illustrates an exemplary mainstream waterline for managing changes according to an implementation scheme.

Figure 17 illustrates the overall process for change management of services deployed on a cloud platform in a data center, according to the implementation plan.

Figure 18 illustrates the process executed during the change management phase of the main waterline according to the implementation plan.

Figure 19 illustrates the process for managing a queue for collecting and changing event information according to the implementation scheme.

Figure 20 is a block diagram showing a functional view of a typical computer system used in the environment of Figure 1 according to one embodiment.

These figures depict various embodiments for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein can be employed without departing from the principles of the embodiments described herein.

Figures use similar reference numerals to identify similar elements. A letter following the reference numeral, such as "115a," indicates that the text specifically refers to the element with that particular reference numeral. Reference numerals without a following letter, such as "115," refer to any or all elements in the figure that carry that reference numeral.

Detailed Implementation

Cloud platforms provide computing resources, such as storage, computing resources, and applications, to computing systems on demand via public networks like the internet. Cloud platforms allow businesses to minimize the upfront costs of building computing infrastructure and enable them to launch and run applications faster with less maintenance overhead. They also allow businesses to adjust computing resources to adapt to rapidly fluctuating and unpredictable demands. Businesses can use cloud platforms to create data centers for their users. However, implementing a data center on each cloud platform requires expertise in the cloud platform technology.

The implementation scheme uses a cloud platform infrastructure language (LPIL) to create data centers within a cloud platform. The system receives a cloud platform-independent declarative specification for the data center. This declarative specification describes the structure of the data center and may not provide instructions on how to create it. The cloud platform-independent declarative specification is configured to generate data centers on any cloud platform across multiple cloud platforms and is specified using the LPIL. The system receives information identifying the target cloud platform used to create the data center and compiles the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. The system sends the cloud platform-specific data center representation and a set of instructions for execution on the target cloud platform. The target cloud platform uses the platform-specific data center representation to execute instructions for configuring the data center. The system provides users with access to the data center computing resources configured on the cloud platform.

The system performs operations related to software releases configured on data centers on a cloud platform, such as deploying software releases, providing resources, and performing software release rollbacks. The system accesses data centers configured on the target cloud platform. Data centers are generated based on a cloud platform-independent declarative specification that includes a hierarchy of data center entities. Each data center entity includes (1) a service or (2) one or more other data center entities. The system generates a cloud platform-independent mainline pipeline that includes: (1) a sequence of phases for deploying software artifacts, such as development, testing, and production phases, and (2) standards for subsequent phases in a sequence of phases to advance software artifacts from one phase to another. Based on the data center layout defined by the declarative specification, the system compiles the cloud platform-independent mainline pipeline to generate a cloud platform-dependent detailed pipeline for the target cloud platform, as well as instructions for performing service-related operations. The system executes the cloud platform-dependent detailed pipeline on the target cloud platform, such as deploying software releases on data center entities within the data center.

This system allows users to specify a period of system freeze (also known as compute suspension) for services configured to run in a data center on a cloud platform. During the system freeze, changes to the system are prohibited or suspended. Changes that can be prevented during a system freeze include upgrades to software artifacts, changes to the configuration of resources used by the system (e.g., infrastructure resources such as load balancers, databases, servers, firewalls, network resources, etc.), changes to data center configuration, modifications to libraries and other system artifacts used by the system, and modifications to applications. While online systems can use continuous delivery platforms to allow developers to modify the source code used in software artifacts, the implementation disclosed herein allows organizations to postpone any such changes for a specific time interval.

For example, organizations providing services through online systems can determine that services running on those systems should not be disrupted during periods of high traffic. Therefore, the implementation plan allows the system to be frozen, ensuring it continues to operate without any changes being made. This reduces the likelihood of system downtime.

As another example, the system may have some issues that require team analysis to determine the cause. However, during the period of system analysis to determine the cause of a specific problem, the system administrator may prefer to freeze the system's services, preventing changes to software artifacts while the system is being monitored and debugged. This helps isolate problems arising from modifications made to the system during monitoring and debugging, making it difficult to determine whether the problem is caused by the original software of the service or by any changes made to the software during debugging.

As another embodiment, multiple subsystems may be running within the system. The disclosed implementation allows for mutually exclusive modification of subsystems, such that only one subsystem (or a small set of subsystems) is modified at a time. A subsystem can be modified at a time by deploying software artifacts for that specific subsystem. A system administrator may prefer to impose a pause on the remaining subsystems, thus modifying only one subsystem at a time. Once the subsystem has been deployed and tested, the pause is lifted to ensure that the subsystem operates reliably before allowing other subsystems to continue making changes to their respective software artifacts. The implementation allows for such a pause to be imposed on the system, thereby selectively preventing modifications to the software artifacts deployed within the system.

The cloud platform is also referred to as the infrastructure in this document. This system can refer to a multi-tenant system, but is not limited to a multi-tenant system, and can be any online system or any computing system with network access to the cloud platform.

Overall system environment

Figure 1 is a block diagram illustrating a system environment for a multi-tenant system configured in a data center on a cloud platform according to an implementation scheme. System environment 100 includes a multi-tenant system 110, one or more cloud platforms 120, and one or more client devices 105. In other implementations, system environment 100 may include more or fewer components.

Multitenant system 110 stores information about one or more tenants 115. Each tenant can be associated with an enterprise that represents a customer of multitenant system 110. Each tenant can have multiple users interacting with the multitenant system via client device 105.

A cloud platform can also be referred to as a cloud computing platform or a public cloud environment. Tenants can use the cloud platform infrastructure language to provide declarative specifications for data centers created on the target cloud platform 120 and use the data centers to perform operations, such as providing resources, performing software releases, etc. Tenant 115 can create one or more data centers on cloud platform 120. A data center represents a set of computing resources, including servers, applications, storage, memory, etc., that can be used by users (e.g., users associated with a tenant). Each tenant can provide different functionalities to its users. Therefore, each tenant can perform different services on the data centers configured for that tenant. Multi-tenant systems can implement different software release and deployment mechanisms for each tenant. Tenants can further obtain or develop software versions including instructions for various services performed in the data centers. Implementation schemes allow tenants to deploy specific versions of software releases for different services running on different computing resources in the data centers.

The computing resources in a data center are secure, and users without authorization to access them may not be able to access them. For example, data center 125a created for tenant 115a may not be accessible to tenant 115b unless access is explicitly authorized. Similarly, data center 125b created for tenant 115b may not be accessible to tenant 115a unless access is explicitly authorized. Furthermore, services provided by the data center can only be accessed by computing systems outside the data center if access is authorized to the computing system according to the data center's declarative specifications.

In a multi-tenant system 110, data from multiple tenants can be stored in the same physical database. However, the database is configured to logically separate one tenant's data from that of other tenants, preventing one tenant from accessing another's data unless explicitly shared. Their data can be stored in tables shared with other clients' data, transparently to the tenants. Database tables can store rows from multiple tenants. Therefore, in a multi-tenant system, various hardware and software components of the system can be shared by one or more tenants. For example, multi-tenant system 110 can execute application servers that simultaneously process requests from multiple tenants. However, multi-tenant systems enforce tenant-level data isolation to ensure that one tenant's job does not access other tenants' data.

Implementations of the cloud platform include AWS (Amazon Web Services), Google Cloud Platform, or Microsoft Azure. Cloud platform 120 provides computing infrastructure services that can be used on demand by tenant 115 or any computing system outside of cloud platform 120. Implementations of the computing infrastructure services provided by the cloud platform include server, storage, database, networking, security, load balancing, software, analytics, intelligence, and other infrastructure service functions. Tenant 115 can use these infrastructure services to build, deploy, and manage applications in a scalable and secure manner.

Multitenant system 110 may include a tenant data store that stores data for various tenants within the multitenant storage. The tenant data store may store data for different tenants in separate physical structures, such as separate database tables or separate databases. Alternatively, the tenant data store may store data for multiple tenants in a shared structure. For example, user accounts for all tenants may share the same database table. However, the multitenant system stores additional information to logically separate the data for different tenants.

Each component shown in Figure 1 represents one or more computing devices. For example, a computing device can be a conventional computer system running a Microsoft ^™ Windows ^™ compatible operating system (OS), Apple ^™ OS X, and/or Linux distribution. A computing device can also be a client device with computer functionality, such as a personal digital assistant (PDA), mobile phone, video game system, etc. Each computing device stores software modules, which store instructions.

Interactions between various components of system environment 100 are typically performed via a network (not shown in Figure 1). In one embodiment, the network uses standard communication technologies and/or protocols. In another embodiment, entities may use custom and/or proprietary data communication technologies to replace or supplement the aforementioned technologies.

Although the techniques disclosed herein are described in the context of multi-tenant systems, these techniques can be implemented using other systems that may not be multi-tenant systems. For example, an online system used by a single organization or enterprise can use the techniques disclosed herein to create one or more data centers on one or more cloud platforms 120.

System Architecture

Multitenant system 110 includes a deployment module for deploying software artifacts on a cloud platform. The deployment module can perform various operations associated with software releases, such as providing resources on the cloud platform, deploying software releases, performing rollbacks of software artifacts installed on data center entities, etc. Figure 2 is a block diagram illustrating the system architecture of deployment module 210 according to an implementation scheme. Deployment module 210 includes a data center generation module 220 and a software release management module 230. Other implementations may have components and/or other components different from those described herein, and functionality may be distributed among the components in different ways.

The data center creation module 220 includes instructions for creating a data center on a cloud platform. The software release management module 230 includes instructions for deploying software releases or software artifacts that run on the data center created by the data center creation module 220.

Data center generation module 220 receives a cloud platform-independent declarative specification of the data center from a user (e.g., a tenant's user). The cloud platform-independent declarative specification of the data center specifies various entities within the data center. In one implementation, the cloud platform-independent declarative specification of the data center includes a hierarchical organization of data center entities, where each data center entity may include one or more services, one or more other data center entities, or a combination of both. Figure 4 illustrates the various types of data center entities in more detail. Data center generation module 220 receives a platform-independent declarative specification and a target cloud platform as input and generates a cloud platform-specific metadata representation for the target cloud platform. Data center generation module 220 deploys the generated cloud platform-specific metadata representation on the target cloud platform to create the data center on the target cloud platform according to the declarative specification.

The software release management module 230 receives (1) an artifact version map 225 and (2) a mainline 235 as input. The artifact version map 225 identifies a specific version of a software release or deployment artifact targeted for deployment on a specific data center entity. The artifact version map 225 maps the data center entity to a software release version targeted for deployment on the data center entity. The mainline 235 includes instructions for operations related to software releases on the data center, such as deploying services, destroying services, providing service resources, destroying service resources, etc.

Mainstream pipeline 235 may include instructions for performing operations related to software releases in different environments (such as development, testing, canary, and production environments), and instructions for determining when to advance a software release from one environment to another. For example, if a software release is deployed in the development environment and executes more than a threshold number of test cases, the release is advanced to the testing environment for further testing, such as system-level and integration testing. If the software release in the testing environment passes a test coverage threshold, it is advanced to the canary environment, where it is offered to a small group of users on a trial basis. If the software release in the canary environment executes without errors within a threshold time, it is advanced to the production environment, where it is offered to all users.

The software release management module 230 compiles the input artifact version map 225 and mainline 235 to generate a cloud-platform-specific detailed pipeline 255 for delivery to the target cloud platform. The cloud-platform-specific detailed pipeline 255 includes instructions for deploying the software release or appropriate version of the deployment artifact on the data center entity, as specified in the artifact version map 225. The software release management module 230 can accept modifications to one of the inputs. For example, a user can modify the input artifact version map 225 and provide the same mainline 235. Thus, the same mainline is used, but a different software release is deployed on the data center entity. The software release management module 230 recompiles the input to generate a new cloud-platform-specific detailed pipeline 255 that deploys the version of the software release according to the new artifact version map 225.

Artifact version mapping can also be referred to as deployment manifest, version manifest, software release mapping, or software artifact version mapping. The main pipeline can also be referred to as the main deployment pipeline or the main programming pipeline.

Figure 2B illustrates the overall process for deploying software artifacts in a data center according to the implementation scheme. Figure 2B shows the layout of data center 265, which includes various data center entities. As shown in Figure 2B, artifact version mapping 225 identifies different versions of software targeted for release on different data center entities 275 of data center 265. Mainstream pipelines represent the flow of deployment artifacts through various environments within the data center. Software release management module 230 combines information from mainstream pipeline 235 with artifact version mapping 225 to determine a cloud platform-specific detailed pipeline 255 that maps the appropriate version of the software artifact to the data center entity based on artifact version mapping 225.

Figure 3 is a block diagram illustrating the architecture of a software release management module 230 according to one embodiment. The software release management module 230 includes a parsing module 310, a pipeline generator module 320, an artifact version mapping store 330, a pipeline store 340, a system configuration freeze module 350, a change processing module 355, and a pipeline execution engine 360. Other embodiments may include more, fewer, or different modules than those indicated herein in Figure 3.

The various modules shown in Figure 3 can be run in a distributed manner on different systems. For example, the pipeline generator module 320 can be executed on a computing system of a multi-tenant system, while the pipeline execution engine 360 can be executed on a computing system of a cloud platform that deploys data center entities and services.

The parsing module 310 parses various types of user input, including declarative specifications for data centers, artifact version mappings 225, and mainstream waterlines 235. The parsing module 310 generates data structures and metadata representations of the processed inputs and provides these data structures and metadata representations to other modules of the software release management module 230 for further processing.

Metadata store 340 stores metadata representations of various transformations in the data center generated by software release management module 230. If an issue is encountered in the current version of the data center, a rollback to a previous version can be performed using the metadata representation of the transformation. The metadata representation of the transformation can be used for verification, auditing, governance, etc., at various stages of the transformation process.

The pipeline generator module 320 processes the main pipeline by combining the artifact version mapping received as input to generate a detailed pipeline for the target cloud platform. The pipeline includes stages that contain instructions for providing services or deploying applications, based on the artifact version mapping to the deployment software release versions for various services on the cloud platform.

The workpiece version mapping store 330 stores the workpiece version mapping received from the user, and the pipeline store 340 stores the main pipeline and the pipeline generated by the pipeline generator module 320.

The system configuration freeze module 350 receives and processes requests to perform system configuration freeze on services running in data center entities configured on a cloud platform. The received request specifies one or more services or data center entities configured on the cloud platform, and the time interval for requesting system configuration freeze on the services or data center entities. Further details of the system configuration freeze module 350 are provided in conjunction with Figure 11. This document describes the various processes performed by the system configuration freeze module 350.

The change processing module 355 identifies requests to change services installed in the data center and tracks information describing the changes. The change processing module 350 interacts with the change management system and records the execution details of the pipeline for implementing changes in services. The recorded details can be used for auditing, for example, to determine why certain changes are needed to the service configuration in the data center.

The pipeline execution engine 360 executes the detailed pipeline generated by the pipeline generator module 320. In one implementation, the pipeline execution engine 360 is a system such as SPINNAKER that executes pipelines for releasing/deploying software. The pipeline execution engine 360 parses the pipeline and executes each stage of the pipeline on the target cloud computing platform. The pipeline execution engine 360 can execute on one or more computing systems on the cloud platform.

Data center generation based on cloud platform

Figure 4 illustrates an embodiment of a declarative specification for a data center according to one implementation. The declarative specification 410 includes multiple data center entities. A data center entity is an instance of a data center entity type, and each data center entity type can have multiple instances. Embodiments of data center entities include data centers, service groups, services, teams, environments, and patterns.

Declarative specification 410 includes definitions for various types of data center entities, including service groups, services, teams, environments, and patterns. A declarative specification includes one or more instances of a data center. The following is a description of the various types of data center entities and their embodiments. These embodiments are illustrative and show some attributes of the data center entity. Other implementations may include different attributes, and attributes with the same functionality may be given names different from those indicated herein. In one implementation, a hierarchical object conforming to a predefined pattern (e.g., JSON (Javascript object notation)) is used to specify the declarative specification.

According to one implementation, service group 520 represents a set of capabilities, features, and services that can be independently built and delivered by one or more computing systems. A service group may also be referred to as a logical service group, functional unit, or bounded context. Service group 520 can also be viewed as a set of services containing a cohesive set of technical use case functions provided by one or more computing systems. Service group 520 enforces security boundaries. Service group 520 defines the scope of modifications. Therefore, any modifications to an entity, such as capabilities, features, or services provided by one or more computing systems within service group 520, may propagate as needed or suitable for the entities within the service group, but not to entities residing outside the bounded definition of service group 520. A data center may include multiple service groups 520. Service group definitions specify attributes, including name, description, identifier, schema version, and service set instance. An example of a service group is a blockchain service group, which includes a set of services for providing blockchain functionality. Similarly, a security service group provides security features. A user interface service group provides functionality for specific user interface features. A shared document service group provides functionality for sharing documents across users. Similarly, there may be several other service groups.

Service groups support specification reusability, providing tenants or users interested in developing data centers with a library of service groups they can readily use. The boundaries of services surrounding a service group are based on security and network issues, among others. A service group is associated with protocols used to perform interactions with it. In one implementation, a service group provides an API (Application Programming Interface) and a collection of services that implement those APIs. Furthermore, service groups are independent of the underlying infrastructure. Service groups provide an blast radius for services within the group, ensuring that the impact of any service failure within the group is limited to the services within the group, with minimal impact outside the group.

The following is an example of a service group specification. A service group specifies various attributes representing the service group's metadata and includes the set of services within the service group. Other types of metadata may exist for service groups, which are not indicated herein.

As shown in the examples above, service groups can specify a set of clusters. A cluster represents a set of compute nodes, such as a set of servers, a set of virtual machines, or a set of containers (such as Kubernetes containers). Physical servers can run multiple containers, each with its own shared file system, CPU, memory, processing space, etc.

A service group specifies a set of services. A service group can specify a cluster for a service, enabling a cluster of compute nodes deployed in a data center on a cloud platform, and if included in a declarative specification, mapping the service to the cluster based on the specified mapping. For example, in the service group embodiment shown above, the service instance "Service Instance 0002" is specified to run on cluster instance cluster 1.

Service groups can specify security groups, each specifying a set of services that are allowed to interact with each other. Services outside a security group require additional authentication to communicate with services within the security group. Alternatively, services within a security group may use one protocol to interact with each other, while services outside the security group may use a different protocol, requiring enhanced authentication to interact with services within the security group. Therefore, security groups specify the policies that determine how services interact with each other. Security policies can specify one or more environments to which a security policy applies. For example, security policy "Policy 1" may apply to a specific environment env1 (e.g., a production environment), and another security policy "Policy 2" may apply to another environment env2 (e.g., a development environment). Security policies can be specified for service group types or specific service types.

In one implementation, the security policy specifies expressions for filtering service groups based on various attributes, making the security policy applicable to the set of filtered service groups. For example, the security policy may specify a list of IP (Internet Protocol) addresses that are whitelisted as a set of service groups identified by the filtered set, and thus allow these computing systems to access the service group or a specific set of services within the service group.

In one implementation, a security policy can specify a source service set and a destination service set for a service group. The source service of a specific service specifies services outside the security group that are allowed to connect to this specific service. The destination service of a specific service specifies services outside the security group that this specific service needs to connect to. During provisioning and deployment, the data center generation module generates instructions for the cloud platform that use cloud platform-specific features and network functions to enforce specific network policies, causing the network policies to implement the security policies specified in the declarative specification.

A data center entity, referred to as a cell, represents a set of services that interact with each other vertically and can be scaled by additional instances or replicas of the cell (i.e., replicas of the service set). Creating multiple instances of a cell allows the system to scale the set of services that interact with each other. A data center instance can include one or more cells. Each cell can include one or more services. A data center can include instances of service groups or cells.

Service definitions specify metadata for the service type, such as database services, load balancer services, etc. Metadata describes various attributes of the service, including the service name, service description, location of the service documentation, any sub-services associated with the service, the service owner, the team associated with the service, service build dependencies specifying other services the service depends on during construction, service startup dependencies specifying other services that should run when the specific service starts, authorized clients, the DNS (Domain Name Server) name associated with the service, service status, and the level of support for the service. Service definitions specify the listening port attribute, which specifies the ports the service can listen on for different communication protocols; for example, the service can listen on port p1 for UDP and port p2 for TCP. Other services within the data center can interact with the service via the ports specified by the service.

The service definition specifies outbound access attributes, which specify the target endpoint. For example, it specifies that the service needs to access a specific external URL (Uniform Resource Locator). During deployment, the datacenter generation module ensures that the cloud platform enforces access policies to provide requests for external URLs for instances of this service type.

Outbound access specifications can identify one or more environment types for which outbound access applies. For example, outbound access for endpoint set S1 can be applied to a specific environment env1 (e.g., production environment), and outbound access for endpoint set S2 can be applied to another environment env2 (e.g., development environment).

The following is an example of a service definition.

{

"Service_Definition":[

{

"Authorization_Client":[],

"Establish dependency relationship":[],

"Description":"Description of the service",

"dns_name":"dns1",

"File":"URL",

"Name":"Name1",

Team definition 450 includes team member names and other team attributes such as names, email addresses, communication channels, etc. The following is an example of a team definition. A service can be associated with one or more teams responsible for modifying that service. Therefore, any modifications made to the service are approved by the team. After being deployed to a cloud platform, a service can be associated with a team responsible for maintaining the service. A team can be associated with a service group, and correspondingly with all services within that service group. For example, a team approves any changes to a service group, such as services that are part of that service group. A team can be associated with a data center, and correspondingly with all service groups within that data center. Team associations specified at the data center level provide a default team for all service groups within the data center, and further provide a default team for all services within that service group.

According to one implementation, team associations specified at the function level override team associations provided at the data center level. Similarly, team associations specified at the service level override default values that may already be provided for team associations specified at the service group level or data center level. Teams can decide how to take certain actions for data center entities associated with the team. Team associations also determine the final metadata representation of the data center on the cloud platform for the compiler to generate for the cloud platform, as well as the number of accounts created for provisioning and deploying the data center on the cloud platform. The data center generation module 210 creates one or more user accounts in the cloud platform and provides team members with access to these user accounts. This allows team members to perform specific actions associated with the data center entities associated with the team, such as making or approving structural changes to or maintaining the data center entity during deployment, including debugging and testing issues that can be identified for the data center entity.

Traditional techniques associate the same team with the data center throughout the design process, resulting in organizational structure influencing the design of the data center or service group. The implementation plan separates team definition from defining the construction of the data center entity, thereby reducing the team's influence on the design and architecture of the data center entity.

Environment definition 460 specifies the type of system environment represented by the data center, such as a development environment, pilot environment, test environment, or production environment. Summary definition 470 refers to a summary of the syntax specifying a particular data center entity definition. Summary definition 470 is used to validate various data center entity definitions. The data center generation module determines the security policy of the data center in the cloud platform-specific metadata representation based on the environment. For example, a specific set of security policies may apply to environment env1, and different sets of security policies may apply to environment env2. For example, security policies in a production environment may provide more restricted access compared to a development environment. Security policies can specify the length of time security tokens are allowed to exist for a specific purpose. For example, long access tokens (e.g., access tokens lasting one week) may be allowed in a development environment, but access tokens used in a production environment have a much shorter lifespan (e.g., several hours). Access tokens can allow users or services to access specific cloud platform resources.

Datacenter definition 420 specifies the attributes and components of a datacenter instance. A declarative specification can specify multiple datacenter instances. Datacenter definition 420 specifies attributes including name, description, environment type, service set, team, datacenter name server, etc. A datacenter definition can specify a summary definition and validate any metadata representation generated from the datacenter definition against the specified summary definition. A datacenter includes core services and a set of features that enable other services to function within the datacenter. Datacenter instances are deployed on a specific cloud platform and can be associated with specific environment types, such as development, testing, pilot, and production.

The following is the definition of a data center instance. A data center instance definition includes a list of service groups included in the data center instance and other attributes, including the data center environment, data center identifier, name, region representing the geographic area, one or more teams associated with the data center, and mode version.

Figure 5 illustrates some exemplary data centers created on a cloud platform according to a declarative specification based on one implementation scheme. Data centers 510 can be created based on declarative specifications processed by the data center generation module 210. As shown in Figure 5, multiple data centers can be configured within the cloud platform 120. Each data center 510 can correspond to tenant 115 of the multi-tenant system 110. Tenant 115 can create one or more data centers 510. Alternatively, data centers 510 can be created by any computing system. Each data center includes one or more service groups. For example, data center 510a includes service groups 520a and 520b, and data center 510b includes service group 520c. A data center can include multiple instances of a specific type of service group. Each service group includes a set of services. For example, service group 520a includes services 530a and 530b, service group 520b includes services 530a, 530b, and 530c, and service group 520c includes services 530e, 530f, and 530g. A service group can include multiple instances of services of the same service type.

Data center generation module 220 uses the following steps to create a data center on a cloud platform based on a declarative specification. Data center generation module 210 receives a cloud platform-independent declarative specification for the data center. The cloud platform-independent declarative specification can be used by tenants in a multi-tenant system or by any other computing system, such as an online system. The cloud platform-independent declarative specification is specified using the cloud platform infrastructure language. The cloud platform-independent declarative specification of the data center is configured to generate a data center on any cloud platform among multiple cloud platforms.

The data center generation module 210 receives information identifying the target cloud platform used to create a data center based on a declarative specification independent of the cloud platform. The target cloud platform can be any of several cloud platforms, such as AWS, Azure, GCP, etc. The data center generation module 210 further receives information about the connection to the target cloud platform, such as credentials for creating a connection to the target cloud platform. The cloud platform can also be referred to as a cloud computing platform.

The data center generation module 210 compiles a cloud platform-independent declarative specification to generate a cloud platform-specific data center representation for creating a data center on the target cloud computing platform. For example, the cloud platform-specific data center representation can refer to user accounts, network addresses, etc., specific to the target cloud computing platform.

The data center generation module 210 sends a platform-specific data center representation and instructions for deploying the data center on the target cloud computing platform. The target cloud computing platform executes the instructions to configure its computing resources to generate the data center based on the platform-specific data center representation. The data center generation module 210 provides users with access to the computing resources of the data center configured on the cloud computing platform. For example, if the data center is created for a tenant in a multi-tenant system, access to the data center is provided to the users associated with that tenant.

Figure 6 is a block diagram illustrating the generation of a data center on a cloud platform based on a declarative specification according to one implementation. The data center generation module 210 receives a cloud platform-independent declarative specification 610 as input. The cloud platform-independent declarative specification 610 can be a version of the declarative specification that has been incrementally modified by the user. The data center generation module 210 processes a specific version of the cloud platform-independent declarative specification 610. Since no cloud platform-independent declarative specification 610 is specified for any particular target cloud platform, the data center generation module 210 can configure a data center on any target cloud platform based on the cloud platform-independent declarative specification 610.

The data center generation module 210 processes the cloud platform-independent declarative specification 610 to generate a cloud platform-independent detailed metadata representation 620 for the data center. The cloud platform-independent detailed metadata representation 620 defines the details of each instance of the data center entity specified in the cloud platform-independent declarative specification 610. The data center generation module 210 creates unique identifiers for data center entity instances (e.g., service instances).

In one implementation, the cloud platform-independent detailed metadata representation 620 includes an array of instances of data center entity types, such as an array of service group instances of a specific service group type. Each service group instance includes an array of service instances. The service instances may also include details of user teams permitted to perform certain actions associated with the service instance. The data center generation module 210 uses these team details during provisioning and deployment, for example, to create user accounts for the service instances and allow team members to access those user accounts.

The cloud-independent detailed metadata representation 620 includes attributes for each instance of the data center entity. Therefore, the description of each instance of the data center entity is expanded to include all details. As a result, the cloud-independent detailed metadata representation 620 of the data center can be significantly larger than the cloud-independent declarative specification 610. For example, the cloud-independent declarative specification 610 can be a few thousand lines of specification, while the cloud-independent detailed data center representation 620 can be millions of lines of generated code. Consequently, the data center generation module 210 keeps the cloud-independent detailed metadata representation 620 immutable; that is, once the representation is finalized, no modifications are made to it. For example, if any updates, deletions, or additions to the data center entity need to be performed, they are performed on the cloud-independent declarative specification 610.

The data center generation module 210 receives a target cloud platform on which a data center is intended to be provisioned and deployed, and generates a cloud platform-specific detailed metadata representation 630 for the data center. For example, the data center generation module 210 interacts with the target cloud platform to generate certain entities (or resources), such as user accounts, Virtual Private Clouds (VPCs), and network resources, such as subnets on the VPC, various connections between entities in the cloud platform, etc. The data center generation module 210 receives resource identifiers of the resources created in the target cloud platform, such as user account names, VPC IDs, etc., and incorporates these into a cloud platform-independent detailed metadata representation 620 to obtain a cloud platform-specific metadata representation 630 for the data center. In one implementation, the data center generation module 210 creates a unique user account on the cloud platform for each team for a given combination of service groups and services. Teams use the user account to perform interactions with specific services of that service group, such as for debugging, for receiving alerts, etc.

The target cloud platform can perform several steps to process the cloud platform-specific detailed metadata representation 630. For example, a cloud platform-independent declarative specification can specify permitted interactions between services. These permitted interactions are specified in the cloud platform-specific detailed metadata representation 630 and implemented as cloud platform network policies. The cloud platform can further create security groups to implement network policies to implement the data center according to the declarative specification.

Declarative specifications, independent of the cloud platform, define the dependencies between services. For example, startup dependencies for each service list all services that should run when a specific service is started. The data center generation module 220 generates a cloud platform-specific, detailed metadata representation of the data center, including information describing these dependencies. This ensures that instructions for deploying services guarantee that the cloud platform starts services in the order specified by the dependencies, and that for each service, when it starts, the services that needed to be started before it are running. Therefore, the dependencies between services represent a dependency graph, and the cloud platform starts services in the order determined by the dependency graph, such that if service A depends on service B, service B is started before service A.

Data center generation module 220 creates trust relationships between user accounts that allow services to access other services via secure communication channels. These trust relationships are generated using base-specific instructions based on declarative specifications (e.g., based on outbound access attributes specified for services). Data center generation module 220 sends instructions to the cloud platform to create network policies based on cloud platform-specific mechanisms that control interactions and access between service groups and services, such as outbound access, security groups, security policies, etc., as specified in the structure of the declarative specifications.

The data center generation module 210 deploys a cloud platform-specific metadata representation 630 on the specific target cloud platform for which it generates the representation. The data center generation module 210 can use the generated metadata representation to perform various verifications, including policy verification, format verification, etc.

The cloud platform-independent declarative specification 610 can be referred to as the declarative data center representation, the cloud platform-independent detailed metadata representation 620 can be referred to as the data center derived metadata representation, and the cloud platform-specific metadata representation 630 can be referred to as the data center hydrated metadata representation.

The overall process for deploying software artifacts on a data center

Figure 7 illustrates the overall process, according to the implementation scheme, for generating a pipeline for configuring software artifacts for deployment on a data center on a cloud platform. The data center generation module generates one or more data centers 710 on the target cloud platform. Each data center is generated according to a declarative specification independent of the cloud platform and has a hierarchical structure of data center entities.

The software release management module 230 generates a cloud platform-independent mainstream pipeline 720. In one implementation, the cloud platform-independent mainstream pipeline includes phases corresponding to data center environments, such as development, testing, canary, and production environments. The mainstream pipeline constitutes a sequence of progressive and/or conditional deployments across various environments (such as development, testing, pilot, or production environments). A mainstream pipeline can be triggered by passing images of software artifacts and includes phases or instructions for deploying builds in development-type environments. The built software artifacts are conditionally advanced to one or more testing environments, then one or more canary environments, and finally deployed to the production environment. The mainstream pipeline can be customized by users (e.g., service owners) to represent specific orchestrations across environments. The mainstream pipeline can be customized to capture specific advancement criteria for moving from one phase to the next. For example, different tenants in a multi-tenant system can customize the mainstream pipeline in different ways. In one implementation, the mainstream pipeline defaults to using the latest version of the software for the service's software artifacts and builds and deploys that version across various environments. Users can use artifact version mapping to ensure that a specific version of a software artifact is deployed on a specific data center entity.

In one implementation, each service deployed in a data center has a cloud platform-independent mainstream waterline generated by the data center entity as defined by the data center's declarative specification. Examples include mainstream waterlines for data center instances, mainstream waterlines for service groups, mainstream waterlines for communities, and mainstream waterlines for services. Mainstream waterlines can be triggered when delivering images of software artifacts. Mainstream waterlines enable continuous deployment controlled by the service owner. Mainstream waterlines also enable on-demand deployment owned by the data center instance owner or the publishing owner.

Certain parts of the mainline pipeline can be customized by users, for example, by tenants of a multi-tenant system deploying services on a data center. For instance, the advance decision pipeline can be customized by a tenant to determine which test cases to execute and what the thresholds are. The software release management module 230 receives customizations 730 of the logic used to advance software artifacts from one stage of the cloud platform-independent mainline pipeline to another.

The software release management module 230 compiles a cloud platform-independent mainstream pipeline to generate a cloud platform-specific detailed deployment pipeline 740 of a hierarchy of data center entities specific to each data center, as specified in the data center's cloud platform-independent declarative specification.

The software release management module 230 further receives code 750 for releasing one or more features of a service deployed on a data center. The software release management module 230 executes a cloud platform-specific detailed deployment pipeline 760 to deploy software artifacts based on the received code.

Figure 8 illustrates an exemplary mainstream pipeline 800 according to an implementation scheme. The mainstream pipeline represents a sequence of stages that represent progressive conditional deployments across various data center environments. Figure 8 shows the stages for different data center environments, including development, test, canary, and production environments. Each stage is further represented as the pipeline executed in that stage. Thus, mainstream pipeline 800 includes a development environment pipeline 810 that feeds to a test environment pipeline 820, a test environment pipeline that feeds to a canary environment pipeline 830, and a canary environment pipeline that feeds to a production environment pipeline 840.

Each stage of the pipeline is a hierarchical pipeline that includes lower-level pipelines. For example, the development environment pipeline 810 includes development main pipelines that feed to data center pipelines D11, D12, ..., depending on the number of data centers specified as having a development environment in the data center's declarative specification.

The test environment pipeline 820 includes test main pipelines that feed to data center pipelines D21, D22, ..., depending on the number of data centers specified as having a test environment in the data center's declarative specification.

The canary environment pipeline 820 includes canary main pipelines feeding into data center pipelines D31, D32, ..., depending on the number of data centers specified as having a canary environment in the data center's declarative specification.

Production environment pipeline 820 includes the main production pipelines that feed to data center pipelines D21, D22, ..., depending on the number of data centers specified as having a test environment in the data center's declarative specification.

Each environment pipeline 810, 820, and 830 includes a procurement decision pipeline 815a, 815b, and 815c, respectively. The procurement decision pipeline 815 collects the output from the data center pipeline of the environment pipeline and determines whether the software artifact is ready to proceed to the next stage. The procurement decision pipeline 815 can determine whether the service's software artifact should proceed to the next stage based on test case results obtained from the data center. For example, if more than a threshold of test cases passes, the procurement decision pipeline 815 advances the software artifact to the next stage. The final environment stage, such as the production environment pipeline, may not have a procurement decision pipeline because there are no subsequent stages to which the software artifact needs to be advanced. As shown in Figure 8, the development environment pipeline's advance decision pipeline 815a determines whether to advance the software artifact from the development phase to the testing phase; the testing environment pipeline's advance decision pipeline 815b determines whether to advance the software artifact from the testing phase to the canary phase; and the canary environment pipeline's advance decision pipeline 815c determines whether to advance the software artifact from the canary phase to the production phase.

The main pipeline comprises multiple pipelines, such as a provisioning pipeline for supplying resources to the target cloud platform and a deployment pipeline for deploying software artifacts on the data center entity. Each pipeline includes a sequence of stages, each stage representing one or more actions that the target cloud platform needs to perform to provision and deploy the data center. The data center generation module 210 generates a detailed pipeline for deploying versions of software artifacts on the data center entity.

In one implementation, the pipeline generator module 320 generates a detailed pipeline using a pipeline template that includes variables. The pipeline template is transformed into a pipeline by providing specific values for the variables in the pipeline. The process of generating a pipeline from a template is called pipeline template hydration. A pipeline template contains template expressions that serve as placeholders for the actual values used in the deployment. For example, a template expression can be replaced by a target-specific parameter value or expression. Multiple pipeline instances can be generated by hydrating pipeline templates for different targets. Template variables represent parameters that can be replaced with specific values for a given target to generate a target-specific pipeline instance. For example, during hydration, the template variable "account_id" can be replaced with the actual value of account_id, such as "12345".

In one implementation, the pipeline generator module 320 generates pipelines hierarchically based on the hierarchy of data center entities. For example, a data center includes different types of data center entities, including data centers, service groups, services, etc. A data center entity may include one or more sub-data center entities. For example, a data center may include one or more service groups as sub-data center entities. A service group may include one or more services as sub-data center entities. Therefore, the data center generation module 210 starts with data center entities at the hierarchical level and generates pipelines for data center entities below that level. For example, the pipeline generator module 320 starts at the data center level and generates pipelines for service groups within the data center. For each service group, the pipeline generator module 320 generates pipelines for the services within the service group.

According to one implementation, the process for executing a pipeline is as follows: Software deployment module 230 receives a request to deploy software artifacts on a set of data center entities in a target cloud platform. Software deployment module 230 executes a main pipeline for one or more data centers. Software deployment module 230 executes an aggregate pipeline for each service group in each data center. The aggregate pipeline includes pipelines for services within the service group. For each service in each service group, the pipeline is executed by performing all stages of the pipeline. Providing the execution of the pipeline results in providing resources for the service, and deploying the pipeline results in deploying the service in the target cloud platform.

Figure 9 illustrates the overall process of staged execution of the main pipeline environment on the cloud platform according to the implementation scheme. Steps 910, 920, 930, 940, and 950 can be performed by each environment pipeline 810, 820, and 830. Production environment pipeline 3 can perform only steps 910 and 920. The steps shown in Figure 9 can be performed for one or more services specified using the manifest file.

The environment pipeline for environment E includes instructions for deploying software 910 on a set of data center entities, for example, a set of data center entities designated as having environment E. In one implementation, software artifacts are generated by compiling the source code of the service. The source code can be obtained from version control software. The data center entity set can include data center instances, service groups, cells, services, or any combination thereof.

The environment pipeline for Environment E also includes instructions for running tests 920 that test software artifacts 920 deployed on the data center entity set. The environment pipeline for Environment E also includes instructions for evaluating test results 930 against advancement criteria, for example, using the advancement decision pipeline 815. If the advancement criteria are not met, steps 910, 920, 930, and 940 can be repeated using revised software artifacts, such as those generated from source code, which include fixes for certain defects identified during testing 920. The environment pipeline for Environment E also includes instructions for proceeding to the next stage 950 if the advancement criteria are met.

In one implementation, a mainstream pipeline comprises a hierarchical structure of pipelines. This hierarchy includes multiple levels, and a pipeline at a specific level includes the next lower-level pipeline as a sub-pipeline. For example, at the highest level of the hierarchy, the mainstream pipeline includes a release mainstream pipeline that deploys a set of services associated with a product. The next level of the hierarchy includes a service mainstream pipeline, which represents all deployments of a specific service across various environments. The next level of the hierarchy may include a service group mainstream pipeline, followed by service mainstream pipelines.

Figure 10 illustrates an exemplary mainline pipeline according to an implementation scheme. The mainline pipeline is a hierarchical pipeline, where each stage of the pipeline may include a pipeline with detailed instructions for performing that stage. The mainline pipeline hierarchy may mirror a data center hierarchy. For example, the top level of the mainline pipeline represents a sequence of stages for different environments. Each environment may include one or more pipelines for data center instances or for other types of data center entities. Data center instance pipeline 1010 may include service group pipeline 1020. Each service group pipeline 1020 may include one or more service pipelines 1030. Data center instance pipeline 1010 may include cell pipelines 1025, each cell pipeline 1025 including one or more service pipelines 1030. Service pipelines 1030 may include stages, each stage representing a pipeline that represents instructions for deploying services for a specific environment. The lowest-level pipeline or leaf-level pipeline in the hierarchy is called a cell pipeline and may include detailed service-specific instructions for performing service-related operations. For example, service deployment may include pre-deployment steps, deployment steps, post-deployment steps, and post-deployment testing and verification steps. A pipeline that is not a leaf-level pipeline and has one or more sub-pipelines is an aggregate pipeline that orchestrates the execution of sub-pipelines.

A mainstream pipeline can be driven by pull requests, which occur when the software's version control system receives requests to consider committing changes to an external repository for inclusion in the project's main repository. Therefore, when a pull request is received, the mainstream pipeline is automatically triggered, and software artifacts are deployed based on the latest software version that received the pull request. The mainstream pipeline performs continuous delivery of software artifacts based on pull requests. For example, the mainstream pipeline can be driven on an on-demand basis by using the application programming interface (API) call request of deployment module 210. On-demand deployment based on the mainstream pipeline can be requested for any set of services and any version of a given service specified using the API. The mainstream pipeline can be invoked to request a rollback from the current version to a previous version, or a rollforward from the currently deployed version to a newer version.

In one implementation, deployment module 210 creates a service main pipeline for each service. These pipelines are triggered when a pull request for the software repository is received. Deployment module 210 receives pipeline templates from users for a specific service. These pipeline templates include detailed instructions for testing, validating, building, etc., for a specific service. Data center generation module 220 receives cloud platform-independent declarative specifications for one or more data centers. Data center generation module 220 generates (or configures) data centers based on the received cloud platform-independent declarative specifications. Deployment module 210 receives the Advance Decision 815 pipeline. The Advance Decision 815 pipeline is integrated into the overall main pipeline.

The pipeline generator creates all pipelines for each data center from a template and combines them in a hierarchical manner via mainstream pipelines, as shown in Figure 10. In one implementation, the pipeline generator generates service pipelines for each service; the pipeline generator generates cell mainstream pipelines to invoke service pipelines; the pipeline generator generates service group mainstream pipelines to invoke cell mainstream pipelines; the pipeline generator generates data center instance mainstream pipelines to invoke service group pipelines; and the pipeline generator generates service mainstream pipelines to invoke data center instance mainstream pipelines.

The following is a snippet showing the main pipeline at each stage. Each stage can specify attributes, including stage name, pipeline type, stage type (e.g., main deployment pipeline or advance pipeline), previous stage, etc.

As shown in the main pipeline review, the first stage is artifact version mapping. The next stage is the main deployment pipeline for deployment to the development environment. The next stage is the advancement pipeline for determining whether the software artifact can be advanced to the next stage. The next stage is the main deployment pipeline for deployment to the test environment. The next stage is the advancement pipeline for determining whether the software artifact can be advanced to the next stage (i.e., the pilot environment).

Software artifact version mapping

In one implementation, deployment module 210 receives an artifact version map that associates various software artifacts and their versions with data center entities. The artifact version map provides a declarative specification of the specific versions of software artifacts that need to be deployed for services in different data center entities. Each data center entity can be uniquely identified based on its position in the data center hierarchy, as specified in the data center's declarative specification. For example, for a service, a software library can act as a software artifact. Software artifacts can have multiple versions, such as V1, V2, V3, etc. The artifact version map can specify that version V1 needs to be deployed in data center entities C1 and C2, and version V2 needs to be deployed in data center entities C3 and C4. Deployment module 210 generates a mainline and instructions that ensure the appropriate software artifact versions are deployed in the data center entities as specified in the artifact version map.

In one implementation, the artifact version map is specified as a JSON (JavaScript Object Notation) file, a YAML file, or a file using any other syntax for representing nested objects. The artifact version map may include a set of <service>:<version> key pairs associated with various data center entities distributed across a data center hierarchy. The artifact version map key pairs act as a whitelist for the corresponding pipelines. If a service's key is not included in the artifact version map, then all pipelines for that service are excluded during pipeline execution. Different artifact version maps can be applied to the same main pipeline, resulting in different services being included/excluded during main pipeline execution.

The following is an example artifact version mapping. The artifact version mapping uses the attribute "environment_type" to specify the environment type. In the following embodiments, an environment type development is specified. An environment type may include one or more datacenter instances; a datacenter instance may include one or more service groups, and a service group may include one or more services. In the following embodiments, the software artifact name is specified as library1, the version is specified as version1, and it is associated with the service instance "instance001". However, the software artifact name and version can be associated with a datacenter entity at any level of the hierarchy. For example, for a specified software artifact name and version or service group, the software artifact name and version apply to all services within the service group, unless the software artifact name and version are overridden by different values of the software artifact name and version specified for a specific service instance within the service group.

Similarly, you can specify software artifact names and versions for a data center instance, and these apply to all service groups or cells within the data center instance, unless an overridden value is specified for the service group.

In one implementation, the artifact version mapping uses the full path of the data center entity, such as "offset_group1/data center1/service_group2/service1", to specify the data center entity. In another implementation, the artifact version mapping uses a regular expression within the full path of the data center entity to specify the set of data center entities. For example, it includes the full path of service_group[?], including service_group1, service_group2, service_group3, etc.

The following is an example of artifact version mapping, which specifies regular expressions to define service sets. The environment type is specified as dev and test, and the data center entities (including data center instances and service groups) in the full path are specified as wildcards, and the service instance is specified as "service*". Therefore, for all data center instances, all service groups, and service names matching service* in both the development (dev) and test environments, version V1 of application "application1" will be deployed.

In some implementations, the job version mapping can specify the parameters used by the pipeline. Therefore, the specified parameters will apply to the interleaved groups for which the specified parameters are assigned.

According to the implementation plan, the following process is used to configure the deployment of software artifacts on data centers on a cloud platform. The data center generation module generates one or more data centers on the target cloud platform. Each data center is generated according to a declarative specification independent of the cloud platform and has a hierarchical structure of data center entities.

The software release management module 230 receives an artifact version mapping that maps data center entities to versions of software artifacts as input. The software release management module 230 also receives a mainstream pipeline independent of the cloud platform as input.

The software release management module 230 combines artifact version mapping to compile a cloud-platform-independent mainstream pipeline to generate a cloud-platform-specific detailed pipeline. In one implementation, the generated cloud-platform-specific detailed pipeline includes artifact version mapping filters prior to certain stages to determine whether certain stages should be enabled or disabled based on the artifact version mapping.

The software release management module 230 further receives code 750 for releasing one or more features of a service deployed on a data center. For example, the code may represent source code obtained from a version control management system that stores a repository of source code to which developers submit changes. The software release management module 230 executes a cloud platform-specific deployment pipeline to deploy software artifacts based on the received code.

Artifact version mapping and mainline pipelines can be used to orchestrate various types of operations related to the continuous delivery of software artifacts in cloud-based data centers. Artifact version mapping and mainline pipelines can be configured to perform aggregated retry operations on services, service groups, or any data center entity. Artifact version mapping includes configuration of retry operations for data center entities, including retry policies, the threshold number of retries to perform in case of pipeline stage failure, whether user acknowledgment is required before retries, or automatic retries. For example, a retry policy could be a fixed fallback policy that pauses execution for a fixed period before a retry. Other retry policies can be configured using artifact version mapping and mainline pipelines. In one implementation, the pipeline generator introduces a call retry phase within the aggregate pipeline to trigger the retry policy if a previous pipeline stage fails. The retry policy and configuration parameters specified for a data center entity apply to all data center entities and services within that data center entity, unless nested data center entities override the value.

Service system configuration freeze

In one implementation, the system configuration freeze module 350 performs a system configuration freeze on services deployed in the target cloud platform. The system configuration freeze can be performed in response to a request received from a user (e.g., a system administrator). This request may identify a data center entity and request the freezing of the system configuration of all services running on that data center entity. The system configuration freeze ensures that no system configuration changes are made to the services of the data center entity, and no software artifacts associated with the services running on the data center entity are modified.

In one implementation, the system receives a request to set a system configuration freeze for a data center entity (e.g., service group, cell, data center, etc.). The system identifies the services within the data center entity, for example, based on the data center's declarative specification or metadata describing the data center's topological hierarchy. For example, the system performs a hierarchical traversal of all data center entities below the data center entity and identifies all services within the data center entity. The system performs a system configuration freeze on all services identified within data center entity D1, unless the system receives a request to prevent a system configuration freeze on service S1 or smaller data center entity D2 within the identified data center entity D1 from being performed on service S1 or smaller data center entity D2.

A system configuration freeze can be implemented in response to the determination that a workload exceeding a threshold is expected during a time interval. For example, if a tenant in a multi-tenant system expects a large number of processing requests during a time interval such as a holiday, the tenant can request to freeze services running on the production system during that time interval to avoid the possibility of service interruption.

In one implementation, the system may receive a modified declarative specification and a request to perform a system configuration freeze on a data center entity. For example, a data center may be modified to add a data center entity D2 within the hierarchy of data center entity D1, or to add a service s1 within the hierarchy of data center entity D1. The system creates the requested data center entity D2 or service S1 and enforces a system configuration freeze on the created service or data center entity for a specified time interval.

System configuration freezes can be performed in response to determining that a problem associated with a data center entity is being diagnosed during a time interval. For example, tenants in a multi-tenant system may identify a problem with a data center entity or service, such as a performance issue causing poor service performance. The tenant's system administrator can run diagnostic processes and tools to identify the root cause of the problem. While diagnosing the problem, the tenant may prefer to freeze the system configuration of the data center entity or service to minimize changes to the system, thus enabling accurate problem diagnosis.

A system configuration freeze can be performed in response to the determination that a specific system configuration change is being executed during a time interval. A system configuration freeze prevents any other system configuration changes from occurring during the time interval. Therefore, the system can ensure that changes to potentially interfering system configurations are executed in a mutually exclusive manner, such that only one change is executed at a time, and that no other changes are prevented during that period.

Although the embodiments described herein illustrate processes in a multi-tenant system context, the disclosed techniques can be applied to any other system, such as a system dedicated to a single organization.

Figure 11 illustrates the system architecture of the system configuration freeze module according to an embodiment. The system configuration freeze module 350 includes a freeze request processing module 1110, a lock manager 1120, and a system configuration freeze metadata store 1130. Other embodiments may include more or fewer components than those indicated herein in Figure 11.

The freeze request processing module 1110 receives and processes a system configuration freeze request. The system configuration freeze request can be received from a user's (e.g., a system administrator's) client device. The system configuration freeze request specifies the time interval during which a system configuration freeze needs to be enforced. The system configuration freeze request can identify one or more services that need to be frozen during the specified time interval. The system configuration freeze request can specify a data center entity for which the system configuration freeze is to be performed. Therefore, the request specifies that all services running in the data center entity need to be frozen during the specified time interval, so that no modifications are made to the software artifacts associated with these services, and no modifications are made to any type of configuration of these services. The freeze request processing module 1110 identifies a list of services that need to be frozen during the specified time interval. For example, if the system configuration freeze request specifies a data center entity, the freeze request processing module 1110 identifies all services configured to run in the data center entity.

The system configuration freeze metadata store 1130 stores metadata related to performing service configuration freezes, including metadata describing the request for a system configuration freeze. The metadata describing the request includes a request identifier, the time period for which the system freeze request needs to be enforced, and the data center entity for which the system configuration freeze needs to be enforced. The time interval can be specified using a start time and an end time, or by using a start time and the length of the time interval. The system configuration freeze metadata store 1130 also stores a mapping from services to locks, which can be obtained by the system configuration freeze module 350 used to enforce a freeze on a service, or by a pipeline used to modify the service configuration. The metadata describing the lock includes an estimate of the length of the time period for which the lock is requested.

Lock manager 1120 receives and acquires a lock request. The lock manager receives the lock acquisition request from freeze request processing module 1110 or from the pipeline execution module. In one embodiment, the lock manager is a distributed lock service that can run on a system different from the computing system running freeze request processing module 1110. Lock manager 1120 also receives and releases a previously acquired lock. Acquiring a lock associated with a service within a specific time period ensures that no pipeline configured to modify the service can proceed during that time period. Releasing the lock after the time interval allows any pipeline attempting to modify the service configuration to continue, thereby ending the system configuration freeze of the service.

Pipeline generator module 320 generates pipelines for changing the system configuration of data center entities. The generated pipelines are configured to allow system configuration freeze module 350 to freeze the configuration of any service.

Figure 12 illustrates an exemplary pipeline for enforcing a system configuration freeze according to an implementation scheme. Figure 12 shows some stages of a pipeline generated for modifying the service configuration of a service, for example, for modifying software artifacts associated with the service. The generated pipeline may include other stages not shown in Figure 12. The pipeline shown in Figure 12 may be part of the service pipeline 1030 shown in Figure 10.

Therefore, the generated pipeline 1200 includes a pre-change phase 1210, a change phase 1220, and a post-change phase 1230. Change phase 1220 includes various types of system configuration changes that can be implemented by the software release management module 230. These system configuration changes include, but are not limited to, deploying new services in a data center entity, destroying services in a data center entity, providing resources in a data center entity, destroying resources in a data center entity, performing any utility operations, or performing a rollback of a service deployment by restoring to an earlier version of a service or software artifact. The system configuration freeze module 350 is configured to freeze all these types of changes to the data center entity during a time interval.

Pre-change phase 1210 includes instructions for acquiring locks associated with a service. Post-change phase 1230 includes instructions for releasing locks acquired in pre-change phase 1210. In one embodiment, information identifying locks associated with each service is stored in system configuration freeze metadata storage 1130. Therefore, pipeline generator module 320 can access locks associated with the pipeline and generate instructions for acquiring locks, and include the generated instructions in the pre-change pipeline. Alternatively, pipeline generator module 320 can generate instructions for accessing system configuration freeze metadata storage 1130 to obtain identifiers of locks associated with services and then acquiring the locks. If a lock cannot be acquired, for example, if the lock has previously been acquired by another entity and has not been released, the instructions for acquiring locks in pre-change phase 1210 cause pipeline execution to be paused. For example, system configuration freeze module 350 acquires locks for all services that need to be frozen during a time interval, thereby pausing the execution of any pipeline attempting to change the system configuration of a service during the time interval. In one implementation, if the lock associated with the service cannot be acquired, the pre-change stage 1210 instruction causes pipeline 1200 to fail. In another implementation, if the lock associated with the service cannot be acquired within a threshold time period, the pre-change stage 1210 instruction causes pipeline 1200 to fail. Therefore, pipeline execution times out and fails after a certain period. If pipeline execution fails, subsequent stages, such as change stage 1220 and post-change stage 1230, are not executed.

The process of freezing system configuration

Figure 13 illustrates the process for changing the system configuration of services deployed on a data center in a cloud platform, according to an implementation scheme. The data center generation module 220 generates one or more data centers 1310 based on a cloud platform-independent declarative specification, as shown in Figures 6 and 7, for example. Each data center has a set of services deployed within it. The software release management module 230 receives an artifact version map 1320, which includes information describing any changes required to the services installed and executed in the data center. For example, changes may include adding a new service to the data center, removing a service currently deployed in the data center, changing the configuration of a service, deploying a new version of a software artifact for the service, etc.

The software release management module 230 generates a cloud platform-independent mainstream pipeline 1330, which includes a change management phase, such as the mainstream pipeline shown in Figure 12. The software release management module 230 compiles a cloud platform-independent mainstream pipeline 1340 to generate a cloud platform-specific detailed pipeline, which includes instructions for deploying services based on artifact version mappings deployed on the data center of the target cloud platform.

The software release management module 230 receives instructions 1350 for modifying the system configuration of services deployed on the data center entity. The software release management module 230 executes a cloud platform-specific detailed pipeline to appropriately modify the system configuration of the services based on artifact version mappings 1360. The cloud platform-specific detailed pipeline includes a change phase, which includes instructions for making changes associated with one or more software artifacts of the data center entity; a pre-change phase, which includes instructions for acquiring a lock; and a post-change phase, which releases the lock after the change is made. If the lock cannot be acquired in the pre-change phase, the execution of the pipeline is suspended.

Figure 14 illustrates the overall process for performing a system configuration freeze on a data center entity configured on a cloud platform, according to an implementation scheme. The system configuration freeze module 350 receives a request 1410 to perform a system configuration freeze on the data center entity for a duration interval. The system configuration freeze module 350 identifies all services configured to be performed within the data center entity. The system configuration freeze module 350 may identify services based on the declarative specifications of the data center to which the data center entity belongs. The request to perform a system configuration freeze specifies the time interval for which the system configuration freeze is requested to be enforced.

The system configuration freeze module 350 acquires locks 1430 associated with services of data center entities within the time interval. In one implementation, once the system configuration freeze time interval begins, the system configuration freeze module 350 accesses the system configuration freeze metadata store 1130 to determine an identifier 1420 corresponding to each identified service, and acquires the lock corresponding to the lock identifier.

After the start of the time interval and before its end, the software release management module 230 receives a request 1440 for modifying the system configuration of a service running on the data center entity during the time interval. The software release management module 230 executes a pipeline 1450 for deploying software artifacts associated with the service running on the data center entity. The execution of the pipeline results in the execution of a pre-change phase, causing the pipeline execution to pause until a lock is acquired. After the time interval ends, the system configuration freeze module 350 releases the lock acquired for the service running on the data center entity 1460. As a result, the pre-change phase of the pipeline is able to acquire the lock and cause the pipeline execution to complete 1470, thereby allowing the requested changes to the service to be executed after the time interval. According to some implementations, after the change phase is completed and the requested modifications to the system configuration are made, the post-change phase releases the lock acquired for the service. This process ensures that any pipeline attempting to modify the service running on the data center entity is prevented from doing so during the time interval.

According to some implementation schemes, the pre-change phase includes instructions to determine whether the pipeline's pre-change phase has failed to acquire a lock within a threshold time. If the pipeline's pre-change phase fails to acquire a lock within the threshold time, the pipeline execution fails. Therefore, the user is provided with an indication of pipeline execution failure, and the user must send subsequent instructions to modify the system configuration of the service.

In one implementation, if a pipeline modifying the system configuration of a service of a data center entity is being executed when a system configuration freeze request is requested, the system postpones the system configuration freeze and correspondingly delays the interval in which the system configuration freeze occurs until the current pipeline completes execution. However, the system prevents any new pipeline execution that might change the system configuration of the data center entity's services from starting until the system configuration freeze is complete. This is because the currently executing pipeline has a lock associated with the service, causing the system configuration freeze request to be shelved when the system configuration freeze module 350 attempts to acquire the lock on the data center entity's service. The system administrator can manually destroy the currently running pipeline to allow the system configuration freeze to continue.

In one implementation, if a failure occurs during pipelined execution when a system configuration freeze is forcibly executed on a data center entity, the pipelined execution may result in locks being acquired but not released. Lock manager 1120 performs a lock garbage collection process that checks if any locks remain unreleased after the system configuration freeze time interval. If lock manager 1120 identifies any locks associated with the services of the data center entity after the time interval, lock manager 1120 releases these locks.

In some implementations, a lock is associated with an estimated duration representing the expected length of time it will take to acquire the lock. The system can use the lock duration as an estimate of the time spent executing a pipeline. The system can prioritize certain pipelines based on their expected execution durations; for example, shorter pipelines may be prioritized over those that take significantly longer. However, if a pipeline execution takes longer than the duration specified for the lock, the system only logs the difference. The system maintains a list of services for which the specified lock duration is significantly shorter than the actual time it takes to acquire the lock. The system prioritizes these pipelines differently from other pipelines that have a precise duration specified for the lock. In one implementation, the system automatically determines the lock duration based on the pipeline's past execution time.

In one implementation, system configuration freezes are associated with priority. Requests to modify services are also associated with priority measurements. If, during a system configuration freeze period, the software release management module 230 receives a service modification request with a priority higher than the system configuration freeze priority, the service modification request for the data center entity is allowed to proceed during the time interval, regardless of the system configuration freeze. This mechanism allows for continued repairs should any issues arise during a system configuration freeze (e.g., a pause) that require urgent fixes.

System architecture of the change processing module

Figure 15 illustrates the system architecture of the change processing module according to an implementation scheme. The change processing module 355 includes a change determination module 1510, a change management client 1520, an event queue manager 1530, an event listener, and an event queue store 1550. Other implementations may include more or fewer components than those indicated herein in Figure 15.

The change determination module 1510 identifies changes being performed on services installed in one or more data centers. In one implementation, the change determination module 1510 receives a set of instructions that identify changes to be performed compared to the current configuration of the services on the data centers. For example, the change may specify a new service to be added, an existing service to be deleted, or a service configuration that needs to be modified.

In one implementation, the change determination module 1510 stores various versions of the artifact version map. The software release management module 230 stores version V1 of the artifact version map corresponding to the current configuration of the data center on the target cloud computer. The software release management module 230 receives version V2 of the artifact version map that needs to be implemented on the target cloud computer. Version V2 of the artifact version map differs from version V1. The change determination module 1510 compares the two versions of the artifact version map to determine the differences between them. Therefore, the change determination module 1510 compares the received new version of the artifact version map with a previous version of the artifact version map currently being deployed to identify the changes requested in the new version of the artifact version map. These changes may include new services that need to be installed on a specific data center entity, versions of existing services that need to be modified (e.g., upgraded) on a specific data center entity, services that need to be removed from a specific data center entity, etc. A pipeline generator generates a main pipeline to implement the requested changes to the service configuration.

The change management client 1520 interacts with the change management system to store information describing the changes being implemented. The change management system can be any system that stores records describing changes and manages tasks related to those changes, such as sending change-related alerts or messages, receiving approvals for specific changes, etc. The change management client 1520 calls the change management system's API to interact with it. The change management system can be an external system that provides change management as a service.

Event Queue Manager 1530 creates and manages queues that store events related to deployment tasks. Event Queue Manager 1530 can create queues for various teams that need to be notified of specific events. For example, a team might be responsible for a data center entity defined in the data center declarative specification. The change processing module identifies specific events, such as the failure or success of a service installed in the data center entity, and notifies the team. Event Listener 1540 listens for events in the pipeline execution engine 360. Events describe detailed actions performed for each pipeline execution, such as whether a particular stage executed successfully or failed, or whether a message was generated for a particular stage, etc. Event Listener 1540 categorizes events and associates them with specific data center entities. Therefore, Event Listener 1540 provides information describing the event to Event Queue Manager 1530 for storage in event queues. Event Queue Store 1550 stores different queues, such as queues for one or more teams defined in the data center declarative specification.

Figure 16 illustrates an exemplary mainstream pipeline for managing changes according to an implementation scheme. The mainstream pipeline includes stages similar to those shown in Figure 8, corresponding to different system environments, through which software artifacts are advanced for deployment to the data center. Therefore, mainstream pipeline 1600 includes stages for different data center environments, including development, testing, canary, and production environments. The mainstream pipeline also includes a change management stage; thus, mainstream pipeline 1600 includes a development environment stage 810 that feeds to a testing environment stage 820, which feeds to a change management stage 1610, which feeds to a canary environment pipeline 830, which feeds to a production environment pipeline 840, which feeds to a change closure stage 1620. Each stage is represented as the pipeline executed in that stage.

A mainline pipeline represents a set of pipelines that form a hierarchical structure. For example, it might correspond to a pipeline for each data center entity in a hierarchy of data center entities, as specified in the declarative specification used to generate the data center. The change management phase of each pipeline executes and manages actions related to changes represented by the mainline. For example, the change management phase includes instructions for performing interactions with the change management system.

In one implementation, the change management phase includes instructions for creating change cases in the change management system, each change case representing a set of changes performed by the mainline. The change cases in the change management system represent information describing a set of changes performed on services within the data center. The change sets may be stored as one or more records in the change management system's database.

In one implementation, the configuration file (e.g., an artifact version map) includes information describing existing change cases that will be used to store information describing the set of changes performed on services in the data center. For example, users (such as system administrators) can create change cases and provide them in the configuration file for use during the change management phase.

The change management stages of various pipelines, represented by the main pipeline, provide the change management system with the deployment status of various services for storage in association with change cases. For example, a pipeline for a specific service provides the change management system with the deployment status of that service. The status can indicate whether the deployment was successful, failed, or generated errors or warnings, etc.

If the deployment of all services specified in the artifact version mapping is successful, the change closure phase closes the change case. Closing the change case prevents further modifications to it; for example, further information may not be stored in the change case. However, users can view the information stored in the change case, for example, to audit specific changes made to the service.

Change management process

Figure 17 illustrates the overall process for change management of services deployed on a data center on a cloud platform according to the configuration of the implementation scheme. The data center generation module 220 generates one or more data centers 1710 based on a declarative specification independent of the cloud platform, as shown in the processes illustrated in Figures 6 and 7. Each data center has a set of services deployed within it. The software release management module 230 receives an artifact version map 1720, which includes information describing any changes required to the services installed and executed in the data center. For example, changes may include adding a new service to the data center, removing a service currently deployed in the data center, changing the configuration of a service, deploying a new version of a software artifact for the service, etc.

The software release management module 230 generates a cloud-platform-independent mainstream pipeline 1730, which includes a change management phase, such as the mainstream pipeline shown in Figure 16. The cloud-platform-independent mainstream pipeline may also include a change closure phase at the end of the pipeline. The software release management module 230 compiles the cloud-platform-independent mainstream pipeline to generate a cloud-platform-specific detailed pipeline 1740, which includes instructions for deploying services based on artifact version mappings deployed on the data center of the target cloud platform.

Software release management module 230 receives source code 1750, which is used to compile and build software artifacts and deploy them on the target cloud platform. Software release management module 230 executes a cloud platform-specific detailed pipeline to deploy the appropriate version 1760 of the software artifacts according to the artifact version mapping. The execution of the cloud platform-specific detailed pipeline results in the execution of a change management phase for each service deployed according to the artifact version mapping. The execution of the change management phase of the main pipeline can result in the creation of change cases in the change management system. The execution of the change management phase for a single service pipeline sends the pipeline's execution status for storage associated with change cases in the change management system. The execution of the change management phase for a single service pipeline can cause the pipeline to await approval before advancing the software artifacts processed by the pipeline to subsequent stages, such as from the testing phase to the canary phase or from the canary phase to the production phase. The execution of the change closure phase of the main pipeline results in the closure of the change case, preventing further modifications from being recorded in the change case.

Figure 18 illustrates the process executed by the change management phase of the mainstream pipeline according to the implementation scheme. The software release management module 230 creates a change case 1810 in the change management system. The cloud-independent mainstream pipeline comprises a pipeline set that includes at least one pipeline for each service deployed or modified in the data center. Each pipeline in this set includes a change management phase.

The software release management module 230 repeats steps 1820, 1830, and 1840 for each pipeline associated with a service in the data center. The software release management module 230 executes the pipeline associated with the service 1820. The software release management module 230 sends the pipeline's execution status 1830 to the change management system. The software release management module 230 receives approval to proceed to the next stage 1840. Approval can be received from the user or team associated with the service. If the software release management module 230 receives an instruction to proceed to the next stage without approval, the software release management module 230 executes steps to repeat some steps of the pipeline, for example, after receiving revised source code or modified software artifacts. For example, if the number of passed test cases is below a threshold, approval may be rejected. Therefore, the source code is modified, and the steps are repeated to pass at least the threshold number of test cases, thereby receiving approval. Once all pipelines in the pipeline set are approved and the pipeline set is successfully executed, the software release management module 230 executes a change closure phase, which results in the change cases being closed.

In one implementation, the software release management module 230 monitors events logged by the pipeline execution engine, which indicate the status of each action performed by the pipeline execution engine. The software release management module 230 analyzes each event in the data center context to determine if the event is significant enough to be recorded in a change case, and then sends information describing the event to a queue for reporting to the team associated with the service.

Figure 19 illustrates a process according to an implementation scheme for managing queues used to collect event information related to changes in service configuration. The event listener 1540 of the change processing module 355 listens for events 1910 from the pipeline execution engine. For each event, the change processing module 355 performs the following steps 1920, 1930, 1940, and 1940. The change processing module 355 determines the service for which the event was generated. Therefore, the change processing module 355 evaluates the event to determine if it is significant enough to be logged and reported by the change management system to the team associated with that service 1920. If the change processing module 355 determines that the event is significant enough to be logged and reported, the change processing module 355 identifies the queue 1930 for the team associated with that service. The change processing module 355 sends the event information to the identified queue 1940. The event is reported to the team associated with the service via the queue. The change processing module 355 sends a status update 1950 to the change management system based on the events stored associated with the change case.

Computer Architecture

Figure 20 is a high-level block diagram illustrating a functional view of a typical computer system used as one of the entities shown in environment 100 of Figure 1, according to an embodiment. At least one processor 2002 is shown connected to chipset 2004. Also connected to chipset 2004 are memory 2006, storage device 2008, keyboard 2010, graphics adapter 2012, indicating device 2014, and network adapter 2016. Display 2018 is connected to graphics adapter 2012. In one embodiment, the functionality of chipset 2004 is provided by memory controller hub 2020 and I/O controller hub 2022. In another embodiment, memory 2006 is directly connected to processor 2002 instead of chipset 2004.

Storage device 2008 is a non-transitory computer-readable storage medium, such as a hard disk drive, optical disc read-only memory (CD-ROM), DVD, or solid-state storage device. Memory 2006 stores instructions and data used by processor 2002. Pointing device 2014 may be a mouse, trackball, or other type of pointing device and is used in conjunction with keyboard 2010 to input data into computer system 200. Graphics adapter 2012 displays images and other information on monitor 2018. Network adapter 2016 connects computer system 2000 to a network.

As is known in the art, computer 2000 may have different and/or other components than those shown in FIG. 20. Furthermore, computer 2000 may lack certain illustrated components. For example, computer system 2000 acting as multi-tenant system 110 may lack keyboard 2010 and indicating device 2014. Additionally, storage device 2008 may be local and/or remote from computer 2000 (e.g., embodied within a storage area network (SAN)).

Computer 2000 is adapted to execute computer modules for providing the functions described herein. As used herein, the term "module" refers to computer program instructions and other logic for providing specified functions. Modules may be implemented in hardware, software, and/or firmware. A module may include one or more processes, and/or be provided only by a portion of those processes. Modules are typically stored on storage device 2008, loaded into memory 2006, and executed by processor 2002.

The type of computer system 2000 used by the entities in the system environment can vary depending on the implementation scheme and the processing power used by the entities. For example, client devices may be mobile phones with limited processing power, small displays 2018, and may lack indicating devices 2014. In contrast, multi-tenant systems or cloud platforms may include multiple blade servers working together to provide the functionality described herein.

Other precautions

Specific naming of components, capitalization of terms, attributes, data structures, or any other programming or structural aspects are not mandatory or important, and the mechanisms for implementing the described embodiments may have different names, formats, or protocols. Furthermore, as described, the system may be implemented via a combination of hardware and software, or entirely with hardware components. Moreover, the specific division of functionality among the various system components described herein is merely exemplary and not mandatory; a function performed by a single system component may alternatively be performed by multiple components, and a function performed by multiple components may alternatively be performed by a single component.

Some of the descriptions above are characterized by the algorithms and symbolic representations of the operations on information. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the essence of their work to others skilled in the art. Although these operations are described functionally or logically, they are understood to be implemented by computer programs. Furthermore, it is sometimes convenient, without loss of generality, to refer to these operations as modules or functional names.

Unless explicitly stated in the discussion above, it should be understood that throughout the description, the use of terms such as “processing” or “computing” or “arithmetic” or “determining” or “displaying” refers to the actions and processes of a computer system or similar electronic computing device that manipulate and convert data represented as physical (electronic) quantities within computer system memory or registers or other such information storage, transmission or display devices.

Some implementations described herein include processing steps and instructions described in algorithmic form. It should be noted that the processing steps and instructions of the implementation can be embodied in software, firmware, or hardware, and when embodied in software, can be downloaded to reside on and operated from different platforms used by a real-time network operating system.

The described embodiments also relate to means for performing the operations described herein. The means may be specifically constructed for the desired purpose, or it may include a general-purpose computer selectively activated or reconfigured by a computer program stored on a computer-readable medium accessible to a computer. Such a computer program may be stored in a non-transitory computer-readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory, random access memory, EPROM, EEPROM, magnetic cards or optical cards, application-specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, and each medium is coupled to a computer system bus. Furthermore, the computer mentioned in the specification may include a single processor, or may be an architecture employing multiple processors to increase computing power.

The algorithms and operations presented herein are not substantially related to any specific computer or other device. Various general-purpose systems can also be used with the programs taught herein, or it can be demonstrated that it is convenient to construct more specialized devices to perform the required method steps. The necessary structures for various such systems, as well as equivalent variations, will be apparent to those skilled in the art. Furthermore, this embodiment is described without reference to any specific programming language. It should be understood that the teachings of the embodiments described herein can be implemented using various programming languages.

The implementation scheme is well-suited for a wide variety of computer network systems across diverse topologies. In this field, the configuration and management of large networks involves storage devices and computers communicatively connected to different computers and storage devices via a network (such as the Internet).

Finally, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and may not be intended to depict or limit the subject matter of the invention. Therefore, the disclosure of embodiments is intended to be illustrative, not restrictive.

Claims (20)

1. A computer-implemented method for managing changes to services configured to run in a data center on a cloud platform, the method comprising: Access a data center configured on a target cloud platform, the data center executing a set of services, the data center including a hierarchy of data center entities, wherein each data center entity includes (1) one or more services or (2) one or more other data center entities; Receive modifications to the service set executed on the data center; Generate a mainstream pipeline for deploying services on the target cloud platform, the mainstream pipeline comprising: Multiple phases, including instructions for deploying the service, each phase corresponding to a system environment, wherein at least some of said system environments belong to a list including development, testing, and production environments; and The change management phase includes instructions for interacting with the change management system. Compile the main pipelines to generate a pipeline set corresponding to the service set, each generated pipeline including a change management phase; and Each of the pipeline sets is executed, wherein the execution of the change management phase of the pipeline provides the deployment status of one or more services to the change management system. 2. The computer-implemented method according to claim 1 further includes: Receive audit requests for specific changes to specific services deployed in the data center; In response to receiving the audit request, identify the deployment status of the specific service from the change management system; and In response to the audit request, provide the identified deployment status of the specific service. 3. The computer-implemented method according to claim 1, wherein the change management phase of the mainstream waterline occurs prior to the phase in the production environment. 4. The computer-implemented method according to claim 1, wherein the pipeline assembly forms a hierarchy corresponding to the hierarchy of the data center entities in the data center. 5. The computer-implemented method of claim 1, wherein the mainline advances the software artifacts of the service through a system environment set. 6. The computer-implemented method of claim 5, wherein a stage of the plurality of stages of the mainline includes instructions for executing a set of test cases to determine whether to advance the software artifact from the current stage to the next stage. 7. The computer-implemented method of claim 1, wherein the main pipeline includes a pipeline hierarchy corresponding to the hierarchy of data center entities specified by a declarative specification independent of the cloud platform. 8. The computer-implemented method of claim 7, wherein the pipeline hierarchy includes a data center instance pipeline, the data center instance pipeline includes one or more service group pipelines, and the service group pipeline includes one or more service pipelines. 9. The computer-implemented method of claim 7, wherein the change management phase of the generated pipeline includes instructions to await approval before proceeding to the next phase. 10. The computer-implemented method according to claim 1, further comprising: Accept declarative specifications independent of the cloud platform; and Compile the cloud platform-independent declarative specification to generate a cloud platform-specific data center representation. 11. The computer-implemented method of claim 10, wherein compiling the cloud-independent declarative specification comprises: A first version of the cloud-independent detailed metadata representation of the data center is generated from the cloud-independent declarative specification; and A second version of the detailed metadata representation of the data center, independent of the cloud platform, is generated from the modified declarative specification. 12. The computer-implemented method according to claim 11, further comprising: Based on the first version of the cloud platform-independent detailed metadata representation, a platform-specific detailed metadata representation is generated for the target cloud platform; and Based on the platform-specific detailed metadata representation, the data center is deployed on the target cloud platform. 13. The computer-implemented method of claim 10, wherein the cloud platform-independent declarative specification includes the definition of one or more data center instances, each data center instance including one or more service groups, wherein each service group includes a service set. 14. The computer-implemented method of claim 1, wherein the modification of the service set performed on the data center is specified via artifact version mapping. 15. The computer-implemented method according to claim 14, further comprising: The modification of the service set performed on the data center is determined by comparing the artifact version mapping with a previously received artifact version mapping. 16. A non-transitory computer-readable storage medium for storing instructions, which, when executed by a computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps comprising: Access a data center configured on a target cloud platform, the data center executing a set of services, the data center including a hierarchy of data center entities, wherein each data center entity includes (1) one or more services or (2) one or more other data center entities; Receive modifications to the service set executed on the data center; Generate a mainstream pipeline for deploying services on the target cloud platform, the mainstream pipeline comprising: Multiple phases, including instructions for deploying the service, each phase corresponding to a system environment, wherein at least some of said system environments belong to a list including development, testing, and production environments; and The change management phase includes instructions for interacting with the change management system. Compile the main pipelines to generate a pipeline set corresponding to the service set, each generated pipeline including a change management phase; and Each of the pipeline sets is executed, wherein the execution of the change management phase of the pipeline provides the deployment status of one or more services to the change management system. 17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions further cause the computer processor to perform the following steps: Receive audit requests for specific changes to specific services deployed in the data center; In response to receiving the audit request, identify the deployment status of the specific service from the change management system; and In response to the audit request, provide the identified deployment status of the specific service. 18. The non-transitory computer-readable storage medium of claim 16, wherein the change management phase of the mainstream pipeline occurs prior to the phase in the production environment. 19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions further cause the computer processor to perform the following steps: The modification of the service set performed on the data center is determined by comparing the artifact version mapping with a previously received artifact version mapping. 20. A computer system, comprising: Computer processor; and A non-transitory computer-readable storage medium for storing instructions, which, when executed by the computer processor, cause the computer processor to perform steps for configuring a data center in a cloud platform, the steps including: Access a data center configured on a target cloud platform, the data center executing a set of services, the data center including a hierarchy of data center entities, wherein each data center entity includes (1) one or more services or (2) one or more other data center entities; Receive modifications to the service set executed on the data center; Generate a mainstream pipeline for deploying services on the target cloud platform, the mainstream pipeline comprising: Multiple phases, including instructions for deploying the service, each phase corresponding to a system environment, wherein at least some of said system environments belong to a list including development, testing, and production environments; and The change management phase includes instructions for interacting with the change management system. Compile the main pipelines to generate a pipeline set corresponding to the service set, each generated pipeline including a change management phase; and Each of the pipeline sets is executed, wherein the execution of the change management phase of the pipeline provides the deployment status of one or more services to the change management system.