diff --git a/README.md b/README.md index dd9a1ddf900be8ee3d4be56d2f93de4881c3b65c..36f3c05db880086b87621b5ec50aa50db0964088 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,10 @@ general process overview and specific documentation for different stakeholders. - [Release template files](https://gitlab.com/gitlab-org/release-tools/tree/master/templates) - [How to create a blog post for a patch release](general/patch/blog-post.md) +## Upcoming changes + +- [Security development on gitlab.com](general/security/future.md) design document + ## Further Reading - ["Release Manager - The invisible hero"](https://about.gitlab.com/2015/06/25/release-manager-the-invisible-hero/) (2015-06-25) diff --git a/general/security/future.md b/general/security/future.md new file mode 100644 index 0000000000000000000000000000000000000000..bb0f2cf84da02a5f2344d81e3de6da25a6689c88 --- /dev/null +++ b/general/security/future.md @@ -0,0 +1,362 @@ +# Security development on gitlab.com + +## Goals + +1. Continue to develop new features and bug fixes in public projects on gitlab.com. +1. Develop security fixes in private projects on gitlab.com. +1. Continue to automatically deploy new features and bug fixes to gitlab.com for real-world testing. +1. Automatically deploy *undisclosed* security fixes to gitlab.com for up-to-date security. +1. Continue to release a new minor version of GitLab every month on the 22nd, built from a specific snapshot of gitlab.com's deployed version. +1. Continue to regularly publish patch versions for the latest minor version to resolve bugs. +1. Continue to regularly publish security versions for the three most recent minor versions to resolve security vulnerabilities. + +## Definitions + +### Projects + +For the sake of example, we'll be focusing on a single project, GitLab EE, +but this setup expands to all GitLab components that need security fixes +(Omnibus, Shell, Pages, Gitaly, etc.). + +The project has three locations: **Canonical**, **Security**, and **Build**. + +* **Canonical**: +* **Security**: +* **Build**: + +Throughout this document, when referencing **Canonical**, **Security**, or **Build** +with **Bold Title Case**, we're referencing these projects. + +### Branches + +* `master`: the default branch +* `stable`: Any versioned stable branch, e.g., `12-3-stable` +* `security/master` and `security/stable`: One of the above branches mirrored to **Security** via pull mirror with branch prefix. +* `security/auto-deploy`: Any versioned auto-deploy branch, e.g., `security/12-3-auto-deploy-20190911` + +Throughout this document, when referencing a branch in `backticks`, or as a +graph node, we're referencing these branches. + +There are two special branches, `security/upstream` and +`security/master-canonical` that are detailed in later sections. + +## Mirroring + +Since there's a lot happening between multiple projects, it may be helpful to +start with a high-level overview: + +```mermaid +graph LR + classDef canonical fill:#74bd3d,stroke:#333,stroke-width:1px + classDef security fill:#ff7272,stroke:#333,stroke-width:1px + classDef fade opacity:0.8,stroke-dasharray:4 4 + + canonical((Canonical)) + security((Security)) + build((Build)) + rt{release-tools} + + canonical -->|push mirror| build + canonical -->|pull mirror| security + security -->|push mirror| build + + build -.-> |git clone| rt + rt -.-> |rake publish| canonical + + class security security + class canonical canonical + style build fill:#72b9ff +``` + +Now let's break it down into smaller, more detailed parts. + +**Canonical** syncs protected branches to **Build** via push mirroring: + +```mermaid +graph TD + classDef canonical fill:#74bd3d,stroke:#333,stroke-width:1px + classDef security fill:#ff7272,stroke:#333,stroke-width:1px + classDef fade opacity:0.8,stroke-dasharray:4 4 + + subgraph Canonical + c-master(master) + c-stable(stable) + c-feature(feature/some-new-feature) + c-bug(15015-some-bug-fix) + end + + subgraph Build + b-master(master) + b-stable(stable) + end + + c-master -->|push mirror| b-master + c-stable -->|push mirror| b-stable + + class c-feature,c-bug fade + class c-master,c-stable canonical + class b-master,b-stable canonical +``` + +**Security** receives protected branches from **Canonical** via pull mirroring with a `security/` branch prefix: + +```mermaid +graph TD + classDef canonical fill:#74bd3d,stroke:#333,stroke-width:1px + classDef security fill:#ff7272,stroke:#333,stroke-width:1px + classDef fade opacity:0.8,stroke-dasharray:4 4 + + subgraph Security + s-master(security/master) + s-stable(security/stable) + end + + subgraph Canonical + c-master(master) + c-stable(stable) + c-feature(feature/some-new-feature) + c-bug(15015-some-bug-fix) + end + + c-master -->|pull mirror| s-master + c-stable -->|pull mirror| s-stable + + class c-master,c-stable,c-auto-deploy canonical + class s-master,s-stable,s-auto-deploy security + class c-feature,c-bug fade +``` + +> :warning: The prefixed `security/*` branches are targets for **Security** merge requests that resolve security issues, and will regularly diverge from their **Canonical** source. + +For reasons that will be explained later in this document, **Security** needs +a pristine version of **Canonical** `master` that never diverges. This branch +is synced via a scheduled [Merge Train] job: + +[Merge Train]: https://gitlab.com/gitlab-org/merge-train + +```mermaid +graph TD + classDef canonical fill:#74bd3d,stroke:#333,stroke-width:1px + classDef security fill:#ff7272,stroke:#333,stroke-width:1px + classDef fade opacity:0.8,stroke-dasharray:4 4 + + subgraph Canonical + c-master(master) + c-stable(stable) + c-feature(feature/some-new-feature) + c-bug(15015-some-bug-fix) + end + + subgraph Security + s-master(security/master) + s-stable(security/stable) + sc-master(security/master-canonical) + end + + c-master -->|pull mirror| s-master + c-stable -->|pull mirror| s-stable + c-master -.-> |merge-train| sc-master + + class c-master,c-stable,c-auto-deploy,sc-master canonical + class s-master,s-stable,s-auto-deploy security + class c-feature,c-bug fade +``` + +**Security** syncs protected branches to **Build** via push mirroring: + +```mermaid +graph TD + classDef canonical fill:#74bd3d,stroke:#333,stroke-width:1px + classDef security fill:#ff7272,stroke:#333,stroke-width:1px + classDef fade opacity:0.8,stroke-dasharray:4 4 + + subgraph Security + s-master(security/master) + s-stable(security/stable) + end + + subgraph Build + bs-master(security/master) + bs-stable(security/stable) + + b-master(master) + b-stable(stable) + end + + s-master -->|push mirror| bs-master + s-stable -->|push mirror| bs-stable + + class b-master,b-stable,b-auto-deploy canonical + class s-master,s-stable,s-auto-deploy security + class bs-master,bs-stable,bs-auto-deploy security + class b-master,b-stable fade +``` + +## Normal development + +Nothing changes for day-to-day development, so we will not document that +procedure in detail here. + +## Security development + +Developers work on security fixes in **Security**, targeting `security/master`. A +fix goes through review, gets merged, and then gets the `unpublished` label. +The purpose of this label is described later. + +Once a fix is merged into `security/master`, a developer can open backport +MRs against the `security/stable` branches. + +```mermaid +graph TD + classDef canonical fill:#74bd3d,stroke:#333,stroke-width:1px + classDef security fill:#ff7272,stroke:#333,stroke-width:1px + classDef fade opacity:0.8,stroke-dasharray:4 4 + + subgraph Security + s-master(security/master) + + s-stable(security/stable) + s-auto-deploy(security/auto-deploy) + s-developer(security/fix-mermaid-xss) + s-developer-backport(security/fix-mermaid-xss-backport) + end + + subgraph Build + b-master(master) + b-stable(stable) + + bs-master(security/master) + bs-stable(security/stable) + bs-auto-deploy(security/auto-deploy) + end + + s-developer -->|merge| s-master + s-master -.->|cherry-pick| s-auto-deploy + s-developer-backport -->|merge| s-stable + + s-master -->|push mirror| bs-master + s-stable -->|push mirror| bs-stable + s-auto-deploy -->|push mirror| bs-auto-deploy + + class sc-master,sc-stable canonical + class s-master,s-stable,s-auto-deploy security + class b-master,b-stable,b-auto-deploy canonical + class bs-master,bs-stable,bs-auto-deploy security + class s-developer,s-developer-backport fade +``` + +## Auto-deploys + +See the [existing documentation](https://gitlab.com/gitlab-org/release/docs/blob/master/general/deploy/auto-deploy.md), +with two key differences: + +1. The auto-deploy branch is created in **Security** with a `security/` prefix: + + ```mermaid + sequenceDiagram + participant RT as release-tools + participant C as Canonical + participant S as Security + + note left of RT: Weekly + + rect rgb(116, 189, 61) + RT->>C:Latest green `master` + C->>RT:d1f38742 + end + + rect rgb(255, 114, 114) + RT->>S:Create `security/auto-deploy` from d1f38742 + end + ``` + +2. We cherry-pick merge requests labeled `Pick into auto-deploy` from **Canonical** _and_ **Security**. + + ```mermaid + sequenceDiagram + participant RT as release-tools + participant C as Canonical + participant S as Security + + rect rgb(116, 189, 61) + RT->>C:MRs with `Pick into auto-deploy` + C->>RT:!5483 + RT->>C:Merge commit SHA for !5483 + C->>RT:601cc959 + RT->>S:Cherry-pick 601cc959 into `security/auto-deploy` + end + + rect rgb(255, 114, 114) + RT->>S:MRs with `Pick into auto-deploy` + S->>RT:!549 + RT->>S:Merge commit SHA for !549 + S->>RT:2a150e3e + RT->>S:Cherry-pick 2a150e3e into `security/auto-deploy` + end + ``` + +This ensures gitlab.com is running a version of the codebase that has a +passing build with the latest bug fixes and features, as well as all known +security fixes. + +## Monthly releases + +Same as before, but using a **Security** `security/auto-deploy` branch as the +source. + +## Patch releases + +Same as before. + +## Security patch releases + +Same as before, but using **Security**'s `security/stable` branches as the +source. + +## Branch syncing after publication + +After a security fix is published, we need to get it from **Security**'s +`security/master` into **Canonical**'s `master`. However, `security/master` +may contain fixes that haven't yet been published, so we can't simply merge +the two branches. + +This is where the `unpublished` label comes into play. When we publish a +security release, we: + +1. Create a `security/upstream` branch in **Security** based off of `security/master-canonical`. +1. Gather `unpublished` **Security** merge requests. +1. Cherry-pick the merge commits into `security/upstream`. +1. Open an upstream merge request in **Canonical** with a source of `security/upstream` and a target of `master`. +1. Automatically Approve and "Merge when pipeline succeeds" the merge request. + +Additionally, the `security/stable` branches on **Build** get merged into +their non-prefixed counterparts and pushed to **Canonical**: + +```mermaid +sequenceDiagram + participant B as Build + participant RT as release-tools + participant C as Canonical + + B->>RT:pull `stable` + B->>RT:pull `security/stable` + RT-->>RT:merge `security/stable` into `stable` + + RT->>C:push `stable` + C->>B:Mirror `stable` +``` + +Here's an overview of the *post-publication* state of all projects and branches: + +* **Canonical** + * `master`: Receiving active development, contains all _released_ security fixes. + * `stable`: Contains all released security backports. +* **Security**: + * `security/master`: Contains latest changes from **Canonical** `master`, plus unreleased security fixes. + * `security/stable`: Contains all released security backports. + * `security/auto-deploy`: Contains latest changes from a green **Canonical** `master`, plus unreleased security fixes. +* **Build**: All protected branches are synced from **Canonical** and **Security**. + +--- + +[Return to Guides](../README.md)