diff --git a/README.md b/README.md index a45b30d711512f440141205f97e4b41be696bc65..a37cd96fc36aff40665cfd465b40e7e13dae54d5 100644 --- a/README.md +++ b/README.md @@ -1,126 +1,127 @@ +[![pipeline status](https://gitlab.com/gitlab-org/gitlab-shell/badges/main/pipeline.svg)](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) +[![coverage report](https://gitlab.com/gitlab-org/gitlab-shell/badges/main/coverage.svg)](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) +[![Code Climate](https://codeclimate.com/github/gitlabhq/gitlab-shell.svg)](https://codeclimate.com/github/gitlabhq/gitlab-shell) + # GitLab Shell -## GitLab Shell handles git SSH sessions for GitLab +GitLab Shell is a SSH server, configured to handle Git SSH sessions for GitLab. +GitLab Shell is not a Unix shell, nor a replacement for Bash or Zsh: -GitLab Shell handles git SSH sessions for GitLab and modifies the list of authorized keys. -GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh. +- It limits shell access to specific `git` commands. +- It provides authorization and transport for these commands. -When you access the GitLab server over SSH then GitLab Shell will: +## Requirements -1. Limit you to predefined git commands (git push, git pull). -1. Call the GitLab Rails API to check if you are authorized, and what Gitaly server your repository is on -1. Copy data back and forth between the SSH client and the Gitaly server +GitLab Shell is written in Go, and needs a Go compiler to build. It still requires +Ruby to build and test, but not to run. -If you access a GitLab server over HTTP(S) you end up in [gitlab-workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse). +Download and install the current version of Go from [golang.org](https://golang.org/dl/) -An overview of the four cases described above: +We follow the [Golang Release Policy](https://golang.org/doc/devel/release.html#policy) +of supporting the current stable version and the previous two major versions. -1. git pull over SSH -> gitlab-shell -> API call to gitlab-rails (Authorization) -> accept or decline -> establish Gitaly session -1. git push over SSH -> gitlab-shell (git command is not executed yet) -> establish Gitaly session -> (in Gitaly) gitlab-shell pre-receive hook -> API call to gitlab-rails (authorization) -> accept or decline push +## Handling `git` SSH sessions -## Code status +When you access the GitLab server over SSH, GitLab Shell: -[![pipeline status](https://gitlab.com/gitlab-org/gitlab-shell/badges/main/pipeline.svg)](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) -[![coverage report](https://gitlab.com/gitlab-org/gitlab-shell/badges/main/coverage.svg)](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) -[![Code Climate](https://codeclimate.com/github/gitlabhq/gitlab-shell.svg)](https://codeclimate.com/github/gitlabhq/gitlab-shell) - -## Requirements +1. Limits you to `git push`, `git pull`, and `git fetch` commands only. +1. Calls the GitLab Rails API to check if you are authorized, and what Gitaly server your repository is on. +1. Copies data back and forth between the SSH client and the Gitaly server. -GitLab Shell is written in Go, and needs a Go compiler to build. It still requires -Ruby to build and test, but not to run. +### `git pull` over SSH -Download and install the current version of Go from https://golang.org/dl/ +Git pull over SSH -> gitlab-shell -> API call to gitlab-rails (Authorization) -> accept or decline -> establish Gitaly session -We follow the [Golang Release Policy](https://golang.org/doc/devel/release.html#policy) -of supporting the current stable version and the previous two major versions. +### `git push` over SSH -## Check +Git push over SSH -> gitlab-shell (git command is not executed yet) -> establish Gitaly session -> (in Gitaly) gitlab-shell pre-receive hook -> API call to gitlab-rails (authorization) -> accept or decline push -Checks if GitLab API access and redis via internal API can be reached: +For more details see [Architecture](doc/architecture.md) - make check +### Modifies `authorized_keys` -## Compile +GitLab Shell modifies the `authorized_keys` file on the client machine. -Builds the `gitlab-shell` binaries, placing them into `bin/`. +- TODO some details needed here. - make compile +### Runs on port 22 -## Install +GitLab Shell runs on `port 22` on an Omnibus installation. To use a regular SSH +service, configure it on an alternative port. -Builds the `gitlab-shell` binaries and installs them onto the filesystem. The -default location is `/usr/local`, but can be controlled by use of the `PREFIX` -and `DESTDIR` environment variables. +### Access GitLab with `https` - make install +If you access a GitLab server over HTTP(S) you end up in +[`gitlab-workhorse`](https://gitlab.com/gitlab-org/gitlab-workhorse). -## Setup +## `gitlab-sshd` -This command is intended for use when installing GitLab from source on a single -machine. In addition to compiling the gitlab-shell binaries, it ensures that -various paths on the filesystem exist with the correct permissions. Do not run -it unless instructed to by your installation method documentation. +See [`gitlab-sshd`](doc/gitlab-sshd). - make setup +## Commands +- `make check`: Checks if GitLab API access and Redis (via internal API) can be reached +- `make compile`: Builds the `gitlab-shell` binaries, placing them into `bin/`. +- `make install`: Builds the `gitlab-shell` binaries and installs them onto the + file system. The default location is `/usr/local`, but you can change it with the `PREFIX` + and `DESTDIR` environment variables. +- `make setup`: Don't run this command unless instructed to by your installation method + documentation. Used when installing GitLab from source on a single machine. Compiles + the `gitlab-shell` binaries, and ensures that file system paths exist and contain + correct permissions. -## Testing +### Testing Run tests: - bundle install - make test +```shell +bundle install +make test +``` -Run gofmt: +Run `gofmt`: - make verify +```shell +make verify +``` Run both test and verify (the default Makefile target): - bundle install - make validate +```shell +bundle install +make validate +``` ### Gitaly Some tests need a Gitaly server. The -[`docker-compose.yml`](./docker-compose.yml) file will run Gitaly on -port 8075. To tell the tests where Gitaly is, set -`GITALY_CONNECTION_INFO`: +[`docker-compose.yml`](docker-compose.yml) file runs Gitaly on port 8075. +To tell the tests the location of Gitaly, set `GITALY_CONNECTION_INFO`: - export GITALY_CONNECTION_INFO='{"address": "tcp://localhost:8075", "storage": "default"}' - make test +```plaintext +export GITALY_CONNECTION_INFO='{"address": "tcp://localhost:8075", "storage": "default"}' +make test +``` -If no `GITALY_CONNECTION_INFO` is set, the test suite will still run, but any -tests requiring Gitaly will be skipped. They will always run in the CI -environment. +If no `GITALY_CONNECTION_INFO` is set, the test suite still runs, but any +tests requiring Gitaly are skipped. These tests always run in the CI environment. -## Git LFS remark +## References -Starting with GitLab 8.12, GitLab supports Git LFS authentication through SSH. +- [Using the GitLab Shell chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/#using-the-gitlab-shell-chart) -## Logging Guidelines - -In general, it should be possible to determine the structure, but not content, -of a gitlab-shell or gitlab-sshd session just from inspecting the logs. Some -guidelines: +## Git LFS remark -- We use [`gitlab.com/gitlab-org/labkit/log`](https://pkg.go.dev/gitlab.com/gitlab-org/labkit/log) - for logging functionality -- **Always** include a correlation ID -- Log messages should be invariant and unique. Include accessory information in - fields, using `log.WithField`, `log.WithFields`, or `log.WithError`. -- Log success cases as well as error cases -- Logging too much is better than not logging enough. If a message seems too - verbose, consider reducing the log level before removing the message. +GitLab supports Git LFS authentication through SSH. ## Releasing -See [PROCESS.md](./PROCESS.md) +See [PROCESS.md](PROCESS.md) ## Contributing -See [CONTRIBUTING.md](./CONTRIBUTING.md). +See [CONTRIBUTING.md](CONTRIBUTING.md). ## License -See [LICENSE](./LICENSE). +See [LICENSE](LICENSE). diff --git a/doc/architecture.md b/doc/architecture.md new file mode 100644 index 0000000000000000000000000000000000000000..78f60d5612159ced1f6bca0154ec7dd87aacaa25 --- /dev/null +++ b/doc/architecture.md @@ -0,0 +1,31 @@ +# Architecture + +```mermaid +sequenceDiagram + participant Git on client + participant SSH server + participant AuthorizedKeysCommand + participant GitLab Shell + participant Rails + participant Gitaly + participant Git on server + + Note left of Git on client: git fetch + Git on client->>+SSH server: ssh git fetch-pack request + SSH server->>+AuthorizedKeysCommand: gitlab-shell-authorized-keys-check git AAAA... + AuthorizedKeysCommand->>+Rails: GET /internal/api/authorized_keys?key=AAAA... + Note right of Rails: Lookup key ID + Rails-->>-AuthorizedKeysCommand: 200 OK, command="gitlab-shell upload-pack key_id=1" + AuthorizedKeysCommand-->>-SSH server: command="gitlab-shell upload-pack key_id=1" + SSH server->>+GitLab Shell: gitlab-shell upload-pack key_id=1 + GitLab Shell->>+Rails: GET /internal/api/allowed?action=upload_pack&key_id=1 + Note right of Rails: Auth check + Rails-->>-GitLab Shell: 200 OK, { gitaly: ... } + GitLab Shell->>+Gitaly: SSHService.SSHUploadPack request + Gitaly->>+Git on server: git upload-pack request + Note over Git on client,Git on server: Bidirectional communication between Git client and server + Git on server-->>-Gitaly: git upload-pack response + Gitaly -->>-GitLab Shell: SSHService.SSHUploadPack response + GitLab Shell-->>-SSH server: gitlab-shell upload-pack response + SSH server-->>-Git on client: ssh git fetch-pack response +``` diff --git a/doc/gitlab-sshd.md b/doc/gitlab-sshd.md new file mode 100644 index 0000000000000000000000000000000000000000..77100b6ae45ebc0d1a37c5165b5198352df546a4 --- /dev/null +++ b/doc/gitlab-sshd.md @@ -0,0 +1,30 @@ +# gitlab-sshd + +`gitlab-sshd` is a binary in [`gitlab-shell`](https://gitlab.com/gitlab-org/gitlab-shell) +which runs as a persistent SSH daemon. It will replace `OpenSSH` on GitLab SaaS, +and eventually other cloud-native environments. Instead of running an `sshd` process, +we run a `gitlab-sshd` process that does the same job, in a more focused manner: + +```mermaid +sequenceDiagram + participant Git on client + participant GitLab SSHD + participant Rails + participant Gitaly + participant Git on server + + Note left of Git on client: git fetch + Git on client->>+GitLab SSHD: ssh git fetch-pack request + GitLab SSHD->>+Rails: GET /internal/api/authorized_keys?key=AAAA... + Note right of Rails: Lookup key ID + Rails-->>-GitLab SSHD: 200 OK, command="gitlab-shell upload-pack key_id=1" + GitLab SSHD->>+Rails: GET /internal/api/allowed?action=upload_pack&key_id=1 + Note right of Rails: Auth check + Rails-->>-GitLab SSHD: 200 OK, { gitaly: ... } + GitLab SSHD->>+Gitaly: SSHService.SSHUploadPack request + Gitaly->>+Git on server: git upload-pack request + Note over Git on client,Git on server: Bidirectional communication between Git client and server + Git on server-->>-Gitaly: git upload-pack response + Gitaly -->>-GitLab SSHD: SSHService.SSHUploadPack response + GitLab SSHD-->>-Git on client: ssh git fetch-pack response +```