diff --git a/doc/administration/database_load_balancing.md b/doc/administration/database_load_balancing.md index 45f27a8a8f2ecfd5a53dfb2f7015a6474854eaf5..92b8342f2518cf25d899677b90dac05a88446acc 100644 --- a/doc/administration/database_load_balancing.md +++ b/doc/administration/database_load_balancing.md @@ -1,272 +1,9 @@ --- -stage: Enablement -group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'postgresql/database_load_balancing.md' +remove_date: '2022-02-19' --- -# Database Load Balancing **(FREE SELF)** +This file was moved to [another location](postgresql/database_load_balancing.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1283) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.0. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60894) from GitLab Premium to GitLab Free in 14.0. - -Distribute read-only queries among multiple database servers. - -## Overview - -Database load balancing improves the distribution of database workloads across -multiple computing resources. Load balancing aims to optimize resource use, -maximize throughput, minimize response time, and avoid overload of any single -resource. Using multiple components with load balancing instead of a single -component may increase reliability and availability through redundancy. -[_Wikipedia article_](https://en.wikipedia.org/wiki/Load_balancing_(computing)) - -When database load balancing is enabled in GitLab, the load is balanced using -a simple round-robin algorithm, without any external dependencies such as Redis. - -In the following image, you can see the load is balanced rather evenly among -all the secondaries (`db4`, `db5`, `db6`). Because `SELECT` queries are not -sent to the primary (unless necessary), the primary (`db3`) hardly has any load. - -![DB load balancing graph](img/db_load_balancing_postgres_stats.png) - -## Requirements - -For load balancing to work, you need at least PostgreSQL 11 or newer, -[**MySQL is not supported**](../install/requirements.md#database). You also need to make sure that you have -at least 1 secondary in [hot standby](https://www.postgresql.org/docs/11/hot-standby.html) mode. - -Load balancing also requires that the configured hosts **always** point to the -primary, even after a database failover. Furthermore, the additional hosts to -balance load among must **always** point to secondary databases. This means that -you should put a load balancer in front of every database, and have GitLab connect -to those load balancers. - -For example, say you have a primary (`db1.gitlab.com`) and two secondaries, -`db2.gitlab.com` and `db3.gitlab.com`. For this setup, you need to have 3 -load balancers, one for every host. For example: - -- `primary.gitlab.com` forwards to `db1.gitlab.com` -- `secondary1.gitlab.com` forwards to `db2.gitlab.com` -- `secondary2.gitlab.com` forwards to `db3.gitlab.com` - -Now let's say that a failover happens and db2 becomes the new primary. This -means forwarding should now happen as follows: - -- `primary.gitlab.com` forwards to `db2.gitlab.com` -- `secondary1.gitlab.com` forwards to `db1.gitlab.com` -- `secondary2.gitlab.com` forwards to `db3.gitlab.com` - -GitLab does not take care of this for you, so you need to do so yourself. - -Finally, load balancing requires that GitLab can connect to all hosts using the -same credentials and port as configured in the -[Enabling load balancing](#enabling-load-balancing) section. Using -different ports or credentials for different hosts is not supported. - -## Use cases - -- For GitLab instances with thousands of users and high traffic, you can use - database load balancing to reduce the load on the primary database and - increase responsiveness, thus resulting in faster page load inside GitLab. - -## Enabling load balancing - -For the environment in which you want to use load balancing, you'll need to add -the following. This balances the load between `host1.example.com` and -`host2.example.com`. - -**In Omnibus installations:** - -1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - - ```ruby - gitlab_rails['db_load_balancing'] = { 'hosts' => ['host1.example.com', 'host2.example.com'] } - ``` - -1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - ---- - -**In installations from source:** - -1. Edit `/home/git/gitlab/config/database.yml` and add or amend the following lines: - - ```yaml - production: - username: gitlab - database: gitlab - encoding: unicode - load_balancing: - hosts: - - host1.example.com - - host2.example.com - ``` - -1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. - -### Load balancing for Sidekiq - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334494) in GitLab 14.1, load balancing for Sidekick is enabled by default. - -Sidekiq jobs mostly write to the primary database, but there are read-only jobs that can benefit -from the use of Sidekiq load balancing. -These jobs can use load balancing and database replicas to read the application state. -This allows to offload the primary database. - -For Sidekiq, we can define -[data consistency](../development/sidekiq_style_guide.md#job-data-consistency-strategies) -requirements for a specific job. - -## Service Discovery **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5883) in GitLab 11.0. - -Service discovery allows GitLab to automatically retrieve a list of secondary -databases to use, instead of having to manually specify these in the -`database.yml` configuration file. Service discovery works by periodically -checking a DNS A record, using the IPs returned by this record as the addresses -for the secondaries. For service discovery to work, all you need is a DNS server -and an A record containing the IP addresses of your secondaries. - -To use service discovery you need to change your `database.yml` configuration -file so it looks like the following: - -```yaml -production: - username: gitlab - database: gitlab - encoding: unicode - load_balancing: - discover: - nameserver: localhost - record: secondary.postgresql.service.consul - record_type: A - port: 8600 - interval: 60 - disconnect_timeout: 120 -``` - -Here, the `discover:` section specifies the configuration details to use for -service discovery. - -### Configuration - -The following options can be set: - -| Option | Description | Default | -|----------------------|---------------------------------------------------------------------------------------------------|-----------| -| `nameserver` | The nameserver to use for looking up the DNS record. | localhost | -| `record` | The record to look up. This option is required for service discovery to work. | | -| `record_type` | Optional record type to look up, this can be either A or SRV (GitLab 12.3 and later) | A | -| `port` | The port of the nameserver. | 8600 | -| `interval` | The minimum time in seconds between checking the DNS record. | 60 | -| `disconnect_timeout` | The time in seconds after which an old connection is closed, after the list of hosts was updated. | 120 | -| `use_tcp` | Lookup DNS resources using TCP instead of UDP | false | - -If `record_type` is set to `SRV`, then GitLab continues to use round-robin algorithm -and ignores the `weight` and `priority` in the record. Since SRV records usually -return hostnames instead of IPs, GitLab needs to look for the IPs of returned hostnames -in the additional section of the SRV response. If no IP is found for a hostname, GitLab -needs to query the configured `nameserver` for ANY record for each such hostname looking for A or AAAA -records, eventually dropping this hostname from rotation if it can't resolve its IP. - -The `interval` value specifies the _minimum_ time between checks. If the A -record has a TTL greater than this value, then service discovery honors said -TTL. For example, if the TTL of the A record is 90 seconds, then service -discovery waits at least 90 seconds before checking the A record again. - -When the list of hosts is updated, it might take a while for the old connections -to be terminated. The `disconnect_timeout` setting can be used to enforce an -upper limit on the time it takes to terminate all old database connections. - -Some nameservers (like [Consul](https://www.consul.io/docs/discovery/dns#udp-based-dns-queries)) can return a truncated list of hosts when -queried over UDP. To overcome this issue, you can use TCP for querying by setting -`use_tcp` to `true`. - -## Balancing queries - -Read-only `SELECT` queries balance among all the secondary hosts. -Everything else (including transactions) executes on the primary. -Queries such as `SELECT ... FOR UPDATE` are also executed on the primary. - -## Prepared statements - -Prepared statements don't work well with load balancing and are disabled -automatically when load balancing is enabled. This should have no impact on -response timings. - -## Primary sticking - -After a write has been performed, GitLab sticks to using the primary for a -certain period of time, scoped to the user that performed the write. GitLab -reverts back to using secondaries when they have either caught up, or after 30 -seconds. - -## Failover handling - -In the event of a failover or an unresponsive database, the load balancer -tries to use the next available host. If no secondaries are available the -operation is performed on the primary instead. - -If a connection error occurs while writing data, the -operation is retried up to 3 times using an exponential back-off. - -When using load balancing, you should be able to safely restart a database server -without it immediately leading to errors being presented to the users. - -## Logging - -The load balancer logs various events in -[`database_load_balancing.log`](logs.md#database_load_balancinglog), such as - -- When a host is marked as offline -- When a host comes back online -- When all secondaries are offline -- When a read is retried on a different host due to a query conflict - -The log is structured with each entry a JSON object containing at least: - -- An `event` field useful for filtering. -- A human-readable `message` field. -- Some event-specific metadata. For example, `db_host` -- Contextual information that is always logged. For example, `severity` and `time`. - -For example: - -```json -{"severity":"INFO","time":"2019-09-02T12:12:01.728Z","correlation_id":"abcdefg","event":"host_online","message":"Host came back online","db_host":"111.222.333.444","db_port":null,"tag":"rails.database_load_balancing","environment":"production","hostname":"web-example-1","fqdn":"gitlab.example.com","path":null,"params":null} -``` - -## Handling Stale Reads **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3526) in GitLab 10.3. - -To prevent reading from an outdated secondary the load balancer checks if it -is in sync with the primary. If the data is determined to be recent enough the -secondary is used, otherwise it is ignored. To reduce the overhead of -these checks we only perform these checks at certain intervals. - -There are three configuration options that influence this behavior: - -| Option | Description | Default | -|------------------------------|----------------------------------------------------------------------------------------------------------------|------------| -| `max_replication_difference` | The amount of data (in bytes) a secondary is allowed to lag behind when it hasn't replicated data for a while. | 8 MB | -| `max_replication_lag_time` | The maximum number of seconds a secondary is allowed to lag behind before we stop using it. | 60 seconds | -| `replica_check_interval` | The minimum number of seconds we have to wait before checking the status of a secondary. | 60 seconds | - -The defaults should be sufficient for most users. Should you want to change them -you can specify them in `config/database.yml` like so: - -```yaml -production: - username: gitlab - database: gitlab - encoding: unicode - load_balancing: - hosts: - - host1.example.com - - host2.example.com - max_replication_difference: 16777216 # 16 MB - max_replication_lag_time: 30 - replica_check_interval: 30 -``` + + diff --git a/doc/administration/img/db_load_balancing_postgres_stats.png b/doc/administration/img/db_load_balancing_postgres_stats.png deleted file mode 100644 index 8b311616e7b600ed5191a13821faf25eaafbdf97..0000000000000000000000000000000000000000 Binary files a/doc/administration/img/db_load_balancing_postgres_stats.png and /dev/null differ diff --git a/doc/administration/index.md b/doc/administration/index.md index 53a3c305aab471d5a528e62fb20c28397f94105b..ef9c56c1615dbf652e6bb716e1ea9d08352cae59 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -31,8 +31,6 @@ Learn how to install, configure, update, and maintain your GitLab instance. ### Installing GitLab - [Install](../install/index.md): Requirements, directory structures, and installation methods. - - [Database load balancing](database_load_balancing.md): Distribute database queries among multiple database servers. - - [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only). - [Reference architectures](reference_architectures/index.md): Add additional resources to support more users. - [Installing GitLab on Amazon Web Services (AWS)](../install/aws/index.md): Set up GitLab on Amazon AWS. - [Geo](geo/index.md): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. @@ -79,6 +77,8 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Enabling and disabling features flags](feature_flags.md): how to enable and disable GitLab features deployed behind feature flags. - [Application settings cache expiry interval](application_settings_cache.md) +- [Database Load Balancing](postgresql/database_load_balancing.md): Distribute database queries among multiple database servers. +- [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only). #### Customizing GitLab appearance diff --git a/doc/administration/logs.md b/doc/administration/logs.md index bf74a96a627d7e9685ffa125d4e8a19a403341fd..c9b1253f04151034b274b1d967411611a47e2fbb 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -830,7 +830,7 @@ are generated in a location based on your installation method: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/15442) in GitLab 12.3. -Contains details of GitLab [Database Load Balancing](database_load_balancing.md). +Contains details of GitLab [Database Load Balancing](postgresql/database_load_balancing.md). Depending on your installation method, this file is located at: - Omnibus GitLab: `/var/log/gitlab/gitlab-rails/database_load_balancing.log` diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index d798feb71a9f7079406db30d49b1ce69a2272347..14a560223f99119e442a4362e67c2adb46ec5af7 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -26,11 +26,10 @@ From left to right, the performance bar displays: details for each query: - **In a transaction**: shows up below the query if it was executed in the context of a transaction - - **Role**: shows up when [database load - balancing](../../database_load_balancing.md) is enabled. It shows - which server role was used for the query. "Primary" means that the query - was sent to the read/write primary server. "Replica" means it was sent - to a read-only replica. + - **Role**: shows up when [Database Load Balancing](../../postgresql/database_load_balancing.md) + is enabled. It shows which server role was used for the query. + "Primary" means that the query was sent to the read/write primary server. + "Replica" means it was sent to a read-only replica. - **Config name**: shows up only when the `GITLAB_MULTIPLE_DATABASE_METRICS` environment variable is set. This is used to distinguish between different databases configured for different diff --git a/doc/administration/postgresql/database_load_balancing.md b/doc/administration/postgresql/database_load_balancing.md new file mode 100644 index 0000000000000000000000000000000000000000..8ddb8beab6a456e4a6447ae1874834938c87cbbb --- /dev/null +++ b/doc/administration/postgresql/database_load_balancing.md @@ -0,0 +1,234 @@ +--- +stage: Enablement +group: Database +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Database Load Balancing **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1283) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60894) from GitLab Premium to GitLab Free in 14.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334494) for Sidekiq in GitLab 14.1. + +With Database Load Balancing, read-only queries can be distributed across +multiple PostgreSQL nodes to increase performance. + +This functionality is provided natively in GitLab Rails and Sidekiq where +they can be configured to balance their database read queries in a round-robin approach, +without any external dependencies: + +```plantuml +@startuml +card "**Internal Load Balancer**" as ilb #9370DB +skinparam linetype ortho + +together { + collections "**GitLab Rails** x3" as gitlab #32CD32 + collections "**Sidekiq** x4" as sidekiq #ff8dd1 +} + +collections "**Consul** x3" as consul #e76a9b + +card "Database" as database { + collections "**PGBouncer x3**\n//Consul//" as pgbouncer #4EA7FF + + card "**PostgreSQL** //Primary//\n//Patroni//\n//PgBouncer//\n//Consul//" as postgres_primary #4EA7FF + collections "**PostgreSQL** //Secondary// **x2**\n//Patroni//\n//PgBouncer//\n//Consul//" as postgres_secondary #4EA7FF + + pgbouncer -[#4EA7FF]-> postgres_primary + postgres_primary .[#4EA7FF]r-> postgres_secondary +} + +gitlab -[#32CD32]-> ilb +gitlab -[hidden]-> pgbouncer +gitlab .[#32CD32,norank]-> postgres_primary +gitlab .[#32CD32,norank]-> postgres_secondary + +sidekiq -[#ff8dd1]-> ilb +sidekiq -[hidden]-> pgbouncer +sidekiq .[#ff8dd1,norank]-> postgres_primary +sidekiq .[#ff8dd1,norank]-> postgres_secondary + +ilb -[#9370DB]-> pgbouncer + +consul -[#e76a9b]r-> pgbouncer +consul .[#e76a9b,norank]r-> postgres_primary +consul .[#e76a9b,norank]r-> postgres_secondary +@enduml +``` + +## Requirements to enable Database Load Balancing + +To enable Database Load Balancing, make sure that: + +- The HA Postgres setup has one or more secondary nodes replicating the primary. +- Each Postgres node is connected with the same credentials and on the same port. + +For Omnibus GitLab, you also need PgBouncer configured on each PostgreSQL node to pool +all load-balanced connections when [configuring a multi-node setup](replication_and_failover.md). + +## Configuring Database Load Balancing + +Database Load Balancing can be configured in one of two ways: + +- (Recommended) [Hosts](#hosts): a list of PostgreSQL hosts. +- [Service Discovery](#service-discovery): a DNS record that returns a list of PostgreSQL hosts. + +### Hosts + +To configure a list of hosts, add the `gitlab_rails['db_load_balancing']` setting into the +`gitlab.rb` file in the GitLab Rails / Sidekiq nodes for each environment you want to balance. + +For example, on an environment that has PostgreSQL running on the hosts `host1.example.com`, +`host2.example.com` and `host3.example.com` and reachable on the same port configured with +`gitlab_rails['db_port']`: + +1. On each GitLab Rails / Sidekiq node, edit `/etc/gitlab/gitlab.rb` and add the following line: + + ```ruby + gitlab_rails['db_load_balancing'] = { 'hosts' => ['host1.example.com', 'host2.example.com', `host3.example.com`] } + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + +### Service Discovery + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5883) in GitLab 11.0. + +Service discovery allows GitLab to automatically retrieve a list of PostgreSQL +hosts to use. It periodically +checks a DNS A record, using the IPs returned by this record as the addresses +for the secondaries. For service discovery to work, all you need is a DNS server +and an A record containing the IP addresses of your secondaries. + +When using Omnibus GitLab the provided [Consul](../consul.md) service works as +a DNS server and returns PostgreSQL addresses via the `postgresql-ha.service.consul` +record. For example: + +1. On each GitLab Rails / Sidekiq node, edit `/etc/gitlab/gitlab.rb` and add the following: + + ```ruby + gitlab_rails['db_load_balancing'] = { 'discover' => { + 'nameserver' => 'localhost' + 'record' => 'postgresql-ha.service.consul' + 'record_type' => 'A' + 'port' => '8600' + 'interval' => '60' + 'disconnect_timeout' => '120' + } + } + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +| Option | Description | Default | +|----------------------|---------------------------------------------------------------------------------------------------|-----------| +| `nameserver` | The nameserver to use for looking up the DNS record. | localhost | +| `record` | The record to look up. This option is required for service discovery to work. | | +| `record_type` | Optional record type to look up, this can be either A or SRV (GitLab 12.3 and later) | A | +| `port` | The port of the nameserver. | 8600 | +| `interval` | The minimum time in seconds between checking the DNS record. | 60 | +| `disconnect_timeout` | The time in seconds after which an old connection is closed, after the list of hosts was updated. | 120 | +| `use_tcp` | Lookup DNS resources using TCP instead of UDP | false | + +If `record_type` is set to `SRV`, then GitLab continues to use round-robin algorithm +and ignores the `weight` and `priority` in the record. Since SRV records usually +return hostnames instead of IPs, GitLab needs to look for the IPs of returned hostnames +in the additional section of the SRV response. If no IP is found for a hostname, GitLab +needs to query the configured `nameserver` for ANY record for each such hostname looking for A or AAAA +records, eventually dropping this hostname from rotation if it can't resolve its IP. + +The `interval` value specifies the _minimum_ time between checks. If the A +record has a TTL greater than this value, then service discovery honors said +TTL. For example, if the TTL of the A record is 90 seconds, then service +discovery waits at least 90 seconds before checking the A record again. + +When the list of hosts is updated, it might take a while for the old connections +to be terminated. The `disconnect_timeout` setting can be used to enforce an +upper limit on the time it takes to terminate all old database connections. + +### Handling Stale Reads **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3526) in GitLab 10.3. + +To prevent reading from an outdated secondary the load balancer checks if it +is in sync with the primary. If the data is recent enough, the +secondary is used, otherwise it is ignored. To reduce the overhead of +these checks we only perform them at certain intervals. + +There are three configuration options that influence this behavior: + +| Option | Description | Default | +|------------------------------|----------------------------------------------------------------------------------------------------------------|------------| +| `max_replication_difference` | The amount of data (in bytes) a secondary is allowed to lag behind when it hasn't replicated data for a while. | 8 MB | +| `max_replication_lag_time` | The maximum number of seconds a secondary is allowed to lag behind before we stop using it. | 60 seconds | +| `replica_check_interval` | The minimum number of seconds we have to wait before checking the status of a secondary. | 60 seconds | + +The defaults should be sufficient for most users. + +To configure these options with a hosts list, use the following example: + +```ruby +gitlab_rails['db_load_balancing'] = { + 'hosts' => ['host1.example.com', 'host2.example.com', `host3.example.com`] + 'max_replication_difference' => 16777216 # 16 MB + 'max_replication_lag_time' => 30 + 'replica_check_interval' => 30 +} +``` + +## Logging + +The load balancer logs various events in +[`database_load_balancing.log`](../logs.md#database_load_balancinglog), such as + +- When a host is marked as offline +- When a host comes back online +- When all secondaries are offline +- When a read is retried on a different host due to a query conflict + +The log is structured with each entry a JSON object containing at least: + +- An `event` field useful for filtering. +- A human-readable `message` field. +- Some event-specific metadata. For example, `db_host` +- Contextual information that is always logged. For example, `severity` and `time`. + +For example: + +```json +{"severity":"INFO","time":"2019-09-02T12:12:01.728Z","correlation_id":"abcdefg","event":"host_online","message":"Host came back online","db_host":"111.222.333.444","db_port":null,"tag":"rails.database_load_balancing","environment":"production","hostname":"web-example-1","fqdn":"gitlab.example.com","path":null,"params":null} +``` + +## Implementation Details + +### Balancing queries + +Read-only `SELECT` queries balance among all the given hosts. +Everything else (including transactions) executes on the primary. +Queries such as `SELECT ... FOR UPDATE` are also executed on the primary. + +### Prepared statements + +Prepared statements don't work well with load balancing and are disabled +automatically when load balancing is enabled. This shouldn't impact +response timings. + +### Primary sticking + +After a write has been performed, GitLab sticks to using the primary for a +certain period of time, scoped to the user that performed the write. GitLab +reverts back to using secondaries when they have either caught up, or after 30 +seconds. + +### Failover handling + +In the event of a failover or an unresponsive database, the load balancer +tries to use the next available host. If no secondaries are available the +operation is performed on the primary instead. + +If a connection error occurs while writing data, the +operation retries up to 3 times using an exponential back-off. + +When using load balancing, you should be able to safely restart a database server +without it immediately leading to errors being presented to the users. diff --git a/doc/development/licensed_feature_availability.md b/doc/development/licensed_feature_availability.md index 10e6d717a1814480e190cac3babbbbe2910502a9..629d0027ffe2d00a7337f08dc68c508005aea921 100644 --- a/doc/development/licensed_feature_availability.md +++ b/doc/development/licensed_feature_availability.md @@ -29,7 +29,7 @@ project.feature_available?(:feature_symbol) ## Restricting global features (instance) However, for features such as [Geo](../administration/geo/index.md) and -[Load balancing](../administration/database_load_balancing.md), which cannot be restricted +[Database Load Balancing](../administration/postgresql/database_load_balancing.md), which cannot be restricted to only a subset of projects or namespaces, the check is made directly in the instance license. diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index cbf3c09b28b336a5285a8ce8b0d2b1af92066ea7..74b1d5680b0966f7f65107a988700430abb2c636 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -160,10 +160,10 @@ query. This in turn makes it much harder for this code to overload a database. ## Use read replicas when possible -In a DB cluster we have many read replicas and one primary. A classic use of scaling the DB is to have read-only actions be performed by the replicas. We use [load balancing](../administration/database_load_balancing.md) to distribute this load. This allows for the replicas to grow as the pressure on the DB grows. +In a DB cluster we have many read replicas and one primary. A classic use of scaling the DB is to have read-only actions be performed by the replicas. We use [load balancing](../administration/postgresql/database_load_balancing.md) to distribute this load. This allows for the replicas to grow as the pressure on the DB grows. By default, queries use read-only replicas, but due to -[primary sticking](../administration/database_load_balancing.md#primary-sticking), GitLab uses the +[primary sticking](../administration/postgresql/database_load_balancing.md#primary-sticking), GitLab uses the primary for some time and reverts to secondaries after they have either caught up or after 30 seconds. Doing this can lead to a considerable amount of unnecessary load on the primary. To prevent switching to the primary [merge request 56849](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/56849) introduced the @@ -187,7 +187,7 @@ Internally, our database load balancer classifies the queries based on their mai - Sidekiq background jobs After the above queries are executed, GitLab -[sticks to the primary](../administration/database_load_balancing.md#primary-sticking). +[sticks to the primary](../administration/postgresql/database_load_balancing.md#primary-sticking). To make the inside queries prefer using the replicas, [merge request 59086](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59086) introduced `fallback_to_replicas_for_ambiguous_queries`. This MR is also an example of how we redirected a diff --git a/doc/development/scalability.md b/doc/development/scalability.md index fdae66b7abc8002d76b03802105976bafd5e4526..7a3f3c7097d5a86974a01fca43d4ba1b0e4e7e09 100644 --- a/doc/development/scalability.md +++ b/doc/development/scalability.md @@ -123,8 +123,7 @@ the read replicas. [Omnibus ships with Patroni](../administration/postgresql/rep #### Load-balancing -GitLab EE has [application support for load balancing using read -replicas](../administration/database_load_balancing.md). This load balancer does +GitLab EE has [application support for load balancing using read replicas](../administration/postgresql/database_load_balancing.md). This load balancer does some actions that aren't traditionally available in standard load balancers. For example, the application considers a replica only if its replication lag is low (for example, WAL data behind by less than 100 MB). diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index e28a328888d1d7e07d8f1929aa3f8d10678c5575..e6d865d56e9ef149746c75b9676c42885dd7de56 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -574,7 +574,7 @@ of reading a stale record is non-zero due to replicas potentially lagging behind When the number of jobs that rely on the database increases, ensuring immediate data consistency can put unsustainable load on the primary database server. We therefore added the ability to use -[database load balancing for Sidekiq workers](../administration/database_load_balancing.md#load-balancing-for-sidekiq). +[Database Load Balancing for Sidekiq workers](../administration/postgresql/database_load_balancing.md). By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas under several strategies outlined below.