Commit Graph

5117 Commits

Author SHA1 Message Date
Rutger Wessels 4b58cb8c6f Override default timeout when running decomposition migration
The default timeout is 5 minutes. But the migration can take
a lot of time. So we set it to 1 day.

Changelog: fixed
2024-05-17 17:59:19 +00:00
Balasankar 'Balu' C 645fc7f4df Merge branch '8200-remove-grafana-deprecation-messages' into 'master'
Remove Grafana attribute and deprecation messages

Closes #8200

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7603

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Clemens Beck <cbeck@gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Robert Marshall <rmarshall@gitlab.com>
2024-05-17 08:07:55 +00:00
Robert Marshall 2bb9ea5d31 Remove Grafana attribute and deprecation messages
- Grafana was removed in GitLab 16.3. This removes the deprecation
  messages and the attribute that was left behind to enable those
  messages to fire without error.

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8200

Changelog: removed
2024-05-17 08:07:55 +00:00
Balasankar 'Balu' C 9cc039d44e Merge branch 'use-progress-message-for-pg-version-check-step' into 'master'
Use progress message on newer PG version check

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7596

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Robert Marshall <rmarshall@gitlab.com>
2024-05-17 08:07:12 +00:00
Robert Marshall 3e7d27b43b Use progress message on newer PG version check
- When `pg-upgrade` checks if a newer version of PostgreSQL is available
  to use, use the progress_message feature to align with all the other
  checks performed.

Related https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8247
2024-05-17 08:07:12 +00:00
Balasankar 'Balu' C c8d2602e5f Add dedicated Redis sentinel master settings for GitLab Rails
- Ensure legacy redis sentinel master configuration for rails
  gets parsed and loaded into resque with the new redis helper.

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8540

Signed-off-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-05-16 18:38:30 +00:00
Balasankar 'Balu' C 82bcde6d90 Migrate Gitlab Rails to NewRedisHelper logic
- Moves the Gitlab Rails library to use the new
  redis helper logic code path.

Signed-off-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-05-15 19:08:58 +00:00
Robert Marshall 74817b6f57 Merge branch 'skip-storage-uniqueness-if-gitaly-not-running' into 'master'
Skip Gitaly storages uniqueness check if Gitaly is not enabled

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7600

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Igor <iwiedler@gitlab.com>
Approved-by: Toon Claes <toon@gitlab.com>
Approved-by: Alessio Caiazza <acaiazza@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-05-10 14:51:38 +00:00
Balasankar 'Balu' C 474076cb89 Skip Gitaly storages uniqueness check if Gitaly is not enabled
- If a node has legacy `git-data` storages configured but no
  longer runs the gitaly service, skip the unique storage path
  validation check.

Signed-off-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-05-10 14:51:37 +00:00
Balasankar 'Balu' C 28ac725403 Merge branch 'sh-fix-unix-redis-with-password' into 'master'
redis: Fix password auth with UNIX domain sockets

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7573

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Ian Baum <ibaum@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Reviewed-by: Gabriel Mazetto <gabriel@gitlab.com>
Reviewed-by: Ian Baum <ibaum@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
2024-05-10 10:16:49 +00:00
Robert Marshall d0eba1b5ae Merge branch 'wc/no-dup-gitaly-storages' into 'master'
Prevent Gitaly storages from using the same path

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7564

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Gabriel Mazetto <gabriel@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Reviewed-by: Gabriel Mazetto <gabriel@gitlab.com>
Reviewed-by: Will Chandler <wchandler@gitlab.com>
Reviewed-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: Will Chandler <wchandler@gitlab.com>
2024-05-09 22:39:30 +00:00
Will Chandler 57396c7aba Prevent Gitaly storages from using the same path
Gitaly is making a breaking change with v17.0 to prevent multiple
storages from sharing the same local path. This is being done as part of
the work to add a write-ahead log to Gitaly, see
https://gitlab.com/gitlab-org/gitaly/-/issues/5598 for further details.

Validate that Gitaly's config does not have more than one storage using
the same path, dereferencing any symlinks.

Changelog: changed
2024-05-09 22:39:29 +00:00
Robert Marshall 6dd718199d Merge branch 'add-reconfigure-documentation' into 'master'
Add documentation on what happens during gitlab-ctl reconfigure

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7213

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Reviewed-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Reviewed-by: Robert Marshall <rmarshall@gitlab.com>
Reviewed-by: Evan Read <eread@gitlab.com>
Co-authored-by: Balasankar "Balu" C <balasankar@gitlab.com>
Co-authored-by: Evan Read <eread@gitlab.com>
2024-05-09 16:16:42 +00:00
Balasankar 'Balu' C 5baed11c98 Improve reconfigure process documentation
- Adds much more detail around the process that happens
  when gitlab-ctl reconfigure executes.
- Adds notes to the recipes to clarify what happens in
  the context of a reconfigure run.

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>
2024-05-09 16:16:41 +00:00
João Cunha e320a46820
Bring EOF back to upgrade_check file 2024-05-08 17:08:07 +02:00
Balasankar 'Balu' C 8bf2c5160c Merge branch 'upgrade-stop-16-11' into 'master'
Enforce upgrade stop at 16.11

Closes #8518

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7575

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Clemens Beck <cbeck@gitlab.com>
2024-05-08 14:53:29 +00:00
GitLab Release Tools Bot 8b6b7af263 Update upgrade stop to 16.7
[ci skip]
2024-05-07 16:28:05 +00:00
GitLab Release Tools Bot aeb111224d Update upgrade stop to 16.11
[ci skip]
2024-05-07 16:15:38 +00:00
Balasankar 'Balu' C 82b342458b Merge branch '8341-remove-pg-13' into 'master'
Remove PostgreSQL 13

Closes #8341

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7546

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Ryan Egesdahl <regesdahl@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Reviewed-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Reviewed-by: Ryan Egesdahl <regesdahl@gitlab.com>
Co-authored-by: Clemens Beck <cbeck@gitlab.com>
2024-05-07 05:24:29 +00:00
Clemens Beck 92f7cf5b6c Remove PostgreSQL 13
* Remove PostgreSQL 13 builds.
* Remove unused PostgreSQL 12 build definition.
* Move PostgreSQL 14 from 'new' to 'default' build definition.
* Update minimum PostgreSQL version notice.

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8341

Changelog: removed
2024-05-07 05:24:28 +00:00
Gregorius Marco cfa756435e Remove deprecated min_concurrency and max_concurrency for Sidekiq
- Eliminates code related to `min_concurrency` and
  `max_concurrency` options in sidekiq.

Related https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/3422

Changelog: removed
2024-05-04 00:39:29 +00:00
Clemens Beck 0c8e92a936
Enforce upgrade stop at 16.11
Before upgrading to GitLab/Omnibus 17.0, one must upgrade to
the GitLab/Omnibus 16.11.

Upgrade path documentation:
  https://docs.gitlab.com/ee/update/#upgrade-paths

Relates https://gitlab.com/gitlab-org/gitlab/-/issues/440234

Changelog: changed
2024-04-29 15:09:31 +02:00
Stan Hu 656fb39a8c
redis: Fix password auth with UNIX domain sockets
Previously if a Redis instance listened on a UNIX socket but a
password were set, GitLab Rails would not be able to authenticate.
This occurred because the UNIX URL doesn't contain a password.

Both Ruby and Go Redis clients support URLs in the form:

unix://<user>:<password>@</path/to/redis.sock>?db=<db_number>

Relates to work started in
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/2194

Changelog: fixed
2024-04-27 09:45:46 -07:00
Stan Hu 1c579dae4d
Fix reconfigure failure if Redis node has Rails Sentinel config
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7500
introduced a regression when a node with a managed Redis node has
Sentinels configured in `gitlab_rails` section. Previously the
`redis['default_host']` would only be set if
`gitlab_rails['redis_sentinels']` were empty.

Relates to
https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8465#note_1879691545

Changelog: fixed
2024-04-24 20:53:36 -07:00
Balasankar 'Balu' C ef9331b689 Merge branch 'mg-remove-queue-selector-and-negate-options' into 'master'
Remove queue_selector and negate options from Sidekiq

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7540

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Gregorius Marco <gmarco@gitlab.com>
2024-04-24 03:59:55 +00:00
Gregorius Marco cae9ce603e Remove queue_selector and negate options from Sidekiq
https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/2220

Changelog: removed
2024-04-24 03:59:55 +00:00
Ian Baum ba9ca699aa Merge branch 'ngala/namespace-in-path-auth-redirect-uri' into 'master'
Update default pages auth-redirect-uri when namespace-in-path is enabled

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7548

Merged-by: Ian Baum <ibaum@gitlab.com>
Approved-by: Vladimir Shushlin <vshushlin@gitlab.com>
Approved-by: Mitchell Nielsen <mnielsen@gitlab.com>
Approved-by: Ian Baum <ibaum@gitlab.com>
Co-authored-by: ngala <ngala@gitlab.com>
2024-04-23 14:46:46 +00:00
Stan Hu 39fa902e77
Fix missing arguments when PostgreSQL upgrade times out
This is a small regression from
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7247.
When the PostgreSQL upgrade times out, previously the script
would error out with `wrong number of arguments (given 0, expected 1)
(ArgumentError)`.

Changelog: fixed
2024-04-19 23:15:08 -07:00
ngala 1f597136de Update default pages auth-redirect-uri when namespace-in-path is enabled
Related: https://gitlab.com/gitlab-org/gitlab/-/issues/452459+

Changelog: fixed
2024-04-16 14:52:14 +05:30
Robert Marshall 8ea9c2a973 Merge branch 'ngala/pages_namespace_in_path_custom_auth_redirect_uri' into 'master'
Support custom auth_redirect_uri when namespace_in_path is enabled

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7516

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Mitchell Nielsen <mnielsen@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: ngala <ngala@gitlab.com>
2024-04-15 23:29:41 +00:00
Naman Jagdish Gala 5bc1b0d5ad Support custom auth_redirect_uri when namespace_in_path is enabled
Related: https://gitlab.com/gitlab-org/gitlab/-/issues/452459

Changelog: fixed
2024-04-15 23:29:41 +00:00
Andrew Patterson f96ce284b5 Merge branch 'avoid-indefined-local-error' into 'master'
Avoid "undefined local" error in pg-upgrade.rb#analyze_cluster

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7532

Merged-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Ian Baum <ibaum@gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Co-authored-by: Katrin Leinweber (GTLB) <kleinweber@gitlab.com>
2024-04-15 19:43:54 +00:00
Robert Marshall 9f89bce2e7 Merge branch 'enable-kas-in-fips-mode' into 'master'
Enable KAS in FIPS mode

Closes gitlab-org/build/CNG#1948

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7528

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Vishal Patel <vpatel@gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: Taka Nishida <tnishida@gitlab.com>
2024-04-15 19:41:06 +00:00
Taka Nishida 10d69a4d34 Enable KAS in FIPS mode
- Removes code that blocked KAS service when GitLab is
  deployed in FIPS mode

Related https://gitlab.com/gitlab-org/gitlab/-/issues/375327

Changelog: changed
2024-04-15 19:41:05 +00:00
Balasankar 'Balu' C afccdd4968 Merge branch 'sh-allow-redis-multiple-bind' into 'master'
Accept multiple bind addresses in Redis config

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7500

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Hossein Pursultani <hpursultani@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>
2024-04-15 05:28:38 +00:00
Katrin Leinweber (GTLB) 0eaa7e61d4 Avoid "undefined local" error (follow-up to 0faf786f)
Changelog: fixed
2024-04-11 15:13:32 +02:00
Robert Marshall 57d2bf8538 Merge branch 'add-grpc-level-config-for-kas' into 'master'
Support optional grpc log level config for KAS

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7518

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Gabriel Mazetto <gabriel@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: Taka Nishida <tnishida@gitlab.com>
2024-04-10 21:20:33 +00:00
Taka Nishida eacda21e30 Support optional grpc log level config for KAS
Adds the gitlab_kas['grpc_log_level'] config option.

Changelog: added
2024-04-10 21:20:33 +00:00
Sylvester Chin f185f8373c Allow routing rules to contain shard information
Changelog: changed
2024-04-09 17:25:41 +00:00
Robert Marshall 611ed6b5ff Merge branch 'support-setting-env-var' into 'master'
Support TLS for kas->kas communication for KAS

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7453

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Timo Furrer <tfurrer@gitlab.com>
Approved-by: João Alexandre Cunha <j.a.cunha@gmail.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: Taka Nishida <tnishida@gitlab.com>
2024-04-08 20:01:00 +00:00
Taka Nishida b55ca1fb9f Support TLS for kas->kas communication in Omnibus
Closes https://gitlab.com/gitlab-org/gitlab/-/issues/375327

Changelog: added
2024-04-08 20:01:00 +00:00
Javiera Tapia e19a65e32e Unified Backups: Add gitlab-backup-cli to Omnibus
- Adds wrapper around `gitlab-backup-cli` command
- Adds recipe and template for `gitlab-backup-cli` yaml configuration

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8273

Changelog: added
2024-04-03 02:49:59 +00:00
Stan Hu faca982898
Accept multiple bind addresses in Redis config
As specified in https://redis.io/docs/management/config-file/, Redis
can bind to multiple addresses with a space-separated field. Previously
attempting to do this without setting `gitlab_rails['redis_host']`
would fail because a URI could not be built with a space in the
hostname.

This commit now splits the string with the space and picks the first
address as the default Redis host.

Relates to https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8465

Changelog: added
2024-04-01 16:37:35 -07:00
Robert Marshall 3f2b0e7618 Merge branch 'redis-version-information-new-redis-helper' into 'master'
Move Redis server information to NewRedisHelper

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7465

Merged-by: Robert Marshall <rmarshall@gitlab.com>
Approved-by: Dustin Collins <714871-dustinmm80@users.noreply.gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Robert Marshall <rmarshall@gitlab.com>
Co-authored-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-04-01 18:32:51 +00:00
Balasankar 'Balu' C be58ec87c6 Move Redis server information to NewRedisHelper
- Continue conversion to the new style redis helper with
  refactors against the redis server functionality methods

Signed-off-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-04-01 18:32:51 +00:00
Balasankar 'Balu' C fca4d4f229 Fix deprecation warnings for root configurations
Do not warn if nothing is set and the default empty
hash is returned for some configurations.

Signed-off-by: Balasankar 'Balu' C <balasankar@gitlab.com>
2024-03-27 18:00:33 +00:00
Balasankar 'Balu' C aee82f523b Merge branch 'create-git-data-dirs-when-gitlab-rails-is-disabled' into 'master'
Create git_data_dirs even if gitlab_rails is disabled

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/7459

Merged-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Approved-by: Jason Plum <jplum@gitlab.com>
Approved-by: Andrew Patterson <apatterson@gitlab.com>
Approved-by: Balasankar 'Balu' C <balasankar@gitlab.com>
Co-authored-by: Ahmad Sherif <ahmad@gitlab.com>
2024-03-25 09:00:17 +00:00
Clemens Beck a340a0c3ba Auto upgrade single node installs to PostgreSQL 14
Postgres 14 is going to become our minimum supported version in
GitLab 17. GitLab 16.11 turns on auto-upgrade for single node installs.

Documentation change: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/147402

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8340

Changelog: changed
2024-03-25 08:55:22 +00:00
Robert Marshall 9ef2e18687 Enable easy SELinux context fixes
- Adds a command to re-apply SELinux file context rules based on the
  current GitLab provided SELinux policy.
- Documents the existence of this new command.
- Moves semanage tests into SELinuxHelper rspec and extends coverage for
  changes made to support the new command.

Related https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7257

Closes https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7452

Changelog: added
Signed-off-by: Robert Marshall <rmarshall@gitlab.com>
2024-03-19 04:50:52 +00:00
Ahmad Sherif 4beb352443 Create git_data_dirs even if gitlab_rails is disabled
Currently, `gitlab_rails` service needs to be enabled in order for
`gitaly` service to function properly. Specifically, `gitlab_rails`
creates `git_data_dirs` (e.g. `/var/opt/gitlab/git-data/repositories`)
which is needed for Gitaly to start-up. If only the `gitaly` service is
enabled, then Gitaly won't start due to config validation error (i.e.
storage directories have to be present).

This MR makes the creation of these directories the responsibility of
the `gitaly` cookbook. The `gitlab` cookbook still creates the
directories as backward-compatibility measure, as I'm not sure if these
directories are needed outside of Gitaly.

This is needed as part of
https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/issues/24530.

Changelog: fixed
2024-03-18 13:45:33 +01:00