Turn on postgresql SSL by default

doc/settings/database.md

@@ -19,20 +19,31 @@ If you are planning to use MySQL/MariaDB, make sure to read the [introductory
 paragraph](#using-a-mysql-database-management-server-enterprise-edition-only)
 before proceeding, as it contains some useful information.
 
-## Enabling SSL
+## Configuring SSL
 
-To enable SSL, you first need to have a number of files:
+Omnibus automatically enables SSL on the PostgreSQL server, but it will accept
+both encrypted and unencrypted connections by default. Enforcing SSL requires
+using the `hostssl` configuration in `pg_hba.conf`. See
+https://www.postgresql.org/docs/9.6/static/auth-pg-hba-conf.html
+for more details.
+
+SSL support depends on a number of files:
 
 1. The public SSL certificate for the database (`server.crt`).
 2. The corresponding private key for the SSL certificate (`server.key`).
-3. Optional: A root certificate bundle that validates the server's certificate
+3. A root certificate bundle that validates the server's certificate
 (`root.crt`). By default, Omnibus GitLab will use the embedded certificate
-bundle in `/opt/gitlab/embedded/ssl/certs/cacert.pem`.
+bundle in `/opt/gitlab/embedded/ssl/certs/cacert.pem`. This is not required for
+self-signed certificates
+
+A self-signed certificate and private key will be automatically generated for
+use. If you'd prefer to use a CA-signed certificate, follow the steps below.
 
 Note that the location of these files can be configurable, but the private key
-MUST be readable by the `gitlab-psql` user. Note that private keys stored in
-`/etc/gitlab/ssl` currently cannot be read by this user, so the key may need
-to be copied to another location and assigned the proper permissions.
+MUST be readable by the `gitlab-psql` user. Omnibus will automatically manage
+the permissions of the files for you, but you *must* ensure that the
+`gitlab-psql` can access the directory the files are placed in, if the paths
+are customized.
 
 For more details, see the [PostgreSQL documentation](https://www.postgresql.org/docs/9.6/static/ssl-tcp.html).
 
@@ -47,34 +58,40 @@ With these files in hand, enable SSL:
 1. Edit `/etc/gitlab/gitlab.rb`:
 
     ```ruby
-    postgresql['ssl'] = 'on'
+    postgresql['ssl_cert_file'] = '/custom/path/to/server.crt'
+    postgresql['ssl_key_file'] = '/custom/path/to/server.key'
+    postgresql['ssl_ca_file'] = '/custom/path/to/bundle.pem'
+    postgresql['internal_certificate'] = "-----BEGIN CERTIFICATE-----
+    ...base64-encoded certificate...
+    -----END CERTIFICATE-----
+    "
+    postgresql['internal_key'] = "-----BEGIN RSA PRIVATE KEY-----
+    ...base64-encoded private key...
+    -----END RSA PRIVATE KEY-----
+    "
     ```
 
-    Note that this does NOT enforce SSL connections on the server side. Enforcing SSL
-    requires using the `hostssl` configuration in `pg_hba.conf`. See https://www.postgresql.org/docs/9.6/static/auth-pg-hba-conf.html
-    for more details.
+    Relative paths will be rooted from the PostgreSQL data directory
+    (`/var/opt/gitlab/postgresql/data` by default).
 
-1. Optional: Customize the location of the required SSL files in `/etc/gitlab/gitlab.rb`. For example:
+1. [Reconfigure GitLab][] to apply the configuration changes.
 
-    ```ruby
-    postgresql['ssl_cert_file'] = 'server.crt'
-    postgresql['ssl_key_file'] = 'server.key'
-    postgresql['ssl_ca_file'] = '/opt/gitlab/embedded/ssl/certs/cacert.pem'
-    ```
-
-    Using a relative path will cause PostgreSQL to look inside its data
-    directory (`/var/opt/gitlab/postgresql/data` by default).
-
-1. Optional: Copy the required SSL files into the PostgreSQL data directory. For example:
+1. Restart PostgreSQL for the changes to take effect:
 
     ```sh
-    cp server.crt server.key /var/opt/gitlab/postgresql/data
-    cd /var/opt/gitlab/postgresql/data
-    chown gitlab-psql:gitlab-psql server.crt server.key
+    gitlab-ctl restart postgresql
     ```
 
-    Note that the PostgreSQL user (by default `gitlab-psql`) must have read access to these files,
-    or PostgreSQL will fail to start.
+   If PostgreSQL fails to start, check the logs
+   (e.g. `/var/log/gitlab/postgresql/current`) for more details.
+
+### Disabling SSL
+
+1. Add the following to `/etc/gitlab/gitlab.rb`:
+
+```ruby
+postgresql['ssl'] = 'off'
+```
 
 1. [Reconfigure GitLab][] to apply the configuration changes.
 

files/gitlab-config-template/gitlab.rb.template

@@ -666,7 +666,7 @@ external_url 'GENERATED_EXTERNAL_URL'
 
 ### SSL settings
 # See https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html#GUC-SSL-CERT-FILE for more details
-# postgresql['ssl'] = 'off'
+# postgresql['ssl'] = 'on'
 # postgresql['ssl_ciphers'] = 'HIGH:MEDIUM:+3DES:!aNULL:!SSLv3:!TLSv1'
 # postgresql['ssl_cert_file'] = 'server.crt'
 # postgresql['ssl_key_file'] = 'server.key'

files/gitlab-cookbooks/gitlab/attributes/default.rb

@@ -373,7 +373,7 @@ default['gitlab']['postgresql']['max_connections'] = 200
 default['gitlab']['postgresql']['md5_auth_cidr_addresses'] = []
 default['gitlab']['postgresql']['trust_auth_cidr_addresses'] = []
 
-default['gitlab']['postgresql']['ssl'] = 'off'
+default['gitlab']['postgresql']['ssl'] = 'on'
 default['gitlab']['postgresql']['ssl_ciphers'] = 'HIGH:MEDIUM:+3DES:!aNULL:!SSLv3:!TLSv1'
 default['gitlab']['postgresql']['ssl_cert_file'] = 'server.crt'
 default['gitlab']['postgresql']['ssl_key_file'] = 'server.key'

files/gitlab-cookbooks/gitlab/libraries/postgresql.rb

@@ -23,6 +23,12 @@ module Postgresql
       parse_mattermost_postgresql_settings
     end
 
+    def parse_secrets
+      gitlab_postgresql_crt, gitlab_postgresql_key = generate_postgresql_keypair
+      Gitlab['postgresql']['internal_certificate'] ||= gitlab_postgresql_crt
+      Gitlab['postgresql']['internal_key'] ||= gitlab_postgresql_key
+    end
+
     def parse_postgresql_settings
       # If the user wants to run the internal Postgres service using an alternative
       # DB username, host or port, then those settings should also be applied to
@@ -97,5 +103,15 @@ module Postgresql
     def postgresql_managed?
       Services.enabled?('postgresql')
     end
+
+    def generate_postgresql_keypair
+      key, cert = SecretsHelper.generate_keypair(
+        bits: 4096,
+        subject: "/C=USA/O=GitLab/OU=Database/CN=PostgreSQL",
+        validity: 365 * 10 # ten years from now
+      )
+
+      [cert.to_pem, key.to_pem]
+    end
   end
 end

files/gitlab-cookbooks/gitlab/libraries/registry.rb

@@ -114,17 +114,11 @@ module Registry
     end
 
     def generate_registry_keypair
-      key = SecretsHelper.generate_rsa(4096)
-      subject = "/C=USA/O=GitLab/OU=Container/CN=Registry"
-
-      cert = OpenSSL::X509::Certificate.new
-      cert.subject = cert.issuer = OpenSSL::X509::Name.parse(subject)
-      cert.not_before = Time.now
-      cert.not_after = (DateTime.now + 365 * 10).to_time
-      cert.public_key = key.public_key
-      cert.serial = 0x0
-      cert.version = 2
-      cert.sign key, OpenSSL::Digest::SHA256.new
+      key, cert = SecretsHelper.generate_keypair(
+        bits: 4096,
+        subject: "/C=USA/O=GitLab/OU=Container/CN=Registry",
+        validity: 365 * 10 # ten years from now
+      )
 
       [cert.to_pem, key.to_pem]
     end

files/gitlab-cookbooks/gitlab/recipes/postgresql.rb

@@ -80,6 +80,29 @@ execute "/opt/gitlab/embedded/bin/initdb -D #{postgresql_data_dir} -E UTF8" do
   not_if { pg_helper.bootstrapped? }
 end
 
+
+##
+# Create SSL cert + key in the defined location. Paths are relative to postgresql_data_dir
+##
+ssl_cert_file = File.absolute_path(node['gitlab']['postgresql']['ssl_cert_file'], postgresql_data_dir)
+ssl_key_file = File.absolute_path(node['gitlab']['postgresql']['ssl_key_file'], postgresql_data_dir)
+
+file ssl_cert_file do
+  content node['gitlab']['postgresql']['internal_certificate']
+  owner postgresql_username
+  group postgresql_username
+  mode 0400
+  only_if { node['gitlab']['postgresql']['ssl'] == 'on' }
+end
+
+file ssl_key_file do
+  content node['gitlab']['postgresql']['internal_key']
+  owner postgresql_username
+  group postgresql_username
+  mode 0400
+  only_if { node['gitlab']['postgresql']['ssl'] == 'on' }
+end
+
 postgresql_config = File.join(postgresql_data_dir, "postgresql.conf")
 postgresql_runtime_config = File.join(postgresql_data_dir, 'runtime.conf')
 should_notify = omnibus_helper.should_notify?("postgresql")

files/gitlab-cookbooks/package/libraries/helpers/secrets_helper.rb

@@ -9,6 +9,26 @@ class SecretsHelper
     OpenSSL::PKey::RSA.new(bits)
   end
 
+  def self.generate_x509(subject:, validity:, key:)
+    cert = OpenSSL::X509::Certificate.new
+    cert.subject = cert.issuer = OpenSSL::X509::Name.parse(subject)
+    cert.not_before = Time.now
+    cert.not_after = (DateTime.now + validity).to_time
+    cert.public_key = key.public_key
+    cert.serial = 0x0
+    cert.version = 2
+    cert.sign(key, OpenSSL::Digest::SHA256.new)
+
+    cert
+  end
+
+  def self.generate_keypair(bits:, subject:, validity:)
+    key = generate_rsa(bits)
+    cert = generate_x509(subject: subject, validity: validity, key: key)
+
+    [key, cert]
+  end
+
   def self.read_gitlab_secrets
     existing_secrets ||= {}
 
@@ -23,7 +43,7 @@ class SecretsHelper
           Gitlab[k][pk] ||= p
         end
       else
-        warn("Ignoring section #{k} in /etc/gitlab/giltab-secrets.json, does not exist in gitlab.rb")
+        warn("Ignoring section #{k} in /etc/gitlab/gitlab-secrets.json, does not exist in gitlab.rb")
       end
     end
   end
@@ -52,6 +72,10 @@ class SecretsHelper
         'email_invite_salt' => Gitlab['mattermost']['email_invite_salt'],
         'file_public_link_salt' => Gitlab['mattermost']['file_public_link_salt'],
         'sql_at_rest_encrypt_key' => Gitlab['mattermost']['sql_at_rest_encrypt_key']
+      },
+      'postgresql' => {
+        'internal_certificate' => Gitlab['postgresql']['internal_certificate'],
+        'internal_key' => Gitlab['postgresql']['internal_key']
       }
     }
 

files/gitlab-ctl-commands-ee/replicate_geo_database.rb

@@ -21,7 +21,7 @@ class GeoReplicationCommand
       slot_name: nil,
       skip_replication_slot: false,
       backup_timeout: 1800,
-      sslmode: 'verify-full',
+      sslmode: 'verify-ca',
     }
 
     parse_options!

spec/chef/recipes/postgresql_spec.rb

@@ -2,7 +2,10 @@ require 'chef_helper'
 
 describe 'postgresql 9.2' do
   let(:chef_run) { ChefSpec::SoloRunner.converge('gitlab::default') }
-  let(:postgresql_conf) { '/var/opt/gitlab/postgresql/data/postgresql.conf' }
+  let(:postgresql_data_dir) { '/var/opt/gitlab/postgresql/data' }
+  let(:postgresql_ssl_cert) { File.join(postgresql_data_dir, 'server.crt') }
+  let(:postgresql_ssl_key) { File.join(postgresql_data_dir, 'server.key') }
+  let(:postgresql_conf) { File.join(postgresql_data_dir, 'postgresql.conf') }
   let(:runtime_conf) { '/var/opt/gitlab/postgresql/data/runtime.conf' }
 
   before do
@@ -67,20 +70,52 @@ describe 'postgresql 9.2' do
     end
 
     context 'sets SSL settings' do
-      it 'disables SSL by default' do
+      it 'enables SSL by default' do
         expect(chef_run.node['gitlab']['postgresql']['ssl'])
-          .to eq('off')
+          .to eq('on')
+
+        expect(chef_run).to render_file(
+          postgresql_conf
+        ).with_content(/ssl = on/)
+      end
+
+      it 'generates a self-signed certificate and key' do
+        stub_gitlab_rb(postgresql: { ssl_cert_file: 'certfile', ssl_key_file: 'keyfile' })
+
+        absolute_cert_path = File.join(postgresql_data_dir, 'certfile')
+        absolute_key_path = File.join(postgresql_data_dir, 'keyfile')
+
+        expect(chef_run).to create_file(absolute_cert_path).with(
+          user: 'gitlab-psql',
+          group: 'gitlab-psql',
+          mode: 0400
+        )
+
+        expect(chef_run).to create_file(absolute_key_path).with(
+          user: 'gitlab-psql',
+          group: 'gitlab-psql',
+          mode: 0400
+        )
+
+        expect(chef_run).to render_file(absolute_cert_path)
+          .with_content(/-----BEGIN CERTIFICATE-----/)
+        expect(chef_run).to render_file(absolute_key_path)
+          .with_content(/-----BEGIN RSA PRIVATE KEY-----/)
+      end
+
+      it 'disables SSL' do
+        stub_gitlab_rb(postgresql: { ssl: 'off' })
 
         expect(chef_run).to render_file(
           postgresql_conf
         ).with_content(/ssl = off/)
+
+        expect(chef_run).not_to render_file(postgresql_ssl_cert)
+        expect(chef_run).not_to render_file(postgresql_ssl_key)
       end
 
       it 'activates SSL' do
-        stub_gitlab_rb(postgresql: {
-                         ssl: 'on',
-                         ssl_crl_file: 'revoke.crl'
-                       })
+        stub_gitlab_rb(postgresql: { ssl_crl_file: 'revoke.crl' })
 
         expect(chef_run).to render_file(
           postgresql_conf
@@ -96,7 +131,7 @@ describe 'postgresql 9.2' do
         ).with_content(/ssl_key_file = 'server.key'/)
         expect(chef_run).to render_file(
           postgresql_conf
-        ).with_content(/ssl_ca_file = '\/opt\/gitlab\/embedded\/ssl\/certs\/cacert.pem'/)
+        ).with_content(%r{ssl_ca_file = '/opt/gitlab/embedded/ssl/certs/cacert.pem'})
         expect(chef_run).to render_file(
           postgresql_conf
         ).with_content(/ssl_crl_file = 'revoke.crl'/)