Merge branch 'registry_garbage_collect' into 'master'

Add registry garbage collect command.

Fixes gitlab-org/omnibus-gitlab#1409

See merge request !987

CHANGELOG.md

@@ -15,7 +15,7 @@ omnibus-gitlab repository.
 - Allow configuring Rack Attack endpoints (Dmitry Ivanov) 7aee63
 - Bundle jemalloc and allow optional enable 1381ba
 - Use single db_host when multi postgresql::listen_adresses
-
+- Add gitlab-ctl registry-garbage-collect command
 
 8.11.4
 

doc/maintenance/README.md

@@ -109,3 +109,81 @@ could not change directory to "/root"
 ```
 
 This is normal behavior and it can be ignored.
+
+### Container registry garbage collection
+
+Container registry can use considerable amounts of disk space. To clear up
+some unused layers, registry includes a garbage collect command.
+
+There are a couple of considerations you need to note before running the
+built in command:
+
+* The built in command will stop the registry before it starts garbage collect
+* The garbage collect command takes some time to complete, depending on the
+amount of data that exists
+* If you changed the location of registry configuration file, you will need to
+specify the path
+* After the garbage collect is done, registry should start up automatically
+
+**Warning** The command below will cause Container registry downtime.
+
+If you did not change the default location of the configuration file, to do
+garbage collection:
+
+```
+sudo gitlab-ctl registry-garbage-collect
+```
+
+This command will take some time to complete, depending on the amount of
+layers you have stored.
+
+If you changed the location of the Container registry config.yml:
+
+```
+sudo gitlab-ctl registry-garbage-collect /path/to/config.yml
+```
+
+#### Doing garbage collect without downtime
+
+You can do a garbage collect without stopping the Container registry by setting
+it into a read only mode. During this time, you will be able to pull from
+the Container registry but you will not be able to push.
+
+These are the steps you need to take in order to complete the garbage collection:
+
+In `/etc/gitlab/gitlab.rb` specify the read only mode:
+
+```ruby
+registry['storage'] = {
+  'maintenance' => {
+    'readonly' => {
+      'enabled' => 'true'
+    }
+  }
+}
+```
+
+Save and run `sudo gitlab-ctl reconfigure`. This will set the Container registry
+into the read only mode.
+
+Next, trigger the garbage collect command:
+
+```
+sudo /opt/gitlab/embedded/bin/registry garbage-collect /var/opt/gitlab/registry/config.yml
+```
+
+This will start the garbage collection. The command will take some time to complete.
+
+Once done, in `/etc/gitlab/gitlab.rb` change the configuration to:
+
+```ruby
+registry['storage'] = {
+  'maintenance' => {
+    'readonly' => {
+      'enabled' => 'false'
+    }
+  }
+}
+```
+
+and run `sudo gitlab-ctl reconfigure`.

files/gitlab-ctl-commands/registry_garbage_collect.rb

@@ -0,0 +1,45 @@
+#
+# Copyright:: Copyright (c) 2016 GitLab Inc.
+# License:: Apache License, Version 2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+add_command_under_category 'registry-garbage-collect', 'container-registry',  'Run Container Registry garbage collection.', 2 do |cmd_name, path|
+  service_name = "registry"
+
+  unless service_enabled?(service_name)
+    log "Container registry is not enabled, exiting..."
+    exit! 1
+  end
+
+  config_file_path = path || '/var/opt/gitlab/registry/config.yml'
+
+  unless File.exists?(config_file_path)
+    log "Didn't find #{config_file_path}, please supply the path to registry config.yml file, eg: gitlab-ctl registry-garbage-collect /path/to/config.yml"
+    exit! 1
+  end
+
+  run_sv_command_for_service('stop', service_name)
+  log "Running garbage-collect using configuration from #{config_file_path}, this might take a while...\n"
+  status = run_command("/opt/gitlab/embedded/bin/registry garbage-collect #{config_file_path}")
+
+  if status.exitstatus == 0
+    run_sv_command_for_service('start', service_name)
+    exit! 0
+  else
+    log "\nFailed to run garbage-collect command, starting registry service."
+    run_sv_command_for_service('start', service_name)
+    exit! 1
+  end
+end