Merge branch 'relative_url' into 'master'

Relative url support for omnibus installations

This MR is partly based on !401 (see !401 for some discussions on implementation way)

TODOs:
 - [x] test if gitlab-shell works
 - [x] test it on centos
 - [x] implement relative url configuration via parsing of `external_url` instead of `gitlab_rails['gitlab_relative_url']`
 - [x] Docs, changelog

See merge request !590

CHANGELOG.md

@@ -3,6 +3,8 @@
 The latest version of this file can be found at the master branch of the
 omnibus-gitlab repository.
 
+- Add experimental support for relative url installations
+
 8.4.0
 
 - Add support for ecdsa and ed25519 keys to Docker image (Matthew Monaco) 3bfcb2617d240937fdb77d38900ee00f1ffbce02

config/templates/gitlab-rails/bundle_exec_wrapper.erb

@@ -17,7 +17,9 @@ fi
 
 cd <%= install_dir %>/embedded/service/gitlab-rails
 
-if [ "$(id -n -u)" = "${gitlab_user}" ] ; then
+if [ -n "$NO_PRIVILEGE_DROP" ]; then
+  privilege_drop=''
+elif [ "$(id -n -u)" = "${gitlab_user}" ] ; then
   # We are already running at the intended privilege; don't try to drop
   # privileges again because only root can do that (and we are apparently not
   # root!).

doc/README.md

@@ -17,6 +17,7 @@
 ## Configuring
 
 - [Configuring the external url](settings/configuration.md#configuring-the-external-url-for-gitlab)
+- [Configuring a relative URL for Gitlab (experimental)](settings/configuration.md#configuring-a-relative-url-for-gitlab)
 - [Storing git data in an alternative directory](settings/configuration.md#storing-git-data-in-an-alternative-directory)
 - [Changing the name of the git user group](settings/configuration.md#changing-the-name-of-the-git-user-group)
 - [Specify numeric user and group identifiers](settings/configuration.md#specify-numeric-user-and-group-identifiers)

doc/common_installation_problems/README.md

@@ -269,6 +269,16 @@ to serve evil JavaScript code to the visitors of your GitLab server.
 If you want to run GitLab with custom JavaScript or CSS code you are probably
 better off running GitLab from source, or building your own packages.
 
+If you really know what you are doing,
+you can execute `gitlab-rake assets:precompile` like this
+
+```shell
+sudo NO_PRIVILEGE_DROP=true USE_DB=false gitlab-rake assets:clean assets:precompile
+# user and path might be different if you changed the defaults of
+# user['username'], user['group'] and gitlab_rails['dir'] in gitlab.rb
+sudo chown -R git:git /var/opt/gitlab/gitlab-rails/tmp/cache
+```
+
 ### 'Short read or OOM loading DB' error
 
 Try cleaning the old redis session by following the [documentation here.](http://doc.gitlab.com/ce/operations/cleaning_up_redis_sessions.html)

doc/settings/configuration.md

@@ -19,6 +19,76 @@ external_url "http://gitlab.example.com"
 
 Run `sudo gitlab-ctl reconfigure` for the change to take effect.
 
+## Configuring a relative URL for Gitlab
+
+_**Note:** Relative URL support is **experimental** and was [introduced][590]
+in Omnibus GitLab 8.5._
+
+The omnibus-gitlab package is shipped with pre-compiled assets (CSS, JavaScript,
+etc.).
+
+In case you configure Omnibus with a relative URL, the assets will need to be
+recompiled. This is a task which consumes a lot of CPU and memory resources, so
+to avoid out-of-memory errors, you should have at least 2GB of RAM available on
+your system, while we recommend 4GB RAM and 4 or 8 CPU cores.
+
+### Enable relative URL in GitLab
+
+Follow the steps below to enable a relative URL in GitLab:
+
+1.  (Optional) If you run short on resources, you can temporarily free up some
+    memory by shutting down Unicorn and Sidekiq with the following command:
+
+    ```shell
+    sudo gitlab-ctl stop unicorn
+    sudo gitlab-ctl stop sidekiq
+    ```
+
+2.  Set the `external_url` in `/etc/gitlab/gitlab.rb`:
+
+    ```ruby
+    external_url "https://gitlab.example.com/gitlab"
+    ```
+
+    In this case, the relative URL under which GitLab will be served will be
+    `/gitlab`. Change it to your liking.
+
+3.  Reconfigure GitLab for the changes to take effect:
+
+    ```shell
+    sudo gitlab-ctl reconfigure
+    ```
+
+4.  Restart GitLab in case you shut down Unicorn and Sidekiq in the first step:
+
+    ```shell
+    sudo gitlab-ctl start
+    ```
+
+---
+
+If for some reason the asset compilation fails (i.e. the server runs out of memory),
+you can execute the task manually after you addressed the issue (i.e. add swap):
+
+```shell
+sudo NO_PRIVILEGE_DROP=true USE_DB=false gitlab-rake assets:clean assets:precompile
+# user and path might be different if you changed the defaults of
+# user['username'], user['group'] and gitlab_rails['dir'] in gitlab.rb
+sudo chown -R git:git /var/opt/gitlab/gitlab-rails/tmp/cache
+```
+
+### Disable relative URL in GitLab
+
+To disable the relative URL, follow the same steps as above and set up the
+`external_url` to a one that doesn't contain a relative path. You may need to
+restart Unicorn after the reconfigure task is done:
+
+```shell
+sudo gitlab-ctl restart unicorn
+```
+
+[590]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/590 "Merge request - Relative url support for omnibus installations"
+
 ## Loading external configuration file from non-root user
 
 Omnibus-gitlab package loads all configuration from `/etc/gitlab/gitlab.rb` file.

files/gitlab-cookbooks/gitlab/definitions/unicorn_config.rb

@@ -15,7 +15,7 @@
 # limitations under the License.
 #
 
-define :unicorn_config, :listen => nil, :working_directory => nil, :worker_timeout => 60, :preload_app => false, :worker_processes => 4, :before_fork => nil, :after_fork => nil, :pid => nil, :stderr_path => nil, :stdout_path => nil, :notifies => nil, :owner => nil, :group => nil, :mode => nil do
+define :unicorn_config, :listen => nil, :working_directory => nil, :worker_timeout => 60, :preload_app => false, :worker_processes => 4, :before_fork => nil, :after_fork => nil, :pid => nil, :stderr_path => nil, :stdout_path => nil, :relative_url => nil, :notifies => nil, :owner => nil, :group => nil, :mode => nil do
   config_dir = File.dirname(params[:name])
 
   directory config_dir do

files/gitlab-cookbooks/gitlab/definitions/unicorn_service.rb

@@ -65,6 +65,7 @@ define :unicorn_service, :rails_app => nil, :user => nil do
     preload_app true
     stderr_path File.join(unicorn_log_dir, "unicorn_stderr.log")
     stdout_path File.join(unicorn_log_dir, "unicorn_stdout.log")
+    relative_url node['gitlab'][svc]['relative_url']
     pid unicorn_pidfile
     before_fork <<-'EOS'
       old_pid = "#{server.config[:pid]}.oldbin"

files/gitlab-cookbooks/gitlab/libraries/gitlab.rb

@@ -135,7 +135,10 @@ module Gitlab
       end
 
       unless ["", "/"].include?(uri.path)
-        raise "Unsupported external URL path: #{uri.path}"
+        relative_url = uri.path.chomp("/")
+        Gitlab['gitlab_rails']['gitlab_relative_url'] ||= relative_url
+        Gitlab['unicorn']['relative_url'] ||= relative_url
+        Gitlab['gitlab_workhorse']['relative_url'] ||= relative_url
       end
 
       Gitlab['gitlab_rails']['gitlab_port'] = uri.port

files/gitlab-cookbooks/gitlab/recipes/gitlab-rails.rb

@@ -208,6 +208,20 @@ template_symlink File.join(gitlab_rails_etc_dir, "smtp_settings.rb") do
   end
 end
 
+template_symlink File.join(gitlab_rails_etc_dir, "relative_url.rb") do
+  link_from File.join(gitlab_rails_source_dir, "config/initializers/relative_url.rb")
+  owner "root"
+  group "root"
+  mode "0644"
+  variables(node['gitlab']['gitlab-rails'].to_hash)
+  notifies :run, 'bash[generate assets]'
+  restarts dependent_services
+
+  unless node['gitlab']['gitlab-rails']['gitlab_relative_url']
+    action :delete
+  end
+end
+
 template_symlink File.join(gitlab_rails_etc_dir, "gitlab.yml") do
   link_from File.join(gitlab_rails_source_dir, "config/gitlab.yml")
   source "gitlab.yml.erb"
@@ -280,6 +294,7 @@ remote_file File.join(gitlab_rails_dir, 'VERSION') do
   source "file:///opt/gitlab/embedded/service/gitlab-rails/VERSION"
   notifies :run, 'bash[migrate gitlab-rails database]' unless postgresql_not_listening
   notifies :run, 'execute[clear the gitlab-rails cache]' unless redis_not_listening
+  notifies :run, 'bash[generate assets]' if node['gitlab']['gitlab-rails']['gitlab_relative_url']
   dependent_services.each do |sv|
     notifies :restart, sv
   end
@@ -298,6 +313,17 @@ execute "clear the gitlab-rails cache" do
   action :nothing
 end
 
+bash "generate assets" do
+  code <<-EOS
+    set -e
+    /opt/gitlab/bin/gitlab-rake assets:clean assets:precompile
+    chown -R #{gitlab_user}:#{gitlab_group} #{gitlab_rails_tmp_dir}/cache
+  EOS
+  # We have to precompile assets as root because of permissions and ownership of files
+  environment ({ 'NO_PRIVILEGE_DROP' => 'true', 'USE_DB' => 'false' })
+  action :nothing
+end
+
 bitbucket_keys = node['gitlab']['gitlab-rails']['bitbucket']
 
 unless bitbucket_keys.nil?

files/gitlab-cookbooks/gitlab/recipes/gitlab-shell.rb

@@ -80,7 +80,7 @@ end
 
 # If no internal_api_url is specified, default to the IP/port Unicorn listens on
 api_url = node['gitlab']['gitlab-rails']['internal_api_url']
-api_url ||= "http://#{node['gitlab']['unicorn']['listen']}:#{node['gitlab']['unicorn']['port']}"
+api_url ||= "http://#{node['gitlab']['unicorn']['listen']}:#{node['gitlab']['unicorn']['port']}#{node['gitlab']['unicorn']['relative_url']}"
 
 redis_port = node['gitlab']['gitlab-rails']['redis_port']
 if redis_port

files/gitlab-cookbooks/gitlab/recipes/nginx.rb

@@ -94,6 +94,7 @@ template gitlab_rails_http_conf do
     {
       :fqdn => node['gitlab']['gitlab-rails']['gitlab_host'],
       :port => node['gitlab']['gitlab-rails']['gitlab_port'],
+      :relative_url => node['gitlab']['gitlab-rails']['gitlab_relative_url'],
       :kerberos_enabled => node['gitlab']['gitlab-rails']['kerberos_enabled'],
       :kerberos_use_dedicated_port => node['gitlab']['gitlab-rails']['kerberos_use_dedicated_port'],
       :kerberos_port => node['gitlab']['gitlab-rails']['kerberos_port'],

files/gitlab-cookbooks/gitlab/templates/default/gitlab.yml.erb

@@ -21,7 +21,7 @@ production: &base
 
     # WARNING: See config/application.rb under "Relative url support" for the list of
     # other files that need to be changed for relative url support
-    # relative_url_root: /gitlab
+    relative_url_root: <%= @gitlab_relative_url %>
 
     # Uncomment and customize if you can't use the default user to run GitLab (default: 'git')
     user: <%= node['gitlab']['user']['username'] %>

files/gitlab-cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb

@@ -91,7 +91,8 @@ server {
   access_log  <%= @log_directory %>/gitlab_access.log gitlab_access;
   error_log   <%= @log_directory %>/gitlab_error.log;
 
-  location / {
+  <% path = @relative_url ? @relative_url : "/" %>
+  location <%= path %> {
     ## If you use HTTPS make sure you disable gzip compression
     ## to be safe against BREACH attack.
     <%= 'gzip off;' if @https %>

files/gitlab-cookbooks/gitlab/templates/default/relative_url.rb.erb

@@ -0,0 +1,7 @@
+# This file is managed by gitlab-ctl. Manual changes will be
+# erased! To change the contents below, edit /etc/gitlab/gitlab.rb
+# and run `sudo gitlab-ctl reconfigure`.
+
+Rails.application.configure do
+  config.relative_url_root = "<%= @gitlab_relative_url %>"
+end

files/gitlab-cookbooks/gitlab/templates/default/sv-gitlab-workhorse-run.erb

@@ -15,7 +15,7 @@ exec chpst -e /opt/gitlab/etc/gitlab-workhorse/env -P \
     -listenNetwork <%= node['gitlab']['gitlab-workhorse']['listen_network'] %> \
     -listenUmask <%= node['gitlab']['gitlab-workhorse']['listen_umask'] %> \
     -listenAddr <%= node['gitlab']['gitlab-workhorse']['listen_addr'] %> \
-    -authBackend <%= node['gitlab']['gitlab-workhorse']['auth_backend'] %> \
+    -authBackend <%= node['gitlab']['gitlab-workhorse']['auth_backend'] %><%= node['gitlab']['gitlab-workhorse']['relative_url'] %> \
     -authSocket <%= node['gitlab']['gitlab-workhorse']['auth_socket'] %> \
     -documentRoot /opt/gitlab/embedded/service/gitlab-rails/public \
     -pprofListenAddr <%= node['gitlab']['gitlab-workhorse']['pprof_listen_addr'] %>\

files/gitlab-cookbooks/gitlab/templates/default/unicorn.rb.erb

@@ -50,3 +50,8 @@ stderr_path '<%= @stderr_path %>'
 stdout_path '<%= @stdout_path %>'
 
 <%- end %>
+<%- if @relative_url %>
+# Relative url from where GitLab is served
+ENV['RAILS_RELATIVE_URL_ROOT'] = "<%= @relative_url %>"
+
+<%- end %>