Allow setting feature flags via configuration parameter

- It requires database is set up because
  feature flags are stored to DB (table `application_settings`)
- Add configuration parameter GITLAB_FEATURE_FLAGS_ENABLE_TARGETS
  and GITLAB_FEATURE_FLAGS_DISABLE_TARGETS
- Add ruby script to configure feature flags from command line
  and invoke runtime (from configure_gitlab())

Author: kkimurak

README.md

@@ -50,6 +50,7 @@
     - [External Issue Trackers](#external-issue-trackers)
     - [Host UID / GID Mapping](#host-uid--gid-mapping)
     - [Piwik](#piwik)
+    - [Feature flags](#feature-flags)
     - [Exposing ssh port in dockerized gitlab-ce](docs/exposing-ssh-port.md)
     - [Available Configuration Parameters](#available-configuration-parameters)
 - [Maintenance](#maintenance)
@@ -801,6 +802,52 @@ These options should contain something like:
 - `PIWIK_URL=piwik.example.org`
 - `PIWIK_SITE_ID=42`
 
+#### Feature flags
+
+In this section, we talk about feature flags that administrators can change the state (See <https://docs.gitlab.com/ee/administration/feature_flags.html>). If you are looking for documentation for "Feature flags" that configured on project deploy settings, see <https://docs.gitlab.com/ee/operations/feature_flags.html>
+
+GitLab adopted feature flags strategies to deploy features in an early stage of development so that they can be incrementally rolled out. GitLab administrators with access to the [Rails console](https://docs.gitlab.com/ee/administration/feature_flags.html#how-to-enable-and-disable-features-behind-flags) or the [Feature flags API](https://docs.gitlab.com/ee/api/features.html) can control them (note that `sameersbn/gitlab` is a container image that provides GitLab installations from the source).  
+You can see all feature flags in GitLab at corresponding version of documentation: <https://docs.gitlab.com/ee/user/feature_flags.html>  
+
+For `sameersbn/gitlab`, you can control them via environment parameter [`GITLAB_FEATURE_FLAGS_DISABLE_TARGETS`](#gitlab_feature_flags_disable_targets) and [`GITLAB_FEATURE_FLAGS_ENABLE_TARGETS`](#gitlab_feature_flags_enable_targets) in addition to the above methods.  
+This image searches yml files in [`${GITLAB_INSTALL_DIR}/config/feature_flags`](https://gitlab.com/gitlab-org/gitlab-foss/-/tree/master/config/feature_flags) (typically `/home/git/gitlab/config/feature_flags/`) recursively and use the file list as a source of active feature flags.
+
+Here is a part of example `docker-compose.yml`:
+
+````yml
+services:
+  gitlab:
+    image: sameersbn/gitlab:latest
+    environment:
+    - GITLAB_FEATURE_FLAGS_DISABLE_TARGETS=auto_devops_banner_disabled,ci_enable_live_trace
+    - GITLAB_FEATURE_FLAGS_ENABLE_TARGETS=git_push_create_all_pipelines,build_service_proxy
+````
+
+Once the container up, you can see following messages in container log like below.  
+
+````sh
+...
+Configuring gitlab::feature_flags...
+- specified feature flags: {:to_be_disabled=>["auto_devops_banner_disabled", "ci_enable_live_trace"], :to_be_enabled=>["git_push_create_all_pipelines", "build_service_proxy"]}
+- auto_devops_banner_disabled : off
+- ci_enable_live_trace : off
+- git_push_create_all_pipelines : on
+- build_service_proxy : on
+...
+````
+
+If specified flag names are not included in the list, they will be ignored and appears to container log like below:
+
+````sh
+...
+Configuring gitlab::feature_flags...
+- specified feature flags: {:to_be_disabled=>["auto_devops_banner_disabled", "invalid_flag_name"], :to_be_enabled=>["git_push_create_all_pipelines", "another_invalid_flag_name"]}
+- Following flags are probably invalid and have been ignored: invalid_flag_name,another_invalid_flag_name
+- auto_devops_banner_disabled : off
+- git_push_create_all_pipelines : on
+...
+````
+
 #### Available Configuration Parameters
 
 *Please refer the docker run command options for the `--env-file` flag where you can specify all required environment variables in a single file. This will save you from writing a potentially long docker run command. Alternatively you can use docker-compose. docker-compose users and Docker Swarm mode users can also use the [secrets and config file options](#docker-secrets-and-configs)*
@@ -1619,6 +1666,17 @@ The value of the `worker-src` directive in the `Content-Security-Policy` header.
 
 The value of the `report-uri` directive in the `Content-Security-Policy` header
 
+##### `GITLAB_FEATURE_FLAGS_DISABLE_TARGETS`
+
+Comma separated list of feature flag names to be disabled. No whitespace is allowed.  
+You can see all feature flags in GitLab at corresponding version of documentation: <https://docs.gitlab.com/ee/user/feature_flags.html>  
+Feature flags name and its statement will be appear to container log. Note that some of the feature flags are implicitly enabled or disabled by GitLab itself, and are not appear to container log.  
+No defaults.
+
+##### `GITLAB_FEATURE_FLAGS_ENABLE_TARGETS`
+
+This parameter is the same as [`GITLAB_FEATURE_FLAGS_DISABLE_TARGETS`](#gitlab_feature_flags_enable_targets), except its purpose is to enable the feature flag. No defaults.
+
 ##### `SSL_SELF_SIGNED`
 
 Set to `true` when using self signed ssl certificates. `false` by default.

assets/runtime/env-defaults

@@ -637,3 +637,7 @@ GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_SCRIPT_SRC=${GITLAB_CONTENT_SECURITY_P
 GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_STYLE_SRC=${GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_STYLE_SRC:-"'self' 'unsafe-inline'"}
 GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_WORKER_SRC=${GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_WORKER_SRC:-"'self' blob:"}
 GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_REPORT_URI=${GITLAB_CONTENT_SECURITY_POLICY_DIRECTIVES_REPORT_URI:-}
+
+## Feature Flags
+GITLAB_FEATURE_FLAGS_DISABLE_TARGETS=${GITLAB_FEATURE_FLAGS_DISABLE_TARGETS:-}
+GITLAB_FEATURE_FLAGS_ENABLE_TARGETS=${GITLAB_FEATURE_FLAGS_ENABLE_TARGETS:-}

assets/runtime/functions

@@ -2006,8 +2006,55 @@ configure_gitlab() {
   rm -rf ${GITLAB_INSTALL_DIR}/tmp/sockets/gitlab.socket
 }
 
+# feature flags are recorded to database (schema "application_settings") so requires DB is (at least) initialized
+gitlab_configure_feature_flags() {  
+  echo "Configuring gitlab::feature_flags..."
+
+  if [[ -z "${GITLAB_FEATURE_FLAGS_ENABLE_TARGETS}" && -z "${GITLAB_FEATURE_FLAGS_ENABLE_TARGETS}" ]]; then
+    # Do nothing and reports no error if no targets specified
+    echo "- No targets specified. skipping..."
+    return 0
+  fi
+
+  # Build command line argument for script only when target is specified
+  # If not, scripts fails because option specifier is recognized as feature flags for example
+  # like "--disable --enable" : for this case, --disable is recognized as a value of option "--enable"
+  if [[ -n "${GITLAB_FEATURE_FLAGS_DISABLE_TARGETS}" ]]; then
+    GITLAB_FEATURE_FLAGS_DISABLE_TARGETS="--disable ${GITLAB_FEATURE_FLAGS_DISABLE_TARGETS}"
+  fi
+  # The same goes for --enable (this is the last option passed to "rails runner" that will be run below)
+  # For this case (final option), it throws "missing argument" error for execution like:
+  # like "--disable feature1,feature2 --enable"
+  if [[ -n "${GITLAB_FEATURE_FLAGS_ENABLE_TARGETS}" ]]; then
+    GITLAB_FEATURE_FLAGS_ENABLE_TARGETS="--enable ${GITLAB_FEATURE_FLAGS_ENABLE_TARGETS}"
+  fi
+
+  PWD_ORG=${PWD}
+  cd "${GITLAB_INSTALL_DIR}"
+
+  # copy the script to temporal directory : to avoid permission issue
+  cp "${GITLAB_RUNTIME_DIR}/scripts/configure_feature_flags.rb" "${GITLAB_TEMP_DIR}/"
+  chown "${GITLAB_USER}:" "${GITLAB_TEMP_DIR}/configure_feature_flags.rb"
+
+  echo "- Launching rails runner to set feature flags. This will take some time...."
+
+  # If arguments are empty, the script will do nothing and print object dump like below:
+  #  - specified feature flags: {:to_be_disabled=>[], :to_be_enabled=>[]}
+  # DO NOT qupte variables : word splitting must be enabled.
+  # If disabled, whole string like '--disable feature_name_1,feature_name_2'
+  # will be recognized as single option and results to invalid argument error
+  #
+  # shellcheck disable=SC2086
+  exec_as_git bundle exec rails runner "${GITLAB_TEMP_DIR}/configure_feature_flags.rb" \
+    ${GITLAB_FEATURE_FLAGS_DISABLE_TARGETS} \
+    ${GITLAB_FEATURE_FLAGS_ENABLE_TARGETS}
+
+  rm "${GITLAB_TEMP_DIR}/configure_feature_flags.rb"
+  cd "${PWD_ORG}"
+}
+
 configure_gitlab_requires_db() {
-  :
+  gitlab_configure_feature_flags
 }
 
 configure_gitlab_shell() {

assets/runtime/scripts/configure_feature_flags.rb

@@ -0,0 +1,90 @@
+#!/usr/bin/env ruby
+
+require "optparse"
+require "set"
+
+# sameersbn/docker-gitlab
+# Ruby script to configure feature flags via CLI
+# Intended to be executed in the context of Rails Runner of Gitlab application
+# (to get valid "Feature" module, defined in (gitlab root)/lib/feature.rb)
+# https://guides.rubyonrails.org/command_line.html#bin-rails-runner
+#   bundle exec rails runner <path to this script> -- --enable <enable target> --disable <disable target>
+
+class FeatureFlagCLI
+  def available_feature_flags()
+    # Feature flag lists are stored in (Gitlab root directory)/config/feature_flags/
+    # We can get the directory by accessing "root" property of "Gitlab" Module
+    # (may returns /home/git/gitlab for sameersbn/docker-gitlab)
+    feature_flag_yamls = Dir.glob("#{Gitlab.root}/config/feature_flags/**/*.yml")
+
+    if Gitlab.ee?
+      feature_flag_yamls.concat(Dir.glob("#{Gitlab.root}/ee/config/feature_flags/**/*.yml"))
+    end if
+
+      list = feature_flag_yamls.map { |p| File.basename(p, File.extname(p)) }
+    list
+  end
+
+  def parse_options(argv = ARGV)
+    op = OptionParser.new
+
+    opts = {
+      to_be_disabled: [],
+      to_be_enabled: [],
+    # TODO support "opt out", "opt out removed"
+    # to_be_opted_out: [],
+    # opt_out_removed: [],
+    }
+
+    op.on("-d", "--disable feature_a,feature_b,feature_c", Array, "comma-separated list of feature flags to be disabled (defaults: ${opts[:to_be_disabled]})") { |v|
+      opts[:to_be_disabled] = v.uniq
+    }
+    op.on("-e", "--enable feature_a,feature_b,feature_c", Array, "comma-separated list of feature flags to be enabled (defaults: ${opts[:to_be_enabled]})") { |v|
+      opts[:to_be_enabled] = v.uniq
+    }
+
+    begin
+      args = op.parse(argv)
+      succeed = true
+    rescue OptionParser::InvalidOption, OptionParser::MissingArgument => e
+      puts e.message
+      puts op.help
+      succeed = false
+    end
+
+    [succeed, opts, args]
+  end
+
+  def run
+    succeed, opts, args = parse_options
+    if succeed
+      puts "- specified feature flags: #{opts.to_s}"
+
+      available_flags = self.available_feature_flags
+      disable_targets = available_flags & opts[:to_be_disabled]
+      enable_targets = available_flags & opts[:to_be_disabled]
+
+      disable_targets.each do |feature|
+        Feature.disable(feature)
+      end
+
+      enable_targets.each do |feature|
+        Feature.enable(feature)
+      end
+
+      invalid_enable_targets = opts[:to_be_enabled] - enable_targets
+      invalid_disable_targets = opts[:to_be_disabled] - disable_targets
+      invalid_targets = invalid_disable_targets | invalid_enable_targets
+      if invalid_targets.length > 0
+        puts "- Following flags are probably invalid and have been ignored: #{invalid_enable_targets.to_a.join(",")}"
+      end
+    end
+
+    Feature.all
+  end
+end
+
+features = FeatureFlagCLI.new.run
+puts features.map { |f|
+  format("- feature %<name>s : %<state>s", name: f.name, state: f.state)
+}

entrypoint.sh

@@ -21,12 +21,12 @@ case ${1} in
         /usr/bin/supervisord -nc /etc/supervisor/supervisord.conf &
         SUPERVISOR_PID=$!
         migrate_database
-        configure_gitlab_requires_db
         kill -15 $SUPERVISOR_PID
         if ps h -p $SUPERVISOR_PID > /dev/null ; then
         wait $SUPERVISOR_PID || true
         fi
         rm -rf /var/run/supervisor.sock
+        configure_gitlab_requires_db
         exec /usr/bin/supervisord -nc /etc/supervisor/supervisord.conf
         ;;
       app:init)