Make PG_TEST_EXTRA env var override the "meson setup" option

"meson test" used to ignore the PG_TEST_EXTRA environment variable,
which meant that in order to run additional tests, you had to run
"meson setup -DPG_TEST_EXTRA=...". That's somewhat expensive, and not
consistent with autoconf builds. Allow PG_TEST_EXTRA environment
variable to override the setup-time option at run time, so that you
can do "PG_TEST_EXTRA=... meson test".

To implement this, the configuration time value is passed as an extra
"--pg-test-extra" argument to testwrap instead of adding it to the
test environment. If the environment variable is set at the time of
running test, testwrap uses the value from the environment variable
and ignores the --pg-test-extra option.

Now that "meson test" obeys the environment variable, we can remove it
from the "meson setup" steps in the CI script. It will now be picked
up from the environment variable like with "make check".

Author: Nazir Bilal Yavuzk, Ashutosh Bapat
Reviewed-by: Ashutosh Bapat with inputs from Tom Lane and Andrew Dunstan

Author: hlinnaka

.cirrus.tasks.yml

@@ -175,7 +175,6 @@ task:
         --buildtype=debug \
         -Dcassert=true -Dinjection_points=true \
         -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-        -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
         -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
         build
     EOF
@@ -364,7 +363,6 @@ task:
             --buildtype=debug \
             -Dcassert=true -Dinjection_points=true \
             ${LINUX_MESON_FEATURES} \
-            -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
             build
         EOF
 
@@ -380,7 +378,6 @@ task:
             -Dllvm=disabled \
             --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
             -DPERL=perl5.36-i386-linux-gnu \
-            -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
             build-32
         EOF
 
@@ -502,7 +499,6 @@ task:
       -Dextra_lib_dirs=/opt/local/lib \
       -Dcassert=true -Dinjection_points=true \
       -Duuid=e2fs -Ddtrace=auto \
-      -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
       build
 
   build_script: ninja -C build -j${BUILD_JOBS}
@@ -574,7 +570,7 @@ task:
   # Use /DEBUG:FASTLINK to avoid high memory usage during linking
   configure_script: |
     vcvarsall x64
-    meson setup --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Dinjection_points=true -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% -DPG_TEST_EXTRA="%PG_TEST_EXTRA%" build
+    meson setup --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Dinjection_points=true -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% build
 
   build_script: |
     vcvarsall x64

doc/src/sgml/installation.sgml

@@ -3080,9 +3080,13 @@ ninja install
       <term><option>-DPG_TEST_EXTRA=<replaceable>TEST_SUITES</replaceable></option></term>
       <listitem>
        <para>
-        Enable test suites which require special software to run.  This option
-        accepts arguments via a whitespace-separated list.  See <xref
-        linkend="regress-additional"/> for details.
+        Enable additional test suites, which are not run by default because
+        they are not secure to run on a multiuser system, require special
+        software to run, or are resource intensive.  The argument is a
+        whitespace-separated list of tests to enable. See
+        <xref linkend="regress-additional"/> for details. If the
+        <envar>PG_TEST_EXTRA</envar> environment variable is set when the
+        tests are run, it overrides this setup-time option.
        </para>
       </listitem>
      </varlistentry>

meson.build

@@ -3289,11 +3289,6 @@ test_env.set('PG_REGRESS', pg_regress.full_path())
 test_env.set('REGRESS_SHLIB', regress_module.full_path())
 test_env.set('INITDB_TEMPLATE', test_initdb_template)
 
-# Test suites that are not safe by default but can be run if selected
-# by the user via the whitespace-separated list in variable PG_TEST_EXTRA.
-# Export PG_TEST_EXTRA so it can be checked in individual tap tests.
-test_env.set('PG_TEST_EXTRA', get_option('PG_TEST_EXTRA'))
-
 # Add the temporary installation to the library search path on platforms where
 # that works (everything but windows, basically). On windows everything
 # library-like gets installed into bindir, solving that issue.
@@ -3357,6 +3352,11 @@ foreach test_dir : tests
     testwrap,
     '--basedir', meson.build_root(),
     '--srcdir', test_dir['sd'],
+    # Some test suites are not run by default but can be run if selected by the
+    # user via variable PG_TEST_EXTRA. Pass configuration time value of
+    # PG_TEST_EXTRA as an argument to testwrap so that it can be overridden by
+    # run time value, if any.
+    '--pg-test-extra', get_option('PG_TEST_EXTRA'),
   ]
 
   foreach kind, v : test_dir

meson_options.txt

@@ -47,7 +47,7 @@ option('injection_points', type: 'boolean', value: false,
   description: 'Enable injection points')
 
 option('PG_TEST_EXTRA', type: 'string', value: '',
-  description: 'Enable selected extra tests')
+  description: 'Enable selected extra tests. Overridden by PG_TEST_EXTRA environment variable.')
 
 option('PG_GIT_REVISION', type: 'string', value: 'HEAD',
   description: 'git revision to be packaged by pgdist target')

src/tools/testwrap

@@ -13,6 +13,7 @@ parser.add_argument('--basedir', help='base directory of test', type=str)
 parser.add_argument('--testgroup', help='test group', type=str)
 parser.add_argument('--testname', help='test name', type=str)
 parser.add_argument('--skip', help='skip test (with reason)', type=str)
+parser.add_argument('--pg-test-extra', help='extra tests', type=str)
 parser.add_argument('test_command', nargs='*')
 
 args = parser.parse_args()
@@ -41,6 +42,15 @@ env_dict = {**os.environ,
             'TESTDATADIR': os.path.join(testdir, 'data'),
             'TESTLOGDIR': os.path.join(testdir, 'log')}
 
+
+# The configuration time value of PG_TEST_EXTRA is supplied via arguement
+# --pg-test-extra. But it can be overridden by environment variable
+# PG_TEST_EXTRA at the time of running a test. Hence use value from arguments
+# only if PG_TEST_EXTRA is not set in the test environment, which already
+# contains all the environment variables at the time of running the test.
+if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
+    env_dict["PG_TEST_EXTRA"] = args.pg_test_extra
+
 sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
 # Meson categorizes a passing TODO test point as bad
 # (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO