Skip to content

Commit ac6eef8

Browse files
KevinBKozanKevinBKozan
authored
Merge branch 'develop' into MQE-1570
2 parents d972c20 + 14a433a commit ac6eef8

File tree

6 files changed

+48
-20
lines changed

6 files changed

+48
-20
lines changed

‎docs/commands/mftf.md

+8-11
Original file line numberDiff line numberDiff line change
@@ -122,19 +122,9 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
122122
| `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
123123
| `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
124124
| `--tests` | Defines the test configuration as a JSON string.|
125-
| `--debug=[<developer> or <none>]`| Set debug level to `default` when the option is not specified, to `developer` with `--debug developer` or `--debug`, to `none` with `--debug none` | |
125+
| `--debug or --debug=[<none>]`| Performs schema validations on XML files. </br> <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br> <br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
126126
| `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
127127

128-
#### Debugging levels
129-
130-
You can run `generate:tests` with any of the following debug levels. XML schema validation errors will be logged as CRITICAL failures.
131-
132-
| Debug level | Description |
133-
| ---| --- |
134-
|`default` `[generate:tests]` | Perform XML schema validation on merged files. Does not indicate the file name where the error is encountered. Use `developer` level for enhanced debugging. |
135-
|`developer` `[generate:tests --debug or generate:tests --debug developer]` | Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. Note: This option takes extra processing time. Use it after test generation has failed once. |
136-
|`none` `[generate:tests --debug none]`| Skip debugging during test generation. Added for backward compatibility, it will be removed with the next MAJOR release.|
137-
138128
#### Examples of the JSON configuration
139129

140130
The configuration to generate a single test with no suites:
@@ -305,6 +295,7 @@ vendor/bin/mftf run:group [--skip-generate|--remove] [--] <group1> [<group2>]
305295
| --------------------- | --------------------------------------------------------------------------------------------------------- |
306296
| `-k, --skip-generate` | Skips generating from the source XML. Instead, the command executes previously-generated groups of tests. |
307297
| `-r, --remove` | Removes previously generated suites and tests before the actual generation and run. |
298+
| `--debug or --debug=[<none>]`| Performs schema validations on XML files. `run:group` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred). `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.|
308299

309300
#### Examples
310301

@@ -336,6 +327,7 @@ vendor/bin/mftf run:test [--skip-generate|--remove] [--] <name1> [<name2>]
336327
|-----------------------|-----------------------------------------------------------------------------------------------------------|
337328
| `-k, --skip-generate` | Skips generating from the source XML. Instead, the command executes previously-generated groups of tests. |
338329
| `-r, --remove` | Remove previously generated suites and tests. |
330+
| `--debug or --debug=[<none>]`| Performs schema validations on XML files. `run:test` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred). `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.
339331

340332
#### Examples
341333

@@ -357,6 +349,11 @@ For more details about `failed`, refer to [Reporting][].
357349
```bash
358350
vendor/bin/mftf run:failed
359351
```
352+
#### Options
353+
354+
| Option | Description |
355+
|-----------------------|-----------------------------------------------------------------------------------------------------------|
356+
| `--debug or --debug=[<none>]`| Performs schema validations on XML files. `run:failed` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred). Use it after test run has failed once. `--debug=none` skips debugging during test run. Added for backward compatibility, it will be removed in the next MAJOR release.|
360357

361358
#### Examples
362359

‎src/Magento/FunctionalTestingFramework/Config/Reader/MftfFilesystem.php

+2-2
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ public function readFiles($fileList)
4242
$configMerger->merge($content, $fileList->getFilename(), $exceptionCollector);
4343
}
4444
// run per file validation with generate:tests -d
45-
if ($debugLevel == MftfApplicationConfig::LEVEL_DEVELOPER) {
45+
if ($debugLevel === MftfApplicationConfig::LEVEL_DEVELOPER) {
4646
$this->validateSchema($configMerger, $fileList->getFilename());
4747
}
4848
} catch (\Magento\FunctionalTestingFramework\Config\Dom\ValidationException $e) {
@@ -52,7 +52,7 @@ public function readFiles($fileList)
5252
$exceptionCollector->throwException();
5353

5454
//run validation on merged file with generate:tests
55-
if ($debugLevel == MftfApplicationConfig::LEVEL_DEFAULT) {
55+
if ($debugLevel === MftfApplicationConfig::LEVEL_DEFAULT) {
5656
$this->validateSchema($configMerger);
5757
}
5858

‎src/Magento/FunctionalTestingFramework/Console/GenerateTestsCommand.php

+1-1
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ protected function configure()
5757
InputOption::VALUE_OPTIONAL,
5858
'Run extra validation when generating tests. Use option \'none\' to turn off debugging --
5959
added for backward compatibility, will be removed in the next MAJOR release',
60-
'default'
60+
MftfApplicationConfig::LEVEL_DEFAULT
6161
);
6262

6363
parent::configure();

‎src/Magento/FunctionalTestingFramework/Console/RunTestCommand.php

+9-1
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,13 @@ protected function configure()
3737
'f',
3838
InputOption::VALUE_NONE,
3939
'force generation of tests regardless of Magento Instance Configuration'
40+
)->addOption(
41+
'debug',
42+
'd',
43+
InputOption::VALUE_OPTIONAL,
44+
'Run extra validation when running tests. Use option \'none\' to turn off debugging --
45+
added for backward compatibility, will be removed in the next MAJOR release',
46+
MftfApplicationConfig::LEVEL_DEFAULT
4047
);
4148

4249
parent::configure();
@@ -58,6 +65,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
5865
$skipGeneration = $input->getOption('skip-generate');
5966
$force = $input->getOption('force');
6067
$remove = $input->getOption('remove');
68+
$debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
6169

6270
if ($skipGeneration and $remove) {
6371
// "skip-generate" and "remove" options cannot be used at the same time
@@ -75,7 +83,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
7583
]),
7684
'--force' => $force,
7785
'--remove' => $remove,
78-
'--debug' => MftfApplicationConfig::LEVEL_NONE
86+
'--debug' => $debug
7987
];
8088
$command->run(new ArrayInput($args), $output);
8189
}

‎src/Magento/FunctionalTestingFramework/Console/RunTestFailedCommand.php

+17-3
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
use Symfony\Component\Console\Output\OutputInterface;
1414
use Symfony\Component\Process\Process;
1515
use Magento\FunctionalTestingFramework\Exceptions\TestFrameworkException;
16+
use Symfony\Component\Console\Input\InputOption;
1617

1718
class RunTestFailedCommand extends BaseGenerateCommand
1819
{
@@ -49,7 +50,15 @@ class RunTestFailedCommand extends BaseGenerateCommand
4950
protected function configure()
5051
{
5152
$this->setName('run:failed')
52-
->setDescription('Execute a set of tests referenced via failed file');
53+
->setDescription('Execute a set of tests referenced via failed file')
54+
->addOption(
55+
'debug',
56+
'd',
57+
InputOption::VALUE_OPTIONAL,
58+
'Run extra validation when running failed tests. Use option \'none\' to turn off debugging --
59+
added for backward compatibility, will be removed in the next MAJOR release',
60+
MftfApplicationConfig::LEVEL_DEFAULT
61+
);
5362

5463
parent::configure();
5564
}
@@ -67,12 +76,13 @@ protected function configure()
6776
*/
6877
protected function execute(InputInterface $input, OutputInterface $output): int
6978
{
79+
$debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
7080
// Create Mftf Configuration
7181
MftfApplicationConfig::create(
7282
false,
7383
MftfApplicationConfig::GENERATION_PHASE,
7484
false,
75-
MftfApplicationConfig::LEVEL_NONE
85+
$debug
7686
);
7787

7888
$testConfiguration = $this->getFailedTestList();
@@ -83,7 +93,11 @@ protected function execute(InputInterface $input, OutputInterface $output): int
8393
}
8494

8595
$command = $this->getApplication()->find('generate:tests');
86-
$args = ['--tests' => $testConfiguration, '--remove' => true];
96+
$args = [
97+
'--tests' => $testConfiguration,
98+
'--remove' => true,
99+
'--debug' => $debug
100+
];
87101

88102
$command->run(new ArrayInput($args), $output);
89103

‎src/Magento/FunctionalTestingFramework/Console/RunTestGroupCommand.php

+11-2
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,13 @@ protected function configure()
3939
'f',
4040
InputOption::VALUE_NONE,
4141
'force generation of tests regardless of Magento Instance Configuration'
42+
)->addOption(
43+
'debug',
44+
'd',
45+
InputOption::VALUE_OPTIONAL,
46+
'Run extra validation when running tests. Use option \'none\' to turn off debugging --
47+
added for backward compatibility, will be removed in the next MAJOR release',
48+
MftfApplicationConfig::LEVEL_DEFAULT
4249
)->addArgument(
4350
'groups',
4451
InputArgument::IS_ARRAY | InputArgument::REQUIRED,
@@ -64,6 +71,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
6471
$force = $input->getOption('force');
6572
$groups = $input->getArgument('groups');
6673
$remove = $input->getOption('remove');
74+
$debug = $input->getOption('debug') ?? MftfApplicationConfig::LEVEL_DEVELOPER; // for backward compatibility
6775

6876
if ($skipGeneration and $remove) {
6977
// "skip-generate" and "remove" options cannot be used at the same time
@@ -77,7 +85,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
7785
$force,
7886
MftfApplicationConfig::GENERATION_PHASE,
7987
false,
80-
MftfApplicationConfig::LEVEL_NONE
88+
$debug
8189
);
8290

8391
if (!$skipGeneration) {
@@ -86,7 +94,8 @@ protected function execute(InputInterface $input, OutputInterface $output): int
8694
$args = [
8795
'--tests' => $testConfiguration,
8896
'--force' => $force,
89-
'--remove' => $remove
97+
'--remove' => $remove,
98+
'--debug' => $debug
9099
];
91100

92101
$command->run(new ArrayInput($args), $output);

0 commit comments

Comments
 (0)