Skip to content

Commit a1508b6

Browse files
jilu1jilu1
authored
Merge pull request #753 from magento/MQE-2207
Mqe 2207 Merge 3.0.0-GA release branch to master
2 parents 757e136 + 26b0d6e commit a1508b6

File tree

454 files changed

+10106
-9498
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

454 files changed

+10106
-9498
lines changed

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,4 @@ dev/tests/mftf.log
1919
dev/tests/docs/*
2020
dev/tests/_output
2121
dev/tests/functional.suite.yml
22+

‎.travis.yml

+6-2
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
11
language: php
22
php:
3-
- 7.1
4-
- 7.2
53
- 7.3
4+
- 7.4
5+
services:
6+
- docker
7+
before_install:
8+
- docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome:3.141.59-zirconium
69
install: composer install --no-interaction --prefer-source
710
env:
811
matrix:
912
- VERIFICATION_TOOL=phpunit-checks
1013
- VERIFICATION_TOOL=static-checks
14+
- VERIFICATION_TOOL=functional
1115
script:
1216
- bin/$VERIFICATION_TOOL
1317
after_success:

‎CHANGELOG.md

+91-3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,94 @@
11
Magento Functional Testing Framework Changelog
22
================================================
3-
2.6.3
3+
3.0.0
4+
---------
5+
6+
### Enhancements
7+
8+
* Customizability
9+
* Introduced MFTF helpers `<helper>` to create custom actions outside of MFTF.[See custom-helpers page for details](./docs/custom-helpers.md)
10+
* Removed deprecated actions `<executeSelenium>` and `<performOn>`.
11+
* `<group value="skip"/>` no longer skips a test. Instead, the test is added to the `skip` group.
12+
13+
* Maintainability
14+
* Added support for PHP 7.4.
15+
* Added support for PHPUnit 9.
16+
* Dropped support for PHP 7.0, 7.1, 7.2.
17+
* Schema updates for test entities to only allow single entity per file except Data and Metadata.
18+
* Support for sub-folders in test modules.
19+
* Removed support to read test entities from `<magento>dev/tests/acceptance/tests/functional/Magento/FunctionalTest`.
20+
* Removed file attribute for `<module>` in suiteSchema.
21+
* Removed action `pauseExecution` and added `pause`. [See actions page for details](./docs/test/actions.md#pause)
22+
* Removed action `formatMoney` and added `formatCurrency`. [See actions page for details](./docs/test/actions.md#formatcurrency)
23+
* Improved assertion actions to support PHPUnit 9 changes. [See assertions page for details](./docs/test/assertions.md)
24+
* Added new actions: `assertEqualsWithDelta`, `assertNotEqualsWithDelta`, `assertEqualsCanonicalizing`, `assertNotEqualsCanonicalizing`, `assertEqualsIgnoringCase`, `assertNotEqualsIgnoringCase`.
25+
* Added new actions: `assertStringContainsString`, `assertStringNotContainsString`, `assertStringContainsStringIgnoringCase`, `assertStringNotContainsStringIgnoringCase` for string haystacks.
26+
* Removed actions: `assertInternalType`, `assertNotInternalType`, `assertArraySubset`.
27+
* Removed delta option from `assertEquals` and `assertNotEquals`.
28+
* Added static check `deprecatedEntityUsage` that checks and reports references to deprecated test entities.
29+
* Added static check `annotations` that checks and reports missing annotations in tests.
30+
* Updated `bin/mftf static-checks` command to allow executing static-checks defined in `staticRuleSet.json` by default. [See command page for details](./docs/commands/mftf.md#static-checks)
31+
* Added support for Two-Factor Authentication (2FA). [See configure-2fa page for details](./docs/configure-2fa.md)
32+
* Added new upgrade script to remove unused arguments from action groups.
33+
* `mftf.log` no longer includes notices and warnings at test execution time.
34+
* Added unhandledPromptBehavior driver capability for Chrome 75+ support.
35+
* Added the Chrome option `--ignore-certificate-errors` to `functional.suite.dist.yml`.
36+
37+
* Traceability
38+
* Removed `--debug` option `NONE` to disallow ability to turn off schema validation.
39+
* Notices added for test entity naming convention violations.
40+
* Metadata file names changed to `*Meta.xml`.
41+
* Introduced new `.env` configuration `VERBOSE_ARTIFACTS` to toggle saving attachments in Allure. [See configuration page for details](./docs/configuration.md)
42+
* Changed the `bin/mftf static-checks` error file directory from the current working directory to `TESTS_BP/tests/_output/static-results/`.
43+
44+
* Readability
45+
* Support only nested assertion syntax [See assertions page for details](./docs/test/assertions.md).
46+
* Documented [3.0.0 Backward Incompatible Changes](./docs/backward-incompatible-changes.md).
47+
* Removed blacklist/whitelist terminology in MFTF.
48+
49+
* Upgrade scripts added to upgrade tests to MFTF major version requirements. See upgrade instructions below.
50+
* Bumped dependencies to support PHP/PHPUnit upgrade.
51+
52+
### Upgrade Instructions
53+
54+
* Run `bin/mftf reset --hard` to remove old generated configurations.
55+
* Run `bin/mftf build:project` to generate new configurations.
56+
* Run `bin/mftf upgrade:tests`. [See command page for details](./docs/commands/mftf.md#upgradetests).
57+
* After running the above command, some tests may need manually updates:
58+
* Remove all occurrences of `<executeInSelenium>` and `<performOn>`.
59+
* Remove all occurrences of `<module file=""/>` from any `<suite>`s.
60+
* Ensure all `<assert*>` actions in your tests have a valid schema.
61+
* Lastly, try to generate all tests. Tests should all be generated as a result of the upgrades.
62+
* If not, the most likely issue will be a changed XML schema. Check error messaging and search your codebase for the attributes listed.
63+
64+
### Fixes
65+
66+
* Throw exception during generation when leaving out .url for `amOnPage`.
67+
* `request_timeout` and `connection_timeout` added to functional.suite.yml.dist.
68+
* Fixed `ModuleResolver` to resolve test modules moved out of deprecated path.
69+
* Fixed issue of resolving arguments of type `entity` in action groups within a custom helper.
70+
* Fixed reporting issue in output file for `testDependencies` static check.
71+
* Fixed a bug in `actionGroupArguments` static check when action group filename is missing `ActionGroup`.
72+
* Fixed issue of running suites under root `_suite` directory in Standalone MFTF.
73+
* Fixed issue with custom helper usage in suites.
74+
* Fixed issue with decryption of secrets during data entity creation.
75+
* Fixed issue with merging of `array` items in data entity.
76+
* Fixed issue where an extended data entity would not merge array items. Array items should merge properly now.
77+
* Fixed issue where Chrome remains running after MFTF suite finishes.
78+
* Fixed javascript error seen on Chrome 83 for dragAndDrop action.
79+
* Fixed allure issue when `WebDriverCurlException` is encountered in `afterStep`.
80+
81+
### GitHub Issues/Pull Requests
82+
83+
* [#567](https://github.com/magento/magento2-functional-testing-framework/pull/567) -- log attachments for failed requests.
84+
85+
### Demo Video links
86+
87+
* [MFTF 3.0.0 RC1](https://www.youtube.com/watch?v=z0ZaZCmnw-A&t=2s)
88+
* [MFTF 3.0.0 RC2](https://www.youtube.com/watch?v=BJOQAw6dX5o)
89+
* [MFTF 3.0.0 RC3](https://www.youtube.com/watch?v=scLb7pi8pR0)
90+
91+
2.6.4
492
-----
593

694
### Fixes
@@ -70,9 +158,9 @@ Magento Functional Testing Framework Changelog
70158
* Command verifies and troubleshoots some configuration steps required for running tests
71159
* Please see DevDocs for more details
72160
* `<*Data>` actions now contain `API Endpoint` and `Request Header` artifacts.
73-
* Introduced new `.env` configurations `ENABLE_BROWSER_LOG` and `BROWSER_LOG_BLACKLIST`
161+
* Introduced new `.env` configurations `ENABLE_BROWSER_LOG` and `BROWSER_LOG_BLOCKLIST`
74162
* Configuration enables allure artifacts for browser log entries if they are present after the step.
75-
* Blacklist filters out logs from specific sources.
163+
* Blocklist filters out logs from specific sources.
76164
* Customizability
77165
* Introduced `timeout=""` to `magentoCLI` actions.
78166

‎bin/blacklist.txt renamed to ‎bin/blocklist.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# #
44
# THIS FILE CANNOT CONTAIN BLANK LINES #
55
###################################################################
6-
bin/blacklist.txt
6+
bin/blocklist.txt
77
dev/tests/static/Magento/Sniffs/Commenting/FunctionCommentSniff.php
88
dev/tests/static/Magento/Sniffs/Commenting/VariableCommentSniff.php
99
dev/tests/verification/_generated

‎bin/copyright-check

+2-2
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,13 @@
55

66

77
FILE_EXTENSIONS='.php\|.xml\|.xsd'
8-
BLACKLIST='bin/blacklist.txt'
8+
BLOCKLIST='bin/blocklist.txt'
99
RESULT=''
1010

1111
# Iterate through the list of tracked files
1212
# that have the expected extensions
1313
# that are not ignored
14-
for i in `git ls-tree --full-tree -r --name-only HEAD | grep $FILE_EXTENSIONS | grep -v -f $BLACKLIST`
14+
for i in `git ls-tree --full-tree -r --name-only HEAD | grep $FILE_EXTENSIONS | grep -v -f $BLOCKLIST`
1515
do
1616
if echo `cat $i` | grep -q -v "Copyright © Magento, Inc. All rights reserved."; then
1717
# Copyright is missing

‎bin/copyright-check.bat

+5-5
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
@echo off
55
SETLOCAL EnableDelayedExpansion
6-
SET BLACKLIST_FILE=bin/blacklist.txt
6+
SET BLOCKLIST_FILE=bin/blocklist.txt
77
SET i=0
88

99
FOR /F %%x IN ('git ls-tree --full-tree -r --name-only HEAD') DO (
@@ -12,14 +12,14 @@ FOR /F %%x IN ('git ls-tree --full-tree -r --name-only HEAD') DO (
1212
if "%%~xx"==".xml" set GOOD_EXT=1
1313
if "%%~xx"==".xsd" set GOOD_EXT=1
1414
IF DEFINED GOOD_EXT (
15-
SET BLACKLISTED=
16-
FOR /F "tokens=* skip=5" %%f IN (%BLACKLIST_FILE%) DO (
15+
SET BLOCKLISTED=
16+
FOR /F "tokens=* skip=5" %%f IN (%BLOCKLIST_FILE%) DO (
1717
SET LINE=%%x
1818
IF NOT "!LINE!"=="!LINE:%%f=!" (
19-
SET BLACKLISTED=1
19+
SET BLOCKLISTED=1
2020
)
2121
)
22-
IF NOT DEFINED BLACKLISTED (
22+
IF NOT DEFINED BLOCKLISTED (
2323
FIND "Copyright © Magento, Inc. All rights reserved." %%x >nul
2424
IF ERRORLEVEL 1 (
2525
SET /A i+=1

‎bin/functional

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Copyright © Magento, Inc. All rights reserved.
2+
# See COPYING.txt for license details.
3+
4+
set -e
5+
6+
echo "==============================="
7+
echo " EXECUTE Functional Tests "
8+
echo "==============================="
9+
bin/mftf build:project
10+
bin/mftf run:test DevDocsTest -f
11+
bin/mftf run:test FormatCurrencyTest -f

‎composer.json

+24-23
Original file line numberDiff line numberDiff line change
@@ -2,53 +2,54 @@
22
"name": "magento/magento2-functional-testing-framework",
33
"description": "Magento2 Functional Testing Framework",
44
"type": "library",
5-
"version": "2.6.4",
5+
"version": "3.0.0",
66
"license": "AGPL-3.0",
77
"keywords": ["magento", "automation", "functional", "testing"],
88
"config": {
99
"sort-packages": true
1010
},
1111
"require": {
12-
"php": "~7.1.0||~7.2.0||~7.3.0",
12+
"php": "^7.3",
1313
"ext-curl": "*",
1414
"ext-dom": "*",
15+
"ext-intl": "*",
1516
"ext-json": "*",
1617
"ext-openssl": "*",
17-
"allure-framework/allure-codeception": "~1.3.0",
18+
"allure-framework/allure-codeception": "~1.4.0",
1819
"aws/aws-sdk-php": "^3.132",
19-
"codeception/codeception": "~2.4.5",
20-
"composer/composer": "^1.6",
20+
"codeception/codeception": "~4.1.4",
21+
"codeception/module-asserts": "^1.1",
22+
"codeception/module-sequence": "^1.0",
23+
"codeception/module-webdriver": "^1.0",
24+
"composer/composer": "^1.9",
2125
"csharpru/vault-php": "~3.5.3",
2226
"csharpru/vault-php-guzzle6-transport": "^2.0",
23-
"flow/jsonpath": ">0.2",
24-
"fzaninotto/faker": "^1.6",
25-
"monolog/monolog": "^1.0",
27+
"monolog/monolog": "^1.17",
2628
"mustache/mustache": "~2.5",
2729
"php-webdriver/webdriver": "^1.8.0",
30+
"spomky-labs/otphp": "^10.0",
2831
"symfony/console": "^4.4",
29-
"symfony/finder": "^4.4",
30-
"symfony/http-foundation": "^4.4",
31-
"symfony/mime": "^4.4",
32+
"symfony/finder": "^5.0",
33+
"symfony/http-foundation": "^5.0",
34+
"symfony/mime": "^5.0",
3235
"symfony/process": "^4.4",
33-
"vlucas/phpdotenv": "^2.4"
36+
"vlucas/phpdotenv": "^2.4",
37+
"weew/helpers-array": "^1.3"
3438
},
3539
"require-dev": {
36-
"squizlabs/php_codesniffer": "~3.2",
37-
"sebastian/phpcpd": "~3.0 || ~4.0",
3840
"brainmaestro/composer-git-hooks": "^2.3.1",
39-
"doctrine/cache": "<1.7.0",
40-
"codeception/aspect-mock": "^3.0",
41-
"goaop/framework": "2.2.0",
4241
"codacy/coverage": "^1.4",
43-
"phpmd/phpmd": "^2.6.0",
44-
"phpunit/phpunit": "~6.5.0 || ~7.0.0",
45-
"rregeer/phpunit-coverage-check": "^0.1.4",
42+
"codeception/aspect-mock": "^3.0",
43+
"doctrine/cache": "<1.7.0",
44+
"goaop/framework": "~2.3.4",
4645
"php-coveralls/php-coveralls": "^1.0",
46+
"phpmd/phpmd": "^2.8.0",
47+
"phpunit/phpunit": "^9.0",
48+
"rregeer/phpunit-coverage-check": "^0.1.4",
49+
"sebastian/phpcpd": "~5.0.0",
50+
"squizlabs/php_codesniffer": "~3.5.4",
4751
"symfony/stopwatch": "~3.4.6"
4852
},
49-
"suggest": {
50-
"epfremme/swagger-php": "^2.0"
51-
},
5253
"replace": {
5354
"facebook/webdriver": "^1.7.1"
5455
},

0 commit comments

Comments
 (0)