Magento Automated Testing

Magento Automated Testing

Magento Automated Testing is a testing infrastructure that allows to run the set of all required Magento Automated Tests against code in Pull Requests. It automatically triggers the tests against the changes on Pull Request creation or new commits pushes into the Pull Request. The running tests are represented both as Pull Request statuses and as Pull Requests checks.

Below there's information about all kinds of tests available currently.

Semantic Version Checker

Semantic Version Checker is a test that compares the changed code against the mainline and detects levels of code changes. It shows a list of changes grouped by the class (level) of changes (MAJOR, MINOR, PATCH) with explanation of the change (Class was removed, [public] Method implementation changed, [private] Property has been removed etc.) The allowed level is PATCH.

Functional Tests

Sample Data Tests

Unit Tests

Integration Tests

WebAPI Tests

Static Tests

Database Compare

Database Compare is a test which compares DB schema and data between freshly installed Magento instance (from the PR code) and another instance which was upgraded to the PR code from the pre-previous latest minor release Magento version. Usually it runs several separate scenarios:

• Upgrade from the latest pre-previous minor release directly to the PR code (e.g. 2.1.x => 2.3-based PR code)
• Upgrade from the latest pre-previous minor release to previous minor release, and the to the PR code (e.g. 2.1.x => 2.2.x => 2.3-based PR code)
• Upgrade from the latest pre-previous minor release directly to the PR code with sample data (e.g. 2.1.x => 2.3-based PR code)
• Upgrade from the latest pre-previous minor release to previous minor release, and the to the PR code with sample data (e.g. 2.1.x => 2.2.x => 2.3-based PR code)

Magento Component Health Index