Refactor PET in test.yml CI workflow

[rockbmb]

Closes #1010 .

This PR will rework the PET job in .github/workflows/test.yml.

• use a test matrix with individual retry/timeout configurations rather than just yarn test to avoid restarting the whole suite on a single suite's failure
• To also avoid RPC timeouts due to throttling/stalling by test network's production RPC endpoints, consider using https://github.com/AcalaNetwork/subway or Chopsticks (as in Rework CI to use Chopsticks servers to reduce RPC load open-web3-stack/polkadot-ecosystem-tests#504)
• Filter end-to-end test suites from PET to be used in finalized workflow
  • This repository is only concerned with the runtimes that are under the Polkadot Fellowship's purview, while PET contains end-to-end tests for other ecosystme chains; as such, it makes sense to filter what is included in CI to further reduce load.
• Re-enable parts of test.yml that were disabled to test PET changes in isolation

Review notes

Main changes:

• new folder in .github/subway-configs; it contains Subway configs for the RPC proxies that will be created during CI: one for each Polkadot/Kusama relay + system parachain .
• PET reenabled in .github/workflows/test.yml, and completely refactored
  • it uses a matrix strategy with two jobs, one for each network: Polkadot and Kusama

Details

Each of the above P/K jobs:

• Fetches Rust toolchain + Subway, builds its binary, and starts a Subway proxy for each of that network's relay/SP
  - Polkadot: 1 proxy for relay + 5 proxies for SPs
  - Kusama: 1 proxy for relay + 4 proxies for SPs
• Downloads WASM artifacts from a previous step of the CI pipeline
• Overrides PET's .env with said runtimes + proxies' endpoints
  • That means PET will run using runtimes built against the code of whichever PR triggered CI, and cache RPC responses, which are shared between all chains' tests
• Gets yarn, checks out PET repo, builds it
• Runs PET's known block number command from within it, to use the latest state in combination with built WASM runtimes
• Runs all end-to-end test suites for that network's chains
  • only runs tests related to relay + SPs; ecosystem chains are skipped, as this repo's purview is only for Fellowship-maintained runtimes
• Cleans up Subway proxies at the end

In particular, the critical change:

          yarn test -u $TEST_FILES --pool=threads --maxWorkers=3 --retry=3
          # `$TEST_FILES` contains all E2E modules for the network: relay + SPs included
          # 1. `-u` updates snapshots so that they don't fail spuriously e.g. proof size/ref time changes
          # 2. `--pool=threads` runs test files from `$TEST_FILES` in parallel using worker threads. Default
          #    is `--pool=forks`, which uses child processes - heavier.
          # 3. `--maxWorkers=3` combined with the above means at most 3 test files running concurrently, each in its
          #    own thread
          # 4. `--retry=3` attempts a failing test 3 times. This operates at the level of individual tests, not modules,
          #    or the whole test suite; one failure will *not* require a full rerun.

---

• Does not require a CHANGELOG entry

[github-actions]

Review required! Latest push from author must always be reviewed

[ggwpez]

thanks, lets see how it goes :)

[rockbmb]

/merge

[ggwpez]

/merge

[fellowship-merge-bot]

Enabled auto-merge in Pull Request

Available commands

• /merge: Enables auto-merge for Pull Request
• /merge cancel: Cancels auto-merge for Pull Request
• /merge help: Shows this menu

For more information see the documentation

[fellowship-merge-bot]

There was a problem running the action.

❌😵❌

Please find more information in the logs.

[bkchr]

Sorry for being that late to the part, but I don't get why we have literally the same file multiple times. Just different endpoints.

[bkchr]

#1077

[rockbmb]

but I don't get why we have literally the same file multiple times.

We don't! You're right.

This was an omission on my part; I planned to improve that point, and then forgot. Thanks for doing it.