Tests V2 #1026

shunjizhan · 2024-09-21T13:39:53Z

Change

Core Updates

finally we have test coverage!
replaced mandala node with chopsticks acala fork as the test node

Cleanups and Refactors

removed a lot of docker stuff in favor for local setups, which improved stability and reduced complexity
removed submodules of { hardhat, truffle }-tutorials, and instead use local e2e-{ hardhat, truffle }, which will be a refined subset of tutorials as e2e tests.
removed e2e tests from workspace packages. This can avoid introducing unnecessary deps confusion, especially when hardhat is using ethers v6, while providers are using v5. We will run them as independent pkgs.
removed waffle examples since waffle is outdated. Also, waffle examples are essentially bodhi signer test, so instead we will put these tests in bodhi package directly

Tests

In the scope of this PR, we only care about setting up the whole flow, so we only run a minimum test for each of the test section. We will actually transform and enable most of the old tests in the next PR.

The whole flow works as expected:

each test section runs in parallel and generates a coverage report
coverage reports are merged into one and is uploaded to codecov
test coverage is "penetrated" across packages. (i.e. when running e2e tests against eth rpc, test coverage also applies for eth-provider codes)

Todo

convert, fix, and enable all tests
deal with subql
some test coverage highlights seems to be off (though coverage data looks solid)

codecov · 2024-09-23T09:45:52Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

shunjizhan · 2024-09-25T09:30:03Z

packages/bodhi/src/BodhiSigner.ts

@@ -58,7 +60,8 @@ export class BodhiSigner extends AbstractSigner implements TypedDataSigner {
  }

  static fromPair(provider: BodhiProvider, pair: KeyringPair): BodhiSigner {
-    return new BodhiSigner(provider, pair.address, new SubstrateSigner(provider.api.registry, pair));
+    const substrateAddr = encodeAddress(pair.address, provider.api.registry.chainSS58);


not sure why chopsticks and mandala behaves a little differently here: when doing extrinsic.signAsync, the payload address will use prefix: 42 on mandala, but prefix: 10 on chopsticks.

So I have to explicitly use prefix: 10 addr here, otherwise it won't be able to find the signer. Take a note here in case it breaks anything

ok mandala does have prefix:42, so this was indeed a bug before, just happened to work on mandala

zjb0807

LGTM.

some tests for precompile contracts in hardhat-tutorials should be in the Acala repo.

zjb0807 · 2024-09-25T10:40:57Z

.github/workflows/test.yml

@@ -4,13 +4,18 @@ on:
    paths-ignore:
      - '**/README.md'

-concurrency:
-  group: ${{ github.workflow }}-${{ github.ref }}
-  cancel-in-progress: true


why remove this?

for the concurrency, they run in parallel by default

sometimes cancel previous tests is slow (in old CI sometimes take a couple minutes), so I removed cancel-in-progress: true so new test can run immediately

we can probably close #832 too if we don't cancel previous run, ubuntu-latest is free anyways (sorry github)

if in the new setup, cancel previous run isn't that slow, we can also add this back, either way

let's try add this back

zjb0807 · 2024-09-25T11:02:23Z

chopsticks/configs/acala.yml

+mock-signature-host: true
+runtime-log-level: 5
+block: 7000000
+wasm-override: ./chopsticks/wasm/acala-2250.wasm


Suggested change

wasm-override: ./chopsticks/wasm/acala-2250.wasm

wasm-override: ./chopsticks/wasm/acala-2260.wasm

old tests worked with runtime 2250, so let's control variable and upgrade to 2260 after all tests are converted and running (otherwise if some tests fail, we don't know if it's the test's issue, or runtime 2260's issue).

shunjizhan · 2024-09-25T12:10:53Z

some tests for precompile contracts in hardhat-tutorials should be in the Acala repo.

Yeah I agree, precompile functionalities should be tested in Acala's scope.

In bodhi's scope, it should be able to assume low level evm stuff works as expected. So bodhi tests only need to focus on toolings compatibilities (hardhat/truffle/viem/ethers/etc), as well as all different interactions with eth rpc

shunjizhan added 8 commits September 10, 2024 11:55

runtime 2260 wip

cbcc2db

basic setup for eth provider tests

8e1f167

fix

6ba5362

fix

279dbc4

update

555a6f6

eth rpc tests

680bd21

fix

aca7e7c

merge coverage report

d3d7c62

shunjizhan added 21 commits September 23, 2024 17:57

polish

ba05f76

fix

6ed5527

eth transactions tests

bf9faa6

hardhat e2e tets

6d7df26

dump docker logs

1b778e7

fix e2e test

b40bbb1

fix

761e89f

fix

623be26

fix

9552e2c

fix

a124bb8

clean up docker files

07a0ede

polish

c2f71a7

clean up

8bee5e7

fix yarn

9de78da

fix

1db2ab9

merge master

579e226

enable wasm override

513789b

truffle e2e

4b724be

fix

8a97cbe

fix

342b859

viem e2e

e9caa22

shunjizhan added 7 commits September 25, 2024 13:27

update

7357874

bodhi tests

4e3e8f2

polish

3563300

fix

caff225

fix merge

399a2c8

clean up

9984ca0

cleanup

48bf785

shunjizhan commented Sep 25, 2024

View reviewed changes

add codecov yml

d2663f6

shunjizhan requested review from xlc and zjb0807 September 25, 2024 09:44

shunjizhan marked this pull request as ready for review September 25, 2024 09:44

zjb0807 reviewed Sep 25, 2024

View reviewed changes

shunjizhan added 3 commits September 25, 2024 20:24

use latest action

addf176

fix deprecation warning

fe40928

checkout and setupnode v4

4ec551a

zjb0807 approved these changes Sep 26, 2024

View reviewed changes

xlc approved these changes Sep 26, 2024

View reviewed changes

shunjizhan merged commit 81996d4 into master Sep 26, 2024
13 checks passed

shunjizhan deleted the tests-v2 branch September 26, 2024 02:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests V2 #1026

Tests V2 #1026

shunjizhan commented Sep 21, 2024 •

edited

Loading

codecov bot commented Sep 23, 2024

shunjizhan Sep 25, 2024

shunjizhan Sep 25, 2024 •

edited

Loading

zjb0807 left a comment

zjb0807 Sep 25, 2024

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan Sep 25, 2024

shunjizhan Sep 25, 2024

zjb0807 Sep 25, 2024

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan commented Sep 25, 2024 •

edited

Loading

	wasm-override: ./chopsticks/wasm/acala-2250.wasm
	wasm-override: ./chopsticks/wasm/acala-2260.wasm

Tests V2 #1026

Tests V2 #1026

Conversation

shunjizhan commented Sep 21, 2024 • edited Loading

Change

Core Updates

Cleanups and Refactors

Tests

Todo

codecov bot commented Sep 23, 2024

Welcome to Codecov 🎉

shunjizhan Sep 25, 2024

Choose a reason for hiding this comment

shunjizhan Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

zjb0807 left a comment

Choose a reason for hiding this comment

zjb0807 Sep 25, 2024

Choose a reason for hiding this comment

shunjizhan Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

shunjizhan Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

shunjizhan Sep 25, 2024

Choose a reason for hiding this comment

shunjizhan Sep 25, 2024

Choose a reason for hiding this comment

zjb0807 Sep 25, 2024

Choose a reason for hiding this comment

shunjizhan Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

shunjizhan commented Sep 25, 2024 • edited Loading

shunjizhan commented Sep 21, 2024 •

edited

Loading

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan Sep 25, 2024 •

edited

Loading

shunjizhan commented Sep 25, 2024 •

edited

Loading