Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not add chess tests to CI unless strictly necessary #1105

Open
newling opened this issue Feb 14, 2025 · 2 comments
Open

Do not add chess tests to CI unless strictly necessary #1105

newling opened this issue Feb 14, 2025 · 2 comments

Comments

@newling
Copy link
Contributor

newling commented Feb 14, 2025

The chess tests take 1 minute to compile each, if we're not careful we're going to quickly get 1hr+ CI runs which will slow development down quite badly. See Time spent in compilation: 59091.0 [ms] in following from CI:

Found HIP installation: /usr, version 5.7.31921
 "/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/llvm-aie/bin/ld.lld" --nmagic /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/input.o /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/mm_npu1.o --gc-sections --orphan-handling=error -T /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/core_1_5.elf.ld -L/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/llvm-aie/bin/../lib/aie2-none-unknown-elf -L/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/llvm-aie/lib/clang/19/lib/aie2-none-unknown-elf /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/llvm-aie/lib/clang/19/lib/aie2-none-unknown-elf/libclang_rt.builtins.a -lc -lm /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/llvm-aie/bin/../lib/aie2-none-unknown-elf/crt0.o /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/llvm-aie/bin/../lib/aie2-none-unknown-elf/crt1.o -o /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/core_1_5.elf
Time spent in compilation: 59091.0 [ms]
Run #1 of 2 for /home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess.mlir
Running the following command:
/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/iree-install/bin/iree-run-module --module=/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess_aie.vmfb --input=512x4096xbf16=@/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess_input1.bin --input=4096x512xbf16=@/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess_input2.bin --device=xrt-lite --output=@/home/nod/actions-runner/_work/iree-amd-aie/iree-amd-aie/test_aie_vs_cpu/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess/matmul_512_512_4096_bf16_f32_O3_npu1_4col_outline_ukernel_chess_aie.bin --xrt_lite_n_core_rows=4 --xrt_lite_n_core_cols=4
Standard output from script:
EXEC @matmul

Maybe we should split the CI tests in chess and non-chess tests too, so that we can see that all peano tests pass quickly?

Update: this includes all ukernel tests that use chess. Maybe the solution will be to change ukernel tests to use peano for assembling (currently this doesn't work out-of-the-box). @jtuyls thoughts?

@jtuyls
Copy link
Collaborator

jtuyls commented Feb 15, 2025

Update: this includes all ukernel tests that use chess. Maybe the solution will be to change ukernel tests to use peano for assembling (currently this doesn't work out-of-the-box). @jtuyls thoughts?

Yeah, that's one of the reasons I have been working on getting in compilation of ukernels through peano. Except for a couple of (performance) tests, I think we should move the remaining tests to peano indeed.

@jtuyls
Copy link
Collaborator

jtuyls commented Feb 17, 2025

#1114 reduced the E2E Strix CI time from 40+ min to 12min.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants