Skip to content

CI: use aws codebuild for job dist-arm-linux #140148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 25, 2025
Merged

CI: use aws codebuild for job dist-arm-linux #140148

merged 1 commit into from
Apr 25, 2025
+91 −13

Conversation

marcoieni
Copy link
Member

We want to use AWS credits to run linux large runners. So we want to start using AWS Codebuild.

r? @Kobzol

try-job: dist-arm-linux

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Apr 22, 2025
@marcoieni
Copy link
Member Author

@bors try

marcoieni
marcoieni commented Apr 22, 2025
View reviewed changes
.github/workflows/ci.yml Outdated
@@ -168,6 +168,8 @@ jobs:
run: src/ci/scripts/install-ninja.sh

- name: enable ipv6 on Docker
# Don't run on codebuild because systemctl is not available
if: ${{ !contains(matrix.os, 'codebuild-ubuntu') }}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests pass even if we don't enable docker ipv6.

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 22, 2025
@bors
Auto merge of rust-lang#140148 - marcoieni:ci-aws-codebuild, r=<try>
6034bd4 
CI: use aws codebuild for job dist-arm-linux

try-job: dist-arm-linux
@bors
Copy link
Collaborator

bors commented Apr 22, 2025

⌛ Trying commit d93449e with merge 6034bd4...

marcoieni
marcoieni commented Apr 22, 2025
View reviewed changes
@Kobzol
Copy link
Contributor

Kobzol commented Apr 22, 2025

In some situations (e.g. freeing disk space), we can just ignore the errors. But for things that are missing on AWS, could we first prepare the AWS image to look as similas to the GHA image as possible (at the beginning of the job)?

@Kobzol
Copy link
Contributor

Kobzol commented Apr 22, 2025

Also we have a bunch of code in bootstrap that depends on the GITHUB_ACTIONS variable, but I suppose that it is set even on the AWS runners?

@marcoieni
Copy link
Member Author

In some situations (e.g. freeing disk space), we can just ignore the errors.

If we ignore the errors, the job will fail later because the image doesn't have enough space. I prefer to fail earlier so that we don't waste too much time.

@Kobzol
Copy link
Contributor

Kobzol commented Apr 22, 2025

I don't understand 😅 If something is not present on the image, you can't really delete it from it to save space, right? 😆 I just meant to ignore these kinds of erorrs.

@marcoieni
Copy link
Member Author

Right, let me explain better.

When you do sudo apt-get remove package_1 package_2 package_3 and one package is missing, the entire command fails, so no package is deleted. I didn't find a way to tell apt to ignore that package and delete the others.
If package deletion fails, the pipeline will fail after 1 hour or so, so we waste 1 hour.

The workaround is to uninstall the packages independently, i.e.:

sudo apt-get remove package_1
sudo apt-get remove package_2
sudo apt-get remove package_3

but that's way slower than removing them all at once.

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@bors
Copy link
Collaborator

bors commented Apr 22, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 22, 2025
@marcoieni
Copy link
Member Author

marcoieni commented Apr 22, 2025 •
edited
Loading

Probably the disk ran out of space, I will bump the machine specs.

@marcoieni
Copy link
Member Author

But for things that are missing on AWS, could we first prepare the AWS image to look as similas to the GHA image as possible (at the beginning of the job)?

I added the Install cargo in AWS CodeBuild CI step. I hope this addresses this concern 👍

@marcoieni
Copy link
Member Author

@bors try

@bors
Copy link
Collaborator

bors commented Apr 22, 2025

⌛ Trying commit 3e30d0f with merge 605f8b1...

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 22, 2025
@bors
Auto merge of rust-lang#140148 - marcoieni:ci-aws-codebuild, r=<try>
605f8b1 
CI: use aws codebuild for job dist-arm-linux

try-job: dist-arm-linux
@marcoieni
Copy link
Member Author

Also we have a bunch of code in bootstrap that depends on the GITHUB_ACTIONS variable, but I suppose that it is set even on the AWS runners?

yes, it's set on CodeBuild as well. Tested here: https://github.com/rust-lang/aws-runners-test/actions/runs/14592125980/job/40929590253
image

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@bors
Copy link
Collaborator

bors commented Apr 22, 2025

💔 Test failed - checks-actions

@marcoieni
Copy link
Member Author

I don't know why cargo wasn't available there. I tried moving the step after freeing disk space, let's see.

@bors try

@marcoieni
Copy link
Member Author

I wonder why PR checks don't start..

@Kobzol
Copy link
Contributor

Kobzol commented Apr 23, 2025

It also happened here: #140191 weird. They started after a repush.

@marcoieni marcoieni force-pushed the ci-aws-codebuild branch 2 times, most recently from 2cb951f to a77f2d3 Compare April 23, 2025 08:47
@marcoieni
Copy link
Member Author

tried two times. Still no luck 😵

@Kobzol
Copy link
Contributor

Kobzol commented Apr 23, 2025

Check Zulip 😆 The workflow has an error (https://github.com/rust-lang/rust/actions/runs/14613694054).

@marcoieni
Copy link
Member Author

@bors try

@bors
Copy link
Collaborator

bors commented Apr 23, 2025

⌛ Trying commit fdc7281 with merge 0392d7d...

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 23, 2025
@bors
Auto merge of rust-lang#140148 - marcoieni:ci-aws-codebuild, r=<try>
0392d7d 
CI: use aws codebuild for job dist-arm-linux

try-job: dist-arm-linux
@bors
Copy link
Collaborator

bors commented Apr 23, 2025

☀️ Try build successful - checks-actions
Build commit: 0392d7d (0392d7d9b6015162f94b8696d55e0ec17451b084)

@marcoieni
Copy link
Member Author

@bors r=Kobzol

@bors
Copy link
Collaborator

bors commented Apr 23, 2025

📌 Commit fdc7281 has been approved by Kobzol

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 23, 2025
@marcoieni
Copy link
Member Author

pushed a small fix

@marcoieni
Copy link
Member Author

@bors r=Kobzol

@bors
Copy link
Collaborator

bors commented Apr 23, 2025

📌 Commit 18c3370 has been approved by Kobzol

It is now in the queue for this repository.

Zalathar added a commit to Zalathar/rust that referenced this pull request Apr 24, 2025
@Zalathar
Rollup merge of rust-lang#140148 - marcoieni:ci-aws-codebuild, r=Kobzol
9c368a8 
CI: use aws codebuild for job dist-arm-linux

try-job: dist-arm-linux
@Zalathar Zalathar mentioned this pull request Apr 24, 2025
Closed
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 24, 2025
@bors
Auto merge of rust-lang#140231 - Zalathar:rollup-6bg0hhp, r=Zalathar
157a2b2 
Rollup of 23 pull requests

Successful merges:

 - rust-lang#139261 (mitigate MSVC alignment issue on x86-32)
 - rust-lang#139307 (std: Add performance warnings to HashMap::get_disjoint_mut)
 - rust-lang#139700 (Autodiff flags)
 - rust-lang#139752 (set subsections_via_symbols for ld64 helper sections)
 - rust-lang#139809 (Don't warn about `v128` in wasm ABI transition)
 - rust-lang#139852 (StableMIR: Implement `CompilerInterface`)
 - rust-lang#139945 (Extend HIR to track the source and syntax of a lifetime)
 - rust-lang#140028 (`deref_patterns`: support string and byte string literals in explicit `deref!("...")` patterns)
 - rust-lang#140139 (rustc_target: Adjust RISC-V feature implication)
 - rust-lang#140143 (Move `sys::pal::os::Env` into `sys::env`)
 - rust-lang#140148 (CI: use aws codebuild for job dist-arm-linux)
 - rust-lang#140150 (fix MAX_EXP and MIN_EXP docs)
 - rust-lang#140172 (Make algebraic functions into `const fn` items.)
 - rust-lang#140177 ([compiletest] Parallelize test discovery)
 - rust-lang#140181 (Remove `synstructure::Structure::underscore_const` calls.)
 - rust-lang#140184 (Update doc of cygwin target)
 - rust-lang#140186 (Rename `compute_x` methods)
 - rust-lang#140187 ([AIX] Handle AIX dynamic library extensions within c-link-to-rust-dylib run-make test)
 - rust-lang#140191 (Remove git repository from git config)
 - rust-lang#140194 (minicore: Have `//@ add-core-stubs` also imply `-Cforce-unwind-tables=yes`)
 - rust-lang#140195 (triagebot: label minicore changes w/ `A-test-infra-minicore` and ping jieyouxu on changes)
 - rust-lang#140202 (Make #![feature(let_chains)] bootstrap conditional in compiler/)
 - rust-lang#140214 (Remove comment about handling non-global where bounds with corresponding projection)

r? `@ghost`
`@rustbot` modify labels: rollup
@matthiaskrgr matthiaskrgr mentioned this pull request Apr 24, 2025
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 25, 2025
@bors
Auto merge of rust-lang#140273 - matthiaskrgr:rollup-rxmuvkg, r=matth…
603744a 
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#137096 (Stabilize flags for doctest cross compilation)
 - rust-lang#140148 (CI: use aws codebuild for job dist-arm-linux)
 - rust-lang#140187 ([AIX] Handle AIX dynamic library extensions within c-link-to-rust-dylib run-make test)
 - rust-lang#140196 (Improved diagnostics for non-primitive cast on non-primitive types (`Arc`, `Option`))
 - rust-lang#140210 (Work around cygwin issue on condvar timeout)
 - rust-lang#140213 (mention about `x.py setup` in `INSTALL.md`)
 - rust-lang#140229 (`DelimArgs` tweaks)
 - rust-lang#140248 (Fix impl block items indent)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 25, 2025
@bors
Auto merge of rust-lang#140273 - matthiaskrgr:rollup-rxmuvkg, r=matth…
5c54aa7 
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#137096 (Stabilize flags for doctest cross compilation)
 - rust-lang#140148 (CI: use aws codebuild for job dist-arm-linux)
 - rust-lang#140187 ([AIX] Handle AIX dynamic library extensions within c-link-to-rust-dylib run-make test)
 - rust-lang#140196 (Improved diagnostics for non-primitive cast on non-primitive types (`Arc`, `Option`))
 - rust-lang#140210 (Work around cygwin issue on condvar timeout)
 - rust-lang#140213 (mention about `x.py setup` in `INSTALL.md`)
 - rust-lang#140229 (`DelimArgs` tweaks)
 - rust-lang#140248 (Fix impl block items indent)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 7ba71d3 into rust-lang:master Apr 25, 2025
6 checks passed
@rustbot rustbot added this to the 1.88.0 milestone Apr 25, 2025
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Apr 25, 2025
@rust-timer
Unrolled build for rust-lang#140148
3e20f34 
Rollup merge of rust-lang#140148 - marcoieni:ci-aws-codebuild, r=Kobzol

CI: use aws codebuild for job dist-arm-linux

try-job: dist-arm-linux
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Kobzol Kobzol Kobzol left review comments

Assignees

@Kobzol Kobzol

Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.88.0
Development

Successfully merging this pull request may close these issues.
5 participants
@marcoieni @bors @Kobzol @rust-log-analyzer @rustbot