Intermittent segmentation faults from rustc on github_arm_linux_runner_2404 #11533

westonpace · 2025-02-03T19:33:03Z

Description

The rustc on the runner image will sometimes abort with a segmentation fault. I have even seen this happen when just using rustc to print the current version of rustc.

Platforms affected

Azure DevOps
GitHub Actions - Standard Runners
GitHub Actions - Larger Runners

Runner images affected

Image version and build link

Here is the log of a failed run:

2025-02-03T19:10:39.9034761Z Current runner version: '2.322.0'
2025-02-03T19:10:39.9042153Z Runner name: 'gha-arm_464b79870998'
2025-02-03T19:10:39.9043013Z Runner group name: 'large-runners'
2025-02-03T19:10:39.9044039Z Machine name: 'runner'
2025-02-03T19:10:39.9062736Z ##[group]VM Image
2025-02-03T19:10:39.9063432Z OS: Linux (arm64)
2025-02-03T19:10:39.9064113Z Source: Marketplace
2025-02-03T19:10:39.9064667Z Publisher: arm
2025-02-03T19:10:39.9065243Z Offer: github_arm_linux_runner_2404
2025-02-03T19:10:39.9065988Z Plan: github_arm_linux_runner_plan_2404
2025-02-03T19:10:39.9066642Z Version: 1.0.10
2025-02-03T19:10:39.9067128Z ##[endgroup]
2025-02-03T19:10:39.9069683Z ##[group]GITHUB_TOKEN Permissions
2025-02-03T19:10:39.9072152Z Actions: write
2025-02-03T19:10:39.9072942Z Attestations: write
2025-02-03T19:10:39.9073466Z Checks: write
2025-02-03T19:10:39.9073971Z Contents: write
2025-02-03T19:10:39.9074606Z Deployments: write
2025-02-03T19:10:39.9075136Z Discussions: write
2025-02-03T19:10:39.9075932Z Issues: write
2025-02-03T19:10:39.9076513Z Metadata: read
2025-02-03T19:10:39.9077093Z Packages: write
2025-02-03T19:10:39.9078025Z Pages: write
2025-02-03T19:10:39.9078582Z PullRequests: write
2025-02-03T19:10:39.9079164Z RepositoryProjects: write
2025-02-03T19:10:39.9079774Z SecurityEvents: write
2025-02-03T19:10:39.9080349Z Statuses: write
2025-02-03T19:10:39.9080881Z ##[endgroup]
2025-02-03T19:10:39.9083583Z Secret source: Actions
2025-02-03T19:10:39.9084643Z Prepare workflow directory
2025-02-03T19:10:39.9498312Z Prepare all required actions
2025-02-03T19:10:39.9535448Z Getting action download info
2025-02-03T19:10:40.1882216Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2025-02-03T19:10:40.3078691Z Download action repository 'actions-rust-lang/setup-rust-toolchain@v1' (SHA:11df97af8e8102fd60b60a77dfbf58d40cd843b8)
2025-02-03T19:10:40.5970122Z Download action repository 'Swatinem/rust-cache@v2' (SHA:f0deed1e0edfc6a9be95417288c0e1099b1eeec3)
2025-02-03T19:10:41.1178898Z Download action repository 'taiki-e/install-action@cargo-llvm-cov' (SHA:8f3491ddae78b96b7fe95d046c68c3934396d5aa)
2025-02-03T19:10:41.6228610Z Download action repository 'codecov/codecov-action@v4' (SHA:b9fd7d16f6d7d1b5d2bec1a2887e65ceed900238)
2025-02-03T19:10:42.2119285Z Getting action download info
2025-02-03T19:10:42.3808315Z Complete job name: build (gha-arm)
2025-02-03T19:10:42.4576115Z ##[group]Run actions/checkout@v4
2025-02-03T19:10:42.4577209Z with:
2025-02-03T19:10:42.4577941Z   submodules: true
2025-02-03T19:10:42.4578493Z   repository: lancedb/redacted
2025-02-03T19:10:42.4579320Z   token: ***
2025-02-03T19:10:42.4579809Z   ssh-strict: true
2025-02-03T19:10:42.4580311Z   ssh-user: git
2025-02-03T19:10:42.4580835Z   persist-credentials: true
2025-02-03T19:10:42.4581418Z   clean: true
2025-02-03T19:10:42.4581927Z   sparse-checkout-cone-mode: true
2025-02-03T19:10:42.4582543Z   fetch-depth: 1
2025-02-03T19:10:42.4583044Z   fetch-tags: false
2025-02-03T19:10:42.4583552Z   show-progress: true
2025-02-03T19:10:42.4584112Z   lfs: false
2025-02-03T19:10:42.4584604Z   set-safe-directory: true
2025-02-03T19:10:42.4585565Z env:
2025-02-03T19:10:42.4586033Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:42.4586599Z   RUST_BACKTRACE: 1
2025-02-03T19:10:42.4587095Z   CC: clang
2025-02-03T19:10:42.4587751Z   CXX: clang++
2025-02-03T19:10:42.4588243Z ##[endgroup]
2025-02-03T19:10:42.5503038Z Syncing repository: lancedb/redacted
2025-02-03T19:10:42.5505060Z ##[group]Getting Git version info
2025-02-03T19:10:42.5505899Z Working directory is '/home/runner/work/redacted/redacted'
2025-02-03T19:10:42.5507074Z [command]/usr/bin/git version
2025-02-03T19:10:42.9266351Z git version 2.48.1
2025-02-03T19:10:42.9292427Z ##[endgroup]
2025-02-03T19:10:42.9316335Z Temporarily overriding HOME='/home/runner/work/_temp/39ad436c-d906-4a77-b7bc-889fed8d40f4' before making global git config changes
2025-02-03T19:10:42.9318318Z Adding repository directory to the temporary git global config as a safe directory
2025-02-03T19:10:42.9322124Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/redacted/redacted
2025-02-03T19:10:42.9646907Z Deleting the contents of '/home/runner/work/redacted/redacted'
2025-02-03T19:10:42.9651288Z ##[group]Initializing the repository
2025-02-03T19:10:42.9655173Z [command]/usr/bin/git init /home/runner/work/redacted/redacted
2025-02-03T19:10:43.1949068Z hint: Using 'master' as the name for the initial branch. This default branch name
2025-02-03T19:10:43.1950322Z hint: is subject to change. To configure the initial branch name to use in all
2025-02-03T19:10:43.1951434Z hint: of your new repositories, which will suppress this warning, call:
2025-02-03T19:10:43.1952413Z hint:
2025-02-03T19:10:43.1953263Z hint: 	git config --global init.defaultBranch <name>
2025-02-03T19:10:43.1954105Z hint:
2025-02-03T19:10:43.1954881Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2025-02-03T19:10:43.1956089Z hint: 'development'. The just-created branch can be renamed via this command:
2025-02-03T19:10:43.1957077Z hint:
2025-02-03T19:10:43.1958071Z hint: 	git branch -m <name>
2025-02-03T19:10:43.2215826Z Initialized empty Git repository in /home/runner/work/redacted/redacted/.git/
2025-02-03T19:10:43.2227660Z [command]/usr/bin/git remote add origin https://github.com/lancedb/redacted
2025-02-03T19:10:43.2406435Z ##[endgroup]
2025-02-03T19:10:43.2407921Z ##[group]Disabling automatic garbage collection
2025-02-03T19:10:43.2410094Z [command]/usr/bin/git config --local gc.auto 0
2025-02-03T19:10:43.2438227Z ##[endgroup]
2025-02-03T19:10:43.2439346Z ##[group]Setting up auth
2025-02-03T19:10:43.2445189Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-02-03T19:10:43.2471383Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-02-03T19:10:43.7668552Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-02-03T19:10:43.7692515Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-02-03T19:10:43.7872386Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-02-03T19:10:43.7902008Z ##[endgroup]
2025-02-03T19:10:43.7902678Z ##[group]Fetching the repository
2025-02-03T19:10:43.7917078Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin +bdca98e03f81b6b59ec2c329d27d6103536c08d5:refs/remotes/pull/3314/merge
2025-02-03T19:10:44.9308955Z From https://github.com/lancedb/redacted
2025-02-03T19:10:44.9309681Z  * [new ref]         bdca98e03f81b6b59ec2c329d27d6103536c08d5 -> pull/3314/merge
2025-02-03T19:10:44.9476918Z ##[endgroup]
2025-02-03T19:10:44.9477623Z ##[group]Determining the checkout info
2025-02-03T19:10:44.9481386Z ##[endgroup]
2025-02-03T19:10:44.9483074Z [command]/usr/bin/git sparse-checkout disable
2025-02-03T19:10:44.9658333Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2025-02-03T19:10:44.9683367Z ##[group]Checking out the ref
2025-02-03T19:10:44.9687349Z [command]/usr/bin/git checkout --progress --force refs/remotes/pull/3314/merge
2025-02-03T19:10:45.0654495Z Note: switching to 'refs/remotes/pull/3314/merge'.
2025-02-03T19:10:45.0654833Z 
2025-02-03T19:10:45.0655044Z You are in 'detached HEAD' state. You can look around, make experimental
2025-02-03T19:10:45.0656045Z changes and commit them, and you can discard any commits you make in this
2025-02-03T19:10:45.0656788Z state without impacting any branches by switching back to a branch.
2025-02-03T19:10:45.0657199Z 
2025-02-03T19:10:45.0657747Z If you want to create a new branch to retain commits you create, you may
2025-02-03T19:10:45.0658334Z do so (now or later) by using -c with the switch command. Example:
2025-02-03T19:10:45.0658607Z 
2025-02-03T19:10:45.0659117Z   git switch -c <new-branch-name>
2025-02-03T19:10:45.0659330Z 
2025-02-03T19:10:45.0659462Z Or undo this operation with:
2025-02-03T19:10:45.0659707Z 
2025-02-03T19:10:45.0659824Z   git switch -
2025-02-03T19:10:45.0660009Z 
2025-02-03T19:10:45.0660262Z Turn off this advice by setting config variable advice.detachedHead to false
2025-02-03T19:10:45.0660558Z 
2025-02-03T19:10:45.0660973Z HEAD is now at bdca98e Merge 7962e877396abbe1fb42497a53ae21036b1abea9 into b0138adea7758a7a8ac02ed312682586c1a481cf
2025-02-03T19:10:45.0662260Z ##[endgroup]
2025-02-03T19:10:45.0662750Z ##[group]Setting up auth for fetching submodules
2025-02-03T19:10:45.0668080Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-02-03T19:10:45.0701703Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf
2025-02-03T19:10:45.0724157Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf [email protected]:
2025-02-03T19:10:45.0751151Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf [email protected]:
2025-02-03T19:10:45.0772600Z ##[endgroup]
2025-02-03T19:10:45.0773156Z ##[group]Fetching submodules
2025-02-03T19:10:45.0775391Z [command]/usr/bin/git submodule sync
2025-02-03T19:10:45.1085055Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --depth=1
2025-02-03T19:10:45.1256869Z Submodule 'src/bpf/src' (https://github.com/albertlockett/bpf.git) registered for path 'src/bpf/src'
2025-02-03T19:10:45.1259560Z Submodule 'lance' (https://github.com/lancedb/lance.git) registered for path 'src/lance'
2025-02-03T19:10:45.1260426Z Submodule 'src/lancedb' (https://github.com/lancedb/lancedb.git) registered for path 'src/lancedb'
2025-02-03T19:10:45.1261388Z Submodule 'src/ocra' (https://github.com/lancedb/ocra.git) registered for path 'src/ocra'
2025-02-03T19:10:45.1283171Z Cloning into '/home/runner/work/redacted/redacted/src/bpf/src'...
2025-02-03T19:10:45.5004861Z Cloning into '/home/runner/work/redacted/redacted/src/lance'...
2025-02-03T19:10:46.4352181Z Cloning into '/home/runner/work/redacted/redacted/src/lancedb'...
2025-02-03T19:10:47.2724732Z Cloning into '/home/runner/work/redacted/redacted/src/ocra'...
2025-02-03T19:10:47.6891783Z Submodule path 'src/bpf/src': checked out '8ea823f64387d6427984b157e1150322cdd559d5'
2025-02-03T19:10:47.7561523Z Submodule path 'src/lance': checked out 'c73d71706c22d499414ab31e06bd3ac3206e1fbe'
2025-02-03T19:10:48.7080719Z From https://github.com/lancedb/lancedb
2025-02-03T19:10:48.7081284Z  * branch            482f1ee1d3c10df737767390657d88c872752238 -> FETCH_HEAD
2025-02-03T19:10:48.7656124Z Submodule path 'src/lancedb': checked out '482f1ee1d3c10df737767390657d88c872752238'
2025-02-03T19:10:49.5138367Z From https://github.com/lancedb/ocra
2025-02-03T19:10:49.5138886Z  * branch            65b02d1f97db164add2c7e0d846a9be298e9e6b4 -> FETCH_HEAD
2025-02-03T19:10:49.5183388Z Submodule path 'src/ocra': checked out '65b02d1f97db164add2c7e0d846a9be298e9e6b4'
2025-02-03T19:10:49.5196042Z [command]/usr/bin/git submodule foreach git config --local gc.auto 0
2025-02-03T19:10:49.5368438Z Entering 'src/bpf/src'
2025-02-03T19:10:49.5386377Z Entering 'src/lance'
2025-02-03T19:10:49.5404858Z Entering 'src/lancedb'
2025-02-03T19:10:49.5423694Z Entering 'src/ocra'
2025-02-03T19:10:49.5449561Z ##[endgroup]
2025-02-03T19:10:49.5450107Z ##[group]Persisting credentials for submodules
2025-02-03T19:10:49.5455314Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :"
2025-02-03T19:10:49.5622252Z Entering 'src/bpf/src'
2025-02-03T19:10:49.5656192Z Entering 'src/lance'
2025-02-03T19:10:49.5691440Z Entering 'src/lancedb'
2025-02-03T19:10:49.5725672Z Entering 'src/ocra'
2025-02-03T19:10:49.5769161Z [command]/usr/bin/git submodule foreach sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url"
2025-02-03T19:10:49.5938550Z Entering 'src/bpf/src'
2025-02-03T19:10:49.5967754Z file:/home/runner/work/redacted/redacted/.git/modules/src/bpf/src/config	remote.origin.url
2025-02-03T19:10:49.5971568Z Entering 'src/lance'
2025-02-03T19:10:49.6003753Z file:/home/runner/work/redacted/redacted/.git/modules/lance/config	remote.origin.url
2025-02-03T19:10:49.6007141Z Entering 'src/lancedb'
2025-02-03T19:10:49.6036926Z file:/home/runner/work/redacted/redacted/.git/modules/src/lancedb/config	remote.origin.url
2025-02-03T19:10:49.6040054Z Entering 'src/ocra'
2025-02-03T19:10:49.6070089Z file:/home/runner/work/redacted/redacted/.git/modules/src/ocra/config	remote.origin.url
2025-02-03T19:10:49.6108217Z [command]/usr/bin/git submodule foreach git config --local --add 'url.https://github.com/.insteadOf' '[email protected]:'
2025-02-03T19:10:49.6275205Z Entering 'src/bpf/src'
2025-02-03T19:10:49.6293352Z Entering 'src/lance'
2025-02-03T19:10:49.6312255Z Entering 'src/lancedb'
2025-02-03T19:10:49.6330439Z Entering 'src/ocra'
2025-02-03T19:10:49.6358616Z [command]/usr/bin/git submodule foreach git config --local --add 'url.https://github.com/.insteadOf' '[email protected]:'
2025-02-03T19:10:49.6528855Z Entering 'src/bpf/src'
2025-02-03T19:10:49.6547232Z Entering 'src/lance'
2025-02-03T19:10:49.6565462Z Entering 'src/lancedb'
2025-02-03T19:10:49.6583709Z Entering 'src/ocra'
2025-02-03T19:10:49.6609076Z ##[endgroup]
2025-02-03T19:10:49.6639137Z [command]/usr/bin/git log -1 --format=%H
2025-02-03T19:10:49.6658237Z bdca98e03f81b6b59ec2c329d27d6103536c08d5
2025-02-03T19:10:49.6931869Z ##[group]Run actions-rust-lang/setup-rust-toolchain@v1
2025-02-03T19:10:49.6932239Z with:
2025-02-03T19:10:49.6932435Z   toolchain: stable
2025-02-03T19:10:49.6932751Z   cache: true
2025-02-03T19:10:49.6932951Z   cache-on-failure: true
2025-02-03T19:10:49.6933204Z   matcher: true
2025-02-03T19:10:49.6933399Z   rustflags: -D warnings
2025-02-03T19:10:49.6933614Z   override: true
2025-02-03T19:10:49.6933875Z env:
2025-02-03T19:10:49.6934062Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:49.6934294Z   RUST_BACKTRACE: 1
2025-02-03T19:10:49.6934507Z   CC: clang
2025-02-03T19:10:49.6934693Z   CXX: clang++
2025-02-03T19:10:49.6934872Z ##[endgroup]
2025-02-03T19:10:49.7077412Z ##[group]Run : construct rustup command line
2025-02-03T19:10:49.7078119Z ␛[36;1m: construct rustup command line␛[0m
2025-02-03T19:10:49.7078614Z ␛[36;1mecho "targets=$(for t in ${targets//,/ }; do echo -n ' --target' $t; done)" >> $GITHUB_OUTPUT␛[0m
2025-02-03T19:10:49.7079267Z ␛[36;1mecho "components=$(for c in ${components//,/ }; do echo -n ' --component' $c; done)" >> $GITHUB_OUTPUT␛[0m
2025-02-03T19:10:49.7079681Z ␛[36;1mecho "downgrade=" >> $GITHUB_OUTPUT␛[0m
2025-02-03T19:10:49.7110034Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-02-03T19:10:49.7110409Z env:
2025-02-03T19:10:49.7110623Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:49.7110851Z   RUST_BACKTRACE: 1
2025-02-03T19:10:49.7111049Z   CC: clang
2025-02-03T19:10:49.7111269Z   CXX: clang++
2025-02-03T19:10:49.7111454Z   targets: 
2025-02-03T19:10:49.7111659Z   components: 
2025-02-03T19:10:49.7111846Z ##[endgroup]
2025-02-03T19:10:49.7236148Z ##[group]Run if [[ ! -v CARGO_INCREMENTAL ]]; then
2025-02-03T19:10:49.7236544Z ␛[36;1mif [[ ! -v CARGO_INCREMENTAL ]]; then␛[0m
2025-02-03T19:10:49.7236866Z ␛[36;1m  echo "CARGO_INCREMENTAL=0" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7237129Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7237373Z ␛[36;1mif [[ ! -v CARGO_PROFILE_DEV_DEBUG ]]; then␛[0m
2025-02-03T19:10:49.7238079Z ␛[36;1m  echo "CARGO_PROFILE_DEV_DEBUG=0" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7238358Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7238620Z ␛[36;1mif [[ ! -v CARGO_TERM_COLOR ]]; then␛[0m
2025-02-03T19:10:49.7238916Z ␛[36;1m  echo "CARGO_TERM_COLOR=always" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7239229Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7239511Z ␛[36;1mif [[ ! -v RUST_BACKTRACE ]]; then␛[0m
2025-02-03T19:10:49.7239802Z ␛[36;1m  echo "RUST_BACKTRACE=short" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7240341Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7240654Z ␛[36;1mif [[ ( ! -v RUSTFLAGS ) && $NEW_RUSTFLAGS != "" ]]; then␛[0m
2025-02-03T19:10:49.7240994Z ␛[36;1m  echo "RUSTFLAGS=$NEW_RUSTFLAGS" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7241290Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7241506Z ␛[36;1m# Enable faster sparse index on nightly␛[0m
2025-02-03T19:10:49.7241839Z ␛[36;1m# The value is ignored on stable and causes no problems␛[0m
2025-02-03T19:10:49.7242332Z ␛[36;1m# https://internals.rust-lang.org/t/call-for-testing-cargo-sparse-registry/16862␛[0m
2025-02-03T19:10:49.7242810Z ␛[36;1mif [[ ! -v CARGO_UNSTABLE_SPARSE_REGISTRY ]]; then␛[0m
2025-02-03T19:10:49.7243156Z ␛[36;1m  echo "CARGO_UNSTABLE_SPARSE_REGISTRY=true" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7243496Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7243752Z ␛[36;1mif [[ ! -v CARGO_REGISTRIES_CRATES_IO_PROTOCOL ]]; then␛[0m
2025-02-03T19:10:49.7244144Z ␛[36;1m  echo "CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse" >> $GITHUB_ENV␛[0m
2025-02-03T19:10:49.7244508Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7266800Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-02-03T19:10:49.7267230Z env:
2025-02-03T19:10:49.7267425Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:49.7268016Z   RUST_BACKTRACE: 1
2025-02-03T19:10:49.7268251Z   CC: clang
2025-02-03T19:10:49.7268493Z   CXX: clang++
2025-02-03T19:10:49.7268679Z   NEW_RUSTFLAGS: -D warnings
2025-02-03T19:10:49.7268946Z ##[endgroup]
2025-02-03T19:10:49.7339862Z ##[group]Run echo "::add-matcher::/home/runner/work/_actions/actions-rust-lang/setup-rust-toolchain/v1/rust.json"
2025-02-03T19:10:49.7340785Z ␛[36;1mecho "::add-matcher::/home/runner/work/_actions/actions-rust-lang/setup-rust-toolchain/v1/rust.json"␛[0m
2025-02-03T19:10:49.7363859Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-02-03T19:10:49.7364200Z env:
2025-02-03T19:10:49.7364400Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:49.7364647Z   RUST_BACKTRACE: 1
2025-02-03T19:10:49.7364854Z   CC: clang
2025-02-03T19:10:49.7365050Z   CXX: clang++
2025-02-03T19:10:49.7365277Z   CARGO_INCREMENTAL: 0
2025-02-03T19:10:49.7365513Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-02-03T19:10:49.7365752Z   RUSTFLAGS: -D warnings
2025-02-03T19:10:49.7366005Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-02-03T19:10:49.7366306Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-02-03T19:10:49.7366568Z ##[endgroup]
2025-02-03T19:10:49.7458969Z ##[group]Run if ! command -v rustup &> /dev/null ; then
2025-02-03T19:10:49.7459382Z ␛[36;1mif ! command -v rustup &> /dev/null ; then␛[0m
2025-02-03T19:10:49.7459959Z ␛[36;1m  curl --proto '=https' --tlsv1.2 --retry 10 --retry-connrefused -fsSL "https://sh.rustup.rs" | sh -s -- --default-toolchain none -y␛[0m
2025-02-03T19:10:49.7460512Z ␛[36;1m  echo "${CARGO_HOME:-$HOME/.cargo}/bin" >> $GITHUB_PATH␛[0m
2025-02-03T19:10:49.7460870Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7483227Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-02-03T19:10:49.7483566Z env:
2025-02-03T19:10:49.7483770Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:49.7484056Z   RUST_BACKTRACE: 1
2025-02-03T19:10:49.7484241Z   CC: clang
2025-02-03T19:10:49.7484429Z   CXX: clang++
2025-02-03T19:10:49.7484662Z   CARGO_INCREMENTAL: 0
2025-02-03T19:10:49.7484871Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-02-03T19:10:49.7485126Z   RUSTFLAGS: -D warnings
2025-02-03T19:10:49.7485393Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-02-03T19:10:49.7485688Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-02-03T19:10:49.7485938Z ##[endgroup]
2025-02-03T19:10:49.7557988Z ##[group]Run if [[ -z "$toolchain" && ( -f "rust-toolchain" || -f "rust-toolchain.toml" ) ]]
2025-02-03T19:10:49.7558523Z ␛[36;1mif [[ -z "$toolchain" && ( -f "rust-toolchain" || -f "rust-toolchain.toml" ) ]]␛[0m
2025-02-03T19:10:49.7558925Z ␛[36;1mthen␛[0m
2025-02-03T19:10:49.7559186Z ␛[36;1m  # Install the toolchain as specified in the file␛[0m
2025-02-03T19:10:49.7559560Z ␛[36;1m  # rustup show is the old way that implicitly installed a toolchain␛[0m
2025-02-03T19:10:49.7560011Z ␛[36;1m  # rustup toolchain install is the new explicit way␛[0m
2025-02-03T19:10:49.7560674Z ␛[36;1m  # https://github.com/rust-lang/rustup/issues/3635#issuecomment-2343511297␛[0m
2025-02-03T19:10:49.7561118Z ␛[36;1m  rustup show active-toolchain || rustup toolchain install␛[0m
2025-02-03T19:10:49.7561509Z ␛[36;1m  if [[ -n $components ]]; then␛[0m
2025-02-03T19:10:49.7561778Z ␛[36;1m    rustup component add ${components//,/ }␛[0m
2025-02-03T19:10:49.7562052Z ␛[36;1m  fi␛[0m
2025-02-03T19:10:49.7562286Z ␛[36;1m  if [[ -n $targets ]]; then␛[0m
2025-02-03T19:10:49.7562561Z ␛[36;1m    rustup target add ${targets//,/ }␛[0m
2025-02-03T19:10:49.7562810Z ␛[36;1m  fi␛[0m
2025-02-03T19:10:49.7563047Z ␛[36;1melse␛[0m
2025-02-03T19:10:49.7563242Z ␛[36;1m  if [[ -z "$toolchain" ]]␛[0m
2025-02-03T19:10:49.7563483Z ␛[36;1m  then␛[0m
2025-02-03T19:10:49.7563693Z ␛[36;1m    toolchain=stable␛[0m
2025-02-03T19:10:49.7563905Z ␛[36;1m  fi␛[0m
2025-02-03T19:10:49.7564227Z ␛[36;1m  rustup toolchain install ${toolchain//,/ }  --profile minimal --no-self-update␛[0m
2025-02-03T19:10:49.7564679Z ␛[36;1m  # Take the last element from the list␛[0m
2025-02-03T19:10:49.7564942Z ␛[36;1m  if [[ "$override" == "true" ]]␛[0m
2025-02-03T19:10:49.7565184Z ␛[36;1m  then␛[0m
2025-02-03T19:10:49.7565440Z ␛[36;1m    rustup override set ${toolchain//*,/ }␛[0m
2025-02-03T19:10:49.7565687Z ␛[36;1m  fi␛[0m
2025-02-03T19:10:49.7565871Z ␛[36;1mfi␛[0m
2025-02-03T19:10:49.7588696Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-02-03T19:10:49.7589155Z env:
2025-02-03T19:10:49.7589357Z   CARGO_TERM_COLOR: always
2025-02-03T19:10:49.7589752Z   RUST_BACKTRACE: 1
2025-02-03T19:10:49.7589994Z   CC: clang
2025-02-03T19:10:49.7590195Z   CXX: clang++
2025-02-03T19:10:49.7590388Z   CARGO_INCREMENTAL: 0
2025-02-03T19:10:49.7590608Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-02-03T19:10:49.7590861Z   RUSTFLAGS: -D warnings
2025-02-03T19:10:49.7591089Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-02-03T19:10:49.7591403Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-02-03T19:10:49.7591665Z   toolchain: stable
2025-02-03T19:10:49.7591860Z   targets: 
2025-02-03T19:10:49.7592078Z   components: 
2025-02-03T19:10:49.7592267Z   override: true
2025-02-03T19:10:49.7592447Z ##[endgroup]
2025-02-03T19:10:50.9443924Z free(): invalid next size (fast)
2025-02-03T19:10:51.1661517Z /home/runner/work/_temp/4aff9113-ab53-497c-8ddf-ac3f0651f9a1.sh: line 25:  2889 Aborted                 (core dumped) rustup toolchain install ${toolchain//,/ } --profile minimal --no-self-update
2025-02-03T19:10:51.2694173Z ##[error]Process completed with exit code 134.
2025-02-03T19:10:51.2871321Z Post job cleanup.
2025-02-03T19:10:51.2927425Z Post job cleanup.
2025-02-03T19:10:51.3709318Z [command]/usr/bin/git version
2025-02-03T19:10:51.3741135Z git version 2.48.1
2025-02-03T19:10:51.3779889Z Temporarily overriding HOME='/home/runner/work/_temp/2313e33d-dc57-41ab-9e35-f66298453445' before making global git config changes
2025-02-03T19:10:51.3781053Z Adding repository directory to the temporary git global config as a safe directory
2025-02-03T19:10:51.3793040Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/redacted/redacted
2025-02-03T19:10:51.3822344Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-02-03T19:10:51.3851426Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-02-03T19:10:51.4022503Z Entering 'src/bpf/src'
2025-02-03T19:10:51.4054993Z Entering 'src/lance'
2025-02-03T19:10:51.4090082Z Entering 'src/lancedb'
2025-02-03T19:10:51.4124370Z Entering 'src/ocra'
2025-02-03T19:10:51.4172471Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-02-03T19:10:51.4190141Z http.https://github.com/.extraheader
2025-02-03T19:10:51.4200286Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2025-02-03T19:10:51.4224713Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-02-03T19:10:51.4394397Z Entering 'src/bpf/src'
2025-02-03T19:10:51.4413971Z http.https://github.com/.extraheader
2025-02-03T19:10:51.4439615Z Entering 'src/lance'
2025-02-03T19:10:51.4460500Z http.https://github.com/.extraheader
2025-02-03T19:10:51.4485667Z Entering 'src/lancedb'
2025-02-03T19:10:51.4504836Z http.https://github.com/.extraheader
2025-02-03T19:10:51.4532143Z Entering 'src/ocra'
2025-02-03T19:10:51.4552522Z http.https://github.com/.extraheader
2025-02-03T19:10:51.4677198Z Cleaning up orphan processes

Is it regression?

Yes, we did not see this on 2.321.0

Expected behavior

I would expect I can either use the version of rustc that is on the runner image or I can download, install, and use a clean version of rustc using rustup.

For reference, here is a log from a successful build (truncated after the step that is failing in 2.322.0) from 2.321.0:

2025-01-27T16:13:32.7503975Z Current runner version: '2.321.0'
2025-01-27T16:13:32.7514457Z Runner name: 'ubuntu-2404-4x-arm64_1bee6828365e'
2025-01-27T16:13:32.7516154Z Runner group name: 'large-runners'
2025-01-27T16:13:32.7517718Z Machine name: 'runner'
2025-01-27T16:13:32.7543333Z ##[group]VM Image
2025-01-27T16:13:32.7544764Z OS: Linux (arm64)
2025-01-27T16:13:32.7545819Z Source: Marketplace
2025-01-27T16:13:32.7546925Z Publisher: arm
2025-01-27T16:13:32.7548204Z Offer: github_arm_linux_runner_2404
2025-01-27T16:13:32.7549488Z Plan: github_arm_linux_runner_plan_2404
2025-01-27T16:13:32.7550811Z Version: 1.0.10
2025-01-27T16:13:32.7551798Z ##[endgroup]
2025-01-27T16:13:32.7555998Z ##[group]GITHUB_TOKEN Permissions
2025-01-27T16:13:32.7559597Z Actions: write
2025-01-27T16:13:32.7560651Z Attestations: write
2025-01-27T16:13:32.7561682Z Checks: write
2025-01-27T16:13:32.7562783Z Contents: write
2025-01-27T16:13:32.7563796Z Deployments: write
2025-01-27T16:13:32.7564774Z Discussions: write
2025-01-27T16:13:32.7566100Z Issues: write
2025-01-27T16:13:32.7567384Z Metadata: read
2025-01-27T16:13:32.7568554Z Packages: write
2025-01-27T16:13:32.7569573Z Pages: write
2025-01-27T16:13:32.7570646Z PullRequests: write
2025-01-27T16:13:32.7571688Z RepositoryProjects: write
2025-01-27T16:13:32.7572827Z SecurityEvents: write
2025-01-27T16:13:32.7573954Z Statuses: write
2025-01-27T16:13:32.7574938Z ##[endgroup]
2025-01-27T16:13:32.7578573Z Secret source: Actions
2025-01-27T16:13:32.7580035Z Prepare workflow directory
2025-01-27T16:13:32.8102936Z Prepare all required actions
2025-01-27T16:13:32.8152589Z Getting action download info
2025-01-27T16:13:33.0964186Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2025-01-27T16:13:33.2419100Z Download action repository 'actions-rust-lang/setup-rust-toolchain@v1' (SHA:11df97af8e8102fd60b60a77dfbf58d40cd843b8)
2025-01-27T16:13:33.5204896Z Download action repository 'Swatinem/rust-cache@v2' (SHA:f0deed1e0edfc6a9be95417288c0e1099b1eeec3)
2025-01-27T16:13:33.8552766Z Download action repository 'taiki-e/install-action@cargo-llvm-cov' (SHA:389abf9c6d64ac884567c06213051f87398d3b3c)
2025-01-27T16:13:34.2527868Z Download action repository 'codecov/codecov-action@v4' (SHA:b9fd7d16f6d7d1b5d2bec1a2887e65ceed900238)
2025-01-27T16:13:34.7380640Z Getting action download info
2025-01-27T16:13:35.0511307Z Complete job name: build (ubuntu-2404-4x-arm64)
2025-01-27T16:13:35.1686217Z ##[group]Run actions/checkout@v4
2025-01-27T16:13:35.1688176Z with:
2025-01-27T16:13:35.1689142Z   submodules: true
2025-01-27T16:13:35.1690190Z   repository: lancedb/redacted
2025-01-27T16:13:35.1691723Z   token: ***
2025-01-27T16:13:35.1692676Z   ssh-strict: true
2025-01-27T16:13:35.1693668Z   ssh-user: git
2025-01-27T16:13:35.1694717Z   persist-credentials: true
2025-01-27T16:13:35.1695843Z   clean: true
2025-01-27T16:13:35.1696876Z   sparse-checkout-cone-mode: true
2025-01-27T16:13:35.1698232Z   fetch-depth: 1
2025-01-27T16:13:35.1699230Z   fetch-tags: false
2025-01-27T16:13:35.1700243Z   show-progress: true
2025-01-27T16:13:35.1701310Z   lfs: false
2025-01-27T16:13:35.1702286Z   set-safe-directory: true
2025-01-27T16:13:35.1703886Z env:
2025-01-27T16:13:35.1704847Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:35.1705986Z   RUST_BACKTRACE: 1
2025-01-27T16:13:35.1706990Z ##[endgroup]
2025-01-27T16:13:35.2998079Z Syncing repository: lancedb/redacted
2025-01-27T16:13:35.3001378Z ##[group]Getting Git version info
2025-01-27T16:13:35.3002934Z Working directory is '/home/runner/work/redacted/redacted'
2025-01-27T16:13:35.3005227Z [command]/usr/bin/git version
2025-01-27T16:13:35.8050681Z git version 2.48.1
2025-01-27T16:13:35.8079430Z ##[endgroup]
2025-01-27T16:13:35.8099278Z Temporarily overriding HOME='/home/runner/work/_temp/c35bb8e2-49d8-49ab-9b9e-8287dd7b48e1' before making global git config changes
2025-01-27T16:13:35.8103036Z Adding repository directory to the temporary git global config as a safe directory
2025-01-27T16:13:35.8105761Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/redacted/redacted
2025-01-27T16:13:35.8364603Z Deleting the contents of '/home/runner/work/redacted/redacted'
2025-01-27T16:13:35.8369775Z ##[group]Initializing the repository
2025-01-27T16:13:35.8374813Z [command]/usr/bin/git init /home/runner/work/redacted/redacted
2025-01-27T16:13:36.2247178Z hint: Using 'master' as the name for the initial branch. This default branch name
2025-01-27T16:13:36.2249873Z hint: is subject to change. To configure the initial branch name to use in all
2025-01-27T16:13:36.2252182Z hint: of your new repositories, which will suppress this warning, call:
2025-01-27T16:13:36.2253874Z hint:
2025-01-27T16:13:36.2255030Z hint: 	git config --global init.defaultBranch <name>
2025-01-27T16:13:36.2256453Z hint:
2025-01-27T16:13:36.2257815Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2025-01-27T16:13:36.2260306Z hint: 'development'. The just-created branch can be renamed via this command:
2025-01-27T16:13:36.2262098Z hint:
2025-01-27T16:13:36.2263048Z hint: 	git branch -m <name>
2025-01-27T16:13:36.2560647Z Initialized empty Git repository in /home/runner/work/redacted/redacted/.git/
2025-01-27T16:13:36.2575715Z [command]/usr/bin/git remote add origin https://github.com/lancedb/redacted
2025-01-27T16:13:36.2791615Z ##[endgroup]
2025-01-27T16:13:36.2793365Z ##[group]Disabling automatic garbage collection
2025-01-27T16:13:36.2796742Z [command]/usr/bin/git config --local gc.auto 0
2025-01-27T16:13:36.2829721Z ##[endgroup]
2025-01-27T16:13:36.2831332Z ##[group]Setting up auth
2025-01-27T16:13:36.2838550Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-01-27T16:13:36.2869406Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-01-27T16:13:36.8670012Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-01-27T16:13:36.8700932Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-01-27T16:13:36.8916597Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-01-27T16:13:36.8953692Z ##[endgroup]
2025-01-27T16:13:36.8954869Z ##[group]Fetching the repository
2025-01-27T16:13:36.8963397Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin +25a5057d25ae4b4b8223fda756c24c230824f7d2:refs/remotes/origin/main
2025-01-27T16:13:38.4116023Z From https://github.com/lancedb/redacted
2025-01-27T16:13:38.4116885Z  * [new ref]         25a5057d25ae4b4b8223fda756c24c230824f7d2 -> origin/main
2025-01-27T16:13:38.4319039Z ##[endgroup]
2025-01-27T16:13:38.4319662Z ##[group]Determining the checkout info
2025-01-27T16:13:38.4320308Z ##[endgroup]
2025-01-27T16:13:38.4325357Z [command]/usr/bin/git sparse-checkout disable
2025-01-27T16:13:38.4516138Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2025-01-27T16:13:38.4549210Z ##[group]Checking out the ref
2025-01-27T16:13:38.4553348Z [command]/usr/bin/git checkout --progress --force -B main refs/remotes/origin/main
2025-01-27T16:13:38.5897216Z Switched to a new branch 'main'
2025-01-27T16:13:38.5898139Z branch 'main' set up to track 'origin/main'.
2025-01-27T16:13:38.5910393Z ##[endgroup]
2025-01-27T16:13:38.5911023Z ##[group]Setting up auth for fetching submodules
2025-01-27T16:13:38.5917310Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-01-27T16:13:38.5959505Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf
2025-01-27T16:13:38.5988591Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf [email protected]:
2025-01-27T16:13:38.6020286Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf [email protected]:
2025-01-27T16:13:38.6046455Z ##[endgroup]
2025-01-27T16:13:38.6047042Z ##[group]Fetching submodules
2025-01-27T16:13:38.6051476Z [command]/usr/bin/git submodule sync
2025-01-27T16:13:38.6423943Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --depth=1
2025-01-27T16:13:38.6633290Z Submodule 'src/bpf/src' (https://github.com/albertlockett/bpf.git) registered for path 'src/bpf/src'
2025-01-27T16:13:38.6636073Z Submodule 'lance' (https://github.com/lancedb/lance.git) registered for path 'src/lance'
2025-01-27T16:13:38.6639424Z Submodule 'src/lancedb' (https://github.com/lancedb/lancedb.git) registered for path 'src/lancedb'
2025-01-27T16:13:38.6645025Z Submodule 'src/ocra' (https://github.com/lancedb/ocra.git) registered for path 'src/ocra'
2025-01-27T16:13:38.6671405Z Cloning into '/home/runner/work/redacted/redacted/src/bpf/src'...
2025-01-27T16:13:39.1709842Z Cloning into '/home/runner/work/redacted/redacted/src/lance'...
2025-01-27T16:13:40.0785503Z Cloning into '/home/runner/work/redacted/redacted/src/lancedb'...
2025-01-27T16:13:40.9416552Z Cloning into '/home/runner/work/redacted/redacted/src/ocra'...
2025-01-27T16:13:41.4009308Z Submodule path 'src/bpf/src': checked out '8ea823f64387d6427984b157e1150322cdd559d5'
2025-01-27T16:13:41.4940616Z Submodule path 'src/lance': checked out '58c5e27ff1534ae82041eeb73c55a555da2bcd65'
2025-01-27T16:13:41.5719227Z Submodule path 'src/lancedb': checked out 'd999d72c8d99acedbabb769344edb949d24e7994'
2025-01-27T16:13:42.5839665Z From https://github.com/lancedb/ocra
2025-01-27T16:13:42.5840311Z  * branch            65b02d1f97db164add2c7e0d846a9be298e9e6b4 -> FETCH_HEAD
2025-01-27T16:13:42.5893058Z Submodule path 'src/ocra': checked out '65b02d1f97db164add2c7e0d846a9be298e9e6b4'
2025-01-27T16:13:42.5909479Z [command]/usr/bin/git submodule foreach git config --local gc.auto 0
2025-01-27T16:13:42.6112061Z Entering 'src/bpf/src'
2025-01-27T16:13:42.6134129Z Entering 'src/lance'
2025-01-27T16:13:42.6155981Z Entering 'src/lancedb'
2025-01-27T16:13:42.6176190Z Entering 'src/ocra'
2025-01-27T16:13:42.6207581Z ##[endgroup]
2025-01-27T16:13:42.6208580Z ##[group]Persisting credentials for submodules
2025-01-27T16:13:42.6213985Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :"
2025-01-27T16:13:42.6415895Z Entering 'src/bpf/src'
2025-01-27T16:13:42.6453949Z Entering 'src/lance'
2025-01-27T16:13:42.6493381Z Entering 'src/lancedb'
2025-01-27T16:13:42.6532969Z Entering 'src/ocra'
2025-01-27T16:13:42.6588287Z [command]/usr/bin/git submodule foreach sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url"
2025-01-27T16:13:42.6791374Z Entering 'src/bpf/src'
2025-01-27T16:13:42.6826074Z file:/home/runner/work/redacted/redacted/.git/modules/src/bpf/src/config	remote.origin.url
2025-01-27T16:13:42.6830346Z Entering 'src/lance'
2025-01-27T16:13:42.6866558Z file:/home/runner/work/redacted/redacted/.git/modules/lance/config	remote.origin.url
2025-01-27T16:13:42.6870269Z Entering 'src/lancedb'
2025-01-27T16:13:42.6909048Z file:/home/runner/work/redacted/redacted/.git/modules/src/lancedb/config	remote.origin.url
2025-01-27T16:13:42.6913485Z Entering 'src/ocra'
2025-01-27T16:13:42.6949406Z file:/home/runner/work/redacted/redacted/.git/modules/src/ocra/config	remote.origin.url
2025-01-27T16:13:42.6998929Z [command]/usr/bin/git submodule foreach git config --local --add 'url.https://github.com/.insteadOf' '[email protected]:'
2025-01-27T16:13:42.7204079Z Entering 'src/bpf/src'
2025-01-27T16:13:42.7227196Z Entering 'src/lance'
2025-01-27T16:13:42.7250145Z Entering 'src/lancedb'
2025-01-27T16:13:42.7273149Z Entering 'src/ocra'
2025-01-27T16:13:42.7309708Z [command]/usr/bin/git submodule foreach git config --local --add 'url.https://github.com/.insteadOf' '[email protected]:'
2025-01-27T16:13:42.7526042Z Entering 'src/bpf/src'
2025-01-27T16:13:42.7549819Z Entering 'src/lance'
2025-01-27T16:13:42.7573440Z Entering 'src/lancedb'
2025-01-27T16:13:42.7595333Z Entering 'src/ocra'
2025-01-27T16:13:42.7629698Z ##[endgroup]
2025-01-27T16:13:42.7668571Z [command]/usr/bin/git log -1 --format=%H
2025-01-27T16:13:42.7693284Z 25a5057d25ae4b4b8223fda756c24c230824f7d2
2025-01-27T16:13:42.8062762Z ##[group]Run actions-rust-lang/setup-rust-toolchain@v1
2025-01-27T16:13:42.8063307Z with:
2025-01-27T16:13:42.8063632Z   toolchain: stable
2025-01-27T16:13:42.8064010Z   cache-workspaces: src/rust
2025-01-27T16:13:42.8064430Z   rustflags: -C debuginfo=1
2025-01-27T16:13:42.8064848Z   components: rustfmt,clippy
2025-01-27T16:13:42.8065260Z   cache: true
2025-01-27T16:13:42.8065616Z   cache-on-failure: true
2025-01-27T16:13:42.8066001Z   matcher: true
2025-01-27T16:13:42.8066348Z   override: true
2025-01-27T16:13:42.8066690Z env:
2025-01-27T16:13:42.8067024Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:42.8067417Z   RUST_BACKTRACE: 1
2025-01-27T16:13:42.8067770Z ##[endgroup]
2025-01-27T16:13:42.8236108Z ##[group]Run : construct rustup command line
2025-01-27T16:13:42.8236642Z ␛[36;1m: construct rustup command line␛[0m
2025-01-27T16:13:42.8237294Z ␛[36;1mecho "targets=$(for t in ${targets//,/ }; do echo -n ' --target' $t; done)" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:42.8238416Z ␛[36;1mecho "components=$(for c in ${components//,/ }; do echo -n ' --component' $c; done)" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:42.8239113Z ␛[36;1mecho "downgrade=" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:42.8280075Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:42.8280602Z env:
2025-01-27T16:13:42.8280920Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:42.8281283Z   RUST_BACKTRACE: 1
2025-01-27T16:13:42.8281616Z   targets: 
2025-01-27T16:13:42.8281944Z   components: rustfmt,clippy
2025-01-27T16:13:42.8282317Z ##[endgroup]
2025-01-27T16:13:42.8438824Z ##[group]Run if [[ ! -v CARGO_INCREMENTAL ]]; then
2025-01-27T16:13:42.8439464Z ␛[36;1mif [[ ! -v CARGO_INCREMENTAL ]]; then␛[0m
2025-01-27T16:13:42.8439978Z ␛[36;1m  echo "CARGO_INCREMENTAL=0" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8440430Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8440800Z ␛[36;1mif [[ ! -v CARGO_PROFILE_DEV_DEBUG ]]; then␛[0m
2025-01-27T16:13:42.8441321Z ␛[36;1m  echo "CARGO_PROFILE_DEV_DEBUG=0" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8441797Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8442143Z ␛[36;1mif [[ ! -v CARGO_TERM_COLOR ]]; then␛[0m
2025-01-27T16:13:42.8442634Z ␛[36;1m  echo "CARGO_TERM_COLOR=always" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8443096Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8443436Z ␛[36;1mif [[ ! -v RUST_BACKTRACE ]]; then␛[0m
2025-01-27T16:13:42.8443922Z ␛[36;1m  echo "RUST_BACKTRACE=short" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8444371Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8444771Z ␛[36;1mif [[ ( ! -v RUSTFLAGS ) && $NEW_RUSTFLAGS != "" ]]; then␛[0m
2025-01-27T16:13:42.8445329Z ␛[36;1m  echo "RUSTFLAGS=$NEW_RUSTFLAGS" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8445807Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8446183Z ␛[36;1m# Enable faster sparse index on nightly␛[0m
2025-01-27T16:13:42.8446725Z ␛[36;1m# The value is ignored on stable and causes no problems␛[0m
2025-01-27T16:13:42.8447486Z ␛[36;1m# https://internals.rust-lang.org/t/call-for-testing-cargo-sparse-registry/16862␛[0m
2025-01-27T16:13:42.8448426Z ␛[36;1mif [[ ! -v CARGO_UNSTABLE_SPARSE_REGISTRY ]]; then␛[0m
2025-01-27T16:13:42.8449022Z ␛[36;1m  echo "CARGO_UNSTABLE_SPARSE_REGISTRY=true" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8449534Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8449933Z ␛[36;1mif [[ ! -v CARGO_REGISTRIES_CRATES_IO_PROTOCOL ]]; then␛[0m
2025-01-27T16:13:42.8450572Z ␛[36;1m  echo "CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:42.8451120Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8486481Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:42.8486992Z env:
2025-01-27T16:13:42.8487306Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:42.8487680Z   RUST_BACKTRACE: 1
2025-01-27T16:13:42.8488253Z   NEW_RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:42.8489011Z ##[endgroup]
2025-01-27T16:13:42.8576917Z ##[group]Run echo "::add-matcher::/home/runner/work/_actions/actions-rust-lang/setup-rust-toolchain/v1/rust.json"
2025-01-27T16:13:42.8578600Z ␛[36;1mecho "::add-matcher::/home/runner/work/_actions/actions-rust-lang/setup-rust-toolchain/v1/rust.json"␛[0m
2025-01-27T16:13:42.8610846Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:42.8611347Z env:
2025-01-27T16:13:42.8611667Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:42.8612039Z   RUST_BACKTRACE: 1
2025-01-27T16:13:42.8612381Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:42.8612747Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:42.8613142Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:42.8613545Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:42.8614001Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:42.8614441Z ##[endgroup]
2025-01-27T16:13:42.8766743Z ##[group]Run if ! command -v rustup &> /dev/null ; then
2025-01-27T16:13:42.8767330Z ␛[36;1mif ! command -v rustup &> /dev/null ; then␛[0m
2025-01-27T16:13:42.8768388Z ␛[36;1m  curl --proto '=https' --tlsv1.2 --retry 10 --retry-connrefused -fsSL "https://sh.rustup.rs" | sh -s -- --default-toolchain none -y␛[0m
2025-01-27T16:13:42.8769310Z ␛[36;1m  echo "${CARGO_HOME:-$HOME/.cargo}/bin" >> $GITHUB_PATH␛[0m
2025-01-27T16:13:42.8769801Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8802944Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:42.8803453Z env:
2025-01-27T16:13:42.8803764Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:42.8804135Z   RUST_BACKTRACE: 1
2025-01-27T16:13:42.8804481Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:42.8804846Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:42.8805231Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:42.8805636Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:42.8806096Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:42.8806528Z ##[endgroup]
2025-01-27T16:13:42.8897020Z ##[group]Run if [[ -z "$toolchain" && ( -f "rust-toolchain" || -f "rust-toolchain.toml" ) ]]
2025-01-27T16:13:42.8897844Z ␛[36;1mif [[ -z "$toolchain" && ( -f "rust-toolchain" || -f "rust-toolchain.toml" ) ]]␛[0m
2025-01-27T16:13:42.8898832Z ␛[36;1mthen␛[0m
2025-01-27T16:13:42.8899234Z ␛[36;1m  # Install the toolchain as specified in the file␛[0m
2025-01-27T16:13:42.8899860Z ␛[36;1m  # rustup show is the old way that implicitly installed a toolchain␛[0m
2025-01-27T16:13:42.8900490Z ␛[36;1m  # rustup toolchain install is the new explicit way␛[0m
2025-01-27T16:13:42.8901205Z ␛[36;1m  # https://github.com/rust-lang/rustup/issues/3635#issuecomment-2343511297␛[0m
2025-01-27T16:13:42.8901915Z ␛[36;1m  rustup show active-toolchain || rustup toolchain install␛[0m
2025-01-27T16:13:42.8902466Z ␛[36;1m  if [[ -n $components ]]; then␛[0m
2025-01-27T16:13:42.8902928Z ␛[36;1m    rustup component add ${components//,/ }␛[0m
2025-01-27T16:13:42.8903378Z ␛[36;1m  fi␛[0m
2025-01-27T16:13:42.8903714Z ␛[36;1m  if [[ -n $targets ]]; then␛[0m
2025-01-27T16:13:42.8904150Z ␛[36;1m    rustup target add ${targets//,/ }␛[0m
2025-01-27T16:13:42.8904592Z ␛[36;1m  fi␛[0m
2025-01-27T16:13:42.8904912Z ␛[36;1melse␛[0m
2025-01-27T16:13:42.8905249Z ␛[36;1m  if [[ -z "$toolchain" ]]␛[0m
2025-01-27T16:13:42.8905647Z ␛[36;1m  then␛[0m
2025-01-27T16:13:42.8905975Z ␛[36;1m    toolchain=stable␛[0m
2025-01-27T16:13:42.8906359Z ␛[36;1m  fi␛[0m
2025-01-27T16:13:42.8907068Z ␛[36;1m  rustup toolchain install ${toolchain//,/ }  --component rustfmt --component clippy --profile minimal --no-self-update␛[0m
2025-01-27T16:13:42.8908075Z ␛[36;1m  # Take the last element from the list␛[0m
2025-01-27T16:13:42.8908578Z ␛[36;1m  if [[ "$override" == "true" ]]␛[0m
2025-01-27T16:13:42.8908991Z ␛[36;1m  then␛[0m
2025-01-27T16:13:42.8909364Z ␛[36;1m    rustup override set ${toolchain//*,/ }␛[0m
2025-01-27T16:13:42.8909800Z ␛[36;1m  fi␛[0m
2025-01-27T16:13:42.8910119Z ␛[36;1mfi␛[0m
2025-01-27T16:13:42.8942857Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:42.8943368Z env:
2025-01-27T16:13:42.8943988Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:42.8944359Z   RUST_BACKTRACE: 1
2025-01-27T16:13:42.8944703Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:42.8945066Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:42.8945456Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:42.8946094Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:42.8946580Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:42.8947020Z   toolchain: stable
2025-01-27T16:13:42.8947346Z   targets: 
2025-01-27T16:13:42.8947672Z   components: rustfmt,clippy
2025-01-27T16:13:42.8948334Z   override: true
2025-01-27T16:13:42.8948663Z ##[endgroup]
2025-01-27T16:13:53.0516802Z info: syncing channel updates for 'stable-aarch64-unknown-linux-gnu'
2025-01-27T16:13:53.3374775Z info: latest update on 2025-01-09, rust version 1.84.0 (9fc6b4312 2025-01-07)
2025-01-27T16:13:53.3932775Z info: component 'clippy' for target 'aarch64-unknown-linux-gnu' is up to date
2025-01-27T16:13:53.3934120Z info: component 'rustfmt' for target 'aarch64-unknown-linux-gnu' is up to date
2025-01-27T16:13:53.3970026Z 
2025-01-27T16:13:53.4097660Z   stable-aarch64-unknown-linux-gnu unchanged - rustc 1.84.0 (9fc6b4312 2025-01-07)
2025-01-27T16:13:53.4098523Z 
2025-01-27T16:13:53.4153622Z info: override toolchain for '/home/runner/work/redacted/redacted' set to 'stable-aarch64-unknown-linux-gnu'
2025-01-27T16:13:53.4195734Z ##[group]Run echo "rustc-version=$(rustc --version)" >> $GITHUB_OUTPUT
2025-01-27T16:13:53.4196453Z ␛[36;1mecho "rustc-version=$(rustc --version)" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:53.4196988Z ␛[36;1mrustc --version --verbose␛[0m
2025-01-27T16:13:53.4197508Z ␛[36;1mecho "cargo-version=$(cargo --version)" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:53.4198338Z ␛[36;1mcargo --version --verbose␛[0m
2025-01-27T16:13:53.4198866Z ␛[36;1mecho "rustup-version=$(rustup --version)" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:53.4199404Z ␛[36;1mrustup --version␛[0m
2025-01-27T16:13:53.4199764Z ␛[36;1m␛[0m
2025-01-27T16:13:53.4200354Z ␛[36;1mDATE=$(rustc --version --verbose | sed -ne 's/^commit-date: \(20[0-9][0-9]\)-\([01][0-9]\)-\([0-3][0-9]\)$/\1\2\3/p')␛[0m
2025-01-27T16:13:53.4201191Z ␛[36;1mHASH=$(rustc --version --verbose | sed -ne 's/^commit-hash: //p')␛[0m
2025-01-27T16:13:53.4201866Z ␛[36;1mecho "cachekey=$(echo $DATE$HASH | head -c12)" >> $GITHUB_OUTPUT␛[0m
2025-01-27T16:13:53.4236988Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:53.4237490Z env:
2025-01-27T16:13:53.4237807Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:53.4238427Z   RUST_BACKTRACE: 1
2025-01-27T16:13:53.4238768Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:53.4239128Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:53.4239514Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:53.4239908Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:53.4240365Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:53.4240822Z ##[endgroup]
2025-01-27T16:13:53.5360145Z rustc 1.84.0 (9fc6b4312 2025-01-07)
2025-01-27T16:13:53.5360622Z binary: rustc
2025-01-27T16:13:53.5361076Z commit-hash: 9fc6b43126469e3858e2fe86cafb4f0fd5068869
2025-01-27T16:13:53.5361556Z commit-date: 2025-01-07
2025-01-27T16:13:53.5361939Z host: aarch64-unknown-linux-gnu
2025-01-27T16:13:53.5362339Z release: 1.84.0
2025-01-27T16:13:53.8330380Z LLVM version: 19.1.5
2025-01-27T16:13:56.6839985Z cargo 1.84.0 (66221abde 2024-11-19)
2025-01-27T16:13:56.6840481Z release: 1.84.0
2025-01-27T16:13:56.6840913Z commit-hash: 66221abdeca2002d318fde6efff516aab091df0e
2025-01-27T16:13:56.6841404Z commit-date: 2024-11-19
2025-01-27T16:13:56.6841778Z host: aarch64-unknown-linux-gnu
2025-01-27T16:13:56.6842188Z libgit2: 1.8.1 (sys:0.19.0 vendored)
2025-01-27T16:13:56.6842713Z libcurl: 8.9.0-DEV (sys:0.4.74+curl-8.9.0 vendored ssl:OpenSSL/1.1.1w)
2025-01-27T16:13:56.6843263Z ssl: OpenSSL 1.1.1w  11 Sep 2023
2025-01-27T16:13:56.6843661Z os: Ubuntu 24.4.0 (noble) [64-bit]
2025-01-27T16:13:56.6874699Z info: This is the version for the rustup toolchain manager, not the rustc compiler.
2025-01-27T16:13:56.7387049Z info: The currently active `rustc` version is `rustc 1.84.0 (9fc6b4312 2025-01-07)`
2025-01-27T16:13:56.7424584Z info: This is the version for the rustup toolchain manager, not the rustc compiler.
2025-01-27T16:13:56.7425581Z rustup 1.27.1 (54dd3d00f 2024-04-24)
2025-01-27T16:13:56.7926876Z info: The currently active `rustc` version is `rustc 1.84.0 (9fc6b4312 2025-01-07)`
2025-01-27T16:13:56.9367296Z ##[group]Run # Not all versions support setting CARGO_REGISTRIES_CRATES_IO_PROTOCOL
2025-01-27T16:13:56.9368348Z ␛[36;1m# Not all versions support setting CARGO_REGISTRIES_CRATES_IO_PROTOCOL␛[0m
2025-01-27T16:13:56.9369098Z ␛[36;1m# On versions 1.66, 1.67, and 1.68.0-nightly the value "sparse" is still unstable.␛[0m
2025-01-27T16:13:56.9369883Z ␛[36;1m# https://github.com/dtolnay/rust-toolchain/pull/69#discussion_r1107268108␛[0m
2025-01-27T16:13:56.9370669Z ␛[36;1m# If we detect an incompatible value, set it to "git" which is always supported.␛[0m
2025-01-27T16:13:56.9371634Z ␛[36;1mif [[ "rustc 1.84.0 (9fc6b4312 2025-01-07)" =~ ^rustc\ (1\.6[67]\.|1\.68\.0-nightly) && "${CARGO_REGISTRIES_CRATES_IO_PROTOCOL}" == "sparse" ]]; then␛[0m
2025-01-27T16:13:56.9372511Z ␛[36;1m  echo "Downgrade cargo registry protocol to git"␛[0m
2025-01-27T16:13:56.9373120Z ␛[36;1m  echo "CARGO_REGISTRIES_CRATES_IO_PROTOCOL=git" >> $GITHUB_ENV␛[0m
2025-01-27T16:13:56.9373672Z ␛[36;1mfi␛[0m
2025-01-27T16:13:56.9408877Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-01-27T16:13:56.9409369Z env:
2025-01-27T16:13:56.9409684Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:56.9410053Z   RUST_BACKTRACE: 1
2025-01-27T16:13:56.9410399Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:56.9410755Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:56.9411142Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:56.9411544Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:56.9411995Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:56.9412431Z ##[endgroup]
2025-01-27T16:13:56.9526266Z ##[group]Run Swatinem/rust-cache@v2
2025-01-27T16:13:56.9526702Z with:
2025-01-27T16:13:56.9527047Z   workspaces: src/rust
2025-01-27T16:13:56.9527407Z   cache-on-failure: true
2025-01-27T16:13:56.9527778Z   prefix-key: v0-rust
2025-01-27T16:13:56.9528295Z   cache-targets: true
2025-01-27T16:13:56.9528658Z   cache-all-crates: false
2025-01-27T16:13:56.9529019Z   save-if: true
2025-01-27T16:13:56.9529373Z   cache-provider: github
2025-01-27T16:13:56.9529743Z   cache-bin: true
2025-01-27T16:13:56.9530071Z   lookup-only: false
2025-01-27T16:13:56.9530411Z env:
2025-01-27T16:13:56.9530716Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:56.9531088Z   RUST_BACKTRACE: 1
2025-01-27T16:13:56.9531423Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:56.9531786Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:56.9532185Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:56.9532580Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:56.9533045Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:56.9533488Z ##[endgroup]
2025-01-27T16:13:58.4568980Z ##[group]Cache Configuration
2025-01-27T16:13:58.4570022Z Cache Provider:
2025-01-27T16:13:58.4570821Z     github
2025-01-27T16:13:58.4571596Z Workspaces:
2025-01-27T16:13:58.4573240Z     /home/runner/work/redacted/redacted/src/rust
2025-01-27T16:13:58.4573726Z Cache Paths:
2025-01-27T16:13:58.4574070Z     /home/runner/.cargo/bin
2025-01-27T16:13:58.4574497Z     /home/runner/.cargo/.crates.toml
2025-01-27T16:13:58.4574936Z     /home/runner/.cargo/.crates2.json
2025-01-27T16:13:58.4575364Z     /home/runner/.cargo/registry
2025-01-27T16:13:58.4575770Z     /home/runner/.cargo/git
2025-01-27T16:13:58.4576209Z     /home/runner/work/redacted/redacted/src/rust/target
2025-01-27T16:13:58.4576673Z Restore Key:
2025-01-27T16:13:58.4577020Z     v0-rust-build-Linux-99e519fa
2025-01-27T16:13:58.4577415Z Cache Key:
2025-01-27T16:13:58.4577766Z     v0-rust-build-Linux-99e519fa-c11580f9
2025-01-27T16:13:58.4578425Z .. Prefix:
2025-01-27T16:13:58.4578761Z   - v0-rust-build-Linux
2025-01-27T16:13:58.4579157Z .. Environment considered:
2025-01-27T16:13:58.4579766Z   - Rust Version: 1.84.0 aarch64-unknown-linux-gnu (9fc6b43126469e3858e2fe86cafb4f0fd5068869)
2025-01-27T16:13:58.4580912Z   - CARGO_INCREMENTAL
2025-01-27T16:13:58.4581941Z   - CARGO_PROFILE_DEV_DEBUG
2025-01-27T16:13:58.4582515Z   - CARGO_REGISTRIES_CRATES_IO_PROTOCOL
2025-01-27T16:13:58.4583135Z   - CARGO_TERM_COLOR
2025-01-27T16:13:58.4583960Z   - CARGO_UNSTABLE_SPARSE_REGISTRY
2025-01-27T16:13:58.4584578Z   - RUST_BACKTRACE
2025-01-27T16:13:58.4585102Z   - RUSTFLAGS
2025-01-27T16:13:58.4585578Z .. Lockfiles considered:
2025-01-27T16:13:58.4586177Z   - /home/runner/work/redacted/redacted/.cargo/config.toml
2025-01-27T16:13:58.4586933Z   - /home/runner/work/redacted/redacted/src/rust/.cargo/config.toml
2025-01-27T16:13:58.4587550Z   - /home/runner/work/redacted/redacted/src/rust/Cargo.lock
2025-01-27T16:13:58.4588322Z   - /home/runner/work/redacted/redacted/src/rust/catalog/Cargo.toml
2025-01-27T16:13:58.4588955Z   - /home/runner/work/redacted/redacted/src/rust/core/Cargo.toml
2025-01-27T16:13:58.4589623Z   - /home/runner/work/redacted/redacted/src/rust/execution-plans/Cargo.toml
2025-01-27T16:13:58.4590320Z   - /home/runner/work/redacted/redacted/src/rust/indexer/Cargo.toml
2025-01-27T16:13:58.4590995Z   - /home/runner/work/redacted/redacted/src/rust/plan-executor/Cargo.toml
2025-01-27T16:13:58.4591773Z   - /home/runner/work/redacted/redacted/src/rust/router/Cargo.toml
2025-01-27T16:13:58.4592466Z   - /home/runner/work/redacted/redacted/src/rust/redacted-caching/Cargo.toml
2025-01-27T16:13:58.4593245Z   - /home/runner/work/redacted/redacted/src/rust/redacted-consistent-hashing/Cargo.toml
2025-01-27T16:13:58.4594066Z   - /home/runner/work/redacted/redacted/src/rust/redacted-interceptors/Cargo.toml
2025-01-27T16:13:58.4594800Z   - /home/runner/work/redacted/redacted/src/rust/redacted-k8s/Cargo.toml
2025-01-27T16:13:58.4595494Z   - /home/runner/work/redacted/redacted/src/rust/redacted-protos/Cargo.toml
2025-01-27T16:13:58.4596258Z   - /home/runner/work/redacted/redacted/src/rust/vector-index-extensions/Cargo.toml
2025-01-27T16:13:58.4597179Z ##[endgroup]
2025-01-27T16:13:58.4597368Z 
2025-01-27T16:13:58.4597530Z ... Restoring cache ...
2025-01-27T16:13:59.5216000Z Cache Size: ~0 MB (1376 B)
2025-01-27T16:13:59.5246072Z [command]/usr/bin/tar -xf /home/runner/work/_temp/3f1c3b29-44a9-4cfc-a1be-1cc07d5a2253/cache.tzst -P -C /home/runner/work/redacted/redacted --use-compress-program unzstd
2025-01-27T16:13:59.5502125Z Cache restored successfully
2025-01-27T16:13:59.5506575Z Restored from cache key "v0-rust-build-Linux-99e519fa-c11580f9" full match: true.
2025-01-27T16:13:59.5623339Z ##[group]Run sudo apt update
2025-01-27T16:13:59.5623831Z ␛[36;1msudo apt update␛[0m
2025-01-27T16:13:59.5624307Z ␛[36;1msudo apt install -y protobuf-compiler libssl-dev␛[0m
2025-01-27T16:13:59.5661229Z shell: /usr/bin/bash -e {0}
2025-01-27T16:13:59.5661632Z env:
2025-01-27T16:13:59.5661961Z   CARGO_TERM_COLOR: always
2025-01-27T16:13:59.5662338Z   RUST_BACKTRACE: 1
2025-01-27T16:13:59.5662697Z   CARGO_INCREMENTAL: 0
2025-01-27T16:13:59.5663079Z   CARGO_PROFILE_DEV_DEBUG: 0
2025-01-27T16:13:59.5663481Z   RUSTFLAGS: -C debuginfo=1
2025-01-27T16:13:59.5663920Z   CARGO_UNSTABLE_SPARSE_REGISTRY: true
2025-01-27T16:13:59.5664392Z   CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
2025-01-27T16:13:59.5664853Z   CACHE_ON_FAILURE: true
2025-01-27T16:13:59.5665211Z ##[endgroup]

Actual behavior

The failure is not consistent. Slightly less than half of our builds succeed. The builds that fail will sometimes fail during the rustup stage, sometimes after that when printing the current rustc version, and sometimes will fail during the compilation itself. Everytime it fails it is some strange segmentation fault. I suspect the rustc on the image is corrupt somehow.

The first failure appeared on January 29th, around 9:30 AM PST.

We sometimes get failures even when we try and use rustup to install an older version of rustc. I believe this is because the first thing that rustup does is list the current rust version and this can raise a segmentation fault.

Repro steps

The repository I am encountering this in is a private repository but I can share the workflow. In this workflow and the example logs I have replaced the repository name with redacted. I will attempt to reproduce this on an open repository:

name: Rust
on:
  push:
    branches:
      - main
  pull_request:
    paths:
      - src/rust/**
      # rebuild on dep changes
      - src/lance
      - src/lancedb
      - .github/workflows/rust.yml

env:
  # This env var is used by Swatinem/rust-cache@v2 for the cache
  # key, so we set it to make sure it is always consistent.
  CARGO_TERM_COLOR: always
  RUST_BACKTRACE: "1"
  CC: clang
  CXX: clang++

jobs:
  build:
    strategy:
      matrix:
        runner: [gha-arm, ubuntu-2404-8x-x64]
    runs-on: ${{ matrix.runner }}
    timeout-minutes: 30
    defaults:
      run:
        working-directory: ./src/rust/
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: true
      - uses: actions-rust-lang/setup-rust-toolchain@v1
        with:
          toolchain: "stable"
      - name: Install dependencies
        run: |
          sudo apt update
          sudo apt install -y protobuf-compiler libssl-dev
      - uses: Swatinem/rust-cache@v2
        with:
          # bumped on may 31st, 2024, because the cache is getting too big
          prefix-key: "redacted-rust-${{ matrix.runner}}"
          workspaces: "src/rust"
      - name: Check lint and cliipy
        run: make check
      - name: Install cargo-llvm-cov
        if: contains(matrix.runner, 'x64')
        uses: taiki-e/install-action@cargo-llvm-cov
      - name: Generate code coverage
        if: contains(matrix.runner, 'x64')
        run: make coverage
      - name: Run tests
        if: contains(matrix.runner, 'arm64')
        env:
          RUST_MIN_STACK: 16777216
        run: make test
      - name: Upload coverage to Codecov
        if: contains(matrix.runner, 'x64')
        uses: codecov/codecov-action@v4
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          codecov_yml_path: src/rust/codecov.yml
          files: coverage.codecov
          fail_ci_if_error: false

The text was updated successfully, but these errors were encountered:

westonpace · 2025-02-03T19:33:51Z

This issue may be related though the failures I see are not exactly the same: rust-lang/rust#125430

westonpace · 2025-02-03T19:35:08Z

We run the same workflow on a non-arm runner and have not seen these failures.

westonpace · 2025-02-03T21:04:53Z

I have not had much luck reproducing locally but I was able to reproduce on the repo in question with a very minimal workflow. This same workflow does not reproduce the issue on my public repo:

workflow:

name: Rust
on:
  push:
    branches:
      - main
  pull_request:
    paths:
      - .github/workflows/rust.yml


jobs:
  build:
    runs-on: gha-arm
    timeout-minutes: 30
    steps:
      - run: which rustc
      - run: md5sum /home/runner/.cargo/bin/rustc
      - run: rustc --version
      - run: rustc --version
      - run: rustc --version
      - run: rustc --version
      - run: rustc --version
      - run: rustc --version

log:

2025-02-03T21:03:52.9941372Z Current runner version: '2.322.0'
2025-02-03T21:03:52.9948097Z Runner name: 'gha-arm_fa3d8516bfb3'
2025-02-03T21:03:52.9948913Z Runner group name: 'large-runners'
2025-02-03T21:03:52.9949952Z Machine name: 'runner'
2025-02-03T21:03:52.9966955Z ##[group]VM Image
2025-02-03T21:03:52.9967740Z OS: Linux (arm64)
2025-02-03T21:03:52.9968436Z Source: Marketplace
2025-02-03T21:03:52.9968970Z Publisher: arm
2025-02-03T21:03:52.9969517Z Offer: github_arm_linux_runner_2404
2025-02-03T21:03:52.9970192Z Plan: github_arm_linux_runner_plan_2404
2025-02-03T21:03:52.9970810Z Version: 1.0.10
2025-02-03T21:03:52.9971302Z ##[endgroup]
2025-02-03T21:03:52.9973466Z ##[group]GITHUB_TOKEN Permissions
2025-02-03T21:03:52.9975865Z Actions: write
2025-02-03T21:03:52.9976418Z Attestations: write
2025-02-03T21:03:52.9977106Z Checks: write
2025-02-03T21:03:52.9977584Z Contents: write
2025-02-03T21:03:52.9978091Z Deployments: write
2025-02-03T21:03:52.9978810Z Discussions: write
2025-02-03T21:03:52.9979495Z Issues: write
2025-02-03T21:03:52.9980129Z Metadata: read
2025-02-03T21:03:52.9980642Z Packages: write
2025-02-03T21:03:52.9981211Z Pages: write
2025-02-03T21:03:52.9981712Z PullRequests: write
2025-02-03T21:03:52.9982288Z RepositoryProjects: write
2025-02-03T21:03:52.9982949Z SecurityEvents: write
2025-02-03T21:03:52.9983476Z Statuses: write
2025-02-03T21:03:52.9983962Z ##[endgroup]
2025-02-03T21:03:52.9986864Z Secret source: Actions
2025-02-03T21:03:52.9987613Z Prepare workflow directory
2025-02-03T21:03:53.0308181Z Prepare all required actions
2025-02-03T21:03:53.0409628Z Complete job name: build
2025-02-03T21:03:53.1215992Z ##[group]Run which rustc
2025-02-03T21:03:53.1217178Z �[36;1mwhich rustc�[0m
2025-02-03T21:03:53.1262565Z shell: /usr/bin/bash -e {0}
2025-02-03T21:03:53.1263733Z ##[endgroup]
2025-02-03T21:03:53.1455040Z /home/runner/.cargo/bin/rustc
2025-02-03T21:03:53.1558976Z ##[group]Run md5sum /home/runner/.cargo/bin/rustc
2025-02-03T21:03:53.1559752Z �[36;1mmd5sum /home/runner/.cargo/bin/rustc�[0m
2025-02-03T21:03:53.1589007Z shell: /usr/bin/bash -e {0}
2025-02-03T21:03:53.1589584Z ##[endgroup]
2025-02-03T21:03:53.7972913Z 21b27f4a24c066ff3c4158998d4f794b  /home/runner/.cargo/bin/rustc
2025-02-03T21:03:53.8028486Z ##[group]Run rustc --version
2025-02-03T21:03:53.8029347Z �[36;1mrustc --version�[0m
2025-02-03T21:03:53.8067917Z shell: /usr/bin/bash -e {0}
2025-02-03T21:03:53.8068617Z ##[endgroup]
2025-02-03T21:03:57.9717134Z malloc(): corrupted top size
2025-02-03T21:03:58.1224937Z /home/runner/work/_temp/09c07160-30d6-4185-a9ef-70e0360ac30f.sh: line 1:  2327 Aborted                 (core dumped) rustc --version
2025-02-03T21:03:58.1239808Z ##[error]Process completed with exit code 134.
2025-02-03T21:03:58.1335207Z Cleaning up orphan processes

westonpace · 2025-02-03T21:06:50Z

Here's a log from the public repo.

2025-02-03T20:59:28.5104332Z Current runner version: '2.322.0'
2025-02-03T20:59:28.5111840Z Runner name: 'gha-arm_9754af55d459'
2025-02-03T20:59:28.5112771Z Runner group name: 'large-runners'
2025-02-03T20:59:28.5113824Z Machine name: 'runner'
2025-02-03T20:59:28.5133452Z ##[group]VM Image
2025-02-03T20:59:28.5134209Z OS: Linux (arm64)
2025-02-03T20:59:28.5134784Z Source: Marketplace
2025-02-03T20:59:28.5135440Z Publisher: arm
2025-02-03T20:59:28.5136002Z Offer: github_arm_linux_runner_2404
2025-02-03T20:59:28.5136649Z Plan: github_arm_linux_runner_plan_2404
2025-02-03T20:59:28.5137402Z Version: 1.0.10
2025-02-03T20:59:28.5138221Z ##[endgroup]
2025-02-03T20:59:28.5140322Z ##[group]GITHUB_TOKEN Permissions
2025-02-03T20:59:28.5142749Z Actions: write
2025-02-03T20:59:28.5143377Z Attestations: write
2025-02-03T20:59:28.5143973Z Checks: write
2025-02-03T20:59:28.5144702Z Contents: write
2025-02-03T20:59:28.5145245Z Deployments: write
2025-02-03T20:59:28.5145751Z Discussions: write
2025-02-03T20:59:28.5146631Z Issues: write
2025-02-03T20:59:28.5147249Z Metadata: read
2025-02-03T20:59:28.5148021Z Packages: write
2025-02-03T20:59:28.5148532Z Pages: write
2025-02-03T20:59:28.5149144Z PullRequests: write
2025-02-03T20:59:28.5149780Z RepositoryProjects: write
2025-02-03T20:59:28.5150390Z SecurityEvents: write
2025-02-03T20:59:28.5151000Z Statuses: write
2025-02-03T20:59:28.5151562Z ##[endgroup]
2025-02-03T20:59:28.5154982Z Secret source: Actions
2025-02-03T20:59:28.5155923Z Prepare workflow directory
2025-02-03T20:59:28.5552183Z Prepare all required actions
2025-02-03T20:59:28.5619153Z Complete job name: build
2025-02-03T20:59:28.6386837Z ##[group]Run which rustc
2025-02-03T20:59:28.6388117Z �[36;1mwhich rustc�[0m
2025-02-03T20:59:28.6434180Z shell: /usr/bin/bash -e {0}
2025-02-03T20:59:28.6435185Z env:
2025-02-03T20:59:28.6435682Z   CARGO_TERM_COLOR: always
2025-02-03T20:59:28.6436436Z   RUST_BACKTRACE: 1
2025-02-03T20:59:28.6437067Z   CC: clang
2025-02-03T20:59:28.6437816Z   CXX: clang++
2025-02-03T20:59:28.6438453Z ##[endgroup]
2025-02-03T20:59:28.6619232Z /home/runner/.cargo/bin/rustc
2025-02-03T20:59:28.6721836Z ##[group]Run rustc --version
2025-02-03T20:59:28.6722523Z �[36;1mrustc --version�[0m
2025-02-03T20:59:28.6757296Z shell: /usr/bin/bash -e {0}
2025-02-03T20:59:28.6758122Z env:
2025-02-03T20:59:28.6758593Z   CARGO_TERM_COLOR: always
2025-02-03T20:59:28.6759372Z   RUST_BACKTRACE: 1
2025-02-03T20:59:28.6759975Z   CC: clang
2025-02-03T20:59:28.6760464Z   CXX: clang++
2025-02-03T20:59:28.6761082Z ##[endgroup]
2025-02-03T20:59:33.8900919Z rustc 1.84.0 (9fc6b4312 2025-01-07)
2025-02-03T20:59:34.4539114Z ##[group]Run md5sum /home/runner/.cargo/bin/rustc
2025-02-03T20:59:34.4539776Z �[36;1mmd5sum /home/runner/.cargo/bin/rustc�[0m
2025-02-03T20:59:34.4578538Z shell: /usr/bin/bash -e {0}
2025-02-03T20:59:34.4579109Z env:
2025-02-03T20:59:34.4579418Z   CARGO_TERM_COLOR: always
2025-02-03T20:59:34.4579712Z   RUST_BACKTRACE: 1
2025-02-03T20:59:34.4580100Z   CC: clang
2025-02-03T20:59:34.4580398Z   CXX: clang++
2025-02-03T20:59:34.4580696Z ##[endgroup]
2025-02-03T20:59:34.8517201Z 21b27f4a24c066ff3c4158998d4f794b  /home/runner/.cargo/bin/rustc

westonpace · 2025-02-03T21:18:31Z

Here is the runner configuration, in case it is important:

hemanthmanga · 2025-02-04T04:16:54Z

Hi @westonpace, Thank you for bringing this issue to our attention. We are looking into this issue and will update you on this issue after investigating.

julienbonastre · 2025-02-05T01:37:43Z

Interesting, same timeframe when this started for us but impacting a terragrunt based pipeline running on GH Large Hosted Arm64 runners 😢

tedchamb · 2025-02-05T16:49:55Z

@westonpace , which public repo are you referring to in this comment? We would like to get a solid repro we can work with.
#11533 (comment)

westonpace · 2025-02-05T16:52:24Z

@tedchamb sorry if I was unclear. I cannot reproduce on a public repo. Here was my attempt: https://github.com/lancedb/rustc-error-repro

If I run this exact same workflow in my private repo then I get the error. It does not occur on a public repo.

subir0071 · 2025-02-05T17:07:49Z

This issue is related to ARM image for Github actions.
Kindly raise the issue in partner-runner-images

westonpace · 2025-02-05T19:46:40Z

Thank you for sharing that repository. I have found actions/partner-runner-images#40 and added some comments there.

Joanna2473 · 2025-02-06T08:27:55Z

.

subir0071 · 2025-02-06T17:17:37Z

This is being tracked in the partner repo.
Hence, closing the ticket.

westonpace added bug report needs triage labels Feb 3, 2025

hemanthmanga added OS: Ubuntu OS: Ubuntu24 and removed needs triage labels Feb 4, 2025

hemanthmanga assigned subir0071 Feb 4, 2025

mitsuyuki418 mentioned this issue Feb 4, 2025

Segmentation fault terraform version -json for an ARM runner hashicorp/setup-terraform#459

Open

julienbonastre mentioned this issue Feb 5, 2025

Large Hosted arm64 Runners - Segmentation Fault since v2.322.0 Ubuntu-24.04 #11542

Closed

15 tasks

westonpace mentioned this issue Feb 5, 2025

ubuntu-24.04-arm - inconsistent rustc failures when compile CRuby actions/partner-runner-images#40

Open

dundargoc mentioned this issue Feb 5, 2025

ci: ARM jobs are unreliable neovim/neovim#32339

Closed

MikeMcC399 mentioned this issue Feb 6, 2025

actions/checkout@v4 SIGSEGV (exit code 139) in ubuntu-24.04-arm actions/partner-runner-images#47

Open

subir0071 closed this as completed Feb 6, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Intermittent segmentation faults from rustc on github_arm_linux_runner_2404 #11533

Intermittent segmentation faults from rustc on github_arm_linux_runner_2404 #11533

westonpace commented Feb 3, 2025 •

edited

Loading

westonpace commented Feb 3, 2025

westonpace commented Feb 3, 2025

westonpace commented Feb 3, 2025

westonpace commented Feb 3, 2025 •

edited

Loading

westonpace commented Feb 3, 2025

hemanthmanga commented Feb 4, 2025

julienbonastre commented Feb 5, 2025

tedchamb commented Feb 5, 2025

westonpace commented Feb 5, 2025

subir0071 commented Feb 5, 2025

westonpace commented Feb 5, 2025

Joanna2473 commented Feb 6, 2025

subir0071 commented Feb 6, 2025

Intermittent segmentation faults from rustc on github_arm_linux_runner_2404 #11533

Intermittent segmentation faults from rustc on github_arm_linux_runner_2404 #11533

Comments

westonpace commented Feb 3, 2025 • edited Loading

Description

Platforms affected

Runner images affected

Image version and build link

Is it regression?

Expected behavior

Actual behavior

Repro steps

westonpace commented Feb 3, 2025

westonpace commented Feb 3, 2025

westonpace commented Feb 3, 2025

westonpace commented Feb 3, 2025 • edited Loading

westonpace commented Feb 3, 2025

hemanthmanga commented Feb 4, 2025

julienbonastre commented Feb 5, 2025

tedchamb commented Feb 5, 2025

westonpace commented Feb 5, 2025

subir0071 commented Feb 5, 2025

westonpace commented Feb 5, 2025

Joanna2473 commented Feb 6, 2025

subir0071 commented Feb 6, 2025

westonpace commented Feb 3, 2025 •

edited

Loading

westonpace commented Feb 3, 2025 •

edited

Loading