Skip to content

.github/workflows: Migrate workflows to Blacksmith runners#7454

Draft
blacksmith-sh[bot] wants to merge 37 commits into
masterfrom
blacksmith-migration-abb1f89
Draft

.github/workflows: Migrate workflows to Blacksmith runners#7454
blacksmith-sh[bot] wants to merge 37 commits into
masterfrom
blacksmith-migration-abb1f89

Conversation

@blacksmith-sh

@blacksmith-sh blacksmith-sh Bot commented May 21, 2026

Copy link
Copy Markdown

This PR has been automatically generated by a team member in your GitHub organization using Blacksmith's Migration Wizard. This PR changes the following:

  1. Your selected workflows will now run on Blacksmith's 2x faster hardware (e.g., runs-on: blacksmith-4vcpu-ubuntu-2204). Learn more about the different instances available to choose from.
  2. Your jobs running on Blacksmith will now have all official GitHub and popular third-party cache actions automatically interact with our 4x faster, colocated cache. Learn more about Blacksmith's actions cache.
  3. Your GitHub Actions will now actually be observable. Learn more about Blacksmith's logging and other observability features.
  4. Your Docker builds will now automatically share their Docker layer cache, resulting in up to 40x faster builds. Learn more about Blacksmith's Docker layer caching.

View with Codesmith
Need help on this PR? Tag @codesmith with what you need. Autofix is enabled.

@m-kuhn

m-kuhn commented May 21, 2026

Copy link
Copy Markdown
Member

This is me doing some tests

m-kuhn and others added 6 commits May 21, 2026 13:23
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@qfield-fairy

qfield-fairy commented May 21, 2026

Copy link
Copy Markdown
Collaborator

🍎 MacOS DMG universal builds

Download a MacOS DMG universal build of this PR for testing.
(Built from commit 0a5dd79)

🪟 Windows builds

Download a Windows build of this PR for testing and for arm64.

(Built from commit 32ab37e)

🐧 Linux AppImage builds

Download a Linux AppImage build of this PR for testing.
(Built from commit 8dbc1b2)

📱 Android builds

Download an Android arm64 build of this PR for testing.
(Built from commit 8dbc1b2)

Other Android architectures

@blacksmith-sh

blacksmith-sh Bot commented May 22, 2026

Copy link
Copy Markdown
Author

Found 4 test failures on Blacksmith runners:

Failures

Test View Logs
test_gallery_editor View Logs
test_image_attachment View Logs
test_projection View Logs
test_qml_text_editor_widgets View Logs

Fix with Codesmith
Need help on this PR? Tag @codesmith with what you need.

@m-kuhn

m-kuhn commented May 22, 2026

Copy link
Copy Markdown
Member

@codesmith please stop committing

@blacksmith-sh

blacksmith-sh Bot commented May 22, 2026

Copy link
Copy Markdown
Author

@m-kuhn

@codesmith please stop committing

Understood, I'll stop committing on this PR.

PR: #7454

View with Codesmith

@m-kuhn m-kuhn force-pushed the blacksmith-migration-abb1f89 branch from 273105a to 9fc3c9f Compare May 22, 2026 08:42
@m-kuhn

m-kuhn commented May 22, 2026

Copy link
Copy Markdown
Member

@codesmith we need ATL installed in the blacksmith runner for the windows (x64) workflow, however the current logic doesn't seem to work. What is required here?

ATL header atlbase.h not found under VCToolsInstallDir=C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\

@blacksmith-sh

blacksmith-sh Bot commented May 22, 2026

Copy link
Copy Markdown
Author

@m-kuhn

@codesmith we need ATL installed in the blacksmith runner for the windows (x64) workflow, however the current logic doesn't seem to work. What is required here?

ATL header atlbase.h not found und...

The problem is that vs_installer.exe modify --wait does NOT actually wait for the component install to complete. It's a well-known quirk of the Visual Studio installer: vs_installer.exe is a launcher that returns to the caller almost immediately (with $LASTEXITCODE == 0) while the real install runs asynchronously in a separate setup.exe process. Your if ($LASTEXITCODE -ne 0) check never trips, but by the time the next step runs, atlbase.h simply isn't on disk yet — and when ilammy/msvc-dev-cmd@v1 snapshots VCToolsInstallDir (it adds the Hostx64\x64 toolset path, not the ATL include path) the verification step rightly fails.

Two things need to change:

  1. Run the bootstrapper, not vs_installer.exe, and actually wait via Start-Process -Wait. The bootstrapper at https://aka.ms/vs/17/release/vs_BuildTools.exe is the same binary the runner image was built from; it's the only invocation that honors --wait reliably. Use Start-Process -Wait -PassThru so PowerShell blocks on the real exit code, and treat 3010 (reboot-required) as success.
  2. Then poll for atlbase.h before moving on. Even with Start-Process -Wait, file system mirroring on the VS install layout can lag by a few seconds; poll for up to ~120 s and only then fail.

Concrete replacement for the 🪳 Install Visual Studio ATL component step:

- name: 🪳 Install Visual Studio ATL component
  if: matrix.platform_arch == 'x64' # blacksmith windows-2025 image
  shell: powershell
  run: |
    $vswhere = "${Env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
    $installPath = & $vswhere -latest -products * -property installationPath
    if (-not $installPath) { throw "Visual Studio install not found via vswhere" }

    $bootstrapper = "$env:RUNNER_TEMP\vs_BuildTools.exe"
    Invoke-WebRequest -Uri "https://aka.ms/vs/17/release/vs_BuildTools.exe" -OutFile $bootstrapper

    $args = @(
      'modify',
      '--installPath', $installPath,
      '--add', 'Microsoft.VisualStudio.Component.VC.ATL',
      '--quiet', '--norestart', '--wait', '--nocache'
    )
    $p = Start-Process -FilePath $bootstrapper -ArgumentList $args -Wait -PassThru -NoNewWindow
    if ($p.ExitCode -ne 0 -and $p.ExitCode -ne 3010) {
      throw "Failed to install ATL component (exit $($p.ExitCode))"
    }

    # Poll for atlbase.h to make sure the on-disk layout caught up before later steps consume it.
    $atlRoot = Join-Path $installPath 'VC\Tools\MSVC'
    $deadline = (Get-Date).AddMinutes(2)
    while ((Get-Date) -lt $deadline) {
      $hit = Get-ChildItem $atlRoot -Recurse -Filter atlbase.h -ErrorAction SilentlyContinue | Select-Object -First 1
      if ($hit) { Write-Host "ATL ready at $($hit.FullName)"; return }
      Start-Sleep -Seconds 5
    }
    throw "ATL header atlbase.h still missing under $atlRoot after install"

Then drop the separate Verify ATL availability step — the polling above already covers it, and the previous version was searching VCToolsInstallDir after msvc-dev-cmd ran, which scopes that env var to the MSVC bin directory and not the ATL header tree. If you keep a verify step, point it at $installPath\VC\Tools\MSVC (or just $installPath) instead of $Env:VCToolsInstallDir.

A few smaller things while we're here:

  • The trailing whitespace on lines 70 and 75 of windows.yml (your pre-commit failure on 26284533921, 26292490706, 26294765860, 26294851109) is the empty PowerShell line after the --wait continuation. Either remove the empty line or strip the trailing spaces; pre-commit run --all-files will do it.
  • If the disk-space failure on qtmultimedia returns once ATL is in place, the only Blacksmith-side path that fits is moving vcpkg's buildtrees/downloads onto useblacksmith/stickydisk; every Blacksmith Windows tier is capped at 130 GB C:.

Maintainer asked me to stop committing on this PR, so I'm answering the "what is required" rather than pushing. Say the word and I'll push the edit.

PR: #7454

View with Codesmith

@m-kuhn

m-kuhn commented May 22, 2026

Copy link
Copy Markdown
Member

@codesmith continue to commit on the windows workflow only, the qtmultimedia issue is unrelated and no longer relevant

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@blacksmith-sh

blacksmith-sh Bot commented May 22, 2026

Copy link
Copy Markdown
Author

@m-kuhn

@codesmith continue to commit on the windows workflow only, the qtmultimedia issue is unrelated and no longer relevant

Pushed.

Issues Identified

The 🪳 Install Visual Studio ATL component step exited with $LASTEXITCODE == 0 but atlbase.h was missing afterward. vs_installer.exe modify --quiet --wait is a launcher: it returns immediately while the real install runs asynchronously in a separate setup.exe, so --wait is effectively a no-op and the next step ran before ATL was on disk. The follow-up Verify ATL availability step then correctly reported the missing header (also pointed at $Env:VCToolsInstallDir, which msvc-dev-cmd sets to the MSVC bin tree, not the include tree, but that was secondary).

Fixes Applied

In .github/workflows/windows.yml:

  • Replaced vs_installer.exe with the fresh vs_BuildTools.exe bootstrapper downloaded from aka.ms/vs/17/release/vs_BuildTools.exe. This is the binary the runner image was built from and is the only invocation that actually honors --wait.
  • Invoked it via Start-Process -Wait -PassThru -NoNewWindow so PowerShell blocks on the real exit code, and accepted 3010 (reboot-required) alongside 0.
  • After the bootstrapper returns, poll for atlbase.h under <installPath>\VC\Tools\MSVC for up to 2 minutes to absorb the file-system layout lag that sometimes follows component installs.
  • Removed the separate Verify ATL availability step (the polling above covers it, and the old check was scoped to the wrong env var). The dev-prompt step ordering is unchanged.

The change is scoped to the x64 Windows job; the arm64 entry and other jobs in the file are untouched. The diff also incidentally removed the stray trailing whitespace on those lines, which should clear pre-commit's trailing-whitespace failure on windows.yml at the same time.

Verification

Pushed e370ae4 to blacksmith-migration-abb1f89; Windows workflow will rerun the x64 build with the new ATL install path.

PR: #7454

View with Codesmith

m-kuhn and others added 6 commits May 22, 2026 15:05
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
m-kuhn and others added 3 commits May 22, 2026 16:02
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@m-kuhn

m-kuhn commented May 22, 2026

Copy link
Copy Markdown
Member

@codesmith on android, use a blacksmith sticky disk for /home/runner/.cache/vcpkg/archives the key should be the matrix triplet

Co-authored-by: Codesmith <codesmith-bot@users.noreply.github.com>
@blacksmith-sh

blacksmith-sh Bot commented May 22, 2026

Copy link
Copy Markdown
Author

@m-kuhn

@codesmith on android, use a blacksmith sticky disk for /home/runner/.cache/vcpkg/archives the key should be the matrix triplet

Pushed.

Fixes Applied

Added a 💾 Mount vcpkg archives sticky disk step in .github/workflows/android.yml immediately before 🌱 Install dependencies and generate project files, using useblacksmith/stickydisk@v1 with key: vcpkg-archives-${{ matrix.triplet }} and path: /home/runner/.cache/vcpkg/archives. One sticky disk per triplet (arm64-android, arm-neon-android, x64-android, x86-android), so the cache stays clean across architectures and each matrix entry gets its own ext4 volume mounted in ~3 s before vcpkg starts pulling sources. No other steps touched.

PR: #7454

View with Codesmith

@m-kuhn

m-kuhn commented May 29, 2026

Copy link
Copy Markdown
Member

@codesmith stop committing

@blacksmith-sh

blacksmith-sh Bot commented May 29, 2026

Copy link
Copy Markdown
Author

@m-kuhn

@codesmith stop committing

Understood, I'll stop committing on this PR.

PR: #7454

View with Codesmith

@nirvn nirvn marked this pull request as draft June 8, 2026 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants