docs: migrate SkiaSharp CI links and NuGet feeds off the xamarin AzDO org (pipeline def 345 + dnceng-public feeds)#4368
Open
mattleibow wants to merge 19 commits into
Open
docs: migrate SkiaSharp CI links and NuGet feeds off the xamarin AzDO org (pipeline def 345 + dnceng-public feeds)#4368mattleibow wants to merge 19 commits into
mattleibow wants to merge 19 commits into
Conversation
…line (def 345) Migrate all references to the old public Azure DevOps pipeline (xamarin/public, definitionId 4, 'SkiaSharp (Public)') to the new public pipeline (dnceng-public/public, definitionId 345, 'mono-SkiaSharp'). Covers the README build-status badge, the PR-artifacts comment workflow, the versioning doc, the ci-status skill (SKILL.md, ci-status.py constants, report-schema examples), and the get-skiasharp-pr download scripts. Internal DevDiv release pipelines and the SkiaSharp-CI/SkiaSharp packaging feed URLs are intentionally left unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
📦 Try the packages from this PRWarning Do not run these scripts without first reviewing the code in this PR. Step 1 — Download the packages bash / macOS / Linux: curl -fsSL https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.sh | bash -s -- 4368PowerShell / Windows: iex "& { $(irm https://raw.githubusercontent.com/mono/SkiaSharp/main/scripts/get-skiasharp-pr.ps1) } 4368"Step 2 — Add the local NuGet source dotnet nuget add source ~/.skiasharp/hives/pr-4368/packages --name skiasharp-pr-4368More options
Or download manually from Azure Pipelines — look for the Remove the source when you're done: dotnet nuget remove source skiasharp-pr-4368 |
Contributor
|
📖 Documentation Preview The documentation for this PR has been deployed and is available at: 🔗 View Staging Site This preview will be updated automatically when you push new commits to this PR. This comment is automatically updated by the documentation staging workflow. |
Move the public NuGet package feeds off the soon-to-be-deleted xamarin Azure DevOps org to dnceng-public/public. Feed names are lowercased to 'skiasharp' and 'skiasharp-ci'. Covers the integration-test nuget.config, the CI artifacts feed URL in shared.cake, the feed docs (building-samples.md, releasing.md), and the feed-management automation (manage-nuget-feed.yml matrix jobs and the manage-nuget-feed.ps1 default Organization/SourceFeed/DestinationFeed). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
mattleibow
changed the title
…not dnceng-public) The skiasharp and skiasharp-ci feeds were provisioned in the dnceng org (dnceng/public/_packaging/...), alongside the existing dotnet-public and dotnet-eng feeds — not in dnceng-public. Point the 6 feed references at the correct org. The public CI *pipeline* remains on dnceng-public/public (def 345); only the package feeds live under dnceng/public. get-skiasharp-pr.* keep the dnceng-public org because they download build artifacts from the pipeline. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add scripts/infra/package/mirror-nuget-feed.ps1 and a temporary workflow (.github/workflows/mirror-nuget-feed.yml) to copy packages from the old xamarin/public feeds to the new dnceng/public feeds before the xamarin org is deleted. Design (mirrors the patterns in manage-nuget-feed.ps1): - Diff-first: inventories both feeds via metadata only, then transfers just the versions missing from the destination. The destination feed is the state, so the tool is fully idempotent and re-runnable at any time. - Opt-in push: DRY RUN by default; -Push required to write anything. - Timeout-safe: honours -MaxDurationMinutes and stops gracefully between packages, never mid-push; re-run to continue where it left off. - Corruption-safe: every download is verified (valid zip + nuspec id/version match) before push; existing packages are never overwritten (409 = ok). - Scope control: default mirrors everything; -GoodVersionsOnly skips -pr.*/ malformed builds; -ReleasesOnly keeps only recognized releases. A version breakdown (releases / ephemeral / unrecognized) is printed every run. Workflow runs dry-run on PR/push to the tool files and a real, opt-in mirror on manual dispatch (needs AZURE_DEVOPS_PAT with packaging r/w on dnceng). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add a -SummaryFile parameter to mirror-nuget-feed.ps1 that appends a GitHub-flavoured Markdown report (diff counts, per-scope copy totals, and a per-package breakdown of what would be copied) and wire the temporary workflow to write it to $GITHUB_STEP_SUMMARY. On a PR that touches the tool, the workflow now runs a dry run for both feeds — metadata-only inventory + diff, no downloads, no writes, no secret required — and shows exactly what a real mirror would copy directly in the PR check summary. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ak dry run The CI job exposes AZURE_DEVOPS_PAT (currently expired). The script was attaching that PAT to the source/destination inventory reads, so the read-only dry run failed with 'Access Denied: PAT expired' even though the public feeds are anonymously readable. Resolve read auth lazily: probe each feed anonymously first and only fall back to the PAT if anonymous access is actually denied. The PAT is now only required for the push. Verified locally by running the dry run with a bogus PAT in the environment — it succeeds via anonymous read. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Harden how the feed-mirror handles AZURE_DEVOPS_PAT so it can never surface in logs and is never persisted: - Workflow: inject the secret ONLY for an opted-in manual push. On PR/push and dispatch dry runs the env value evaluates to '' so the token is not in the job environment at all (it cannot appear in those logs, even masked). - Push auth now uses the Azure Artifacts Credential Provider via the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS process env var instead of a temp nuget.config. The PAT is never written to disk and never passed as a 'dotnet nuget push' argument. The provider is installed only on the push path. - Register the PAT with the GitHub log masker (::add-mask::) as defense in depth, and scrub it from any captured 'dotnet' output before logging. - Dry runs read the public feeds anonymously and need no PAT at all (verified locally with the token unset). Note: the previous run's log showed the PAT as '***' (already masked by GitHub) plus an 'expired' error message — the secret value itself was never exposed. This change removes it from that scope entirely regardless. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ageRegex - Summary tables now list every package id, not just the top 25. - Add -ExcludePackageRegex to drop packages by id (e.g. '^_' to keep the internal underscore-prefixed CI wrapper packages out of the public feed). - Workflow defaults: the public 'skiasharp' feed excludes '^_'; the internal 'skiasharp-ci' feed keeps them. So a dispatch does the right thing per feed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Delete scripts/infra/package/manage-nuget-feed.ps1 and its workflow. The mirror-nuget-feed tool supersedes it: rather than deleting junk in place from a live feed, it mirrors only the packages/versions that pass the rules into a clean destination (public 'skiasharp' excludes the internal '_' wrapper packages; internal 'skiasharp-ci' keeps them). Drop the 'temporary' framing from the mirror workflow since it is now the go-forward feed tool, and make the version-classification comments self-contained (no longer reference the deleted script). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…versions
Make the feed behaviour a property of the script, not the workflow:
- Add -Feed skiasharp|skiasharp-ci. Each feed profile encodes its source,
destination, and rules intrinsically (public 'skiasharp' excludes the
internal '_' wrapper packages; internal 'skiasharp-ci' keeps them).
- Known-bad versions ('-pr.*' and malformed '-preview-*') are now ALWAYS
skipped for every feed. Removed the -GoodVersionsOnly / -ReleasesOnly /
-ExcludeVersionRegex / -ExcludePackageRegex knobs and the exploratory
'scope' input — we never copy junk, so there is nothing to choose.
- Remove the -IncludeBadVersions escape hatch entirely (never wanted).
- Slim the workflow to a dumb passthrough: inputs are just 'feed' (both/one)
and 'push' (dry by default); the matrix runs both feeds in parallel and
simply calls the script with -Feed. No source/dest/exclude/scope in YAML.
Local usage is now just: mirror-nuget-feed.ps1 -Feed skiasharp [-Push].
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…d pushes Two independent expert reviews of the irreversible-push path. Fixes: Correctness / safety: - Public feed uses an ALLOWLIST (releases only) instead of a denylist, so an unrecognized version on a non-underscore package can never reach the public feed. Internal CI feed keeps its denylist (drop -pr.* only). Encoded as a per-feed VersionPolicy in the feed profile. - Broaden bad-version detection to any '-pr.*' (was '-pr.<num>' only). - End-of-run verification: re-inventory the destination and assert every copied version is actually present; fail (exit 1) if not — a push that reports success but does not commit can no longer look green. - Push success is now determined only by exit code (--skip-duplicate makes a genuine duplicate exit 0); a non-zero exit is always a real failure and can no longer be masked as 'already present'. - Abort (exit 2) if the source feed unexpectedly returns 0 versions. - ValidateRange on MaxPushRetries (1-10) so 0 can't silently skip everything. - Full-archive zip validation (decompress every entry) so a truncated/corrupt payload with an intact nuspec is caught before push; pick the root .nuspec. Timeout: - 'dotnet nuget push --timeout 300' and reserve 12 min of budget so we always stop between packages, never mid-push. Secret handling: - PAT comes from env vars only; removed the -Pat/-SourcePat CLI parameters. - Source reads use only AZURE_DEVOPS_SOURCE_PAT, never the destination PAT. - Disable the credential provider's on-disk session-token cache. Workflow: - permissions: contents:read; checkout persist-credentials:false. - Pushes only from refs/heads/main (protected) in addition to opt-in dispatch. - Per-feed job-level concurrency so the same feed can't mirror concurrently. Verified locally: public -> 42,343 (allowlist), CI -> 47,388 (denylist), full-CRC validation passes a real 7 MB package. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse the existing manage-nuget-feed workflow/script paths (they already exist on main) so the workflow is registered on the default branch and can be dispatched against this branch. Pure rename of the mirror tool + workflow; no behaviour change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ggers This workflow operates on live NuGet feeds and must never run automatically. Drop the pull_request and push triggers; leave only workflow_dispatch. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The workflow is manual-dispatch only and dispatch is limited to write-access users, so a real copy (push=true) may be run from any branch — needed to populate the new dnceng feeds before this migration PR merges. Drop the refs/heads/main restriction on the real-copy gate; the secret is still injected only when push=true is explicitly selected. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… [switch]$Push param Local-fake-PAT testing revealed the push path was completely broken: every push threw 'Cannot convert Hashtable to SwitchParameter' before ever calling dotnet. Root cause: the loop stored the result in $push, which is the SAME variable (PowerShell is case-insensitive) as the script's [switch]$Push parameter — so assigning the result hashtable tried to coerce it to a switch. - Rename the local to $pushResult. - Rename the helper functions Save-Package -> Save-FeedPackage and Push-Package -> Publish-FeedPackage to avoid colliding with the built-in PackageManagement 'Save-Package' cmdlet (hygiene). Verified locally with a fake PAT: the tool now actually attempts the push, gets a real 401 Unauthorized from the feed, reports 5 failures, exits 1, and never prints the PAT. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… + robustness)
Second GPT-5.5 review after the $push/$Push shadowing bug. Shadowing class
confirmed fully swept (only $CacheDir remains, a safe string->string reuse).
- BLOCKER: workflow passed dispatch inputs (feed/push) by interpolating
${{ ... }} directly into bash/pwsh run blocks — a script-injection vector.
Pass inputs via env: and strictly validate feed (both|skiasharp|skiasharp-ci)
and push (true|false) before use. matrix.feed / gate outputs also via env.
- Add ValidateRange(13,1440) to MaxDurationMinutes so a value <= the 12-min
reserve can't silently no-op with exit 0.
- Use a per-feed cache subdirectory so a cached download for one feed can never
be reused for another.
Re-verified with a fake PAT: push still attempts correctly, 401s, exits 1, no
PAT leak.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Multi-line dotnet errors were stored verbatim in the failed list and wrapped in single backticks, breaking the Markdown step summary. Collapse whitespace to a single line so both the console list and the summary render cleanly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ches Add -Limit N (0 = unlimited) to cap how many missing versions are copied in one run. Useful for smoke-testing a huge feed (e.g. skiasharp-ci, where the smallest package is ~462 versions of ~23 MB native assets) and for migrating in controlled, resumable batches. Stops cleanly between packages like the timeout path, reports 'not yet attempted', and exits 0 to re-run (idempotent). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… disk exhaustion
The mirror downloaded every package to the cache dir but only removed the file
on the integrity-failure path — successfully-copied .nupkg files were never
deleted. For the skiasharp-ci feed (whose _NativeAssets* packages are ~23 MB
each) the cache grew until the GitHub-hosted runner's ~14 GB disk filled and the
runner process was killed mid-run ("No space left on device"), so each round
crashed after ~37 min instead of running the full time budget.
Reclaim disk after every processed item (copied, skipped, or failed) so the
cache never holds more than one package at a time. This only removes the local
download; it never touches anything already pushed to the destination feed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59d57876-0d1a-4a70-a95a-f44c7e784dfd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
xamarinAzure DevOps organization is being deleted. This PR migrates every in-repo reference that lives on / points at that org to the new Azure DevOps homes. It is a link/config migration only — no code logic changes.Two related migrations (note the two different target orgs — this mirrors how dnceng splits things: public CI pipelines live in
dnceng-public, public package feeds live indnceng):xamarin/publicdef4(SkiaSharp (Public))dnceng-public/publicdef345(mono-SkiaSharp)xamarin/publicfeedsSkiaSharp/SkiaSharp-CIdnceng/publicfeedsskiasharp/skiasharp-ciFiles changed (14 total)
Pipeline →
dnceng-public/publicdef 345:README.md— build-status badge → numeric-id form_apis/build/status/345.github/workflows/pr-artifacts-comment.yml—adoUrldocumentation/dev/versioning.md— manual-download link.agents/skills/ci-status/SKILL.md— org ref + pipeline table row.agents/skills/ci-status/scripts/ci-status.py— renamed constants +PUBLIC_PIPELINES.agents/skills/ci-status/references/report-schema.md— example JSONscripts/get-skiasharp-pr.sh/scripts/get-skiasharp-pr.ps1— pipeline constants + fallback URLs (these download build artifacts from the pipeline, so they stay ondnceng-public)Feeds →
dnceng/publicskiasharp/skiasharp-ci:tests/SkiaSharp.Tests.Integration/nuget.config—skiasharpsource URLscripts/infra/shared/shared.cake—CI_ARTIFACTS_FEED_URL(skiasharp-ci)externals-downloaddocumentation/dev/building-samples.md+documentation/dev/releasing.md— feed doc URLs.github/workflows/manage-nuget-feed.yml— 3 matrix jobs (Organization=dnceng/skiasharp/skiasharp-ci)scripts/infra/package/manage-nuget-feed.ps1— defaultOrganization/SourceFeed/DestinationFeedVerification done
grepfor oldxamarin/public/_build,definitionId=4,SkiaSharp (Public),xamarinfeed URLs,Organization = "xamarin"→ no matches remainpy_compile), Bash (bash -n), PowerShell (parser) syntax checks passhttps://pkgs.dev.azure.com/dnceng/public/_packaging/skiasharp{,-ci}/nuget/v3/index.jsonreturn HTTP200(verified via curl + authenticatedazfeed enumeration; feed IDsd9d80f9d…/1fdc4244…, project-scoped topublic)xamarinstrings are unrelated: Skia C-shim source paths (include/xamarin,sk_xamarin.cpp), Xamarin product/SDK names, XcodeORGANIZATIONNAME, and the DevDiv repoDevDiv/Xamarin.yaml-templates⛔ Outstanding concerns — please review before merging
dnceng/publicskiasharp+skiasharp-ciare created and public (HTTP 200). (Corrected from the initial assumption ofdnceng-public/public, where they do not exist — the feeds actually landed in thedncengorg next todotnet-public/dotnet-eng.)shared.cake'sCI_ARTIFACTS_FEED_URLdrivesdotnet cake --target=externals-download(viadownload.cake) and the integration-testnuget.config— until packages are published, native bootstrap and integration tests that rely on these feeds will find nothing. Publisher must push before this is functional.xamarinfeeds must be updated to publish to the newdnceng/publicfeeds.AZURE_DEVOPS_PATsecret scope. Themanage-nuget-feed.ymlworkflow and theget-skiasharp-pr.*scripts authenticate withsecrets.AZURE_DEVOPS_PAT. Confirm this PAT is valid for thedncengorg (feeds) anddnceng-publicorg (pipeline). A xamarin-scoped PAT will 401 after the switch.aka.ms/skiasharp-eap/index.jsonredirect (external). The "Preview" feed alias is referenced ~10 places (README, docs, release skills, integration tests). Its redirect target is configured in the aka.ms portal, not in this repo. If it currently points at axamarin-org feed, it must be repointed or all preview-feed consumers break. Not fixable in-repo — flagging for action.❓ Assumptions to confirm
345/mono-SkiaSharpindnceng-public/publicis correct (provided by the release owner). Worth a visual check that the README badge renders.✅ Intentionally NOT changed (leave as-is)
10789,26493,15756,25298,25328) anddevdiv.visualstudio.comDevDiv/Xamarin.yaml-templates(a repo inside DevDiv, not the xamarin org)include/xamarin,src/xamarin,sk_xamarin.cpp), Xamarin product/SDK names, XcodeORGANIZATIONNAMEdc.services.visualstudio.com(App Insights),governance.visualstudio.com(Component Governance),download.visualstudio.microsoft.com(VSIX CDN)Recommended merge order
dnceng/publicskiasharp/skiasharp-cifeeds — done.AZURE_DEVOPS_PATscope.aka.ms/skiasharp-eapalias (if it targeted xamarin).xamarinorg.