Skip to content

Commit c9f6fdb

Browse files
authored
[ci] Switch to using the 1ES mandated pipeline template. (#844)
As part of Microsoft's continued push for supply chain security, our CI that builds shipping software must extend an "official" template that can be used to ensure various safety checks have run. Unfortunately, this requires extensive changes to our CI to fit their model. This PR requires both necessary changes and cleanup done to make our process mesh better with the template. The only functional difference should be: - Previously the outputs of both the `Windows` and `MacOS` builds were copied to the same artifact directory (`"nuget"`) which was signed and released. This meant that the last one written "won" and that's what we shipped. The new template didn't like multiple agents writing to the same output directory, so now we only write to `output-windows` and `output-macos`, and we always sign and ship the `output-windows` output.
1 parent b741a7b commit c9f6fdb

File tree

5 files changed

+134
-200
lines changed

5 files changed

+134
-200
lines changed

azure-pipelines.yml

Lines changed: 63 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -9,27 +9,74 @@ variables:
99
BUILD_NUMBER: $(Build.BuildNumber)
1010
BUILD_COMMIT: $(Build.SourceVersion)
1111

12+
# Build variables
13+
mainBranchName: main # Name of Git "main" branch
14+
configuration: Release # Build configuration: 'Debug', 'Release'
15+
16+
# Reporting variables
17+
areaPath: DevDiv\VS Client - Runtime SDKs\Android # AzDo area path to log any issues
18+
19+
# Windows specific variables
20+
windowsAgentPoolName: Maui-1ESPT # Windows VM pool name
21+
windowsImage: 1ESPT-Windows2022 # Windows VM image name
22+
windowsClassicInstaller: https://aka.ms/xamarin-android-commercial-d17-4-windows # Windows Classic XA installer URL
23+
24+
# macOS specific variables
25+
macosAgentPoolName: Azure Pipelines # macOS VM pool name
26+
macosImage: internal-macos12 # macOS VM image name
27+
macosClassicInstaller: https://aka.ms/xamarin-android-commercial-d17-4-macos # macOS Classic XA installer URL
28+
1229
resources:
1330
repositories:
31+
- repository: 1esPipelines
32+
type: git
33+
name: 1ESPipelineTemplates/1ESPipelineTemplates
34+
ref: refs/tags/release
1435
- repository: internal-templates
1536
type: github
1637
name: xamarin/yaml-templates
1738
endpoint: xamarin
1839
ref: refs/heads/main
1940

20-
jobs:
21-
- template: build/ci/build.yml
22-
23-
- ${{ if eq(variables['System.TeamProject'], 'devdiv') }}:
24-
- template: sign-artifacts/jobs/v2.yml@internal-templates
25-
parameters:
26-
dependsOn: [ 'build' ]
27-
condition: startsWith(variables['Build.SourceBranch'], 'refs/tags/')
28-
29-
- template: compliance/sbom/job.v1.yml@internal-templates
30-
parameters:
31-
dependsOn: signing
32-
condition: startsWith(variables['Build.SourceBranch'], 'refs/tags/')
33-
artifactNames: [ nuget-signed ]
34-
packageName: androidx
35-
packageFilter: '*.nupkg'
41+
extends:
42+
template: v1/1ES.Official.PipelineTemplate.yml@1esPipelines
43+
parameters:
44+
pool:
45+
name: AzurePipelines-EO
46+
image: 1ESPT-Windows2022
47+
os: windows
48+
49+
stages:
50+
- stage: Build
51+
52+
jobs:
53+
- template: build/ci/build.yml@self
54+
parameters:
55+
name: windows
56+
buildPool:
57+
name: $(windowsAgentPoolName)
58+
image: $(windowsImage)
59+
os: windows
60+
classicInstallerUrl: $(windowsClassicInstaller)
61+
mainBranchName: $(mainBranchName)
62+
configuration: $(configuration)
63+
runAPIScan: true
64+
65+
- template: build/ci/build.yml@self
66+
parameters:
67+
name: macos
68+
buildPool:
69+
name: $(macosAgentPoolName)
70+
vmImage: $(macosImage)
71+
os: macOS
72+
classicInstallerUrl: $(macosClassicInstaller)
73+
mainBranchName: $(mainBranchName)
74+
configuration: $(configuration)
75+
76+
- template: sign-artifacts/jobs/v2.yml@internal-templates
77+
parameters:
78+
dependsOn: [ 'build_windows' ]
79+
artifactName: output-windows
80+
usePipelineArtifactTasks: true
81+
use1ESTemplate: true
82+
condition: startsWith(variables['Build.SourceBranch'], 'refs/tags/')

build/ci/api-scan.yml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ steps:
1414
TargetFolder: ${{ parameters.apiScanDirectory }}
1515
OverWrite: true
1616
flattenFolders: true
17-
condition: and(succeeded(), eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
17+
condition: and(succeeded(), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
1818

1919
- task: CmdLine@2
2020
displayName: 'Remove System assemblies from APIScan'
@@ -23,14 +23,14 @@ steps:
2323
del ${{ parameters.apiScanDirectory }}\System.*
2424
del ${{ parameters.apiScanDirectory }}\mscorlib.dll
2525
del ${{ parameters.apiScanDirectory }}\netstandard.dll
26-
condition: and(succeeded(), eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
26+
condition: and(succeeded(), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
2727

2828
- task: CmdLine@2
2929
displayName: 'List Files for APIScan'
3030
inputs:
3131
script: |
3232
tree ${{ parameters.apiScanDirectory }} /f
33-
condition: and(succeeded(), eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
33+
condition: and(succeeded(), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
3434

3535
### Run latest version of APIScan listed at https://www.1eswiki.com/wiki/APIScan_Build_Task
3636
- task: APIScan@2
@@ -41,7 +41,7 @@ steps:
4141
softwareVersionNum: '$(Build.BuildId)'
4242
isLargeApp: true
4343
toolVersion: Latest
44-
condition: and(succeeded(), eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
44+
condition: and(succeeded(), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
4545
env:
4646
AzureServicesAuthConnectionString: runAs=App;AppId=$(ApiScanClientId);TenantId=$(ApiScanTenant);AppKey=$(ApiScanSecret)
4747

@@ -51,7 +51,7 @@ steps:
5151
GdnExportAllTools: false
5252
GdnExportGdnToolApiScan: true
5353
GdnExportOutputSuppressionFile: source.gdnsuppress
54-
condition: and(eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
54+
condition: eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch'])
5555

5656
- task: PublishSecurityAnalysisLogs@3
5757
displayName: Publish Guardian Artifacts
@@ -61,11 +61,11 @@ steps:
6161
AllTools: false
6262
APIScan: true
6363
ToolLogsNotFoundAction: Warning
64-
condition: and(eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
64+
condition: eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch'])
6565

6666
- task: PostAnalysis@2
6767
displayName: Fail Build on Guardian Issues
6868
inputs:
6969
GdnBreakAllTools: false
7070
GdnBreakGdnToolApiScan: true
71-
condition: and(eq(variables['runAPIScan'], 'true'), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
71+
condition: eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch'])

build/ci/build.yml

Lines changed: 63 additions & 103 deletions
Original file line numberDiff line numberDiff line change
@@ -1,122 +1,82 @@
11
parameters:
22
# Environment Parameters
3-
name: 'build' # the name of the build job for dependency purposes
4-
timeoutInMinutes: 300 # the timeout in minutes
5-
mainBranchName: 'main' # the "main" branch that should be used - can be something other than "main"
6-
macosAgentPoolName: 'Azure Pipelines' # the name of the macOS VM pool
7-
# https://github.com/actions/runner-images
8-
macosImage: internal-macos12 # macOS VM image name, must be "internal" locked down image
9-
windowsAgentPoolName: android-win-2022 # the name of the Windows VM pool
10-
windowsImage: 'windows-latest' # the name of the Windows VM image
3+
name: # Job display name
4+
buildPool: # VM pool information
5+
classicInstallerUrl: # URL to retrieve the Classic XA installer
116

7+
# Build Parameters
8+
mainBranchName: 'main' # Name of Git "main" branch
9+
configuration: 'Release' # Build configuration: 'Debug', 'Release'
10+
verbosity: 'normal' # Build verbosity: 'minimal', 'normal', 'diagnostic'
11+
timeoutInMinutes: 300 # Max job runtime in minutes
12+
runAPIScan: false # Run APIScan analysis
13+
1214
# Tool Parameters
13-
dotnetVersion: '7.0.405' # the version of .NET to use
14-
dotnetWorkloadRollbackFile: 'workloads.json'
15-
dotnetWorkloadSource: 'https://aka.ms/dotnet6/nuget/index.json'
16-
dotnetNuGetOrgSource: 'https://api.nuget.org/v3/index.json'
17-
classicXAPkg: https://aka.ms/xamarin-android-commercial-d17-4-macos
18-
classicXAVsix: https://aka.ms/xamarin-android-commercial-d17-4-windows
19-
skipUnitTests: false # do not run unit test step
15+
dotnetVersion: '7.0.406' # .NET version to install on agent
16+
dotnetWorkloadRollbackFile: 'workloads.json' # Rollback file specifying workload versions to install
17+
dotnetNuGetOrgSource: 'https://api.nuget.org/v3/index.json' # NuGet.org URL to find workloads
18+
dotnetWorkloadSource: 'https://aka.ms/dotnet6/nuget/index.json' # .NET engineering URL to find workloads
19+
skipUnitTests: false # Skip running unit tests
2020

21-
tools: # a list of additional .NET global tools needed
21+
tools: # Additional .NET global tools to install
2222
- 'xamarin.androidbinderator.tool': '0.5.7'
2323
- 'Cake.Tool': '4.0.0'
2424
- 'boots': '1.1.0.712-preview2'
2525
- 'private-api-tools': '1.0.2'
26-
27-
# Build Parameters
28-
verbosity: 'normal' # the build verbosity: 'minimal', 'normal', 'diagnostic'
29-
configuration: 'Release' # the build configuration: 'Debug', 'Release'
3026

3127
# Reporting/Analysis Parameters
32-
areaPath: 'DevDiv\VS Client - Runtime SDKs\Android' # the areaPath to log any issues
33-
publishJob: '' # the job to use as the source of the 'nuget' artifact: '', 'windows', 'macos', 'linux'
34-
publishOutputSuffix: '' # the artifact suffix to use when publishing the output folder
35-
signListPath: 'SignList.xml' # the path to the SignList.xml to copy into the nuget artifact for signing
36-
artifactsPath: 'output' # the path to the NuGet packages that need to be signed, verified and published
28+
artifactsPath: 'output' # Path to the NuGet packages that need to be signed, verified and published
29+
signListPath: 'SignList.xml' # Path to 'SignList.xml' used for signing NuGet packages
3730

3831
jobs:
39-
- job: ${{ parameters.name }}
40-
strategy:
41-
matrix:
42-
macos:
43-
poolName: ${{ parameters.macosAgentPoolName }}
44-
imageName: ${{ parameters.macosImage }}
45-
classicInstallerUrl: ${{ parameters.classicXAPkg }}
46-
runCodeQL: false
47-
windows:
48-
poolName: ${{ parameters.windowsAgentPoolName }}
49-
imageName: ${{ parameters.windowsImage }}
50-
classicInstallerUrl: ${{ parameters.classicXAVsix }}
51-
runCodeQL: true
52-
runAPIScan: true
53-
displayName: Build
54-
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
55-
variables:
56-
Codeql.Enabled: $(runCodeQL)
57-
pool:
58-
name: $(poolName)
59-
vmImage: $(imageName)
60-
61-
steps:
62-
- template: setup-environment.yml
63-
parameters:
64-
dotnetVersion: ${{ parameters.dotnetVersion }}
65-
dotnetWorkloadRollbackFile: ${{ parameters.dotnetWorkloadRollbackFile }}
66-
dotnetWorkloadSource: ${{ parameters.dotnetWorkloadSource }}
67-
dotnetNuGetOrgSource: ${{ parameters.dotnetNuGetOrgSource }}
68-
dotnetTools: ${{ parameters.tools }}
69-
classicInstallerUrl: $(classicInstallerUrl)
32+
- job: build_${{ parameters.name }}
33+
displayName: ${{ parameters.name }}
34+
timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
35+
pool: ${{ parameters.buildPool }}
36+
37+
templateContext:
38+
sdl:
39+
spotBugs:
40+
enabled: false
41+
binskim:
42+
scanOutputDirectoryOnly: true
43+
outputs:
44+
- output: pipelineArtifact
45+
targetPath: ${{ parameters.artifactsPath }}
46+
artifactName: output-${{ parameters.name }}
47+
48+
steps:
49+
- template: setup-environment.yml
50+
parameters:
51+
dotnetVersion: ${{ parameters.dotnetVersion }}
52+
dotnetWorkloadRollbackFile: ${{ parameters.dotnetWorkloadRollbackFile }}
53+
dotnetWorkloadSource: ${{ parameters.dotnetWorkloadSource }}
54+
dotnetNuGetOrgSource: ${{ parameters.dotnetNuGetOrgSource }}
55+
dotnetTools: ${{ parameters.tools }}
56+
classicInstallerUrl: ${{ parameters.classicInstallerUrl }}
7057

71-
- template: build-and-test.yml
72-
parameters:
73-
artifactsPath: ${{ parameters.artifactsPath }}
74-
verbosity: ${{ parameters.verbosity }}
75-
configuration: ${{ parameters.configuration }}
76-
skipUnitTests: ${{ parameters.skipUnitTests }}
58+
- template: build-and-test.yml
59+
parameters:
60+
artifactsPath: ${{ parameters.artifactsPath }}
61+
verbosity: ${{ parameters.verbosity }}
62+
configuration: ${{ parameters.configuration }}
63+
skipUnitTests: ${{ parameters.skipUnitTests }}
7764

65+
- ${{ if eq(parameters.runAPIScan, true) }}:
7866
- template: api-scan.yml
7967
parameters:
8068
mainBranchName: ${{ parameters.mainBranchName }}
8169

82-
# after the build is complete
83-
- pwsh: |
84-
$srcExists = (Test-Path "${{ parameters.signListPath }}")
85-
$dstExists = (Test-Path "${{ parameters.artifactsPath }}\SignList.xml")
86-
if ($srcExists -and !$dstExists) {
87-
Copy-Item "${{ parameters.signListPath }}" "${{ parameters.artifactsPath }}\SignList.xml"
88-
Write-Host "Copied ${{ parameters.signListPath }} to ${{ parameters.artifactsPath }}\SignList.xml"
89-
} elseif (!$srcExists) {
90-
Write-Host "${{ parameters.signListPath }} did not exist, nothing copied."
91-
} elseif ($dstExists) {
92-
Write-Host "${{ parameters.artifactsPath }}\SignList.xml already existed, nothing copied."
93-
}
94-
displayName: 'Copy SignList.xml to the nuget artifact'
95-
- task: PublishBuildArtifacts@1
96-
displayName: 'Publish artifacts'
97-
condition: or(eq('${{ parameters.publishJob }}', ''), eq('${{ parameters.publishJob }}', variables['System.JobName']))
98-
inputs:
99-
PathToPublish: ${{ parameters.artifactsPath }}
100-
ArtifactName: nuget
101-
- task: PublishBuildArtifacts@1
102-
displayName: 'Publish platform artifacts'
103-
condition: always()
104-
inputs:
105-
PathToPublish: output
106-
ArtifactName: output-$(System.JobName)${{ parameters.publishOutputSuffix }}
107-
# run any required checks
108-
- ${{ if eq(variables['System.TeamProject'], 'devdiv') }}:
109-
- task: ComponentGovernanceComponentDetection@0
110-
displayName: 'Run component detection'
111-
condition: and(always(), eq('refs/heads/${{ parameters.mainBranchName }}', variables['Build.SourceBranch']))
112-
inputs:
113-
scanType: 'Register'
114-
verbosity: 'Verbose'
115-
alertWarningLevel: 'High'
116-
117-
- template: code-analysis.yml
118-
parameters:
119-
name: ${{ parameters.name }}
120-
mainBranchName: ${{ parameters.mainBranchName }}
121-
areaPath: ${{ parameters.areaPath }}
122-
configuration: ${{ parameters.configuration }}
70+
# Copy SignList.xml to output
71+
- pwsh: |
72+
$srcExists = (Test-Path "${{ parameters.signListPath }}")
73+
$dstExists = (Test-Path "${{ parameters.artifactsPath }}\SignList.xml")
74+
if ($srcExists -and !$dstExists) {
75+
Copy-Item "${{ parameters.signListPath }}" "${{ parameters.artifactsPath }}\SignList.xml"
76+
Write-Host "Copied ${{ parameters.signListPath }} to ${{ parameters.artifactsPath }}\SignList.xml"
77+
} elseif (!$srcExists) {
78+
Write-Host "${{ parameters.signListPath }} did not exist, nothing copied."
79+
} elseif ($dstExists) {
80+
Write-Host "${{ parameters.artifactsPath }}\SignList.xml already existed, nothing copied."
81+
}
82+
displayName: Copy SignList.xml to output

0 commit comments

Comments
 (0)