Skip to content

Commit

Permalink
chore: publish code coverage (#170)
Browse files Browse the repository at this point in the history
* chore: publish code coverage

* pr-fix: ci trigger

* pr-fix: collect code coverage

* pr-fix: coverlage update

* pr-fix: project var name

* pr-fix: add coverage to integration test project

* pr-fix: use az devops code coverge

* pr-fix: add coverage settings to integration test project

* pr-fix: check integrtion test coverage

* pr-fix: single publish coverage result

* pr-fix: use single test stage

* pr-fix: adds depends on

* pr-fix: publish artifacts directly

* pr-fix: publish artifacts directly

* pr-fix: publish artifacts directly

* pr-fix: publish artifacts directly

* pr-fix: publish artifacts directly

* pr-fix: single test job

* pr-fix: single test job

* pr-fix: remove msbuild from csproj

* pr-fix: force result directory

* pr-fix: force result directory

* pr-fix: explicitly publish results

* pr-fix: Cobertura type

* Update publish-coverage.yml

* pr-fix: add gh comment

* pr-fix: add summary report

* pr-fix: secrets

* pr-fix: pr number

* pr-fix: pr number

* pr-fix: pr number

* pr-fix: publish code coverage in gh workflow

* pr-fix: correct gh action

* pr-fix: publish on pr

* pr-fix: correct path

* pr-fix: use gh cli

* pr-fix: install other way

* pr-fix: remove useless test

* pr-fix: use pr number

* pr-fix: upsert comment

* pr-fix: change summary

* pr-fix: use gh summary

* pr-fix: separate jobs

* pr-fix: use wildards

* pr-fix: use own test

* pr-fix: use correct artifacts

* pr-fix: use correct var name

* pr-fix: correct multi-line

* pr-fix: correct coverage path

* pr-ifix: correct multi-line

* pr-fix: correct format

* pr-fix: sinlge line

* pr-fix: w/o no-build

* pr-fix: publish to gh

* pr-fix: update comment

* pr-fix: motivational message

* pr-fix: correct syntax error

* pr-fix: improve coverage

* pr-fix: error multi-line

* pr-fix: more coverage + flexibe retrieval

* pr-fix: write host

* pr-fix: correct pattern

* pr-fix: update with more coverage

* pr-fix: improve branch coverage

* pr-fix: improve coverage

* pr-fix: improve coverage

* pr-fix: correct tests

* pr-fix: resuse min coverage level
  • Loading branch information
stijnmoreels authored Aug 2, 2024
1 parent d59d161 commit 6c0cb08
Show file tree
Hide file tree
Showing 22 changed files with 564 additions and 101 deletions.
37 changes: 0 additions & 37 deletions .github/workflows/code-coverage.yml

This file was deleted.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Arcus - Testing
[![Build Status](https://dev.azure.com/codit/Arcus/_apis/build/status/Commit%20builds/CI%20-%20Arcus.Testing?branchName=main)](https://dev.azure.com/codit/Arcus/_build/latest?definitionId=804&branchName=main)
[![NuGet Badge](https://buildstats.info/nuget/Arcus.Testing.Logging.Xunit?includePreReleases=true)](https://www.nuget.org/packages/Arcus.Testing.Logging/)
[![codecov](https://codecov.io/gh/arcus-azure/arcus.testing/graph/badge.svg?token=GIXXJO815Q)](https://codecov.io/gh/arcus-azure/arcus.testing)
![Azure DevOps coverage](https://img.shields.io/azure-devops/coverage/codit/Arcus/804)

Reusable testing components for Arcus repo's.

Expand Down
18 changes: 5 additions & 13 deletions build/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pr:
paths:
include:
- src/*
- build/ci-build.yml
- build/*

parameters:
- name: 'Package.Version.ManualTrigger'
Expand All @@ -31,7 +31,7 @@ resources:
endpoint: arcus-azure

variables:
- group: 'Arcus - GitHub Package Registry'
- group: 'GitHub Configuration'
- group: 'Build Configuration'
- template: ./variables/build.yml

Expand Down Expand Up @@ -66,27 +66,19 @@ stages:
targetPath: '$(Pipeline.Workspace)/build'
artifactName: Build

- stage: UnitTests
displayName: Unit Tests
- stage: Tests
dependsOn: Build
condition: succeeded()
jobs:
- template: templates/unit-tests.yml

- stage: IntegrationTests
displayName: Integration Tests
dependsOn: Build
condition: succeeded()
jobs:
- template: templates/integration-tests.yml
parameters:
azureServiceConnection: '${{ parameters.azureServiceConnection }}'
- template: templates/publish-coverage.yml

- stage: ReleaseToMyget
displayName: 'Release to MyGet'
dependsOn:
- UnitTests
- IntegrationTests
dependsOn: Tests
condition: succeeded()
jobs:
- job: PushToMyGet
Expand Down
38 changes: 33 additions & 5 deletions build/templates/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,36 @@ jobs:
Set-AzDevOpsVariable -Name 'Arcus.Testing.ServicePrincipal.ClientId' -Value $env:servicePrincipalId
Set-AzDevOpsVariable -Name 'Arcus.Testing.ServicePrincipal.ClientSecret' -Value $env:servicePrincipalKey
- template: test/run-integration-tests.yml@templates
parameters:
dotnetSdkVersion: '$(DotNet.Sdk.Version)'
projectName: '$(Project).Tests.Integration'
includePreviewVersions: $(DotNet.Sdk.IncludePreviewVersions)
- task: UseDotNet@2
displayName: 'Import .NET Core SDK ($(DotNet.Sdk.Version))'
inputs:
packageType: 'sdk'
version: '$(DotNet.Sdk.Version)'
includePreviewVersions: $(DotNet.Sdk.IncludePreviewVersions)

- task: qetza.replacetokens.replacetokens-task.replacetokens@6
displayName: 'Replace integration test tokens'
inputs:
root: 'src/$(Project).Tests.Integration/'
sources: 'appsettings.json'
missingVarLog: 'error'
logLevel: 'debug'
addBOM: true
tokenPrefix: '#{'
tokenSuffix: '}#'

- script: |
dotnet test src/**/$(Project).Tests.Integration.csproj --configuration $(Build.Configuration) --collect:"XPlat Code Coverage" --logger trx --results-directory $(Pipeline.Workspace)
displayName: 'Run integration tests'
- task: PublishTestResults@2
displayName: 'Publish integration test results'
inputs:
testResultsFormat: 'VSTest'
testResultsFiles: '$(Pipeline.Workspace)/*.trx'

- task: PublishPipelineArtifact@0
displayName: 'Publish code coverage artifacts'
inputs:
targetPath: '$(Pipeline.Workspace)'
artifactName: Coverage-Integration
98 changes: 98 additions & 0 deletions build/templates/publish-coverage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
jobs:
- job: PublishCoverage
displayName: 'Publish code coverage'
pool:
vmImage: '$(Vm.Image)'
dependsOn:
- UnitTests
- IntegrationTests
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download coverage unit artifacts'
inputs:
artifact: 'Coverage-Unit'
path: '$(Build.SourcesDirectory)/TestResults'

- task: DownloadPipelineArtifact@2
displayName: 'Download coverage integration artifacts'
inputs:
artifact: 'Coverage-Integration'
path: '$(Build.SourcesDirectory)/TestResults'

- task: reportgenerator@5
displayName: 'Generate code coverage report'
inputs:
reports: '$(Build.SourcesDirectory)/TestResults/**/coverage.cobertura.xml'
targetdir: coveragereport
reporttypes: Html;HtmlInline_AzurePipelines_Dark;Cobertura;MarkdownSummaryGithub
publishCodeCoverageResults: true

- script: sudo apt install gh
displayName: 'Install GitHub CLI'

- powershell: |
gh auth login --with-token $(GITHUB_TOKEN)
gh repo view
$summary = Get-Content -Path './coveragereport/SummaryGithub.md' -Raw
Write-Host "Summary: $summary"
$lineCoveragePattern = "\*\*Line coverage:\*\*\s*\|\s*(\d+(\.\d+)?%)"
$branchCoveragePattern = "\*\*Branch coverage:\*\*\s*\|\s*(\d+(\.\d+)?%)"
$lineCoverage = "N/A"
if ($summary -match $lineCoveragePattern) {
$lineCoverage = $matches[1]
Write-Host "Line coverage: $lineCoverage"
} else {
Write-Host "Line coverage not found"
}
$branchCoverage = "N/A"
if ($summary -match $branchCoveragePattern) {
$branchCoverage = $matches[1]
Write-Host "Branch coverage: $branchCoverage"
} else {
Write-Host "Branch coverage not found"
}
$lineCoverageValue = [double]($lineCoverage -replace '%', '')
$branchCoverageValue = [double]($branchCoverage -replace '%', '')
$buildId = $env:BUILD_BUILDID
$reportUrl = "https://dev.azure.com/codit/Arcus/_build/results?buildId=$buildId&view=codecoverage-tab"
$minCoverage = 80
if ($lineCoverageValue -ge $minCoverage -and $branchCoverage -ge $minCoverage) {
$motivationalMessage = "Great job! 😎 Your code coverage is higher than my caffeine levels! ☕"
} else {
$motivationalMessage = @"
😟 Your code coverage is not fully reaching **$minCoverage %** yet.
Please inspect [the report]($reportUrl) to find possible improvements and before you know it, your code coverage will reach the stars! 🚀
"@
}
if ($lineCoverageValue -ge $minCoverage) { $lineCoverageBadge = "🟢" } else { $lineCoverageBadge = "🔴" }
if ($branchCoverageValue -ge $minCoverage) { $branchCoverageBadge = "🟢" } else { $branchCoverageBadge = "🔴" }
$comments = gh pr view $env:PR_NUMBER --json comments --jq '.comments[].body'
$commentExists = $comments | Where-Object { $_ -like "*Code coverage*" -and $_ -like "*summary*" }
$content = @"
## 🧪 [Code coverage]($reportUrl) summary
| Metric | Value |
| ------ | ----- |
| Line coverage | $lineCoverageBadge **$lineCoverage** |
| Branch coverage | $branchCoverageBadge **$branchCoverage** |
$motivationalMessage
"@
if ($commentExists) {
gh pr comment $env:PR_NUMBER --body $content --edit-last
} else {
gh pr comment $env:PR_NUMBER --body $content
}
env:
GITHUB_TOKEN: $(GITHUB_TOKEN)
PR_NUMBER: $(System.PullRequest.PullRequestNumber)
displayName: 'Publish code coverage summary on GitHub'
27 changes: 22 additions & 5 deletions build/templates/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,25 @@ jobs:
packageType: 'sdk'
version: '$(DotNet.Sdk.VersionBC)'

- template: test/run-unit-tests.yml@templates
parameters:
dotnetSdkVersion: '$(DotNet.Sdk.Version)'
projectName: '$(Project).Tests.Unit'
includePreviewVersions: $(DotNet.Sdk.IncludePreviewVersions)
- task: UseDotNet@2
displayName: 'Import .NET Core SDK ($(DotNet.Sdk.Version))'
inputs:
packageType: 'sdk'
version: '$(DotNet.Sdk.Version)'
includePreviewVersions: $(DotNet.Sdk.IncludePreviewVersions)

- script: |
dotnet test src/**/$(Project).Tests.Unit.csproj --configuration $(Build.Configuration) --collect:"XPlat Code Coverage" --logger trx --results-directory $(Pipeline.Workspace)
displayName: 'Run unit tests'
- task: PublishTestResults@2
displayName: 'Publish unit test results'
inputs:
testResultsFormat: 'VSTest'
testResultsFiles: '$(Pipeline.Workspace)/*.trx'

- task: PublishPipelineArtifact@0
displayName: 'Publish code coverage artifacts'
inputs:
targetPath: '$(Pipeline.Workspace)'
artifactName: Coverage-Unit
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<Project Sdk="Microsoft.NET.Sdk">
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
Expand All @@ -7,6 +7,10 @@

<ItemGroup>
<PackageReference Include="Bogus" Version="29.0.2" />
<PackageReference Include="coverlet.collector" Version="6.0.2">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Guard.Net" Version="3.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="6.0.0" />
Expand Down
27 changes: 0 additions & 27 deletions src/Arcus.Testing.Tests.Integration/DemoIntegrationTest.cs

This file was deleted.

4 changes: 0 additions & 4 deletions src/Arcus.Testing.Tests.Unit/Arcus.Testing.Tests.Unit.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,6 @@
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="coverlet.msbuild" Version="6.0.2">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="FsCheck.Xunit" Version="2.16.6" />
<PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0" />
Expand Down
Loading

0 comments on commit 6c0cb08

Please sign in to comment.