Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: policy docker build #1310

Merged
merged 4 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 90 additions & 81 deletions .github/workflows/multibuild.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the changes noted, the rest of the file is prettier formatting

Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Multibuild
on:
push:
tags:
- 'v*.*.*'
- "v*.*.*"
workflow_dispatch:
inputs:
main_build_only:
Expand All @@ -27,7 +27,9 @@ jobs:
git config --global user.name 'Atsign Robot'
git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com'
git checkout -b multibuild-${{github.run_number}}
- name: Ensure pubspec.yaml matches git ref (if current git ref is a version tag)
- name:
Ensure pubspec.yaml matches git ref (if current git ref is a version
tag)
shell: bash
if: startsWith(github.ref, 'refs/tags/v')
working-directory: ./packages/dart/sshnoports
Expand All @@ -53,20 +55,20 @@ jobs:
include:
- os: ubuntu-latest
output-name: sshnp-linux-x64
ext: ''
bundle: 'shell'
ext: ""
bundle: "shell"
- os: macos-13
output-name: sshnp-macos-x64
ext: ''
bundle: 'shell'
ext: ""
bundle: "shell"
- os: macos-14
output-name: sshnp-macos-arm64
ext: ''
bundle: 'shell'
ext: ""
bundle: "shell"
- os: windows-latest
output-name: sshnp-windows-x64
ext: '.exe'
bundle: 'windows'
ext: ".exe"
bundle: "windows"
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
Expand Down Expand Up @@ -118,7 +120,8 @@ jobs:
name: Import certificates
env:
MACOS_CODESIGN_CERT: ${{ secrets.MACOS_CODESIGN_CERT }}
MACOS_CODESIGN_CERT_PASSWORD: ${{ secrets.MACOS_CODESIGN_CERT_PASSWORD }}
MACOS_CODESIGN_CERT_PASSWORD:
${{ secrets.MACOS_CODESIGN_CERT_PASSWORD }}
MACOS_SIGNING_IDENTITY: ${{ secrets.MACOS_SIGNING_IDENTITY }}
MACOS_KEYCHAIN_PASSWORD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
run: |
Expand All @@ -144,11 +147,14 @@ jobs:
sshnp/{sshnp,sshnpd,srv,srvd,at_activate,debug/srvd,npt,npa_file}
# zip the build
- if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' }}
run: ditto -c -k --keepParent sshnp tarball/${{ matrix.output-name }}.zip
run:
ditto -c -k --keepParent sshnp tarball/${{ matrix.output-name }}.zip
- if: ${{ matrix.os == 'ubuntu-latest' }}
run: tar -cvzf tarball/${{ matrix.output-name }}.tgz sshnp
- if: ${{ matrix.os == 'windows-latest' }}
run: Compress-Archive -Path sshnp -Destination tarball/${{ matrix.output-name }}.zip
run:
Compress-Archive -Path sshnp -Destination tarball/${{
matrix.output-name }}.zip
# notarize the build
- if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' }}
env:
Expand All @@ -164,16 +170,15 @@ jobs:
# upload the build
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: ${{ matrix.output-name }}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
name:
${{ matrix.output-name
}}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/sshnoports/tarball
if-no-files-found: error

other_build:
needs: verify_tags
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./packages/dart
Comment on lines -174 to -176
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't change working directory anymore, use the root of the repo

strategy:
matrix:
platform: [linux/arm/v7, linux/arm64, linux/riscv64]
Expand All @@ -195,15 +200,17 @@ jobs:
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1
- if: ${{ ! inputs.main_build_only }}
run: |
docker buildx build -t atsigncompany/sshnptarball -f sshnoports/tools/Dockerfile.package \
docker buildx build -t atsigncompany/sshnptarball -f ./tools/multibuild/Dockerfile.package \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New dockerfile path

--platform ${{ matrix.platform }} -o type=tar,dest=bins.tar .
mkdir tarballs
tar -xvf bins.tar -C tarballs
- if: ${{ ! inputs.main_build_only }}
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: ${{ matrix.output-name }}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/tarballs/${{ matrix.output-name }}.tgz
name:
${{ matrix.output-name
}}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./tarballs/${{ matrix.output-name }}.tgz
if-no-files-found: error

universal_sh:
Expand Down Expand Up @@ -240,13 +247,12 @@ jobs:
working-directory: ./packages/dart/sshnoports/bundles
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: universal.ps1-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/sshnoports/bundles/universal.ps1
if-no-files-found: error

- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: universal.ps1-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/sshnoports/bundles/universal.ps1
if-no-files-found: error

github-release:
name: >-
Expand All @@ -256,55 +262,54 @@ jobs:
outputs:
hashes: ${{ steps.hash.outputs.hashes }}
permissions:
contents: write # Mandatory for making GitHub Releases
id-token: write # Mandatory for sigstore
contents: write # Mandatory for making GitHub Releases
id-token: write # Mandatory for sigstore
attestations: write
steps:
- name: Checkout pubspec.lock
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
sparse-checkout: packages/dart/sshnoports/pubspec.lock
sparse-checkout-cone-mode: false
- name: Install Syft
uses: anchore/sbom-action/download-syft@61119d458adab75f756bc0b9e4bde25725f86a7a # v0.17.2
- name: Download all the tarballs
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: tarballs/
- name: Generate SBOMs
run: |
syft scan file:./packages/dart/sshnoports/pubspec.lock \
-o 'spdx-json=tarballs/dart_sshnoports_sbom.spdx.json' \
-o 'cyclonedx-json=tarballs/dart_sshnoports_sbom.cyclonedx.json'
- name: Move packages for signing
run: |
cd tarballs
mv */*.sh .
mv */*.ps1 .
mv */*.tgz .
mv */*.zip .
rm -Rf -- */
- name: Generate SHA256 checksums
working-directory: tarballs
run: sha256sum * > checksums.txt
- name: Upload artifacts to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `tarballs/` contains the built packages, and the
# Syft produced SBOMs
run: >-
gh release upload
'${{ github.ref_name }}' tarballs/**
--repo '${{ github.repository }}'
- id: hash
name: Pass artifact hashes for SLSA provenance
working-directory: tarballs
run: |
echo "hashes=$(cat checksums.txt | base64 -w0)" >> "$GITHUB_OUTPUT"
- uses: actions/attest-build-provenance@6149ea5740be74af77f260b9db67e633f6b0a9a1 # v1.4.2
with:
subject-path: 'tarballs/**'
- name: Checkout pubspec.lock
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
sparse-checkout: packages/dart/sshnoports/pubspec.lock
sparse-checkout-cone-mode: false
- name: Install Syft
uses: anchore/sbom-action/download-syft@61119d458adab75f756bc0b9e4bde25725f86a7a # v0.17.2
- name: Download all the tarballs
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: tarballs/
- name: Generate SBOMs
run: |
syft scan file:./packages/dart/sshnoports/pubspec.lock \
-o 'spdx-json=tarballs/dart_sshnoports_sbom.spdx.json' \
-o 'cyclonedx-json=tarballs/dart_sshnoports_sbom.cyclonedx.json'
- name: Move packages for signing
run: |
cd tarballs
mv */*.sh .
mv */*.ps1 .
mv */*.tgz .
mv */*.zip .
rm -Rf -- */
- name: Generate SHA256 checksums
working-directory: tarballs
run: sha256sum * > checksums.txt
- name: Upload artifacts to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `tarballs/` contains the built packages, and the
# Syft produced SBOMs
run: >-
gh release upload '${{ github.ref_name }}' tarballs/** --repo '${{
github.repository }}'
- id: hash
name: Pass artifact hashes for SLSA provenance
working-directory: tarballs
run: |
echo "hashes=$(cat checksums.txt | base64 -w0)" >> "$GITHUB_OUTPUT"
- uses: actions/attest-build-provenance@6149ea5740be74af77f260b9db67e633f6b0a9a1 # v1.4.2
with:
subject-path: "tarballs/**"

provenance:
needs: [github-release]
Expand All @@ -325,11 +330,11 @@ jobs:
permissions:
contents: write # Needed to delete workflow branch
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ref: multibuild-${{github.run_number}}
- name: Delete workflow branch
run: git push origin --delete multibuild-${{github.run_number}}
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ref: multibuild-${{github.run_number}}
- name: Delete workflow branch
run: git push origin --delete multibuild-${{github.run_number}}

notify_on_completion:
needs: [github-release, cleanup]
Expand All @@ -338,7 +343,9 @@ jobs:
- name: Google Chat Notification
uses: Co-qn/google-chat-notification@3691ccf4763537d6e544bc6cdcccc1965799d056 # v1
with:
name: SSH no ports binaries were built by GitHub Action ${{ github.run_number }}
name:
SSH no ports binaries were built by GitHub Action ${{
github.run_number }}
url: ${{ secrets.GOOGLE_CHAT_WEBHOOK }}
status: ${{ job.status }}

Expand All @@ -350,6 +357,8 @@ jobs:
- name: Google Chat Notification
uses: Co-qn/google-chat-notification@3691ccf4763537d6e544bc6cdcccc1965799d056 # v1
with:
name: SSH no ports binaries build by GitHub Action ${{ github.run_number }}
name:
SSH no ports binaries build by GitHub Action ${{ github.run_number
}}
url: ${{ secrets.GOOGLE_CHAT_WEBHOOK }}
status: failure
33 changes: 0 additions & 33 deletions packages/dart/sshnoports/tools/Dockerfile.package
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was moved to tools/multibuild.yaml

This file was deleted.

57 changes: 57 additions & 0 deletions tools/multibuild/Dockerfile.package
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Dockerfile.package
# A dockerfile for packaging SSH No Ports releases using docker buildx

FROM atsigncompany/buildimage:3.5.2_3.6.0-149.3.beta@sha256:df67b9e3271381fc0c5b20e7350cf4de8dad6ac62e075b49b1a866c49af47409 AS build
# Using atsigncompany/buildimage until official dart image has RISC-V support
WORKDIR /noports

# install node for later (keep at the top file to increase cache hits)
# hadolint ignore=DL3008
RUN apt-get update; \
apt-get install -y --no-install-recommends npm
Comment on lines +8 to +11
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Install npm for the angular app


COPY . .

# Build packages/dart/sshnoports
WORKDIR /noports/packages/dart/sshnoports
Comment on lines +13 to +16
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed some pathing here, the new build context is the root of the repo instead of packages/dart so that the apps directory is also within the context (docker context doesn't work with ../ paths)

RUN set -eux; \
mkdir -p /sshnp/debug; \
mkdir /tarball; \
dart pub get --enforce-lockfile; \
dart run build_runner build --delete-conflicting-outputs; \
dart compile exe bin/activate_cli.dart -v -o /sshnp/at_activate; \
dart compile exe bin/sshnp.dart -v -o /sshnp/sshnp; \
dart compile exe bin/npt.dart -v -o /sshnp/npt; \
dart compile exe bin/npa_file.dart -v -o /sshnp/npa_file; \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added npa_file build

dart compile exe bin/sshnpd.dart -v -o /sshnp/sshnpd; \
dart compile exe bin/srv.dart -v -o /sshnp/srv; \
dart compile exe bin/npp.dart -v -o /sshnp/npp; \
dart compile exe bin/srvd.dart -v -o /sshnp/srvd; \
dart compile exe bin/srvd.dart -D ENABLE_SNOOP=true -v -o /sshnp/debug/srvd; \
cp -r bundles/core/* /sshnp/; \
cp -r bundles/shell/* /sshnp/; \
cp LICENSE /sshnp/;

# Build apps/admin/admin_api
WORKDIR /noports/apps/admin/admin_api
RUN dart pub get --enforce-lockfile; \
dart compile exe bin/admin_api.dart -v -o /sshnp/admin_api
Comment on lines +35 to +38
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dart admin api app


# Build apps/admin/webapp
WORKDIR /noports/apps/admin/webapp
RUN npm install; \
npm run build; \
mkdir -p /sshnp/web/admin; \
cp -r ./dist /sshnp/web/admin
Comment on lines +40 to +45
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build the angular app


RUN set -eux; \
case "$(dpkg --print-architecture)" in \
amd64) ARCH="x64";; \
armhf) ARCH="arm";; \
arm64) ARCH="arm64";; \
riscv64) ARCH="riscv64";; \
esac; \
tar -cvzf /tarball/sshnp-linux-"${ARCH}".tgz /sshnp

FROM scratch
COPY --from=build /tarball/* /