Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: policy service beta #1315

Merged
merged 53 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
000d62d
feat: (initial sane commit): file-based policy service; policy servic…
gkc Jul 18, 2024
168bc5b
chore: ran dart format; cleaned an 'unused function' lint
gkc Jul 18, 2024
d1aa276
refactor: renamed policy.dart to npa_file.dart
gkc Jul 18, 2024
0d67c65
feat: some normalization of the yaml structure by adding userGroups
gkc Jul 23, 2024
206ea45
docs: corrected code comments
gkc Jul 23, 2024
24a5631
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Aug 8, 2024
090ff36
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Aug 12, 2024
7b263a5
interim commit
gkc Aug 23, 2024
e46f0c7
interim commit
gkc Aug 28, 2024
8b380c4
UI & API are good
gkc Aug 28, 2024
2b86e67
doc: added some more comments to npa_file.dart
gkc Aug 28, 2024
080cf29
feat: completed policy UI, policy API and policy service (npp) using …
gkc Sep 1, 2024
cb58e29
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Sep 2, 2024
6a2955b
build: Remove at_client dependency override, use published version 3.1.0
gkc Sep 2, 2024
3af7dbb
build: Remove at_client dependency override, use published version 3.1.0
gkc Sep 2, 2024
8d6edfb
feat: Added support for "-p $policy_atsign" to installation scripts
gkc Sep 2, 2024
ad167ad
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Sep 2, 2024
ba80840
build: updated buildArchive script to include the admin API and webapp
gkc Sep 3, 2024
40a7beb
build: updated multibuild.yaml to include the npp binary, and the adm…
gkc Sep 3, 2024
58aca59
build: fixed syntax in multibuild.yaml
gkc Sep 3, 2024
5e9334d
build: fixed syntax in multibuild.yaml
gkc Sep 3, 2024
c6a01fb
build: don't run build_runner for the admin_api
gkc Sep 3, 2024
37baa91
build: correct output path for compilation of admin_api
gkc Sep 3, 2024
3bc7a1c
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Sep 4, 2024
9c743b5
feat: sshnpd, if using a policy service, pings it periodically
gkc Sep 4, 2024
b8dda7b
refactor: SshnpdImpl: have _shareUsername() use _notify()
gkc Sep 4, 2024
2bfab25
fix: SshnpdImpl: when sending heartbeat to policy, set ttln correctly
gkc Sep 4, 2024
33bb4c3
chore: lint
gkc Sep 4, 2024
ce287e4
feat: NPAImpl now sends notifications to its loggingAtsign (currently…
gkc Sep 4, 2024
1be9c68
feat: expose log events via the policy admin API
gkc Sep 4, 2024
bea419a
feat: (interim): Added event streaming to the UI/API
gkc Sep 4, 2024
4544d1c
feat: policy dashboard UI tweaking
gkc Sep 4, 2024
43cc538
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Sep 4, 2024
bf67971
ci: add policy to other_build in multibuild
XavierChanth Sep 4, 2024
a1269fe
fix: node is node
XavierChanth Sep 4, 2024
2d8ef68
Merge branch 'feat/file-based-policy-service' into ci-policy-docker-b…
XavierChanth Sep 4, 2024
6d70e14
fix: install npm not nodejs
XavierChanth Sep 4, 2024
6a5779d
Merge pull request #1310 from atsign-foundation/ci-policy-docker-build
gkc Sep 4, 2024
a25c579
fix: don't include dist in path
XavierChanth Sep 4, 2024
357131d
Merge pull request #1311 from atsign-foundation/ci-policy-docker-build
gkc Sep 5, 2024
374e2b1
build(deps): Update dependabot for new Dockerfile location
cpswan Sep 5, 2024
de9e1bb
fix: use urls relative to /
gkc Sep 5, 2024
ccdb4f0
build: prep for merge
gkc Sep 5, 2024
32ef698
Merge remote-tracking branch 'origin/trunk' into feat/file-based-poli…
gkc Sep 5, 2024
73591b5
chore: removed stuff that shouldn't have been committed
gkc Sep 5, 2024
20c5928
fix: ugh, typo
gkc Sep 5, 2024
de0dfc1
Merge pull request #1313 from atsign-foundation/cpswan-dockerfile-mul…
gkc Sep 5, 2024
699a9c2
fix: remove link from dockerfile
XavierChanth Sep 5, 2024
3db2410
build: back out the concept of a 'beta' directory in the distribution
gkc Sep 5, 2024
ad81254
Merge branch 'feat-policy-service-beta' into feat-file-base-policy-se…
gkc Sep 5, 2024
dde8204
Merge pull request #1316 from atsign-foundation/feat-file-base-policy…
gkc Sep 5, 2024
bafb8dc
build: remove ln from buildArchive
gkc Sep 5, 2024
42dc02a
fix: Don't try to compile stuff that doesn't exist, duh
gkc Sep 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ updates:
- "/packages/c/sshnpd/tools/"
- "/packages/dart/sshnoports/tools/"
- "/tests/end2end_tests/image/"
- "/tools/multibuild/"
schedule:
interval: "daily"
groups:
Expand Down
192 changes: 109 additions & 83 deletions .github/workflows/multibuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Multibuild
on:
push:
tags:
- 'v*.*.*'
- "v*.*.*"
workflow_dispatch:
inputs:
main_build_only:
Expand All @@ -27,7 +27,9 @@ jobs:
git config --global user.name 'Atsign Robot'
git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com'
git checkout -b multibuild-${{github.run_number}}
- name: Ensure pubspec.yaml matches git ref (if current git ref is a version tag)
- name:
Ensure pubspec.yaml matches git ref (if current git ref is a version
tag)
shell: bash
if: startsWith(github.ref, 'refs/tags/v')
working-directory: ./packages/dart/sshnoports
Expand All @@ -53,28 +55,31 @@ jobs:
include:
- os: ubuntu-latest
output-name: sshnp-linux-x64
ext: ''
bundle: 'shell'
ext: ""
bundle: "shell"
- os: macos-13
output-name: sshnp-macos-x64
ext: ''
bundle: 'shell'
ext: ""
bundle: "shell"
- os: macos-14
output-name: sshnp-macos-arm64
ext: ''
bundle: 'shell'
ext: ""
bundle: "shell"
- os: windows-latest
output-name: sshnp-windows-x64
ext: '.exe'
bundle: 'windows'
ext: ".exe"
bundle: "windows"
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ref: multibuild-${{github.run_number}}
- uses: dart-lang/setup-dart@0a8a0fc875eb934c15d08629302413c671d3f672 # v1.6.5
- uses: actions/setup-node@1e60f620b9541d16bece96c5465dc8ee9832be0b # v4.0.3
with:
node-version: '20.17.0'
# create directories need for build
- run: |
mkdir sshnp
mkdir -p sshnp/web/admin
mkdir tarball
- if: ${{ matrix.os != 'windows-latest' }}
run: mkdir sshnp/debug
Expand All @@ -85,8 +90,21 @@ jobs:
dart compile exe bin/activate_cli.dart -v -o sshnp/at_activate${{ matrix.ext }}
dart compile exe bin/sshnp.dart -v -o sshnp/sshnp${{ matrix.ext }}
dart compile exe bin/npt.dart -v -o sshnp/npt${{ matrix.ext }}
dart compile exe bin/npp_file.dart -v -o sshnp/npa_file${{ matrix.ext }}
dart compile exe bin/npp_file.dart -v -o sshnp/npp_file${{ matrix.ext }}
dart compile exe bin/sshnpd.dart -v -o sshnp/sshnpd${{ matrix.ext }}
dart compile exe bin/srv.dart -v -o sshnp/srv${{ matrix.ext }}
dart compile exe bin/npp_atserver.dart -v -o sshnp/npp_atserver${{ matrix.ext }}
- name: build admin API
working-directory: ./apps/admin/admin_api
run: |
dart pub get --enforce-lockfile
dart compile exe bin/np_admin.dart -v -o ../../../packages/dart/sshnoports/sshnp/np_admin${{ matrix.ext }}
- name: build admin webapp
working-directory: ./apps/admin/webapp
run: |
npm install
npm run build
- if: ${{ matrix.os != 'windows-latest' }}
run: |
dart compile exe bin/srvd.dart -v -o sshnp/srvd${{ matrix.ext }}
Expand All @@ -95,13 +113,15 @@ jobs:
- run: |
cp -r bundles/core/* sshnp/
cp -r bundles/${{ matrix.bundle }}/* sshnp/
cp -r ../../../apps/admin/webapp/dist/* sshnp/web/admin/
cp LICENSE sshnp
# codesign for apple
- if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' }}
name: Import certificates
env:
MACOS_CODESIGN_CERT: ${{ secrets.MACOS_CODESIGN_CERT }}
MACOS_CODESIGN_CERT_PASSWORD: ${{ secrets.MACOS_CODESIGN_CERT_PASSWORD }}
MACOS_CODESIGN_CERT_PASSWORD:
${{ secrets.MACOS_CODESIGN_CERT_PASSWORD }}
MACOS_SIGNING_IDENTITY: ${{ secrets.MACOS_SIGNING_IDENTITY }}
MACOS_KEYCHAIN_PASSWORD: ${{ secrets.MACOS_KEYCHAIN_PASSWORD }}
run: |
Expand All @@ -124,14 +144,17 @@ jobs:
--prefix "com.atsign." \
--timestamp \
-v \
sshnp/{sshnp,sshnpd,srv,srvd,at_activate,debug/srvd,npt}
sshnp/{sshnp,sshnpd,srv,srvd,at_activate,debug/srvd,npt,npa_file,npp_file,npp_atserver,np_admin}
# zip the build
- if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' }}
run: ditto -c -k --keepParent sshnp tarball/${{ matrix.output-name }}.zip
run:
ditto -c -k --keepParent sshnp tarball/${{ matrix.output-name }}.zip
- if: ${{ matrix.os == 'ubuntu-latest' }}
run: tar -cvzf tarball/${{ matrix.output-name }}.tgz sshnp
- if: ${{ matrix.os == 'windows-latest' }}
run: Compress-Archive -Path sshnp -Destination tarball/${{ matrix.output-name }}.zip
run:
Compress-Archive -Path sshnp -Destination tarball/${{
matrix.output-name }}.zip
# notarize the build
- if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' }}
env:
Expand All @@ -147,16 +170,15 @@ jobs:
# upload the build
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: ${{ matrix.output-name }}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
name:
${{ matrix.output-name
}}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/sshnoports/tarball
if-no-files-found: error

other_build:
needs: verify_tags
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./packages/dart
strategy:
matrix:
platform: [linux/arm/v7, linux/arm64, linux/riscv64]
Expand All @@ -178,15 +200,17 @@ jobs:
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1
- if: ${{ ! inputs.main_build_only }}
run: |
docker buildx build -t atsigncompany/sshnptarball -f sshnoports/tools/Dockerfile.package \
docker buildx build -t atsigncompany/sshnptarball -f ./tools/multibuild/Dockerfile.package \
--platform ${{ matrix.platform }} -o type=tar,dest=bins.tar .
mkdir tarballs
tar -xvf bins.tar -C tarballs
- if: ${{ ! inputs.main_build_only }}
uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: ${{ matrix.output-name }}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/tarballs/${{ matrix.output-name }}.tgz
name:
${{ matrix.output-name
}}-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./tarballs/${{ matrix.output-name }}.tgz
if-no-files-found: error

universal_sh:
Expand Down Expand Up @@ -223,13 +247,12 @@ jobs:
working-directory: ./packages/dart/sshnoports/bundles
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: universal.ps1-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/sshnoports/bundles/universal.ps1
if-no-files-found: error

- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
with:
name: universal.ps1-${{github.ref_name}}-${{github.run_number}}-${{github.run_attempt}}
path: ./packages/dart/sshnoports/bundles/universal.ps1
if-no-files-found: error

github-release:
name: >-
Expand All @@ -239,55 +262,54 @@ jobs:
outputs:
hashes: ${{ steps.hash.outputs.hashes }}
permissions:
contents: write # Mandatory for making GitHub Releases
id-token: write # Mandatory for sigstore
contents: write # Mandatory for making GitHub Releases
id-token: write # Mandatory for sigstore
attestations: write
steps:
- name: Checkout pubspec.lock
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
sparse-checkout: packages/dart/sshnoports/pubspec.lock
sparse-checkout-cone-mode: false
- name: Install Syft
uses: anchore/sbom-action/download-syft@61119d458adab75f756bc0b9e4bde25725f86a7a # v0.17.2
- name: Download all the tarballs
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: tarballs/
- name: Generate SBOMs
run: |
syft scan file:./packages/dart/sshnoports/pubspec.lock \
-o 'spdx-json=tarballs/dart_sshnoports_sbom.spdx.json' \
-o 'cyclonedx-json=tarballs/dart_sshnoports_sbom.cyclonedx.json'
- name: Move packages for signing
run: |
cd tarballs
mv */*.sh .
mv */*.ps1 .
mv */*.tgz .
mv */*.zip .
rm -Rf -- */
- name: Generate SHA256 checksums
working-directory: tarballs
run: sha256sum * > checksums.txt
- name: Upload artifacts to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `tarballs/` contains the built packages, and the
# Syft produced SBOMs
run: >-
gh release upload
'${{ github.ref_name }}' tarballs/**
--repo '${{ github.repository }}'
- id: hash
name: Pass artifact hashes for SLSA provenance
working-directory: tarballs
run: |
echo "hashes=$(cat checksums.txt | base64 -w0)" >> "$GITHUB_OUTPUT"
- uses: actions/attest-build-provenance@6149ea5740be74af77f260b9db67e633f6b0a9a1 # v1.4.2
with:
subject-path: 'tarballs/**'
- name: Checkout pubspec.lock
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
sparse-checkout: packages/dart/sshnoports/pubspec.lock
sparse-checkout-cone-mode: false
- name: Install Syft
uses: anchore/sbom-action/download-syft@61119d458adab75f756bc0b9e4bde25725f86a7a # v0.17.2
- name: Download all the tarballs
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: tarballs/
- name: Generate SBOMs
run: |
syft scan file:./packages/dart/sshnoports/pubspec.lock \
-o 'spdx-json=tarballs/dart_sshnoports_sbom.spdx.json' \
-o 'cyclonedx-json=tarballs/dart_sshnoports_sbom.cyclonedx.json'
- name: Move packages for signing
run: |
cd tarballs
mv */*.sh .
mv */*.ps1 .
mv */*.tgz .
mv */*.zip .
rm -Rf -- */
- name: Generate SHA256 checksums
working-directory: tarballs
run: sha256sum * > checksums.txt
- name: Upload artifacts to GitHub Release
env:
GITHUB_TOKEN: ${{ github.token }}
# Upload to GitHub Release using the `gh` CLI.
# `tarballs/` contains the built packages, and the
# Syft produced SBOMs
run: >-
gh release upload '${{ github.ref_name }}' tarballs/** --repo '${{
github.repository }}'
- id: hash
name: Pass artifact hashes for SLSA provenance
working-directory: tarballs
run: |
echo "hashes=$(cat checksums.txt | base64 -w0)" >> "$GITHUB_OUTPUT"
- uses: actions/attest-build-provenance@6149ea5740be74af77f260b9db67e633f6b0a9a1 # v1.4.2
with:
subject-path: "tarballs/**"

provenance:
needs: [github-release]
Expand All @@ -308,11 +330,11 @@ jobs:
permissions:
contents: write # Needed to delete workflow branch
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ref: multibuild-${{github.run_number}}
- name: Delete workflow branch
run: git push origin --delete multibuild-${{github.run_number}}
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ref: multibuild-${{github.run_number}}
- name: Delete workflow branch
run: git push origin --delete multibuild-${{github.run_number}}

notify_on_completion:
needs: [github-release, cleanup]
Expand All @@ -321,7 +343,9 @@ jobs:
- name: Google Chat Notification
uses: Co-qn/google-chat-notification@3691ccf4763537d6e544bc6cdcccc1965799d056 # v1
with:
name: SSH no ports binaries were built by GitHub Action ${{ github.run_number }}
name:
SSH no ports binaries were built by GitHub Action ${{
github.run_number }}
url: ${{ secrets.GOOGLE_CHAT_WEBHOOK }}
status: ${{ job.status }}

Expand All @@ -333,6 +357,8 @@ jobs:
- name: Google Chat Notification
uses: Co-qn/google-chat-notification@3691ccf4763537d6e544bc6cdcccc1965799d056 # v1
with:
name: SSH no ports binaries build by GitHub Action ${{ github.run_number }}
name:
SSH no ports binaries build by GitHub Action ${{ github.run_number
}}
url: ${{ secrets.GOOGLE_CHAT_WEBHOOK }}
status: failure
3 changes: 3 additions & 0 deletions apps/admin/admin_api/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# https://dart.dev/guides/libraries/private-files
# Created by `dart pub`
.dart_tool/
3 changes: 3 additions & 0 deletions apps/admin/admin_api/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
## 1.0.0

- Initial version.
2 changes: 2 additions & 0 deletions apps/admin/admin_api/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
A sample command-line application with an entrypoint in `bin/`, library code
in `lib/`, and example unit test in `test/`.
16 changes: 16 additions & 0 deletions apps/admin/admin_api/analysis_options.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Defines a default set of lint rules enforced for
# projects at Google. For details and rationale,
# see https://pub.dev/packages/lints.
include: package:lints/recommended.yaml

# For lint rules and documentation, see http://dart-lang.github.io/linter/lints.
# Uncomment to specify additional rules.
linter:
rules:
annotate_overrides: true
prefer_final_fields: true
camel_case_types : true
unnecessary_string_interpolations : true
await_only_futures : true
unawaited_futures: true
depend_on_referenced_packages : false
Loading
Loading