Upgrade to 1.24

Remove rotten protoc-gen-doc
Upgrade tools version to correspond to those in general codebase.
Use new avid-tools

Add .gitattributes
Add dependabot.yml

Install protoc-gen-doc binary in ci workflow

Use new go get -tool command to set tools and remove tools.go
Use go tool install to install all tools.
Move all helper go code into separate directory and copy over
when appropriate otherwise installing tools fails with unknown
import.

AB#10624

Signed-off-by: Paul Hewlett <[email protected]>

Author: eccles

.env.tools

@@ -1 +1 @@
-export TOOLS_BUILDNUMBER=20241108.3
+export TOOLS_BUILDNUMBER=20250328.1

.gitattributes

@@ -0,0 +1,2 @@
+**/go.sum              -diff -merge
+**/go.sum              linguist-generated=true

.github/dependabot.yml

@@ -0,0 +1,22 @@
+# refer: https://github.com/dependabot/dependabot-core/blob/main/.github/dependabot.yml
+version: 2
+updates:
+  # Maintain dependencies for GitHub Actions
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      # Check for updates managed by Composer once a week
+      interval: "weekly"
+      day: "wednesday"
+      time: "03:00"
+      timezone: "Europe/London"
+
+  # Maintain dependencies for Golang
+  - package-ecosystem: "gomod"
+    directory: "/"
+    schedule:
+      # Check for updates managed by Composer once a week
+      interval: "weekly"
+      day: "wednesday"
+      time: "03:00"
+      timezone: "Europe/London"

.github/workflows/ci.yml

@@ -10,30 +10,40 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
+
       - name: Setup go-task
         uses: arduino/setup-task@v1
         with:
           version: 3.x
           repo-token: ${{ secrets.GITHUB_TOKEN }}
+
       - name: Setup Go
         uses: actions/setup-go@v4
         with:
-          go-version: '1.22.x'
+          go-version: '1.24.x'
           token: ${{ secrets.GITHUB_TOKEN }}
           cache-dependency-path: |
             datatrails-common-api/go.sum
+
       - name: Setup protobuf
         uses: arduino/setup-protoc@v2
         with:
           version: "24.3"
           repo-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Setup protoc-gen-doc
+        run: |
+          # download protoc-gen-doc as binary
+          go install github.com/pseudomuto/protoc-gen-doc/cmd/[email protected]
+          which protoc-gen-doc
+
       - name: Generate, Test and Export
         run: |
           task all
 
       - name: Show exports
         working-directory: exported/
-        run: find . -type d
+        run: find . -type f
 
       - name: Publish release artifacts
         if: ${{ github.event_name == 'release' }}

README.md

@@ -2,19 +2,15 @@
 
 Common public api definitions for the DataTrails platform
 
-## Finding and including proto files for depdendecnies
+## Finding and including proto files for dependencies
 
-tools/go.mod is the source of truth for all proto providing dependencies. That file alone specifies both the upstream version we are using *and* is used, via go install, to make the .proto files available locally
+go.mod is the source of truth for all proto providing dependencies. That file alone specifies both the upstream version we are using *and* is used, via go get -tool, to make the .proto files available locally
 
-This corresponds to the practice recommended by grpc-gateway and elsewhere
+This corresponds to the practice recommended by grpc-gateway and elsewhere.
 
-1. *** ALWAYS *** Use tools/go.mod to specify the dependency version.
-2. Add the package to the `go install` command in the apis:preflight task
-3. If necessary, add a var for the new path in any-api **and** then add a reference to that var in the PROTO_INC var.
-
-Following this practice removes the need for dual maintenance of dependency versions in the builder image. It also produces a build cycle that is significantly faster.
-
-Cross repository builds in docker while using go.work to refer to locally modified sources don't work. And this setup is essential for an efficient workflow.
+1. Execute the `go get -tool` command to add the package to the go.mod file.
+2. If necessary, add a var for the new path in any-api **and** then add a reference to that var in the PROTO_INC var.
+3. Executing 'go install tool' is then sufficient to install required tools.
 
 ## bootstrap proto files
 
@@ -24,29 +20,12 @@ For this reason we curl the proto's and make them available in our aggregate pro
 
 ## Workflow for updating common apis
 
-### Ensure the go tool chain is setup on your host
-
-It is inordinately cumbersome to set this all up for docker. It can't be
-achieved without imposing significant inneficiencies in the workflow. For any
-engineer working on the api's it is reasonable to expect they can install the
-correct version of go for their host, and likely already have.
-
-### Use an avid/src/go.work
-
-First point avid at a clone of this repo so that it's imports will resolve
-directly against the sources in `./api`
+### Ensure tools are setup on your host
 
-### Familiarize with the workflow sub tasks
-
-The general use entry points all use Docker and do not support efficiently and
-iteratively developing avid and other dependents against a local clone of this
-repo.
+This repo use the go-task tool and also the go toolchain.
 
 #### For this repo
 
-You can do everything in the builder image using `task all`. This uses the
-blessed builder image and requires no host setup other than docker.
-
 If you are iterating it is a lot more efficient to use the sub tasks, because they all run on your host.
 
 `task` `apis:bootstrap` -> `task:apis:generate` -> `task:apis:test` -> `task:apis:export`
@@ -68,15 +47,11 @@ If you want to iterate on *just* the helper go code and there tests, do one roun
 
 Then just iterate using `task apis:generate` and `task apis:test`
 
-#### For avid
-
-See the README.md in avid/src/api/README.md
-
 ##### Build one api against locally cloned go-datatrails-common-api
 
 The protos can be included exactly as they are found from a clone of go-datatrails-common-api. *Without* needing to generate, build or export. Eg,
 
     task apis:assetsv2-api \
         DATATRAILS_COMMON_API="../../go-datatrails-common-api"
 
-It is necessary however to run `task apis:bootsrap` after cloning go-datatrails-common
+It is necessary however to run `task apis:bootstrap` after cloning go-datatrails-common-api.

datatrails-common-api/.gitignore

@@ -1 +1,5 @@
+marshalers/
 go.work
+*.go
+*.json
+*.storage