Use docker buildx bake to build images

[willcl-ark]

This is much cleaner than maintaining a list of complete shell commands.

[bdp-DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #692 (Add docker buildx bake and support for cmake by danvergara)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[willcl-ark]

Thanks Drahtbot, added the commit from #606

[danvergara]

bake is awesome and did not know about it.

[danvergara]

I'm running the commands provided as example, but can we add a reference on how to create a builder before running the buildx command? Mostly for folks not used to build images for multiple platforms.
Even that I'm used to it I always the forget the commands.

https://docs.docker.com/reference/cli/docker/buildx/create/
https://www.docker.com/blog/how-to-rapidly-build-multi-architecture-images-with-buildx/

[deadmanoz]

Agreed, the instructions should outline all required steps. Assuming we're ok with the docker-container build driver (which supports multi-arch image builds), can simply add this as a preliminary step

docker buildx create --name=warnet-builder --use --bootstrap

[danvergara]

Shouldn't this be as shown below?

Suggested change

	REPO = "bitcoin"
	REPO = "bitcoin/bitcoin"

[deadmanoz]

Referring to a comment in src/warnet/constants.py (replicated below), do you think somehow referencing or parsing the targets here would be a potential solution?

# Constants used throughout the project
# Storing as constants for now but we might want a more sophisticated config management
# at some point.
SUPPORTED_TAGS = ["27.0", "26.0", "25.1", "24.2", "23.2", "22.2"]

[deadmanoz]

Could leverage the tags of @Onyekachukwu-Nweke's suggestion to achieve this: #669 (comment)

[danvergara]

I opened this PR to include cmake for future releases. It's against the @willcl-ark's branch so it can be included in this PR.

willcl-ark#32

[deadmanoz]

We can eliminate the need to duplicate the target names in the sub-groups (maintained, practice, vulnerable) and the main all group by using variables and concat:

variable "maintained_targets" {
  default = ["bitcoin-28", "bitcoin-27", "bitcoin-26"]
}

variable "practice_targets" {
  default = ["bitcoin-unknown-message", "bitcoin-invalid-blocks", "bitcoin-50-orphans", "bitcoin-no-mp-trim", "bitcoin-disabled-opcodes", "bitcoin-5k-inv"]
}

variable "vulnerable_targets" {
  default = ["v0-21-1", "v0-20-0", "v0-19-2", "v0-17-0", "v0-16-1"]
}

group "maintained" {
  targets = maintained_targets
}

group "practice" {
  targets = practice_targets
}

group "vulnerable" {
  targets = vulnerable_targets
}

group "all" {
  targets = concat(maintained_targets, practice_targets, vulnerable_targets)
}

[Onyekachukwu-Nweke]

Implement a matrix pattern for version management

I recommend implementing a matrix pattern for targets that differ primarily by version and commit hash:

// Define a matrix of maintained versions
variable "MAINTAINED_VERSIONS" {
  default = {
    "28" = {
      tag = "28.0"
      commit = "110183746150428e6385880c79f8c5733b1361ba"
    }
    "27" = {
      tag = "27.2"
      commit = "bf03c458e994abab9be85486ed8a6d8813313579" 
    }
    "26" = {
      tag = "26.2"
      commit = "7b7041019ba5e7df7bde1416aa6916414a04f3db"
    }
  }
}

// Then reference in target definitions:
target "bitcoin-maintained" {
  name = "bitcoin-${key}"
  inherits = ["maintained-base"]
  matrix = {
    key = keys(variable.MAINTAINED_VERSIONS)
  }
  tags = ["bitcoindevproject/bitcoin:${variable.MAINTAINED_VERSIONS[key].tag}"]
  args = {
    COMMIT_SHA = "${variable.MAINTAINED_VERSIONS[key].commit}"
  }
}

This pattern could be extended to both practice-base and cve-base targets.

This approach:

1. Makes the configuration more maintainable
2. Reduces duplication
3. Makes it easier to add new versions
4. It creates a clear separation between configuration and implementation

[Onyekachukwu-Nweke]

For improved modularity, consider implementing this as a reusable variable."

variable "DEFAULT_PLATFORMS" {
  default = ["linux/amd64", "linux/arm64", "linux/arm/v7"]
}

platforms = "${variable.DEFAULT_PLATFORMS}"

[macgyver13]

variable. is not needed from my experience to reference a variable

variable "DEFAULT_PLATFORMS" {
  default = ["linux/amd64", "linux/arm64", "linux/arm/v7"]
}

platforms = "${DEFAULT_PLATFORMS}"

*Note: platform can be overridden as needed as well when baking
docker buildx bake <target> --set "*.platform="linux/arm64""

[Onyekachukwu-Nweke]

Yeah, works like @macgyver13 suggested

[Onyekachukwu-Nweke]

Following the same pattern as my previous comment, I recommend making this a reusable variable as well.