build: Build amd64 and arm64 multi-platform Docker image

[chadlwilson]

Description of Change

Uses docker buildx to build and push the docker image for linux/arm64 as well as existing linux/amd64

• No changes to docker workflow secrets should be needed
• Implementation builds the image for both platforms, and then loads into docker for the native platform (--load)
• Adds testing of the docker image to the PR workflow so contributors can also tweak the docker build more safely
• Currently only runs docker tests for the "native" image for the platform, i.e linux/amd64 on default runners.
  • If we want to specifically test the arm64 image, we could split to test on arm runners and/or force a test of the emulated images. However, it's a bit clunky with the current scripting based docker build (as opposed to using the docker actions that make this easier) - especially if you want to build once, and then test/push the exact same built image, rather than rebuilding.
• Since my local platform is arm64-native, I have run test-docker.sh locally on the arm64 image though

Sample images built and pushed using this approach: https://hub.docker.com/repository/docker/chadwilson/dependency-check/tags

See sample actions run at https://github.com/dependency-check/DependencyCheck/actions/runs/17936406550/job/51003624010?pr=7952

Related issues

• fixes Include ARM build docker #6420

Have test cases been added to cover the new functionality?

yes

[chadlwilson]

@jeremylong perhaps have a go with this locally to make sure build+test works for you? If you use MacOS you might need to brew install docker-buildx to get the plugin installed if not already.

I actively use buildx for other projects so I might have forgotten magic I did to initialize the builders etc which is otherwise taken care of by the GitHub action.

if so, I can improve the scripting to create a deterministic builder - or ensure it creates one if not already created.

[marcelstoer]

I quickly tested this on macOS. buildx has been installed for years, but the --load flag fails the build

docker exporter does not currently support exporting manifest lists

One needs to enable "Use containerd for pulling and storing images" in the settings. Is this the same reason you added the daemon-config?

[chadlwilson]

@marcelstoer thanks for that. Are you using docker desktop or something like Colima, rancher etc?

I can understand why you’d get this with Docker Desktop given its default configuration, but not sure what my Colima install is doing to avoid this (I am using Colima with docker runtime)

And yes, this is why the containerd snapshotter is being used on GHA. I thought it was perhaps a linux-only problem, where docker is native, but seems not.

The alternative is to add more steps to manually export from the builder and then later import/load a single-arch tar into docker but it’s a pain-in-the-ass without slopes/regctl so I was hoping the simpler option works. As you can see from docker/buildx#59 (comment) this is an irritating saga that has been going on for years without simple functionality being addressed in docker so allow basic interoperability between buildx and the runtime/engine.

[chadlwilson]

Ahh, I see now why I don’t have an issue: abiosoft/colima#1360

Would it be sufficient if I added a shell test to the build script to fail fast if this isn’t enabled and point to instructions?

[marcelstoer]

Yes, I'm using Docker Desktop which has the containerd flag disabled by default.

Would it be sufficient if I added a shell test to the build script to fail fast if this isn’t enabled and point to instructions?

That'd be way more than sufficient 😄 I have no objection to merging as-is as hardly anyone will run these scripts locally (I reckon).

At $work, I opted for less efficient but more stable and risk free pipelines. We run docker buildx in CI (w/o --load) and docker buildx --push for releases; both with exactly the same parameters. Of course, if you prune the build(x) cache in between, building such images takes a lot longer. Btw, in the pre-buildx days we also had a Docker build step followed by a Docker push step.

[chadlwilson]

Yeah, the issue is with running sanity tests before push, which requires you to get a built image into Docker Engine. Then you're in the land of all the 'traditional' commands on the docker CLI which cannot handle multi-arch images, nor OCI format etc.

--load from buildx is the most convenient for this purpose, and since docker seems to be moving toward the containerd snapshotter (as they did towards buildkit over time) it's probably ok for this purpose. Having said this, I don't feel super comfortable imposing a system wide setting on contributors (unless Jeremy is OK!) as it affects how all containers are run, I believe?

[jeremylong]

@chadlwilson can you add the test for the build failure that points to the setup instructions as you mentioned above? I'm okay with the PR - I just think this would be helpful as we have people build things locally a lot and I want to avoid as many questions as we can.

[chadlwilson]

Ack, will do shortly

[chadlwilson]

Rebased and added. Will now fail with:

Docker Engine is not running with the containerd snapshotter - this is currently needed to build and test ODC multi-platform images using docker buildx.
If using Docker Desktop, enable "Use containerd for pulling and storing images" per https://docs.docker.com/desktop/settings-and-maintenance/settings/#general
For more technical information on Docker Engine, see https://docs.docker.com/engine/storage/containerd/

[marcelstoer]

This is great, thanks!

[jeremylong]

LGTM