ci(docker): incorporate docker release enhancements from uv (#13274)

## Summary This PR updates `ruff` to match `uv` updated [docker releases approach](https://github.com/astral-sh/uv/blob/main/.github/workflows/build-docker.yml). It's a combined PR with changes from these PR's * https://github.com/astral-sh/uv/pull/6053 * https://github.com/astral-sh/uv/pull/6556 * https://github.com/astral-sh/uv/pull/6734 * https://github.com/astral-sh/uv/pull/7568 Summary of changes / features 1. This change would publish an additional tags that includes only `major.minor`. For a release with `x.y.z`, this would publish the tags: * ghcr.io/astral-sh/ruff:latest * ghcr.io/astral-sh/ruff:x.y.z * ghcr.io/astral-sh/ruff:x.y 2. Parallelizes multi-platform builds using multiple workers (hence the new docker-build / docker-publish jobs), which cuts docker releases time in half. 3. This PR introduces additional images with the ruff binaries from scratch for both amd64/arm64 and makes the mapping easy to configure by generating the Dockerfile on the fly. This approach focuses on minimizing CI time by taking advantage of dedicating a worker per mapping (20-30s~ per job). For example, on release `x.y.z`, this will publish the following image tags with format `ghcr.io/astral-sh/ruff:{tag}` with manifests for both amd64/arm64. This also include `x.y` tags for each respective additional tag. Note, this version does not include the python based images, unlike `uv`. * From **scratch**: `latest`, `x.y.z`, `x.y` (currently being published) * From **alpine:3.20**: `alpine`, `alpine3.20`, `x.y.z-alpine`, `x.y.z-alpine3.20` * From **debian:bookworm-slim**: `debian-slim`, `bookworm-slim`, `x.y.z-debian-slim`, `x.y.z-bookworm-slim` * From **buildpack-deps:bookworm**: `debian`, `bookworm`, `x.y.z-debian`, `x.y.z-bookworm` 4. This PR also fixes `org.opencontainers.image.version` for all tags (including the one from `scratch`) to contain the right release version instead of branch name `main` (current behavior). ``` > docker inspect ghcr.io/astral-sh/ruff:0.6.4 | jq -r '.[0].Config.Labels' { ... "org.opencontainers.image.version": "main" } ``` Closes https://github.com/astral-sh/ruff/issues/13481 ## Test Plan Approach mimics `uv` with almost no changes so risk is low but I still tested the full workflow. * I have a working CI release pipeline on my fork run 1096665773 * The resulting images were published to https://github.com/samypr100/ruff/pkgs/container/ruff
2025-07-07 13:15:06 +00:00 · 2024-10-22 12:06:49 +00:00 · 2024-10-22 12:06:49 +00:00 · 7dbd8f0f8e
commit 7dbd8f0f8e
parent 46c0961b0b
3 changed files with 262 additions and 15 deletions
--- a/.github/workflows/build-docker.yml
+++ b/.github/workflows/build-docker.yml
@ -17,12 +17,21 @@ on:
    paths:
      - .github/workflows/build-docker.yml

+env:
+  RUFF_BASE_IMG: ghcr.io/${{ github.repository_owner }}/ruff
+
 jobs:
-  docker-publish:
-    name: Build Docker image (ghcr.io/astral-sh/ruff)
+  docker-build:
+    name: Build Docker image (ghcr.io/astral-sh/ruff) for ${{ matrix.platform }}
    runs-on: ubuntu-latest
    environment:
      name: release
+    strategy:
+      fail-fast: false
+      matrix:
+        platform:
+          - linux/amd64
+          - linux/arm64
    steps:
      - uses: actions/checkout@v4
        with:
@ -36,12 +45,6 @@ jobs:
          username: ${{ github.repository_owner }}
          password: ${{ secrets.GITHUB_TOKEN }}

-      - name: Extract metadata (tags, labels) for Docker
-        id: meta
-        uses: docker/metadata-action@v5
-        with:
-          images: ghcr.io/astral-sh/ruff
-
      - name: Check tag consistency
        if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
        run: |
@ -55,14 +58,233 @@ jobs:
            echo "Releasing ${version}"
          fi

-      - name: "Build and push Docker image"
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.RUFF_BASE_IMG }}
+          # Defining this makes sure the org.opencontainers.image.version OCI label becomes the actual release version and not the branch name
+          tags: |
+            type=raw,value=dry-run,enable=${{ inputs.plan == '' || fromJson(inputs.plan).announcement_tag_is_implicit }}
+            type=pep440,pattern={{ version }},value=${{ inputs.plan != '' && fromJson(inputs.plan).announcement_tag || 'dry-run' }},enable=${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
+
+      - name: Normalize Platform Pair (replace / with -)
+        run: |
+          platform=${{ matrix.platform }}
+          echo "PLATFORM_TUPLE=${platform//\//-}" >> $GITHUB_ENV
+
+      # Adapted from https://docs.docker.com/build/ci/github-actions/multi-platform/
+      - name: Build and push by digest
+        id: build
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          platforms: ${{ matrix.platform }}
+          cache-from: type=gha,scope=ruff-${{ env.PLATFORM_TUPLE }}
+          cache-to: type=gha,mode=min,scope=ruff-${{ env.PLATFORM_TUPLE }}
+          labels: ${{ steps.meta.outputs.labels }}
+          outputs: type=image,name=${{ env.RUFF_BASE_IMG }},push-by-digest=true,name-canonical=true,push=${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
+
+      - name: Export digests
+        run: |
+          mkdir -p /tmp/digests
+          digest="${{ steps.build.outputs.digest }}"
+          touch "/tmp/digests/${digest#sha256:}"
+
+      - name: Upload digests
+        uses: actions/upload-artifact@v4
+        with:
+          name: digests-${{ env.PLATFORM_TUPLE }}
+          path: /tmp/digests/*
+          if-no-files-found: error
+          retention-days: 1
+
+  docker-publish:
+    name: Publish Docker image (ghcr.io/astral-sh/ruff)
+    runs-on: ubuntu-latest
+    environment:
+      name: release
+    needs:
+      - docker-build
+    if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
+    steps:
+      - name: Download digests
+        uses: actions/download-artifact@v4
+        with:
+          path: /tmp/digests
+          pattern: digests-*
+          merge-multiple: true
+
+      - uses: docker/setup-buildx-action@v3
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.RUFF_BASE_IMG }}
+          # Order is on purpose such that the label org.opencontainers.image.version has the first pattern with the full version
+          tags: |
+            type=pep440,pattern={{ version }},value=${{ fromJson(inputs.plan).announcement_tag }}
+            type=pep440,pattern={{ major }}.{{ minor }},value=${{ fromJson(inputs.plan).announcement_tag }}
+
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      # Adapted from https://docs.docker.com/build/ci/github-actions/multi-platform/
+      - name: Create manifest list and push
+        working-directory: /tmp/digests
+        # The jq command expands the docker/metadata json "tags" array entry to `-t tag1 -t tag2 ...` for each tag in the array
+        # The printf will expand the base image with the `<RUFF_BASE_IMG>@sha256:<sha256> ...` for each sha256 in the directory
+        # The final command becomes `docker buildx imagetools create -t tag1 -t tag2 ... <RUFF_BASE_IMG>@sha256:<sha256_1> <RUFF_BASE_IMG>@sha256:<sha256_2> ...`
+        run: |
+          docker buildx imagetools create \
+            $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
+            $(printf '${{ env.RUFF_BASE_IMG }}@sha256:%s ' *)
+
+  docker-publish-extra:
+    name: Publish additional Docker image based on ${{ matrix.image-mapping }}
+    runs-on: ubuntu-latest
+    environment:
+      name: release
+    needs:
+      - docker-publish
+    if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
+    strategy:
+      fail-fast: false
+      matrix:
+        # Mapping of base image followed by a comma followed by one or more base tags (comma separated)
+        # Note, org.opencontainers.image.version label will use the first base tag (use the most specific tag first)
+        image-mapping:
+          - alpine:3.20,alpine3.20,alpine
+          - debian:bookworm-slim,bookworm-slim,debian-slim
+          - buildpack-deps:bookworm,bookworm,debian
+    steps:
+      - uses: docker/setup-buildx-action@v3
+
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Generate Dynamic Dockerfile Tags
+        shell: bash
+        run: |
+          set -euo pipefail
+
+          # Extract the image and tags from the matrix variable
+          IFS=',' read -r BASE_IMAGE BASE_TAGS <<< "${{ matrix.image-mapping }}"
+
+          # Generate Dockerfile content
+          cat <<EOF > Dockerfile
+          FROM ${BASE_IMAGE}
+          COPY --from=${{ env.RUFF_BASE_IMG }}:latest /ruff /usr/local/bin/ruff
+          ENTRYPOINT []
+          CMD ["/usr/local/bin/ruff"]
+          EOF
+
+          # Initialize a variable to store all tag docker metadata patterns
+          TAG_PATTERNS=""
+
+          # Loop through all base tags and append its docker metadata pattern to the list
+          # Order is on purpose such that the label org.opencontainers.image.version has the first pattern with the full version
+          IFS=','; for TAG in ${BASE_TAGS}; do
+            TAG_PATTERNS="${TAG_PATTERNS}type=pep440,pattern={{ version }},suffix=-${TAG},value=${{ fromJson(inputs.plan).announcement_tag }}\n"
+            TAG_PATTERNS="${TAG_PATTERNS}type=pep440,pattern={{ major }}.{{ minor }},suffix=-${TAG},value=${{ fromJson(inputs.plan).announcement_tag }}\n"
+            TAG_PATTERNS="${TAG_PATTERNS}type=raw,value=${TAG}\n"
+          done
+
+          # Remove the trailing newline from the pattern list
+          TAG_PATTERNS="${TAG_PATTERNS%\\n}"
+
+          # Export image cache name
+          echo "IMAGE_REF=${BASE_IMAGE//:/-}" >> $GITHUB_ENV
+
+          # Export tag patterns using the multiline env var syntax
+          {
+            echo "TAG_PATTERNS<<EOF"
+            echo -e "${TAG_PATTERNS}"
+            echo EOF
+          } >> $GITHUB_ENV
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        # ghcr.io prefers index level annotations
+        env:
+          DOCKER_METADATA_ANNOTATIONS_LEVELS: index
+        with:
+          images: ${{ env.RUFF_BASE_IMG }}
+          flavor: |
+            latest=false
+          tags: |
+            ${{ env.TAG_PATTERNS }}
+
+      - name: Build and push
        uses: docker/build-push-action@v6
        with:
          context: .
          platforms: linux/amd64,linux/arm64
-          # Reuse the builder
-          cache-from: type=gha
-          cache-to: type=gha,mode=max
-          push: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
-          tags: ghcr.io/astral-sh/ruff:latest,ghcr.io/astral-sh/ruff:${{ (inputs.plan != '' && fromJson(inputs.plan).announcement_tag) || 'dry-run' }}
+          # We do not really need to cache here as the Dockerfile is tiny
+          #cache-from: type=gha,scope=ruff-${{ env.IMAGE_REF }}
+          #cache-to: type=gha,mode=min,scope=ruff-${{ env.IMAGE_REF }}
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
+          annotations: ${{ steps.meta.outputs.annotations }}
+
+  # This is effectively a duplicate of `docker-publish` to make https://github.com/astral-sh/ruff/pkgs/container/ruff
+  # show the ruff base image first since GitHub always shows the last updated image digests
+  # This works by annotating the original digests (previously non-annotated) which triggers an update to ghcr.io
+  docker-republish:
+    name: Annotate Docker image (ghcr.io/astral-sh/ruff)
+    runs-on: ubuntu-latest
+    environment:
+      name: release
+    needs:
+      - docker-publish-extra
+    if: ${{ inputs.plan != '' && !fromJson(inputs.plan).announcement_tag_is_implicit }}
+    steps:
+      - name: Download digests
+        uses: actions/download-artifact@v4
+        with:
+          path: /tmp/digests
+          pattern: digests-*
+          merge-multiple: true
+
+      - uses: docker/setup-buildx-action@v3
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        env:
+          DOCKER_METADATA_ANNOTATIONS_LEVELS: index
+        with:
+          images: ${{ env.RUFF_BASE_IMG }}
+          # Order is on purpose such that the label org.opencontainers.image.version has the first pattern with the full version
+          tags: |
+            type=pep440,pattern={{ version }},value=${{ fromJson(inputs.plan).announcement_tag }}
+            type=pep440,pattern={{ major }}.{{ minor }},value=${{ fromJson(inputs.plan).announcement_tag }}
+
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      # Adapted from https://docs.docker.com/build/ci/github-actions/multi-platform/
+      - name: Create manifest list and push
+        working-directory: /tmp/digests
+        # The readarray part is used to make sure the quoting and special characters are preserved on expansion (e.g. spaces)
+        # The jq command expands the docker/metadata json "tags" array entry to `-t tag1 -t tag2 ...` for each tag in the array
+        # The printf will expand the base image with the `<RUFF_BASE_IMG>@sha256:<sha256> ...` for each sha256 in the directory
+        # The final command becomes `docker buildx imagetools create -t tag1 -t tag2 ... <RUFF_BASE_IMG>@sha256:<sha256_1> <RUFF_BASE_IMG>@sha256:<sha256_2> ...`
+        run: |
+          readarray -t lines <<< "$DOCKER_METADATA_OUTPUT_ANNOTATIONS"; annotations=(); for line in "${lines[@]}"; do annotations+=(--annotation "$line"); done
+          docker buildx imagetools create \
+            "${annotations[@]}" \
+            $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
+            $(printf '${{ env.RUFF_BASE_IMG }}@sha256:%s ' *)
--- a/2
+++ b/2
@ -1,4 +1,4 @@
-FROM --platform=$BUILDPLATFORM ubuntu as build
+FROM --platform=$BUILDPLATFORM ubuntu AS build
 ENV HOME="/root"
 WORKDIR $HOME

--- a/docs/integrations.md
+++ b/docs/integrations.md
@ -131,3 +131,28 @@ When running without `--fix`, Ruff's formatter hook can be placed before or afte
 [mdformat](https://mdformat.readthedocs.io/en/stable/users/plugins.html#code-formatter-plugins) is
 capable of formatting code blocks within Markdown. The [`mdformat-ruff`](https://github.com/Freed-Wu/mdformat-ruff)
 plugin enables mdformat to format Python code blocks with Ruff.
+
+
+## Docker
+
+Ruff provides a distroless Docker image including the `ruff` binary. The following tags are published:
+
+- `ruff:latest`
+- `ruff:{major}.{minor}.{patch}`, e.g., `ruff:0.6.6`
+- `ruff:{major}.{minor}`, e.g., `ruff:0.6` (the latest patch version)
+
+In addition, ruff publishes the following images:
+
+<!-- prettier-ignore -->
+- Based on `alpine:3.20`:
+  - `ruff:alpine`
+  - `ruff:alpine3.20`
+- Based on `debian:bookworm-slim`:
+  - `ruff:debian-slim`
+  - `ruff:bookworm-slim`
+- Based on `buildpack-deps:bookworm`:
+  - `ruff:debian`
+  - `ruff:bookworm`
+
+As with the distroless image, each image is published with ruff version tags as
+`ruff:{major}.{minor}.{patch}-{base}` and `ruff:{major}.{minor}-{base}`, e.g., `ruff:0.6.6-alpine`.