mirror_images: introduce dd-repo-tools wrapper and seed config#7022
Draft
pawelchcki wants to merge 1 commit into
Draft
mirror_images: introduce dd-repo-tools wrapper and seed config#7022pawelchcki wants to merge 1 commit into
pawelchcki wants to merge 1 commit into
Conversation
Adds `utils/mirror_images.sh` as a thin wrapper around the shared
`mirror_images.py` tool and seeds `mirror_images.yaml` with the public
images this repo's Dockerfiles and docker-compose files reference.
The image list covers the 29 real public images that should be mirrored
into `registry.ddbuild.io/ci/system-tests/mirror/...`. Ignore patterns
suppress 161 false-positive references the linter picks up:
- Dockerfile stage names (`base`, `build`, `dd-lib-init_<env>`)
- docker-compose local build targets (`system-tests/*:latest`,
`reverseproxy:latest`)
- Build-time ARG-substituted refs (`${TARGETARCH}`, `$RUNTIME`)
- `public.ecr.aws/*` (AWS's trusted public Docker Hub mirror)
The same source list also feeds the ghcr.io mirror for GitHub-hosted CI
by setting `MIRROR_DEST_REGISTRY=ghcr.io/datadog/system-tests` (see
header comment); no second yaml needed.
Dockerfile/docker-compose source files still reference public registries
directly — rewriting those to use the mirror prefix is a follow-up.
Contributor
|
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
utils/mirror_images.sh— thin wrapper around the shareddd-repo-tools/mirror_images.py(pinned to commitfb4f39a5).mirror_images.yamlwith the 29 real public images this repo's Dockerfiles/docker-compose files reference, plus ignore patterns that suppress 161 false-positive lint hits.What's covered
Mirrored (→
registry.ddbuild.io/ci/system-tests/mirror/...by default):mcr.microsoft.com/dotnet/{aspnet,sdk}:*— multiple versionsgolang:1.25,maven:3-eclipse-temurin-21,node:13,node:18.10-slim,python:2.7,python:3.11-slim,ruby:3.1.3,rust:1.87-slim-bookworm,ubuntu:22.04,openjdk:7-alpine,debian:bookworm-slim,amazonlinux:2023,apache/spark:3.4.4,datadog/dd-trace-ci:php-8.2_bookworm-6,docker.io/datadog/dd-lib-ruby-init:latestgcr.io/datadoghq/agent:7.78.4ghcr.io/datadog/dd-trace-rb/dd-lib-ruby-init:latest_snapshot669783387624.dkr.ecr.us-east-1.amazonaws.com/dockerhub/library/php:5.6-cliIgnored (regex patterns in
ignore.images):base,build,dd-lib-init_.+reverseproxy:latest,system-tests/.+:latest.*\${?RUNTIME}?.*,.*\${?TARGETARCH}?.*public\.ecr\.aws/.*— AWS's trusted public Docker Hub mirror, no re-mirroring neededMirroring to both registries
Same
mirror_images.yamlfeeds both destinations via env var (documented in the header):Follow-up
utils/mirror_images.sh lintstill reports 40 refs in 29 unique images — these are the actual DockerfileFROM/ docker-composeimage:lines that need to be rewritten to useregistry.ddbuild.io/ci/system-tests/mirror/.... That's a much larger refactor and is intentionally out of scope here.Test plan
utils/mirror_images.sh lintruns and only flags the expected 29 unmirrored sources (nobase/build/system-tests/*noise).utils/mirror_images.sh lockresolves digests for all declared images.utils/mirror_images.sh mirror --dry-runshows the expected copy plan.MIRROR_DEST_REGISTRY=ghcr.io/datadog/system-teststargets ghcr.io paths.