Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Artifactory for image registry #911

Merged
merged 10 commits into from
Nov 18, 2024
Merged

Conversation

kevinAlbs
Copy link
Contributor

This PR is intended to address rate-limiting errors pulling from dockerhub causing task failures on Evergreen. Example:

docker: Error response from daemon: toomanyrequests: Too Many Requests (HAP429).

DEVPROD-12779 notes:

In the meantime there is an artifactory mirror for dockerhub that can be used for pull containers.

Running docker pull artifactory.corp.mongodb.com/dockerhub/<DOCKER_IMAGE>:<DOCKER_TAG> appears to pull and add the image to the artifactory registry. DEVPROD-12779 also notes:

The proper long term solution seems to be to use authentication for pull requests as suggested by the Docker support.

Getting credentials is tracked in DRIVERS-3047.

As an added fix, the check-format target is updated to install make to address an observed error:

+check-format *failed* | Some possibly relevant errors from pip install:
+check-format *failed* |     error: subprocess-exited-with-error
+check-format *failed* |     CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
+check-format *failed* |     ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (clang-format)

Verified with this patch build

@kevinAlbs kevinAlbs requested a review from eramongodb November 18, 2024 16:45
To avoid "not implemented" error referenced in mongodb/mongo-c-driver#1794
To agree with Evergreen distro best practices: "If selecting Ubuntu, for example, prefer a 22.04 (or 20.04) distro version over 18.04 if you can."
@kevinAlbs
Copy link
Contributor Author

docker login is added to authenticate to Artifactory.

Docker distros are updated to remove ubuntu2004 (after seeing note in mongodb/mongo-c-driver#1794) and ubuntu1804 to follow Evergreen guidelines.

--persist-build=false is passed to Earthly commands to address an observed failure logging:

internal | WARN: The internal command 'docker-image://docker.io/docker/dockerfile-copy:v0.1.9@sha256:e8f159d3f00786604b93c675ee2783f8dc194bb565e61ca5788f6a6e9d304061' failed: failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3Adocker%2Fdockerfile-copy%3Apull&service=registry.docker.io: 429 Too Many Requests

I expect this is due to the CACHE command in the Earthfile (not present in C driver). Earthly appears to hardcode use of a dockerhub image to implement the CACHE command:

// This only has an effect when the Target contains at least one `CACHE /my/directory` command.

Filed MONGOCRYPT-747 to track removing workaround.

Latest changes verified by this patch build.

@kevinAlbs kevinAlbs requested a review from eramongodb November 18, 2024 19:26
.evergreen/config.yml Show resolved Hide resolved
@kevinAlbs kevinAlbs merged commit 7890557 into mongodb:master Nov 18, 2024
41 of 44 checks passed
joshbsiegel pushed a commit to joshbsiegel/libmongocrypt that referenced this pull request Nov 18, 2024
* replace dockerhub references with artifactory
* install `make` to build clang-format
* remove ubuntu2004 from docker distros.
  * Avoids "not implemented" error referenced in mongodb/mongo-c-driver#1794
* replace ubuntu1804 with ubuntu2204 in docker distros
  * Agrees with Evergreen distro best practices.
* pass `--persist-build=false` to avoid internal use of Docker Hub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants