Unable to authenticate to private Google Artifact Registry for gradle packages

[andyfrith-rm]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab SaaS, Renovate 38.142

Please tell us more about your question or problem

We are migrating from an on-premise Nexus, which did not require authentication for retrieving packages, to Google Artifact Registry, which does. Renovate was working successfully against Nexus.

The configuration below works for NPM packages and Docker images, but not for Gradle packages. This suggests to me that the credentials themselves are correct, but something in the configuration is not quite right.

We get 401 errors, for both gradle plugins and regular libraries. Java libraries from Maven Central are found no problem.

I have masked the project-name, company-name and gitlab-id but all other config is an exact representation.

Renovate runner config:

  RENOVATE_EXTRA_FLAGS: >
    --autodiscover=true
    --autodiscover-filter=$CI_PROJECT_NAMESPACE/**
  GITHUB_COM_TOKEN: $RENOVATE_GITHUB_COM_TOKEN
  RENOVATE_HOST_RULES: '[{"matchHost": "gitlab.com", "token": "$RENOVATE_TOKEN"},
  {"matchHost":"https://europe-west2-npm.pkg.dev/<project-name>/npm-all/","token":"{{ secrets.ARTIFACT_REGISTRY_NPM_RELEASE_PASSWORD }}"},
  {"matchHost":"https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/","username": "_json_key_base64","password": "{{ secrets.ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD }}"},
  {"matchHost":"https://europe-west2-docker.pkg.dev/<project-name>","username":"_json_key_base64","password":"{{ secrets.ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD }}"}]'

include: '/templates/renovate.gitlab-ci.yml'

renovate:
  before_script:
    - |
      echo "Running renovate with extra flags: $RENOVATE_EXTRA_FLAGS"
    - echo "@<company-name>-internal:registry=https://europe-west2-npm.pkg.dev/<project-name>/npm-all/" >> .npmrc
    - echo "//europe-west2-npm.pkg.dev/<project-name>/npm-all/:always-auth=true" >> .npmrc
    - sa_key=$(mktemp)
    - echo $PIPELINE_SA_GCP_KEY > $sa_key
    - export GOOGLE_APPLICATION_CREDENTIALS=$sa_key
    - npx google-artifactregistry-auth --repo-config ./.npmrc --credential-config ./.npmrc
    - export RENOVATE_NPMRC=$(sed 's/$/\\n/' .npmrc | tr -d '\n' )
    - export NPM_TOKEN=$(cat .npmrc | grep _authToken | cut -d '=' -f2)
    - export RENOVATE_SECRETS="{\"ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD\":\"${GCP_SA_KEY_BASE64}\", \"ARTIFACT_REGISTRY_NPM_RELEASE_PASSWORD\":\"${NPM_TOKEN}\"}"
  rules:
    - when: always
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
    - if: '$CI_PIPELINE_SOURCE == "push"'
  artifacts:
    paths:
      - renovate-log.ndjson

Target project renovate.json:
{ "$schema": "https://docs.renovatebot.com/renovate-schema.json", "extends": ["local><gitlab-id>/renovate-config/renovate-config//config/default.json"], "packageRules": [ { "matchPackageNames": ["com.<company-name>**"], "matchDatasources": ["maven","gradle"], "registryUrls": ["https://europe-west2-maven.pkg.dev/<project-name>/gradle-all"] } ] }

Logs (if relevant)

Logs

DEBUG: GET https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/com/<company-name>/build-properties/com.<company-name>.build-properties.gradle.plugin/maven-metadata.xml = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=401 retryCount=0, duration=43) (repository=r2617/developer-experience/microservices/application-data-centre-status)
DEBUG: Dependency lookup unauthorized. Please add authentication with a hostRule for https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/com/<company-name>/build-properties/com.<company-name>.build-properties.gradle.plugin/maven-metadata.xml (repository=r2617/developer-experience/microservices/application-data-centre-status)
DEBUG: Content is not found for Maven url (repository=r2617/developer-experience/microservices/application-data-centre-status)
       "url": "https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/com/<company-name>/build-properties/com.<company-name>.build-properties.gradle.plugin/maven-metadata.xml"

[rarkins]

Dependency lookup unauthorized. Please add authentication with a hostRule indicates that no credentials have been detected/used. You show no hostRules anywhere either. How would Renovate know to use credentials for maven datasource lookups?

[andyfrith-rm]

The host rules are defined in the above config

RENOVATE_HOST_RULES: '[{"matchHost": "gitlab.com", "token": "$RENOVATE_TOKEN"},
  {"matchHost":"https://europe-west2-npm.pkg.dev/<project-name>/npm-all/","token":"{{ secrets.ARTIFACT_REGISTRY_NPM_RELEASE_PASSWORD }}"},
  {"matchHost":"https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/","username": "_json_key_base64","password": "{{ secrets.ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD }}"},
  {"matchHost":"https://europe-west2-docker.pkg.dev/<project-name>","username":"_json_key_base64","password":"{{ secrets.ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD }}"}]'

From the debug log

DEBUG: Combined config
       "config": {
         "hostRules": [
           {"matchHost": "gitlab.com", "token": "***********"},
           {
             "matchHost": "https://europe-west2-npm.pkg.dev/<project-name>/npm-all/",
             "token": "{{ secrets.ARTIFACT_REGISTRY_NPM_RELEASE_PASSWORD }}"
           },
           {
             "matchHost": "https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/",
             "username": "_json_key_base64",
             "password": "{{ secrets.ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD }}"
           },
           {
             "matchHost": "https://europe-west2-docker.pkg.dev/<project-name>",
             "username": "_json_key_base64",
             "password": "{{ secrets.ARTIFACT_REGISTRY_MAVEN_RELEASE_PASSWORD }}"
           },
           {"hostType": "github", "matchHost": "github.com", "token": "***********"}
         ],

DEBUG: Adding token authentication for https://europe-west2-npm.pkg.dev/<project-name>/npm-all/ (hostType=undefined) to hostRules
DEBUG: Adding password authentication for https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/ (hostType=undefined) to hostRules
DEBUG: Adding password authentication for https://europe-west2-docker.pkg.dev/<project-name> (hostType=undefined) to hostRules

[rarkins]

Thanks, I searched for hostRules and missed HOST_RULES.

Nevertheless, this line applies:

DEBUG: Dependency lookup unauthorized. Please add authentication with a hostRule for https://europe-west2-maven.pkg.dev/<project-name>/gradle-all/com/<company-name>/build-properties/com.<company-name>.build-properties.gradle.plugin/maven-metadata.xml

This implies that no authentication was present for this request, which implies no hostRules matched it.

The log message you printed just logs that the hostRules were registered, not that they were applied. You can search for the string "hostRules" in logs to find more applicable messages.

[andyfrith-rm]

Thanks for the clarification - I can see further down in the log it does appear to apply

DEBUG: hostRules: applying Basic authentication for europe-west2-maven.pkg.dev (repository=<gitlab-id>/developer-experience/microservices/application-data-centre-status)

I appreciate it's difficult as I can't share the whole unredacted log

[github-actions[bot]]

Hi there,

The maintainers have labeled this discussion as a use case specific to "enterprise" users. Enterprise users typically self-host Renovate and have advanced requirements like custom certificates, HTTP proxies, private registries, air gapping, etc.

Such cases usually need a lot of back-and-forth to resolve, because the problem is related to the user's environment. Usually the user's environment can not be reproduced publicly, which means that maintainer debugging is not possible.

The maintainers decided their time is best spent on issues that help large numbers of users. This means enterprise-specific issues affecting one user (or a small number of users), are not a priority. If we can help you quickly, we will. But if you see this message, then a maintainer believes this is not a quick fix. The maintainers will not actively work on this issue, unless new information is found.

If you are an enterprise user, please consider the following options:

• Troubleshoot and contribute a fix yourself, or
• Hire a consultant to help you, or
• Consider an Enterprise support contract from Mend.io, or
• Wait for a community member to help you, if they have the same problem

If you choose to wait, then this discussion can remain open indefinitely. But please be patient, do not nudge or bump this discussion just to get attention.

Thanks, the Renovate team

[github-actions[bot]]

Hi there,

You redacted too much information from your config or logs. This makes it hard for us to pinpoint the problem.

Please only redact truly sensitive information. Or consider an enterprise support contract, if Open Source support does not suit you.

Thanks, the Renovate team