Make `--config` from buildx command string generation optional #1673

robfrank · 2023-05-08T11:06:35Z

As per this comment:

removing the "--config" let the sample project "multi-architecture" to build fine

robfrank · 2023-05-08T13:30:29Z

ok, let me fix also the tests

codecov · 2023-05-08T14:10:04Z

Codecov Report

Merging #1673 (1afb1e1) into master (cf48a2d) will increase coverage by 0.04%.
The diff coverage is 100.00%.

❗ Current head 1afb1e1 differs from pull request most recent head 0e7fd07. Consider uploading reports for the commit 0e7fd07 to get more accurate results

@@             Coverage Diff              @@
##             master    #1673      +/-   ##
============================================
+ Coverage     64.29%   64.34%   +0.04%     
- Complexity     2223     2225       +2     
============================================
  Files           172      172              
  Lines         10053    10059       +6     
  Branches       1381     1382       +1     
============================================
+ Hits           6464     6472       +8     
+ Misses         3051     3050       -1     
+ Partials        538      537       -1

Files Changed	Coverage Δ
...io/fabric8/maven/docker/service/BuildXService.java	`75.16% <100.00%> (+1.01%)`	⬆️

... and 1 file with indirect coverage changes

sonarqubecloud · 2023-05-08T16:37:15Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

100.0% Coverage
0.0% Duplication

rohanKanojia · 2023-05-08T16:40:00Z

@chonton : polite ping, Could you please review this PR whenever you get some time?

chonton · 2023-05-09T01:24:53Z

With this change, you are no longer isolated from the build environment. The docker configs for the current user are being used and changed.

The problems that this code change fixes could also probably be fixed by updating your docker and docker buildx to more recent version.

arauchberger · 2023-06-14T13:34:20Z

hi @chonton,

i think i'm on the latest DockerDesktop version (Docker version 24.0.2, build cb74dfc) and therefor on buildX Version github.com/docker/buildx v0.10.5 86bdced7766639d56baa4c7c449a4f6468490f87.

i would very much appreciate the PR-merge, this would be a great improvement to my build pipeline when providing multiarch images with no extra effort.

thanks in advance.

rohanKanojia · 2023-06-14T15:58:00Z

@arauchberger : Did you get some time to test this PR out? I think instead of dropping config straight away maybe we should do it conditionally (either only apply it if it's present or enable it via some flag)

rohanKanojia · 2023-06-14T16:02:16Z

src/main/java/io/fabric8/maven/docker/service/BuildXService.java

@@ -63,7 +63,7 @@ protected <C> void useBuilder(ProjectPaths projectPaths, ImageConfiguration imag
        BuildDirs buildDirs = new BuildDirs(projectPaths, imageConfig.getName());

        Path configPath = getDockerStateDir(imageConfig.getBuildConfiguration(),  buildDirs);
-        List<String> buildX = Arrays.asList("docker", "--config", configPath.toString(), "buildx");
+        List<String> buildX = Arrays.asList("docker", "buildx");


@robfrank : Sorry for the late reply, what was the value of configPath when you were debugging this issue?

for me it was : --config <path_to_my_git_repo>/product-pitdata/target/docker/index.docker.io/pineit/at.pineit.pitdata/latest/docker

for @robfrank i think it was: --config /<long path>/docker-maven-plugin/samples/multi-architecture/target/docker/hello/multiarch/0.43-SNAPSHOT/docker

arauchberger · 2023-06-14T18:15:24Z

I did test locally and it works fine.
In detail i do not really understand what exactly the problem is, but it definitely looks like, that if the buildkit is used (and therefor a container is started) credentials are somehow lost.

amizurov · 2023-06-18T13:57:32Z

Hey folks, I'm not sure if this plugin related bug created the issue in docker/for-mac#6928 project.

rhuss · 2023-07-19T16:35:30Z

@rohanKanojia @chonton I haven't followed the issue closely, but it looks like folks have problems by being forced to provide a configuration. It looks like that authentication information is still looked up in the private config directory managed via docker login etc. I understand that for reusable builds, the configuration should be pinned. Still, authentication should come from outside the project, so I wonder whether we can at least make it optional whether the local configuration (I haven't looked into it what this contains) should be used or, instead, the one from outside of the project (which possibly holds the authentication information). The ideal would be if the authentication config lookup would follow the strategy of http://dmp.fabric8.io/#authentication as it is for "normal" docker usage.

rohanKanojia · 2023-07-20T04:15:37Z

Will prioritize this for next release.

remove --config from command string generation add --node option to buildx command Signed-off-by: Roberto.Franchini <ro.franchini@gmail.com>

…empty Signed-off-by: Rohan Kumar <rohaan@redhat.com>

sonarqubecloud · 2023-07-28T17:45:18Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

100.0% Coverage
0.0% Duplication

amizurov · 2023-07-29T09:35:11Z

Folks i think we mixed here several issues one related to docker desktop for arm64 MacOs (docker/for-mac#6928) docker --config /bla-bla buildx this is not the plugin issue. And maybe some potential problem with authentication but i'm didn't look closely to it and i have no reproducer. Also i want to mentioned that we used buildx feature in our production environment and everything work properly before this change.

rohanKanojia · 2023-07-29T09:59:12Z

@amizurov : Are you using v0.43.2 in your production pipeline? I just released it this morning.

I think the change should not be breaking. It would only add --config if there is non empty dockerStateDir configured by the user.

amizurov · 2023-07-29T10:25:52Z

@amizurov : Are you using v0.43.2 in your production pipeline? I just released it this morning.

I think the change should not be breaking. It would only add --config if there is non empty dockerStateDir configured by the user.

No, we use previous versions 0.43.0 and 0.43.1. I'll checked new version as well.I also hope that this not break our builds, but i'm wondering some changes look strange (e.g. "--node", builderName + "0" what happens if docker team changes the naming of nodes, why we need set this name ?)

rohanKanojia · 2023-07-29T11:07:28Z

@amizurov : You're right.Rather than hardcoding it, it should be picked up from a field in BuildXConfiguration

rohanKanojia · 2023-08-13T18:14:02Z

@amizurov : I've released 0.43.3 . Could you please try again on this version and see if you're still facing this issue?

amizurov · 2023-08-13T18:32:09Z

@amizurov : I've released 0.43.3 . Could you please try again on this version and see if you're still facing this issue?

Sure, I'do it tomorrow, thanks a lot.

amizurov · 2023-08-14T14:58:23Z

@rohanKanojia Thanks a lot, looks like everything works properly in our case (with 0.43.3).

arauchberger · 2023-08-14T15:43:49Z

hi,
i don't remember why it was needed to add the --node, but i remember that i had to "patch it" this way to make it work for me.
now 0.43.3 works ask expected 'out of the box' for me too.

many thanks

oktayderman · 2024-12-06T08:40:51Z

folks I do have error for our private registry when using buildx
Head "...": dial tcp 195.214.161.37:5050: connect: connection refused

@arauchberger what should I do in the configuration of the plugin in pom.xml to make that plugin not add --config ?
I could not figure it out ? Can you give a example ? The BuildXService code seems to have --config without any condition Didnt understand how it may be omitted now.

List buildX = new ArrayList<>(Arrays.asList(DOCKER, "--config", configPath.toString(), "buildx"));

arauchberger · 2024-12-06T09:05:03Z

folks I do have error for our private registry when using buildx Head "...": dial tcp 195.214.161.37:5050: connect: connection refused

@arauchberger what should I do in the configuration of the plugin in pom.xml to make that plugin not add --config ? I could not figure it out ? Can you give a example ? The BuildXService code seems to have --config without any condition Didnt understand how it may be omitted now.

List buildX = new ArrayList<>(Arrays.asList(DOCKER, "--config", configPath.toString(), "buildx"));

Hi @oktayderman,

i'm currently using this version of the plugin

<plugin>
    <groupId>io.fabric8</groupId>
    <artifactId>docker-maven-plugin</artifactId>
    <version>0.45.1</version>
</plugin>

and this setup

<plugin>
    <groupId>io.fabric8</groupId>
    <artifactId>docker-maven-plugin</artifactId>
    <extensions>true</extensions>
    <configuration>
        <images>
            <image>
                <name>${docker.image.name}:%l</name>
                <alias>${docker.image.alias}</alias>
                <build>
                    <tags>
                        <tag>${docker.image.tag}</tag>
                        <tag>${docker.image.tag2}</tag>
                    </tags>
                    <contextDir>${project.basedir}</contextDir>
                    <dockerFile>${docker.dockerfile}</dockerFile>
                    <args>
                        <!-- used as Labels in dockerfile -->
                        <buildAppDesc>${project.description}</buildAppDesc>
                        <buildAppName>${project.artifactId}</buildAppName>
                        <buildBranch>${git.branch}</buildBranch>
                        <buildCommitId>${git.commit.id.abbrev}</buildCommitId>
                        <buildDate>${maven.build.timestamp}</buildDate>
                        <buildHost>${git.build.host}</buildHost>
                        <buildLic>${license.name}, ${license.url}</buildLic>
                        <buildOrg>${project.organization.name}, ${project.organization.url}</buildOrg>
                        <buildOrgDev>${organization.developer}</buildOrgDev>
                    </args>
                    <buildx>
                        <platforms>
                            <!-- linux/amd64,linux/arm64 -->
                            <platform>${docker.platforms}</platform>
                        </platforms>
                    </buildx>
                </build>
            </image>
        </images>
        <retries>3</retries>
    </configuration>
    <executions>
        <execution>
            <id>docker-build</id>
            <phase>install</phase>
            <goals>
                <goal>build</goal>
            </goals>
        </execution>
        <execution>
            <id>docker-push</id>
            <phase>deploy</phase>
            <goals>
                <goal>push</goal>
            </goals>
        </execution>
    </executions>
</plugin>

everything is working as expected now.

oktayderman · 2024-12-06T09:58:16Z

Hi arauchberger
thx for the quick response but same error still exists for me as --config and --arguments still coming at the generated docker command. Can you share your dockerfile also ?
I am referring a private registry image the problem is that for me. -->
FROM gitlab.i2i.systems:5050/i2i/ocs/fizz/fizz-source/base-image-17-jre:latest

[INFO] DOCKER> docker --config /myData/dev/projects/fizz-source/AOM/aom-parent/aomif/target/docker/aomif2/docker buildx build --progress=plain --builder maven --platform linux/amd64 --tag aomif2:latest --tag aomif2:latest --tag aomif2:20.0-SNAPSHOT --build-arg buildAppDesc=aomif --build-arg buildAppName=aomif --build-arg buildDate=06.12.2024_09:53:30 --build-arg buildOrg=${project.organization.name}, ${project.organization.url} --file=/myData/dev/projects/fizz-source/AOM/aom-parent/aomif/target/docker/aomif2/tmp/docker-build/Dockerfile /myData/dev/projects/fizz-source/AOM/aom-parent/aomif/target/docker/aomif2/tmp/docker-build --load

arauchberger · 2024-12-06T10:18:24Z

here is my Dockerfile. the storage thing is that i do not/no longer see the line with the docker command (as you just posted) in my output, even though i have set verbose=true for the plugin 🤷‍♂️

#####################################
# first stage
#
# package a minimal JRE
#####################################
FROM alpine AS packager

# install openjdk
RUN apk --no-cache add openjdk11-jdk openjdk11-jmods
ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk

# echo jdk version
RUN { \
  echo "OpenJDK Version: $($JAVA_HOME/bin/java --version)" ; \
  }

ENV JAVA_MINIMAL=/opt/jre

# build modules distribution
RUN $JAVA_HOME/bin/jlink \
  --verbose \
  --add-modules \
  jdk.management.agent,java.base,java.net.http,java.sql,java.naming,java.desktop,java.xml,jdk.crypto.cryptoki,jdk.unsupported,java.management,java.security.jgss \
  --compress 2 \
  --strip-debug \
  --no-header-files \
  --no-man-pages \
  --output "$JAVA_MINIMAL"

#####################################
# Second stage
#
# adding the minimal "JRE" distr from the first stage
# setup pineIT-Env and copy the application to the image
#####################################
FROM alpine

# Buildtime arguments to be overwritten while building with 'docker build --build-arg ARG=VAL '
ARG port=8904

# Buildtime arguments provided by maven bpl3 build routine
ARG buildAppName
ARG buildAppDesc
ARG buildBranch
ARG buildCommitId
ARG buildDate
ARG buildHost
ARG buildLic
ARG buildOrg
ARG buildOrgDev
ARG buildOS
ARG buildScmUrl
ARG buildTag

# Labels pineit
LABEL at.pineit.bpl3.build.branch=${buildBranch}
LABEL at.pineit.bpl3.build.host=${buildHost}
LABEL at.pineit.bpl3.build.os=${buildOS}
LABEL at.pineit.bpl3.build.tag=${buildTag}

# Lables OCI
LABEL org.opencontainers.image.authors=${buildOrgDev}
LABEL org.opencontainers.image.created=${buildDate}
LABEL org.opencontainers.image.description=${buildAppDesc}
LABEL org.opencontainers.image.licenses=${buildLic}
LABEL org.opencontainers.image.revision=${buildCommitId}
LABEL org.opencontainers.image.source=${buildScmUrl}
LABEL org.opencontainers.image.title=${buildAppName}
LABEL org.opencontainers.image.url=${buildScmUrl}
LABEL org.opencontainers.image.vendor=${buildOrg}


# Runtime environment for JRE
ENV JAVA_MINIMAL=/opt/jre
ENV PATH="$PATH:$JAVA_MINIMAL/bin"

# Runtime environment for pineIT
ENV PINEIT_HOME=/opt/pineit
ENV PINEIT_LIB_DIR=${PINEIT_HOME}/lib
ENV PINEIT_APP_MAIN="at.pineit.product.main.PineApp"

# Runtime environment for healthcheck
ENV HC_URL=http://localhost:${port}/pineit/${buildAppName}/healthz/ready

# Runtime environment to be overwritten via docker compose and k8s
ENV PCC_LOGBACKFILE=${PINEIT_HOME}/conf/${buildAppName}/logback.xml
ENV PCC_JAVA_OPTS="-Xms128M -Xmx128M"

# install some utils
RUN apk --no-cache add curl iputils

# Copy the minimal-java to image
COPY --from=packager "$JAVA_MINIMAL" "$JAVA_MINIMAL"

# create app directories
RUN mkdir -p ${PINEIT_LIB_DIR} && chgrp -R 0 ${PINEIT_HOME} && chmod -R g=u ${PINEIT_HOME}
# Copy all dependencies to the image
COPY /target/dependency/* ${PINEIT_LIB_DIR}/

# Expose some ports to the "outer rim"
EXPOSE ${port}

# Set working directory
WORKDIR ${PINEIT_HOME}

# Execute Java Main
ENTRYPOINT ["/bin/sh", "-c"]
CMD ["java $PCC_JAVA_OPTS -cp \"${PINEIT_LIB_DIR}/*\" -Dlogback.configurationFile=${PCC_LOGBACKFILE} ${PINEIT_APP_MAIN} $PCC_MAIN_ARGS"]

oktayderman · 2024-12-06T14:07:42Z

thx for the dockerfile you don't use a private registry so there will be no problem I opened an issue for this.
#1841

oktayderman · 2024-12-08T07:00:18Z

@arauchberger

I have resolved the problem by the reference of this issue: docker/buildx#2239
I added following config to buildx configuration, this option applies at docker builder create step. I am surprised at default we cannot use host network.

 <buildx>
<driverOpts>
    <network>host</network>
</driverOpts>
...

robfrank force-pushed the fix/1583_buildx branch from e987d46 to bb6be66 Compare May 8, 2023 13:59

robfrank force-pushed the fix/1583_buildx branch from bb6be66 to cdb7adf Compare May 8, 2023 16:33

robfrank mentioned this pull request May 8, 2023

revert to fabric8 maven docker-plugin (it's not deprecated) ArcadeData/arcadedb#1070

Merged

arauchberger mentioned this pull request Jun 14, 2023

Multi-platform buildx: unauthorized for fetching metadata for the base image #1583

Closed

rohanKanojia reviewed Jun 14, 2023

View reviewed changes

fabric8io#1583

0641d04

remove --config from command string generation add --node option to buildx command Signed-off-by: Roberto.Franchini <ro.franchini@gmail.com>

rohanKanojia force-pushed the fix/1583_buildx branch from cdb7adf to 8fa1ee2 Compare July 28, 2023 16:52

fix (buildx) : Only add --config to buildx when config directory not …

0e7fd07

…empty Signed-off-by: Rohan Kumar <rohaan@redhat.com>

rohanKanojia force-pushed the fix/1583_buildx branch from 8fa1ee2 to 0e7fd07 Compare July 28, 2023 17:42

rohanKanojia changed the title ~~remove --config from command string generation~~ Make --config from buildx command string generation optional Jul 28, 2023

rohanKanojia approved these changes Jul 28, 2023

View reviewed changes

rohanKanojia merged commit a359cda into fabric8io:master Jul 28, 2023

rohanKanojia mentioned this pull request Aug 13, 2023

fix (buildx) : Remove hardcoded --node value while creating buildx …builder #1700

Merged

oktayderman mentioned this pull request Dec 6, 2024

When using buildx getting "Head "https://gitlab..../manifests/latest": dial tcp ...:5050: connect: connection refused" #1841

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make `--config` from buildx command string generation optional #1673

Make `--config` from buildx command string generation optional #1673

robfrank commented May 8, 2023

robfrank commented May 8, 2023

codecov bot commented May 8, 2023 •

edited

Loading

sonarqubecloud bot commented May 8, 2023

rohanKanojia commented May 8, 2023

chonton commented May 9, 2023

arauchberger commented Jun 14, 2023

rohanKanojia commented Jun 14, 2023

rohanKanojia Jun 14, 2023

arauchberger Jun 14, 2023

arauchberger commented Jun 14, 2023

amizurov commented Jun 18, 2023

rhuss commented Jul 19, 2023

rohanKanojia commented Jul 20, 2023

sonarqubecloud bot commented Jul 28, 2023

amizurov commented Jul 29, 2023 •

edited

Loading

rohanKanojia commented Jul 29, 2023 •

edited

Loading

amizurov commented Jul 29, 2023 •

edited

Loading

rohanKanojia commented Jul 29, 2023 •

edited

Loading

rohanKanojia commented Aug 13, 2023

amizurov commented Aug 13, 2023

amizurov commented Aug 14, 2023 •

edited

Loading

arauchberger commented Aug 14, 2023

oktayderman commented Dec 6, 2024 •

edited

Loading

arauchberger commented Dec 6, 2024

oktayderman commented Dec 6, 2024

arauchberger commented Dec 6, 2024

oktayderman commented Dec 6, 2024

oktayderman commented Dec 8, 2024

Make --config from buildx command string generation optional #1673

Make --config from buildx command string generation optional #1673

Conversation

robfrank commented May 8, 2023

robfrank commented May 8, 2023

codecov bot commented May 8, 2023 • edited Loading

Codecov Report

sonarqubecloud bot commented May 8, 2023

rohanKanojia commented May 8, 2023

chonton commented May 9, 2023

arauchberger commented Jun 14, 2023

rohanKanojia commented Jun 14, 2023

rohanKanojia Jun 14, 2023

Choose a reason for hiding this comment

arauchberger Jun 14, 2023

Choose a reason for hiding this comment

arauchberger commented Jun 14, 2023

amizurov commented Jun 18, 2023

rhuss commented Jul 19, 2023

rohanKanojia commented Jul 20, 2023

sonarqubecloud bot commented Jul 28, 2023

amizurov commented Jul 29, 2023 • edited Loading

rohanKanojia commented Jul 29, 2023 • edited Loading

amizurov commented Jul 29, 2023 • edited Loading

rohanKanojia commented Jul 29, 2023 • edited Loading

rohanKanojia commented Aug 13, 2023

amizurov commented Aug 13, 2023

amizurov commented Aug 14, 2023 • edited Loading

arauchberger commented Aug 14, 2023

oktayderman commented Dec 6, 2024 • edited Loading

arauchberger commented Dec 6, 2024

oktayderman commented Dec 6, 2024

arauchberger commented Dec 6, 2024

oktayderman commented Dec 6, 2024

oktayderman commented Dec 8, 2024

Make `--config` from buildx command string generation optional #1673

Make `--config` from buildx command string generation optional #1673

codecov bot commented May 8, 2023 •

edited

Loading

amizurov commented Jul 29, 2023 •

edited

Loading

rohanKanojia commented Jul 29, 2023 •

edited

Loading

amizurov commented Jul 29, 2023 •

edited

Loading

rohanKanojia commented Jul 29, 2023 •

edited

Loading

amizurov commented Aug 14, 2023 •

edited

Loading

oktayderman commented Dec 6, 2024 •

edited

Loading