Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hotfix] Synchronize CI pipeline setup #78

Merged
merged 2 commits into from
Jan 18, 2024

Conversation

MartijnVisser
Copy link
Contributor

The Flink Kafka connector now tests:

  • All PRs against main are tested against Flink 1.17.2 and 1.18.1, instead of 1.17.1 and 1.18.0
  • The weekly run tests -- main against 1.17-SNAPSHOT, 1.18-SNAPSHOTand1.19-SNAPSHOT(1.19 is newly added) --v3.0against1.17-SNAPSHOTand1.18-SNAPSHOT` (as was before)

Comment on lines 38 to 43
strategy:
matrix:
flink: [ 1.17.1, 1.18.0 ]
uses: apache/flink-connector-shared-utils/.github/workflows/python_ci.yml@ci_utils
with:
flink_version: ${{ matrix.flink }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we remove ci for python part?

IIRC currently it is done separately

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow I completely missed the Python part. Let me check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep them in for PRs, and actually we should also have them for weekly runs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@snuyanzin
Copy link
Contributor

snuyanzin commented Jan 16, 2024

since it is touched a couple of more ideas

  1. by default python tests are running with jdk8, after [FLINK-33796] Add ability to customize java version for python ci in connectors flink-connector-shared-utils#30
    it could be also customized for python tests, so may be would make sense to run python ci with different jdk as well
  2. it seems python ci is not included in weekly, I wonder whether we should add it there

The Flink Kafka connector now tests:
- All PRs against `main` are tested against Flink `1.17.2` and `1.18.1`, instead of `1.17.1` and `1.18.0`
- The weekly run tests
-- `main` against `1.17-SNAPSHOT, `1.18-SNAPSHOT` and `1.19-SNAPSHOT` (1.19 is newly added)
-- `v3.0` against `1.17-SNAPSHOT` and `1.18-SNAPSHOT` (as was before)
@MartijnVisser
Copy link
Contributor Author

  1. it could be also customized for python tests

I think we should mirror what we do with Flink, so I think we should indeed also run them for JDK11 and JDK17? We could argue that JDK11/JDK17 would only be run during a weekly, to avoid overly hitting the servers. I believe we also run those tests during the nightly builds for Flink

  1. it seems python ci is not included in weekly, I wonder whether we should add it there

We should indeed

@snuyanzin
Copy link
Contributor

yeah... initially there were already 2 jdks (8, 11) in ci for PRs and that's why I've just added other jdks there...
Now amount of ci jobs per is growing together with number of jdks + possibly twice because of python...

it would make sense to follow something similar we have in Flink main repo where everything is built with jdk8 for PRs and all other jdks are participating only during nightlies

@MartijnVisser MartijnVisser merged commit d3bda90 into apache:main Jan 18, 2024
8 checks passed
@MartijnVisser MartijnVisser deleted the sync_ci_1.19 branch January 18, 2024 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants