Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update avro to 1.11.3 (backport to 2.18 branch) #512

Merged
merged 3 commits into from
Aug 24, 2024

Conversation

rafalh
Copy link
Contributor

@rafalh rafalh commented Aug 21, 2024

Update Apache avro library to 1.11.3.
To make it compatible namespace generation for nested classes had to be changed.
Now dots are used instead of dollar signs as indicator of nesting.
AFAIK dollar sign is not an allowed character in Avro namespace according to the specification and this is why this change was implemented in avro library.
Note that avro library generates namespaces not ending with dollar character since version 1.9, but it actually removes all dollar signs in classes with multiple levels of nesting since 1.11.

See: AVRO-2143 and AVRO-2757
Fixes #167

Compatibility:

  • over-the-wire protocol should be compatible
  • reading schemas from older versions should work
  • schemas generated after this change for classes with no nesting are unchanged
  • schemas generated after this change for classes with one nesting level are readable by Jackson 2.11+ (commit c570549)
  • schemas generated after this change for classes with multiple nesting levels will not be readable by older Jackson versions

rafalh added 2 commits August 21, 2024 12:08
Namespace for nested classes no longer ends with '$'. This is how avro library generates schema since version 1.9. See: AVRO-2143
Please note that resolution of nested classes without '$' was implemented long ago in c570549.

Fixes FasterXML#167
…levels

Avro before 1.11 was generating schemas with '$' in namespace if class had multiple nesting levels. To fix compatibility with avro 1.11+ make sure all dollar characters are replaced by dots.
Related: AVRO-2757
@rafalh rafalh mentioned this pull request Aug 21, 2024
@cowtowncoder cowtowncoder added avro cla-needed PR looks good (although may also require code review), but CLA needed from submitter 2.18 labels Aug 21, 2024
@cowtowncoder
Copy link
Member

Quick note: as per my comments, happy to review, help get merged.
But one thing we'll need before merging, CLA, from:

https://github.com/FasterXML/jackson/blob/master/contributor-agreement.pdf

It is only needed once before the first contribution, and the usual way is to print it, fill & sign, scan/photo, email to cla at fasterxml dot com.
Once I get that I can proceed with final review and merging.
Thank you in advance, looking forward to FINALLY resolving the upgrade problem!

@rafalh
Copy link
Contributor Author

rafalh commented Aug 23, 2024

I've sent CLA photo. I just noticed that I used info e-mail instead of cla, because that's what was used in CONTRIBUTING.md. Thanks for letting me know.

@cowtowncoder
Copy link
Member

@rafalh info is fine too; will need to change CONTRIBUTING.md

@cowtowncoder cowtowncoder removed the cla-needed PR looks good (although may also require code review), but CLA needed from submitter label Aug 24, 2024
@cowtowncoder
Copy link
Member

CLA received, will do one more review.

@cowtowncoder cowtowncoder merged commit 7e3c869 into FasterXML:2.18 Aug 24, 2024
4 checks passed
@cowtowncoder cowtowncoder added this to the 2.18.0 milestone Aug 24, 2024
@cowtowncoder
Copy link
Member

Thank you again, @rafalh -- this is awesome! Hoping to wrap up 2.18.0-rc1 planning Very Soon Now, to get the release candidate out.

@rafalh rafalh deleted the avro-1.11-backport-2.18 branch August 27, 2024 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incompatibility with Avro >=1.9.0 (upgrade to Avro 1.11.3)
2 participants