You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This makes the connector unusable, because the name changes with every execution (apparently with every batch as specified by batch.max.rows, making it incompatible with previously registered versions of the schema. This then crashes the connector after the first batch was fetched, because it tries to set the same schema with a new name, which is incompatible with schema registry (unless using compatibility NONE, which I cannot in good conscience use in production).
This seems broken to me @elakito
Can you please find a way to re-use the same schema name with every query? Maybe use a hash of the query or something?
The text was updated successfully, but these errors were encountered:
More insights: this only happens when the queryMode is set to query.
When I set it to table then it uses the table.name as the name for the avro schema, which is stable enough.
pantaoran
changed the title
Set avro schema name (source connector)?
Avro schema name in source connector is randomly recomputed with every query execution
Apr 25, 2023
When using this as a source connector with the
queryMode
set toquery
, then the name of the avro schema which gets registered in Schema Registry contains a random integer.This happens here in the code: https://github.com/SAP/kafka-connect-sap/blob/master/src/main/scala/com/sap/kafka/connect/source/querier/TableQuerier.scala#L95
This makes the connector unusable, because the name changes with every execution (apparently with every batch as specified by
batch.max.rows
, making it incompatible with previously registered versions of the schema. This then crashes the connector after the first batch was fetched, because it tries to set the same schema with a new name, which is incompatible with schema registry (unless using compatibilityNONE
, which I cannot in good conscience use in production).This seems broken to me @elakito
Can you please find a way to re-use the same schema name with every query? Maybe use a hash of the query or something?
The text was updated successfully, but these errors were encountered: