[define] usage of CloudEvents' dataschema conflicts with the Confluent wire format

[BirgitBader]

(i) This issue has been manually transferred from a former internal repository, as a private repository issue cannot be transferred to a public repository.

Context

CloudEvents' dataschema should contain a reference to the schema that has been used to write the cloud event. If the Confluent schema registry and it's SerDes is used, this information is encoded using the Confluent wire format directly in the key/value of the record.

Currently, it is not possible to write the schema ID into a custom record header when using the Confluent Schema Registry SerDes. An open issue exists: confluentinc/schema-registry#881

Other SerDe Libraries support writing to a custom header: https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/getting-started/assembly-configuring-kafka-client-serdes.html#registry-serdes-config-props_registry

Therefore we need to define how this should be handled.

[BirgitBader]

Comment history

May 6, 2022 by @thake

@soenkeHH will ask his contact at confluent about the status/roadmap for confluentinc/schema-registry#881