Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to register schemas with canonical forms that reduce to a primitive type #157

Open
BryceCicada opened this issue Oct 25, 2021 · 3 comments

Comments

@BryceCicada
Copy link

I'd like to use a schema-registry avro schema to encode a kafka message key. Most of the time, the key is pretty simple like a string or a long. Other times, it's more complex and I can use a record type.

When the key isn't a record, but it's just a primitive type, I'm struggling to register the schema. For example:

      const schemaRegistry = new SchemaRegistry({host: "http://my-registry:8081"})
      await schemaRegistry.register({type: SchemaType.AVRO, schema: '"string"'})

gives:

     ConfluentSchemaRegistryArgumentError: Invalid name: undefined
      at AvroHelper.validate (node_modules/@kafkajs/confluent-schema-registry/src/AvroHelper.ts:26:13)
      at SchemaRegistry.register (node_modules/@kafkajs/confluent-schema-registry/src/SchemaRegistry.ts:106:12)
      at Context.<anonymous> (test/service-bus/serde_test.ts:45:28)
      at processImmediate (internal/timers.js:461:21)

I've also tried:

      await schemaRegistry.register({type: SchemaType.AVRO, schema: JSON.stringify({type: "string"})})
      await schemaRegistry.register({type: SchemaType.AVRO, schema: JSON.stringify({name: "foo", type: "string"})})

which have schemas that reduce to the same avro canonical form as the first example, but these give the same error.

I can register the schemas with the registry directly, eg:

$ curl -X POST localhost:8081/subjects/my-topic-key/versions -H "Content-Type: application/json" -d '{"schema": "\"string\""}'
{"id":1}
$ curl -X POST localhost:8081/subjects/my-topic-key/versions -H "Content-Type: application/json" -d '{"schema": "{\"type\": \"string\"}"}'  # NB Same canonical form -> same id from registry
{"id":1} 
$ curl -X POST localhost:8081/subjects/my-topic-key/versions -H "Content-Type: application/json" -d '{"schema": "{\"name\": \"foo\", \"type\": \"string\"}"}' # NB Same canonical form again
{"id":1} 
$ curl localhost:8081/schemas/ids/1
{"schema":"\"string\""}

which makes me think that AvroHelper.validate() might be a bit too strict in requiring a name field on the schema.

@BryceCicada
Copy link
Author

Commenting on my own issue here as I understand more...

I suspect the reason for requiring a name on the schema is because of the implicit subject naming that uses the name field from the schema. Would it make sense to relax this constraint in AvroHelper.validate() if the subject name were passed in to register()?

@filipstiglic
Copy link

Hello, any update on this?

@rogen-code
Copy link

Hit this today and would love to know if anyone has a workaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants