ConfluentSchemaRegistryClient picks wrong subject version when using SpecificRecord types

[palmski]

Describe the issue
ConfluentSchemaRegistryClient picks wrong subject version when serialising a message if the schema registry has a newer version of the schema registered. This results in the wrong version being encoded in the ContentType header as calculated here https://github.com/spring-cloud/spring-cloud-stream/blob/main/schema-registry/spring-cloud-stream-schema-registry-client/src/main/java/org/springframework/cloud/stream/schema/registry/client/ConfluentSchemaRegistryClient.java#L103-L111

To Reproduce
Steps to reproduce the behavior:

1. Send an event which registers an avro schema with the schema registry
2. Observe the ContentType header (let's say it is application/vnd.user.v1+avro)
3. Evolve the schema, produce a new version of the SpecificRecord type
4. Send a new version of the event which registers a new version of the schema
5. Observe the ContentType header (it will be application/vnd.user.v2+avro)
6. Send an event using the older SpecificRecord type
7. Observe the ContentType header (it will be application/vnd.user.v2+avro)
8. Observe the consumer fail to deserialise because the reader schema (v2) does not match the writer schema (v1)

Version of the framework
4.2.0
Expected behavior
When registering the schema on sending the event the schema registry client should not assume that the most recent subject version is being used, but should instead take the ID returned from the POST to "/subjects/" + subject + "/versions" and invoke https://docs.confluent.io/platform/current/schema-registry/develop/api.html#get--schemas-ids-int-%20id-versions to fetch the correct version

[palmski]

For reference, using this code (Kotlin, sorry) to replace the linked code above does the job

            template.getForEntity<List<Map<String, Any>>>("$endpoint/schemas/ids/$id/versions").body?.let {
                if (!CollectionUtils.isEmpty(it) && it.isNotEmpty()) {
                    version = it[0]["version"] as Int
                }
            }

[sobychacko]

@palmski Thanks for the report. That seems like an issue. Are you willing to send a PR to fix this issue? If so, please do it; we will gladly review it. Thanks!

[palmski]

@sobychacko #3107 raised

Sorry it took so long, getting the build running within our corporate network has been... challenging :)