[FLINK-34193][Connectors/Kafka] Remove usage of Flink-Shaded Jackson and Snakeyaml in flink-connector-kafka

[jinsuichen]

What is the purpose of the change

This pull request removes the dependency on shaded.

Brief change log

• Due to the @PublicEvolving annotation marking JSONKeyValueDeserializationSchema.java, the specific version of Jackson it uses is retained and cannot be removed. Therefore, in KafkaRecordDeserializationSchemaTest.java, the ObjectNode is changed to use the same dependency instead of relying on shaded.
• In KafkaRecordDeserializationSchemaTest.java, JsonDeserializationSchema indirectly depends on shaded, so it is replaced with an inner class SimpleDeserializationSchema.
• The YAML serialization and deserialization logic in YamlFileMetaDataService.java is replaced with JSON.

Verifying this change

This change is already covered by existing tests, such as JsonFileMetadataServiceTest.java.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[boring-cyborg]

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

[jinsuichen]

Hi, @MartijnVisser. All CI jobs have been passed in my own repo forked from here. PR link

Could you help me retry the failed one?

[MartijnVisser]

Superseded by #84