Skip to content

Fix Kafka Connect consumer override config - #15

Open
jruaux wants to merge 3 commits into
mainfrom
jruaux/create-pr
Open

Fix Kafka Connect consumer override config#15
jruaux wants to merge 3 commits into
mainfrom
jruaux/create-pr

Conversation

@jruaux

@jruaux jruaux commented May 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • enable Kafka Connect connector client overrides in the standalone worker config
  • use consumer.override.auto.offset.reset so the file sink's latest offset reset setting is applied
  • update the sample docs to match

Verification

  • docker compose config
  • local smoke test confirmed Kafka Connect logs show auto.offset.reset = latest and records are written to output/dw-scte.jsonl

Note

Low Risk
Documentation and sample-only Docker/Connect configuration; no production service or auth changes.

Overview
Adds a Kafka Connect sample under samples/kafka-connect that runs Korvet with Redis, a standalone Connect worker, a built-in file sink, and a looping JSON producer on topic dw-scte, with Makefile helpers and docs linked from the root and samples READMEs. Generated sink output is gitignored via samples/kafka-connect/output/.

Connect behavior is tuned so connector consumer settings actually apply: the worker sets connector.client.config.override.policy=All, and the file sink uses consumer.override.auto.offset.reset=latest (documented in the sample README) so the sink starts at latest offsets instead of ignoring a non-prefixed override.

Reviewed by Cursor Bugbot for commit 00487a3. Bugbot is set up for automated code reviews on this repo. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant