MINOR: Cherry-pick KAFKA-19851: Delete dynamic configs that were removed by Kafka to 4.2#22157
Merged
Conversation
…e#21053) When upgrading from Kafka 3.x to 4.0, the metadata log may contain dynamic configurations that were removed in 4.0 (e.g., message.format.version per KIP-724). These removed configs cause InvalidConfigurationException when users attempt to modify any configuration, because validation checks all existing configs including the removed ones. Adds filtering to prevent unsupported or invalid configurations from being applied during metadata replay. The filtering is implemented using a SupportedConfigChecker interface that is injected via dependency injection through Builder patterns. When a ConfigRecord is replayed, the checker validates whether the configuration name is supported for the given resource type. Unsupported configurations are silently ignored during replay, ensuring that only valid configurations enter the in-memory state. The SupportedConfigChecker interface provides a default TRUE implementation that accepts all configurations. The actual filtering logic is implemented by DefaultSupportedConfigChecker, which maintains a whitelist of valid configuration names per resource type (TOPIC, CLIENT_METRICS, GROUP) based on the actual config definitions. The filtering occurs in both ConfigurationDelta#replay and ConfigurationControlManager#replay methods. Added unit tests to ensure: - Removed configurations are filtered during the replay operations - Only supported configurations appear in the resulting metadata images - The filtering works correctly for all resource types (TOPIC, BROKER, CLIENT_METRICS, GROUP) - DefaultSupportedConfigChecker correctly identifies supported vs unsupported configurations for each resource type Reviewers: José Armando García Sancio <jsancio@apache.org>, Jun Rao <junrao@apache.org>, Alyssa Huang <ahuang@confluent.io>, Kevin Wu <kevin.wu2412@gmail.com>, Andrew Grant <agrant@confluent.io> (cherry picked from commit a35d649)
tzy-0x7cf
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KAFKA-19851; Delete dynamic configs that were removed by Kafka (#21053)
When upgrading from Kafka 3.x to 4.0, the metadata log may contain
dynamic configurations that were removed in 4.0 (e.g.,
message.format.version per KIP-724). These removed configs cause
InvalidConfigurationException when users attempt to modify any
configuration, because validation checks all existing configs including
the removed ones.
Adds filtering to prevent unsupported or invalid configurations from
being applied during metadata replay. The filtering is implemented using
a SupportedConfigChecker interface that is injected via dependency
injection through Builder patterns. When a ConfigRecord is replayed, the
checker validates whether the configuration name is supported for the
given resource type. Unsupported configurations are silently ignored
during replay, ensuring that only valid configurations enter the
in-memory state.
The SupportedConfigChecker interface provides a default TRUE
implementation that accepts all configurations. The actual filtering
logic is implemented by DefaultSupportedConfigChecker, which maintains a
whitelist of valid configuration names per resource type (TOPIC,
CLIENT_METRICS, GROUP) based on the actual config definitions. The
filtering occurs in both ConfigurationDelta#replay and
ConfigurationControlManager#replay methods.
Added unit tests to ensure:
CLIENT_METRICS, GROUP)
unsupported configurations for each resource type
Reviewers: José Armando García Sancio jsancio@apache.org, Jun Rao
junrao@apache.org, Alyssa Huang ahuang@confluent.io, Kevin Wu
kevin.wu2412@gmail.com, Andrew Grant agrant@confluent.io
(cherry picked from commit a35d649)