[DVC] Add RequestBasedMetaRepository to enable metadata retrieval directly from server

[pthirun]

[DVC] Add RequestBasedMetaRepository to enable metadata retrieval directly from server

This PR introduces the use of the /store_properties/{store_name} endpoint to retrieve and cache store metadata for the DaVinci client. The endpoint was implemented in the Venice Server in this PR: #1374.

The RequestBasedMetaRepository leverages this endpoint to maintain a cache of Store and SchemaData objects for requested stores. The cache is refreshed by the NativeMetadataRepository, which this new class extends.

By default, this feature is disabled in the ClientConfig passed to the NativeMetadataRepository. The default configuration preserves the current behavior of using the ThinClientMetaStoreBasedRepository. The relevant configuration field is ClientConfig.useRequestBasedMetaRepository. For more details, see NativeMetadataRepository.getInstance.

How was this PR tested?

Unit and Integration test included in PR.

Does this PR introduce any user-facing changes?

• No. You can skip the rest of this section.
• Yes. Make sure to explain your proposed changes and call out the behavior change.

NOTE: behavior will not change by default, feature flag in ClientConfig must be turned on.

[xunyin8]

Looks good overall, thanks for the refactoring. Just some minor comments and also looks like test coverage is not passing.

[xunyin8]

Thanks for the changes and unit tests, I think we are a bit lacking on the integration tests. See comments below.

[xunyin8]

Maybe put a comment on NativeMetadataRepository that it does not handle StoreDataChangedListener#handleStoreDeleted events? Since the current implementation will throw exceptions during metadata fetch and not really differentiating store deleted vs unable to retrieve metadata.

[pthirun]

Wrote a comment above the method stating it will throw the VeniceNoStoreException on either deleting or missing stores. Is this an accurate representation?

[xunyin8]

The comment doesn't match the assertion and also this is a weird place to put a test for RequestBasedMetaRepository in a file called MetaSystemStoreTest. Can we add a new test file where we run a more comprehensive integration test with a DVC? i.e push some new versions or make some store config changes and make sure the DVC pick up those changes. You can see examples like the recently added testBatchOnlyMaterializedViewDVCConsumer. You probably don't need a VeniceTwoLayerMultiRegionMultiClusterWrapper, a single region will be sufficient.