feat: heuristic rule for fuse parquet dictionary page

[dantengsky]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Enabled Parquet dictionary encoding by default, plus an NDV-based heuristic that disables per-column dictionaries when a block’s NDV/row ratio exceeds 10%.

During a streaming fuse table write, the first block’s statistics determine whether dictionaries should be disabled for the remaining blocks in that stream.

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

---

This change is 

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

[dantengsky]

@codex review

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

[BohuTANG]

@codex review

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[BohuTANG]

Review comment:

• [P2] Disable dictionary using wrong column path for nested fields — databend/src/query/storages/common/blocks/src/parquet_rs.rs:130-139
  In build_parquet_writer_properties the dictionary is selectively disabled by calling set_column_dictionary_enabled(ColumnPath::from(field.name().as_str()), false).
  TableSchema::leaf_fields() intentionally returns field names using the "parent:child" colon notation, but Parquet’s ColumnPath expects a vector of the actual nested components
  (e.g. ["parent", "child"]). When Arrow writes nested columns it uses those real components, so the per-column configuration registered under "nested:leaf" is never looked up and
  the dictionary remains enabled. As a result, high-cardinality nested fields (struct members, array elements, etc.) always keep dictionary encoding despite the statistics indicating
  they should be disabled. To make the heuristic effective you need to populate the ColumnPath with the individual path components rather than the colon-joined string.

[BohuTANG]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[dantengsky]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. What shall we delve into next?

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[github-actions]

Docker Image for PR

• tag: pr-19024-fba3f0f-1764930416

note: this image tag is only available for internal use.

[github-actions]

ClickBench Report

• hits: https://benchmark.databend.com/clickbench/pr/19024/19958616223/hits.html
• tpch100: https://benchmark.databend.com/clickbench/pr/19024/19958616223/tpch100.html
• tpch1000: https://benchmark.databend.com/clickbench/pr/19024/19958616223/tpch1000.html

[github-actions]

Docker Image for PR

• tag: pr-19024-2a4ba64-1765024271

note: this image tag is only available for internal use.

[github-actions]

ClickBench Report

• hits: https://benchmark.databend.com/clickbench/pr/19024/19988014185/hits.html
• tpch100: https://benchmark.databend.com/clickbench/pr/19024/19988014185/tpch100.html
• tpch1000: https://benchmark.databend.com/clickbench/pr/19024/19988014185/tpch1000.html

[dantengsky]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".