chore(datafusion): upgrade to DataFusion 54.0.0#2777
Closed
kszucs wants to merge 2 commits into
Closed
Conversation
Bumps datafusion, datafusion-cli, datafusion-ffi, and datafusion-sqllogictest from 53.x to 54.0.0, and home to =0.5.12 to satisfy rustyline's transitive requirement pulled in by datafusion-cli. DataFusion 54 removes the as_any() method requirement from ExecutionPlan, TableProvider, CatalogProvider, CatalogProviderList, SchemaProvider, and PhysicalExpr in favor of native Rust trait upcasting, so the now-redundant as_any impls and call sites are removed. Expr::Cast also replaced its data_type field with a field: FieldRef, so downstream date-type checks are updated to use field.data_type().
Fixes lint failure from the DataFusion 54 upgrade commit.
kszucs
changed the title
Collaborator
|
Thanks @kszucs! I'll compare to #2648 which already has a couple of reviews. I think @xanderbailey just needs to upmerge main and fix the false failure. |
Member
Author
|
Ohh sorry, didn't see the other PR. Feel free to close this one if not relevant anymore. |
Collaborator
No worries, thanks for the enthusiasm! If for some reason @xanderbailey can't get around to it, we can review this one. They seem mostly identical from a code standpoint, just Cargo.lock differences. |
Collaborator
|
Closing as duplicate |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
What changes are included in this PR?
datafusion,datafusion-cli,datafusion-ffi, anddatafusion-sqllogictestfrom 53.x to 54.0.0.hometo=0.5.12to satisfyrustyline's transitive requirement pulled in bydatafusion-cli54.as_any()impls and call sites: DataFusion 54 drops theas_any()method requirement fromExecutionPlan,TableProvider,CatalogProvider,CatalogProviderList,SchemaProvider, andPhysicalExprin favor of native Rust trait upcasting.Expr::Castdate-type checks inexpr_to_predicate.rsto usecast.field.data_type()instead of the removedcast.data_typefield.crates/integrations/datafusion/public-api.txtto drop the removedas_any()entries.Are these changes tested?
Covered by the existing
iceberg-datafusionandiceberg-playgroundunit/integration test suites, which all pass against DataFusion 54. Also verified withcargo build --workspace --all-targets,cargo clippy --workspace --all-targets --all-features, andcargo test --workspace.