Replies: 2 comments
-
Thanks for bringing up this discussion! Not sure which is better. @FelixYBW, this issue was mentioned in the project meeting. Did you have any comment? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Current solution in Gluten
Currently, functions under
velox/functions/prestosql
andvelox/functions/sparksql
are registered without prefix in Glutenhttps://github.com/oap-project/gluten/blob/25223aab8ece0c420fac35c0e370c6305884080c/cpp/velox/operators/functions/RegistrationAllFunctions.cc#L66-L79
This solution can be risky in some scenarios:
Proposed change
We can maintain a presto function mapping in Gluten cpp during substrait->velox expresion conversion, and register all presto functions with "presto" prefix. To create the mapping, we need to find all supported functions that being offloaded to prestosql. For example, function
array_distinct
is using prestosql implementation, then addarray_distinct -> presto_array_distinct
to the mapping.With the mapping, we can be sure that which functions goes to sparksql implementation, and which goes to prestosql.
Beta Was this translation helpful? Give feedback.
All reactions