You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/substrait-spark/README.md
-19Lines changed: 0 additions & 19 deletions
Original file line number
Diff line number
Diff line change
@@ -388,25 +388,6 @@ To recap on the steps above
388
388
389
389
The structure of the query plans for both Spark and Substrait are structurally very similar.
390
390
391
-
### Aggregate and Sort
392
-
393
-
Spark's plan has a Project that filters down to the colour, followed by the Aggregation and Sort.
394
-
```
395
-
+- Sort [count(1)#18L ASC NULLS FIRST], true
396
-
+- Aggregate [colour#5], [colour#5, count(1) AS count(1)#18L]
397
-
+- Project [colour#5]
398
-
```
399
-
400
-
When converted to Substrait the Sort and Aggregate is in the same order, but there are additional projects; it's not reduced the number of fields as early.
These look different due to two factors. Firstly the Spark optimizer has swapped the project and aggregate functions.
409
-
Secondly projects within the Substrait plan joined the fields together but don't reduce the number of fields. Any such filtering is done on the outer relations.
0 commit comments