Skip to content

Commit fe8096b

Browse files
committed
feat: clean up example to address later
Signed-off-by: MBWhite <[email protected]>
1 parent dc89c80 commit fe8096b

File tree

1 file changed

+0
-19
lines changed

1 file changed

+0
-19
lines changed

examples/substrait-spark/README.md

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -388,25 +388,6 @@ To recap on the steps above
388388

389389
The structure of the query plans for both Spark and Substrait are structurally very similar.
390390

391-
### Aggregate and Sort
392-
393-
Spark's plan has a Project that filters down to the colour, followed by the Aggregation and Sort.
394-
```
395-
+- Sort [count(1)#18L ASC NULLS FIRST], true
396-
+- Aggregate [colour#5], [colour#5, count(1) AS count(1)#18L]
397-
+- Project [colour#5]
398-
```
399-
400-
When converted to Substrait the Sort and Aggregate is in the same order, but there are additional projects; it's not reduced the number of fields as early.
401-
402-
```
403-
+- Sort:: FieldRef#/I64/StructField{offset=1} ASC_NULLS_FIRST
404-
+- Project:: [Str, I64, Str, I64]
405-
+- Aggregate:: FieldRef#/Str/StructField{offset=0}
406-
```
407-
408-
These look different due to two factors. Firstly the Spark optimizer has swapped the project and aggregate functions.
409-
Secondly projects within the Substrait plan joined the fields together but don't reduce the number of fields. Any such filtering is done on the outer relations.
410391

411392
### Inner Join
412393

0 commit comments

Comments
 (0)