Skip to content
This repository was archived by the owner on Jan 7, 2025. It is now read-only.

Commit c6ab0d7

Browse files
authored
docs: update miscellaneous.md
1 parent db485ce commit c6ab0d7

File tree

1 file changed

+12
-2
lines changed

1 file changed

+12
-2
lines changed

docs/src/miscellaneous.md

+12-2
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ optd supports exploring SQL expressions in the optimization process. However, th
1414

1515
We do not have something like a binding iterator as in the Cascades paper. Before applying a rule, we will generate all bindings of a group, which might take a lot of memory. This should be fixed in the future.
1616

17-
## Cycle Detection
17+
## Cycle Detection + DAG
1818

1919
Consider the case for join commute rule.
2020

@@ -26,10 +26,20 @@ Consider the case for join commute rule.
2626

2727
After applying the rule twice, the memo table will have self-referential groups. Currently, we detect such self-referential things in optimize group task. Probably there will be better ways to do that.
2828

29+
The same applies to DAG / Recursive CTEs -- we did not test if the framework works with DAG but in theory it should support it. We just need to ensure a node in DAG does not get searched twice.
30+
31+
# DAG
32+
33+
For DAG, another challenge is to recover the reusable fragments from the optimizer output. The optimizer can give you a DAG output but by iterating through the plan, you cannot know which parts can be reused/materialized. Therefore, we might need to produce some extra information with the plan node output. i.e., a graph-representation with metadata of each node, instead of `RelNode`. This also helps the process of inserting the physical collector plan nodes, which is currently a little bit hacky in the implementation.
34+
35+
## Memo Table
36+
37+
Obviously, it is not efficient to simply store a mapping from RelNode to the expression id. Cannot imaging how many levels of depths will it require to compute a hash of a tree structure.
38+
2939
## Partial Exploration
3040

3141
Each iteration will only be slower because we have to invoke the optimize group tasks before we can find a group to apply the rule. Probably we can keep the task stack across runs to make it faster.
3242

33-
## Physical Property
43+
## Physical Property + Enforcer Rules
3444

3545
A major missing feature in the optimizer. Need this to support shuffling and sort optimizations.

0 commit comments

Comments
 (0)