Skip to content
This repository was archived by the owner on Jan 7, 2025. It is now read-only.

Commit ba22fa1

Browse files
committed
docs: add misc
Signed-off-by: Alex Chi Z <[email protected]>
1 parent 63dd5e6 commit ba22fa1

File tree

2 files changed

+35
-0
lines changed

2 files changed

+35
-0
lines changed

docs/src/SUMMARY.md

+4
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,7 @@
2626

2727
- [SQLPlannerTest](./sqlplannertest.md)
2828
- [Datafusion CLI](./datafusion_cli.md)
29+
30+
# Miscellaneous
31+
32+
- [Miscellaneous](./miscellaneous.md)

docs/src/miscellaneous.md

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Miscellaneous
2+
3+
This is a note covering things that do not work well in the system right now.
4+
5+
## Type System
6+
7+
Currently, we hard code decimal type to have `15, 2` precision. Type inferences should be done in the schema property inference.
8+
9+
## Expression
10+
11+
optd supports exploring SQL expressions in the optimization process. However, this might be super inefficient as optimizing a plan node (i.e., join to hash join) usually needs the full binding of an expression tree. This could have exponential plan space and is super inefficient.
12+
13+
## Bindings
14+
15+
We do not have something like a binding iterator as in the Cascades paper. Before applying a rule, we will generate all bindings of a group, which might take a lot of memory. This should be fixed in the future.
16+
17+
## Cycle Detection
18+
19+
Consider the case for join commute rule.
20+
21+
```
22+
(Join A B) <- group 1
23+
(Projection (Join B A) <expressions list>) <- group 2
24+
(Projection (Projection (Join A B) <expressions list>) <expressions list>) <- group 1 may refer itself
25+
```
26+
27+
After applying the rule twice, the memo table will have self-referential groups. Currently, we detect such self-referential things in optimize group task. Probably there will be better ways to do that.
28+
29+
## Partial Exploration
30+
31+
Each iteration will only be slower because we have to invoke the optimize group tasks before we can find a group to apply the rule. Probably we can keep the task stack across runs to make it faster.

0 commit comments

Comments
 (0)