Feature: Support hierarchical data with optimised methods

[rad-pat]

It would be great to have some optimised functions to process hierarchical data rather than having to build recursive CTEs to get information about trees of data - siblings/children etc.

This link provides information about one particular implementation, but it could perhaps be done differently https://help.sap.com/docs/SAP_HANA_PLATFORM/4f9859d273254e04af6ab3e9ea3af286/9aa2f6d912f24dc5baca772bd40745fc.html?locale=en-US&version=2.0.05

[inviscid]

There does appear to be some discussion about handing nested/hierarchical data in Arrow and Parquet here: https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/

This would be a very nice feature to have since most "business" data includes some form of organizational layout like accounts, cost centers, departments, products, etc... Often times this data is jagged which makes using a predetermined number of table columns problematic.

The ability to quickly find ancestors, descendants, siblings, leaves, etc... without having to run recursive CTEs would be a very welcome capability.

[rad-pat]

There does appear to be some discussion about handing nested/hierarchical data in Arrow and Parquet here: https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/

I think this is referring to hierarchical data like JSON structure which I believe is a VARIANT column in Databend, not actually building a tree structure from parent-child or level relationships within a table of data.

To have the ability to create a TREE structure from a table and to then use the speed of RUST to get answers to the ancestors, descendants, siblings, leaves questions would be great. Perhaps the usage could be a little similar to a CTE.

e.g. sudo code

with TREE(table, parent_col, child_col) as t
select * 
from data_table d
where d.tree_node_col in siblings(t, node1, node2);

[b41sh]

It looks a bit similar to the json_path function, we can consider supporting these functions, it should be useful.

[sundy-li]

SAP supports CREATE COLUMN TABLE

[rad-pat]

It looks a bit similar to the json_path function, we can consider supporting these functions, it should be useful.

I wonder if I can leverage json_path functions for what we need in the interim, are those functions fully implemented @b41sh ?

[rad-pat]

I wonder if I can leverage json_path functions for what we need in the interim, are those functions fully implemented @b41sh ?

For now it seems that I cannot get the answers I desire using json_path because a recursive descent operator is not yet available. I raised an issue on that in the JSONB repo, I hope that is the correct place.