Skip to content

perf(streaming): optimize array map and jsonb array mapping patterns #23624

@kwannoel

Description

@kwannoel

Consider the case where we want to transform an array's items in some way. We will apply an unnest operation, run some operation on each unnested item, finally use array_agg or jsonb_build_object to reconstruct the object.

The unnesting and application step can lead to logical apply. This makes the runtime rather slow for the query.

Ideally we can add an optimizer rule to transform an unnest, apply function, and re-aggregate to project: map(array, function).

However, this also means we need to support first class functions in our function framework. Where map is a higher order function, which applies the specified function to all items in an array.

Will need significant refactoring. Welcome other ideas @chenzl25 @xiangjinwu @BugenZhao

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions