Skip to content

Pipeline Slicer for PQPs#6

Closed
tobodner wants to merge 87 commits intopr-750-headfrom
pr-751-head
Closed

Pipeline Slicer for PQPs#6
tobodner wants to merge 87 commits intopr-750-headfrom
pr-751-head

Conversation

@tobodner
Copy link
Copy Markdown
Member

@tobodner tobodner commented Nov 3, 2025

Depends on

  • #740

ToDos

  • Define ExchangeStrategy interface to serve data shuffling etc.
  • Refactor
    • adopt recent changes of PqpPipeline and PipelineFragmentTemplate
    • adopt ExchangeStrategy
  • Create tests
    • TPC-H Q1 with staged aggregation
    • Test data shuffle strategy

julianmenzler and others added 30 commits May 20, 2022 16:54
More files will be added to the `physical_query_plan` folder. To stay organized, it makes sense to put all proxies into a dedicated subfolder.
This PR adds pipeline data structures so that PQPs can be modeled as a series of interdependent physical pipeline plans.

Each pipeline consists of

* an identity string (unique identifier)
* pipeline fragments, described by 
  * `PipelineFragmentTemplate`, a templated physical plan serving as a blueprint for all physical plans of the pipeline (data parallelism)
  * `PipelineFragmentDefinition`s encapsulating import/export configuration data, so that physical plans can be generated from the templated physical plan
* references to successor pipelines
* references to predecessor pipelines (weak)
…roughput_parallel_benchmark (#752)

* add repetition metrics to LambdaBenchmarkOutput-builder

* use new metrics in network_throughput_parallel_benchmark + add bool repetition metric to LambdaBenchmarkOutput-Builder (#713)

* Fix clang format issues (#713)

* trigger pipeline

* Add throughput aggregates (percentiles) for invocations of a repetition to LambdaOutput of ThroughputParallelBenchmark (#713)

* Implement review findings (#713)

* fix readme

* ...

* Adjust includes in benchmark-result  (#713)

Co-authored-by: d-justen <33455240+d-justen@users.noreply.github.com>
This commit increases the compiler compatibility of the project. No functional features are added. The changes involve adding missing includes and refactoring a function to obtain the full path of the executable. The project (i.e. the unit tests) now compiles on MacOS.
@tobodner tobodner closed this Nov 3, 2025
@tobodner tobodner deleted the pr-751-head branch November 3, 2025 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants