Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Transitive Paths using binary search instead of a hash map #1313

Merged
merged 98 commits into from
Apr 17, 2024
Merged
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
fd1bdf8
Added graphblas dependecies
JoBuRo Jan 30, 2024
285528b
Added wrapper for graphblas matrix
JoBuRo Jan 30, 2024
935bc0d
Replaced transitiveHull computation
JoBuRo Jan 30, 2024
069819f
Added extern keyword around include
JoBuRo Jan 31, 2024
4b257fe
Replaced std map with abseil map
JoBuRo Jan 31, 2024
d4ba6f6
Added graphblas dependency for GH action
JoBuRo Jan 31, 2024
1b20a1e
Added library for mac build
JoBuRo Jan 31, 2024
92621d0
Removed finalize()
JoBuRo Jan 31, 2024
b3d71bd
Added fallback for GraphBLAS
JoBuRo Jan 31, 2024
d1de484
Reworks GrBMatrix
JoBuRo Feb 1, 2024
303ff2f
More reworks on GrbMatrix
JoBuRo Feb 1, 2024
958b4f7
Renamed getMatrix -> matrix
JoBuRo Feb 1, 2024
87bc650
Added documentation to GrbMatrix
JoBuRo Feb 2, 2024
370ad43
Reworked build function
JoBuRo Feb 2, 2024
0018c84
Reworked extractRow and extractCol
JoBuRo Feb 2, 2024
622dddd
Reworked use of C arrays in GrbMatrix
JoBuRo Feb 2, 2024
7c7d5b6
Additional reworks for GrbMatrix
JoBuRo Feb 2, 2024
21a5994
Reworked extractTuples
JoBuRo Feb 5, 2024
6f0f8cd
Added a quick fix for GrB_init issue
JoBuRo Feb 5, 2024
7269e4c
Reworked IdMapping
JoBuRo Feb 5, 2024
97d766a
Reworks on TransitivePath
JoBuRo Feb 6, 2024
c8c6526
Merge branch 'master' into use-graphblas
joka921 Feb 7, 2024
a06ed71
A tiny bugfix and make the stuff configurable.
joka921 Feb 7, 2024
6e84e8a
Fix build error, add move assignment to GrbMatrix
JoBuRo Feb 7, 2024
005dc9c
Simplifications
JoBuRo Feb 8, 2024
e7dbf4d
Reworks
JoBuRo Feb 9, 2024
714744a
Added timer to transitive path computation
JoBuRo Feb 9, 2024
0fd9f92
**WIP** Refactor of TransitivePath into Fallback and Graphblas
JoBuRo Feb 12, 2024
60a37b6
Fixed timing conversion
JoBuRo Feb 15, 2024
d07401e
Added singleton class for GraphBLAS global context
JoBuRo Feb 15, 2024
1f424de
Added some checkCancellation
JoBuRo Feb 15, 2024
d64f4b6
Build fix
JoBuRo Feb 20, 2024
816ba9a
Removed dead code
JoBuRo Feb 20, 2024
ecdf2a7
Sonar fixes
JoBuRo Feb 21, 2024
12f0e3d
Add timings to runtime info
JoBuRo Feb 21, 2024
ed6dc70
Some fixes.
joka921 Feb 21, 2024
dd8c841
Added binary search, updated tests
JoBuRo Mar 8, 2024
50e26ba
Merge branch 'master' into use-graphblas
joka921 Mar 13, 2024
4e64224
Fix merge conflicts.
joka921 Mar 13, 2024
b79470d
Moved sort to constructor
JoBuRo Mar 13, 2024
f6e2429
Merge remote-tracking branch 'origin/master' into use-graphblas
Mar 14, 2024
965d25a
Removed GraphBlas references and implementation
JoBuRo Mar 16, 2024
9eea0f6
Use TransitivePathFallback by default, so tests pass
JoBuRo Mar 16, 2024
7852bc3
Updated TransitiveFallback computeResult
JoBuRo Mar 19, 2024
2c9b117
Moved Map and Set definition
JoBuRo Mar 25, 2024
2fea8d7
Added checkCancellation to BinSearch
JoBuRo Mar 25, 2024
8039acc
Removed suite-sparse from git workflows
JoBuRo Mar 25, 2024
95f6dad
Style fix
JoBuRo Mar 25, 2024
75f4593
Rebased against current master
JoBuRo Mar 25, 2024
f580604
Merge branch 'master' into improve-transitive-path
JoBuRo Mar 25, 2024
c532162
Style fix
JoBuRo Mar 25, 2024
a65f6c5
Merge branch 'improve-transitive-path' of https://github.com/JoBuRo/q…
JoBuRo Mar 25, 2024
34beef3
Removed unnecessary include
JoBuRo Mar 25, 2024
f9811f9
Removed code duplication
JoBuRo Apr 8, 2024
1692d40
Fixed linker error
JoBuRo Apr 10, 2024
19d1c2e
Added TransitivePathImpl and added it to TransitivePathBinSearch
JoBuRo Apr 10, 2024
1c37ce4
Added TransitivePathImpl to TransitivePathFallback
JoBuRo Apr 10, 2024
d0f9330
Fixed build error
JoBuRo Apr 11, 2024
b17701a
Removed debug print statement
JoBuRo Apr 11, 2024
077f1fb
Added move statement
JoBuRo Apr 11, 2024
42f3878
Moved getChildren definition to cpp file
JoBuRo Apr 11, 2024
036c6fc
Removed unnecessary include
JoBuRo Apr 11, 2024
4ba0cab
Removed unused file
JoBuRo Apr 11, 2024
1511e35
Fixed copyright strings
JoBuRo Apr 11, 2024
c17477f
Moved computeResult to TransitivePathImpl
JoBuRo Apr 11, 2024
3e6330e
Renamed TransitivePathFallback to TransitivePathHashMap
JoBuRo Apr 11, 2024
c52e16e
Renamed runtime constant "use-binsearch" to "use-binsearch-transitive…
JoBuRo Apr 12, 2024
56a06f2
Paramteterized TransitivePathTests
JoBuRo Apr 12, 2024
b175cce
Added unit tests for bound cases of transitive paths
JoBuRo Apr 12, 2024
f395516
Merge branch 'master' into improve-transitive-path
JoBuRo Apr 12, 2024
05f5894
Fixed merge errors
JoBuRo Apr 12, 2024
669c29c
Move values in TransitivePath ctors
JoBuRo Apr 12, 2024
5580e14
Changed return type to auto on BinSearchMap.successors
JoBuRo Apr 12, 2024
dff2aa4
Added comment for C++23: use ranges::to
JoBuRo Apr 12, 2024
e9b63ec
Added docs
JoBuRo Apr 12, 2024
ee12697
Declared TransitivePathBase dtor as pure virtual
JoBuRo Apr 12, 2024
47c19ee
Simplified setupEdges in TransitivePathBinSearch
JoBuRo Apr 12, 2024
da97798
Fixed an issue with move semantics and ctors
JoBuRo Apr 12, 2024
e31afd7
Implemented HashMapWrapper
JoBuRo Apr 15, 2024
0a33997
Moved transitiveHull function to TransitiveHullImpl
JoBuRo Apr 15, 2024
6398363
const auto& in transitiveHull
JoBuRo Apr 15, 2024
8391366
Replaced assertSameUnorderedContent with gmock function
JoBuRo Apr 15, 2024
9821e28
Changed RuntimeParameter for transitivePath to true
JoBuRo Apr 15, 2024
0d6977f
Format fix
JoBuRo Apr 15, 2024
6cf3af1
Added some docs
JoBuRo Apr 15, 2024
d2ea890
Revert "Format fix"
JoBuRo Apr 15, 2024
7f70dd3
Format fix
JoBuRo Apr 16, 2024
199d390
Sonar Fixes
JoBuRo Apr 16, 2024
c139244
Added unit test for exception
JoBuRo Apr 16, 2024
5a48b5f
Added tests for 'zero or more' transitive path
JoBuRo Apr 16, 2024
79f5420
HashMapWrapper successors function returns const Set&
JoBuRo Apr 16, 2024
555aa6f
Use shorthand for QueryPlanner sort matcher
JoBuRo Apr 16, 2024
0faa6d8
Simplified tests a bit V(x) -> x
JoBuRo Apr 16, 2024
4eb14ed
Added a todo
JoBuRo Apr 16, 2024
1d496a3
Added doc for HashMapWrapper
JoBuRo Apr 16, 2024
9d6eaf3
Format fix
JoBuRo Apr 16, 2024
a768eb0
Try to fix the MacOS build
joka921 Apr 16, 2024
8a0bee4
Format fix
JoBuRo Apr 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/engine/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ add_library(engine
Distinct.cpp OrderBy.cpp Filter.cpp
Server.cpp QueryPlanner.cpp QueryPlanningCostFactors.cpp
OptionalJoin.cpp CountAvailablePredicates.cpp GroupBy.cpp HasPredicateScan.cpp
Union.cpp MultiColumnJoin.cpp TransitivePath.cpp Service.cpp
Union.cpp MultiColumnJoin.cpp TransitivePathBase.cpp
TransitivePathHashMap.cpp TransitivePathBinSearch.cpp Service.cpp
Values.cpp Bind.cpp Minus.cpp RuntimeInformation.cpp CheckUsePatternTrick.cpp
VariableToColumnMap.cpp ExportQueryExecutionTrees.cpp
CartesianProductJoin.cpp TextIndexScanForWord.cpp TextIndexScanForEntity.cpp
Expand Down
7 changes: 4 additions & 3 deletions src/engine/QueryExecutionTree.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
#include "engine/Sort.h"
#include "engine/TextIndexScanForEntity.h"
#include "engine/TextIndexScanForWord.h"
#include "engine/TransitivePath.h"
#include "engine/TransitivePathBase.h"
#include "engine/Union.h"
#include "engine/Values.h"
#include "engine/ValuesForTesting.h"
Expand Down Expand Up @@ -158,7 +158,7 @@ void QueryExecutionTree::setOperation(std::shared_ptr<Op> operation) {
type_ = VALUES;
} else if constexpr (std::is_same_v<Op, Service>) {
type_ = SERVICE;
} else if constexpr (std::is_same_v<Op, TransitivePath>) {
} else if constexpr (std::is_same_v<Op, TransitivePathBase>) {
type_ = TRANSITIVE_PATH;
} else if constexpr (std::is_same_v<Op, OrderBy>) {
type_ = ORDER_BY;
Expand Down Expand Up @@ -204,7 +204,8 @@ template void QueryExecutionTree::setOperation(std::shared_ptr<Sort>);
template void QueryExecutionTree::setOperation(std::shared_ptr<Distinct>);
template void QueryExecutionTree::setOperation(std::shared_ptr<Values>);
template void QueryExecutionTree::setOperation(std::shared_ptr<Service>);
template void QueryExecutionTree::setOperation(std::shared_ptr<TransitivePath>);
template void QueryExecutionTree::setOperation(
std::shared_ptr<TransitivePathBase>);
template void QueryExecutionTree::setOperation(std::shared_ptr<OrderBy>);
template void QueryExecutionTree::setOperation(std::shared_ptr<GroupBy>);
template void QueryExecutionTree::setOperation(
Expand Down
11 changes: 7 additions & 4 deletions src/engine/QueryPlanner.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
#include "engine/Sort.h"
#include "engine/TextIndexScanForEntity.h"
#include "engine/TextIndexScanForWord.h"
#include "engine/TransitivePath.h"
#include "engine/TransitivePathBase.h"
#include "engine/Union.h"
#include "engine/Values.h"
#include "parser/Alias.h"
Expand Down Expand Up @@ -435,8 +435,11 @@ std::vector<QueryPlanner::SubtreePlan> QueryPlanner::optimize(
right.value_ = getSideValue(arg._right);
size_t min = arg._min;
size_t max = arg._max;
auto plan = makeSubtreePlan<TransitivePath>(_qec, sub._qet, left,
right, min, max);
auto transitivePath = TransitivePathBase::makeTransitivePath(
_qec, std::move(sub._qet), std::move(left), std::move(right), min,
max);
auto plan =
makeSubtreePlan<TransitivePathBase>(std::move(transitivePath));
candidatesOut.push_back(std::move(plan));
}
joinCandidates(std::move(candidatesOut));
Expand Down Expand Up @@ -1890,7 +1893,7 @@ auto QueryPlanner::createJoinWithTransitivePath(
std::shared_ptr<QueryExecutionTree> otherTree =
aIsTransPath ? b._qet : a._qet;
auto& transPathTree = aIsTransPath ? a._qet : b._qet;
auto transPathOperation = std::dynamic_pointer_cast<TransitivePath>(
auto transPathOperation = std::dynamic_pointer_cast<TransitivePathBase>(
transPathTree->getRootOperation());

// TODO: Handle the case of two or more common variables
Expand Down
Loading
Loading