Fix dimension filtering with BETWEEN in date-type Dims #370
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DimitrisStaratzis
force-pushed
the
dstara/sc-53762/mariadb-issue-with-datetime-dimension-filtering
branch
from
4842a52 to
bc6de95
Compare
Shelnutt2
approved these changes
Aug 27, 2024
robertbindar
approved these changes
Aug 28, 2024
DimitrisStaratzis
deleted the
dstara/sc-53762/mariadb-issue-with-datetime-dimension-filtering
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bug description
This PR fixes the BETWEEN operator when used in Datetime dims.
Query:
select * from nyc where `tpep_pickup_datetime` BETWEEN '2022-01-01T00:00:00' AND '2022-01-02T00:00:00';More specifically, querying the nyc_taxi 18GB array before the fix was creating these ranges:
[Note] pushed conditions: [1640995200000000000, 4061240611000000000]It now creates:
[Note] pushed conditions: [1640995200000000000, 1641081600000000000]Bug fix
The bug was happening because the
BETWEENoperator when used with datetimes has 2 arguments which are not constants. This led us to wrongly cast toItem_basic_constantand thus making the ranges nullptrs. I have also checked whether this bug is present to non-datetime dims and it is not.