Skip to content

Commit 2cc53db

Browse files
msgpack: support datetime interval arithmetic
Support datetime and interval arithmetic with the same rules as in Tarantool [1]. Valid operations: - `tarantool.Datetime` + `tarantool.Interval` = `tarantool.Datetime` - `tarantool.Datetime` - `tarantool.Interval` = `tarantool.Datetime` - `tarantool.Datetime` - `tarantool.Datetime` = `tarantool.Interval` - `tarantool.Interval` + `tarantool.Interval` = `tarantool.Interval` - `tarantool.Interval` - `tarantool.Interval` = `tarantool.Interval` Since `tarantool.Interval` could contain `month` and `year` fields and such operations could be ambiguous, you can use `adjust` field to tune the logic. - `tarantool.IntervalAdjust.NONE` -- only truncation toward the end of month performed (default mode). ``` >>> dt = tarantool.Datetime(year=2022, month=3, day=31) datetime: Timestamp('2022-03-31 00:00:00'), tz: "" >>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.NONE) >>> dt + di datetime: Timestamp('2022-04-30 00:00:00'), tz: "" ``` - `tarantool.IntervalAdjust.EXCESS` -- overflow mode, without any snap or truncation to the end of month, straight addition of days in month, stopping over month boundaries if there is less number of days. ``` >>> dt = tarantool.Datetime(year=2022, month=1, day=31) datetime: Timestamp('2022-01-31 00:00:00'), tz: "" >>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.EXCESS) >>> dt + di datetime: Timestamp('2022-03-02 00:00:00'), tz: "" ``` - `tarantool.IntervalAdjust.LAST` -- mode when day snaps to the end of month, if happens. ``` >>> dt = tarantool.Datetime(year=2022, month=2, day=28) datetime: Timestamp('2022-02-28 00:00:00'), tz: "" >>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.LAST) >>> dt + di datetime: Timestamp('2022-03-31 00:00:00'), tz: "" ``` Tarantool does not yet correctly support subtraction of datetime objects with different timezones [2] and addition of intervals to datetimes with non-fixed offset timezones [3]. tarantool-python implementation support them, but it could be reworked later if core team choose another solution. 1. https://github.com/tarantool/tarantool/wiki/Datetime-Internals#interval-arithmetic 2. tarantool/tarantool#7698 3. tarantool/tarantool#7700 Closes #229
1 parent 51268a1 commit 2cc53db

File tree

5 files changed

+552
-1
lines changed

5 files changed

+552
-1
lines changed

Diff for: CHANGELOG.md

+47
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
8686
Its attributes (same as in init API) are exposed, so you can
8787
use them if needed.
8888

89+
- Datetime interval arithmetic support (#229).
90+
91+
Valid operations:
92+
- `tarantool.Datetime` + `tarantool.Interval` = `tarantool.Datetime`
93+
- `tarantool.Datetime` + `tarantool.Interval` = `tarantool.Datetime`
94+
- `tarantool.Datetime` - `tarantool.Datetime` = `tarantool.Interval`
95+
- `tarantool.Interval` + `tarantool.Interval` = `tarantool.Interval`
96+
- `tarantool.Interval` - `tarantool.Interval` = `tarantool.Interval`
97+
98+
Since `tarantool.Interval` could contain `month` and `year` fields
99+
and such operations could be ambiguous, you can use `adjust` field
100+
to tune the logic.
101+
102+
- `tarantool.IntervalAdjust.NONE` -- only truncation toward the end of
103+
month performed (default mode).
104+
105+
```python
106+
>>> dt = tarantool.Datetime(year=2022, month=3, day=31)
107+
datetime: Timestamp('2022-03-31 00:00:00'), tz: ""
108+
>>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.NONE)
109+
>>> dt + di
110+
datetime: Timestamp('2022-04-30 00:00:00'), tz: ""
111+
```
112+
113+
- `tarantool.IntervalAdjust.EXCESS` -- overflow mode, without any snap
114+
or truncation to the end of month, straight addition of days in month,
115+
stopping over month boundaries if there is less number of days.
116+
117+
```python
118+
>>> dt = tarantool.Datetime(year=2022, month=1, day=31)
119+
datetime: Timestamp('2022-01-31 00:00:00'), tz: ""
120+
>>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.EXCESS)
121+
>>> dt + di
122+
datetime: Timestamp('2022-03-02 00:00:00'), tz: ""
123+
```
124+
125+
- `tarantool.IntervalAdjust.LAST` -- mode when day snaps to the end of month,
126+
if happens.
127+
128+
```python
129+
>>> dt = tarantool.Datetime(year=2022, month=2, day=28)
130+
datetime: Timestamp('2022-02-28 00:00:00'), tz: ""
131+
>>> di = tarantool.Interval(month=1, adjust=tarantool.IntervalAdjust.LAST)
132+
>>> dt + di
133+
datetime: Timestamp('2022-03-31 00:00:00'), tz: ""
134+
```
135+
89136
### Changed
90137
- Bump msgpack requirement to 1.0.4 (PR #223).
91138
The only reason of this bump is various vulnerability fixes,

Diff for: tarantool/msgpack_ext/types/datetime.py

+80
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66
import tarantool.msgpack_ext.types.timezones as tt_timezones
77
from tarantool.error import MsgpackError
88

9+
from tarantool.msgpack_ext.types.interval import Interval, Adjust
10+
911
# https://www.tarantool.io/en/doc/latest/dev_guide/internals/msgpack_extensions/#the-datetime-type
1012
#
1113
# The datetime MessagePack representation looks like this:
@@ -47,6 +49,7 @@
4749
NSEC_IN_SEC = 1000000000
4850
NSEC_IN_MKSEC = 1000
4951
SEC_IN_MIN = 60
52+
MONTH_IN_YEAR = 12
5053

5154
def get_bytes_as_int(data, cursor, size):
5255
part = data[cursor:cursor + size]
@@ -168,6 +171,83 @@ def __init__(self, data=None, *, timestamp=None, year=None, month=None,
168171
self._datetime = datetime
169172
self._tz = ''
170173

174+
def _interval_operation(self, other, sign=1):
175+
self_dt = self._datetime
176+
177+
# https://github.com/tarantool/tarantool/wiki/Datetime-Internals#date-adjustions-and-leap-years
178+
months = other.year * MONTH_IN_YEAR + other.month
179+
180+
res = self_dt + pandas.DateOffset(months = sign * months)
181+
182+
# pandas.DateOffset works exactly like Adjust.NONE
183+
if other.adjust == Adjust.EXCESS:
184+
if self_dt.day > res.day:
185+
res = res + pandas.DateOffset(days = self_dt.day - res.day)
186+
elif other.adjust == Adjust.LAST:
187+
if self_dt.is_month_end:
188+
# day replaces days
189+
res = res.replace(day = res.days_in_month)
190+
191+
res = res + pandas.Timedelta(weeks = sign * other.week,
192+
days = sign * other.day,
193+
hours = sign * other.hour,
194+
minutes = sign * other.minute,
195+
seconds = sign * other.sec,
196+
microseconds = sign * (other.nsec // NSEC_IN_MKSEC),
197+
nanoseconds = sign * (other.nsec % NSEC_IN_MKSEC))
198+
199+
if res.tzinfo is not None:
200+
tzoffset = compute_offset(res)
201+
else:
202+
tzoffset = 0
203+
return Datetime(year=res.year, month=res.month, day=res.day,
204+
hour=res.hour, minute=res.minute, sec=res.second,
205+
nsec=res.nanosecond + res.microsecond * NSEC_IN_MKSEC,
206+
tzoffset=tzoffset, tz=self.tz)
207+
208+
def __add__(self, other):
209+
if not isinstance(other, Interval):
210+
raise TypeError(f"unsupported operand type(s) for +: '{type(self)}' and '{type(other)}'")
211+
212+
return self._interval_operation(other, sign=1)
213+
214+
def __sub__(self, other):
215+
if isinstance(other, Datetime):
216+
self_dt = self._datetime
217+
other_dt = other._datetime
218+
219+
# Tarantool datetime subtraction ignores timezone info, but it is a bug:
220+
#
221+
# Tarantool 2.10.1-0-g482d91c66
222+
#
223+
# tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
224+
# ---
225+
# - +0 seconds
226+
# ...
227+
#
228+
# Refer to https://github.com/tarantool/tarantool/issues/7698
229+
# for possible updates.
230+
231+
if self_dt.tzinfo != other_dt.tzinfo:
232+
other_dt = other_dt.tz_convert(self_dt.tzinfo)
233+
234+
self_nsec = self_dt.microsecond * NSEC_IN_MKSEC + self_dt.nanosecond
235+
other_nsec = other_dt.microsecond * NSEC_IN_MKSEC + other_dt.nanosecond
236+
237+
return Interval(
238+
year = self_dt.year - other_dt.year,
239+
month = self_dt.month - other_dt.month,
240+
day = self_dt.day - other_dt.day,
241+
hour = self_dt.hour - other_dt.hour,
242+
minute = self_dt.minute - other_dt.minute,
243+
sec = self_dt.second - other_dt.second,
244+
nsec = self_nsec - other_nsec,
245+
)
246+
elif isinstance(other, Interval):
247+
return self._interval_operation(other, sign=-1)
248+
else:
249+
raise TypeError(f"unsupported operand type(s) for -: '{type(self)}' and '{type(other)}'")
250+
171251
def __eq__(self, other):
172252
if isinstance(other, Datetime):
173253
return self._datetime == other._datetime

Diff for: tarantool/msgpack_ext/types/interval.py

+64
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,70 @@ def __init__(self, data=None, *, year=0, month=0, week=0,
102102
self.nsec = nsec
103103
self.adjust = adjust
104104

105+
def __add__(self, other):
106+
if not isinstance(other, Interval):
107+
raise TypeError(f"unsupported operand type(s) for +: '{type(self)}' and '{type(other)}'")
108+
109+
# Tarantool saves adjust of the first argument
110+
#
111+
# Tarantool 2.10.1-0-g482d91c66
112+
#
113+
# tarantool> dt1 = datetime.interval.new{year = 2, adjust='last'}
114+
# ---
115+
# ...
116+
#
117+
# tarantool> dt2 = datetime.interval.new{year = 1, adjust='excess'}
118+
# ---
119+
# ...
120+
#
121+
# tarantool> (dt1 + dt2).adjust
122+
# ---
123+
# - 'cdata<enum 112>: 2'
124+
# ...
125+
126+
return Interval(
127+
year = self.year + other.year,
128+
month = self.month + other.month,
129+
day = self.day + other.day,
130+
hour = self.hour + other.hour,
131+
minute = self.minute + other.minute,
132+
sec = self.sec + other.sec,
133+
nsec = self.nsec + other.nsec,
134+
adjust = self.adjust,
135+
)
136+
137+
def __sub__(self, other):
138+
if not isinstance(other, Interval):
139+
raise TypeError(f"unsupported operand type(s) for -: '{type(self)}' and '{type(other)}'")
140+
141+
# Tarantool saves adjust of the first argument
142+
#
143+
# Tarantool 2.10.1-0-g482d91c66
144+
#
145+
# tarantool> dt1 = datetime.interval.new{year = 2, adjust='last'}
146+
# ---
147+
# ...
148+
#
149+
# tarantool> dt2 = datetime.interval.new{year = 1, adjust='excess'}
150+
# ---
151+
# ...
152+
#
153+
# tarantool> (dt1 - dt2).adjust
154+
# ---
155+
# - 'cdata<enum 112>: 2'
156+
# ...
157+
158+
return Interval(
159+
year = self.year - other.year,
160+
month = self.month - other.month,
161+
day = self.day - other.day,
162+
hour = self.hour - other.hour,
163+
minute = self.minute - other.minute,
164+
sec = self.sec - other.sec,
165+
nsec = self.nsec - other.nsec,
166+
adjust = self.adjust,
167+
)
168+
105169
def __eq__(self, other):
106170
if not isinstance(other, Interval):
107171
return False

Diff for: test/suites/__init__.py

+3-1
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,16 @@
1919
from .test_uuid import TestSuite_UUID
2020
from .test_datetime import TestSuite_Datetime
2121
from .test_interval import TestSuite_Interval
22+
from .test_datetime_arithmetic import TestSuite_DatetimeArithmetic
2223

2324
test_cases = (TestSuite_Schema_UnicodeConnection,
2425
TestSuite_Schema_BinaryConnection,
2526
TestSuite_Request, TestSuite_Protocol, TestSuite_Reconnect,
2627
TestSuite_Mesh, TestSuite_Execute, TestSuite_DBAPI,
2728
TestSuite_Encoding, TestSuite_Pool, TestSuite_Ssl,
2829
TestSuite_Decimal, TestSuite_UUID, TestSuite_Datetime,
29-
TestSuite_Interval)
30+
TestSuite_Interval, TestSuite_DatetimeArithmetic)
31+
3032

3133
def load_tests(loader, tests, pattern):
3234
suite = unittest.TestSuite()

0 commit comments

Comments
 (0)