|
1 | | -Announcing Python-Blosc2 4.10.1 |
| 1 | +Announcing Python-Blosc2 4.11.0 |
2 | 2 | =============================== |
3 | 3 |
|
4 | | -A correctness release: lazy indexing and reductions now follow NumPy in a |
5 | | -batch of cases where they quietly did not, the stores close several |
6 | | -cross-process read races, and wheels finally ship usable C-Blosc2 development |
7 | | -files. Bundled C-Blosc2 moves to 3.3.2. |
8 | | - |
9 | | -- **Lazy indexing now matches NumPy.** Integer indexing no longer squeezes |
10 | | - length-1 axes the index kept, a ``None`` in the key stops shifting operand |
11 | | - axes for ``LazyUDF`` and broadcast operands, indexing a full reduction |
12 | | - slices the operands instead of evaluating over everything, and |
13 | | - ``datetime64``/``timedelta64`` comparisons work in expressions rather than |
14 | | - raising. |
15 | | - |
16 | | -- **``NDArray.nbytes`` reports the logical size**, ``size * itemsize``, as |
17 | | - NumPy does. ``.schunk.nbytes`` still gives the padded figure, which is what |
18 | | - ``cratio`` keeps measuring. |
19 | | - |
20 | | -- **``CTable.where()`` applied a short boolean mask to the wrong rows.** A |
21 | | - mask no longer than the live-row count is now logical — entry *i* selects |
22 | | - the *i*-th live row — instead of being padded out to the physical length |
23 | | - and picking up rows outside the view. |
24 | | - |
25 | | -- **Cross-process store fixes.** ``EmbedStore`` and ``DictStore`` resolved a |
26 | | - key under the store lock but read the data after releasing it; the resolve |
27 | | - and the read now share one lock. Overwriting an external ``DictStore`` leaf |
28 | | - is atomic too — the new leaf is built beside its final name and moved into |
29 | | - place — so a concurrent reader can no longer open a half-rewritten file. |
30 | | - |
31 | | -- **Wheels ship working C-Blosc2 development files** (``pkg-config`` and |
32 | | - ``find_package(Blosc2)`` both failed against an installed wheel before), |
33 | | - and are ~1.5 MB smaller, carrying two copies of ``libblosc2`` rather than |
34 | | - three. |
| 4 | +Nullability in ``CTable`` is rebuilt on Arrow's own model: a nullable column |
| 5 | +keeps its nulls in a validity sidecar instead of reserving a value from its own |
| 6 | +range. That makes it lossless, and everything above it — predicates, indexes, |
| 7 | +Arrow/Parquet/CSV round-trips — follows. Wheels become a single Stable ABI |
| 8 | +build per platform. |
| 9 | + |
| 10 | +- **Mask-based nullable columns, and they are the default.** A bare |
| 11 | + ``nullable=True`` no longer steals a value from the dtype, so an ``int8`` |
| 12 | + column can hold ``-128``, a ``utf8`` one can hold ``""``, a ``float64`` one |
| 13 | + can tell ``NaN`` from missing, and ``complex128`` is nullable at all for the |
| 14 | + first time. ``None`` is how you write a null. Note that a table with a |
| 15 | + mask column records schema version 3, which readers older than 4.11.0 refuse |
| 16 | + to open. |
| 17 | + |
| 18 | +- **Predicates over nulls follow three-valued (Kleene) logic.** A comparison |
| 19 | + against a null is now *unknown* rather than ``False``, so ``~(t.price > 10)`` |
| 20 | + returns the rows definitely not above 10 instead of every null row, and |
| 21 | + ``~((a > 10) & (b == 999))`` stops dropping rows that qualify. Both the |
| 22 | + operator and the string query form agree with SQL. A predicate can also be |
| 23 | + asked about its unknown rows: ``p.is_null()``, ``p.null_count()``, |
| 24 | + ``p.fillna(True)``. |
| 25 | + |
| 26 | +- **Column indexes are null-aware.** Per-segment ``min``/``max`` are taken over |
| 27 | + the rows that carry a value, so ``Column.min``/``Column.max`` answer from the |
| 28 | + index for a nullable column instead of scanning, and ``where()`` with an ``OR`` |
| 29 | + over a nullable indexed column no longer falls back to a full scan (**1.6x**). |
| 30 | + ``rebuild_index()`` promotes indexes written by an earlier release. |
| 31 | + |
| 32 | +- **A single Stable ABI (abi3) wheel per platform**, serving CPython 3.11 and |
| 33 | + every later version — so a new CPython is installable from a wheel without |
| 34 | + waiting for a blosc2 release. Free-threaded 3.14 and 3.15 ship alongside as |
| 35 | + version-specific wheels. No measurable performance cost. |
| 36 | + |
| 37 | +- **Plus a long list of fixes** around nullable columns: CSV import/export, |
| 38 | + ``extend()`` between storages, sorted-view reductions, descending sorts of |
| 39 | + the widest integers, timestamp writes through ``col[key] = value``, scalar |
| 40 | + broadcast on mask columns, nested columns surviving ``convert_nulls()`` and a |
| 41 | + save/reopen cycle, and more. Outside ``CTable``, ``asarray()`` no longer |
| 42 | + corrupts arrays whose chunks overhang the shape. |
35 | 43 |
|
36 | 44 | Install it with:: |
37 | 45 |
|
|
0 commit comments