Skip to content

Commit 81b7894

Browse files
committed
Getting ready for release 4.11.0
1 parent 9f456ae commit 81b7894

4 files changed

Lines changed: 194 additions & 163 deletions

File tree

ANNOUNCE.rst

Lines changed: 40 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,45 @@
1-
Announcing Python-Blosc2 4.10.1
1+
Announcing Python-Blosc2 4.11.0
22
===============================
33

4-
A correctness release: lazy indexing and reductions now follow NumPy in a
5-
batch of cases where they quietly did not, the stores close several
6-
cross-process read races, and wheels finally ship usable C-Blosc2 development
7-
files. Bundled C-Blosc2 moves to 3.3.2.
8-
9-
- **Lazy indexing now matches NumPy.** Integer indexing no longer squeezes
10-
length-1 axes the index kept, a ``None`` in the key stops shifting operand
11-
axes for ``LazyUDF`` and broadcast operands, indexing a full reduction
12-
slices the operands instead of evaluating over everything, and
13-
``datetime64``/``timedelta64`` comparisons work in expressions rather than
14-
raising.
15-
16-
- **``NDArray.nbytes`` reports the logical size**, ``size * itemsize``, as
17-
NumPy does. ``.schunk.nbytes`` still gives the padded figure, which is what
18-
``cratio`` keeps measuring.
19-
20-
- **``CTable.where()`` applied a short boolean mask to the wrong rows.** A
21-
mask no longer than the live-row count is now logical — entry *i* selects
22-
the *i*-th live row — instead of being padded out to the physical length
23-
and picking up rows outside the view.
24-
25-
- **Cross-process store fixes.** ``EmbedStore`` and ``DictStore`` resolved a
26-
key under the store lock but read the data after releasing it; the resolve
27-
and the read now share one lock. Overwriting an external ``DictStore`` leaf
28-
is atomic too — the new leaf is built beside its final name and moved into
29-
place — so a concurrent reader can no longer open a half-rewritten file.
30-
31-
- **Wheels ship working C-Blosc2 development files** (``pkg-config`` and
32-
``find_package(Blosc2)`` both failed against an installed wheel before),
33-
and are ~1.5 MB smaller, carrying two copies of ``libblosc2`` rather than
34-
three.
4+
Nullability in ``CTable`` is rebuilt on Arrow's own model: a nullable column
5+
keeps its nulls in a validity sidecar instead of reserving a value from its own
6+
range. That makes it lossless, and everything above it — predicates, indexes,
7+
Arrow/Parquet/CSV round-trips — follows. Wheels become a single Stable ABI
8+
build per platform.
9+
10+
- **Mask-based nullable columns, and they are the default.** A bare
11+
``nullable=True`` no longer steals a value from the dtype, so an ``int8``
12+
column can hold ``-128``, a ``utf8`` one can hold ``""``, a ``float64`` one
13+
can tell ``NaN`` from missing, and ``complex128`` is nullable at all for the
14+
first time. ``None`` is how you write a null. Note that a table with a
15+
mask column records schema version 3, which readers older than 4.11.0 refuse
16+
to open.
17+
18+
- **Predicates over nulls follow three-valued (Kleene) logic.** A comparison
19+
against a null is now *unknown* rather than ``False``, so ``~(t.price > 10)``
20+
returns the rows definitely not above 10 instead of every null row, and
21+
``~((a > 10) & (b == 999))`` stops dropping rows that qualify. Both the
22+
operator and the string query form agree with SQL. A predicate can also be
23+
asked about its unknown rows: ``p.is_null()``, ``p.null_count()``,
24+
``p.fillna(True)``.
25+
26+
- **Column indexes are null-aware.** Per-segment ``min``/``max`` are taken over
27+
the rows that carry a value, so ``Column.min``/``Column.max`` answer from the
28+
index for a nullable column instead of scanning, and ``where()`` with an ``OR``
29+
over a nullable indexed column no longer falls back to a full scan (**1.6x**).
30+
``rebuild_index()`` promotes indexes written by an earlier release.
31+
32+
- **A single Stable ABI (abi3) wheel per platform**, serving CPython 3.11 and
33+
every later version — so a new CPython is installable from a wheel without
34+
waiting for a blosc2 release. Free-threaded 3.14 and 3.15 ship alongside as
35+
version-specific wheels. No measurable performance cost.
36+
37+
- **Plus a long list of fixes** around nullable columns: CSV import/export,
38+
``extend()`` between storages, sorted-view reductions, descending sorts of
39+
the widest integers, timestamp writes through ``col[key] = value``, scalar
40+
broadcast on mask columns, nested columns surviving ``convert_nulls()`` and a
41+
save/reopen cycle, and more. Outside ``CTable``, ``asarray()`` no longer
42+
corrupts arrays whose chunks overhang the shape.
3543

3644
Install it with::
3745

0 commit comments

Comments
 (0)