Skip to content

Commit 56790ef

Browse files
committed
Post 4.10.0 release actions done
1 parent 732ca31 commit 56790ef

4 files changed

Lines changed: 45 additions & 32 deletions

File tree

ANNOUNCE.rst

Lines changed: 39 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,44 @@
1-
Announcing Python-Blosc2 4.9.1
1+
Announcing Python-Blosc2 4.10.0
22
==============================
33

4-
This is a hot-fix release for the Arrow interop work introduced in 4.9.0 —
5-
one real performance regression and one clearer error message, both in
6-
``CTable``.
7-
8-
- **Faster dictionary-column Arrow export**: ``CTable.iter_arrow_batches()``
9-
(and therefore ``to_arrow()`` and the Arrow PyCapsule interchange,
10-
``__arrow_c_stream__``) was recomputing the full live-row-position array
11-
from scratch on every batch, for every dictionary-encoded string column —
12-
an ``O(n_rows)`` scan repeated ``O(n_rows / batch_size)`` times. It's now
13-
computed once per export call instead. 6-14x faster export for
14-
dictionary columns on a 1M-row benchmark.
15-
16-
- **Worth knowing regardless of this fix**: the Arrow PyCapsule protocol
17-
(``__arrow_c_stream__``) has no column-projection pushdown — a consumer
18-
that only needs two columns (DuckDB, pyarrow, Polars, pandas) still
19-
triggers export of *every* column in the table, since the raw Arrow C
20-
Stream interface has no way to say "I only need these." Use
21-
``CTable.select([...])`` to project down to the columns you actually
22-
need before handing the table off, especially if any column holds an
23-
expensive nested/list type::
24-
25-
sub = t.select(["company", "fare"])
26-
duckdb.sql("SELECT company, avg(sub.fare) FROM sub GROUP BY company").show()
27-
28-
- **Clearer error on ``mode="a"``**: opening a ``CTable`` with
29-
``mode="a"`` at a path that doesn't exist yet now raises a
30-
``FileNotFoundError`` explaining that ``mode="a"`` opens an existing
31-
table (use ``mode="w"`` to create one), instead of silently creating a
32-
new, empty table.
4+
This is the string-support release: string expressions and DSL kernels now
5+
run on miniexpr, ``utf8()`` and ``dictionary()`` columns gain full indexing
6+
and comparisons, and NumPy's ``StringDType`` is understood by the array
7+
constructors. Alongside, slicing with plain keys is up to 1.7x faster and a
8+
new ``blosc2.random`` module brings chunk-parallel, NumPy-quality random
9+
constructors.
10+
11+
- **String expressions and DSL kernels over strings.** Concatenation,
12+
``lower``/``upper``/``strip``/``replace``/``substr``/``split_part`` and
13+
friends now run on miniexpr over fixed-width ``<Un`` and bytes ``S``
14+
arrays, producing string results sized by miniexpr itself. ``utf8()``
15+
columns can be queried in expression form
16+
(``t.where("name == 'x'")``), with scalar comparisons 5-6x faster via a
17+
raw-byte scan, and new ``blosc2.utf8_array()`` builds variable-length
18+
arrays directly.
19+
20+
- **Full indexing for string columns.** ``create_index()`` now works on
21+
``utf8()`` and ``dictionary()`` columns via alphabetical ranks —
22+
``sort_by`` drops from 424 ms to 7 ms at 1M rows — and scalar comparisons
23+
are served from the index.
24+
25+
- **New ``blosc2.random`` module**: 42 of NumPy's 43 ``Generator`` methods,
26+
each chunk generated in parallel with its own seeded ``PCG64`` stream
27+
(~3x faster than the NumPy path on 100M elements).
28+
29+
- **Slicing up to 1.7x faster.** Plain slice/int keys skip ndindex's general
30+
machinery (it was 43% of a scattered-read loop); strided steps, ellipsis
31+
and fancy indexing still use it.
32+
33+
- **String plumbing**: ``from_utf8()``/``to_utf8()`` conversions,
34+
``CTable.add_column(values=)``, ``Column.assign()`` on variable-length
35+
columns, ``StringDType`` dispatch in the array constructors, and DSL
36+
operands can be native NumPy arrays or pandas ``Series``.
37+
38+
- **Important fixes**: string column indexes returning zero rows at the
39+
default column width, ``SChunk`` slices for typesizes above 255 bytes
40+
(upstream, via C-Blosc2 3.3.1), scalar bools in tuple keys now matching
41+
NumPy, and a batch of miniexpr correctness fixes.
3342

3443
Install it with::
3544

RELEASE_NOTES.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# Release notes
22

3+
## Changes from 4.10.0 to 4.10.1
4+
5+
XXX version-specific blurb XXX
6+
37
## Changes from 4.9.1 to 4.10.0
48

59
This release is the string-support milestone: string expressions and DSL

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ dependencies = [
4141
"rich",
4242
"threadpoolctl; platform_machine != 'wasm32'",
4343
]
44-
version = "4.10.0"
44+
version = "4.10.1.dev0"
4545
[project.entry-points."array_api"]
4646
blosc2 = "blosc2"
4747

src/blosc2/version.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
__version__ = "4.10.0"
1+
__version__ = "4.10.1.dev0"
22
__array_api_version__ = "2024.12"

0 commit comments

Comments
 (0)