Rework the utf8 querying tip around the new lookup cost

The tip claimed a FULL index only pays off on a utf8 column while the
text repeats, and blamed the sorted rank list for growing with the number
of distinct values. The cost was the vocabulary load, not the lookup, and
it is gone now: the index is worth having at either cardinality.

The lookup figure gains warm bars beside the first-lookup ones (the
harness times one fresh process per variant, so it can only ever report
first lookups) and takes the full width; the index build panel below it
now reports peak memory as well as time, which is where the two flavours
differ most. Also fixes the previous subsection's claim that a utf8
column reads back in a tenth of the fixed-width memory: measured, it is
about a quarter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: FrancescAlted

bench/optim_tips/tip_13_utf8_strings.py

@@ -20,9 +20,11 @@
 #   (c) tip_13c_utf8_ondisk.png -- bytes on disk, column and index
 #
 # Both cardinalities (20k distinct, near-unique) are measured throughout, so
-# the FULL index's cardinality ceiling on utf8() -- values are indexed by
-# alphabetical rank, and the rank table grows with the number of distinct
-# values -- is visible rather than hidden.
+# any cardinality sensitivity of the FULL index is visible rather than hidden:
+# a utf8 column is indexed by alphabetical rank, and a query literal becomes a
+# rank by bisecting the index's vocabulary sidecar.  The first lookup in a
+# process opens that sidecar and the ones after it reuse the handle, which is
+# what the "1st lookup" and "warm" bars separate.
 #
 # Synthetic free-text catalogue, built once and reused: every measured
 # variant runs in a fresh subprocess that re-imports this module, so the
@@ -53,9 +55,12 @@
 # one-shot work anyway.
 BENCH_REPS = 1
 
-# Lighter shades of the two series colours, for the "+ FULL index" bars.
+# Lighter shades of the two series colours, for the "+ FULL index" bars:
+# _IDX is the first lookup in a process, _WARM every lookup after it.
 COLOR_NAIVE_IDX = "#9dc0ea"
 COLOR_TIP_IDX = "#8fd9bd"
+COLOR_NAIVE_WARM = "#5b9bd5"
+COLOR_TIP_WARM = "#45bf94"
 
 # A vocabulary with accented and CJK entries, so "UTF-8 bytes vs UCS-4
 # codepoints" is a real gap and not an ASCII artefact.
@@ -215,6 +220,26 @@ def numpy_string_20k():   return _numpy("string", "20k")
 # fmt: on
 
 
+def warm_lookup(flavour, card, reps=5):
+    """Steady-state indexed lookup: repeated calls on one open table.
+
+    ``measure()`` runs every variant in a fresh process, and ``_where()`` opens
+    the table anew on each rep, so its "+ FULL" numbers are all *first* lookups
+    -- which pay for opening the index sidecars (the vocabulary among them, on
+    utf8) before answering.  Timed here rather than through the harness because
+    the whole point is to reuse one table object across calls; peak memory is
+    not of interest.
+    """
+    t = blosc2.CTable.open(path_for(flavour, card, indexed=True))
+    needle = NEEDLE[card]
+    best = float("inf")
+    for _ in range(reps):
+        t0 = time.perf_counter()
+        len(t.where(f"title == {needle!r}")[:])
+        best = min(best, time.perf_counter() - t0)
+    return best
+
+
 def grouped_bars(ax, title, groups, series, values, fmt, ylabel="Time (s)", legend_cols=1, title_size=9.5):
     """One panel: len(groups) clusters of len(series) direct-labeled bars."""
     x = np.arange(len(groups), dtype=float)
@@ -272,6 +297,16 @@ def save(fig, name, rect=(0, 0, 1, 0.9)):
             f"utf8 {fmt_bytes(m[f'read_utf8_{card}'])}"
             f"   ({m[f'read_string_{card}'] / m[f'read_utf8_{card}']:.2f}x)"
         )
+    warm = {(f, c): warm_lookup(f, c) for f, c in COMBOS}
+    print("\nindexed lookup, warm (same open table):")
+    for card in CARDS:
+        s, u = warm["string", card], warm["utf8", card]
+        print(
+            f"  {card:>4}: string {s * 1000:6.1f}ms  utf8 {u * 1000:6.1f}ms   "
+            f"(1st lookup: {t[f'whereidx_string_{card}'] * 1000:.1f}ms / "
+            f"{t[f'whereidx_utf8_{card}'] * 1000:.1f}ms)"
+        )
+
     print(
         f"\nNumPy read+compare (20k): string {t['numpy_string_20k']:.3f}s  utf8 {t['numpy_utf8_20k']:.3f}s"
     )
@@ -290,6 +325,14 @@ def save(fig, name, rect=(0, 0, 1, 0.9)):
         (f"string({MAX_LENGTH}) + FULL", COLOR_NAIVE_IDX),
         ("utf8() + FULL", COLOR_TIP_IDX),
     )
+    series_query = (
+        (f"string({MAX_LENGTH})", COLOR_NAIVE),
+        ("utf8()", COLOR_TIP),
+        (f"string({MAX_LENGTH}) + FULL, 1st lookup", COLOR_NAIVE_IDX),
+        ("utf8() + FULL, 1st lookup", COLOR_TIP_IDX),
+        (f"string({MAX_LENGTH}) + FULL, warm", COLOR_NAIVE_WARM),
+        ("utf8() + FULL, warm", COLOR_TIP_WARM),
+    )
     groups = (f"titles repeat\n({CARD_LOW // 1000}k different ones)", "titles nearly\nall different")
     mrows = f"{N // 1_000_000} Mrow"
     secs = lambda v: f"{v:.2f}s"  # noqa: E731
@@ -315,23 +358,34 @@ def save(fig, name, rect=(0, 0, 1, 0.9)):
     )  # fmt: skip
     save(fig, "tip_13a_utf8_read.png")
 
-    # (b) Querying, with and without a FULL index.
-    fig, axes = plt.subplots(1, 2, figsize=(9.5, 3.6))
+    # (b) Querying, with and without a FULL index.  Six bars per cluster need
+    # the full width, so the lookup panel takes a row of its own and the two
+    # index-build panels share the one below it.
+    fig, axd = plt.subplot_mosaic(
+        [["lookup", "lookup"], ["build_t", "build_m"]],
+        figsize=(9.5, 6.6), height_ratios=(1.2, 0.8),
+    )  # fmt: skip
     fig.suptitle(
         f"Querying a text column — {mrows}, where('title == ...')",
         fontsize=11.5, color=INK,
     )  # fmt: skip
     grouped_bars(
-        axes[0], "Equality lookup", groups, series_idx,
+        axd["lookup"], "Equality lookup", groups, series_query,
         [[t[f"where_string_{c}"], t[f"where_utf8_{c}"],
-          t[f"whereidx_string_{c}"], t[f"whereidx_utf8_{c}"]] for c in CARDS],
-        msecs, ylabel="Time (ms)", legend_cols=2,
+          t[f"whereidx_string_{c}"], t[f"whereidx_utf8_{c}"],
+          warm["string", c], warm["utf8", c]] for c in CARDS],
+        msecs, ylabel="Time (ms)", legend_cols=3,
     )  # fmt: skip
     grouped_bars(
-        axes[1], "FULL index build", groups, series_short,
+        axd["build_t"], "FULL index build — time", groups, series_short,
         [[t[f"index_string_{c}"], t[f"index_utf8_{c}"]] for c in CARDS], secs,
     )  # fmt: skip
-    save(fig, "tip_13b_utf8_query.png")
+    grouped_bars(
+        axd["build_m"], "FULL index build — peak memory", groups, series_short,
+        [[m[f"index_string_{c}"], m[f"index_utf8_{c}"]] for c in CARDS], fmt_bytes,
+        ylabel="Peak memory",
+    )  # fmt: skip
+    save(fig, "tip_13b_utf8_query.png", rect=(0, 0, 1, 0.95))
 
     # (c) Bytes on disk, column and column + FULL index.
     fig, ax = plt.subplots(1, 1, figsize=(6.6, 3.6))

doc/guides/optim_tips/tip_13a_utf8_read.png

@@ -384,12 +384,14 @@ The measurements below use a 1 Mrow table of free text averaging 76 bytes per ro
 
 ![Full column read: utf8() vs string()](optim_tips/tip_13a_utf8_read.png)
 
-The time gap is real but modest — decompression dominates, and both flavours decompress about the same payload. The memory gap is the important one: the fixed-width array is *rows × 800 B* whatever the text actually weighs, so it does not depend on the data at all, while the `utf8()` array pays for the bytes that are there. How often titles repeat makes no difference either — the padding is charged per row, not per different value. And it scales linearly: the same column at 100 Mrows would need 80 GB of RAM to be read whole as `string(200)`, against roughly a tenth of that as `utf8()`.
+The time gap is real but modest — decompression dominates, and both flavours decompress about the same payload. The memory gap is the important one: the fixed-width array is *rows × 800 B* whatever the text actually weighs, so it does not depend on the data at all, while the `utf8()` array pays for the bytes that are there. How often titles repeat makes no difference either — the padding is charged per row, not per different value. And it scales linearly: the same column at 100 Mrows would need 80 GB of RAM to be read whole as `string(200)`, against roughly a quarter of that as `utf8()`.
 
 Anything that materializes the column benefits from this — a NumPy comparison, {meth}`to_pandas() <blosc2.CTable.to_pandas>`, a plot. UTF-8 is also the ecosystem's common currency: a `utf8()` column *is* int64 offsets plus a UTF-8 blob — Arrow's `large_string` layout — so {meth}`to_arrow() <blosc2.CTable.to_arrow>` builds straight from the stored buffers, and pandas, Polars and DuckDB take it from there. Fixed width has to transcode UCS-4 on the way out. See {ref}`utf8 and NumPy's StringDType <Utf8AndStringDType>`.
 
 ### Querying columns, with and without a FULL index
 
+Reading a column whole is one thing; finding values on it is another. {meth}`where() <blosc2.CTable.where>` never materializes the column — it scans chunk by chunk — so the memory blow-up above does not happen here at all. A FULL index replaces that scan with a direct lookup, on either flavour.
+
 ```python
 t.where("title == 'some exact title'")  # scans, one chunk at a time
 t.create_index("title", kind=blosc2.IndexKind.FULL)
@@ -398,9 +400,9 @@ t.where("title == 'some exact title'")  # looks it up, no scan
 
 ![Equality lookup and index build: utf8() vs string()](optim_tips/tip_13b_utf8_query.png)
 
-{meth}`where() <blosc2.CTable.where>` never materializes the column: it scans chunk by chunk, so the memory blow-up above simply does not happen on either flavour, and `utf8()`'s edge is just fewer bytes to decompress and compare.
+A `utf8()` column is indexed by *alphabetical rank*: the query literal is located by bisecting the index's vocabulary, and the rows that match are a contiguous run of the sorted-positions sidecar. None of that depends on how many different values the column holds, so the index is worth having at either cardinality — a scan costs tens of milliseconds, a lookup a few. The first lookup of a session is the dearer one only because it opens the sidecars; later ones reuse them.
 
-A FULL index turns that scan into a direct lookup — but on a `utf8()` column it only pays off while the text repeats. The index sorts the *different* values alphabetically and stores each row's position in that sorted list, so the more different values there are, the bigger that list gets and the more work the lookup does. When titles repeat, the index is a clear win, and it costs a fraction of what the fixed-width one costs to build. When almost every title is different, the lookup ends up *slower than no index at all*, while `string(200)` — whose index reads raw values straight out of a known slot — keeps the same lookup time either way. So index a `utf8()` column when its text repeats, and leave wide-open free text unindexed.
+`string(200)` answers just as directly, from the values themselves. `utf8()` is the faster of the two — comparing ranks is integer work — and by far the cheaper to build: ~4.6x faster when titles repeat, 1.3x when they do not, and 2.4 GiB of peak memory for the fixed-width build whatever the data, against 200 MiB / 1.2 GiB.
 
 Caveat emptor: that sorted list is built once, so adding rows leaves it out of date: blosc2 falls back to a scan (correct results, no speedup) until you call {meth}`rebuild_index() <blosc2.CTable.rebuild_index>`. Also, note how no index accelerates `startswith` or substring search, on any flavor.