Skip to content

Commit ce53c9f

Browse files
jeremymanningclaude
andcommitted
Enable the swap control, and document the whole transform
The Lexicon Lab's `mint` control was rendered disabled: `LexiconLab` never passed `mint` into `vacParams` at all, so it could not have worked. It works now, and the interesting part is how its one real constraint is presented. `mint = "swap"` is injective only at p in {0, 1} — a theorem (contract §5.2a), not a rough edge: a map stable in p whose images are domain types is injective everywhere only if it is the identity. So the UI never clamps p, never falls back to nonce, and never catches-and-replaces the typed error. `LexiconLab` asks the engine and carries the refusal up as a string; the panel prints it verbatim with the four exits offered as buttons, and with no vocabulary the counters, the trainer and the invariance check have nothing to report — which is the honest state of that configuration. Beside the mint control the theorem is COUNTED rather than asserted: every domain type is pushed through the real transform at the current p and the distinct images counted. Measured on the shipped corpus at seed 0, swap loses 244 / 322 / 233 image slots at p = .25/.5/.75 and 0 at both endpoints; nonce loses 0 everywhere. The `bijective` chip now reads "injective at p = 0, 1" under swap instead of a bare tick. Info tab gains #vacancy: the T4 2x2 with the vacancy cell marked, the transform's definition, nesting and stability and the four properties the source implementation claims and breaks, the invariance theorem with §7.4's framing (the exact zero IS the finding), the swap decomposition with "cost of unknown form" stated as an upper bound, the stress table's real status (61 rule-seeded entries, never human-checked, 5.1% of this corpus's tokens), and what the static build refuses by name — plus the WebGPU/CI coverage gap. #real, #limits and #refs updated. Every number in that prose is pinned by five new tests in docs.spec.ts: the counts come from a live POST /api/lex/vacancy, the swap collisions and the stress-table size are read off the running panel, and the static build's ±0.2 nats and 700-token floor are read out of the constants that enforce them. No number from the source document appears anywhere; its prosody figures are its own, on a corpus we do not have. Also fixes a stale comment in staticClient/arch.ts that still documented a ±0.1 nats quantization uncertainty its own constant had superseded with 0.2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 1d09552 commit ce53c9f

10 files changed

Lines changed: 882 additions & 58 deletions

File tree

code/frontend/src/lib/staticClient/arch.ts

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -721,7 +721,11 @@ export class ArchSection {
721721
* is 14–23 % pooled with sign flips, so quantization eats exactly the contrast that
722722
* makes the result mean something;
723723
* - pooled `swap − english` and `nonce − english` are reported, with the measured
724-
* ±0.1 nats quantization uncertainty stated beside the sampling standard error.
724+
* `VACANCY_Q8_UNCERTAINTY_NATS` quantization uncertainty stated beside the sampling
725+
* standard error. (This line said "±0.1" until that constant was re-derived on the
726+
* configuration that actually ships and came out at 0.2 — see its own comment. A
727+
* number written twice is a number that drifts, so it is named here rather than
728+
* retyped.)
725729
*
726730
* A dtype with no measured bound is refused outright rather than given a ± copied from
727731
* a different dtype: a stated error bar that was never measured is a fabrication, and

code/frontend/src/viz/info/InfoTab.svelte

Lines changed: 334 additions & 5 deletions
Large diffs are not rendered by default.

code/frontend/src/viz/lex/LexiconLab.svelte

Lines changed: 83 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -64,9 +64,11 @@
6464
import {
6565
buildVacancyMap,
6666
mapVocabWords,
67+
typeCounts,
6768
vacancyDomain,
6869
vacancyParams,
6970
vacateText,
71+
type MintStrategy,
7072
type VacancyMap,
7173
type VacancyParams,
7274
} from "../../lib/lexEngine/vacancy";
@@ -134,7 +136,7 @@
134136
let vacCondition = $state<VacancyCondition>("consistent");
135137
let vacRevealAfter = $state(1);
136138
let vacProsody = $state(true);
137-
let vacMint = $state("nonce");
139+
let vacMint = $state<MintStrategy>("nonce");
138140
139141
/**
140142
* What a training run produced, or null while nothing has been trained at the CURRENT
@@ -205,25 +207,50 @@
205207
consistent: vacCondition !== "inconsistent",
206208
matchProsody: vacProsody,
207209
revealAfter: vacCondition === "reveal" ? vacRevealAfter : 0,
210+
mint: vacMint,
208211
}),
209212
);
210213
211214
/**
212-
* The nonce assignment, built ONCE over the domain (corpus types ∪ the full Dolch list)
213-
* in canonical order — contract §5.2. It reads `vacSeed` and `vacProsody` and
214-
* deliberately NOT `vacP`: the map is `p`-independent, which is what makes a stem's
215-
* nonce the same string at every `p` where it is vacated, and building it inside a
216-
* derived that also read `p` would silently re-mint the whole corpus on every tick of
217-
* the slider — visibly breaking the stability the panel exists to demonstrate.
215+
* The replacement assignment, built ONCE over the domain (corpus types ∪ the full Dolch
216+
* list) in canonical order — contract §5.2. It reads `vacSeed`, `vacProsody` and the
217+
* minting strategy, and deliberately NOT `vacP`: the map is `p`-independent, which is what
218+
* makes a stem's replacement the same string at every `p` where it is vacated, and building
219+
* it inside a derived that also read `p` would silently re-mint the whole corpus on every
220+
* tick of the slider — visibly breaking the stability the panel exists to demonstrate.
221+
*
222+
* `consistent` IS passed, even though the map itself does not depend on it, because the
223+
* engine refuses `mint = "swap"` under the inconsistent control (contract §8.3: 1 680
224+
* open-class stems against 8 202 vacated tokens, so there is no supply of real words to
225+
* mint a fresh one per occurrence). Deciding that here would be re-deriving a rule the
226+
* engine owns; asking the engine and CARRYING its refusal is the honest form. Nothing is
227+
* substituted for a refused map — `vacRefusal` renders the engine's own sentence and the
228+
* panels below have no vocabulary, which is exactly the state the reader is in.
218229
*/
219-
const vacMap = $derived.by<VacancyMap | null>(() =>
220-
corpus
221-
? buildVacancyMap(
222-
vacancyDomain(tokenize(corpus.text)),
223-
vacancyParams({ seed: vacSeed, matchProsody: vacProsody }),
224-
)
225-
: null,
226-
);
230+
const vacBuild = $derived.by<{ map: VacancyMap | null; refusal: string }>(() => {
231+
if (!corpus) return { map: null, refusal: "" };
232+
const tokens = tokenize(corpus.text);
233+
try {
234+
return {
235+
map: buildVacancyMap(
236+
vacancyDomain(tokens),
237+
vacancyParams({
238+
seed: vacSeed,
239+
matchProsody: vacProsody,
240+
mint: vacMint,
241+
consistent: vacParams.consistent,
242+
}),
243+
// The swap control ranks its replacement pool by corpus frequency, so it needs the
244+
// TOKEN STREAM's counts; `nonce` never looks at them.
245+
vacMint === "swap" ? typeCounts(tokens) : undefined,
246+
),
247+
refusal: "",
248+
};
249+
} catch (e) {
250+
return { map: null, refusal: e instanceof Error ? e.message : String(e) };
251+
}
252+
});
253+
const vacMap = $derived(vacBuild.map);
227254
228255
/** The corpus every panel below measures, trains on and generates from. */
229256
const vacatedText = $derived.by(() =>
@@ -251,26 +278,45 @@
251278
* tab's normal rule. Coverage then collapses — and the collapse is the measurement, not
252279
* a failure to be papered over.
253280
*/
254-
const vocab = $derived.by<LexVocab | null>(() => {
255-
if (!corpus || !baseVocab) return null;
256-
if (!vacated) return baseVocab;
281+
const vocabResult = $derived.by<{ vocab: LexVocab | null; refusal: string }>(() => {
282+
if (vacBuild.refusal) return { vocab: null, refusal: vacBuild.refusal };
283+
if (!corpus || !baseVocab) return { vocab: null, refusal: "" };
284+
if (!vacated) return { vocab: baseVocab, refusal: "" };
257285
if (vacMapped && vacMap) {
258-
return new LexVocab(
259-
mapVocabWords(baseVocab.words, vacMap, vacParams),
260-
baseVocab.source,
261-
baseVocab.budgetName,
262-
);
286+
try {
287+
return {
288+
vocab: new LexVocab(
289+
mapVocabWords(baseVocab.words, vacMap, vacParams),
290+
baseVocab.source,
291+
baseVocab.budgetName,
292+
),
293+
refusal: "",
294+
};
295+
} catch (e) {
296+
// Contract §5.2a, and the one refusal a reader will actually meet: `mint = "swap"`
297+
// draws its replacements FROM the domain, so at an intermediate `p` a vacated type
298+
// can land on one that has not moved and two budget words would share an embedding
299+
// row. The theorem there proves no `p`-stable swap avoids it, so this is not a defect
300+
// to re-draw away. The engine's sentence is carried up verbatim and NOTHING is put in
301+
// its place: rebuilding the budget from the vacated corpus here would be a different
302+
// measurement (§7.2's rebuilt rule belongs to the control conditions) wearing the
303+
// mapped condition's label.
304+
return { vocab: null, refusal: e instanceof Error ? e.message : String(e) };
305+
}
263306
}
264-
return buildVocab(budgetSource, budgetName, vacatedText);
307+
return { vocab: buildVocab(budgetSource, budgetName, vacatedText), refusal: "" };
265308
});
309+
const vocab = $derived(vocabResult.vocab);
310+
/** The engine's own words for a configuration it declines, or `""`. Never paraphrased. */
311+
const vacRefusal = $derived(vocabResult.refusal);
266312
const coverage = $derived.by<Coverage | null>(() =>
267313
vocab && corpus ? vocab.coverage(vacatedText) : null,
268314
);
269315
/** What the trainer, the sampler and the geometry are describing. */
270316
const activeCorpusLabel = $derived(
271317
corpus
272318
? vacated
273-
? `${corpus.label} · vacated p=${vacP.toFixed(2)}, seed ${vacSeed}, ${vacCondition}`
319+
? `${corpus.label} · vacated p=${vacP.toFixed(2)}, seed ${vacSeed}, ${vacCondition}, ${vacMint}`
274320
: corpus.label
275321
: "",
276322
);
@@ -545,10 +591,16 @@
545591
one matrix counted twice.
546592
</li>
547593
<li>
548-
<b>Not shipped:</b> nonce-word minting (it needs a parameter-matched control that
549-
does not exist), and the meter/rhyme "fingerprint" (the source's meter score does
550-
not measure meter — under its scheme every word's stress pattern begins the same
551-
way, so the score converges to the template's own density whatever you feed it).
594+
<b>Shipped, corrected:</b> nonce-word minting — the vacancy panel above. Feature 006
595+
deferred it for want of a parameter-matched control; the control turned out to be the
596+
design, since under the mapped condition the transform preserves the vocabulary
597+
exactly. Four properties the source claims for itself are broken by its own
598+
implementation and fixed here (its map is built lazily while rewriting, so a nonce
599+
depends on <code>p</code>; its give-up path and its seam fix are order-dependent; and
600+
injectivity is assumed rather than verified).
601+
<b>Not shipped:</b> the meter/rhyme "fingerprint" (the source's meter score does not
602+
measure meter — under its scheme every word's stress pattern begins the same way, so
603+
the score converges to the template's own density whatever you feed it).
552604
</li>
553605
<li>
554606
Browser and Python run the <b>same recipe</b>, held to ≤1e-5 on the forward pass,
@@ -623,12 +675,13 @@
623675
condition={vacCondition}
624676
revealAfter={vacRevealAfter}
625677
mint={vacMint}
678+
refusal={vacRefusal}
626679
onP={(v) => (vacP = v)}
627680
onSeed={(v) => (vacSeed = v)}
628681
onCondition={(c) => (vacCondition = c as VacancyCondition)}
629682
onRevealAfter={(v) => (vacRevealAfter = v)}
630683
onProsody={(v) => (vacProsody = v)}
631-
onMint={(m) => (vacMint = m)}
684+
onMint={(m) => (vacMint = m as MintStrategy)}
632685
/>
633686
</div>
634687

0 commit comments

Comments
 (0)