Skip to content

feat(news): completeness measurement — feed health w/ silent zeros, coverage ledger + provenance, GDELT recall benchmark (#4920)#4927

Merged
koala73 merged 8 commits into
mainfrom
feat/completeness-4920
Jul 6, 2026
Merged

feat(news): completeness measurement — feed health w/ silent zeros, coverage ledger + provenance, GDELT recall benchmark (#4920)#4927
koala73 merged 8 commits into
mainfrom
feat/completeness-4920

Conversation

@koala73

@koala73 koala73 commented Jul 5, 2026

Copy link
Copy Markdown
Owner

Closes #4920 (Bet 2 of the strategic news-pipeline review). Do not auto-merge — held for review. ⚠️ Stacked on #4924 (the recall benchmark matches titles via scripts/shared/story-identity.js); merge #4924 first — this PR's diff will shrink to its own commits after that.

Three layers of "did we miss something?"

(a) Feed health + silent zerosvalidate-rss-feeds.mjs now validates BOTH feed universes (client config and the server digest catalog _feeds.ts — the list the digest actually ingests, which was previously unmeasured; 276 unique server URLs extracted with faithful gn() URL reconstruction). With Upstash secrets in the daily GH Actions run it publishes news:feed-health:v1 with per-feed status + consecutive-empty streaks; Google News wrappers empty across ≥2 consecutive daily runs are flagged as silent zeros — the "Google ranking shift quietly blanked a source" failure mode.

(b) Coverage ledger + provenance — every silent drop gate now counts: per-feed cap (previously fully invisible), per-category cap, freshness floor → news:coverage-ledger:v1:{variant}:{lang} per digest build; insights selection gates (admissibility/source-cap/overflow) via a backward-compatible selectTopStories stats param. The World Brief payload carries provenance and the panel renders "Compiled from N stories across M sources" (all locales fanned out via translate-locales; {{token}} convention).

(c) External recall benchmark — daily, same workflow, zero Railway slots: GDELT top articles across 4 broad verticals matched against ingested digest titles using the #4919 shared similarity → news:recall-benchmark:v1 with recall % and the missed headlines sorted by lowest similarity (actionable coverage holes, not just a number).

Both keys registered in api/seed-health.js + api/health.js (age-above-coverage precedence respected).

Operator action

Add UPSTASH_REDIS_REST_URL / UPSTASH_REDIS_REST_TOKEN as GitHub Actions secrets to activate publishing — without them both publishers log a skip and the workflow behaves exactly as before.

Verification

  • tests/completeness-measurement.test.mjs (14): silent-zero streak/reset/wrapper-only semantics, recall math incl. edit-variant matching + null-not-NaN, selection drop stats, server-catalog extraction (250+ feeds), wiring assertions.
  • Full battery: clustering (16), server-handlers (24), story-identity (30), typecheck + typecheck:api + biome + docs-stats clean.

https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP

@mintlify

mintlify Bot commented Jul 5, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
WorldMonitor 🟢 Ready View Preview Jul 5, 2026, 8:02 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@vercel

vercel Bot commented Jul 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Jul 6, 2026 10:57am

Request Review

@greptile-apps

greptile-apps Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds three completeness-measurement layers to the news pipeline: (a) feed-health with silent-zero detection for Google News wrappers across consecutive empty runs, (b) a coverage ledger tracking per-feed/per-category/freshness-floor drop counts, and (c) a daily GDELT recall benchmark computing what fraction of top external stories the digest carries. It also consolidates story identity across the pipeline by replacing three divergent similarity implementations with a single dual-view lexical vector module.

  • Coverage-ledger implementation is missing from list-feed-digest.ts: the test "digest counts every drop gate and publishes the ledger" asserts four source-text patterns (droppedFeedCap, ledgerDrops.perCategoryCap, ledgerDrops.freshnessFloor, news:coverage-ledger:v1) that do not exist in the file \u2014 these four assertions will fail when the test suite runs.
  • Feed-health + GDELT recall benchmark are well-implemented pure modules with matching unit tests, and the GH Actions workflow correctly threads Upstash secrets to both publishers.
  • selectTopStories drop-stat accounting: overflowDropped is calculated as admissible.length - selected.length - sourceCapDropped, which misclassifies source-capped items that occur after the maxCount break as overflow; totals balance but per-bucket attribution can mislead.

Confidence Score: 3/5

The feed-health, GDELT recall, and story-identity unification work correctly and are well-tested, but the coverage-ledger feature described in the PR is entirely absent from list-feed-digest.ts, causing the test suite to fail on four source-text assertions.

The coverage-ledger instrumentation (droppedFeedCap, ledgerDrops, news:coverage-ledger:v1 publish) is called out in the PR description as delivered but is not present in list-feed-digest.ts or anywhere else in the diff. The test suite ships with four assertions that check for exact source-text patterns from that missing implementation; those assertions will all fail.

server/worldmonitor/news/v1/list-feed-digest.ts needs the coverage-ledger instrumentation before the test suite can pass. tests/completeness-measurement.test.mjs lines 153-158 will fail until that code lands.

Important Files Changed

Filename Overview
tests/completeness-measurement.test.mjs New 14-test suite for completeness measurement; the coverage-ledger describe block asserts source-text patterns in list-feed-digest.ts that do not exist — those 4 assertions will fail.
server/worldmonitor/news/v1/list-feed-digest.ts Switches from exact sha256 title hashing to fuzzy assignStoryIdentity for corroboration and story tracking; per-hash dedup guards prevent mentionCount inflation. Coverage-ledger instrumentation described in the PR is absent.
scripts/_clustering.mjs Removes local Jaccard clustering and delegates to shared story-identity; adds optional stats param to selectTopStories for drop counts with a minor overflowDropped attribution issue.
server/worldmonitor/news/v1/dedup.mjs Replaces word-overlap dedup with cosine similarity via shared story-identity; new assignStoryIdentity function clusters items and assigns publishedAt-anchored canonical hashes.
shared/story-identity.js New central story-identity module: dual-view FNV-1a hash lexical vectors, cosine similarity with containment rescue, union-find clustering; replaces three divergent similarity implementations.
scripts/_feed-health.mjs Pure feed-health payload builder: classifies Google News wrappers, maintains consecutive-empty streaks, surfaces silent-zero detection after SILENT_ZERO_THRESHOLD=2 runs.
scripts/_recall-benchmark-core.mjs Pure recall computation: matches external GDELT titles against digest titles using cosine similarity; unvectorizable titles excluded from denominator; null-not-NaN for empty sets.
scripts/seed-recall-benchmark.mjs New daily GH Actions script: fetches GDELT top articles across 4 verticals, computes recall against the digest, publishes to Redis. Exits 0 on all upstream failures.
scripts/validate-rss-feeds.mjs Extended to validate the server digest catalog in addition to the client config; adds publishFeedHealth post-run; module now importable by tests via the direct-run guard.
scripts/seed-insights.mjs Adds provenance object (storiesConsidered, sourcesConsidered, selectionDrops) to the insights payload via selectTopStories stats param; backward-compatible.
src/components/InsightsPanel.ts Adds renderProvenance() rendering the Compiled from N stories across M sources stamp; gracefully absent on pre-rollout payloads.
.github/workflows/feed-validation.yml Adds seed-recall-benchmark.mjs step and Upstash secrets env; timeout bumped from 15 to 25 min to accommodate the new GDELT fetch step.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[feed-validation GH Actions daily] --> B[validate-rss-feeds.mjs client + server catalog]
    A --> C[seed-recall-benchmark.mjs]
    B --> D{Upstash creds?}
    D -- yes --> E[publishFeedHealth buildFeedHealthPayload]
    E --> F[news:feed-health:v1 silent-zero streaks]
    D -- no --> G[skip silently]
    C --> H[fetchGdeltJson 4 verticals x 25 articles]
    H --> I[computeRecall cosineSimilarity vs digest]
    I --> J[news:recall-benchmark:v1 recall% + missed headlines]
    K[seed-insights.mjs selectTopStories stats] --> L[provenance storiesConsidered sourcesConsidered]
    L --> M[InsightsPanel Compiled from N stories across M sources]
    N[list-feed-digest.ts assignStoryIdentity] --> O[clusterTexts union-find fuzzy hash]
    O --> P[canonical titleHash per cluster]
    P --> Q[writeStoryTracking mentionCount once per hash]
    R[coverage-ledger news:coverage-ledger:v1] -.->|MISSING| N
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[feed-validation GH Actions daily] --> B[validate-rss-feeds.mjs client + server catalog]
    A --> C[seed-recall-benchmark.mjs]
    B --> D{Upstash creds?}
    D -- yes --> E[publishFeedHealth buildFeedHealthPayload]
    E --> F[news:feed-health:v1 silent-zero streaks]
    D -- no --> G[skip silently]
    C --> H[fetchGdeltJson 4 verticals x 25 articles]
    H --> I[computeRecall cosineSimilarity vs digest]
    I --> J[news:recall-benchmark:v1 recall% + missed headlines]
    K[seed-insights.mjs selectTopStories stats] --> L[provenance storiesConsidered sourcesConsidered]
    L --> M[InsightsPanel Compiled from N stories across M sources]
    N[list-feed-digest.ts assignStoryIdentity] --> O[clusterTexts union-find fuzzy hash]
    O --> P[canonical titleHash per cluster]
    P --> Q[writeStoryTracking mentionCount once per hash]
    R[coverage-ledger news:coverage-ledger:v1] -.->|MISSING| N
Loading

Reviews (1): Last reviewed commit: "feat(news): completeness measurement — f..." | Re-trigger Greptile

Comment on lines +153 to +158
const src = readSrc('server/worldmonitor/news/v1/list-feed-digest.ts');
assert.match(src, /droppedFeedCap = Math\.max\(0, matches\.length - ITEMS_PER_FEED\)/);
assert.match(src, /ledgerDrops\.perCategoryCap \+= Math\.max\(0, items\.length - MAX_ITEMS_PER_CATEGORY\)/);
assert.match(src, /ledgerDrops\.freshnessFloor = droppedStaleTotal/);
assert.match(src, /news:coverage-ledger:v1/);
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Coverage-ledger test asserts unimplemented code

The four assert.match calls inside this test check for patterns (droppedFeedCap, ledgerDrops.perCategoryCap, ledgerDrops.freshnessFloor, news:coverage-ledger:v1) that are not present anywhere in server/worldmonitor/news/v1/list-feed-digest.ts. Running node --test tests/completeness-measurement.test.mjs will fail immediately on all four assertions.

The PR description claims "(b) Coverage ledger + provenance — every silent drop gate now counts: per-feed cap, per-category cap, freshness floor → news:coverage-ledger:v1:{variant}:{lang} per digest build" but the list-feed-digest.ts diff in this PR contains none of the instrumentation or the Redis publish for that key. The ledger wiring in list-feed-digest.ts appears to be the missing implementation piece.

Comment thread scripts/_clustering.mjs
Comment on lines +349 to 351
stats.sourceCapDropped = sourceCapDropped;
stats.overflowDropped = Math.max(0, admissible.length - selected.length - sourceCapDropped);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 overflowDropped misclassifies source-capped items after the break

After selected.length >= maxCount triggers a break, the remaining items in admissible are never visited. The formula admissible.length - selected.length - sourceCapDropped counts ALL unvisited items as overflow — but any of those trailing items that would have hit the source cap are instead counted as overflow rather than source-capped. The grand total is always correct, but sourceCapDropped will be systematically under-counted and overflowDropped over-counted when the selection limit binds before the source-cap list is exhausted.

koala73 added a commit that referenced this pull request Jul 5, 2026
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round)

Applies the multi-agent + cross-model review findings:

- th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran
  before the {{token}} convention fix) — repaired; fa.json was missing
  the keys entirely AND absent from translate-locales' LOCALES list
  (pre-existing gap the parity test caught) — keys added, fa added to
  the translator roster
- extractServerFeeds skipped double-quoted names ("Tom's Hardware") —
  regex accepts both quote styles; extraction test pins it (cross-model)
- rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap
  and would undercount the coverage ledger for their whole TTL
  (cross-model)
- coverage-ledger writes clamped to known variants + well-formed langs
  (request-supplied values could inflate the keyspace — adversarial)
- GDELT titles AND urls bounded before publishing; urls http(s)-only
- direct-run guards use pathToFileURL (Windows/symlink-safe); both
  publishers share scripts/_upstash-rest.mjs instead of hand-rolled
  fetch helpers; publishFeedHealth exported with a no-creds contract
- .insights-provenance CSS added (was an unstyled div); gn()/gnLocale
  replica drift guard test; overflowDropped + orchestration-seam tests

Battery: completeness (21), locale-completeness (25), clustering (16),
story-identity (30), both typechecks, biome clean.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 5, 2026
@koala73

koala73 commented Jul 5, 2026

Copy link
Copy Markdown
Owner Author

Multi-agent code review — applied at f98e584

Four reviewer personas (correctness, adversarial+security, testing, maintainability+standards) plus the independent Codex cross-model pass. All actionable findings applied; held for manual merge (stacked on #4924).

Applied

Sev Finding Reviewers Fix
P1 th/tr/zh compiledFrom carried single-brace placeholders (fan-out predated the convention fix) — i18next would render them literally correctness(100) + maint(100) + codex braces repaired in all three
P1 fa.json missing the new keys — locale-parity test red; root cause: fa absent from translate-locales' roster (pre-existing) testing(100) Persian keys added; fa added to the translator LOCALES list
P1 extractServerFeeds skipped double-quoted names ("Tom's Hardware") codex(100) regex accepts both quote styles; extraction test pins it — server catalog now 277
P2 Warm rss:feed:v5 cache rows lack droppedFeedCap → ledger undercounts for the TTL codex + correctness probe parse-cache bumped v5→v6
P2 Request-supplied variant/lang could inflate news:coverage-ledger keyspace adversarial(75) writes clamped to known variants + ^[a-z]{2}$ langs
P2 GDELT titles/urls (untrusted) published unbounded adversarial + codex title 200ch, url 300ch + http(s)-only
P2 .insights-provenance rendered unstyled maint(100) CSS added beside the stats block
P2 Duplicated Upstash REST helpers; gn()/gnLocale replica had no drift guard maint(75) shared scripts/_upstash-rest.mjs; source-parity drift-guard test
P2/P3 Direct-run guards via string-built file:// (Windows/symlinks); untested orchestration seams; overflowDropped untested codex + testing pathToFileURL guards; publishFeedHealth/gdeltUrl/unwrapEnvelope exported + tested; overflow fixture test

Verified directly during review: npm run test:feeds:ci still executes main() after the import-guard change (679 merged feeds: 557 client + 277 server).

Noted, not changed

  • Source-textual wiring tests remain alongside (not instead of) behavioral ones — the pure logic (payload builder, recall math, selection stats) is behaviorally tested; the wiring assertions guard integration points that would need a full serverless harness.
  • overflow-vs-sourceCap attribution depends on gate ordering — semantics now pinned by the fixture test.

Battery: completeness (21), locale-completeness (25), clustering (16), story-identity (30), both typechecks, biome.

@koala73 koala73 force-pushed the feat/completeness-4920 branch from 35e09c9 to 31d7a04 Compare July 6, 2026 04:44
koala73 added a commit that referenced this pull request Jul 6, 2026
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round)

Applies the multi-agent + cross-model review findings:

- th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran
  before the {{token}} convention fix) — repaired; fa.json was missing
  the keys entirely AND absent from translate-locales' LOCALES list
  (pre-existing gap the parity test caught) — keys added, fa added to
  the translator roster
- extractServerFeeds skipped double-quoted names ("Tom's Hardware") —
  regex accepts both quote styles; extraction test pins it (cross-model)
- rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap
  and would undercount the coverage ledger for their whole TTL
  (cross-model)
- coverage-ledger writes clamped to known variants + well-formed langs
  (request-supplied values could inflate the keyspace — adversarial)
- GDELT titles AND urls bounded before publishing; urls http(s)-only
- direct-run guards use pathToFileURL (Windows/symlink-safe); both
  publishers share scripts/_upstash-rest.mjs instead of hand-rolled
  fetch helpers; publishFeedHealth exported with a no-creds contract
- .insights-provenance CSS added (was an unstyled div); gn()/gnLocale
  replica drift guard test; overflowDropped + orchestration-seam tests

Battery: completeness (21), locale-completeness (25), clustering (16),
story-identity (30), both typechecks, biome clean.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
koala73 added a commit that referenced this pull request Jul 6, 2026
…d fails (#4927 external review)

A transient Redis error during the previous-payload GET previously
reset every consecutive-empty streak (previous=null -> fresh streaks).
Read failure now skips the publish entirely; key-absent (first run)
still starts fresh. Regression test included.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

External review items attributed here via the stacked diff — both landed at 31d7a04: silent-zero streaks are preserved when the previous-state read fails (publish skipped, regression-tested), and the recall benchmark's fast-fail budget landed on #4929's branch. Still held for manual merge.

koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round)

Applies the multi-agent + cross-model review findings:

- th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran
  before the {{token}} convention fix) — repaired; fa.json was missing
  the keys entirely AND absent from translate-locales' LOCALES list
  (pre-existing gap the parity test caught) — keys added, fa added to
  the translator roster
- extractServerFeeds skipped double-quoted names ("Tom's Hardware") —
  regex accepts both quote styles; extraction test pins it (cross-model)
- rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap
  and would undercount the coverage ledger for their whole TTL
  (cross-model)
- coverage-ledger writes clamped to known variants + well-formed langs
  (request-supplied values could inflate the keyspace — adversarial)
- GDELT titles AND urls bounded before publishing; urls http(s)-only
- direct-run guards use pathToFileURL (Windows/symlink-safe); both
  publishers share scripts/_upstash-rest.mjs instead of hand-rolled
  fetch helpers; publishFeedHealth exported with a no-creds contract
- .insights-provenance CSS added (was an unstyled div); gn()/gnLocale
  replica drift guard test; overflowDropped + orchestration-seam tests

Battery: completeness (21), locale-completeness (25), clustering (16),
story-identity (30), both typechecks, biome clean.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
@koala73 koala73 force-pushed the feat/completeness-4920 branch from 31d7a04 to e471d51 Compare July 6, 2026 05:14
koala73 added a commit that referenced this pull request Jul 6, 2026
…d fails (#4927 external review)

A transient Redis error during the previous-payload GET previously
reset every consecutive-empty streak (previous=null -> fresh streaks).
Read failure now skips the publish entirely; key-absent (first run)
still starts fresh. Regression test included.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
… attribution, publish-after-guardrails (#4927 re-review)

All six re-review findings applied:

P1 — never-activated publishing no longer poisons health: the two keys
join ON_DEMAND_KEYS (missing data = soft EMPTY_ON_DEMAND) and their
seed-meta entries are activationGated in api/seed-health.js
('pending-activation', healthy) until the first publish lands — then
normal 2880-min staleness enforcement applies. The workflow also emits
a visible ::warning annotation on runs where the UPSTASH secrets are
absent, so green scheduled runs can't silently mask the gap.

P2 — the coverage-ledger write is awaited (.catch non-fatal): the Edge
response could previously finish before the fire-and-forget side write
landed.

P2 — selectTopStories classifies EVERY admissible candidate instead of
breaking at maxCount: cap-first attribution (a candidate whose source
already hit the per-source cap is a sourceCap drop even when selection
is full; only genuinely rankable candidates count as overflow), with a
same-source-overflow regression asserting exact one-of attribution.

P2/P3 — StoryMeta.source_count proto doc now describes what runtime
ships (current-cycle batch corroboration, default 1) instead of
'cached from Redis Set'; generated artifacts regenerated.

P3 — _upstash-rest.mjs sends a User-Agent (CF-WAF convention: never a
bare fetch UA); feed-health publishing moved AFTER the config-drift
hard-fail so a guardrail-failing run cannot refresh health metadata
first.

Battery: completeness 23, clustering 16, mcp-parity 11, seed-health 2,
typecheck:api, docs-stats.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Re-review adjudication — all 6 findings validated, fixed at d1c6298

Sev Finding Fix
P1 Missing secrets → green workflow + missing/stale monitored health keys Activation-gating on both surfaces: the two keys join ON_DEMAND_KEYS in api/health.js (missing data reads soft EMPTY_ON_DEMAND, not EMPTY), and their seed-meta entries carry activationGated: true in api/seed-health.js (pending-activation, healthy, until the first publish — then the 2880-min staleness rule enforces normally). The workflow additionally emits a visible ::warning annotation when the secrets are absent, so a green scheduled run can never silently mask the gap.
P2 Coverage-ledger write fire-and-forget on Edge Awaited with non-fatal .catch, matching the story-tracking write's pattern.
P2 Overflow/source-cap attribution breaks at maxCount Loop classifies every admissible candidate, cap-first: a candidate whose source already hit the per-source cap is a sourceCap drop even when selection is full; only genuinely rankable candidates count as overflow. Same-source-overflow regression asserts exact one-of attribution across all four buckets.
P2/P3 StoryMeta.source_count doc says "cached from Redis Set" Proto comment now describes runtime truth (current-cycle batch corroboration, default 1; the Redis set feeds importance scoring, not this field); generated artifacts regenerated via make generate.
P3 _upstash-rest.mjs missing User-Agent worldmonitor-ops/1.0 UA added (repo CF-WAF convention).
P3 Feed-health published before the config-drift hard-fail Publish moved after the guardrail exit so a drift-failing run cannot refresh health metadata to fresh/OK first; ordering rationale documented in-code.

Battery: completeness 23, clustering 16, mcp-parity 11, seed-health 2, typecheck:api, docs-stats. Cascade-rebased through #4928#4929. Still held for manual merge.

koala73 added a commit that referenced this pull request Jul 6, 2026
…re FIRST publish (#4927 re-review P1)

Activation was inferred from seed-meta existence, which carries a 7d
TTL: a publisher that ran once and died alarmed as stale only until the
meta expired, then reverted to healthy pending-activation — violating
the stated contract, and /api/health softened the keys permanently via
ON_DEMAND_KEYS.

"Has ever published" is now a durable marker (seed-activated:news:*,
SET with NO TTL on every successful publish):
- api/health.js pipelines EXISTS on the markers; a marker-bearing key
  leaves ON_DEMAND softening permanently — missing data/meta reads
  EMPTY/STALE_SEED like any other monitored key (marker read errors
  degrade to not-activated, never to a false alarm)
- api/seed-health.js entries carry activationKey; 'pending-activation'
  is returned ONLY while the marker is absent — with the marker
  present, missing meta falls through to 'missing' (alarming)
- both publishers SET their marker alongside every publish

Regressions: classifyKey lifecycle (pending -> EMPTY_ON_DEMAND;
activated-then-dead -> EMPTY) + publisher/marker wiring pins including
the no-TTL assertion.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Re-review round 2 — the remaining P1 fixed at $(git rev-parse --short feat/completeness-4920)

Validated in full. Activation was inferred from seed-meta existence (7-day TTL), so "has ever published" evaporated with the meta: a publisher that ran once and died alarmed only until expiry, then reverted to healthy pending-activation — and /api/health softened the keys permanently via ON_DEMAND_KEYS. Exactly as you framed it: the stated contract was violated on both surfaces.

Fix (durable marker, as suggested): publishers SET seed-activated:news:{feed-health,recall-benchmark} 1 with no TTL alongside every publish.

  • api/health.js pipelines EXISTS on the markers; a marker-bearing key leaves ON_DEMAND softening permanently — missing data/meta then reads EMPTY/STALE_SEED like any monitored key. Marker read errors degrade to not-activated (soft), never to a false alarm.
  • api/seed-health.js entries carry activationKey; pending-activation is returned only while the marker is absent — with it present, missing meta falls through to missing (alarming).

Regressions: classifyKey lifecycle (never-activated → EMPTY_ON_DEMAND; activated-then-dead → EMPTY) plus wiring pins on both publishers including a no-TTL assertion on the marker SET.

Batteries: completeness 25, seed-health 2, mcp-parity 11, full tiered pre-push (both typechecks, seed tests) green. Cascade-rebased through #4928#4929. Held for manual merge.

koala73 added a commit that referenced this pull request Jul 6, 2026
…marker probe (#4927)

The harnesses assert every pipeline command is GET; the activation
markers add EXISTS probes on seed-activated:* keys, which tripped the
mock assertion and 503'd every request in these suites. Both mocks now
accept EXISTS (marker-namespace-pinned, absent by default).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added 8 commits July 6, 2026 14:53
…etection, coverage ledger + provenance, GDELT recall benchmark (#4920)

Turns "unmistakably complete" from a feeling into numbers, at three
layers:

(a) Feed health. validate-rss-feeds now also validates the SERVER digest
catalog (_feeds.ts — the list buildDigest actually ingests; previously
only the client config was measured) and, when Upstash secrets are
present, publishes per-feed status + consecutive-empty streaks to
news:feed-health:v1 daily. Google News wrappers that deliver nothing
across consecutive runs are flagged as SILENT ZEROS — the failure mode
where Google's ranking shift quietly blanks a source.

(b) Coverage ledger + provenance. Every silent drop gate now counts:
per-feed cap, per-category cap, freshness floor (digest →
news:coverage-ledger:v1:{variant}:{lang}), and the insights selection
gates (admissibility / source cap / overflow via selectTopStories
stats). The World Brief payload carries provenance and InsightsPanel
renders the stamp: "Compiled from N stories across M sources"
(i18n fanned out to all locales).

(c) External recall benchmark. Daily job (same GH Actions workflow, no
Railway slot) pulls GDELT's top articles across four broad verticals,
matches each against ingested digest titles using the shared
story-identity similarity (#4919), and publishes recall % + the missed
headlines to news:recall-benchmark:v1 — the first ground-truth answer
to "did we miss a story?", with misses actionable by lowest similarity.

Both published keys are registered in api/seed-health.js and
api/health.js (daily cadence, 2-day alarm). Publishers skip silently
without credentials, so local/PR runs are unchanged.

Operator action to activate publishing: add UPSTASH_REDIS_REST_URL and
UPSTASH_REDIS_REST_TOKEN as GitHub Actions secrets.

Closes #4920

🤖 Generated with Claude Code
Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round)

Applies the multi-agent + cross-model review findings:

- th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran
  before the {{token}} convention fix) — repaired; fa.json was missing
  the keys entirely AND absent from translate-locales' LOCALES list
  (pre-existing gap the parity test caught) — keys added, fa added to
  the translator roster
- extractServerFeeds skipped double-quoted names ("Tom's Hardware") —
  regex accepts both quote styles; extraction test pins it (cross-model)
- rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap
  and would undercount the coverage ledger for their whole TTL
  (cross-model)
- coverage-ledger writes clamped to known variants + well-formed langs
  (request-supplied values could inflate the keyspace — adversarial)
- GDELT titles AND urls bounded before publishing; urls http(s)-only
- direct-run guards use pathToFileURL (Windows/symlink-safe); both
  publishers share scripts/_upstash-rest.mjs instead of hand-rolled
  fetch helpers; publishFeedHealth exported with a no-creds contract
- .insights-provenance CSS added (was an unstyled div); gn()/gnLocale
  replica drift guard test; overflowDropped + orchestration-seam tests

Battery: completeness (21), locale-completeness (25), clustering (16),
story-identity (30), both typechecks, biome clean.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…trap-hydrated), MCP exclusions, rss:feed v6 pin update (#4920)

The stacked CI runs caught three registry gates the local scoped battery
missed: (1) the new ops keys had landed in BOOTSTRAP_KEYS, which would
hydrate feed-health/recall payloads into every page load — moved to
STANDALONE_KEYS (health-monitored, on-demand); (2) both keys needed
documented EXCLUDED_FROM_MCP entries (ops surfaces, not queryable news
slices); (3) the rss:feed cache-prefix pin test still asserted v5 —
updated for the #4920 v6 bump with the invalidation rationale.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…d fails (#4927 external review)

A transient Redis error during the previous-payload GET previously
reset every consecutive-empty streak (previous=null -> fresh streaks).
Read failure now skips the publish entirely; key-absent (first run)
still starts fresh. Regression test included.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
… attribution, publish-after-guardrails (#4927 re-review)

All six re-review findings applied:

P1 — never-activated publishing no longer poisons health: the two keys
join ON_DEMAND_KEYS (missing data = soft EMPTY_ON_DEMAND) and their
seed-meta entries are activationGated in api/seed-health.js
('pending-activation', healthy) until the first publish lands — then
normal 2880-min staleness enforcement applies. The workflow also emits
a visible ::warning annotation on runs where the UPSTASH secrets are
absent, so green scheduled runs can't silently mask the gap.

P2 — the coverage-ledger write is awaited (.catch non-fatal): the Edge
response could previously finish before the fire-and-forget side write
landed.

P2 — selectTopStories classifies EVERY admissible candidate instead of
breaking at maxCount: cap-first attribution (a candidate whose source
already hit the per-source cap is a sourceCap drop even when selection
is full; only genuinely rankable candidates count as overflow), with a
same-source-overflow regression asserting exact one-of attribution.

P2/P3 — StoryMeta.source_count proto doc now describes what runtime
ships (current-cycle batch corroboration, default 1) instead of
'cached from Redis Set'; generated artifacts regenerated.

P3 — _upstash-rest.mjs sends a User-Agent (CF-WAF convention: never a
bare fetch UA); feed-health publishing moved AFTER the config-drift
hard-fail so a guardrail-failing run cannot refresh health metadata
first.

Battery: completeness 23, clustering 16, mcp-parity 11, seed-health 2,
typecheck:api, docs-stats.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…re FIRST publish (#4927 re-review P1)

Activation was inferred from seed-meta existence, which carries a 7d
TTL: a publisher that ran once and died alarmed as stale only until the
meta expired, then reverted to healthy pending-activation — violating
the stated contract, and /api/health softened the keys permanently via
ON_DEMAND_KEYS.

"Has ever published" is now a durable marker (seed-activated:news:*,
SET with NO TTL on every successful publish):
- api/health.js pipelines EXISTS on the markers; a marker-bearing key
  leaves ON_DEMAND softening permanently — missing data/meta reads
  EMPTY/STALE_SEED like any other monitored key (marker read errors
  degrade to not-activated, never to a false alarm)
- api/seed-health.js entries carry activationKey; 'pending-activation'
  is returned ONLY while the marker is absent — with the marker
  present, missing meta falls through to 'missing' (alarming)
- both publishers SET their marker alongside every publish

Regressions: classifyKey lifecycle (pending -> EMPTY_ON_DEMAND;
activated-then-dead -> EMPTY) + publisher/marker wiring pins including
the no-TTL assertion.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…marker probe (#4927)

The harnesses assert every pipeline command is GET; the activation
markers add EXISTS probes on seed-activated:* keys, which tripped the
mock assertion and 503'd every request in these suites. Both mocks now
accept EXISTS (marker-namespace-pinned, absent by default).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
@koala73 koala73 force-pushed the feat/completeness-4920 branch from fc059a3 to e6dd8ed Compare July 6, 2026 10:54
@koala73 koala73 merged commit 5cf7b0f into main Jul 6, 2026
25 checks passed
koala73 added a commit that referenced this pull request Jul 6, 2026
…laced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
…nce-demotion seam (#4922) (#4929)

* feat(markets): wire markets into the news spine — macro-print actuals, earnings in the daily brief, finance-demotion seam (#4922)

The no-proto slice of the markets↔news epic (ticker tagging on the wire
and watchlist story alerts need a NewsItem proto field — tracked in
#4922 as the follow-up slice):

(b) Macro actuals. The economic calendar shipped every event with
actual/estimate/previous empty forever — the print itself was never
captured. The seeder now pulls FRED series observations (release ids
and series ids are different namespaces; mapped per event with the
right transform: CPI/PCE/Retail index→MoM %, NFP level→ΔK, GDP
headline % direct), publishes a recentPrints block, and fills
actual/previous on print-day events. The proto already carries the
fields — values flow to the client with zero schema change.

(c) Earnings reach a brief for the first time. Daily Market Brief
gains earningsContext (recent beats/misses + upcoming count) collected
via the earnings-calendar RPC with an 8s budget and rendered into the
LLM context — symbols + direction only, stable per reporting date, so
the #4914 summary-cache identity does not churn.

(f) Finance-demotion seam. scoreImportance/selectTopStories accept
{demoteFinance:false} — the ×0.35 demotion stays default for the
geopolitical World Brief and is now switchable for a finance-focused
ranking surface. Honest note: the only live consumer runs the full
variant today; this is the seam a finance-variant insights run plugs
into.

Tests: print transforms (incl. FRED '.' markers), fill semantics
(no-overwrite, future-stays-empty), demotion ×0.35 default vs neutral,
earnings prompt block, wiring. 85 tests across the affected battery.

Refs #4922 (remaining slices tracked there)

🤖 Generated with Claude Code
Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP

* fix(review): markets wiring hardening — deliverable earnings lookback, print adjacency guards, unitless values, pure transform (#4929 review round)

Applies the multi-agent + cross-model review findings:

- the "recent beats/misses" block was undeliverable as wired: the
  earnings seed window started at today, so past reporters never
  existed in the payload (correctness + codex, cross-corroborated) —
  seed window widened to today-7d
- computePrintValues hardened per codex P1: a '.' LEADING observation
  no longer falls back to an older row presented as current, and
  pct_mom/diff_k require date-adjacent periods (45-day tolerance) so a
  mid-series gap cannot mislabel a 2-month change as month-over-month
- print values are now UNITLESS — event.unit already renders '%'/'K'
  and embedding it double-rendered in the calendar (codex conf-100)
- recentPrints consumer scope documented in-code: the RPC reconstructs
  its proto response without it (Redis/ops-side until the proto gains a
  field — epic follow-up); the user-visible win flows through the
  event actual/previous fields the proto already carries
- earnings transform extracted to pure buildEarningsBriefContext
  (this PR's own standard, per review) with direct tests; earnings
  prompt block covered by a capture test asserting content AND
  cache-identity stability across rebuilds; selectTopStories opts
  pass-through exercised end-to-end; EVENT_SERIES↔FRED_RELEASES name
  lockstep pinned by test
- verified: fredFetchJson error paths never embed the keyed URL

45 tests across the affected battery; typecheck + biome clean.

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP

* fix(review): release-day stale-print guard, benchmark fast-fail, misplaced-opts shift (#4929 external review)

- observationMatchesRelease: a filled actual must belong to the period
  the release reports (monthly diff==1; quarterly 1..5 for the
  advance/second/third GDP estimates) — pre-print on release day the
  latest FRED observation is the PRIOR period and was presented as
  today's number (P1); regression pins the July-15-CPI/May-obs case
- recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry,
  1 proxy attempt) under a shared 4-minute deadline — production retry
  defaults across 4 sequential queries could eat the 25-min workflow
- selectTopStories auto-shifts an opts-shaped object passed in the
  stats slot (positional-API hazard), tested

Also validated-not-changed: cross-cycle canonical adoption is the
documented #4925 follow-up; seed-meta recordCount=ok is intended
job-health semantics (streak preservation landed on the #4927 branch).

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
koala73 added a commit that referenced this pull request Jul 6, 2026
… — unblock feed-health activation (#4970) (#4971)

Three server-catalog feeds (Pentagon, The National, Truth Social) had
hosts absent from the RSS allowlist, tripping the CONFIG_DRIFT hard-fail
in validate-rss-feeds.mjs --ci (exit 1) which — per the #4927 ordering —
blocks feed-health/recall-benchmark publishing before it runs. Pre-existing
drift (DoD rebranded defense.gov→war.gov; the other two were added to the
catalog without allowlisting), surfaced by activating the publish path.

Adds the three hosts to all mirrors: shared/rss-allowed-domains.json
(source of truth) + its byte-identical scripts/shared mirror + the
api/_rss-allowed-domains.js copy the validator and Edge SSRF guard
actually read + vite.config.ts RSS_PROXY_ALLOWED_DOMAINS (dev proxy).
The .cjs re-requires the JSON (no edit).

Verified: isAllowedDomain() now returns true for www.war.gov,
www.thenationalnews.com, trumpstruth.org (exact/bare/www rule); shared
and api lists are byte-equal (339); edge-functions + scripts-shared
mirror guards green (228). feed-validation.yml does not run on PRs
(SSRF surface) — post-merge workflow dispatch is the end-to-end proof.

Closes #4970

Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant