feat(news): completeness measurement — feed health w/ silent zeros, coverage ledger + provenance, GDELT recall benchmark (#4920)#4927
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds three completeness-measurement layers to the news pipeline: (a) feed-health with silent-zero detection for Google News wrappers across consecutive empty runs, (b) a coverage ledger tracking per-feed/per-category/freshness-floor drop counts, and (c) a daily GDELT recall benchmark computing what fraction of top external stories the digest carries. It also consolidates story identity across the pipeline by replacing three divergent similarity implementations with a single dual-view lexical vector module.
Confidence Score: 3/5The feed-health, GDELT recall, and story-identity unification work correctly and are well-tested, but the coverage-ledger feature described in the PR is entirely absent from list-feed-digest.ts, causing the test suite to fail on four source-text assertions. The coverage-ledger instrumentation (droppedFeedCap, ledgerDrops, news:coverage-ledger:v1 publish) is called out in the PR description as delivered but is not present in list-feed-digest.ts or anywhere else in the diff. The test suite ships with four assertions that check for exact source-text patterns from that missing implementation; those assertions will all fail. server/worldmonitor/news/v1/list-feed-digest.ts needs the coverage-ledger instrumentation before the test suite can pass. tests/completeness-measurement.test.mjs lines 153-158 will fail until that code lands. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[feed-validation GH Actions daily] --> B[validate-rss-feeds.mjs client + server catalog]
A --> C[seed-recall-benchmark.mjs]
B --> D{Upstash creds?}
D -- yes --> E[publishFeedHealth buildFeedHealthPayload]
E --> F[news:feed-health:v1 silent-zero streaks]
D -- no --> G[skip silently]
C --> H[fetchGdeltJson 4 verticals x 25 articles]
H --> I[computeRecall cosineSimilarity vs digest]
I --> J[news:recall-benchmark:v1 recall% + missed headlines]
K[seed-insights.mjs selectTopStories stats] --> L[provenance storiesConsidered sourcesConsidered]
L --> M[InsightsPanel Compiled from N stories across M sources]
N[list-feed-digest.ts assignStoryIdentity] --> O[clusterTexts union-find fuzzy hash]
O --> P[canonical titleHash per cluster]
P --> Q[writeStoryTracking mentionCount once per hash]
R[coverage-ledger news:coverage-ledger:v1] -.->|MISSING| N
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[feed-validation GH Actions daily] --> B[validate-rss-feeds.mjs client + server catalog]
A --> C[seed-recall-benchmark.mjs]
B --> D{Upstash creds?}
D -- yes --> E[publishFeedHealth buildFeedHealthPayload]
E --> F[news:feed-health:v1 silent-zero streaks]
D -- no --> G[skip silently]
C --> H[fetchGdeltJson 4 verticals x 25 articles]
H --> I[computeRecall cosineSimilarity vs digest]
I --> J[news:recall-benchmark:v1 recall% + missed headlines]
K[seed-insights.mjs selectTopStories stats] --> L[provenance storiesConsidered sourcesConsidered]
L --> M[InsightsPanel Compiled from N stories across M sources]
N[list-feed-digest.ts assignStoryIdentity] --> O[clusterTexts union-find fuzzy hash]
O --> P[canonical titleHash per cluster]
P --> Q[writeStoryTracking mentionCount once per hash]
R[coverage-ledger news:coverage-ledger:v1] -.->|MISSING| N
Reviews (1): Last reviewed commit: "feat(news): completeness measurement — f..." | Re-trigger Greptile |
| const src = readSrc('server/worldmonitor/news/v1/list-feed-digest.ts'); | ||
| assert.match(src, /droppedFeedCap = Math\.max\(0, matches\.length - ITEMS_PER_FEED\)/); | ||
| assert.match(src, /ledgerDrops\.perCategoryCap \+= Math\.max\(0, items\.length - MAX_ITEMS_PER_CATEGORY\)/); | ||
| assert.match(src, /ledgerDrops\.freshnessFloor = droppedStaleTotal/); | ||
| assert.match(src, /news:coverage-ledger:v1/); | ||
| }); |
There was a problem hiding this comment.
Coverage-ledger test asserts unimplemented code
The four assert.match calls inside this test check for patterns (droppedFeedCap, ledgerDrops.perCategoryCap, ledgerDrops.freshnessFloor, news:coverage-ledger:v1) that are not present anywhere in server/worldmonitor/news/v1/list-feed-digest.ts. Running node --test tests/completeness-measurement.test.mjs will fail immediately on all four assertions.
The PR description claims "(b) Coverage ledger + provenance — every silent drop gate now counts: per-feed cap, per-category cap, freshness floor → news:coverage-ledger:v1:{variant}:{lang} per digest build" but the list-feed-digest.ts diff in this PR contains none of the instrumentation or the Redis publish for that key. The ledger wiring in list-feed-digest.ts appears to be the missing implementation piece.
| stats.sourceCapDropped = sourceCapDropped; | ||
| stats.overflowDropped = Math.max(0, admissible.length - selected.length - sourceCapDropped); | ||
| } |
There was a problem hiding this comment.
overflowDropped misclassifies source-capped items after the break
After selected.length >= maxCount triggers a break, the remaining items in admissible are never visited. The formula admissible.length - selected.length - sourceCapDropped counts ALL unvisited items as overflow — but any of those trailing items that would have hit the source cap are instead counted as overflow rather than source-capped. The grand total is always correct, but sourceCapDropped will be systematically under-counted and overflowDropped over-counted when the selection limit binds before the source-cap list is exhausted.
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round) Applies the multi-agent + cross-model review findings: - th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran before the {{token}} convention fix) — repaired; fa.json was missing the keys entirely AND absent from translate-locales' LOCALES list (pre-existing gap the parity test caught) — keys added, fa added to the translator roster - extractServerFeeds skipped double-quoted names ("Tom's Hardware") — regex accepts both quote styles; extraction test pins it (cross-model) - rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap and would undercount the coverage ledger for their whole TTL (cross-model) - coverage-ledger writes clamped to known variants + well-formed langs (request-supplied values could inflate the keyspace — adversarial) - GDELT titles AND urls bounded before publishing; urls http(s)-only - direct-run guards use pathToFileURL (Windows/symlink-safe); both publishers share scripts/_upstash-rest.mjs instead of hand-rolled fetch helpers; publishFeedHealth exported with a no-creds contract - .insights-provenance CSS added (was an unstyled div); gn()/gnLocale replica drift guard test; overflowDropped + orchestration-seam tests Battery: completeness (21), locale-completeness (25), clustering (16), story-identity (30), both typechecks, biome clean. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…p + rss parse-cache v6 bump (#4927 review round, missed pathspec) Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
Multi-agent code review — applied at f98e584Four reviewer personas (correctness, adversarial+security, testing, maintainability+standards) plus the independent Codex cross-model pass. All actionable findings applied; held for manual merge (stacked on #4924). Applied
Verified directly during review: Noted, not changed
Battery: completeness (21), locale-completeness (25), clustering (16), story-identity (30), both typechecks, biome. |
35e09c9 to
31d7a04
Compare
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round) Applies the multi-agent + cross-model review findings: - th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran before the {{token}} convention fix) — repaired; fa.json was missing the keys entirely AND absent from translate-locales' LOCALES list (pre-existing gap the parity test caught) — keys added, fa added to the translator roster - extractServerFeeds skipped double-quoted names ("Tom's Hardware") — regex accepts both quote styles; extraction test pins it (cross-model) - rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap and would undercount the coverage ledger for their whole TTL (cross-model) - coverage-ledger writes clamped to known variants + well-formed langs (request-supplied values could inflate the keyspace — adversarial) - GDELT titles AND urls bounded before publishing; urls http(s)-only - direct-run guards use pathToFileURL (Windows/symlink-safe); both publishers share scripts/_upstash-rest.mjs instead of hand-rolled fetch helpers; publishFeedHealth exported with a no-creds contract - .insights-provenance CSS added (was an unstyled div); gn()/gnLocale replica drift guard test; overflowDropped + orchestration-seam tests Battery: completeness (21), locale-completeness (25), clustering (16), story-identity (30), both typechecks, biome clean. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…p + rss parse-cache v6 bump (#4927 review round, missed pathspec) Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…d fails (#4927 external review) A transient Redis error during the previous-payload GET previously reset every consecutive-empty streak (previous=null -> fresh streaks). Read failure now skips the publish entirely; key-absent (first run) still starts fresh. Regression test included. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round) Applies the multi-agent + cross-model review findings: - th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran before the {{token}} convention fix) — repaired; fa.json was missing the keys entirely AND absent from translate-locales' LOCALES list (pre-existing gap the parity test caught) — keys added, fa added to the translator roster - extractServerFeeds skipped double-quoted names ("Tom's Hardware") — regex accepts both quote styles; extraction test pins it (cross-model) - rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap and would undercount the coverage ledger for their whole TTL (cross-model) - coverage-ledger writes clamped to known variants + well-formed langs (request-supplied values could inflate the keyspace — adversarial) - GDELT titles AND urls bounded before publishing; urls http(s)-only - direct-run guards use pathToFileURL (Windows/symlink-safe); both publishers share scripts/_upstash-rest.mjs instead of hand-rolled fetch helpers; publishFeedHealth exported with a no-creds contract - .insights-provenance CSS added (was an unstyled div); gn()/gnLocale replica drift guard test; overflowDropped + orchestration-seam tests Battery: completeness (21), locale-completeness (25), clustering (16), story-identity (30), both typechecks, biome clean. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…p + rss parse-cache v6 bump (#4927 review round, missed pathspec) Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
31d7a04 to
e471d51
Compare
…d fails (#4927 external review) A transient Redis error during the previous-payload GET previously reset every consecutive-empty streak (previous=null -> fresh streaks). Read failure now skips the publish entirely; key-absent (first run) still starts fresh. Regression test included. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
… attribution, publish-after-guardrails (#4927 re-review) All six re-review findings applied: P1 — never-activated publishing no longer poisons health: the two keys join ON_DEMAND_KEYS (missing data = soft EMPTY_ON_DEMAND) and their seed-meta entries are activationGated in api/seed-health.js ('pending-activation', healthy) until the first publish lands — then normal 2880-min staleness enforcement applies. The workflow also emits a visible ::warning annotation on runs where the UPSTASH secrets are absent, so green scheduled runs can't silently mask the gap. P2 — the coverage-ledger write is awaited (.catch non-fatal): the Edge response could previously finish before the fire-and-forget side write landed. P2 — selectTopStories classifies EVERY admissible candidate instead of breaking at maxCount: cap-first attribution (a candidate whose source already hit the per-source cap is a sourceCap drop even when selection is full; only genuinely rankable candidates count as overflow), with a same-source-overflow regression asserting exact one-of attribution. P2/P3 — StoryMeta.source_count proto doc now describes what runtime ships (current-cycle batch corroboration, default 1) instead of 'cached from Redis Set'; generated artifacts regenerated. P3 — _upstash-rest.mjs sends a User-Agent (CF-WAF convention: never a bare fetch UA); feed-health publishing moved AFTER the config-drift hard-fail so a guardrail-failing run cannot refresh health metadata first. Battery: completeness 23, clustering 16, mcp-parity 11, seed-health 2, typecheck:api, docs-stats. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
Re-review adjudication — all 6 findings validated, fixed at d1c6298
Battery: completeness 23, clustering 16, mcp-parity 11, seed-health 2, |
…re FIRST publish (#4927 re-review P1) Activation was inferred from seed-meta existence, which carries a 7d TTL: a publisher that ran once and died alarmed as stale only until the meta expired, then reverted to healthy pending-activation — violating the stated contract, and /api/health softened the keys permanently via ON_DEMAND_KEYS. "Has ever published" is now a durable marker (seed-activated:news:*, SET with NO TTL on every successful publish): - api/health.js pipelines EXISTS on the markers; a marker-bearing key leaves ON_DEMAND softening permanently — missing data/meta reads EMPTY/STALE_SEED like any other monitored key (marker read errors degrade to not-activated, never to a false alarm) - api/seed-health.js entries carry activationKey; 'pending-activation' is returned ONLY while the marker is absent — with the marker present, missing meta falls through to 'missing' (alarming) - both publishers SET their marker alongside every publish Regressions: classifyKey lifecycle (pending -> EMPTY_ON_DEMAND; activated-then-dead -> EMPTY) + publisher/marker wiring pins including the no-TTL assertion. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
Re-review round 2 — the remaining P1 fixed at $(git rev-parse --short feat/completeness-4920)Validated in full. Activation was inferred from seed-meta existence (7-day TTL), so "has ever published" evaporated with the meta: a publisher that ran once and died alarmed only until expiry, then reverted to healthy Fix (durable marker, as suggested): publishers
Regressions: Batteries: completeness 25, seed-health 2, mcp-parity 11, full tiered pre-push (both typechecks, seed tests) green. Cascade-rebased through #4928 → #4929. Held for manual merge. |
…marker probe (#4927) The harnesses assert every pipeline command is GET; the activation markers add EXISTS probes on seed-activated:* keys, which tripped the mock assertion and 503'd every request in these suites. Both mocks now accept EXISTS (marker-namespace-pinned, absent by default). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…etection, coverage ledger + provenance, GDELT recall benchmark (#4920) Turns "unmistakably complete" from a feeling into numbers, at three layers: (a) Feed health. validate-rss-feeds now also validates the SERVER digest catalog (_feeds.ts — the list buildDigest actually ingests; previously only the client config was measured) and, when Upstash secrets are present, publishes per-feed status + consecutive-empty streaks to news:feed-health:v1 daily. Google News wrappers that deliver nothing across consecutive runs are flagged as SILENT ZEROS — the failure mode where Google's ranking shift quietly blanks a source. (b) Coverage ledger + provenance. Every silent drop gate now counts: per-feed cap, per-category cap, freshness floor (digest → news:coverage-ledger:v1:{variant}:{lang}), and the insights selection gates (admissibility / source cap / overflow via selectTopStories stats). The World Brief payload carries provenance and InsightsPanel renders the stamp: "Compiled from N stories across M sources" (i18n fanned out to all locales). (c) External recall benchmark. Daily job (same GH Actions workflow, no Railway slot) pulls GDELT's top articles across four broad verticals, matches each against ingested digest titles using the shared story-identity similarity (#4919), and publishes recall % + the missed headlines to news:recall-benchmark:v1 — the first ground-truth answer to "did we miss a story?", with misses actionable by lowest similarity. Both published keys are registered in api/seed-health.js and api/health.js (daily cadence, 2-day alarm). Publishers skip silently without credentials, so local/PR runs are unchanged. Operator action to activate publishing: add UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN as GitHub Actions secrets. Closes #4920 🤖 Generated with Claude Code Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…uble-quoted names, parse-cache bump, bounded external fields (#4927 review round) Applies the multi-agent + cross-model review findings: - th/tr/zh compiledFrom carried single-brace placeholders (fan-out ran before the {{token}} convention fix) — repaired; fa.json was missing the keys entirely AND absent from translate-locales' LOCALES list (pre-existing gap the parity test caught) — keys added, fa added to the translator roster - extractServerFeeds skipped double-quoted names ("Tom's Hardware") — regex accepts both quote styles; extraction test pins it (cross-model) - rss:feed parse cache bumped v5→v6: warm v5 rows lack droppedFeedCap and would undercount the coverage ledger for their whole TTL (cross-model) - coverage-ledger writes clamped to known variants + well-formed langs (request-supplied values could inflate the keyspace — adversarial) - GDELT titles AND urls bounded before publishing; urls http(s)-only - direct-run guards use pathToFileURL (Windows/symlink-safe); both publishers share scripts/_upstash-rest.mjs instead of hand-rolled fetch helpers; publishFeedHealth exported with a no-creds contract - .insights-provenance CSS added (was an unstyled div); gn()/gnLocale replica drift guard test; overflowDropped + orchestration-seam tests Battery: completeness (21), locale-completeness (25), clustering (16), story-identity (30), both typechecks, biome clean. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…p + rss parse-cache v6 bump (#4927 review round, missed pathspec) Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…trap-hydrated), MCP exclusions, rss:feed v6 pin update (#4920) The stacked CI runs caught three registry gates the local scoped battery missed: (1) the new ops keys had landed in BOOTSTRAP_KEYS, which would hydrate feed-health/recall payloads into every page load — moved to STANDALONE_KEYS (health-monitored, on-demand); (2) both keys needed documented EXCLUDED_FROM_MCP entries (ops surfaces, not queryable news slices); (3) the rss:feed cache-prefix pin test still asserted v5 — updated for the #4920 v6 bump with the invalidation rationale. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…d fails (#4927 external review) A transient Redis error during the previous-payload GET previously reset every consecutive-empty streak (previous=null -> fresh streaks). Read failure now skips the publish entirely; key-absent (first run) still starts fresh. Regression test included. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
… attribution, publish-after-guardrails (#4927 re-review) All six re-review findings applied: P1 — never-activated publishing no longer poisons health: the two keys join ON_DEMAND_KEYS (missing data = soft EMPTY_ON_DEMAND) and their seed-meta entries are activationGated in api/seed-health.js ('pending-activation', healthy) until the first publish lands — then normal 2880-min staleness enforcement applies. The workflow also emits a visible ::warning annotation on runs where the UPSTASH secrets are absent, so green scheduled runs can't silently mask the gap. P2 — the coverage-ledger write is awaited (.catch non-fatal): the Edge response could previously finish before the fire-and-forget side write landed. P2 — selectTopStories classifies EVERY admissible candidate instead of breaking at maxCount: cap-first attribution (a candidate whose source already hit the per-source cap is a sourceCap drop even when selection is full; only genuinely rankable candidates count as overflow), with a same-source-overflow regression asserting exact one-of attribution. P2/P3 — StoryMeta.source_count proto doc now describes what runtime ships (current-cycle batch corroboration, default 1) instead of 'cached from Redis Set'; generated artifacts regenerated. P3 — _upstash-rest.mjs sends a User-Agent (CF-WAF convention: never a bare fetch UA); feed-health publishing moved AFTER the config-drift hard-fail so a guardrail-failing run cannot refresh health metadata first. Battery: completeness 23, clustering 16, mcp-parity 11, seed-health 2, typecheck:api, docs-stats. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…re FIRST publish (#4927 re-review P1) Activation was inferred from seed-meta existence, which carries a 7d TTL: a publisher that ran once and died alarmed as stale only until the meta expired, then reverted to healthy pending-activation — violating the stated contract, and /api/health softened the keys permanently via ON_DEMAND_KEYS. "Has ever published" is now a durable marker (seed-activated:news:*, SET with NO TTL on every successful publish): - api/health.js pipelines EXISTS on the markers; a marker-bearing key leaves ON_DEMAND softening permanently — missing data/meta reads EMPTY/STALE_SEED like any other monitored key (marker read errors degrade to not-activated, never to a false alarm) - api/seed-health.js entries carry activationKey; 'pending-activation' is returned ONLY while the marker is absent — with the marker present, missing meta falls through to 'missing' (alarming) - both publishers SET their marker alongside every publish Regressions: classifyKey lifecycle (pending -> EMPTY_ON_DEMAND; activated-then-dead -> EMPTY) + publisher/marker wiring pins including the no-TTL assertion. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…marker probe (#4927) The harnesses assert every pipeline command is GET; the activation markers add EXISTS probes on seed-activated:* keys, which tripped the mock assertion and 503'd every request in these suites. Both mocks now accept EXISTS (marker-namespace-pinned, absent by default). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
fc059a3 to
e6dd8ed
Compare
…laced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
…nce-demotion seam (#4922) (#4929) * feat(markets): wire markets into the news spine — macro-print actuals, earnings in the daily brief, finance-demotion seam (#4922) The no-proto slice of the markets↔news epic (ticker tagging on the wire and watchlist story alerts need a NewsItem proto field — tracked in #4922 as the follow-up slice): (b) Macro actuals. The economic calendar shipped every event with actual/estimate/previous empty forever — the print itself was never captured. The seeder now pulls FRED series observations (release ids and series ids are different namespaces; mapped per event with the right transform: CPI/PCE/Retail index→MoM %, NFP level→ΔK, GDP headline % direct), publishes a recentPrints block, and fills actual/previous on print-day events. The proto already carries the fields — values flow to the client with zero schema change. (c) Earnings reach a brief for the first time. Daily Market Brief gains earningsContext (recent beats/misses + upcoming count) collected via the earnings-calendar RPC with an 8s budget and rendered into the LLM context — symbols + direction only, stable per reporting date, so the #4914 summary-cache identity does not churn. (f) Finance-demotion seam. scoreImportance/selectTopStories accept {demoteFinance:false} — the ×0.35 demotion stays default for the geopolitical World Brief and is now switchable for a finance-focused ranking surface. Honest note: the only live consumer runs the full variant today; this is the seam a finance-variant insights run plugs into. Tests: print transforms (incl. FRED '.' markers), fill semantics (no-overwrite, future-stays-empty), demotion ×0.35 default vs neutral, earnings prompt block, wiring. 85 tests across the affected battery. Refs #4922 (remaining slices tracked there) 🤖 Generated with Claude Code Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP * fix(review): markets wiring hardening — deliverable earnings lookback, print adjacency guards, unitless values, pure transform (#4929 review round) Applies the multi-agent + cross-model review findings: - the "recent beats/misses" block was undeliverable as wired: the earnings seed window started at today, so past reporters never existed in the payload (correctness + codex, cross-corroborated) — seed window widened to today-7d - computePrintValues hardened per codex P1: a '.' LEADING observation no longer falls back to an older row presented as current, and pct_mom/diff_k require date-adjacent periods (45-day tolerance) so a mid-series gap cannot mislabel a 2-month change as month-over-month - print values are now UNITLESS — event.unit already renders '%'/'K' and embedding it double-rendered in the calendar (codex conf-100) - recentPrints consumer scope documented in-code: the RPC reconstructs its proto response without it (Redis/ops-side until the proto gains a field — epic follow-up); the user-visible win flows through the event actual/previous fields the proto already carries - earnings transform extracted to pure buildEarningsBriefContext (this PR's own standard, per review) with direct tests; earnings prompt block covered by a capture test asserting content AND cache-identity stability across rebuilds; selectTopStories opts pass-through exercised end-to-end; EVENT_SERIES↔FRED_RELEASES name lockstep pinned by test - verified: fredFetchJson error paths never embed the keyed URL 45 tests across the affected battery; typecheck + biome clean. Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP * fix(review): release-day stale-print guard, benchmark fast-fail, misplaced-opts shift (#4929 external review) - observationMatchesRelease: a filled actual must belong to the period the release reports (monthly diff==1; quarterly 1..5 for the advance/second/third GDP estimates) — pre-print on release day the latest FRED observation is the PRIOR period and was presented as today's number (P1); regression pins the July-15-CPI/May-obs case - recall benchmark uses fast-fail GDELT limits (10s timeout, 1 retry, 1 proxy attempt) under a shared 4-minute deadline — production retry defaults across 4 sequential queries could eat the 25-min workflow - selectTopStories auto-shifts an opts-shaped object passed in the stats slot (positional-API hazard), tested Also validated-not-changed: cross-cycle canonical adoption is the documented #4925 follow-up; seed-meta recordCount=ok is intended job-health semantics (streak preservation landed on the #4927 branch). Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
… — unblock feed-health activation (#4970) (#4971) Three server-catalog feeds (Pentagon, The National, Truth Social) had hosts absent from the RSS allowlist, tripping the CONFIG_DRIFT hard-fail in validate-rss-feeds.mjs --ci (exit 1) which — per the #4927 ordering — blocks feed-health/recall-benchmark publishing before it runs. Pre-existing drift (DoD rebranded defense.gov→war.gov; the other two were added to the catalog without allowlisting), surfaced by activating the publish path. Adds the three hosts to all mirrors: shared/rss-allowed-domains.json (source of truth) + its byte-identical scripts/shared mirror + the api/_rss-allowed-domains.js copy the validator and Edge SSRF guard actually read + vite.config.ts RSS_PROXY_ALLOWED_DOMAINS (dev proxy). The .cjs re-requires the JSON (no edit). Verified: isAllowedDomain() now returns true for www.war.gov, www.thenationalnews.com, trumpstruth.org (exact/bare/www rule); shared and api lists are byte-equal (339); edge-functions + scripts-shared mirror guards green (228). feed-validation.yml does not run on PRs (SSRF surface) — post-merge workflow dispatch is the end-to-end proof. Closes #4970 Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
Closes #4920 (Bet 2 of the strategic news-pipeline review). Do not auto-merge — held for review.⚠️ Stacked on #4924 (the recall benchmark matches titles via
scripts/shared/story-identity.js); merge #4924 first — this PR's diff will shrink to its own commits after that.Three layers of "did we miss something?"
(a) Feed health + silent zeros —
validate-rss-feeds.mjsnow validates BOTH feed universes (client config and the server digest catalog_feeds.ts— the list the digest actually ingests, which was previously unmeasured; 276 unique server URLs extracted with faithfulgn()URL reconstruction). With Upstash secrets in the daily GH Actions run it publishesnews:feed-health:v1with per-feed status + consecutive-empty streaks; Google News wrappers empty across ≥2 consecutive daily runs are flagged as silent zeros — the "Google ranking shift quietly blanked a source" failure mode.(b) Coverage ledger + provenance — every silent drop gate now counts: per-feed cap (previously fully invisible), per-category cap, freshness floor →
news:coverage-ledger:v1:{variant}:{lang}per digest build; insights selection gates (admissibility/source-cap/overflow) via a backward-compatibleselectTopStoriesstats param. The World Brief payload carries provenance and the panel renders "Compiled from N stories across M sources" (all locales fanned out via translate-locales;{{token}}convention).(c) External recall benchmark — daily, same workflow, zero Railway slots: GDELT top articles across 4 broad verticals matched against ingested digest titles using the #4919 shared similarity →
news:recall-benchmark:v1with recall % and the missed headlines sorted by lowest similarity (actionable coverage holes, not just a number).Both keys registered in
api/seed-health.js+api/health.js(age-above-coverage precedence respected).Operator action
Add
UPSTASH_REDIS_REST_URL/UPSTASH_REDIS_REST_TOKENas GitHub Actions secrets to activate publishing — without them both publishers log a skip and the workflow behaves exactly as before.Verification
tests/completeness-measurement.test.mjs(14): silent-zero streak/reset/wrapper-only semantics, recall math incl. edit-variant matching + null-not-NaN, selection drop stats, server-catalog extraction (250+ feeds), wiring assertions.https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP