Skip to content

fix(search-core): filter same-domain links and harden feed ingestion#11

Merged
lurkshark merged 13 commits intomainfrom
ballard
Mar 4, 2026
Merged

fix(search-core): filter same-domain links and harden feed ingestion#11
lurkshark merged 13 commits intomainfrom
ballard

Conversation

@lurkshark
Copy link
Owner

  • resolve markdown links against the source canonical URL before deriving target IDs
  • skip same-domain outbound links while preserving self-link filtering and dedupe behavior
  • pass source canonical URL through chunking/link insertion flow
  • tighten crawler backoff detection (reCAPTCHA marker) and simplify default request headers
  • enforce a minimum 1s interval between paginated feed requests
  • update link-graph/ranking/feed tests and archive the OpenSpec change docs

@lurkshark lurkshark merged commit 87c5b6d into main Mar 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant