docs: CDP serve production guide + AI agent integration patterns

[Nicolas0315]

Summary

Adds two new documentation files in docs/ covering production deployment and AI agent integration — the two areas where we found the most friction as early adopters.

docs/cdp-serve-guide.md — CDP Serve Production Guide

Covers the "Getting Started → Production" gap that currently exists in the README:

• --timeout gotcha: The default --timeout 10 silently disconnects CDP clients that pause for >10 seconds (common in AI agent workflows where the agent waits for LLM responses between page interactions). Documents the fix (--timeout 0) prominently.
• --cdp_max_connections tuning: Memory estimates per connection count (e.g., 128 connections ≈ 900 MB).
• HTTP tuning table: --http_max_concurrent, --http_max_host_open, --http_timeout with recommended values for crawl workloads.
• Health check examples: curl script, Docker HEALTHCHECK, and docker-compose.yml with health checks.
• Process management: systemd unit file and macOS launchd plist.
• Troubleshooting table: Links to known issues ([BUG] CDP WebSocketDebuggerUrl returns 0.0.0.0 causing remote CDP clients to fail (ECONNREFUSED) #1922, etc.) with workarounds.

docs/ai-agent-integration.md — AI Agent Integration Patterns

5 connection patterns with complete, tested code examples:

1. Direct Puppeteer — single agent, sequential browsing
2. Parallel Page Pool — concurrent crawling with memory estimates (20 pages ≈ 140 MB vs Chrome's 5.6 GB)
3. Lightpanda → Chrome Fallback — automatic fallback with detection thresholds (DOM element count, body text length, JS error count, noscript detection)
4. Playwright (Python) — with connect_over_cdp
5. chromedp (Go) — RemoteAllocator

Also includes:

• Real-device benchmarks from macOS aarch64 (M4 Max) testing: fetch mode (example.com 0.5s, HN 1.1s, GitHub 2.0s) and CDP mode (example.com 501ms, HN 1030ms with 229 links)
• Recommended server configs for 3 workload profiles (AI agent, research/crawl, CI/testing)
• Known limitations table linking to open issues ([BUG] CDP WebSocketDebuggerUrl returns 0.0.0.0 causing remote CDP clients to fail (ECONNREFUSED) #1922, OpenClaw integration is blocked by missing Chromium-style discovery endpoints (/json, /json/list, /json/protocol) despite successful CDP connect and navigation #1932, Attempt to load a page that uses WebSocket resulted in "WebSocket is not Defined" #1952, [Bug]: Missing console API coverage breaks browser automation scripts that rely on console.log interception #1953)

Context

We're building next-browse, a knowledge-first browser that uses Lightpanda as the default CDP engine for autonomous agent crawling with Chrome as a fallback for complex SPAs. These docs are distilled from our real integration experience.

Checklist

• No code changes — documentation only
• All code examples tested on macOS aarch64 nightly build
• References existing open issues where relevant

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[Nicolas0315]

I have read the CLA Document and I hereby sign the CLA