Skip to content

Add request/response body capture for network request tracking#933

Closed
ctate wants to merge 1 commit intomainfrom
fix/issue-932-1773928676890
Closed

Add request/response body capture for network request tracking#933
ctate wants to merge 1 commit intomainfrom
fix/issue-932-1773928676890

Conversation

@ctate
Copy link
Collaborator

@ctate ctate commented Mar 19, 2026

Enhances network request tracking to capture and display request and response bodies, addressing a critical gap for agent workflows that need to analyze HTTP request/response content.

Changes Made

  • Extended TrackedRequest struct to include:

    • request_body: Captures POST data from outgoing requests
    • response_body: Stores response content (fetched lazily after loading completes)
    • status: HTTP response status code
    • response_headers: Response headers from server
    • request_id: Internal CDP correlation ID
  • Enhanced event handling in process_events():

    • Network.requestWillBeSent: Now extracts and stores request body (postData)
    • Network.responseReceived: Captures response metadata (status, headers)
    • Network.loadingFinished: Triggers lazy fetching of response body via Network.getResponseBody
  • Added response body fetching in handle_requests():

    • Asynchronously fetches response bodies for completed requests
    • Handles both text and binary responses appropriately
    • Updates tracked requests with fetched content
  • Updated CLI output formatting:

    • Enhanced format_request_table() to display request/response bodies
    • Added --clear flag for clearing request logs
    • Improved command documentation

Implementation Details

  • Response bodies are fetched lazily to avoid performance overhead during active browsing
  • Uses CDP's Network.getResponseBody API for reliable content retrieval
  • Maintains backward compatibility with existing network tracking features
  • Integrates seamlessly with existing HAR recording functionality

This enables agents to inspect full HTTP request/response cycles including payloads, which is essential for understanding API interactions and web application behavior.

Fixes #932

Enhances network request tracking to capture and display request and response bodies, addressing a critical gap for agent workflows that need to analyze HTTP request/response content.

## Changes Made

- **Extended TrackedRequest struct** to include:
  - `request_body`: Captures POST data from outgoing requests
  - `response_body`: Stores response content (fetched lazily after loading completes)
  - `status`: HTTP response status code
  - `response_headers`: Response headers from server
  - `request_id`: Internal CDP correlation ID

- **Enhanced event handling** in `process_events()`:
  - `Network.requestWillBeSent`: Now extracts and stores request body (postData)
  - `Network.responseReceived`: Captures response metadata (status, headers)
  - `Network.loadingFinished`: Triggers lazy fetching of response body via `Network.getResponseBody`

- **Added response body fetching** in `handle_requests()`:
  - Asynchronously fetches response bodies for completed requests
  - Handles both text and binary responses appropriately
  - Updates tracked requests with fetched content

- **Updated CLI output formatting**:
  - Enhanced `format_request_table()` to display request/response bodies
  - Added `--clear` flag for clearing request logs
  - Improved command documentation

## Implementation Details

- Response bodies are fetched lazily to avoid performance overhead during active browsing
- Uses CDP's `Network.getResponseBody` API for reliable content retrieval
- Maintains backward compatibility with existing network tracking features
- Integrates seamlessly with existing HAR recording functionality

This enables agents to inspect full HTTP request/response cycles including payloads, which is essential for understanding API interactions and web application behavior.

Fixes #932
@vercel
Copy link
Contributor

vercel bot commented Mar 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-browser Ready Ready Preview, Comment Mar 19, 2026 1:58pm
agent-browser-env-demo Ready Ready Preview, Comment Mar 19, 2026 1:58pm

@ChunHao-dev
Copy link
Contributor

Hi @ctate, I've been working on the same issue (#932) and opened #935 with an overlapping implementation.

The core request/response capture is similar to yours, but my PR also adds filtering flags for network requests:

  • --type xhr,fetch (resource type)
  • --method POST (HTTP method)
  • --status 2xx (supports exact, class, and range matching)

Happy to rebase on top of this PR if you'd prefer to merge yours first — I can then submit just the filtering additions as a follow-up.

@ctate ctate closed this Mar 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Request and Response Body is not being captured in network request

2 participants