Skip to content

mcp: add detectForms tool for structured form discovery#1951

Open
mvanhorn wants to merge 13 commits intolightpanda-io:mainfrom
mvanhorn:osc/feat-mcp-detect-forms
Open

mcp: add detectForms tool for structured form discovery#1951
mvanhorn wants to merge 13 commits intolightpanda-io:mainfrom
mvanhorn:osc/feat-mcp-detect-forms

Conversation

@mvanhorn
Copy link
Contributor

Summary

Adds a detectForms MCP tool and corresponding lp.detectForms CDP command that return structured form metadata from the current page. Each form includes its action URL, HTTP method, and an array of fields with names, types, required status, current values, select options, and backendNodeIds for use with the fill tool.

Why this matters

AI agents filling forms currently need multiple round-trips: call interactiveElements, filter for form-related elements, figure out which fields belong to which form, then call fill per field. There is no way to get a form's structure - its action, method, field names, types, or select options - in a single call.

  • #1890 - Multi-step form POST breaks agent workflow (SAP SAML login)
  • #1161 - Focus/blur/text selection improvements - 5 PRs landed for form interaction
  • #335 - Cookie management for form/auth workflows (7 comments)
  • #1869 - Turnstile/auth widget failures - form/auth flows are a top use case

No existing tool provides form structure. interactiveElements returns flat lists without form grouping. structuredData extracts JSON-LD/OpenGraph but not forms.

Changes

New file: src/browser/forms.zig

  • FormInfo, FormField, SelectOption structs with jsonStringify
  • collectForms() walks the DOM via TreeWalker, finds <form> elements, collects child <input>, <textarea>, and <select> fields
  • Skips hidden inputs and submit buttons (not fillable)
  • Skips empty forms (no fields)
  • For <select> elements, collects all <option> values and text

Modified: src/mcp/tools.zig

  • detectForms tool in tool_list with optional URL parameter
  • handleDetectForms handler: collects forms, registers nodes in node_registry, serializes with backendNodeIds

Modified: src/cdp/domains/lp.zig

  • detectForms CDP command with same data, returns forms + formNodeIds arrays

Modified: src/lightpanda.zig

  • Export pub const forms

Example output

[{
  "backendNodeId": 42,
  "action": "/login",
  "method": "POST",
  "fields": [
    {"backendNodeId": 43, "tagName": "input", "name": "email", "inputType": "email", "required": true, "placeholder": "you@example.com"},
    {"backendNodeId": 44, "tagName": "select", "name": "role", "value": "user", "options": [{"value": "user", "text": "User"}, {"value": "admin", "text": "Admin"}]},
    {"backendNodeId": 45, "tagName": "input", "name": "password", "inputType": "password", "required": true}
  ]
}]

Testing

src/browser/forms.zig includes 6 inline test blocks covering: login forms, select options, textarea, empty form exclusion, hidden input exclusion, and multiple forms. Build passes with zig build.

Implementation follows the collectInteractiveElements pattern in interactive.zig: TreeWalker over the DOM, collect matching elements, register nodes for backendNodeId references.

This contribution was developed with AI assistance (Claude Code).

Add a detectForms MCP tool and lp.detectForms CDP command that return
structured form metadata from the current page. Each form includes its
action URL, HTTP method, and fields with names, types, required status,
values, select options, and backendNodeIds for use with the fill tool.

This lets AI agents discover and fill forms in a single step instead of
calling interactiveElements, filtering for form fields, and guessing
which fields belong to which form.

New files:
- src/browser/forms.zig: FormInfo/FormField structs, collectForms()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@arrufat arrufat self-assigned this Mar 21, 2026
mvanhorn and others added 2 commits March 21, 2026 08:54
Form.getAction() resolves relative URLs against the page base, which
causes test failures when the page URL is a test server address. Use
the raw action attribute value instead, which matches what agents need
to understand the form's target path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@arrufat
Copy link
Contributor

arrufat commented Mar 22, 2026

Thank again for the PR, I just ran the tests locally and fixed the failing one.
Is this ready for review?

@mvanhorn
Copy link
Contributor Author

Yes, ready for review. The wba-test failure looks like a CI config issue (missing URL in curl step) - unrelated to this PR. All Zig tests pass.

@arrufat arrufat requested a review from karlseguin March 23, 2026 01:19
Copy link
Collaborator

@karlseguin karlseguin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is missing two things:

1 - a 'disabled' flag (which could come directly from the element, or from a parent) and form fields outside of the form. FormData handles this, we should extract the logic and make it as simple function call.

2 - Fields outside of the form. For example:

<input form="spice">
<form id=spice>...</form>

We have code to handle this, but, looking at it, it could be better. For now, you'd have to do:

const form_id = self.asElement().getAttributeSafe(comptime .wrap("id"));
const root = if (form_id != null)
    self.asNode().getRootNode(null) // Has ID: walk entire document to find form=ID controls
else
    self.asNode(); // No ID: walk only form subtree (no external controls possible)

const node_live = collections.NodeLive(.form).init(root, self, page);

while (node_live.next()) |form_element| {
    // ...
}

karlseguin added a commit that referenced this pull request Mar 23, 2026
Meant to help things like #1951

Small optimization to form node_live iterator

Disable iframes test (not related, but they are super-flaky, and I'm tired of
CI's failing because of it. I'll look at them later today).
arrufat and others added 5 commits March 23, 2026 15:11
Co-authored-by: Karl Seguin <karlseguin@users.noreply.github.com>
Co-authored-by: Karl Seguin <karlseguin@users.noreply.github.com>
Co-authored-by: Karl Seguin <karlseguin@users.noreply.github.com>
Address review feedback from @karlseguin:

1. Use Form.getElements() instead of manual TreeWalker for field
   collection. This reuses NodeLive(.form) which handles fields
   outside the <form> via the form="id" attribute per spec.

2. Add disabled detection: checks both the element's disabled
   attribute and ancestor <fieldset disabled> (with first-legend
   exemption per spec). Fields are flagged rather than excluded -
   agents need visibility into disabled state.

3. allocator is now the first parameter in collectForms/helpers.

4. handleDetectForms returns InvalidParams on bad input instead
   of silently swallowing parse errors.

5. Added tests for disabled fields, disabled fieldsets, and
   external form fields via form="id".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mvanhorn
Copy link
Contributor Author

Addressed in 35551ac:

  1. External form fields: Replaced the manual TreeWalker with Form.getElements(page) which uses NodeLive(.form) - handles fields outside the form via the form="id" attribute automatically.

  2. Disabled flag: Added disabled: bool to FormField. Checks both the element's disabled attribute and ancestor <fieldset disabled> with the first-legend exemption per spec. Fields are flagged rather than excluded since agents need visibility into disabled state.

  3. allocator first in collectForms and helpers.

  4. Invalid input handling: handleDetectForms returns InvalidParams instead of silently swallowing parse errors.

  5. Added tests for disabled fields, disabled fieldsets, and external form fields.

@mvanhorn
Copy link
Contributor Author

Thx all!

@arrufat
Copy link
Contributor

arrufat commented Mar 23, 2026

Hmm. I thought that would fix the tests. They pass locally...

@mvanhorn
Copy link
Contributor Author

The CI failure is in src/browser/tests/frames/frames.html - the link_click script reports "no assertions". Looks like a flaky frames navigation test unrelated to detectForms. Happy to rebase to re-trigger CI if that helps.

@karlseguin
Copy link
Collaborator

Yes, sorry about that. Hopefully that test become reliable in #1958 That failure won't stop us from merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants