Debug failed Playwright tests and auto-fix with PRs
libretto-playwright-debugger investigates failed Playwright runs with an AI agent and automatically opens pull requests to fix broken scripts.
Why it matters
Automatically investigate and repair broken browser automation scripts by analyzing failed Playwright test runs on live pages, diagnosing the root cause, and opening pull requests with fixes to keep end-to-end tests running reliably.
Outcomes
What it gets done
Capture browser context and page state when Playwright automation fails
Analyze failure stack traces and DOM state to diagnose script breakage
Generate code fixes for broken selectors, timing issues, and workflow changes
Open GitHub pull requests with proposed fixes to restore automation
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/saffron-health-libretto | bash Overview
Libretto
libretto-playwright-debugger is an AI debugging agent that hooks into an existing Playwright automation's failure path, investigates a failed run on the live page, and automatically opens a GitHub pull request proposing a fix to the broken script. Use it in Playwright projects with an existing failure-handling path where you want failures automatically diagnosed and turned into a fix PR; it requires a Libretto Cloud key plus an OpenAI or Anthropic key and sits alongside, not in place of, existing retry/fallback logic.
What it does
libretto-playwright-debugger adds a debugging agent to a Playwright automation project: when a run fails, it investigates the failure on the supplied live page and automatically opens a pull request to fix the broken script. It preserves the page's browser context while investigating, and it treats its own infrastructure failures as best-effort results rather than replacing or masking the original automation error - if the debugger itself can't run, the original failure still surfaces normally.
The debugger is initialized once at module scope with a GitHub repo target (owner, repo, base branch) and a model configuration (e.g. openai/gpt-5.4), then invoked at the project's existing failure point by awaiting debugFailure(error, page) before teardown closes the failed Page - existing fallback, retry, logging, and rethrow behavior around that call stay in place unchanged. Under the hood, debugFailure() uses stack frames to locate the relevant source files automatically; when the stack trace doesn't identify every relevant file, an includeFiles option accepts explicit repository-relative paths to widen what the agent looks at.
When to use - and when NOT to
Use libretto-playwright-debugger when you run Playwright browser automation in a project that already has a failure-handling path (a try/catch around automation runs) and want failures on a live page investigated and turned into a proposed fix automatically, rather than just logged for a human to debug from scratch. It's a fit for teams maintaining a meaningful volume of browser automation scripts where failures recur often enough that auto-generated fix PRs save real debugging time.
It requires a Libretto Cloud API key (via the Libretto GitHub App setup flow) and an API key for the configured model provider (OpenAI or Anthropic), plus an existing Playwright dependency in the project - it's an addition to an existing automation failure path, not a standalone testing framework or a replacement for Playwright itself.
Inputs and outputs
Input is a caught automation error plus the Playwright Page object that observed the failure, passed to debugFailure(error, page) inside the existing catch block, optionally with includeFiles naming source files the stack trace doesn't already surface. Setup requires LIBRETTO_API_KEY plus the relevant model-provider key (OPENAI_API_KEY or ANTHROPIC_API_KEY) set in the automation's environment, both stored in the project's existing secret-management system.
pnpm add libretto-playwright-debugger
Output is a pull request against the configured GitHub repository and base branch proposing a fix to the broken script, opened via the Libretto GitHub App.
Integrations
libretto-playwright-debugger integrates with an existing Playwright project (it depends on Playwright being already installed), GitHub via the Libretto GitHub App (for opening pull requests against a specified owner/repo/base branch), and either OpenAI or Anthropic as the underlying model provider for the debugging agent, selected by the agent.model string prefix (openai/... or anthropic/...).
Who it's for
Teams running browser automation with Playwright who want failed runs on a live page automatically diagnosed and turned into a proposed GitHub pull request fix, reducing the manual debugging loop for recurring or flaky automation breakage.
Source README
libretto-playwright-debugger
libretto-playwright-debugger adds a Playwright debugging agent that investigates
failed runs on the supplied live page and automatically opens pull requests to
fix broken scripts. It preserves the page's browser context and treats debugger
infrastructure failures as best-effort results instead of replacing the original
automation error.
Install
Add the package to the project that runs the Playwright automation:
pnpm add libretto-playwright-debugger
The project must already depend on Playwright.
Configure authentication
- Open the Libretto setup flow, install the
Libretto GitHub App for the target repository, and create a Libretto Cloud
API key. - Set
LIBRETTO_API_KEYin the environment that runs the automation. - Set the API key for the configured model provider:
OPENAI_API_KEYfor anopenai/...model orANTHROPIC_API_KEYfor ananthropic/...model.
Store both keys in the project's existing secret-management system.
Add it to the existing failure path
Initialize the debugger once at module scope:
import { createPlaywrightDebugger } from "libretto-playwright-debugger";
const playwrightDebugger = createPlaywrightDebugger({
github: {
owner: "acme",
repo: "automations",
baseBranch: "main",
},
agent: {
model: "openai/gpt-5.4",
},
});
At the existing failure point, await debugFailure() before teardown closes
the failed Page:
try {
await runAutomation(page);
} catch (error) {
await playwrightDebugger.debugFailure(error, page);
// Existing fallback and logging stay here.
throw error;
}
Use the Page that observed the failure. Keep the existing automation,
fallback, retry, logging, and rethrow behavior in place around the new call.
debugFailure() uses stack frames to find source files. When the stack does not
identify every relevant file, pass repository-relative paths:
await playwrightDebugger.debugFailure(error, page, {
includeFiles: ["src/workflows/checkout.ts"],
});
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.