Denshin / Blog / Testing
AI browser testing or Playwright scripts? We use both, for different jobs
Scripted Playwright tests and AI agents that drive a browser solve different problems. Where each one earns its place, what agents still get wrong, and the hybrid workflow we run on testing projects.
Denshin Engineering · Quality Engineering · 22 October 2026 · 4 min read
There are now two quite different ways to test a web app in a real browser. The older one is a script: Playwright, usually, with a test file that says go to this URL, fill this field, click this button, expect this text. The newer one is an agent: a model drives the browser, reads what is on the screen, decides what to click, and tells you whether the result looks right.
Clients ask which one they should pay for. Our answer is that they solve different problems, and treating them as rivals gets you a test suite that is either brittle or expensive.
What scripts are good at
A Playwright test does the same thing every time. That is the whole point. If it passed yesterday and fails today, something changed, and the failure tells you where: a locator that matched nothing, a timeout on a specific step, an assertion with expected and actual values side by side.
Modern Playwright is also less flaky than its reputation, which mostly dates from older tools. Locators wait for elements to be ready on their own, and if you locate by role and accessible name instead of CSS classes, a redesign that shuffles the layout often breaks nothing.
await page.getByRole('button', { name: 'Create invoice' }).click();
await expect(page.getByRole('alert')).toContainText('Invoice saved');
Tests like that are cheap to run, fast, and fit in CI on every pull request. For the flows that must never break (login, checkout, creating the core record your business runs on) this is what we want guarding the door.
The cost is upfront. Someone has to write each test and keep it in step with the product. Teams that skip that second part end up with a suite that faithfully covers the screens that existed a year ago.
What an agent is good at
An AI agent in a browser is better at the things scripts are worst at. Give it a goal in plain language ("create a quotation for a new customer with two line items and a discount, then convert it to an order") and it works out the clicks. Point it at a screen it has never seen and it will try things. Ask whether a page looks broken and it can often tell you, even when nothing threw an error.
That makes it useful for exploratory passes on a new build, for visual and acceptance checks where "does this look right" is the actual question, for flows that still change every week, and for drafting test cases or Playwright code from a description. It is also decent at reading a failing test and suggesting why it started failing.
The trade-offs are real. An agent run is slower and spends model tokens on every step, and the cost climbs as a session gets longer because more of the page history sits in context. It is less repeatable: two runs of the same instruction can take different paths. When it fails, the explanation is a paragraph of reasoning instead of a line number, and sometimes the reasoning is wrong. A typical miss: the agent sees a green "saved" toast and reports a pass, without noticing the total on screen is off by the tax amount.
How we combine them
On projects where we do independent testing, the pattern looks like this:
- An agent explores the build first, following the business flows we describe and anything else it finds. It writes down what it did and what looked wrong.
- A person reviews every finding before it becomes a bug report. Agents raise false alarms, and a client should not have to triage them.
- Flows that have settled down, and matter, get turned into Playwright tests. The agent can draft these. A person reads and tightens them, especially the assertions, because a test that asserts too little passes forever.
- CI runs the scripts on every change. The agent runs before a release, as a second net.
Two guardrails keep agent runs sane. Cap the number of steps in a session, and have it summarise and start fresh instead of wandering. And fail fast: if the same action fails twice, stop and report, rather than letting the model try fifteen creative workarounds. On that note, point agents at a test environment with test data, never production. An agent that can click "delete" will eventually click "delete".
A rough rule
If you can write down the steps and the expected result, and you will want to check it on every release for the next year, write a script. If you are not sure yet what you are looking for, or the screen changes too often to pin down, send an agent. Most products need both. The scripts are the floor; the agent finds the things nobody thought to script.
Letting an AI tool click around inside your systems raises the same questions as letting one write code for you, and our notes on guardrails for AI coding agents carry over almost unchanged.
Tags: Playwright, AI Testing, Test Automation, End-to-End Testing, QA
All posts · Work with Denshin