In today’s newsletter:

• [Hands-On] Build a Browser Automation Agent

• Let Agents Rewrite Their Own Harness

Reading time: 5 minutes.

[Hands-On] Build a Browser Automation Agent

Most of the data on the web doesn't live behind an API. Prices, news, weather, top post on Hacker News. Getting that data normally means writing scrapers, which are susceptible to breaking.

How about a browser agent that skips the scraper entirely? You describe the task in plain English, like "find the latest news about AI agents and summarize the top 3 stories," and an LLM drives a real browser to complete it. The agent plans the steps, navigates, clicks, reads pages, and returns a structured answer.

This project builds one as a chat app. You type an instruction into a Gradio interface, watch the agent browse step by step, and get back a summary plus every page it visited and an action log.

Tech Stack

• browser-use for the autonomous browser agent loop

• Playwright to drive a real Chromium browser

• Orq.ai AI Router to serve the LLM through one OpenAI-compatible endpoint

• Gradio for the chat interface

Prerequisites

• Python 3.10 or higher

• An Orq.ai API key, set as ORQ_API_KEY in a .env file

How the Agent Works

The easiest way to understand the project is to follow one instruction through it. You type "find the top post on Hacker News right now and summarize it" into the chat.

1. The UI hands off and starts streaming. The Gradio handler sends your instruction to the agent on a background thread, and a queue feeds progress messages back into the chat, so you see lines like "Step 2: Navigating to news.ycombinator.com..." while the agent browses

2. The LLM plans the first action. agent.py wraps your instruction in a browser-use Agent backed by the Orq.ai-routed model. The model reads the task and decides what to do first, in this case opening Hacker News

3. Playwright executes it in a real browser. browser-use translates each decision into Chromium actions. Pages actually render, links actually get clicked, and text gets read straight off the page

4. Observe, decide, repeat. After every action, the agent feeds the new page state back to the LLM, which picks the next action. The loop runs until the model declares the task done or hits a 25-step cap

5. The whole run becomes one answer. The run history is parsed into a structured result, with the summary, pages visited, and an action log, and rendered back into the chat as markdown

Building the agent takes few steps, and all of them live in agent.py. Connect the LLM, define what the agent returns, run the browsing loop, and parse the results. Here's each step.

Step 1: Point the LLM at the Router

The agent builds a ChatOpenAI client aimed at Orq.ai's router endpoint. Models are addressed as provider/model strings, so switching from alibaba/qwen3.6-flash to any other supported model is a one-line change with no new client library and no new API key.

Step 2: Define the Result Shape

Instead of returning raw agent output, the project defines an AgentResult dataclass holding the summary, a success flag, the URLs visited, the action log, and any error.

Deciding the output shape before running the agent is what keeps the UI layer simple. The chat code never inspects agent internals; it just renders one object.

Step 3: Run the Agent

The core function creates a browser-use Agent with the task, the LLM, and a step callback, then awaits the run. A hard cap of 25 steps stops runaway sessions, because an agent that can't finish a task shouldn't get to browse forever.

Step 4: Parse the History

The browser-use library changes its API often; it’s a best practice to never trust it blindly. Every read from the run history, like the final answer or the visited URLs, sits inside its own try/except. If a future version breaks one of them, the agent still returns everything else instead of crashing.

Run It

Clone the repo:

Create a virtual environment and install the dependencies:

Add your environment variables:

cp .env.example .env

Then launch the app:

gradio app.py

Then open the local URL and try instructions like "what is the current weather in Tokyo" or "look up the price of the latest iPhone on Apple's website." The agent streams its progress as it browses and ends with the summary, sources, and action log.

Where This Pattern Fits

A browser agent is the right tool when the data is public but has no API, when the task needs interaction like searching or clicking through results, or when tasks vary too much to hard-code a scraper for each one. It's the wrong tool for high-volume extraction, where a dedicated scraper is faster and far cheaper per page.

Full implementation on GitHub →

Let Agents Rewrite Their Own Harness

Two agents running the same model can perform completely differently.

The difference lies in the prompts, tools, memory, and verification rules that wrap around the model (Agent Harness). Harnesses today are still built as they were in the ReAct era, with human engineers inspecting failures and hand-tuning the scaffolding.

That approach has a scaling problem. Every model has its own habits, tool-use quirks, and error modes, so a harness tuned for one model might fit the next one poorly. New models ship every few months. Human harness engineering can't keep up.

Self-Harness, a new paper from Shanghai AI Lab, uses a different idea. The Idea is simple: let the agent improve its own harness. No human engineers, no stronger external model is doing the tuning. The model weights stay frozen the entire time. Only the layer around the model changes, and the model itself is the one changing it.

The Loop That Turns Failures Into Edits

Self-Harness runs as an iterative loop with three stages.

• Weakness Mining. The agent runs on a set of tasks and collects execution traces with verifiable outcomes. It then clusters its failed traces, so it reasons about recurring model-specific failure patterns instead of isolated mistakes.

• Harness Proposal. Based on those patterns, the agent generates a small set of minimal harness edits, each tied to a specific failure mechanism. The minimality constraint keeps edits targeted instead of turning into generic instruction bloat.

• Proposal Validation. Every candidate edit goes through regression testing. An edit is only merged into the next harness version if it improves performance without measurable degradation on held-out tasks.

The validation stage is what makes the loop trustworthy. Self-modification without regression tests is how systems drift. With them, every accepted edit has evidence behind it.

The Numbers

The team tested three models from different families, MiniMax M2.5, Qwen3.5-35B-A3B, and GLM-5. Each one started with a bare-bones harness, just a default system prompt and basic file and shell tools, and ran the improvement loop on Terminal-Bench-2.0.

All three got noticeably better. On tasks the loop had never seen, MiniMax M2.5 went from 40.5% to 61.9%, Qwen3.5-35B-A3B from 23.8% to 38.1%, and GLM-5 from 42.9% to 57.1%.

The models themselves never changed. Every gain, over 20 points in the best case, came from the agent rewriting the instructions and tools around itself.

Each Model Fixed Different Things

The most interesting part is what the edits actually contained, because each model repaired different weaknesses.

• MiniMax M2.5 gave itself rules to create required output files earlier, handle structured tool outputs more carefully, and cut off unproductive tool-use loops.

• Qwen3.5 focused on checking dependencies up front, not repeating failed commands, and remembering to produce required artifacts after tool errors.

• GLM-5 mostly taught itself to preserve environment settings across shell commands and to stop exploring and start implementing sooner.

Some runs went further than local fixes, introducing structural mechanisms like subagent-based decomposition and middleware. The loop didn't just patch failure cases. It occasionally reorganized how the agent solves problems.

This is the strongest evidence for the paper's premise. Harness design really is model-specific, and the model's own failure traces are the best guide to what its harness should say.

Harness tuning is one of the biggest costs of every model migration. If an agent can mine its own failures and patch the layer around itself, that cost stops being an engineering bottleneck and becomes something you run overnight.

Read the Paper →

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.