In today’s newsletter:

• 5 AI Engineering Skills for Using Coding Agents Effectively

• AI Agents Can Build and Improve Their Own Harnesses

Reading time: 5 minutes.

Five Coding Agent Skills Every AI Engineer Needs

Coding agents can write code faster than most engineers can review it. As implementation speeds up, the hard work shifts to deciding what to build, controlling how the agent works, and proving the result is correct.

Andrew Ng recently published an AI Engineering Skills Map based on interviews with dozens of AI engineers and his team’s experience using coding agents. The map identifies five skills that separate productive agent use from handing over a task and hoping the code works.

The five skills cover the entire development cycle. You direct the work, choose how independently the agent can operate, review the result, improve its working environment, and understand enough of the underlying system to intervene when the run goes wrong.

1. Direct the Workflow

Software development still moves through planning, execution, deployment, and monitoring. Coding agents change how much human effort each stage requires, but they don’t remove the stages.

Directing the workflow means deciding where the agent should work and where you should retain control. You might let an agent research the codebase and propose an implementation plan while keeping architecture decisions and acceptance criteria with a human engineer.

The right split depends on four constraints:

• Speed. How quickly does the task need to ship?

• Cost. How many agent runs and review cycles can the task justify?

• Risk. What happens if the implementation is wrong?

• Human effort. Which decisions require knowledge the agent doesn’t have?

A small prototype may start from a short prompt. A change to an existing system with real users needs a detailed spec, explicit constraints, and a plan that can be reviewed before implementation begins.

The skill is knowing how much planning the task requires. Too little leaves the agent guessing. Too much costs more time than the code is worth.

2. Calibrate Agent Autonomy

Giving an agent more autonomy isn’t always better. The useful autonomy level depends on the task and the checks surrounding it.

A low-risk refactor with strong tests can run for a long time without intervention. A database migration that touches production data should stop at defined checkpoints and require approval before any destructive action.

Agent autonomy also depends on context. If the agent loses a changed requirement halfway through the build, more runtime only gives it more time to implement the wrong behavior.

You need to decide:

• which requirements remain in the agent’s active context

• when the agent should stop and request input

• which tools and files the agent can access

• when several agents should work in parallel

• how their results return to the main task

Permissions set the outer boundary. Context keeps the work pointed at the correct goal. Verification tells the agent when the task is actually finished.

3. Review the Work

A coding agent can produce a clean diff that implements the wrong behavior. Reviewing agent-generated work therefore starts with the expected outcome, not the appearance of the code.

For a backend change, verification may include unit tests, integration tests, and checks against the API contract. For a user interface, the agent can run the application and provide screenshots of the completed flow. Qualitative outputs may need an eval set or an LLM judge.

The tests need review too. An agent can satisfy weak tests while missing the requirement those tests were meant to represent.

Human review remains useful when the cost of failure is high or the expected behavior is difficult to encode. The reviewer may inspect the resulting behavior, security boundary, or architecture without reading every generated line.

As agents produce more code, verification capacity becomes the limit. The engineer who can design reliable checks can safely delegate more implementation.

4. Customize the Agent’s Environment

A default coding agent begins each project with little knowledge of how your system works. The agent has to rediscover build commands, architecture decisions, naming conventions, and data-access patterns during the task.

Persistent project instructions remove that repeated work. Files such as AGENTS.md and CLAUDE.md can record the commands and constraints that should apply across sessions.

Other extensions handle more specialized needs:

• Skills provide reusable procedures for tasks such as code review or deployment.

• Plugins package related skills, commands, and integrations.

• MCP servers connect the agent to external tools and data.

• Hooks trigger checks at defined points in the workflow.

Customization also includes removing instructions and tools that no longer help. An outdated skill can push a newer model toward a workaround it no longer needs. A large tool catalog can consume context and make tool selection harder.

A useful agent environment gives the model enough structure to work consistently without filling its context with instructions unrelated to the current task.

5. Understand Coding Agent Foundations

You don’t need to build a coding agent from scratch. You do need a working model of what happens after you submit a task.

The model operates inside a harness that manages repository search, context, tools, permissions, subagents, and execution. Each part introduces its own failure modes.

An agent may miss a file because retrieval did not surface it. It may forget a requirement because compaction removed the detail. It may stop early because the harness interpreted a text response as completion. It may overengineer a feature because the prompt never defined the acceptable scope.

Those failures require different corrections. A larger prompt will not repair weak tests. Another subagent will not fix a missing permission. A longer context window will not help if the wrong files were retrieved.

Understanding the harness lets you diagnose the layer that failed and give the agent the missing context, tool, constraint, or verifier.

Very long autonomous runs attract attention because they produce impressive logs and large token counts. Andrew Ng’s point is that effective agent use is usually more iterative. Skilled engineers know when to let the agent continue and when a small intervention will prevent hours of work in the wrong direction.

Coding agents reduce the effort required to produce code. That makes direction, control, and verification more valuable because those skills determine whether the generated code belongs in the system at all.

Read Andrew Ng’s Coding Agent Skills Map → 

AI Agents Can Build and Improve Their Own Harnesses

The same model can behave like two different coding agents depending on the harness running around it.

GPT-5 scored 35.2% on Terminal-Bench 2.1 with one harness and 49.6% with another. The model weights never changed. The agent loop, tools, context management, and recovery logic did.

HarnessDev asks whether an AI agent can write that surrounding infrastructure for itself. Let’s look at what the agent builds, how it improves the code, and why those improvements don’t always survive outside the development tasks.

The Agent Starts With Almost Nothing

HarnessDev gives the agent a weak but runnable starting point. The starter code can load a task, expose basic tools, and save the result.

Everything that makes the system behave like a capable agent is missing:

• Agent loop: Decide when to call the model or execute another tool.

• Context management: Keep useful observations without filling the context window with old output.

• Planning: Break a large task into smaller pieces and track unfinished work.

• Recovery: Retry failed actions and recover after interrupted runs.

• Verification: Check whether the task was actually completed before stopping.

The agent receives a description of the task family and a few development cases. It then edits the starter code until it has a complete harness.

That is a much harder problem than writing a system prompt. The agent is building the runtime that will control its own future tool calls.

The Finished Harness Is Tested on New Tasks

Once the agent finishes building the harness, HarnessDev freezes the code and tests it on tasks the agent never saw.

This separation matters because an agent can easily write rules that only solve the examples used during development. A coding harness might memorize repository layouts. A research harness might create search queries that only work for the given questions.

Testing the frozen harness on unseen tasks reveals whether the agent learned a reusable workflow or encoded a collection of shortcuts.

Across writing and machine-learning tasks, several generated harnesses matched or exceeded the selected human-built references. Coding and web research remained harder.

The amount of generated code didn’t predict success either. One model produced the smallest harnesses and still achieved the strongest Terminal-Bench result. Focused control logic and reliable verification mattered more than adding more components.

Execution Feedback Becomes a Code Change

HarnessDev also tests whether agents can improve an existing harness.

The agent receives execution traces and scores from previous runs. It can inspect where the harness wasted tokens, selected the wrong tool, lost important context, or stopped before verifying the result.

The improvement loop looks like this:

1. Run the current harness on development tasks.

2. Inspect the traces and task scores.

3. Modify the harness code.

4. Run the updated version.

5. Keep the version that performs better.

The model stays fixed throughout the process. Any improvement comes from changing the system around the model.

All five agents improved their results on the tasks used for feedback. The gains on held-out tasks were much smaller, averaging 3.11 percentage points.

Better Feedback Scores Can Hide Worse Harnesses

A higher development score doesn’t necessarily mean the harness became more capable.

Across 64 harness updates, development and held-out scores moved in the same direction only 53.1% of the time. Only two of the nine final versions selected from development results were also the best versions on unseen tasks.

Some updates added hundreds of lines while leaving functions unused. Other updates tuned prompts, budgets, or stopping rules so closely to one executor model that performance dropped when another model ran the same harness.

One coding harness scored 69.3% with the model that created it and 33% after the executor changed. A search harness saw its duplicate-query rate jump from 10.1% to 88.2% under another model.

The harness and model had learned to depend on each other.

Generated Harnesses Need Their Own Test Suite

The 14.4-point gap between two harnesses running the same model shows how much orchestration code can affect agent performance. HarnessDev shows that agents can close part of that gap by writing the orchestration code themselves.

The held-out results also show what your evaluation process must protect against.

If an agent edits its own harness, the updated version should pass:

• Held-out regression tests that were never included in the feedback loop.

• Executor-transfer tests that run the harness with another compatible model.

• Cost checks that measure tokens and tool calls alongside task success.

• Reachability checks that catch generated code that never runs.

• Sandboxed execution because the generated harness is executable code.

An agent can write the harness it runs on. Whether a fix it wrote after one bad run helps the next one is a different question.

HarnessDev paper→

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.