Reduce AI Agent Token Costs by 95%

... PLUS: Build AI Agents That Work Independently In Production

Author

Sumanth P
July 02, 2026

In today’s newsletter:

  • Glean: Build AI Agents That Work Independently In Production

  • Headroom: Reduce AI Agent Token Costs by 95%

Reading time: 5 minutes.

Most AI agents today are reactive. They wait for a prompt, answer a question, complete a task, and stop.

That works well for chatbots and copilots, but many workplace tasks don't begin with someone asking a question. An incident needs investigating. A support ticket needs routing. A deployment fails. A document requires approval.

The work starts before anyone opens a chat window.

Glean's Independent Agents are built around that idea. Instead of waiting for instructions, they monitor work, gather context, take action within defined guardrails, and collaborate with people across the tools they already use.

Beyond Chat

Most enterprise AI today follows the same pattern:

User → Prompt → Response

The agent only exists for the duration of that interaction. Once the conversation ends, so does the agent's involvement.

Independent Agents take a different approach. They persist beyond a single chat session and can work across applications like Slack, Jira, and Microsoft Teams while carrying context between them. Instead of being tied to one interface, they're designed to participate wherever work happens.

What Makes Them Independent?

Every Independent Agent is built around four capabilities:

  • Identity: Each agent has its own identity and permissions instead of borrowing a user's access. That makes its actions traceable and allows administrators to control exactly what it can see and do.

  • Memory: The agent learns from company documentation, runbooks, and previous interactions, improving its behavior over time.

  • Proactivity: Rather than waiting to be asked, the agent can investigate issues, suggest next steps, notify teammates, and surface useful information when it's needed.

  • Accountability: Every decision and tool call is recorded, with governance controls and an emergency stop if administrators need to disable the agent.

What makes all four work in practice is the context layer underneath. Glean indexes knowledge across an organization's entire stack - Jira, Confluence, GitHub, Slack, Google Drive, and 100+ other connectors - building a knowledge graph that understands relationships between documents, people, projects, and decisions. The agent doesn't just have access to data. It understands how everything connects.

Together, these capabilities make the agent behave less like a chatbot and more like a teammate assigned to a specific responsibility.

An Example

Glean's first Independent Agent is an OnCall Assistant for engineering teams.

When an alert is triggered, the agent gathers logs, reviews documentation, investigates possible root causes, drafts a potential fix, and identifies the right engineer to involve. By the time someone joins the incident, much of the initial investigation has already been completed. As it handles more incidents, it also learns from company procedures and previous resolutions to improve future investigations.

A Different Way to Think About Agents

Much of the conversation around AI agents has focused on making them better at answering questions. Glean suggests another direction: building agents that own part of a workflow rather than part of a conversation.

That requires more than reasoning. It requires identity, memory, permissions, and governance so the agent can operate independently without losing accountability.

As agents become more capable, the question shifts from "How do I chat with an agent?" to "What work can I trust an agent to own?" Independent agents are where that answer starts.

AI agents are getting smarter, but they're also consuming more tokens. Every tool call, database query, log file, RAG retrieval, and code search gets sent to the model as context. Much of that content is repetitive boilerplate, yet you still pay for every token the model processes.

Headroom tackles this problem by compressing that context before it reaches the model, reducing tokens by 60–95% while preserving the information the agent needs to complete its task.

How It Works

Headroom sits between your AI agent and the LLM. Instead of forwarding raw tool outputs directly to the model, it analyzes the content and chooses the best way to compress it.

The pipeline has three main components:

  • CacheAligner stabilizes prompts to improve provider KV-cache hits.

  • ContentRouter detects whether the input is code, JSON, logs, or plain text and routes it to the most suitable compressor.

  • IntelligentContext ranks the information by importance, fits it within the available token budget, and keeps the original content available if it needs to be retrieved later.

The result is a smaller prompt that still contains the information the model needs to reason effectively.

Where It Helps

Context compression becomes more valuable as agents perform more work. For example, imagine a coding agent that:

  • Searches a large repository

  • Reads multiple files

  • Runs unit tests

  • Collects compiler logs

  • Queries documentation

Without compression, every one of those outputs contributes to the prompt sent to the model. Headroom compresses those intermediate results before they're counted as input tokens, reducing both cost and context size.

Headroom supports several integration options depending on your stack.

You can use it as a Python library, run it as a transparent proxy with no code changes, or wrap existing tools such as Claude Code, Codex, and Aider. It also works with LangChain, CrewAI, LiteLLM, MCP, Cursor, and other agent frameworks.

The Bottom Line

Prompt engineering helped developers communicate better with language models. Retrieval engineering helped them find better context. Context optimization asks a different question:

Does the model need to read everything you're sending it?

As agents spend more of their time reading tool outputs instead of responding to users, optimizing context may become just as important as optimizing prompts.

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.