Turn Any GitHub Repository Into an RL Environment

In today’s newsletter:

• Maximem: Give AI Agents Long-Term Memory

• Turn Any GitHub Repository Into an RL Environment

Reading time: 5 minutes.

Give AI Agents Long-Term Memory

Your agent worked perfectly in the demo. It answered questions, pulled the right context, called tools without a hitch. You deployed it. A week later, complaints start coming in. People start asking questions like:

"Why does it keep forgetting what I told it?" "It gave me information I corrected last month." "Every session feels like starting from scratch."

The first response is usually to add a vector database. You store conversations, generate embeddings, set up retrieval and with that recall improves, but the agent still gives wrong answers.

This is where most teams get stuck. The vector database is working. The similarity scores look correct. But it keeps retrieving the wrong information: an outdated fact, a preference that changed, two references to the same person treated as two different people.

Adding better chunking, larger context windows, more retrieval passes. None of it gets to the root. Storage is not the issue. Nothing is deciding which facts are still accurate, which memories actually matter, and what the agent should see before it responds.

Maximem Synap solves that gap. It sits between your agent and its memory as the context management layer, deciding what to retain, how to keep it accurate as facts change, and what to surface at the right moment. 

By the end of this post, you will understand what long-term memory actually requires for production agents, where most approaches fall short, and how to build it right.

What Long-Term Memory Actually Means

A context window is what your agent sees right now, in this session. Long-term memory is what persists after that session ends and needs to be accurate weeks later.

The difference shows up in production quickly. A support agent needs to remember a customer's preferences from three months ago. A research agent needs findings from dozens of sessions. A coding agent needs architectural decisions made six months earlier. The context window provides visibility for today. Memory provides continuity across time.

Most teams discover this gap only after deployment. A bigger context window does not close it. It just gives the agent more room to surface outdated, irrelevant, or contradictory information.

Where Most Approaches Fall Short

The first attempt at memory is usually a vector database: 

1. Store conversations: log each exchange to the database

2. Generate embeddings: convert text into vectors for semantic search

3. Retrieve the most similar chunks: surface what matches the current query closest

For prototypes, this works. The problems appear once users start interacting over time.

A user tells your agent they live in England. Two months later they say they moved to France. Both facts now exist in the database. A similarity search retrieves both with nothing to determine which one is current.

It is not just location. Every time a user updates anything, both the old and new version live in the same database. None of those corrections erase what came before. Over time, the agent is working from a memory full of contradictions with no way to know which version of anything still reflects reality.

Not all information works the same way. A user preference needs to be treated differently from a fact. A fact is different from an event. An event is different from a relationship between two people. Most memory systems ignore those distinctions and store everything as text chunks. When the agent searches for a preference, it searches through facts, events, and relationships at the same time. The right answer is in there. So is everything else.

The Gap Most Memory Systems Leave Open

Even when retrieval works, there is a harder problem underneath. Deciding what context should actually appear in front of the model.

For instance, take a 128K context window. Before the user's message even arrives, tokens are already being used by the system prompt, conversation history, tool descriptions, retrieved documents, and skill instructions. In a typical setup that alone can consume 40K tokens. Everything the agent actually needs for the current task has to fit in what remains.

Nothing in a standard retrieval pipeline manages that. It retrieves by similarity. It does not know what is still accurate, what matters right now, or what the user already corrected last week. Most memory systems leave that decision completely unhandled. That is the gap.

How Synap Builds Long-Term Memory

Maximem Synap is an agentic context management layer that sits between your agent and its memory. It is the decision layer that determines what is worth keeping, whether a stored fact is still current, and what the agent should actually see before it responds.

Most memory systems accumulate. When a user updates a preference, both the old and new version sit in the same database. Synap classifies information before storing it, which means a preference update touches the preference record directly. The outdated version is replaced, not stored alongside the new one.

The same issue appears with how agents track people. A user refers to their contact as "Jamie" in one session and "the project lead" in another. Without a way to link those references, the agent treats them as two different people. Synap connects both to a single entity in the graph, so when the agent needs to recall anything about that person, it draws from one unified record.

Memory Across Sessions and Agents

Single-session memory is only part of the problem. In multi-agent systems, agents working on the same user or task should not re-derive the same knowledge from scratch, but should not share one undifferentiated context either.

Synap organizes memory across four scopes:

• User: individual preferences, history, and corrections tied to a specific person

• Customer: shared context across a user's organization or account

• Client: context specific to your product or service relationship

• World: general knowledge accessible to all agents

Each agent draws on exactly the scopes it needs. A sales agent pulls from customer history and organizational context. A support agent pulls from product knowledge and the complaint history for that specific user. When a fact gets updated in one agent, downstream agents see the corrected version automatically. Memory becomes structured and shared rather than siloed or blended into one growing blob.

The Numbers

LongMemEval and LoCoMo are the benchmarks that test exactly what we have been describing: recall across long multi-session histories where facts change, entities overlap, and context grows stale. They are the hardest conditions for a memory system to perform well in, because they replicate what production agents actually face.

Synap scores 92% on LongMemEval and 93.2% on LoCoMo. Retrieval runs at under 15ms P50. Here is how that compares to other memory systems on the same open-source benchmark:

Scoring 92% at 15ms means Synap handles both the accuracy problem and the latency problem at the same time.

Getting Started

Synap ships with 18 native framework integrations including LangChain, LangGraph, LlamaIndex, CrewAI, and Google ADK. If you already have an agent running on any of these, you are not rebuilding anything. Synap drops into your existing stack as the memory layer.

The fastest way to see it working is the playground, where you can test retrieval against your own data before writing a single line of integration code. There is a free tier that  offers accuracy evaluation on your existing agent if you want a baseline before switching anything.

Synap playground →

Synap SDK →

Turn Any GitHub Repository Into an RL Environment

Coding agents are improving fast.

The challenge is no longer just building better models. It's finding enough realistic environments to train and evaluate them. Most reinforcement learning environments are handcrafted, requiring researchers to manually create tasks, rewards, and evaluation logic.

Repo2RLEnv takes a different approach: it turns existing GitHub repositories into RL environments.

Repositories Contain Training Data

Every active repository contains a record of problems that have been solved.

Issues describe what needed to be done. Pull requests contain proposed solutions. Commits capture the implementation. Tests provide a way to verify correctness.

Together, they form the raw ingredients of an RL environment.

Repo2RLEnv extracts those ingredients automatically.

From Development History to RL Tasks

The project analyzes a repository's history and reconstructs software engineering tasks from it.

A bug fix becomes a task.

A feature request becomes a task.

A merged pull request becomes an example of a successful solution.

Instead of creating synthetic coding challenges, the environment is generated from work that developers actually performed.

What Gets Generated?

For each task, Repo2RLEnv creates:

• A starting repository state

• A goal for the agent to complete

• Automated evaluation logic

• A reward signal based on task success

The result is a reinforcement learning environment that agents can train on or be evaluated against.

Why This Matters

Most coding benchmarks are static.

Once agents perform well on them, researchers need to create new tasks or entirely new benchmarks. GitHub repositories are different. They are constantly evolving.

Every new issue, pull request, and commit can become fresh training data.

That means the software ecosystem itself can become a continuously expanding source of RL environments.

The Real Bottleneck

As coding agents improve, model architecture may stop being the primary constraint.

The harder problem may be generating enough realistic tasks, evaluations, and environments to keep those agents learning.

Repo2RLEnv suggests a different path forward: instead of building environments by hand, generate them from the repositories where real software engineering already happens.

Repo2RLEnv On GitHub →

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.