Agentic data scientist to create high quality synthetic data

In today’s newsletter:

• ZenMux: Know Which Model Is Worth Your Tokens

• Build Better Synthetic Data with AI Agents

Reading time: 5 minutes.

Know Which Model Is Worth Your Tokens

AI benchmarks are great for comparing capabilities, but they don't always tell you which model you'll actually enjoy using.

The model with the highest score isn't always the one that offers the best balance of quality, speed, reliability, and cost for your workload.

ZenMux's Token Economics Arena is built around that idea. Instead of ranking models with benchmark scores, it lets real usage determine which models developers keep coming back to.

How It Works

The campaign includes more than ten leading AI models across coding, AI agents, long-context reasoning, multilingual tasks, and content creation.

Every 1 million tokens you use counts as one vote for that model. At the end of the campaign, ZenMux awards two titles:

• Most Used Model: the model with the highest token usage.

• People's Choice Model: the community favorite.

Rather than asking developers to vote directly, the leaderboard reflects how models are actually used.

Why It Matters

Benchmark scores measure capability under controlled tests. Production workloads are different.

The best model for writing code may not be the best for running agents. A model with slightly lower benchmark scores may produce better outputs for your use case, respond faster, or simply cost less to run.

The Token Economics Arena focuses on that practical side of model selection: which models developers choose when they're solving real problems.

Lowering the Cost to Experiment

To make comparison easier, ZenMux is offering many popular models at discounts of up to 80% during the campaign.

That includes models such as GLM 5.2, Kimi K2.7, Seed 2.1 Pro, and Qwen 3.7, making it much cheaper to try models that many developers may not have used before.

Instead of deciding based on benchmark charts alone, you can test the models on your own workloads and see which one performs best for you.

The Takeaway

Choosing an AI model has become less about finding the highest benchmark score and more about understanding trade-offs.

Benchmarks show what a model can do. Real usage shows which models developers actually choose to build with.

Browse discounted models →

Build Better Synthetic Data with AI Agents

Today, most synthetic data pipelines follow a fixed workflow. Engineers design prompts, generate examples, filter low-quality outputs, and build training datasets. Once the pipeline produces acceptable results, it usually stays unchanged.

Meta's new research asks a different question:

What if an AI agent became the data scientist?

Instead of following a fixed pipeline, the agent plans how to generate data, evaluates the results, uses tools when needed, and continuously improves the dataset it produces.

The system is called Autodata.

The Problem

Synthetic data has become one of the most important ingredients for training modern language models. But most generation pipelines are static.

Developers handcraft prompts, define filtering rules, and tune generation parameters. If the resulting dataset isn't good enough, they manually adjust the pipeline and try again.

The pipeline improves only when a person improves it.

How Autodata Works

Autodata treats synthetic data generation as an agentic task. Instead of executing a predefined pipeline, it gives an AI agent the goal of building high-quality training and evaluation data.

The paper introduces an implementation called Agentic Self-Instruct, which extends the classic Self-Instruct method.

Rather than simply generating instruction-response pairs, the agent can plan multiple steps ahead, use external tools, evaluate intermediate results, and refine its own data generation strategy before producing the final dataset.

In other words, the pipeline isn't fixed. The agent decides how to improve it.

What Changes

The interesting idea isn't that AI generates synthetic data. Models have been doing that for years. The interesting idea is that the process of generating synthetic data also becomes adaptive.

Instead of repeatedly tuning prompts by hand, developers can optimize the agent responsible for producing the data.

As the agent improves, the data improves with it.

The paper describes this as meta-optimization, optimizing the system that creates the training data, rather than only optimizing the training data itself.

The Results

Meta evaluated Autodata across several domains, including:

• Computer science research

• Legal reasoning

• Mathematical reasoning

Across these tasks, Agentic Self-Instruct consistently outperformed traditional synthetic data generation methods.

The researchers also found that improving the data-generation agent itself produced even larger gains, suggesting that better agents can create better datasets without redesigning the entire pipeline.

Looking Ahead

Agentic AI is often discussed in the context of coding, research, and automation. This paper applies the same idea to another part of the machine learning stack: data generation.

Instead of treating synthetic data pipelines as workflows that engineers occasionally update, Autodata treats them as systems that can continuously improve through an AI agent.

If that approach continues to scale, the next generation of training datasets may be built less by handcrafted pipelines and more by agents that learn how to create better data over time.

Read the paper →

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.