In today’s newsletter:

• Fine-Tune Muse Glimmer 30B

• Cut Agent Output Tokens by 65%

Reading time: 5 minutes.

Fine-Tune Muse Glimmer 30B

Muse Glimmer 30B is the first open model from Meta Superintelligence Labs. It's a dense 30B model with a built-in vision encoder, a 131K context window that extends to 262K, and controllable reasoning effort across four levels.

It was built for local agentic work specifically, so it plans multi-step tasks, chains tool calls, recovers from failures, and reads images alongside text. On agentic benchmarks it leads Gemma4-31B and Qwen3.6-27B on MCP Atlas at 75.5, DeepSearch QA at 74.6, and SWE-Bench Pro at 51.2.

Fine-tuning a model that size has normally meant renting a data center GPU like an 80GB A100.

Unsloth changed that. Their optimizations bring Muse Glimmer down to a single RTX 4090 or 3090, and they've published free Kaggle notebooks that run it on Tesla T4s at no cost at all. Training comes out around 1.5x faster while using about half the VRAM of an FA2 setup, with no accuracy loss.

Start With One of the Free Notebooks

The fastest path is Kaggle, which gives 30 free GPU hours a week on two Tesla T4s. Unsloth ships three ready notebooks, and each one is a different training setup rather than a different model:

• Conversational. Standard instruction tuning on text.

• Vision and Text. Adds image inputs, for screenshots, documents, charts, and GUI tasks.

• RL GRPO. Reinforcement learning with reward functions instead of labeled answers.

Open one, point it at your dataset, run it. Nothing to install.

Loading the Model Yourself

On your own card, the whole memory story lives in the arguments you pass at load time:

load_in_4bit does the heavy lifting, taking the weights from 56GB to 21GB.

But 21GB on a card as small as 24GB leaves nothing to train with. offload_embedding frees up another 2.5GB.

Muse Glimmer has a 202,048-token vocabulary, so its embedding table is enormous, and any single batch reads only a handful of rows from it. offload_embedding keeps that table in system RAM and copies over just the rows a batch asks for.

That returns 2.5GB, taking the loaded model from 20.72GB to 18.22GB.

Adapter and Training Settings

The recommended starting point is a rank 16 adapter:

r = 16
lora_alpha = 16
lora_dropout = 0

Bump both to 32 for harder agentic tasks, but start at 16 and only raise it if the results ask for it.

For the trainer itself:

trainer = SFTTrainer(
    max_seq_length = 4096,
    packing = True,
)

The rest of the recommended configuration is batch size 1 to 8, gradient accumulation 4 to 16, learning rate 2e-4, one to three epochs, AdamW, and BF16.

Start with a short context and raise it gradually. Muse Glimmer handles 131K, but training at that length on a consumer card is not where you begin.

When it's done, save the adapter and optionally merge it:

model.save_pretrained("muse-glimmer-lora")
model.merge_and_unload()

From there you can export to GGUF, NVFP4, or FP8 and run it locally.

GRPO Runs on the Free Tier Too

Reinforcement learning on a 30B model should be out of reach on free hardware, but the GRPO notebook runs on Kaggle's two 16GB T4s.

At two generations per prompt and a 1536-token sequence length, the run needs 4.10GB. Double the sequence length to 3072 and it needs 7.57GB, which no longer fits.

One setting changes the difference between running and not running, so treat that notebook as a starting point rather than something you scale up casually.

Watch Your Dataset Mix

Working inside those margins pushes you toward short sequences and short examples, and this brings up another constraint.

Muse Glimmer's reasoning effort is controllable across four levels, and fine-tuning only on short direct answers degrades its multi-step reasoning. The fix is to mix reasoning-style examples in with the direct ones.

Decide that before you start, because a run that quietly weakened your model still finishes and still looks fine.

Read the Fine-tuning Guide →

Cut Agent Output Tokens by 65%

You pay for every token your coding agent writes back to you, including the filler. Openers like "Sure! I'd be happy to help you with that" and hedges like "most likely caused by" get billed at the same rate as the fix.

Caveman is a skill that strips that out. Install it and the agent replies in fragments, while keeping code, commands, and error strings byte-for-byte exact.

It runs on Claude Code, Codex, Cursor, Windsurf, Cline, Copilot, and around 30 other agents.

The headline number comes from 10 prompts measured against real Claude API token counts. Output dropped 65% on average, ranging from 22% on a callback refactor up to 87% on a React error boundary explanation.

The 65% Only Covers Output

Caveman only shrinks output. Your prompts, your files, your context, and the model's thinking tokens all stay exactly the same size. The skill also carries a fixed cost of roughly 1,000 to 1,500 input tokens per turn, because its rules get injected into context on every single one.

So the arithmetic flips on short replies. If your normal answers run around 150 output tokens, caveman saves maybe 100 of them and charges you a thousand or more for the privilege. That's a net loss, and the docs say so plainly, crediting the user who measured it.

It also can't help if your agent bills per request instead of per token. A shorter Copilot answer is still one premium request.

The repo's own rule of thumb is the practical version. If your typical reply runs longer than 1,500 to 2,000 output tokens, caveman probably saves you money. Shorter than that, it probably costs you.

Shorter Answers Also Score Better

There's a second reason to run it that has nothing to do with cost. A March 2026 paper tested brevity constraints across 31 models and found that forcing large models to answer briefly improved accuracy by about 26 points on some benchmarks. The padding isn't free for the model either.

So install it if your agent writes long explanations, and skip it if your replies are already short or your agent charges per request.

Check your own usage page after a week rather than trusting the 65%.

Caveman On GitHub →

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.