• AI Engineering
  • Posts
  • Hands-on Neural Networks: Zero to Hero by Andrej Karpathy

Hands-on Neural Networks: Zero to Hero by Andrej Karpathy

.. PLUS: A Unified Framework for Multimodal RAG

Author

Sumanth P
September 05, 2025

In today’s newsletter:

  • Neural Networks: Zero to Hero by Karpathy

  • A Unified Framework for Multimodal RAG

  • AGENTS.md – A README for Coding Agents

Reading time: 3 minutes.

Andrej Karpathy’s Zero to Hero is one of the most practical, hands-on courses for learning neural networks. It takes you from fundamentals to advanced architectures with a clear, code-first approach.

What it covers:

  • Backpropagation & micrograd - Build an autograd engine from scratch

  • Language modeling (makemore) - Train a bigram character-level model

  • MLPs, activations, gradients, BatchNorm - Core neural net building blocks

  • WaveNet-style CNNs & PyTorch internals - Convolutions and framework mechanics

  • Building GPT from scratch - Step-by-step transformer implementation

  • GPT Tokenizer - How tokenization works for LLMs

All lecture code is open source and available on GitHub.

Traditional RAG systems excel at text but fall short with diverse content.

RAG-Anything extends RAG into a multimodal pipeline with end-to-end document processing and retrieval.

Key Features:

  • End-to-End Multimodal Pipeline - From ingestion to query answering

  • Universal Document Support - PDFs, Office docs, images, and more

  • Specialized Content Analysis - Tables, equations, and heterogeneous data

  • Multimodal Knowledge Graph - Cross-modal entity extraction and relationships

  • Adaptive Processing Modes - MinerU parsing or direct multimodal injection

  • Direct Content List Insertion - Use pre-parsed lists without document parsing

  • Hybrid Intelligent Retrieval - Contextual search across text and multimodal content

It’s 100% open source and built for deep document understanding.

OpenAI released AGENTS.md a simple and open format designed for coding agents.

It provides a predictable place for instructions that help agents work on your project: setup commands, test instructions, code style, and more.

Why it matters:

  • Keeps READMEs concise and human-focused

  • Gives agents a clear, consistent source of instructions

  • Provides precise guidance without cluttering repos

  • Works across a growing ecosystem of coding agents and tools

Instead of proprietary formats, AGENTS.md is one open file that’s already adopted by 20k+ open-source projects.

That’s a Wrap

That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.

PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.

Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.

WORK WITH US

Looking to promote your company, product, or service to 150K+ AI developers? Get in touch today by replying to this email.