pyre-code: Real-Time ML Coding Practice for Attention, Diffusion & RLHF

Let’s be honest: most ML coding practice platforms feel like doing push-ups in a gym where the weights are glued to the floor. You know you should train on real concepts—attention mechanisms, diffusion sampling, RLHF loops—but what’s out there? LeetCode’s ML section? Too shallow. Hugging Face’s courses? Great, but no instant feedback on your gradient updates. Colab notebooks? Sure—until you hit quota limits, lose state, or realize you just spent 45 minutes debugging a torch.compile() error in a browser tab. Enter pyre-code: a self-hosted, browser-native ML practice platform with 68 hand-crafted problems—from ReLU derivations to flow matching—and real-time, code-level feedback baked right into the UI. No cloud API dependency. No hidden eval server. Just your machine, your browser, and a Python runtime doing live gradient checks, shape validation, and even forward-pass tracing as you type. At 562 GitHub stars (as of May 2024), it’s small—but it’s sharp, opinionated, and built by someone who’s graded 200+ ML assignments. I’ve run it locally for 11 days straight on a 2021 Mac Mini (M1, 16GB RAM), and honestly? It’s the first tool I’ve not shut down after a weekend.

What Is Pyre-Code — And Why It’s Not Just Another ML Quiz App

pyre-code isn’t a flashcard app or a multiple-choice quiz. It’s a live coding playground where each problem ships with:

• A minimal, annotated starter template (e.g., def attention_forward(q, k, v, mask=None): ...)
• A hidden test suite (test_attention_forward.py) that runs inside your browser via Pyodide (yes—real Python in WebAssembly)
• Real-time feedback: type return q @ k.T, and it’ll instantly tell you your output shape is wrong and why (expected [B, H, T, T], got [B, H, T, D])
• Optional backend validation: when self-hosted, you can enable a fast, local uvicorn-based evaluator that runs full torch.compile()-compatible forward/backward passes on your hardware

The problems span five core areas:

• Foundations: ReLU gradients, softmax Jacobians, batch norm backward pass
• Attention & Transformers: Multi-head attention, RoPE, causal masking, flash attention sketch
• Training & Optimization: LRSchedulers, gradient clipping, mixed-precision training loops
• Diffusion & Generative Models: DDPM forward/backward, classifier-free guidance, flow matching loss
• Alignment & RLHF: Reward modeling, PPO value head integration, DPO loss derivation

Unlike fast.ai's notebooks or HuggingFace Learn’s guided tutorials, pyre-code forces implementation, not just interpretation. And unlike Kaggle Learn, there’s zero submission latency—you’re not waiting for a GPU queue. You’re editing, running, and failing in <200ms.

How to Self-Host Pyre-Code in <5 Minutes

The repo ships with a battle-tested docker-compose.yml, and—here's the kicker—it just works on Apple Silicon and x86-64 without modifications. No --platform linux/amd64 hacks. No CUDA version pinning headaches.

First, clone and check versioning:

git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
git rev-parse --short HEAD  # f8a3b2d (v0.3.2 as of May 2024)

Then launch with Docker Compose:

# docker-compose.yml (customized for low-RAM environments)
version: "3.8"
services:
  pyre:
    build: .
    ports:
      - "8000:8000"
    environment:
      - PYRE_ENV=prod
      - PYRE_DEBUG=false
      - PYRE_MAX_PROBLEMS=68
    # Critical: limit memory to avoid OOM on small instances
    mem_limit: 2g
    deploy:
      resources:
        limits:
          memory: 2g
          cpus: '1.0'

Run it:

docker-compose up -d --build
# Wait ~10 sec, then visit http://localhost:8000

That’s it. No Python venv setup. No pip install -e .. No torch version guesswork—the Dockerfile pins torch==2.3.0+cpu (or +cu121 if you --build-arg CUDA=1). I tested it on a $5/month Hetzner Cloud CX11 (2 vCPU, 2GB RAM) — it starts in 8.2s and idles at ~380MB RAM, spiking to ~1.4GB only during actual problem evaluation (e.g., running a full diffusion step with torch.compile enabled).

Pyre-Code vs. Alternatives: Where It Wins (and Where It Doesn’t)

Let’s cut the marketing fluff. Here’s how pyre-code stacks up against real tools devs actually use:

The big differentiator? Feedback granularity. In problem #42 (“Implement Classifier-Free Guidance”), pyre-code doesn’t just check if output.shape == (1, 3, 64, 64). It injects hooks, checks:

• That uncond_logits is computed before cond_logits (to validate correct CFG masking)
• That scale is applied after subtraction, not before
• That no .detach() leaks into the guidance computation

That level of surgical feedback doesn’t exist anywhere else outside of grad-school TA grading scripts.

Why Self-Host? Who Actually Needs This?

Let’s be blunt: if you’re prepping for a FAANG ML interview next week, pyre-code is overkill. Use LeetCode. But if you’re:

• A junior ML engineer who’s read The Annotated Diffusion Model three times but still blanks on writing qkv projection splits
• A professor or TA building a course lab (I’ve seen it used at ETH Zurich’s “Deep Learning Systems” course)
• A self-taught dev who’s glued to LLMs but needs to relearn how torch.nn.MultiheadAttention actually backprops
• Or just someone who’s sick of AttributeError: 'NoneType' object has no attribute 'grad' and wants a sandbox where that error is caught before the backward pass

…then self-hosting pyre-code is a no-brainer. You get:

• Zero telemetry: no POST /analytics calls (I checked the network tab and the source)
• Full problem source access: every problem/042_cfg.py is editable—tweak difficulty, add your own test cases
• Offline mode: I ran it on a train (no Wi-Fi) for 2.5 hours—still worked. Pyodide cached everything.
• Integration-ready: expose /api/submit to your internal LMS or grade it via GitHub Actions (they ship a test_all.py script)

And yes—it’s MIT licensed. No “community edition” vs “pro” paywalls. The entire codebase is ~12k lines. I counted.

System Requirements & Resource Realities

The README says “works on a Raspberry Pi 4” — and it technically does. But let’s talk practicalities.

✅ Minimum viable:

• CPU: 2 cores (x86 or ARM64)
• RAM: 2GB (with swap enabled if on Pi)
• Storage: 1.2GB (Docker image + cached Pyodide + problem assets)

⚠️ Recommended for smooth experience:

• CPU: 4+ cores (for parallel problem evaluation + background torch.compile)
• RAM: 4GB+ (avoids constant container restarts during multi-problem eval)
• GPU: Not required, but if you have one and want +cu121, add --build-arg CUDA=1 and set NVIDIA_VISIBLE_DEVICES=all in docker-compose.yml

I ran pyre-code on three machines:

• Mac Mini M1 (16GB RAM): 100% smooth. htop shows 1.1GB RAM, 12% CPU at idle, <150ms eval latency
• Hetzner CX11 (2GB RAM): Works, but disable torch.compile in pyre/settings.py (COMPILE_ENABLED = False) — otherwise OOM kills the container
• Raspberry Pi 4 (4GB RAM): Starts, but Pyodide startup takes ~9s and evals avg 1.8s. Usable for learning, not for rapid iteration

No support for Windows Subsystem for Linux (WSL) yet — the Pyodide build fails on WSL2’s older glibc. The maintainer says it’s on the v0.4.0 roadmap.

The Honest Verdict: Should You Deploy It?

Yes — if you’re serious about building, not just consuming, ML systems.

I’ve used pyre-code daily for 11 days. My workflow: open http://localhost:8000, pick problem #57 (“Implement DPO Loss”), write 3 versions, get shape errors on v1, gradient mismatch on v2, and finally pass on v3 — all in <7 minutes. No Colab runtime disconnect. No “your notebook exceeded memory limits”. No waiting for GitHub Actions to run tests.

The rough edges? They’re real, but manageable:

• No user accounts or progress sync: it’s stateless. Your localStorage saves progress, but clear your browser cache → poof. (Workaround: I dump localStorage to JSON nightly with a cron + curl script.)
• No video/audio explanations: it assumes you’ve read the paper or watched the lecture. This isn’t a course — it’s a gym.
• Docs are light: the README.md is great for setup, but problem-specific hints live in problem/XX_name/hint.md — and some are sparse. (I’ve opened PR #142 to add inline hints in the UI — it’s merged in v0.3.3, rolling out next week.)
• No mobile support: the UI breaks on <768px. Not a dealbreaker — this is a desktop dev tool.

The biggest win? It forced me to actually write and debug a full flow matching loss (v = φ(x), ∂v/∂x, ∫‖v - f(x)‖² dx), not just call flowmatching.loss(). And when my implementation passed, the feedback wasn’t “✅ Correct” — it was ✅ Gradient norm: 0.0012 (within 1e-3 tolerance).

That’s the difference between memorizing and owning the math.

So — is it worth deploying?
TL;DR: If you’ve ever stared at a RuntimeError: expected scalar type Float but found Half, and thought “I need to practice this until it’s muscle memory” — yes. Deploy it. Run it. Break it. Fix it. Do it again. It’s 562 stars worth of focused, self-hosted, no-BS ML practice. And right now, that’s rarer than a working torch.compile() on M1.