Someone developed a framework and trained a VLM from scratch that can play Snake.

A VLM That Can Play Snake, Backed by a Custom Training Framework

A pretty interesting project recently popped up on Reddit’s r/MachineLearning board. Developer murdock_aubry trained a vision-language model (VLM) to play Snake—not with scripts or RL agents reading memory states, but by letting the model look directly at game screenshots and output the next move direction.

Sounds like a toy project. In fact, the author admits it himself: “Using a VLM to play Snake is overkill.” But the point isn’t Snake itself—it’s the training framework FeynRL he wrote for it: an open-source scaffold meant to help developers truly understand how an LLM/VLM is trained, from the ground up.

In a world dominated by billion-parameter models costing millions of dollars to train, a “train your own toy model from scratch” project like this actually has more educational value. You can see clearly what’s happening at every step rather than treating Hugging Face’s Trainer.train() as a black box.

What the Project Actually Does

To summarize: FeynRL is an end-to-end learning framework for training VLMs, covering the entire pipeline—data prep, pretraining, SFT, RL post-training, and evaluation. The author used Snake only as a demo because it’s simple and visual—you can instantly tell whether the model “gets it.”

The training process for the Snake scenario goes roughly like this:

• Data generation: Run the Snake game programmatically, collecting a large number of (game frame, optimal action) pairs. Since the optimal Snake policy is computable (via BFS/A* pathfinding), data labeling costs almost nothing.
• Visual encoding: The model learns to interpret an 8×8 or 16×16 grid screenshot—identify where the snake’s head, food, and body are.
• Decision output: The model outputs one of up/down/left/right.
• RL fine-tuning: Optimize further using rewards based on survival time and food collected.

In the demo gif, the model reliably moves toward food and avoids hitting itself. That’s not difficult for a pure CV model, but getting it to work within a VLM architecture demonstrates that the same framework could train other “see-and-decide” tasks—GUI control, robot vision, chart QA, and so on.

Why It’s Worth Attention: It’s About the Training Pipeline, Not the Model

The year 2025 has seen visibly accelerating progress in VLMs. Hugging Face’s July VLM overview listed many new players: Qwen2.5-Omni’s Thinker-Talker architecture, Kimi-VL-A3B’s MoE decoder, Nanyang Tech’s NEO achieving parity with mainstream modular VLMs using 345 million image-text pairs. But for most developers, these models are for use, not training.

If you actually try training a VLM from scratch, you’ll hit a wall of messy questions:

• Which vision encoder to use: CLIP ViT-L/14, SigLIP, or your own?
• Is a two-layer MLP projector enough, or do you need Q-Former or LDP?
• What pretraining data to use: a 595K subset of CC3M or your own crawl?
• Should the vision encoder be unfrozen during SFT?
• How to design rewards for RL post-training—GRPO or PPO?

Each paper—LLaVA, MobileVLM, Xmodel-VLM—handles these differently, but their answers are scattered across many codebases. Reproducing one often means spending days reading code.

FeynRL’s goal is to fix that—to make every step of the VLM training procedure readable, modifiable, and composable. The author calls it a learning framework rather than a production framework—making it clear: it’s not meant for SOTA performance, but for transparency.

Architecture Guesswork: A Teaching Version of the LLaVA Paradigm

From the GitHub repo and the demo, FeynRL seems to follow the classic three-part LLaVA-style pipeline:

Vision encoder (ViT/CLIP)
        ↓
Projector (MLP)
        ↓  
Language model (LLM Decoder)
        ↓
Output token

This architecture is practically the default for open-source VLMs now. Xmodel-VLM used it for a lightweight 1B VLM; MobileVLM optimized it for mobile devices; LLaVA-1.5 achieved SOTA at 13B scale. The differences lie in which components are used and how they’re trained.

FeynRL’s educational value lies precisely here—it exposes all those design choices. Want to replace ViT with SigLIP? Change one line of config. Try LDP as a projector? Swap a module. Add LoRA? Just toggle a switch.

RL Post-training Is the Focus

The “RL” in its name suggests a strong emphasis on post-training reinforcement learning—one of the hottest trends in VLMs in 2025. Tencent’s recently released unified RL framework aims to bridge diffusion models, autoregressive models, and VLMs under a single RL training scheme. Following DeepSeek-R1, GRPO has practically become a staple of inference models.

Snake, conveniently, is a textbook RL environment—dense rewards (every step counts), short episodes (die in a few dozen moves), fully observable state space. Explaining VLM+RL in this context is far more intuitive than with something like MathVista.

Ease of Use: The Real Key

The author emphasizes “easy” repeatedly. From the repo’s examples directory, the Snake demo seems runnable end-to-end with a single consumer GPU.

Compare that to big-company VLM entry barriers:

• Qwen2.5-VL: requires assembling training scripts manually, understanding Megatron or DeepSpeed.
• LLaVA: official scripts work but are a nightmare to modify.
• NEO: its clever Pre-Buffer/Post-LLM pretraining strategy is costly to reproduce.

FeynRL isn’t competing on performance; it’s a sandbox for learners who want to tinker. Once you fully understand how each module works using Snake, architectures like Qwen or Kimi-VL start making much more sense.

Some Caveats

FeynRL will likely share the usual early-project weaknesses:

Documentation may be sparse. It’s a one-person project—getting the code open-sourced is already impressive. Don’t expect Hugging Face–level tutorials.

Scalability is questionable. Abstractions made for teaching clarity may not survive large-scale training. Try pushing it to a 3B VLM, and you’ll probably hit limits.

Dependency lock-in. Small learning frameworks often tie tightly to specific PyTorch or transformers versions—they may break in a few months.

And a more fundamental concern: Is the Snake demo representative? Snake visuals are hyper-structured—grid-based, clearly colored—making the task much easier. Real-world images (like GUIs or natural scenes) are far more complex. The demo proves the framework runs, but not that it trains useful models.

Real Value for Developers

Regardless of its production viability, the greatest value of such projects is demystification.

Many developers see multimodal models as “too big to touch”—the domain of big labs. But when someone trains a VLM end-to-end on a consumer GPU with just a few hundred lines of code and a Snake game, you realize:

• Cross-modal alignment isn’t magical; it’s a trainable projection layer.
• SFT on multimodal data is fundamentally no different from text-only fine-tuning.
• RL post-training is tricky but implementable—GRPO-style algorithms aren’t huge.

For indie developers, grad students, or engineers pivoting into AI, such “see-the-whole-picture” projects are more useful than any MOOC. Spend a week running and tweaking FeynRL’s Snake demo—swap data, change backbones—and you’ll gain deeper VLM insight than from reading ten papers.

One More Thing

If you finish training your own model and want to benchmark it against mainstream commercial VLMs—say, let GPT-5, Claude, or Gemini also play Snake to compare gaps—the OpenAI Hub supports invoking all major models (GPT, Claude, Gemini, DeepSeek, etc.) with a single key, OpenAI-format compatible, with domestic direct access—handy for unified benchmarking.

Conclusion

FeynRL won’t be the next LLaVA, nor will it top SOTA charts. But in 2026, when “everyone’s talking about big models but few have actually trained one,” a project that patiently unpacks the training process is far more valuable than another score-boosting benchmark paper.

That little green pixel snake moving square by square in the author’s gif may, in some sense, be more convincing than many high-scoring MMMU models—it at least proves the pipeline works, and that you can run it too.

References

• I trained a vision-language model to play Snake, and so can you - Reddit — Original project post by murdock_aubry, includes demo gif
• FeynRL GitHub Repository — Framework source code, examples includes full Snake training demo
• Vision-Language Models (Better, Faster, Stronger) - Hugging Face Blog — 2025 VLM field review covering Qwen2.5-Omni, Kimi-VL, Llama 4, and others
• Recent Open-Source VLM Review - Zhihu — Comparison of mainstream open-source VLM architectures and training strategies