An independent developer has brought RNNs back onto the stage of large models.

RNN Is Back Again—This Time Built by One Person

On July 15, an independent developer posted a preprint on r/MachineLearning, introducing an architecture named DABSN (Dynamic Adaptive Bias State Network). The author had been working on it solo for several months; after finishing the architecture paper, he trained a 24M‑parameter language model. To his surprise, the results exceeded expectations, prompting him to make a post: “Looking for collaborators for scaled‑up training and independent replication.”

The news itself isn’t big, but it’s notable in the context of 2026—three years after RWKV’s “Reinventing RNNs for the Transformer Era.” After Mamba, RetNet, RWKV‑7, and xLSTM took turns topping benchmarks, cracks have begun to appear in the Transformer’s dominance. This time, however, the person driving the effort isn’t a 27‑institution consortium or a big‑tech lab—it’s an independent developer with no organizational suffix at all.

What Exactly Is DABSN

Let’s unpack the terminology. The core of DABSN is a recurrent cell; as with a traditional RNN, its hidden state updates along the time dimension, avoiding the O(n²) global matching of self‑attention. But compared to a vanilla RNN, it does two additional things:

• Dynamic Bias: The bias term inside the cell isn’t a fixed constant learned at training time—it’s computed in real time from the current input and historical state. In other words, the model decides at every step “what to remember, what to forget, what to amplify.” This belongs to the same family of ideas as the GRU’s update gate and Mamba’s selective scan—making state updates input‑dependent rather than tied to fixed weights.
• Adaptive State Network: The state itself isn’t a single vector—it’s structured into multiple subspaces that evolve independently, with a learned routing mechanism deciding where information flows. It’s somewhat like moving a Mixture‑of‑Experts idea inside the RNN’s hidden state.

The goal is clear: solve the long‑term memory problem while keeping RNN‑level constant inference cost. In the preprint, the author tackled a few of the “Olympiad problems” of recurrent architectures:

• MQAR (Multi‑Query Associative Recall): Multi‑query associative memory, introduced in the Mamba paper to stress‑test state‑space models; it checks whether the model can store multiple key‑value pairs in finite state.
• Copy Task: Feed a sequence and see if the model can reproduce it verbatim—tests pure memory capacity.
• Key‑Value Retrieval: Explicit key‑value lookup.
• A5/60: A combinatorial reasoning task on the permutation group A5, now a common benchmark for long‑sequence research because it cleanly separates “state‑evolution ability” from “attention retrieval ability.”

The author didn’t post full numerical results but concluded that on tasks where RNNs traditionally crash, DABSN held up. That’s what gave him the confidence to move on and train a language model.

24M Parameters, 1B Tokens—Why It’s Worth Discussing

Many people see “24M parameters, 1 billion‑token pretraining, GPT‑2 tokenizer” and instinctively close the page: looks like a toy.

It is a toy—but the purpose of the toy is falsification.

In recent years, many new architectures followed the same failure path: nice small‑scale scores, but once scaled to 100M, 1B, or 7B parameters, they collapse; the loss curve diverges from the Transformer’s, and the larger you go, the worse it gets. So for any new architecture, its first milestone isn’t “beating GPT‑4,” but “training to convergence, stable loss curve, expected behavior.” DABSN has passed that stage—it proved it’s not just a paper architecture.

More importantly, the author observed a few interesting things, hinted at by the three directions he plans for his second paper:

1. Language modeling: Standard next‑token prediction performs competitively.
2. Long‑context behavior: Extrapolating over long contexts—the classic home turf of recurrent models and the Achilles heel of Transformers.
3. Scaling: He intends to study the shape of its scaling laws.

The third point carries the most signal. If a 24M‑parameter model shows early signs of loss curves obeying a consistent relationship across sizes, that means it may scale predictably. If this holds, DABSN stops being an academic curiosity and becomes a serious engineering candidate.

How One Person Pulled It Off: PyTorch, C++, Triton

A small but telling detail—the author released three implementations: PyTorch, C++, and Triton.

That level of completeness is rare for a solo project. The PyTorch version serves academic reproducibility, the Triton kernel is for GPU training, and the C++ one is likely for CPU inference or embedded deployment. Taken together, it means this person isn’t just a paper‑writer—he builds from the kernel layer up.

It’s reminiscent of RWKV’s first author Bo Peng—physics background, twenty years of coding, sells lighting equipment by trade, built the architecture and CUDA kernels alone in spare time. Though DABSN’s author remains anonymous, the code structure suggests the same species: an independent developer comfortable with algorithms, systems, and engineering alike.

That also explains why he’s seeking collaborators now. One person can train a 24M model, but scaling to a billion parameters requires serious compute, data pipelines, evaluation infrastructure, and baseline alignment—issues of money and manpower, not intellect.

The Three Years of Recurrent Architectures—A Quick Recap

Putting DABSN on the timeline clarifies things:

• May 2023: The RWKV joint paper formalized “re‑writing Transformers as recurrent matrices,” the first large‑scale proof that RNN‑type architectures can serve as LLMs.
• Dec 2023: Mamba advanced this line with selective state‑space modeling, prompting an 18‑month surge of SSM/linear‑recurrence research.
• 2024: xLSTM, GLA, RWKV‑6, and Griffin appeared; the paths diverged—some took selective scans, others linear‑attention variants, others gated RNNs.
• 2025: Hybrid architectures (Transformer + recurrent layers) entered production; Mistral, AI21, and DeepMind all released work.
• Early 2026: Pure recurrent models at 7B scale began to appear, but mainstream pretraining still relied on Transformer or hybrid designs.

DABSN cuts in at an interesting point—it isn’t hybrid nor a Mamba copy, but reworks the cell internals around dynamic bias. Whether it can beat existing players isn’t clear yet, but at least it’s not reinventing an existing wheel.

Does It Matter That an Independent Developer Built It?

Yes—more than people might think.

Architectural innovation in large models has been dominated by big companies—not because they’re smarter, but because training a 7B model just to see if it converges costs six‑figure dollars. That barrier keeps most “idea‑rich but cash‑poor” researchers out.

Yet breakthroughs often come from those unbound by the dominant paradigm. RWKV was one; Mamba’s early selective‑scan wasn’t considered a promising mainstream route either. DABSN now sits in that fragile spot: the architecture looks promising, but the sole developer can’t afford large‑scale training.

So his call for help is practical, not publicity‑seeking:

• Independent replication: Others run his code to verify results aren’t implementation quirks.
• Scaled‑up training: People or organizations with compute push it to 100M, 1B parameters, or beyond.
• Comparative baselines: Evaluate against Mamba, RWKV‑7, and Transformer under the same data and budget.

Any of these would move DABSN forward. If none happen, it stays a preprint—joining the hundreds of new architectures that die annually on arXiv.

How to Tell If a New Architecture Is Worth Following

For technical readers, it’s better to have a quick‑filter framework than wait for big‑firm endorsements. For early projects like DABSN, I’d check:

1. Runnable code: Not just “code exists,” but “it runs and reproduces the paper’s curves.” DABSN passes this—three tiers of open‑sourced implementations.
2. Benchmark choice: MQAR, Copy, KV Retrieval, A5/60—none random; they’re the hardest for recurrent models. Willingness to target weak spots shows real domain awareness.
3. Honesty about failures: Good papers state what they can’t do. The current preprint shows mainly successes; whether the second paper self‑critiques is a test of seriousness.
4. Scaling behavior: The ultimate filter. Anything that looks good at 24M may collapse at 1B. What DABSN most needs now is a serious large‑scale run.

Unlikely to Disrupt Soon—but Worth Watching

In short: DABSN won’t dethrone Transformers yet—24M parameters and 1B tokens are far too small for firm conclusions. But it did several things right: three implementations, four hard benchmarks, a clear next‑step roadmap, and an open, down‑to‑earth independent author.

For architecture researchers, it’s something you can clone and run now. For application builders, it can wait—check back once it’s trained to 1B or independently replicated. The first signals of a new architecture never come from press releases—they come from GitHub commits and arXiv v2 updates.

The upcoming second paper will cover language modeling, long‑context behavior, and scaling. If its curves still hold, DABSN joins the “track seriously” list; if not, it becomes another footnote in the 2026 RNN revival wave.

Three years ago RWKV began as one person’s project; today it’s a multi‑institution open‑source effort ranking steadily on Chatbot Arena. Whether DABSN follows that path depends on finding collaborators willing to burn compute over the next few months. Independent developers are seldom short of ideas—they’re short of people willing to run those ideas.

By the way, OpenAI Hub (openai‑hub.com) acts as an aggregation platform: one key accesses GPT, Claude, Gemini, DeepSeek, and other major models via the OpenAI‑compatible API—directly accessible within China. If you want to benchmark these mainstream models against new architectures, you can switch models in a single codebase without separate API registrations.

References

• Seeking collaborators for scaling and independent evaluation of a new recurrent language model architecture — The original preprint and collaboration request post by the DABSN author on r/MachineLearning, including architectural explanation and code links.