Arena Reached $100 Million ARR in Eight Months, Proving the AI Evaluation Business Model Works

Arena Reaches $100 Million ARR in Eight Months: The AI Evaluation Business Has Proven Viable

On June 29, AI model evaluation platform Arena (formerly LMArena) revealed that its enterprise product AI Evaluations surpassed $100 million in annual recurring revenue run rate (ARR) just eight months after launch. For comparison, when the company raised a $150 million Series A at a $1.7 billion valuation in January this year, its ARR was only $30 million. That’s 3x growth in half a year.

This is a highly revealing number. In the ecosystem of tooling around large language models, evaluation has always been considered the least glamorous layer, with what appears to be the weakest technical moat. The previous version of this story was smartphone benchmark websites, many of which ultimately got absorbed by manufacturers and lost their credibility. But Arena seems to have taken a different path.

A Unicorn That Grew Out of an Academic Project

Arena’s origins trace back to 2023. The LMSYS team at UC Berkeley (whose core members included Chinese researchers such as Lianmin Zheng, Wei-Lin Chiang, and Hao Zhang) released the open-source model Vicuna, but quickly encountered an awkward problem: traditional benchmark suites like MMLU, GSM8K, and HumanEval were no longer reliably distinguishing model quality.

Their solution was simple and direct: build a website where users could input prompts freely, the system would randomly select two models to answer, and users would blindly choose the better response. Winners gained points, losers lost points. This became the now-famous Chatbot Arena. It was renamed LMArena in September 2024, and simplified again this year to just Arena.

The site’s flywheel spun up quickly:

• More models attracted more users
• More users generated richer preference data
• More data increased the credibility of the leaderboard
• Greater credibility pushed more model providers to participate

Today, more than 400 large models worldwide are rated on Arena, with over 10 million cumulative user battles recorded. OpenAI, Google, Anthropic, DeepSeek, Alibaba Tongyi, Moonshot AI—virtually every company launching a new model now submits it to Arena for scoring. Posting an Arena leaderboard screenshot during launch events has practically become standard procedure.

This de facto status matters. In its investment memo, a16z directly described Arena as “critical infrastructure for the AI industry”—and that assessment is not exaggerated.

How AI Evaluations Turns Traffic Into Revenue

How does a free website make money from enterprises? That’s the common challenge for evaluation platforms. Arena’s answer is AI Evaluations, an enterprise-grade product officially launched in September 2025.

The core logic is this: when model providers and enterprise users make decisions about model selection, training optimization, and product iteration, they need more than a single aggregate leaderboard score. They need granular data across specific dimensions:

• Where exactly does my model underperform a competitor in coding scenarios?
• In Chinese long-context tasks, which answer styles do users actually prefer?
• After this round of fine-tuning, did users genuinely perceive quality improvements or declines?
• Can you provide traceable samples and SLAs for regulatory or customer reporting?

Arena processes the human preference data accumulated from its community into consumable analytics services sold on a usage-based pricing model (founder Angelopoulos specifically clarified this point, so the $100 million figure is technically closer to annualized run rate than traditional recurring revenue).

Its customers mainly fall into two categories:

• Model labs like OpenAI and Anthropic that train their own models and need human preference data for post-training
• Enterprises embedding AI into their businesses that require model selection and ongoing monitoring

Horizontally, Arena is competing for the same budgets as companies like Scale AI, Surge, and Mercor, which focus on human annotation and post-training data. Mercor also surpassed $1 billion in revenue this year, while Handshake’s AI training business is approaching $1 billion. The entire sector built around “feeding data to” and “evaluating” large models is becoming a much larger market than previously imagined.

Arena has also launched Inclusion Arena, which embeds evaluation into real AI applications through APIs and SDKs to collect production-environment feedback. This direction arguably has even greater potential—it is effectively building a CI/CD pipeline for AI products, continuously monitoring shifts in user preferences from the day a model goes live.

Yupp Shut Down, Showing This Business Isn’t Easy

A notable comparison: Yupp, another crowdsourced AI evaluation platform, announced it was shutting down on March 31 this year.

Founded in 2024, Yupp’s model was highly similar to Arena’s—a two-sided marketplace where consumers could freely try multiple models, while enterprises paid for evaluation data. At its peak, it attracted 1.3 million users and secured paid contracts with AI labs, but it never found product-market fit and ultimately shut down.

This illustrates several points:

First, the network effects of evaluation platforms are winner-take-all. Model providers are not going to optimize scores across five different leaderboards simultaneously, and developers are not going to monitor five separate rankings. Arena’s early start, large community, and strong academic credibility turned its first-mover advantage into a near-monopoly position.

Second, without scale, data has little value. Yupp’s 1.3 million users sounded impressive, but for granular evaluations segmented by specific tasks and model comparison dimensions, it was nowhere near enough to support reliable statistical conclusions. Arena’s 10 million-plus battles are barely sufficient.

Third, selling raw data alone is not enough—you need to sell insights. The real value of AI Evaluations lies not in the raw voting data, but in the analytical reports derived from that data that can directly guide model iteration decisions. That requires long-term product refinement and methodological accumulation.

Credibility Is the Achilles’ Heel of This Business

But Arena is not without problems. In April 2025, researchers from Cohere, Stanford, MIT, and other institutions jointly published accusations that Arena showed favoritism in testing before Meta’s Llama 4 launch. Meta allegedly tested 27 model variants privately and only published the best-performing one; meanwhile, Arena supposedly assigned disproportionately more battles to models from Meta, OpenAI, and Google.

Arena responded that “some claims are inconsistent with the facts,” emphasizing that publishing scores for pre-release models is meaningless in itself. But the controversy exposed the core vulnerability shared by all evaluation platforms: when your customers are also the subjects being evaluated, how do you guarantee neutrality?

a16z openly acknowledged in its investment memo that this is Arena’s biggest challenge. Arena’s current response has been to open-source parts of its code and regularly publish battle datasets for third-party research. But as commercialization accelerates, this tension will only intensify. Paying customers want their models’ strengths amplified, while public users and regulators want unbiased evaluations.

a16z’s long-term vision for Arena is for it to become a “green certification” for AI products—especially in regulated industries like healthcare, finance, and critical infrastructure, where model reliability cannot rely solely on vendor promises and requires continuous third-party evaluation. If that vision materializes, Arena’s upside could far exceed its current $1.7 billion valuation. But first it must resolve the structural conflict of “acting as both referee and paid service provider.”

What This Means for Developers

From a developer’s perspective, Arena’s commercialization is actually good news.

If you are selecting models, the public data you could previously rely on was basically limited to Arena’s overall leaderboard—a very coarse signal. AI Evaluations now productizes more detailed comparison data across narrower scenarios and dimensions, theoretically enabling more precise model selection decisions. Of course, that assumes you are willing to pay for it.

For teams building applications, tools like Inclusion Arena that embed evaluation capabilities into production environments are even more interesting. They solve the question: “After switching the underlying model, did users actually perceive quality improvements or declines?” This is critical in multi-model routing and A/B testing scenarios.

Incidentally, for developers who frequently switch between and compare mainstream models, using aggregation platforms (such as OpenAI Hub) to access GPT, Claude, Gemini, DeepSeek, and other models through a unified API key—combined with Arena-style evaluation data for model selection—creates a fairly convenient workflow by eliminating the hassle of maintaining multiple vendor accounts.

One Conclusion

The significance of Arena reaching $100 million ARR is not about the company itself—it’s about proving that providing infrastructure services for the large-model ecosystem is a real business. While everyone else focused on the models themselves, the “shovel sellers” quietly built unicorn-scale valuations.

Scale AI received massive investment from Meta, Mercor surpassed $1 billion in revenue, Surge’s valuation has soared, and Arena hit $100 million ARR in eight months—the players in this sector are all scaling rapidly. The logic is straightforward: the faster models iterate, the greater the demand for high-quality evaluation and data, and that demand is structural and long-term.

As for whether Arena can withstand credibility controversies and continue proving itself after its valuation doubles—that question will probably have to wait for either the next funding round or the next “cheating scandal.”

References

• ITHome: Arena reveals monetization results for AI model evaluation rankings, commercial evaluation service AI Evaluations surpasses $100 million in annual recurring revenue - Report on Arena’s commercialization progress
• Zhihu: In-depth analysis of the drivers, market conditions, and commercialization progress of the AI evaluation sector - Overall analysis of the AI evaluation and data services sector