Baidu Famu 2.0 Tops the MLE-Bench Again, Officially Released in May

Baidu Famo Agent 2.0 Has Taken First Place Globally on MLE-Bench Again

This is already the second time. The last time was in October 2025, when this enterprise-level algorithm self-optimization agent from Baidu Intelligent Cloud topped the list for the first time. After half a year, version 2.0 has made a strong comeback and refreshed its SOTA (state of the art) performance once more. The official release is scheduled for May this year at Create 2026, Baidu’s AI Developer Conference.

The news itself isn’t particularly surprising, but it’s worth discussing seriously—because of the weight of the MLE-Bench ranking, and what it reflects about the shifting competitive focus in the AI industry.

What Does MLE-Bench Actually Test?

Let’s start with the list itself. MLE-Bench was established by OpenAI and is not the kind of academic leaderboard you can climb just by score-hacking. It includes 75 real machine learning engineering problems sourced from Kaggle competitions, covering the entire ML pipeline—from data preprocessing and feature engineering to model selection, hyperparameter tuning, training, and deployment.

In other words, it doesn’t test whether your model can write neat code; it tests whether your AI agent can act like an experienced ML engineer, taking a real Kaggle problem and independently completing the entire workflow—from reading data, analyzing the problem, choosing a solution, writing code, tuning parameters, to submitting results—all through autonomous decision-making.

This differs from the usual “large model benchmarks” we talk about (e.g., MMLU, HumanEval), which mostly measure a model’s inherent knowledge and reasoning ability—like a written test. MLE-Bench, by contrast, evaluates the agent’s practical engineering capability—more like a hands-on exam. It’s not enough to know what to do; you must actually do it well.

Each of the 75 tasks is a real Kaggle competition with real scoring standards. Agents must independently complete the full process from data exploration to model submission within limited time and computational resources. The final score is based on the proportion of tasks in which the agent reaches at least the Kaggle bronze level.

That setup is quite tricky. On Kaggle, the bronze threshold usually means your solution outperforms most human competitors. For an AI agent, this requires not only code-writing ability but also engineering intuition—knowing which model to use when, how to handle a certain data structure, and which features are worth the time to build.

Two Victories, Two Different Meanings

When it first topped the list in October 2025, the industry reaction was mostly, “Oh, Baidu can do agents too.” Frankly, at that time, attention was still on the large-model parameter race, and agents hadn’t become the main narrative.

But this time, with version 2.0 topping the chart again, the context is completely different.

Over the past half year, the AI industry’s direction has clearly shifted. Since the end of last year, nearly all major players have been emphasizing agents, engineering, and practical deployment. The reason is simple: the model capability curve is flattening. Just scaling parameters and training data no longer brings meaningful differentiation. The real competitive barrier is shifting from “model capability” to “engineering capability”—that is, how to transform model ability into usable productivity tools.

MLE-Bench happens to sit at this critical junction. It doesn’t test whether a model is “smart,” but whether an agent can actually get work done. Baidu Famo’s two wins prove that at least in the vertical domain of ML engineering, its autonomous decision-making and execution capabilities are world-class.

And note one detail: MLE-Bench is led by OpenAI. Winning first place on your competitor’s home turf says it all.

What Exactly Is Famo Agent?

Baidu Famo positions itself as an “enterprise-level algorithm self-optimization agent.” A mouthful, but broken down: it’s an AI agent designed for enterprise clients to automate algorithm optimization tasks in machine learning engineering.

Traditionally, the ML development process works like this: data scientists receive business requirements, manually conduct data analysis, feature engineering, model selection, training, tuning, and evaluation—iterating repeatedly. A moderately complex ML project typically takes weeks to months from initiation to production, with much of that time spent on repetitive tuning and experimentation.

Famo Agent aims to automate most of these steps. Give it a dataset and business objective, and it autonomously explores the data, engineers features, selects models, tunes parameters, and outputs a usable solution.

This isn’t a new concept—AutoML has been around for years, with tools like Google AutoML, Auto-sklearn, and H2O. But Famo’s key difference is that it’s based on a large-model agent architecture, not traditional search/optimization algorithms.

Traditional AutoML works more like a brute-force searcher—finding the best combination within a predefined search space using Bayesian optimization or evolutionary algorithms. Its performance ceiling is limited by search design, and it consumes considerable compute.

A large-model-based agent is different. It can “look at” the data first, then decide what methods to use—its decisions come from understanding and reasoning, not blind enumeration. That means it can handle more open-ended, complex problems, and do so more efficiently.

Based on its MLE-Bench performance, this approach clearly works.

Key Technical Highlights

Although Baidu hasn’t disclosed full technical details of Famo 2.0, based on public info and MLE-Bench’s design, several aspects are noteworthy:

1. Multi-step reasoning and planning.
MLE-Bench problems can’t be solved in one shot; agents must devise a multi-step plan and adjust dynamically as they proceed. This requires strong long-range planning ability, not just single-turn dialogue.

2. Closed-loop code generation and execution.
The agent must not only generate code but also execute it, observe outcomes, diagnose errors, and fix bugs. This “write-run-observe-revise” loop is the core of engineering capability. Many agents perform well at code generation but poorly at execution/debugging.

3. Domain knowledge utilization.
Kaggle tasks cover image, NLP, tabular, and time-series data. The agent must apply relevant knowledge per task type—for instance, knowing to use LightGBM over ResNet for tabular data, or how to do data augmentation for text classification.

4. Resource management.
Given limited time and compute resources, the agent must allocate them wisely—deciding which problems warrant more effort and which just need a quick baseline. This kind of “meta-strategy” decision-making often separates top agents from average ones.

What It Means for Developers

If you’re an ML engineer, tools like Famo reaching maturity may change how you work.

It doesn’t mean ML engineers will be replaced—at least not anytime soon. But the repetitive work of parameter tuning, feature testing, and baseline building can indeed be delegated to an agent. Engineers will shift toward problem definition, solution evaluation, and business integration.

Practically speaking, it also lowers the barrier for enterprise ML projects. What previously required a 3–5 person data science team and months of work may soon be doable by one engineer with an agent in just a few days. That’s good news for smaller companies.

Of course, that depends on whether such agents can run reliably in production—not just on benchmarks. The formal version announced at the May Create Conference will be the true test.

Industry Context

Currently, AI agent strategies vary by company:

• OpenAI focuses on general-purpose agents (like ChatGPT’s plugin and Operator ecosystem), taking a platform approach
• Anthropic’s Claude emphasizes safety and controllability, with agent features growing but conservatively
• Google DeepMind invests heavily in research agents—AlphaCode and AlphaFold being typical examples
• In China, Baidu Famo focuses on ML engineering, while Alibaba and ByteDance build agents within their own domain strengths

Baidu’s ML engineering angle is a smart one: it’s vertical enough, with clear demand and quantifiable performance. Unlike general agents, which are harder to evaluate, ML performance is measurable through model metrics—and MLE-Bench results prove its strength.

That said, benchmark performance doesn’t always translate directly to product experience. Those 75 Kaggle problems are standardized challenges; real-world enterprise ML projects are messier—data quality varies, business constraints are irregular, and deployment environments differ widely. Whether Famo 2.0 can maintain comparable performance in production remains to be seen after release.

On API Integration

For developers wanting to integrate AI model capabilities into their applications, the options are expanding. Baidu’s Wenxin models, OpenAI’s GPT series, Anthropic’s Claude, and Google’s Gemini all offer API access.

If you don’t want to manage separate API keys and connection setups for each provider, you can use aggregation platforms like OpenAI Hub, which provide unified access. One key works across mainstream models, the interface is compatible with OpenAI’s standard, and it connects directly from within China—saving lots of hassle.

For example, to compare multiple models, the code looks like this:

from openai import OpenAI

client = OpenAI(
    api_key="your OpenAI Hub API Key",
    base_url="https://api.openai-hub.com/v1"
)

# Same interface format — just change the model parameter
models = ["gpt-4o", "claude-sonnet-4", "gemini-2.5-pro", "deepseek-chat"]

for model in models:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "user", "content": "Write a simple feature engineering pipeline in Python"}
        ],
        max_tokens=1024
    )
    print(f"--- {model} ---")
    print(response.choices[0].message.content)

This is especially convenient for capability benchmarking and model selection.

What to Watch Next

The Create 2026 Baidu AI Developer Conference in May will be a crucial milestone. When Famo 2.0 is officially released, we’ll see:

• The specific product form and pricing strategy
• Real enterprise-case results and performance metrics
• Integration with other Baidu Cloud products (like the Qianfan platform)
• Whether APIs will be opened for developer access

For developers focused on AI engineering deployment, this is a product worth following—not just because it topped a leaderboard, but because of what it represents: the industry-wide shift toward automating ML engineering via agents.

Whether it can evolve from a benchmark champion into a truly productive tool—we’ll find out in a month.

---

References:

• Baidu Famo Agent 2.0 Tops MLE-Bench Again – 36Kr (Chinese): original news flash covering the new SOTA result and the Create 2026 release plan