HexGrid Cloud Open Beta Testing Channel: You pick the model and card, it runs the stress test.

HexGrid Cloud Moved the Benchmarking Station to the Community Entrance

This week, HexGrid Cloud, focused on open‑source model deployment, posted a rather hard‑core thread on r/MachineLearning: whoever wants to see the real performance of an open‑source LLM on a given GPU—report the model, the card, the quantization setup—they’ll run it, and publish the results.

It’s not a marketing event; it’s more like an attempt to make their internal performance‑tuning process “semi‑transparent.” The team said it bluntly—their focus is on the serving/deployment layer, and instead of guessing what model‑hardware combinations users want to see, they’re letting the community place direct orders.

A Well‑Rounded Menu Covering Today’s Most Debated Models

The first batch of models open for benchmarking hit most of the open‑source names dominating discussions in early 2026:

• Nemotron‑3 Super 120B‑A12B (NVFP4 only)
• Nemotron‑3 Nano 30B‑A3B
• Qwen‑3.6 27B
• Llama 3.3 70B Instruct
• Gemma‑4 31B
• Devstral‑Small‑2‑24B‑Instruct‑2512
• plus one open slot—you can suggest any open‑source chat model that fits on a single H200 (141 GB)

The list itself merits comment. The Nemotron‑3 Super 120B‑A12B only runs NVFP4, clearly aimed at the new Blackwell architecture—FP4 precision can’t run efficiently on the H200; it needs a card with native FP4 tensor‑core support. The Nano 30B A3B follows the typical MoE sparse‑activation path, with about 3 B active parameters—what’s really being tested is scheduler and KV‑cache efficiency, not raw compute.

Qwen‑3.6 27B and Gemma‑4 31B are the two dense models most often compared head‑to‑head in the 27B–31B range; Llama 3.3 70B is the old benchmark—virtually every inference‑framework performance comparison revolves around it. Devstral‑Small‑2‑24B is a code‑oriented instruct variant, likely included to cover code‑assistant workloads.

Hardware and Quantization: From RTX PRO 6000 to H200 All the Way Up

The hardware pool spans four tiers, from workstation to data‑center grade:

• RTX PRO 6000: Ada‑architecture workstation GPU, 96 GB VRAM, suited for local deployments
• L40S: cost‑effective option, 48 GB, focused on single‑stream latency
• H100: current production‑grade workhorse
• H200: 141 GB HBM3e, the go‑to for 70B+ long‑context runs

Quantization choices include FP8 / AWQ / BF16; context windows from 8 K to 128 K. Users can specify metrics—maximum throughput, single‑stream speed, or long‑context prefill latency.

There’s a lot of insight in this setup. For example, the same Llama 3.3 70B running FP8 on H100 vs BF16 on H200 doesn’t yield a simple “which is faster” result—it produces two different trade‑off curves: memory‑bandwidth vs precision‑loss vs KV‑cache usage. Veterans know AWQ often loses to FP8 in low‑concurrency single‑stream cases, but once concurrency rises and KV cache becomes the bottleneck, AWQ may outperform by using less memory for weights, freeing up batch space.

Why It Matters

The core tension in open‑source LLM deployment over the past two years has been the gap between vendor‑marketing numbers and users’ real‑world observations. Benchmarks from vLLM, SGLang, TensorRT‑LLM, TGI look great—but run them on your own model, load, and card, and the numbers often differ by a wide margin. There are many reasons—continuous‑batching parameters, whether chunked prefill is on, speculative‑decoding configuration, KV‑cache paging policy—each can cause 2–3× token/s differences.

HexGrid Cloud’s approach essentially says: “Don’t trust the whitepaper. We’ll open the test environment—tell us the combo, we’ll run real data.” This used to be something done in sales‑support workflows; now it’s being moved into the public community.

For context, Google offers Hex‑LLM on TPU + Vertex Model Garden; AWS has SageMaker + Trainium/Inferentia; domestic platforms like ModelScope and OpenCSG also provide one‑stop deployment solutions. But most platforms run and publish their own benchmarks—users see only the results, not how it was tuned. HexGrid is going a step further by exposing the process itself.

A Hidden Message: Serving‑Layer Competition Has Reached the Fine‑Detail Stage

Worth noting, HexGrid admits they’re “obsessing over the serving/deployment layer.” The industry context behind that:

1. Inference engines have converged. vLLM 0.9.x, SGLang 0.4.x, TensorRT‑LLM all implement PagedAttention, continuous batching, prefix caching, chunked prefill—the feature gap on paper is shrinking fast.
2. Differentiation has shifted to “scheduler + hardware adaptation.” Running Llama 3.3 70B FP8 on different platforms, small differences in batch‑size adaptation, prefill/decode scheduling, multi‑request KV‑sharing can yield 30%–50% throughput variance.
3. Native support for new formats like NVFP4 and MXFP4 has become the new battleground. The fact that Nemotron‑3 Super uses only NVFP4 shows model vendors are starting to release hardware‑specific precision editions.

By opening benchmarking, HexGrid is effectively putting its scheduler under a microscope—if it runs slower, the community will notice instantly.

Practical Value for Developers

If you’re doing any of the following, this channel is worth using:

• Choosing a GPU: Want to know whether, for a 27B‑class model, dual L40S or a single H100 gives better value? Have HexGrid run a direct comparison.
• Choosing quantization: Need to weigh AWQ vs FP8 performance at your target batch size? Real data beats theory.
• Selecting architecture: Between Nemotron Nano 30B A3B (MoE) and Qwen‑3.6 27B (dense), which yields better throughput under the same budget? Third‑party runs make conditions cleaner and fairer.

By the Way

For developers in China who just want to quickly call open‑source models to test responses—without self‑deployment—OpenAI Hub (openai‑hub.com) already integrates major open‑weight models like Qwen‑3.6, Llama 3.3, Gemma‑4. One key is OpenAI‑compatible, connects domestically, and is convenient for prompt testing or small‑scale production.
But for large‑scale use cases pursuing extreme cost optimization, you’ll want to look at benchmark data from serving‑layer‑focused platforms like HexGrid—they solve problems at a different level.

Some Perspective

Whether this “community on‑demand benchmarking” model can last depends on two things:
1. HexGrid must disclose full serving configurations—batch size, concurrency model, scheduling parameters—mere token/s numbers aren’t enough;
2. It needs regular updates, a rolling open leaderboard rather than a one‑off campaign.

If they achieve both, this could become a truly useful reference source for open‑deployment choices—far more credible than self‑published blog posts. The inference‑service market urgently needs a third‑party, reproducible benchmark; whoever nails it first will earn a lasting spot in developers’ minds.

Now it’s up to the data HexGrid publishes over the next few weeks to show what actually happens.

References

• Reddit r/MachineLearning original post: HexGrid Cloud community benchmarking — details on models, hardware, and quantization options
• Hugging Face open‑source model repository — sources for Nemotron, Qwen, Llama, Gemma weights mentioned above