# 1.1B Outperforms 7B: LingBot-Vision Redefines Self-Supervised Pretraining with Boundary Awareness

1.1B Parameters: Knocking DINOv3’s 7B Off the Top

On July 5, LingBot-Vision was open-sourced simultaneously on Hugging Face and GitHub, releasing weights in four sizes at once (ViT-S/B/L and the 1.1B ViT-H+), with the accompanying paper posted on arXiv. The core takeaway can be summed up in one sentence: on the NYUv2 depth estimation linear probing task, the 1.1B patch-16 model achieved 0.296 RMSE, outperforming DINOv3’s 7B flagship model (0.309). It still trails DINOv3 slightly on ImageNet-1k linear classification, but on semantic segmentation it has already matched the distilled version of DINOv3 ViT-H+.

Since Meta released DINOv3 last August, this is the first open-source self-supervised vision foundation model to achieve “better dense prediction performance with less than one-seventh the parameter count.” For engineering teams working on detection, segmentation, depth, and robotic perception, that delta is hard to ignore.

A Different Approach: Stop Betting That Masking Will Somehow Learn Boundaries

To understand why LingBot-Vision has an edge on dense tasks, you first need to look at its methodological divergence from DINOv3.

DINOv3 follows the DINO family’s self-distillation path, adding iBOT-style masked patch modeling, KoLeo regularization on the CLS token, and Gram anchoring to prevent dense feature maps from degrading during long training runs. The overall idea is: “randomly mask patches and hope the model learns geometric structure during reconstruction.” This works flawlessly on ImageNet, but for dense prediction tasks, even Meta acknowledged that feature maps drift after long training, which is why Gram anchoring had to be added as a patch.

The LingBot-Vision authors simply stopped betting on that assumption: instead of hoping boundary structure emerges from random masks, they force the student model to reconstruct regions “where the answer cannot simply be copied from context.” Specifically:

• Online boundary field prediction: during the forward pass, the teacher outputs a dense boundary field labeling which tokens lie on geometric boundaries;
• Force masking of boundary tokens: these boundary-bearing tokens are inserted into the student’s mask set, meaning the student must reconstruct precisely the semantic/depth discontinuity regions;
• Self-generated boundary targets: no labels and no external edge detectors like Canny or HED are used; everything is iteratively produced by the teacher itself.

This approach directly injects “what is hardest to learn” into the training loop. Semantically, geometric boundaries are shared low-level structures across depth, normals, and segmentation tasks. From a training perspective, forcing the model to explain these discontinuous regions carries far higher information density than reconstructing large uniformly textured wall surfaces.

Two Unassuming but Critical Engineering Choices

There are two design decisions in the paper worth paying close attention to, and the authors explicitly call them load-bearing in the abstract.

First, the boundary field is rewritten as a per-pixel categorical distribution, rather than a continuous regression target. This may look like a superficial change, but the consequences are significant. DINO-style anti-collapse mechanisms like centering and sharpening in self-distillation were designed for classification distributions. Continuous regression targets tend to drift toward meaningless fixed points under EMA teachers. By switching to categorical targets, the geometry branch can directly reuse existing stabilization machinery. This “adapting new tasks to old defenses” approach is far more engineering-friendly than inventing entirely new defenses for new tasks.

Second, decoded boundary segments must pass an a-contrario validation step before they are allowed to supervise the student. A-contrario is a classical statistical significance framework from computer vision. Put simply, it asks: “What is the probability that this boundary would appear in a random noise image?” High-probability boundaries are discarded immediately. This effectively gives the teacher a self-auditing mechanism, preventing fake early-stage boundaries from leading the student astray. It also addresses the classic chicken-and-egg problem in self-distillation: when the teacher is unreliable, don’t let it teach yet.

Looking Closely at the Numbers

The paper includes a comparison table. A few notable figures:

| Task | Metric | LingBot-Vision 1.1B | DINOv3 7B | |------|------|---------------------|-----------| | NYUv2 depth linear probing | RMSE ↓ | 0.296 | 0.309 | | ADE20k segmentation (linear) | mIoU ↑ | Matches distilled DINOv3 ViT-H+ | — | | ImageNet-1k linear classification | Top-1 ↑ | Trails by ~1.2 points | Leads |

Note that these are all self-reported results, using patch-16 resolution. DINOv3’s 7B version officially uses patch-16 with 512-resolution training. Resolution and patch size significantly affect dense tasks, so the 0.296 vs 0.309 comparison still deserves some skepticism until third-party reproduction arrives. But methodologically, the idea of boundary-aware masking is solid.

The weaker ImageNet performance is also expected. Classification tasks are inherently more sensitive to global textures and category prototypes; boundary structure is not the decisive factor there. In fact, shifting training capacity toward boundary regions slightly defocuses the globally pooled CLS representation. This is a classic tradeoff: models optimized for dense tasks naturally sacrifice a bit of pure classification performance.

All Four Model Sizes Have Released Weights

The authors released four variants in one go: ViT-S, ViT-B, ViT-L, and ViT-H+ (1.1B), all patch-16. This is good news for downstream applications:

• S/B tiers (22M / 86M): suitable for edge deployment, such as robot SLAM front-ends or AR depth estimation scenarios with strict latency requirements;
• L tier (~300M): the sweet spot for standard server-side inference, with feature quality already sufficient for most segmentation/detection heads;
• H+ tier (1.1B): positioned against DINOv3-7B, delivering equivalent or better dense features at less than one-sixth the compute cost.

Weights are directly downloadable from Hugging Face, and the authors provide a timm-compatible loader, making integration into existing stacks low-cost. The license is Apache 2.0, with no commercial barriers — notably more permissive than DINOv3’s DINOv3 License, which carries some commercial restrictions.

The Diverging Paths of LingBot-Vision and DINOv3

Looking at the two projects side by side reveals two distinct directions for SSL vision foundation models this year:

DINOv3’s path is “scale up + patch problems later”: data grew from DINOv2’s 1.4B to 1.7B samples, model size expanded from 1.1B to 6.7B parameters, and training objectives stacked iBOT, KoLeo, and Gram anchoring together, followed by a three-stage process (pretraining → Gram anchoring → high-resolution adaptation). The engineering effort is enormous, and the results are state-of-the-art across the board, but the reproduction barrier is extremely high. The SLURM scripts alone are enough to scare off 90% of teams.

LingBot-Vision’s path is “redesign the pretext task”: no extra data, no extra parameters, no extra training stages — just a different masking strategy. Swapping 1.1B for 7B-level dense performance is an excellent engineering tradeoff.

Of course, this route has obvious weaknesses too:

• it still loses to DINOv3 on classification tasks, proving boundary structure is not a universal key;
• a-contrario validation and dense boundary decoding reduce training throughput, and the authors acknowledge training is about 30% slower than a same-sized DINOv2 baseline;
• all comparisons are currently self-reported, and community reproductions have not caught up yet.

Who Should Pay Attention to This Model

If you work on:

• Monocular depth estimation / 3D perception: directly swapping the backbone may help; that 0.013 RMSE gap on NYUv2 could widen after downstream finetuning;
• General segmentation / open-vocabulary segmentation: boundary-aware dense features are naturally well suited as mask decoder inputs;
• Robotic vision / embodied AI: a 1.1B model can fit on a Jetson AGX, making it far more practical than a 7B model;
• Medical imaging / remote sensing: geometric boundaries are critical in these domains, making continued in-domain pretraining worth exploring.

Teams focused purely on classification, retrieval, or CLIP alignment can probably wait for more community reproductions and ImageNet validation first.

Some Thoughts

From MAE, DINO, iBOT, to I-JEPA, the central question in self-supervised vision has always been: “What exactly should the pretext task force the model to learn?” LingBot-Vision’s answer is: making the model explain geometric discontinuities is far more useful than making it guess uniform textures. This isn’t a new idea — traditional vision methods like Canny and SIFT followed the same logic — but engineering it into an EMA self-distillation framework, making it compatible with anti-collapse machinery via categorical distributions, and adding self-auditing through a-contrario validation: that combination is genuinely substantial work.

DINOv3 proved that scale can produce miracles. LingBot-Vision proves that clever design still matters beyond brute force. For developers, that’s good news — not every team can train a 7B model, and the more capable 1.1B open-source backbones exist, the richer the downstream application ecosystem becomes.

A few things worth watching next:

• whether third-party reproductions can match the self-reported numbers;
• whether the authors extend boundary-aware masking to video or multimodal models (this logic should work even better for temporal discontinuities in video);
• whether the next version will challenge DINOv3’s 6.7B tier directly — if the scaling curve holds, things could get very interesting.

References

• LingBot-Vision: masked boundary modeling for self-supervised pretraining (Reddit r/MachineLearning discussion thread): includes the paper abstract, key methodological details, and author Q&A
• DINOv3 Hands-on Guide: Changing How Vision Models Are Used (Zhihu): Chinese-language explanation of DINOv3’s training objectives and three-stage process
• Detailed Breakdown of the DINO Self-Supervised Vision Series (Zhihu): evolution of DINO/DINOv2/DINOv3
• DINOv3: A New Benchmark for Self-Supervised Vision Foundation Models (Zhihu): DINOv3’s data mixing strategy and architectural details