Babies Are Still More Efficient at Learning to Speak Than Large Models

On August 24, MIT Technology Review published an analysis of the training efficiency of large language models and interviewed several experts who study language acquisition. Its conclusion does not flatter today’s race for scale: even though the most advanced chatbots can already write code, summarize text, and imitate complex tones, human babies remain more than an order of magnitude more sample-efficient at learning language. The two may not even use directly comparable learning paradigms.

Michael C. Frank, a cognitive scientist at Stanford University, offered a set of intuitive figures: after being exposed to roughly 10 million to 30 million words in their everyday environment, young children can already speak in clear, understandable sentences. Train GPT-2 on just 30 million words, however, and the result would not resemble a two- or three-year-old child. It would more likely be a “gibberish generator.”

AI developers should take this comparison seriously. It does not mean that “large models are useless,” nor is it another attempt to romanticize babies as possessing some mysterious intelligence. Rather, it is a reminder to the industry that fluent output and efficient learning are two different things. Today’s large models are extremely good at trading scale for capability, but they are still poor at forming stable concepts from the limited, continuous, feedback-rich experiences through which humans learn.

Why 30 Million Words Are Enough for a Baby but Far From Enough for a Model

On the surface, babies and large models both “hear a lot of language and then predict what should come next.” But reducing both processes to the same next-token prediction problem misses precisely the most important differences.

First, babies do not receive their input as a cleaned text file.

When a parent says, “The ball fell,” the child simultaneously sees the ball’s trajectory, hears it hit the floor, senses the adult’s gaze and tone of voice, and may even reach out to pick it up. Every word is repeatedly annotated by vision, action, causality, emotion, and joint attention. For a model, “ball” is usually just a token that co-occurs in a corpus with words such as “round,” “movement,” and “game.” For a baby, it is an object that can be touched and rolled, and that continues to exist even when hidden from view.

Second, although babies receive less data, that data is highly relevant. Language in the home has a structure of repetition, correction, and gradual progression: the same object is named repeatedly in different situations, adults adjust how they speak based on the child’s reactions, and children actively point, pick things up, and ask questions, using their behavior to select the next piece of training data.

Pretraining a large model is more like dumping libraries, forums, code repositories, and snapshots of the web into a blender all at once. The volume of data is staggering and its coverage extremely broad, but it includes duplicated content, contradictory claims, context-free fragments, and machine-generated text. During pretraining, the model cannot pause and ask, “When you say ‘it,’ do you mean the cup on the table or the cat from a moment ago?”

Finally, the two do not share the same learning objective. A language model’s core loss function typically rewards it for “predicting the next token correctly.” Babies learn language to accomplish intentions: asking for water, finding their mother, getting attention, or explaining what is happening in front of them. The former optimizes statistical fit; the latter is embedded in a closed loop of action and feedback. Predictive objectives can certainly produce powerful language capabilities, but they do not necessarily yield a robust world model automatically, nor do they guarantee that a model can establish new concepts from very few examples.

Do Not Be Misled by the “Word Count” Comparison: This Is Not a Strict Head-to-Head Benchmark

That said, the figure of 10 million to 30 million words cannot simply be treated as the “size of a baby’s training set.”

Babies do not receive language input alone. Every minute after birth, vision, hearing, touch, proprioception, and social feedback provide information. Humans also come equipped with perceptual structures, attention mechanisms, and learning biases shaped by evolution. In other words, children’s limited exposure to words does not mean they receive little information.

There are also measurement problems on the model side. After processing by a tokenizer, a single “word” may become one or more tokens, and direct conversion is impossible across different languages and tokenizers. GPT-2 is also an older architecture released in 2019. It vividly illustrates the difficulty of training on small datasets, but it does not represent the upper limit of data efficiency for every model in 2026. Modern models have become substantially more efficient through improved data mixtures, synthetic data, distillation, curriculum learning, retrieval augmentation, and multimodal training.

The reasonable interpretation, therefore, is not “babies have fewer parameters, so Transformers have failed,” but rather this: today’s mainstream models still depend heavily on consuming offline, static data at massive scale, while humans bind language to their environment and goals and learn through continuous interaction. The gap in sample efficiency is real, but its causes cannot be reduced to a table of parameter counts or token totals.

Scaling Still Works, but Its Marginal Gains Are Becoming More Expensive

Progress in recent years has demonstrated that increasing parameters, compute, and high-quality data remains effective. There is no point in denying scaling laws: when training budgets allow, larger, cleaner, and better-proportioned datasets generally continue to improve capabilities. The problem is that this path is becoming increasingly expensive, while the supply of high-quality human-generated data is finite.

Frank described today’s training methods as “burning down a forest and then scraping almost all of human knowledge.” Although rhetorical, the description gets to the heart of the industry’s cost structure: we use enormous GPU clusters, vast amounts of energy, and huge corpora to reproduce a language milestone that occurs naturally in an ordinary household. The resulting models ultimately possess far broader knowledge than children, but the sample cost of acquiring basic language abilities remains astonishingly high.

More troublingly, adding more data improves average capability but does not necessarily improve reliability. A model may have read tens of thousands of physics textbooks and still make mistakes involving simple spatial relationships. It may generate beautifully structured code without understanding a real constraint in the deployment environment. Knowledge coverage and conceptual understanding remain separated by the problem of grounding: the correspondence between linguistic symbols and real-world objects and the outcomes of actions.

The question of whether continued scaling can lead to human-level intelligence should therefore not be treated as a binary choice. Greater scale will likely continue to deliver product value, but if the goal is efficient, continuous, and transferable general learning, more tokens alone may not be enough. Scale may be necessary, but it looks increasingly unlikely to be sufficient.

The Practical Lesson for Developers: Do Not Just Give Models More Context

For engineering practice, the most useful lesson from this analysis is not to imitate babies by building some grand cognitive architecture. It is to reconsider what kinds of learning and feedback conditions an application actually provides to its model.

1. Highly Relevant Context Is Often More Valuable Than Extremely Long Context

When many teams encounter performance problems, their first response is to stuff more documents into the prompt. The context window grows from 20K to 200K, and retrieval noise grows along with it. The lesson from infant learning is not to “provide less data,” but to make the input highly relevant to the current task.

In a RAG system, this means getting document chunking, metadata filtering, reranking, and permission isolation right before discussing context-window size. Ten passages that are directly relevant to the question and have clearly identified sources will usually produce more consistent output than a hundred passages that “might be useful.”

2. Feed Outcomes Back into the System

A basic chat interface often has only inputs and outputs, with no consequences. A more reliable agent should observe the results of tool calls, execution states, and error messages, then use them to correct its next action.

For example, after a coding model generates SQL, do not merely ask another model to evaluate its syntax. Execute it in a controlled environment and return the query plan, errors, and number of result rows. Exposing the model to the consequences of a task is equivalent to putting the fact that “the ball really did fall to the floor” back into the language-learning process.

3. Evaluations Should Cover Few-Shot Transfer, Not Just Benchmark Accuracy

If a model scores highly on public benchmarks but needs dozens of examples to adapt to a company’s internal terminology, its learning efficiency remains limited. Teams can add three categories of evaluation:

• After a new concept is defined only once, can the model apply it correctly in subsequent tasks?
• When conditions change, can the model update its judgment instead of repeating an outdated answer?
• Does the capability persist when the same rule is expressed differently or applied to a different data format or business scenario?

These tests are closer to the transfer capabilities required in real applications than simply running another set of multiple-choice questions.

4. Training Small Models Requires More Attention to Data Structure, Not Blind Expansion

When training budgets are limited, expanding one billion low-quality samples into two billion may be less effective than curating data with a clear progression of difficulty, negative examples, and feedback signals. Deduplication, conflict detection, data provenance, and curriculum design are often cheaper improvements than continuing to pile on tokens.

This is especially true for domain-specific models. In medicine, law, industrial operations, and similar fields, the truly scarce resource is not general text but process data showing “which decisions lead to which outcomes under which conditions.” If the data contains conclusions without environments or consequences, the model may still learn nothing more than superficial statistics about industry jargon.

The Next Competitive Frontier May Be “How Models Learn,” Not “How Much They Have Read”

Current research is approaching this problem from several directions. Multimodal models seek to connect language with vision and audio. Robotics and embodied intelligence incorporate the outcomes of actions into training. Active learning allows systems to select the most valuable data. Continual learning aims to let models absorb new knowledge after deployment without catastrophically forgetting existing capabilities. World models, memory mechanisms, and online reinforcement learning are also addressing the shortcomings of text-only pretraining.

But none of these approaches has yet replicated the learning efficiency of babies. Multimodality is not the same as embodiment, and pairing images with captions is not the same as having a body. Adding a reinforcement-learning phase is not the same as achieving genuine continual learning; offline preference data may still be just another form of static corpus. The industry does not need to package “infant learning” as a new marketing term. It needs mechanisms that can be empirically validated: which priors reduce sample requirements, which interactions lead to concept formation, and which forms of memory can be updated reliably without retraining the entire model.

That is the most valuable point in this analysis. It does not deny the achievements of large models over the past several years. Instead, it exposes a fact that fluent conversation can easily conceal: a model speaking like a human does not mean it learns like one.

For today’s developers, large models remain among the most capable and accessible general-purpose computing interfaces available. But product design cannot assume that larger models, longer contexts, and more training data will automatically solve every problem. Retrieval quality, environmental feedback, executable tools, persistent memory, and task evaluation are becoming as important as model parameters.

Babies do not read the entire internet, yet they rapidly understand a word by watching, listening, trying, and correcting their mistakes. The truly difficult breakthrough for AI’s next stage may not be to “scrape human knowledge” all over again, but to enable machines to learn more from every limited experience.

References

• ITHome: Study Finds Babies Far Outperform AI Chatbots in Language Learning, with Much Greater Learning Efficiency—A summary of the relevant MIT Technology Review analysis and the views and data provided by Stanford cognitive scientist Michael C. Frank.