My phone keyboard once tried to finish “see you at the” with “see you at the vet,” which would have been a strange way to confirm dinner plans. We’ve all had a predictive-text fail like that, and it’s tempting to assume a proper AI chatbot is just doing a cleverer version of the same trick. That turns out to be closer to the truth than most people expect — just not in the way the “fancy autocomplete” jibe usually means it.
Ask a chatbot to write a short story, explain quantum physics, or draft a polite email to your boss, and the result can feel like magic. It produces fluent paragraphs in seconds, remembers what you said ten minutes ago, follows your instructions, and can mimic Shakespeare or a tired teenager on request. Underneath all of that sits something surprisingly simple: a system that’s constantly predicting what comes next.
ChatGPT, Claude and Gemini all generate text by repeatedly working out which bit of text is most likely to follow whatever’s already been written. There’s a genuinely enormous amount of maths and hardware behind that sentence, but the core idea doesn’t need a maths degree to follow. And technically, these systems aren’t even predicting whole words — they’re predicting something called tokens.
What Does “Predicting the Next Word” Actually Mean?
Give a language model the start of a sentence — “the cat sat on the…” — and it isn’t rifling through a filing cabinet for the “correct” sentence. It’s calculating which continuation makes the most statistical sense, based on everything it absorbed during training.
Internally, it assigns rough odds to thousands of candidates: “mat” might come out at 48%, “floor” at 21%, “chair” at 12%, “roof” at 4%, and something odd like “banana” at a fraction of a percent. It picks one of the sensible options, bolts it onto the sentence, and immediately starts the whole calculation again from scratch. Pick “mat,” and the new input becomes “the cat sat on the mat…” — and off it goes, working out what should come after that. Do this hundreds or thousands of times in a row and you get a full paragraph. That repeating loop, as mundane as it sounds, is genuinely the engine room of modern generative AI.
It’s Actually Predicting Tokens, Not Words
“Next-word prediction” is a handy shorthand, but it’s not quite accurate. Language models actually chop text into smaller pieces called tokens — a token might be a whole word, part of a word, a single letter, or a punctuation mark.
A short, simple sentence might get tokenised one word at a time. Throw a longer or less common word at it, though, and it gets broken apart like Lego bricks: “unpredictability” might come out as something like “un,” “predict,” and “ability.” The reason for doing this is fairly practical — trying to store a dedicated entry for every possible word, in every language, including typos, slang, and made-up names, would be hopelessly inefficient. Reusable chunks let the model cope with unfamiliar names, technical terms, and entirely new languages far more gracefully than a fixed word list ever could.
Where Do the Probabilities Actually Come From?
This is where training comes in. Before any chatbot reaches the public, it reads a genuinely enormous amount of text — books, articles, websites, code — and plays an absolutely enormous game of fill-in-the-blank the entire way through.
Picture handing an untrained model the sentence “the Earth travels around the ___.” At first its guesses are essentially random — it might say “sandwich.” It checks against the real answer, “Sun,” gets it wrong, and nudges its internal settings very slightly in a better direction. Repeat that trillions of times and the system gradually builds up an intricate map of how human language actually fits together — not just grammar, but facts, writing styles, programming syntax, and plenty of basic reasoning besides. Nobody ever sat down and typed a rule saying “if the user says ‘peanut butter,’ respond with ‘jam.'” The model simply absorbed that pattern from seeing the two words together an enormous number of times.
Why Doesn’t It Always Pick the Most Likely Word?
If a model always reached for the single highest-probability word, every single time, its writing would be dull, repetitive, and faintly robotic. Ask it to “write a creative story about a robot discovering the ocean,” and if “robot” sits at 35%, “machine” at 25%, and “explorer” at 18%, always taking the top choice would flatten the writing fast.
So developers build in a bit of deliberate randomness, letting the model occasionally pick the second or third option rather than always the first. That’s why you can ask the exact same question three times and get three genuinely different answers back — and it’s a large part of why generative AI feels like it has some kind of personality, rather than behaving like a rigid calculator spitting out the same output every time.
What Does “Temperature” Actually Control?
If you’ve ever poked around in an AI tool’s advanced settings, you may have spotted a slider called temperature. It simply controls how adventurous the model is allowed to be when choosing its next token.
A low temperature keeps things safe and predictable — ideal for code, factual summaries, or anything technical where consistency matters more than flair. A high temperature nudges the model towards less obvious choices: where a low-temperature model might describe a night sky as “beautiful,” a high-temperature one might reach for something like “shattered glass scattered across an endless black ocean.” The model doesn’t have imagination in any human sense — it’s simply been given permission to wander further from the statistically obvious choice.
How Does It Learn From Its Mistakes?
You’ll sometimes hear intimidating terms like loss functions or gradient descent thrown around, but the underlying idea is closer to a very large game of hot-and-cold. Picture standing on a foggy hillside, trying to find the lowest point in the valley. You can’t see the whole landscape, but you can feel which way the ground slopes under your feet, so you take a small step downhill, then another, then another.
Training a language model works much the same way. Every time it makes a poor prediction, a formula works out roughly how wrong it was, and the system nudges millions of its internal settings very slightly closer to a better answer. Do that across billions of pages of text, and those small steps eventually add up to a genuinely capable model.
Why Context Changes Everything
A model doesn’t just glance at the last word you typed — it reads the whole conversation so far to work out what’s actually being asked. Compare “I deposited my money at the…” with “we sat and watched the ducks by the river…”. The word “bank” fits neatly after both, but means something completely different in each. Because modern chatbots can hold thousands of words of context at once, they can look back across your conversation and work out which meaning actually makes sense right now, rather than guessing blind from a single sentence.
The Flaw Baked Into the System: Hallucinations
Understanding next-token prediction also explains AI’s biggest weakness: it can be completely, confidently wrong, and sound exactly as convincing while doing it.
A model’s actual job is to produce a sequence of words that sounds plausible — and plausible is not the same thing as true. Ask it to write about an obscure academic topic, and its internal patterns know that academic papers tend to come with author names, institutions, and publication dates. So it will happily assemble something that looks exactly like a real citation, even if the author and the book never existed. That’s a hallucination, and it happens because the model isn’t retrieving a fact from a trusted source — it’s calculating the most statistically plausible next few words. It’s precisely why anything that actually matters — medical, legal, or financial claims especially — is worth checking independently rather than taking on faith.
So Is It Really Just “Fancy Autocomplete”?
People dismiss large language models as “autocomplete on steroids” fairly often, and there’s a kernel of truth buried in there — it’s just also a significant understatement. The predictive text on your phone guesses that “see you” is probably followed by “tomorrow,” based on a fairly shallow pattern. A chatbot is doing something structurally similar, but the network behind that prediction is so vast that, in the process of getting good at it, it’s effectively built a working model of logic, code structure, and reasoning along the way. The mechanism is prediction. What comes out the other end is something considerably more than that phrase suggests.
The Magic of the Loop
Next time you’re using an AI chatbot, try typing an unfinished sentence and just watching it continue. You’re not watching a machine pull a ready-made answer out of a digital filing cabinet. You’re watching an almost embarrassingly simple loop — work out the odds, pick a token, repeat — running at a genuinely staggering speed.
It’s simple maths, repeated billions of times over, producing something that feels remarkably like a mind at work.