What Is an LLM, Really? A Gigantic Autocomplete

A large language model (LLM) estimates which token or sequence of tokens is likely in a given context. Many chat products use an LLM along with instructions, tools, safety checks, and application code. The phone keyboard comparison is a useful starting analogy, but a modern LLM can use much broader context and learned representations. Google's LLM introduction explains the token-prediction foundation.

My favorite analogy is the next-word suggestion on your phone's keyboard. When you type "The weather today is very...", the keyboard offers "nice," "hot" or "cold." At its core, an LLM does exactly this; but unlike that tiny model on your phone, it has read a large slice of the internet and practiced on billions of sentences. That is why it can produce not just one word but an entire paragraph, an essay, even working computer code, weaving them one piece at a time.

It feels counterintuitive at first that such a simple idea turns into something that can summarize a legal document, write poetry or spot a bug in code. The secret lies not in any single prediction, but in repeating that prediction billions of times, and in the model absorbing the patterns of language, reasoning and the world along the way. In the rest of this article we'll unpack, step by step, how such a small idea becomes such a large leap.

The Idea at the Core: Predicting the Next Word (Token)

LLMs have a single core skill: predicting the most likely unit that comes next in a piece of text. In technical terms this is called "next-token prediction." Here a "token" is the smallest piece the model uses while processing text. Often it corresponds to a word, but a long word like "autocompletion" may be split into several tokens. So a token is sometimes a whole word, sometimes a fragment of one, sometimes just a punctuation mark.

At each step the model produces a probability distribution over all possible tokens, in other words a numerical answer to the question "out of the thousands of words that could come next, how likely is each one?" After the phrase "The capital of Turkey is..." it assigns a very high probability to the token "Ankara" and a near-zero probability to "banana." It then picks a token according to those probabilities, appends it to the end of the text, and repeats the whole process from the top. Predict, append, predict again: this loop is the mechanism that weaves the fluent paragraphs you see, one word at a time.

The model does not always choose the highest-probability token. Sampling settings such as temperature can change the distribution of outputs, but system instructions, tools, context, and implementation details also affect an answer. Repeatability must be tested in the application rather than assumed from one setting.

Training on Massive Text: Where Does the Model Learn This?

A model knows that the word "capital" is usually followed by the name of a city, even though no one explicitly told it so. It figures this out on its own during a process called "training," by reading enormous amounts of text. Training data typically consists of web pages, books, encyclopedias, articles, forum discussions and code repositories; in total, on the order of hundreds of billions, even trillions, of words.

The mechanics of training are surprisingly simple. The model is shown a fragment of a sentence, the next word is hidden, and the model is asked to predict it. The model makes a guess, the guess is compared with the real word, and to the extent it was wrong, the numerical settings inside the model are nudged by a tiny amount. This procedure is repeated countless times over the massive text. No one teaches the model grammar rules, historical facts or logic; it captures all of these as patterns, as a byproduct, simply while trying to "predict the next word a little better."

Many assistant models are adapted after pre-training with instruction examples and preference data. The methods vary: supervised fine-tuning, reinforcement learning from human feedback (RLHF), and direct preference optimization are examples. No single post-training recipe applies to every model. The DPO paper describes one alternative to a conventional RLHF pipeline.

Parameters: The Billions of Tiny Knobs the Model "Knows"

Those "numerical settings" adjusted during training are called parameters. You can think of a parameter as a tiny knob, a dial on a gigantic audio mixing board. All of the model's knowledge, linguistic intuition and pattern-recognition ability is hidden in exactly where these dials sit. Training is essentially nothing more than the process of setting these billions of dials, one by one, to the right positions.

Modern large models typically have tens of billions, even hundreds of billions, of parameters. The big numbers you hear next to names like "GPT" or "Llama" often refer to precisely this parameter count. As a rough rule, more parameters mean more capacity to capture patterns and behave with more nuance; but the relationship is not linear. Beyond a certain point, the quality of the data, the training method and the model's architecture become far more decisive than raw parameter count.

Parameters are not an ordinary searchable database of training documents. They encode learned patterns, yet models can sometimes reproduce memorized training material. Treat generated quotations, citations, and private data claims as items to verify, not as guaranteed summaries of what the model saw during training.

The Secret Behind Seeming to "Understand": Attention and the Transformer

If the model only predicts the next word, how does it manage to grasp a question and give a coherent, context-appropriate answer? The answer lies in the architecture underlying almost every LLM today, called the "Transformer," and the "attention" mechanism at its heart. Attention lets the model decide, while processing a given word, which of the other words in the text to weight and by how much; in effect, it keeps asking the model "what should I focus on right now?"

An example makes this concrete: in the sentence "I put the bag on the table because it was very heavy," what does the pronoun "it" refer to? A human mind instantly says "the bag." Thanks to attention, the model too forms a strong link to the word "bag" while processing "it," and a weak link to "table." Had you changed the sentence to "because it was very large," attention would most likely shift to "table" instead. This ability to capture the relationships between words in order to predict the next one correctly produces much of what looks, from the outside, like "understanding."

Whether machine behavior counts as “understanding” is debated, and the answer depends on what the word means. For engineering, test the specific capability you need: can the system answer from supplied evidence, follow constraints, and recognize uncertainty on representative examples? A fluent explanation alone is not proof of reliable understanding.

The Context Window: The Model's Short-Term Memory

When you chat with an LLM, you notice it behaves as if it remembers what you just said. What makes this possible is a concept called the "context window." The context window is the amount of text the model can "see" and take into account at once, measured in tokens. You can think of it as the size of a desk in front of the model: it can consider everything that fits on that desk at once, and it cannot see anything that doesn't fit.

The context window is limited, but applications handle overflow differently. They may reject a long input, truncate older messages, summarize history, or retrieve selected notes. A model can only use material provided in the current inference request; check the product's actual context-management behavior before assuming which earlier details remain available.

In recent years context windows have grown dramatically; today some models can take in hundreds of thousands, even millions, of tokens at once. This makes it possible for a model to read a long contract or an entire case file in one go and answer questions about it. Still, no matter how large the context window grows, the distinction between the permanent knowledge the model gained through training and the temporary information specific to the current conversation always holds. We clarify that critical distinction in the next section.

The Difference Between Training and Inference

One of the most commonly confused aspects of LLMs is the difference between the moment the model "learns" and the moment it is "used." Training is the process in which the model acquires knowledge by adjusting its parameters from massive text; it takes weeks, is extremely expensive, and is done only once (or rarely). You can compare it to the long years spent in school: intensive and costly, but a one-time period of accumulation.

Inference, on the other hand, is the moment you take the trained model and ask it a question. Here the model no longer learns anything new; its parameters are fixed, essentially frozen. The model simply uses those fixed dial positions to produce a response to your text. So when you ask ChatGPT something, the model does not "learn" from you; it merely applies what it learned earlier to your current context. It's like a graduated expert using their existing knowledge to solve a new case, without needing to re-earn their diploma.

A model's parameters usually do not change during an ordinary inference request. Its application may still store conversation history, saved memories, or documents and supply them in a later request; that storage is separate from parameter training. A training cutoff also does not prevent the application from accessing newer information through search or retrieval. Keep these three layers—model weights, current context, and external storage—distinct.

What Can It Do, What Can't It? Hallucination and Limits

LLMs are extraordinarily good at summarizing, rewriting, translating, classifying, generating ideas and linguistic tasks in general. They can explain a topic at different levels, change the tone of a text, write code, or simplify a dense paragraph. Where they are strong is in tasks dominated by language and patterns, where a single exact correct answer isn't required.

Their best-known weakness, by contrast, is the phenomenon called "hallucination": the model producing information that sounds perfectly confident and fluent but is actually wrong, or even entirely made up. The reason, as we saw earlier, is that the model is not pulling facts from a database but generating what "looks statistically likely." Citing a court ruling that doesn't exist or giving a wrong date stems not from ill intent but from its nature, which optimizes for probability rather than truth. This is exactly why, in sensitive domains, an LLM's output must always be independently verified.

Models can miss recent events, make arithmetic errors, and reflect biases. Tools and retrieval-augmented generation (RAG) can give them current source material, but retrieval may select the wrong passage and a generated answer may misread a correct one. Verify source dates, access rights, and whether each cited passage actually supports the claim.

For a document assistant, a practical design is to retrieve permitted source passages, attach stable document identifiers, and require the answer to point to those passages. A reviewer should be able to open each source and check the statement. This is an engineering example; it does not claim that every Hatson Tech deployment already implements it. Adapting a model like this to your own terminology and tasks is the subject of our large language model training and fine-tuning page.

Sources and Further Reading