Before looking at gears and numbers, let us agree on what we are talking about.

The definition in one sentence

A large language model is a computer program that has learned the patterns of human language by studying an enormous amount of text, and that uses those patterns to read, write, summarize, translate, and answer questions. It does not follow a hand-written list of grammar rules. Instead, it absorbed statistical habits of language from billions of pages and now applies those habits to whatever you give it.

The name, one word at a time

What an LLM is not

An LLM is not a search engine. A search engine looks up existing web pages and shows them to you, while an LLM writes new text one piece at a time. It is also not a database, because it does not keep a neat table of facts that it can query. And it is not a person, even though it speaks in the first person and can sound warm.

Where you already meet LLMs

You probably use LLMs without noticing. They draft and polish emails, write summaries of long meetings, power customer-support chat windows on shopping and banking sites, suggest code as developers type, and produce the short summaries that now sit above some search results. Behind each of those features is the same basic machine, wrapped in different software.

LLMs look like a sudden arrival, but they sit at the end of a long road of small ideas.

Understanding the history helps you see why LLMs work the way they do. Each step below solved a specific problem left behind by the step before it.

Strip away the mystery and an LLM does one job, over and over.

One job: guess what comes next

Given some text, an LLM calculates which piece of text is most likely to come next. That is the entire task. If the text so far is “The capital of France is”, the model assigns a very high chance to ” Paris”, a tiny chance to ” Lyon”, and almost no chance to ” banana”.

How one answer is built

The model does not write a whole paragraph in one go. It produces a single piece of text, adds it to the conversation, and then predicts the next piece, again and again. Each new piece is chosen from a list of probabilities. This loop is why you see answers appear word by word on screen. It is not just a visual effect; the model truly generates them one at a time.

Why such a simple task produces such rich behavior

To guess the next word well across billions of different texts, the model has to pick up many things along the way. It must learn grammar, because ungrammatical guesses are usually wrong. It must learn facts, because “the tallest mountain on Earth is” demands knowledge. It must learn tone and style, because a legal letter continues differently from a joke. It must even learn patterns of reasoning, because many texts contain worked-out arguments and solutions. These abilities are side effects of getting very good at one narrow game.

A computer cannot work with letters directly, so text is first chopped into small, numbered pieces.

What a token is

A token is a small chunk of text that the model treats as a single unit. A token can be a whole common word like “table”, a piece of a longer word like “believ” plus “able”, a bit of punctuation, or even a single space-and-letter combination. The piece of software that does the chopping is called a tokenizer.

Why not use whole words or single letters?

Designers tried both extremes. Whole words create a vocabulary so large that rare words, names, and typos break the system. Single letters make every sentence very long, which slows the model and makes it harder to see meaning. Subword tokens strike the balance: common words stay whole, and rare words are assembled from familiar pieces.

From text to numbers

Each token in the vocabulary has a number, called a token ID. A common method, byte pair encoding, builds the vocabulary by repeatedly merging the pairs of characters that appear together most often in the training text. The final vocabulary usually holds tens of thousands to a couple of hundred thousand entries. When you type a sentence, the tokenizer turns it into a list of these IDs, and that list is what the model actually receives.

Why tokens matter to you

As a rough guide, one token in English is about four characters, or three quarters of a word, so 1,000 tokens is around 750 words. This matters because nearly everything about an LLM is measured in tokens: how much text it can read at once, how long its answer can be, how fast it responds, and how much a provider charges. Many non-English languages need more tokens for the same sentence, so they can cost more and fit less into the model’s working space.

A token ID is just a label. To work with meaning, the model needs something richer.

What an embedding is

An embedding is a long list of numbers that represents the meaning of a token. Where a token ID such as 4821 says only “this is entry number 4821”, an embedding says “this token is a bit like other animal words, a bit like a friendly pet, and nothing like a tax form”. The model learns these lists during training, so that tokens used in similar ways end up with similar numbers.

Why this helps

Once words are positions in a space, the model can do arithmetic with meaning. A famous early example from word2vec showed that the numbers for “king” minus “man” plus “woman” land very near the numbers for “queen”. Modern embeddings are far richer, and they change depending on the sentence, so the word “bank” ends up in different places in “river bank” and “bank account”.

Order matters too

The Transformer reads all tokens together, so it needs a way to know which came first. The model adds position information to each embedding. Without it, “the dog bit the man” and “the man bit the dog” would look like the same bag of words.

Embeddings outside the model

Embeddings are useful on their own. Companies use them for semantic search, where a query like “how do I get my money back” finds a page titled “Refund policy” even though they share no words. Recommendation systems, duplicate detection, and document clustering use them too. In Chapter 14 you will see how they power a popular technique called retrieval-augmented generation.

The word “large” is not marketing. It refers to something you can count.

Neural networks in plain words

An LLM is built from a neural network, a structure loosely inspired by the brain. It is arranged in layers of simple units. Each unit takes in numbers, multiplies them by adjustable weights, adds them up, and passes the result onward. No single unit is clever. The skill comes from millions of them working together in layers.

Parameters are the knobs

The adjustable weights are called parameters. Training is the process of tuning all of them until the network predicts text well. When people say a model has 70 billion parameters, they mean 70 billion numbers that were tuned, and that together hold everything the model has learned.

Why size matters, and its cost

Each parameter is stored as a number that takes up memory, commonly two bytes. A model with 70 billion parameters therefore needs about 140 gigabytes just to hold its numbers, before any extra working memory. That is far beyond a normal laptop, which is why large models run on data-center computers with many specialized chips called GPUs. Smaller models, with a few billion parameters, can run on a laptop or even a phone, at the price of being less capable.

Scaling laws

Around 2020, researchers noticed that model quality improves in a smooth, predictable way as you increase three things together: parameters, training data, and computing power. Later, DeepMind’s Chinchilla study in 2022 showed that many earlier models were too big for the data they saw. A 70-billion-parameter model trained on about 1.4 trillion tokens beat much larger models trained on less. The lesson is that bigger is not automatically better; balance between model size and data matters.

This is the architecture inside almost every modern LLM, and its secret ingredient is learning where to look.

The big picture

The Transformer takes a list of token embeddings and passes them through a tall stack of identical blocks. Each block lets the tokens exchange information and then refines each token’s representation. After the last block, a final layer converts the result into probabilities for the next token. GPT-3, for example, stacked 96 such blocks.

flowchart TD
    A["Your text"] --> B["Tokenizer"]
    B --> C["Token IDs"]
    C --> D["Embedding layer"]
    D --> E["Add position information"]
    E --> F["Transformer block 1"]
    F --> G["Transformer block 2"]
    G --> H["Many more blocks"]
    H --> I["Output layer"]
    I --> J["Probability for every possible next token"]
    J --> K["Choose one token"]
    K --> L["Add it to the text and repeat"]

Read the diagram from top to bottom. Your text is split into tokens, the tokens become IDs, the IDs become embeddings with position information, and the stack of blocks does the heavy thinking. At the bottom, the model scores every possible next token, one is chosen, and the loop starts again with the longer text.

Attention: deciding what to focus on

Consider the sentence “The trophy did not fit in the suitcase because it was too big.” What does “it” refer to? You know it means the trophy, because a big suitcase would not be the problem. A model needs a way to make the same link. Attention is the mechanism that lets each token look at all the other tokens and decide how much weight to give to each one when updating its own meaning.

Queries, keys, and values

Inside attention, each token creates three versions of itself. The query says what this token is looking for. The key says what this token offers. The value holds the information it will hand over. The model compares each query with every key, and the better the match, the more of that token’s value flows in. Think of a library: your query is the topic you want, the keys are the labels on the shelves, and the values are the books you pull down.

Many heads, many highlighters

One round of attention can only capture one kind of relationship at a time. So each block runs many attention “heads” side by side, a bit like using highlighters in different colors: one for grammar links, one for who-did-what, one for nearby words, and so on. Their results are combined, giving a much richer view of the sentence.

Inside a single block

flowchart TD
    A["Input from previous layer"] --> B["Self-attention with many heads"]
    B --> C["Add the input back and normalize"]
    C --> D["Feed-forward network"]
    D --> E["Add the input back and normalize"]
    E --> F["Output to next layer"]
    A -.-> C
    C -.-> E

Why it beat older designs

Older networks such as LSTMs read text strictly one token after another, like a person reading with a finger under each word. That made them slow to train and forgetful over long passages. A Transformer processes all tokens at once during training, which suits GPUs, and attention lets any word connect directly to any other word, however far apart they are. Together these two traits made training on internet-scale data practical.

Decoder-only models and the “no peeking” rule

Chat-style LLMs such as the GPT, Claude, and Llama families use a version called decoder-only. During training, each token is allowed to look only at tokens before it, never at the ones that follow. This “no peeking” rule, called causal masking, keeps the game honest: the model must truly predict the next token, not copy it. BERT, in contrast, was built to look both ways and is better at understanding than at writing.

Training turns a blank network of random numbers into something that can hold a conversation.

flowchart TD
    A["Raw text from the web, books, and code"] --> B["Cleaning and removing duplicates"]
    B --> C["Tokenization"]
    C --> D["Stage 1: Pretraining by predicting the next token"]
    D --> E["Base model"]
    E --> F["Stage 2: Supervised fine-tuning on example conversations"]
    F --> G["Stage 3: Preference tuning using human feedback"]
    G --> H["Safety testing and red teaming"]
    H --> I["Released assistant model"]

Stage 0: Preparing the data

Before any learning, teams gather huge collections of text: web pages, books, articles, encyclopedias, and computer code. Then comes the unglamorous but vital cleaning. Duplicate pages are removed, low-quality and spam text is filtered out, and personal data is scrubbed where possible. Data quality has an outsized effect on the final model, and many labs treat their exact data recipe as a closely guarded secret.

Stage 1: Pretraining

In pretraining, the model plays the next-token game on trillions of tokens. It sees a stretch of text with the next token hidden, makes a guess, and measures how wrong it was using a score called the loss. An algorithm called backpropagation works out which parameters contributed to the error, and another method, gradient descent, nudges each of them a tiny amount in a better direction. This happens an astronomical number of times, spread across thousands of GPUs, for weeks or months, and can cost millions of dollars in computing.

The result is a base model. It is very good at continuing text, but it is not yet a helpful assistant. Ask a base model “What is the capital of France?” and it might continue with a list of more quiz questions, because that is what such a line often looks like in its training text.

Stage 2: Supervised fine-tuning

To make the model behave like an assistant, teams train it further on a much smaller set of carefully written examples: a question or instruction, paired with an ideal answer. The model learns the format of being helpful, how to follow instructions, how to answer politely, and when to say it is unsure. OpenAI’s InstructGPT research in 2022 showed that this kind of tuning can make a far smaller model preferred by users over a much larger untuned one.

Stage 3: Learning from preferences

Next, the model is shown several different answers to the same prompt, and human reviewers rank which they prefer. A separate scoring model learns those preferences, and the LLM is then trained to produce answers that score well. This approach is known as reinforcement learning from human feedback, or RLHF, and newer simpler variants exist too. Some labs also write down a set of principles and have models critique their own answers against them. This stage shapes tone, helpfulness, and refusal of harmful requests.

Stage 4: Testing before release

Finally, the model is tested hard. Evaluation suites measure skills like math, coding, and factual accuracy. Red teams, groups of people paid to attack the model, try to make it produce dangerous or biased output so the problems can be fixed before the public sees them.

Training versus using

Training is a one-time, enormously expensive event done by the model’s creator. Using the model, called inference, is what happens every time you send a message, and it does not change the parameters. This is why an LLM does not learn from your conversation in the way a person would; the next chapter shows what actually happens during use.

Follow a single question from your keyboard to the answer on your screen.

The prompt

The text you send is called a prompt. Real products usually add more text to it behind the scenes: a system prompt that sets the assistant’s role and rules, the earlier messages in the conversation, and sometimes documents or tool results. All of it is joined into one long sequence of tokens.

The KV cache: not repeating work

Generating each new token requires attention over all earlier tokens. Redoing the calculations for the earlier ones every time would be wasteful, so servers store the keys and values already computed in a memory area called the KV cache. It makes generation much faster, but it takes up valuable GPU memory, and it grows with every token in the conversation. This is one big reason long conversations cost more to serve.

The context window

Every model has a limit on how many tokens it can consider at once, called the context window. It covers your prompt, the history, any attached documents, and the answer being written, all together. Modern models range from a few thousand tokens to hundreds of thousands or more. When a conversation exceeds the limit, older material must be dropped or summarized.

Settings that change the answer

Serving at scale

A popular assistant answers millions of requests a day, so the serving system around the model matters as much as the model. A typical setup puts a load balancer and API gateway at the front, a layer of checks for safety and limits, and a fleet of GPU servers that run the model. Requests are grouped into batches so that GPUs stay busy, and logs and metrics flow out to monitoring tools.

flowchart TD
    A["Client app or website"] --> B["API gateway"]
    B --> C["Authentication and rate limiting"]
    C --> D["Load balancer"]
    D --> E["Input safety checks"]
    E --> F["Request queue and batching"]
    F --> G["GPU inference server 1"]
    F --> H["GPU inference server 2"]
    F --> I["GPU inference server N"]
    G --> J["Output safety checks"]
    H --> J
    I --> J
    J --> K["Streamed response to client"]
    G --> L["Logs and metrics"]
    H --> L
    I --> L
    L --> M["Monitoring dashboards and alerts"]

Two speed measures matter most to users. Time to first token tells you how long you wait before anything appears, and tokens per second tells you how quickly the rest streams in. Providers tune both with techniques such as batching, caching, and using smaller helper models to speed up generation.

The skills below all come from the same trick of predicting text, applied to different kinds of text.

Real-world examples

Knowing where an LLM fails is as valuable as knowing where it shines.

Hallucination: fluent and wrong

A hallucination is an answer that sounds confident but is false or made up, such as an invented book title, a fake web address, or a non-existent legal case. It happens because the model is built to produce plausible text, and a plausible-sounding falsehood can score just as well as a truth when the model lacks the facts. The model has no built-in feeling of “I am not sure”, though training can reduce the problem.

Other limits to keep in mind

Does it understand?

Experts genuinely disagree. Some argue that modeling language well enough requires building a real internal picture of the world, and point to the models’ ability to solve new problems. Others argue that the model only manipulates patterns of text without grounding in real experience. For practical work you do not need to settle this. What matters is that the system is powerful, uneven, and best treated as a capable assistant whose work needs review.

The big trade-offs

An LLM follows instructions written in ordinary language, which makes it both easy to use and easy to trick.

The main threats

The OWASP project, a well-known security community, publishes a “Top 10 for LLM Applications” that lists risks like these and is a good starting point for any team building with LLMs.

Defense in layers

No single fix stops all attacks, so teams stack several protections, much like a building that has a fence, a locked door, and a guard.

Privacy and compliance

Before sending customer or employee data to any provider, check where it is stored, whether it is used for training, and how long it is kept. Regulated fields such as health and finance may need contracts, regional hosting, or self-hosted models. Redact personal data when it is not needed for the task.

An LLM feature that nobody watches will quietly get slower, costlier, or worse.

What to measure

Logging with care

Useful logs record the prompt version, model name and settings, token counts, timing, and a trace ID that links all the steps of one request. Storing full prompts and replies helps debugging, but those can contain personal or confidential data, so redact sensitive fields, restrict who can read logs, and set a retention period.

Testing quality over time

Models change when providers update them, and prompts change when teams edit them. To catch regressions, keep a fixed set of representative questions with known good answers, and rerun it before every change. Scoring can combine automatic checks, human review, and sometimes another LLM acting as a judge, which is fast but should itself be spot-checked by people.

Most real products wrap the raw model in a handful of well-proven patterns.

Retrieval-augmented generation

Retrieval-augmented generation, or RAG, solves two LLM weaknesses at once: missing knowledge and hallucination. Instead of trusting the model’s memory, the app first searches a collection of trusted documents for passages relevant to the question, then places those passages in the prompt and tells the model to answer from them.

flowchart TD
    subgraph P1["Preparation done in advance"]
        D1["Company documents"] --> D2["Split into chunks"]
        D2 --> D3["Create an embedding for each chunk"]
        D3 --> D4["Vector database"]
    end
    subgraph P2["At question time"]
        Q1["User question"] --> Q2["Create an embedding of the question"]
        Q2 --> Q3["Search the vector database for closest chunks"]
        Q3 --> Q4["Build prompt with question and top chunks"]
        Q4 --> Q5["LLM writes the answer"]
        Q5 --> Q6["Answer with source references"]
    end
    D4 --> Q3

The diagram has two halves. In the preparation half, documents are split into manageable chunks, each chunk is turned into an embedding, and the embeddings are stored in a vector database that can quickly find the closest matches. At question time, the question is embedded the same way, the closest chunks are retrieved, and the model writes an answer based on them. Because the sources are known, the answer can include references that a reader can verify.

More patterns worth knowing

Prompting, RAG, or fine-tuning?

Teams often ask which technique to start with. A useful rule is to begin with the simplest one and move on only when it falls short.

The same few errors catch nearly every new team. Here is how to sidestep them.

Best practices

Common mistakes

• Treating the first impressive demo as proof that the system is ready for real users.
• Forgetting that every token costs money and time, and then pasting giant documents into every request.
• Assuming the model has current information when its training ended long ago.
• Changing a prompt without rerunning tests, which quietly breaks older cases.
• Logging full conversations without thinking about privacy.

Short answers to questions that come up again and again.

You now have the full picture, from a single token to a production system.

A large language model is a very large neural network that learned the patterns of language by predicting the next token across a vast collection of text. Text is split into tokens, tokens become embeddings, and a stack of Transformer blocks uses attention to decide what to focus on before scoring the next token. Training happens in stages, from broad pretraining to instruction tuning to feedback-based shaping, and then the finished model is used over and over without further learning. Its strengths are fluency and flexibility; its weaknesses are invented facts, inconsistency, and exposure to tricky input. Good products surround the model with sources, tools, checks, monitoring, and human review.