Before anything else, we need to agree on what the word “learning” means when the learner is a computer program.

When a person learns, they gain memories, habits and understanding. When Claude “learns,” something more mechanical and more interesting happens: a huge collection of numbers inside a program is adjusted, again and again, until the program becomes good at a task. Those numbers are called parameters or weights. You can think of them as millions upon millions of tiny dials. Learning is the process of turning those dials a little at a time so that the program’s answers get better.

Claude is what people call a large language model, often shortened to LLM. “Language model” means a program built to work with words and sentences. “Large” means it has a very big number of dials and was trained on a very big amount of text. Claude is made by a company called Anthropic.

Computers cannot read letters the way you do, so the very first job is translation.

A computer only understands numbers. So before any learning can happen, text has to be chopped into small pieces and each piece has to be given a number. These pieces are called tokens. A token can be a whole short word, a part of a longer word, or even a single punctuation mark. A common word like “water” might be one token, while a rare long word might be split into several.

Why not just use whole words? Because languages have an endless supply of words, names, spelling mistakes and made-up terms. Pieces are more flexible. With a few tens of thousands of well-chosen pieces, a model can build almost any word, in many languages.

Meaning as a position on a map

After tokens get their numbers, each one is turned into an embedding. Picture a giant map where every token has an address. Words with related meanings, like “doctor” and “nurse,” end up as neighbours. Words that rarely appear in similar situations, like “doctor” and “volcano,” live far apart. Importantly, nobody draws this map by hand. It forms by itself during training, because treating similar words similarly helps the model make better guesses.

The first and largest stage of learning is called pretraining, and it rests on one deceptively simple game.

During pretraining, the model is shown an enormous amount of text and asked, over and over, to predict the next token. Take the half-finished sentence “The cat sat on the …”. A good guess is “mat” or “sofa.” A bad guess is “galaxy.” The model makes a guess, the real answer is revealed, and the model is told how far off it was. Then its dials are nudged so that a similar guess next time is a little better.

This sounds too simple to produce anything impressive. But consider what it takes to guess well. To complete a sentence about a chess game, you need to know something about chess. To finish a recipe, you need to know how cooking works. To continue a sad story, you need a feel for emotions. To predict the next word in a legal argument, you need to follow the logic. The simple game forces the model to pick up grammar, facts, reasoning patterns, writing styles and a great deal more, because all of those help with guessing.

flowchart LR
  A["Text from the training data"] --> B["Hide the next token"]
  B --> C["Model makes a guess"]
  C --> D["Compare guess with the real token"]
  D --> E["Measure the error"]
  E --> F["Nudge the dials slightly"]
  F --> B
        

The learner itself is a particular kind of program, and understanding its shape makes everything else easier.

Claude is built on a neural network, a program loosely inspired by the way brain cells connect. It is made of many layers. Numbers enter at one end, each layer transforms them using the current dial settings, and a prediction comes out the other end. The learning in the previous chapter is simply the process of adjusting those dial settings.

Modern language models use a design called the transformer. Its most important trick is called attention. Attention lets the model look at all the words in the context and decide which ones matter most for understanding any given word. Consider the sentence: “The trophy did not fit in the suitcase because it was too big.” What does “it” refer to? A person quickly sees it means the trophy. Attention is the mechanism that lets a model weigh the surrounding words and connect “it” to the right thing.

flowchart TD
  A["Your text"] --> B["Split into tokens"]
  B --> C["Tokens become embeddings"]
  C --> D["Many layers with attention"]
  D --> E["Scores for every possible next token"]
  E --> F["Pick one token"]
  F --> G["Add it to the text and repeat"]
        

Notice the last step. When Claude writes an answer, it produces one token, adds it to the conversation, and then produces the next one, over and over. The same guess-the-next-piece skill built in training is what writes the reply.

The quality of what a learner reads shapes the quality of what it learns.

Pretraining needs a very large and varied amount of text. Anthropic has described that Claude’s training data comes from several kinds of sources, including publicly available information from the internet, data licensed from other companies, data supplied by people who have agreed to share it, and data created synthetically, meaning produced by other AI systems for training purposes. Anthropic has also said that when it collects public web pages with its own crawler, the crawler follows website owners’ instructions about whether they allow it and does not go behind passwords or logins.

Raw text from the world is messy. It contains duplicates, broken pages, spam and plenty of low-quality writing. So the data is cleaned and filtered before use. The goal is a rich mixture: everyday conversation, explanations, stories, reference material, and writing about many subjects and in many languages.

Let us slow down and watch one tiny moment of learning as it unfolds.

A model that has only finished pretraining is clever, but it is not yet a good conversation partner.

After pretraining, the model is a powerful text continuer. If you give it a question, it might continue with another question, or with a made-up forum thread, because that is the kind of text it saw. It has absorbed knowledge, but it has not learned the manners and goals of an assistant. The next stage, often called fine-tuning, teaches those.

In fine-tuning, the model is trained further on carefully prepared examples of good conversations: a person asks something, and a high-quality answer follows. The model keeps playing the same predict-what-comes-next game, but now the text it practises on shows the style of response we want, which is clear, honest, useful and appropriately careful. This stage is usually much smaller than pretraining, but it changes how the model behaves enormously.

flowchart LR
  A["Pretraining: learn from vast text"] --> B["Fine-tuning: learn from example conversations"]
  B --> C["Feedback training: learn what people prefer"]
  C --> D["Testing and safety checks"]
  D --> E["Release as an assistant"]
        

Some qualities, like being genuinely helpful, are easier to recognise than to describe, so the model also learns from judgments.

It is hard to write a rule that says exactly what a “good” answer is. But people can usually tell which of two answers is better. This observation powers a method called reinforcement learning from human feedback, shortened to RLHF. The word “reinforcement” means rewarding behaviour you want so that it becomes more likely.

In a simplified picture, the model writes several possible answers to the same question. People compare them and say which is better. Those comparisons are used to build a separate helper program, called a reward model, that learns to predict which answers people would prefer. The main model is then trained to produce answers that the reward model scores highly. Over many rounds, the assistant drifts toward responses that people find clearer, more useful and more trustworthy.

Anthropic has described an additional approach that uses written principles to guide behaviour.

Collecting human judgments for every possible situation is slow and expensive, and it can hide the reasons behind the choices. Anthropic developed a technique it calls Constitutional AI. The idea is to give the model a written set of principles, a kind of “constitution,” and have the model use those principles to critique and improve its own draft answers. Then the improved answers are used as training examples, and AI-generated judgments based on the principles can help provide feedback signals too.

The benefit is transparency. Instead of values being buried inside thousands of unexplained individual ratings, the guiding ideas can be written down, read, discussed and revised. Anthropic has also published a longer document describing the character and values it hopes Claude will have, covering things like honesty, care for the people it talks with, and avoiding serious harm. The details of exactly how these ideas are applied in each training run are not fully public, so it is best to treat any simple summary, including this one, as a high-level picture.

Learning is only useful if we can check what was actually learned.

Before a model is released, it goes through many rounds of testing. Developers run it through collections of questions called benchmarks, which measure skills such as reading, mathematics, coding and reasoning. They also run safety evaluations, checking whether the model can be pushed into giving dangerous help, producing biased output or behaving deceptively.

An important practice is red teaming. A red team is a group whose job is to attack the system on purpose, looking for weaknesses before real bad actors find them. Anthropic has also published a framework describing how it ties the level of safety measures to how capable its models become. The more powerful the model, the stronger the safeguards and the more careful the testing are meant to be.

This is the question most beginners care about, so let us answer it carefully.

In everyday use, Claude’s weights are frozen. When you send a message, the dials do not move. The model reads everything in the conversation, produces a reply, and that is all. So it does not instantly become smarter or change its beliefs because of what you told it. The proper word for what happens inside a conversation is in-context learning: the model can adapt to the examples, instructions and facts you place in the conversation, but only while they remain in its view.

The amount of text the model can see at once is called the context window. Think of it as the model’s working desk. Everything on the desk can be used for the current answer. When a brand-new conversation starts, the desk is cleared.

So how can Claude seem to remember things?

Some products add extra features around the model. For example, a memory feature can save short notes about your preferences and place them back into the model’s view at the start of later chats. A search tool can fetch fresh information from the web and put it on the desk. These help Claude use up-to-date or personal information, but they are different from changing the weights. The notes live outside the model and can be viewed, edited or removed depending on the settings of the product you use.

What about future versions?

Companies may use some conversation data to help improve future models, depending on their policies and the choices offered to users. Even then, that happens later, in a new, deliberate training run done by the developers, not live during your chat. If this matters to you, the privacy settings and policies of the product you use are the right place to check what applies.

flowchart TD
  A["Frozen trained model"] --> B["Reads your conversation and any shared documents"]
  C["Optional saved notes or search results"] --> B
  B --> D["Writes a reply"]
  D --> E["Chat ends and the working desk is cleared"]
  F["Developers prepare a new training run"] -.-> G["A new model version is released later"]
        

Understanding how learning works also explains why mistakes happen.

Take hallucination. During training, the model is rewarded for producing text that looks like the real thing. A fake-but-believable book title looks very much like a real one. If the model does not truly know a fact, the habit of fluent guessing can fill the gap with something plausible. Developers work to reduce this by training the model to say “I do not know” when appropriate, and by letting it consult reliable sources through tools, but the problem has not vanished.

A few wrong ideas appear again and again. Let us clear them up.

Every method of learning has strengths and weaknesses, and the one used for language models is no exception.

Not in the sense of changing its weights. Within a conversation it can use what you tell it, but the underlying model stays the same. Lasting change comes only from new training runs carried out by developers.

Because it generates likely-sounding text from patterns rather than looking facts up in a verified list. When it lacks solid knowledge, fluent guessing can produce errors. Using tools like search and checking important facts both help.

The core of language-model training is text, but some modern models can also work with images and other inputs. This guide focuses on the text-based learning that forms the foundation.

Pretraining is the long first stage in which the model learns language and knowledge by predicting the next token across a huge amount of text. Fine-tuning is a shorter later stage that teaches the model to behave as a helpful, honest assistant using curated examples.

Human feedback is an important part of how assistants are shaped, with reviewers comparing answers according to guidelines. Because people cannot review everything, the judgments are also used to train helper models and, in some approaches, AI-generated feedback guided by written principles supplements them.

Its training text stops at a certain date, called the knowledge cutoff. Anything newer is unknown to the weights unless the product gives the model a search tool or you paste the information into the conversation.

The model itself does not, since each new conversation starts with a clear desk. Some products add a separate memory feature that saves short notes and shows them to the model again later. Whether that is available, and how to control it, depends on the product and its settings.

There are loose similarities, such as improving through practice and feedback, but the details are very different. A child learns from a lifetime of physical experience, emotions and relationships. Claude learns from text and training signals, in a process that is far more compressed and mechanical.

Claude’s learning is a layered story. First comes pretraining, where a very large network learns language, facts and reasoning patterns by playing the simple game of predicting the next token. Next comes fine-tuning, where it practises on example conversations to behave like an assistant. Then feedback methods, including human preferences and written principles, polish its helpfulness, honesty and care. Extensive testing checks the result before release. Once released, the model’s weights stay fixed, and it adapts within each chat by reading what is in front of it. Understanding this picture makes it easier to use Claude well: lean on its strengths, supply the context it needs, and verify what matters.