What an LLM actually is
A large language model predicts the next token, over and over. Understand training vs inference, why models hallucinate, and what 'knowing' really means for an LLM.
A does exactly one thing: given some text, it predicts what comes next. Answering questions, writing code and summarizing documents are all that one trick, repeated.
The loop
You send text. For every possible next (a word or piece of a word), the model works out how likely it is to come next. That list of chances is a .
"The capital of France is" →
- Paris92%
- the4%
- a2%
- located1%
One token is picked and added to the end of the text. Then the whole thing runs again: "The capital of France is Paris" gets a fresh list of chances, this time favoring a period. This repeats until the model decides it is finished or hits a length limit. A few hundred rounds give you a paragraph.
Your text so far
"The capital of France is"
Model scores every token
a chance for each possible next token
One token is picked
" Paris"
It is added to the text
"…France is Paris"
Then the longer text goes back in, and the next token is predicted.
Nothing in that loop looks anything up. The model isn't checking a database of facts. It has : billions of numbers that captured patterns in its training text. "Paris" follows "the capital of France is" because that pattern appeared again and again in what it learned from.
Training vs inference
Training happened once, months ago, on thousands of machines. It produced a fixed set of weights. Inference is what happens when you call the : your input runs through those frozen weights to produce an answer.
So your prompt doesn't teach the model anything, your conversation doesn't change it, and the next person's request isn't affected by yours.
Under the hood — What is a "weight", and why is it frozen?
A model is, physically, a very large collection of numbers stored in files — billions of them. Each number is a weight: a learned strength of connection that decides how much one piece of information influences another.
Training is the process that found those numbers. It repeatedly showed the model text, let it predict the next token, measured how wrong it was, and nudged every weight slightly in the direction that would have been less wrong. That nudging is , and it is the expensive part: months of computing.
Once training finished, the numbers were saved and never changed again. That is what "frozen" means. A forward pass is what happens on every API call: your tokens flow through those fixed numbers, arithmetic happens, probabilities come out the other end. Nothing is written back. This is why the model cannot learn from your conversation — there is no mechanism during a forward pass to change anything.
Why hallucination is not a bug
When a model confidently states something false, that's called a . It happens because the model is built to produce plausible text, not true text.
When it knows a fact well, plausible and true are the same thing. When it doesn't, it doesn't stop. It still writes whatever sounds most likely, which is how you get a confident citation to a paper that doesn't exist.
The rest of this course teaches three ways to reduce that:
- give the model the source text to work from ()
- force its answer into a shape you can check ()
- let it call your real code for real answers ()
None of these make the model smarter. They give it fewer ways to be wrong.
The vocabulary you will meet
- — the chunk the model actually reads. Roughly ¾ of a word in English. You are billed by them and limited by them.
- — how many tokens the model can consider at once: your prompt, the conversation history, and its own answer.
- — the frozen numbers produced by training.
- — running the model on your input (one per token). What you pay for per call.
- — how adventurous the model is when it picks the next token. Covered in a later lesson.
One more distinction: a is not an application. Gemini is a model. The Gemini app, ChatGPT and the assistant you will build in this course are applications built around a model. Keeping these apart makes the difference between an LLM and an much easier to see later.
Key takeaways
- An LLM predicts the next token, one at a time, over and over. That is the whole mechanism.
- Its weights are frozen: it doesn't learn from your prompts or remember earlier calls.
- It aims for plausible text, not true text, which is why it can confidently make things up.
Sign in to run the exercise
Reading is free. Writing code here needs an account so we have somewhere to keep your Gemini key and the +15 XP you are about to earn.