How Large Language Models Work — training and inference
How large language models work, shown on an interactive canvas: pre-training, fine-tuning, and the token-by-token loop that produces every response.
A large language model is a next-word predictor at scale: it learns statistical patterns from enormous amounts of text, then generates responses one token at a time by predicting what comes next.
How Large Language Models Work — training and inference
The interactive FlowJam canvas for this explanation — every lane, row and arrow above is a real QueryChart diagram you can open and edit.
How to read this visual
- Read the "Training" column first — it explains how the model got its ability — then the "Inference" column for how that ability is used.
- The feedback loop is the key move: the "Yes" branch of "Keep generating?" points back into the attention layers, and the "Stop" branch flows down to the decoded text.
- The three inference rows are the pipeline: input becomes tokens, tokens become predictions, predictions become output.
Training: where the model comes from
"Pre-training on huge amounts of text" is the foundation — the model is exposed to an enormous corpus and trained on one task: predict a missing word. From that single objective it learns grammar, facts and reasoning patterns. "The model learns statistical patterns between words" makes the nature of that learning explicit, and "Fine-tuning adapts the model to a specific task" is the steer — a smaller curated dataset that changes behaviour toward a desired style or domain.
Inference: turning input into output
"The prompt is split into tokens" and "Tokens become numerical embeddings" are the input half: text is broken into small units and converted into vectors the model can compute with. "Attention layers weigh which words matter together" is the mechanism that gives context its power — the word "bank" is read differently next to "river" than next to "money" because attention lets every token consider every other token.
The generation loop
"The model predicts the most likely next token" is the atomic act of generation. "Keep generating?" makes the loop explicit: each prediction is fed back as new input, which is why responses are built one token at a time rather than composed whole. "Tokens are decoded back into readable text" and "The response is returned to the user" complete the pipeline — the output the user sees is the decoded tail of that loop.
Key relationships and takeaways
- An LLM is a next-token predictor, not an answer-retrieval system — every response is a chain of single-token predictions.
- Pre-training supplies the capability; fine-tuning and prompting steer it toward a task.
- Tokens, not characters or whole words, are the units the model thinks in.
- Attention lets context reshape meaning, which is what makes the same words mean different things in different sentences.
- Generation is recursive: each output token becomes new input, which is why responses can be long, coherent and wrong with equal confidence.
When to use this visual
- Explaining to a non-technical audience what an LLM actually does and why it can be confidently wrong.
- Teaching a product team the difference between training, fine-tuning and prompting so they can reason about model behaviour.
- Grounding a discussion of token limits, cost and latency in the token-by-token mechanics.
How it works
Annotate the training stage you rely on
On the pre-training box, note the model family you use and whether it was open- or closed-weight; on fine-tuning, note whether you actually fine-tune or prompt instead.
Add the system prompt
Insert a step before the prompt is tokenized where the system instructions are injected, since for most users that is where behaviour is actually shaped.
Show the safety and guardrail layer
Add the moderation step between the prediction loop and the output, ending in an explicit block or rewrite, to reflect how production systems gate generation.
Track the tokens
Annotate the tokenizer step with the actual token budget of your model, and the decode step with the streaming that shows output as it is generated.
Frequently asked questions
What is a large language model?
A large language model (LLM) is a neural network trained to predict the next word in a sequence, at enormous scale. From that single objective it learns grammar, facts, reasoning and style well enough to generate fluent text. The name describes the mechanism: it is large (billions of parameters), trained on language, and fundamentally a next-token predictor.
Why does an LLM generate text one token at a time?
Because its only operation is predicting the most likely next token given everything so far. After each prediction, that token is added to the input and the process repeats — the "Keep generating?" loop in the visual. That recursive design is what lets the model produce arbitrarily long text, and also why output can drift or repeat: each step only sees the tokens it has already generated.
What is the difference between pre-training and fine-tuning?
Pre-training is the massive initial phase: the model learns general language patterns from terabytes of text, which gives it its raw ability. Fine-tuning is a smaller, later phase on curated data for a specific task or style, which changes how the pre-trained model behaves. Most users of LLMs never fine-tune — they prompt, which steers the same model at inference time.
Why do LLMs sometimes give wrong answers confidently?
Because the model optimises for likely text, not for truth. It has no internal database or correctness checker; it produces the continuation that fits its training patterns. When the pattern is a plausible-sounding claim, it states it confidently. That is why the visual describes the model as a predictor rather than a retriever — the honesty caveat is a property of the mechanism, not a bug that will be patched by scaling alone.
Edit this visual in QueryChart (FlowJam)
Open this exact LLM canvas as your own chart, rename the stages to your pipeline, and annotate the model you use.