02How Large Language Models Work

Step 1 — Text becomes tokens

LLMs don't read letters or whole words. They read tokens: chunks of text, usually a word or part of a word. In English, one token averages about ¾ of a word, so 1,000 tokens ≈ 750 words. Filipino, emoji and code often use more tokens per word.

Illustrative tokenization — each colored chip is one token. Exact splits vary by model.

Tokens matter because everything is measured in them: API pricing (per million input and output tokens), context limits and speed.

Step 2 — Tokens go into the context window

The context window is everything the model can “see” at once: the system prompt, the conversation so far, any documents you pasted, tool results — and its own answer as it writes. Modern models have windows from about 8,000 tokens (small local models, by default) to hundreds of thousands or more (frontier cloud models).

Step 3 — The model predicts the next token, over and over

An LLM is fundamentally a next-token predictor. Given all tokens so far, it calculates a probability for every possible next token, picks one, appends it, and repeats. That simple loop — run by a neural network with billions of parameters — produces essays, code and reasoning.

INPUTYour prompt“Write a tagline for…”

TOKENIZETokens[Write][ a][ tag]…

MODELNeural networkbillions of parameters

PREDICTNext-token odds“Fresh” 31% · “Your” 18%…

OUTPUTPick → append → repeatuntil done

Training vs. inference

Most modern models then go through extra training steps — instruction tuning and reinforcement learning from human (and AI) feedback — which teach them to follow instructions, be helpful and refuse harmful requests. That's why a chat model answers questions rather than just continuing your text.

The settings you control

Why models hallucinate — and how to reduce it

Because the model's job is to produce the most plausible continuation, it will produce something plausible even when it doesn't actually know. It has no built-in “I'm unsure” signal unless trained and prompted to express one. You reduce hallucination by:

1. Grounding — paste the source document or connect a knowledge base (RAG, Lesson 08) and instruct: “Answer only from the provided text.”
2. Permission to not know — “If the answer isn't in the document, say you don't know.”
3. Asking for quotes or citations — the model must point to where it found each claim.
4. Tools — give it search, a calculator or database lookups instead of relying on memory.
5. Verification — a human or a second AI pass checks critical outputs.

Reasoning (“thinking”) models

Many current models can spend extra tokens “thinking” before answering — working through a problem step by step internally. This improves results on math, logic, planning and coding, at the cost of time and tokens. Use reasoning modes for complex analysis and agent planning; skip them for simple drafting where speed matters.

Open-weight vs. closed models

You don't have to choose one. A common setup: open-weight models locally for private, high-volume or simple tasks (Track 4), frontier cloud models for the hardest work.