
An LLM, or large language model, is an AI system trained on enormous amounts of text to predict what comes next in a sentence, one small piece at a time.
That sounds too simple to explain a chatbot that writes essays and fixes code. Practiced on trillions of pieces of text, that guessing game piles up into grammar, facts, reasoning habits, and style. IBM describes LLMs as deep learning models trained on immense amounts of data that can understand and generate natural language.
Tokens: the pieces an LLM reads
An LLM never sees a sentence the way you do. Before anything else happens, a program called a tokenizer chops your text into pieces called tokens. A token is often a whole common word, like “dog”. Rarer words split into familiar chunks: “tokenization” might become “token” plus “ization”.
OpenAI’s rule of thumb puts one token at about four characters, or roughly three quarters of an English word, so a thousand words run to about 1,300 tokens. Providers charge by the token, and a model’s memory is measured in tokens, not words. OpenAI’s text generation guide explains how models work with tokens when they produce text. The GPT-4o guide shows what one widely used token-based model can do.
Here is the odd part: a model reads chunks, not letters, so it can write a polished essay and still miscount the letters in a word. Ask an older chatbot how many times the letter r appears in “strawberry” and it often answered two instead of three. It never saw the letters. Newer reasoning models get around this by spelling words out first, letter by letter.

The transformer: the engine under every LLM
The engine that turns tokens into answers is called the transformer. Eight researchers at Google introduced it in a 2017 paper titled “Attention Is All You Need”. Nearly every LLM since, from GPT to Claude to Gemini, builds on it. The original paper proposed a network based solely on attention mechanisms, with no recurrence or convolutions.
The transformer’s core move is attention. Each token looks back at the tokens before it and weighs which ones matter for the next guess. In “The animal did not cross the street because it was too tired”, attention helps the model connect “it” to “animal” rather than “street”. Stack dozens of these layers, and the model tracks meaning across long passages.
GPT spells the design out: generative pre-trained transformer. It writes new text, and it learned from vast amounts of text before it ever met you. The Claude Sonnet guide looks at one transformer model family in detail.

How training turns text into a model
Training happens in stages. First comes pretraining. The model reads a huge pile of books, articles, websites, and code, and practices next-token prediction over and over. Each wrong guess nudges its billions of parameters closer to the patterns in the text. GPT-3 had 175 billion parameters. Meta’s largest Llama 3 model has 405 billion and trained on 15.6 trillion tokens.
Raw pretraining only teaches text completion, not polite instruction-following. So builders add fine-tuning on examples of instructions and good answers, then human raters rank responses so the model learns which style people prefer. That last stage is why a chatbot refuses some requests and apologizes so much.
Training takes thousands of specialized chips running for weeks or months, which is why only a handful of companies build the biggest models. The Llama AI guide covers Meta’s openly available models, and the Gemini 3 guide covers Google’s latest family.

LLM examples you probably already use
ChatGPT runs on OpenAI’s GPT models. Claude is Anthropic’s family. Gemini is Google’s. Llama is Meta’s, and its weights are openly available for developers to download and run. Each family has larger models for hard work and smaller ones for everyday tasks.
People use them to draft emails, explain hard topics, write code, summarize documents, and translate between languages. The chatbot is only the visible part.
Agents go a step past chat. An agent model can take actions, such as searching the web or running code, and then report back. The ChatGPT Agent guide explains how one agent product works, and the best AI tools compared puts the main chatbots side by side.

Where LLMs go wrong
An LLM can state something false in perfect, confident English. Researchers call this hallucination. A model that predicts plausible text will sometimes predict plausible text that is wrong, especially on topics thinly covered in its training data. It has no built-in fact checker.
That is why numbers, quotes, and citations from a chatbot need checking. A made-up reference looks exactly like a real one. Treat chatbot output as a first draft, not a source. The free AI tools for students roundup lists study helpers worth trying, with the same check-your-work warning attached.
Models also forget. A chatbot has no memory between separate chats unless the product adds one, and what it “knows” ends with its training data.

Context windows and reasoning models
A context window is how much text a model can consider at once: your message, the chat history, any documents you pasted, and its own reply, all counted in tokens. Early models held a few thousand tokens. Current ones hold hundreds of thousands, and some reach a million. A bigger window sounds strictly better. It is not.
A 2023 study from Stanford and UC Berkeley, nicknamed “Lost in the Middle”, found that models recall facts best when they sit at the start or end of a long input, and worst when buried in the middle. The study showed that performance drops when the relevant information sits in the middle of a long context. A bigger desk does not mean equal care for every page. Put the important part first or last.
Reasoning models attack a different weakness: instead of answering at once, they work through a problem step by step before replying. That costs more time and tokens, but helps with math and logic puzzles where the first plausible answer is often wrong.

Frequently asked questions about LLMs
What is LLM in simple words?
LLM stands for large language model: an AI system that predicts the next piece of text.
Is ChatGPT an LLM?
ChatGPT is a product built on OpenAI’s GPT models. The model does the language work; the app adds the chat window and tools.
How is an LLM different from AI in general?
AI covers every system that mimics intelligent behavior. An LLM is one kind: a model that works with language by predicting tokens.
Do LLMs understand what they write?
They track language patterns well enough to produce sensible text, but they do not verify facts or hold beliefs the way people do. Check important claims yourself.
Why do LLMs hallucinate?
They generate the most plausible continuation of the text in front of them. When the right answer is missing from training data or context, a fluent wrong answer is still the most plausible continuation.
What is a context window?
The total text, measured in tokens, that a model can take in during one request. Anything past the limit is dropped or summarized.