Parameters
In shortParameters are the numbers, also called weights, that a model adjusts during training and then uses to answer; their count is a rough measure of size.
In plain words
A language model is a large set of numbers arranged in layers. Training adjusts these numbers until the model predicts text well. Each adjustable number is a parameter, often called a weight. When you read "7B" or "70B" in a model name, it means roughly 7 or 70 billion parameters. For scale, the 2020 paper that introduced GPT-3 describes a model with 175 billion parameters.
After training, the parameters stay fixed. Inference is the use of those fixed numbers to answer your prompt. Fine-tuning changes some of them.
Do not mix this up with the "parameters" of an API call. Settings such as temperature or the maximum answer length are also called parameters. They are choices you make per request. The parameters of a model are what it learned.
Why it matters
- Memory: Each parameter takes space. As rough arithmetic, 7 billion parameters at 16 bits need about 14 GB. This sets what you can run as a local model.
- Cost and speed: Larger models need more computing power per answer, so they are usually slower and dearer per token.
- Not the whole story on quality: Research by Kaplan and colleagues found that performance improves with model size, data and computing power. Work by Hoffmann and colleagues (the "Chinchilla" paper) found that, for a fixed computing budget, size and the amount of training data should grow together, and that several large models had been trained on too little data. A smaller model trained well can match a bigger one.
- Transparency: Providers of closed models often do not publish the parameter count, so you cannot compare by this number alone. The count is known for open-weight models.
Example
You compare two models on your laptop. One has 7 billion parameters, the other 70 billion. The small one answers fast and fits in memory. The large one needs roughly ten times the memory and runs slowly, but handles harder questions better. For summarising short notes, the small model is enough. For a hard analysis, test the larger one.
Common misconceptions
- "More parameters always means a better model." Size helps, but training data, method and tuning matter as much.
- "Parameters are stored facts." They are learned patterns. The model does not keep a lookup table of facts, which is one reason it can hallucinate.
- "Parameters in an API call and in a model are the same thing." They are not. One is a setting, the other is learned.
Questions and answers
What does 7B or 70B mean in a model name?
It means about 7 or 70 billion parameters, the learned numbers inside the model.
Are more parameters always better?
No. Training data and method matter too. Research on compute-optimal training shows that a smaller model trained on enough data can match a larger one.
How much memory does a model need?
Roughly the parameter count times the bytes per parameter. At 16 bits per weight, 7 billion parameters take about 14 GB. Lower-bit versions need less.
Read next
- Large language model (LLM)
A large language model (LLM) is an AI system trained on huge amounts of text to predict the next token, which lets it write, summarise and answer.
- Open-weight model
An open-weight model is an AI model whose trained weights are published for download, so you can run and often adapt it yourself.
- Local model
A local model is an AI model that runs on your own computer or server instead of a provider's cloud, so prompts need not leave your device.
- Fine-tuning
Fine-tuning trains an existing model further on your own examples, so it follows a style, format or task more reliably.
- Inference
Inference is the act of running a trained AI model to produce an output from your input, such as an answer to a prompt.