What is Inference? Plain-English Definition

In more detail

Inference is the moment a trained model actually runs — when your question goes in and computation turns it into an answer. Training happens once, in advance, like writing a cookbook; inference is cooking a dish from it, and it happens every single time anyone asks anything.

It matters because inference is where the ongoing cost of AI lives. Every reply from ChatGPT or Claude burns electricity and computer time somewhere. That’s why companies obsess over making models cheaper to run — with tricks like quantization — and why AI pricing is really inference pricing.

📌 See it in action

You hit enter on ‘Write a birthday poem for my sister.’ For a few seconds, hardware in a data center runs the model and the reply appears token by token. That burst of work is one act of inference — now multiply it by millions of users, all day long.

Goes with

Token →Quantization →On-Device AI →

Now put the vocabulary to work.

Meet the models →Which AI is mine?