Kivorent News

Wednesday, November 18, 2026

Tech and AIExplainer

What a GPU does that a CPU does not, and why AI runs on GPUs

A CPU runs a few complicated tasks very quickly, one after another. A GPU runs thousands of simple calculations at once, which is the kind of work training and running AI models needs.

Kiln, covering chips, data centers and power
, 3 min read

1,000ssimple cores on a data center GPU, vs dozens on a CPU

What to know

  • A CPU has a small number of powerful cores built to run varied, branching instructions quickly.
  • A GPU has thousands of simpler cores that apply the same operation to huge batches of numbers at the same time.
  • AI models are mostly giant multiplications of number grids, so they suit the GPU's design, as long as memory can keep it fed.

Every computer has a CPU, the central processing unit. AI data centers are built around a different chip, the GPU, or graphics processing unit, and the companies that buy them spend far more on GPUs than on CPUs. The difference comes down to what kind of work each chip is designed for.

What a CPU is built for

A CPU runs the operating system, the web server, the database and most ordinary software. That work is varied and full of decisions: check this value, jump to that part of the program, wait for a file, handle a click. Each step often depends on the one before it.

So a CPU is built to finish one stream of instructions as fast as possible. It has a modest number of cores, from a handful in a laptop to around a couple of hundred in the largest server chips, such as AMD's EPYC line. Each core is large and clever. It guesses which way the program will branch, runs instructions out of order when that saves time, and keeps big caches of nearby memory so it rarely waits.

What a GPU is built for

GPUs were first made to draw 3D graphics, which means working out the color of millions of pixels many times a second. Each pixel follows the same steps with different numbers. There is little branching and no need to wait for the neighbor.

A GPU spends its silicon on arithmetic instead of cleverness. A modern data center GPU has thousands of small cores grouped into blocks that all execute the same instruction on different data at the same moment. It also has dedicated units, which Nvidia calls tensor cores, that multiply small grids of numbers in a single step. Any one of those cores is far slower and simpler than a CPU core. Together they do vastly more arithmetic per second.

Why that fits AI

A neural network is mostly matrix multiplication: large grids of numbers, the model's weights, multiplied by grids of input data, layer after layer. Training repeats this across trillions of examples. Running a model for a user, called inference, repeats it for every word it produces.

That is the GPU's ideal job: the same simple operation on enormous batches of numbers, all at once. A CPU can do the same math, but it would take far longer and use more power per answer.

AI also tolerates lower precision. Models can often use 8-bit or even 4-bit numbers instead of the 64-bit numbers scientific software needs, and GPUs add hardware to do many more of these small calculations per second.

The catch: feeding it

Thousands of cores are useless if data cannot reach them fast enough. That is why AI GPUs sit beside stacks of high-bandwidth memory, explained here, and why tens of thousands of them are wired together with fast networks so one model can span many chips. All of this draws a great deal of electricity, which is why AI data centers need so much power and cooling.

CPUs do not go away. Each AI server still has CPUs to run the system, move data and hand work to the GPUs, which is one reason server CPU market share is still watched closely.

Beyond the GPU

Some cloud companies design their own AI chips that drop the graphics parts and keep only the matrix hardware, such as Google's TPUs. They follow the same idea as the GPU: many simple units doing the same math in parallel.

Why it matters

The split between the two chips explains where AI money goes. Because AI work is mostly parallel arithmetic, the chips that do it best take most of the spending, which is why Nvidia's data center sales have grown so fast and why supply of GPUs, memory and power now shapes how quickly AI can expand.

AI chips and data centersNvidia

Written by an AI correspondent under standards set by editor Jacob Knight. Spotted an error? Tell us and we will fix it.