Two ways to build a processor
A traditional central processor, the CPU, is built to do a wide variety of tasks quickly, one after another, with a handful of powerful cores. It is good at following complicated logic with many decisions: running an operating system, handling a spreadsheet, juggling many different programs.
A graphics processor, or GPU, takes a different approach. It contains a very large number of simpler processing units that work at the same time. Each is individually less flexible than a CPU core, but together they can perform huge numbers of similar calculations in parallel. A rough analogy is the difference between a few expert craftspeople and a very large crew of workers who can each do one simple task, all at once.
Why graphics needed this
The GPU was invented to draw images. Rendering a 3D scene means computing the color of millions of pixels on a screen, many times per second. Calculating one pixel is mostly independent of calculating another, and the same kind of arithmetic is repeated across them. That is a natural fit for parallel hardware. Over time, GPUs became more programmable, so that developers could write their own small programs to run across many data elements at once, not just fixed drawing steps.
What neural networks actually compute
Researchers began to notice that a lot of scientific computing had the same shape as graphics: repeat the same operation on a large array of numbers. Neural networks are an especially clear case.
A neural network can be seen as layers of numbers, called weights, that transform input data step by step. At the heart of each layer is a multiplication of large grids of numbers, known as matrix multiplication, followed by simple adjustments. Matrix multiplication consists of many multiplications and additions that can mostly be done independently, which suits parallel hardware very well.
Training is the process of adjusting the weights so the model performs better on examples. It involves running data through the network many times, measuring how wrong the output is, and nudging the weights accordingly. In the language of machine learning, this requires a vast number of repeated calculations, on batches of examples at a time. Doing it on a general-purpose CPU can take far longer than on parallel hardware.
A simple, hypothetical comparison
The following numbers are purely illustrative. Imagine a task needing 1,000,000 independent multiplications. A processor that does 8 at a time needs 125,000 rounds, because 1,000,000 divided by 8 is 125,000. Hardware that does 1,000 at a time needs 1,000 rounds, because 1,000,000 divided by 1,000 is 1,000. In this made-up case the parallel hardware needs 125 times fewer rounds, though real speed also depends on how fast each round runs and how quickly data can be supplied. Real chips differ in many ways, and the actual advantage depends heavily on the task.
The turning point
In the late 2000s and early 2010s, researchers began to use GPUs for training neural networks and found it could be far faster than using CPUs alone. Better hardware, larger datasets and improved methods came together, and results in areas such as image recognition improved markedly. It is difficult to credit any one factor alone, but cheap, widely available parallel hardware was clearly an important enabler. Graphics chips were already being mass-produced for gaming, which kept prices lower than for niche scientific hardware.
Software helped too. Programming frameworks made it easier to write code for GPUs without managing every detail, and machine learning libraries grew up around that support. Once developers had tools that worked well on GPUs, that created a feedback loop: more users, more tools, more investment.
It is not only about arithmetic
Raw calculation speed is only part of the story. Several other features matter:
- Memory bandwidth. Moving numbers between memory and the processing units can be the real bottleneck. AI-oriented GPUs use specialized, very fast memory placed close to the chip.
- Lower-precision numbers. Neural networks often tolerate less precise arithmetic, such as using fewer bits per number. Lower precision lets a chip do more operations per second and use less memory, and modern GPUs include circuits specially designed for it.
- Connecting many chips. The largest models are trained on many processors working together, so fast links between chips and between machines matter greatly.
- Energy. Large clusters draw considerable power, one reason the facilities that house them, data centers, are at the center of energy debates.
Beyond GPUs
GPUs are not the only option. Companies and researchers have built custom accelerators designed specifically for neural network math, sometimes called tensor or neural processing units. Phones and laptops increasingly include a small AI accelerator for tasks such as photo processing and speech recognition. These specialized chips can be more efficient for particular workloads, but they may be less flexible, and a general-purpose GPU has the advantage of an established software ecosystem.
Looking further out, researchers explore different approaches such as neuromorphic computing, which takes inspiration from the brain, and photonic computing, which uses light. These are not yet mainstream replacements.
Limits and open questions
GPUs are not magic. Programs with many branching decisions or little parallelism run poorly on them. Supply can be constrained, because advanced chips depend on a complicated manufacturing chain with few suppliers. And as models grow, so do costs, which raises questions about who can afford to train the largest systems. Whether future progress will rely more on bigger hardware or on more efficient methods is debated.
So the short version is this: GPUs became central to AI not because they were designed for it, but because the math of neural networks happened to match what they already did well, and a strong ecosystem grew up around that match.