AlexNet Architecture Explained (Layer by Layer)

Here is the AlexNet architecture explained the way you’d actually want it as a reference: eight layers deep — five convolutional layers followed by three fully-connected layers ending in a 1000-way softmax — with roughly 60 million parameters and 650,000 neurons, per the NIPS 2012 paper by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. This is the network whose win at ILSVRC-2012 kicked off the deep-learning era, and it remains the single best diagram to learn from because it is small enough to hold in your head all at once. This page is the reference companion: a layer-by-layer walkthrough of the shapes, the kernel sizes, where the parameters hide, and the four design tricks the paper leaned on — in the paper’s own order of importance.

If you’d rather rebuild it in code than read the map, the sibling to this article — Build a CNN Image Classifier From Scratch — defines these same blocks in PyTorch line by line. And if you want the story of why 2012 mattered at all, our Learn cornerstone How AI Learned to See tells it. Keep this page open as the lookup table the build tutorial points back to.

The AlexNet architecture as a labelled stack of 8 layers — five convolutional plus three fully-connected — narrowing to a 1000-way output

🗺️ The Whole Network at a Glance

AlexNet is best read as “LeNet, scaled up, plus a few tricks that made scaling work.” The conv → pool → conv → … → fully-connected → softmax template is Yann LeCun’s 1990s LeNet; AlexNet did not invent the convolutional network. What it proved was that a deep CNN, trained on ImageNet-scale data with a couple of then-new ideas and a pair of GPUs, could crush the competition. The paper’s abstract puts it exactly: the network “has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.”

The whole thing fits in one table. A 224×224×3 image enters on the left and exits as a 1,000-class probability vector on the right. Watch two things as your eye moves down: the spatial map shrinks while the channels deepen, and the parameter count is almost nothing until the fully-connected layers, where it explodes.

Layer	Type	Kernel / stride	Output (approx.)	Params (approx.)
Conv 1	Convolution + ReLU	11×11, stride 4, 96 filters	55×55×96	~35K
—	Overlapping max-pool	3×3, stride 2	27×27×96	0
Conv 2	Convolution + ReLU	5×5, 256 filters	27×27×256	~614K
—	Overlapping max-pool	3×3, stride 2	13×13×256	0
Conv 3	Convolution + ReLU	3×3, 384 filters	13×13×384	~885K
Conv 4	Convolution + ReLU	3×3, 384 filters	13×13×384	~1.3M
Conv 5	Convolution + ReLU	3×3, 256 filters	13×13×256	~885K
—	Overlapping max-pool	3×3, stride 2	6×6×256	0
FC 6	Fully connected + ReLU + dropout	9216 → 4096	4096	~37.7M
FC 7	Fully connected + ReLU + dropout	4096 → 4096	4096	~16.8M
FC 8	Fully connected + softmax	4096 → 1000	1000	~4.1M

That is the entire architecture. Everything below is just reading this table more slowly.

Where AlexNet's parameters live — a tiny convolutional feature extractor feeding two enormous fully-connected layers

🔍 The Five Convolutional Layers (11 → 5 → 3)

The single most important pattern in the convolutional stack is that the kernels shrink with depth: 11×11, then 5×5, then three stacked 3×3 layers. The first layer uses a big 11×11 window with a stride of 4 to survey the whole 224×224 image cheaply — a wide receptive field that scans coarsely and immediately drops the spatial resolution. As the network deepens, the filters narrow to 5×5 and then 3×3 because by then the feature maps are small and dense, and small filters are enough to refine detail. Big-and-coarse early, small-and-fine late.

The channel counts run 96 → 256 → 384 → 384 → 256. So as the picture gets spatially smaller, it gets deeper — each layer trades resolution for a richer stack of learned feature detectors. Three overlapping max-pooling steps (after Conv 1, Conv 2, and Conv 5) do the shrinking, taking the map from 55×55 down to 6×6 by the end.

Here is the part that surprises people: despite doing all the actual “seeing,” the entire convolutional stack holds only a few million parameters — well under a tenth of the network’s total. Convolutions are parameter-thrifty by design, because the same small kernel slides across the whole image and its weights are shared at every position. The heavy lifting, parameter-wise, is still ahead.

🎩 The Three Fully-Connected Layers (Where the Parameters Live)

After Conv 5 and its pool, the 6×6×256 feature map is flattened into a single vector of 6 × 6 × 256 = 9,216 numbers, and that vector is fed into three fully-connected layers: 9216 → 4096, 4096 → 4096, and finally 4096 → 1000 for the class scores. This is where nearly all the weight sits. Roughly 58 of the ~60 million parameters live in these three fully-connected layers — the first one alone (9216 → 4096) is about 37.7 million weights, because every one of 9,216 inputs connects to every one of 4,096 outputs. The part doing the “seeing” is small; the part doing the “deciding” is enormous.

That imbalance is not a footnote — it’s the defining weakness of the design, and it drove much of what came next. VGG went deeper with small filters; GoogLeNet and ResNet replaced the giant fully-connected block with global average pooling, cutting the parameter count dramatically. AlexNet’s front-light, back-heavy shape is exactly what later architectures set out to fix, and it’s why its 233 MB checkpoint feels heavy for its accuracy.

One detail worth carrying forward: the 4,096-dimensional vector produced by FC 7, just before the final classification layer, is an image embedding — a compact numeric fingerprint of the picture. That is the same idea our explainer What Is an Embedding? unpacks, and a CNN’s penultimate layer is one of the cleanest examples you’ll ever meet. Swap the final 4096 → 1000 layer for a 4096 → your-classes layer and you’ve repurposed the whole network — which is precisely the trick the from-scratch build guide leans on.

🛠️ The Four Tricks From §3, in the Paper’s Own Priority Order

Section 3 of the paper lists four architectural ingredients, and the authors are explicit that they’re ordered “according to our estimation of their importance, with the most important first.” Reading them in that order tells you what the team thought actually mattered:

ReLU nonlinearity (§3.1). The max(0, x) activation, non-saturating and fast. The paper reports a four-layer CNN with ReLUs reached 25% training error on CIFAR-10 roughly six times faster than the same network with tanh. AlexNet did not invent ReLU — it popularized it for deep CNNs, and it’s still the default activation today (with descendants like GELU).
Training on multiple GPUs (§3.2). The network was split across two GPUs so it would fit in memory at all. Crucially, this was an engineering workaround, not a design principle — more on that in the sidebar below.
Local Response Normalization (§3.3). A lateral-inhibition scheme that normalizes each neuron’s response against its neighbors across adjacent kernel maps. The paper credited it with a ~1–2% error reduction. It has since been abandoned: batch normalization (2015) did the job better, and nobody adds LRN to new networks anymore. Note that batch norm is a later invention and is not part of AlexNet — don’t retrofit it onto the 2012 model.
Overlapping pooling (§3.4). The max-pooling windows are 3×3 with a stride of 2, so the windows overlap (window wider than stride). The paper says this slightly reduced error and made the model marginally harder to overfit.

Two more anti-overfitting measures live in Section 4 rather than Section 3, but they’re part of the same story: dropout (p=0.5, applied in the first two fully-connected layers, randomly zeroing half the activations during training) and heavy data augmentation (224×224 random crops and horizontal reflections from 256×256 images, plus “fancy PCA” color jitter). Dropout wasn’t an AlexNet invention either — it was a concurrent idea from the same Hinton group — but this was one of its first high-profile outings. With ~60M parameters trained on 1.2M images, the network would happily memorize without these guards.

🖥️ Sidebar: Why Two GPUs in 2012, One Now

The original AlexNet diagram is famously split into two parallel “towers,” and it’s easy to read that as a deep architectural statement. It isn’t. The model was trained on two NVIDIA GTX 580 GPUs with 3 GB of memory each, and a 60-million-parameter network simply did not fit in a single 2012 GPU’s 3 GB. So the authors spread the layers across two cards, letting the GPUs talk only at certain layers to save bandwidth. Training took “between five and six days” over roughly 90 epochs on 1.2 million images.

The two-tower split, in other words, was a memory hack, not a law of the design. Modern reimplementations — including the one in torchvision — run the whole thing single-stream on one GPU, because a single modern card has ample memory. If you’re learning the architecture today, mentally collapse the two towers into one column: that’s the network. The parallel drawing is a 2012 hardware artifact, nothing more.

🔢 Why torchvision Says 61.1M and the Paper Says 60M

If you load AlexNet in code and count its parameters, you’ll get a number that doesn’t match the paper — and that’s expected. The paper reports ~60 million parameters. But torchvision.models.alexnet(weights=AlexNet_Weights.IMAGENET1K_V1) reports exactly 61,100,840 parameters (about 233 MB, reaching 56.522% top-1 / 79.066% top-5 accuracy on ImageNet). The gap isn’t an error in either source.

The reason is that torchvision doesn’t implement the literal two-tower 2012 diagram. It follows Krizhevsky’s later single-stream reformulation, the 2014 “One weird trick for parallelizing convolutional neural networks” paper, which collapses the split network into one clean stream with slightly different channel bookkeeping. So the two counts describe two slightly different implementations of the same idea. The rule of thumb: if you quote ~60M, attribute it to the paper; if you quote 61.1M, attribute it to torchvision’s one-weird-trick variant. They’re not in conflict — they’re two editions of the same architecture.

While we’re citing numbers, one more that’s easy to garble: AlexNet’s famous winning score. The ILSVRC-2012 top-5 error was 15.3%, versus 26.2% for the runner-up — but that 15.3% was a seven-model ensemble (five AlexNets plus two pre-trained on the ImageNet Fall-2011 release). A single AlexNet scored 18.2% top-5; averaging five gave 16.4%. When you cite the score, cite the configuration with it; no single model hit 15.3%.

⚠️ A Reference, Not a Deployment Target

Say it plainly so nobody ships the wrong thing: AlexNet is superseded. ResNet (2015) made 100-plus-layer networks trainable with residual connections and pushed ImageNet top-5 error to around 3.6%; EfficientNet (2019) improved accuracy-per-FLOP dramatically; Vision Transformers (2020) became the default for large-scale image work. AlexNet’s role in 2026 is pedagogical — a historical baseline and a clean diagram you study once to understand what every later network is reacting to.

That’s exactly why this reference exists: not so you can deploy AlexNet, but so the pieces have names when you meet them again in a modern network — the shrinking kernels, the pooling, the activation, the embedding-shaped penultimate layer. Understand the map here, rebuild it once in the companion build guide, and then reach for a pretrained modern backbone every time you have a real problem to solve.

❓ Frequently Asked Questions

How many layers does AlexNet have? Eight learned layers: five convolutional layers followed by three fully-connected layers, ending in a 1000-way softmax. Pooling and normalization steps sit between some of those but aren’t counted as learned layers because they have no trainable weights. The “eight layers” figure is the standard way the architecture is described in the paper.

Why do the kernel sizes go 11 → 5 → 3? The first layer uses a large 11×11 window (with stride 4) to survey the full 224×224 image cheaply and drop the resolution fast; later layers use small 3×3 filters because the feature maps are already small and dense, so a narrow window is enough to refine detail. Big receptive field early, fine detail late — a pattern later networks like VGG pushed to the extreme with 3×3 filters everywhere.

Where are AlexNet’s parameters concentrated? Overwhelmingly in the three fully-connected layers — roughly 58 of the ~60 million parameters, with the first FC layer (9216 → 4096) alone holding about 37.7 million. The convolutional feature extractor, despite doing all the visual work, holds only a few million because convolution weights are shared across the whole image.

Why does torchvision report 61.1M parameters instead of 60M? Because torchvision implements the single-stream “one weird trick” reformulation (Krizhevsky 2014), not the literal two-tower 2012 diagram. The two implementations count slightly differently: the paper’s ~60M and torchvision’s 61,100,840 are two editions of the same architecture. Attribute the number to whichever source you’re quoting.

Was AlexNet’s two-GPU design necessary? No. The model was split across two GTX 580 (3 GB) GPUs purely because it wouldn’t fit in a single 2012 card’s memory. It was a memory workaround, not an architectural requirement — modern reimplementations run single-stream on one GPU.

Should I use AlexNet today? Only to learn from. It’s been superseded by ResNet, EfficientNet, and Vision Transformers on both accuracy and efficiency. Use it as a teaching reference and historical baseline, then deploy a pretrained modern network via transfer learning.

📚 Further Reading

Build a CNN Image Classifier From Scratch — the hands-on sibling to this reference: define these same features and classifier blocks in PyTorch, line by line, and train on CIFAR-10.
How AI Learned to See — the story of the 2012 AlexNet moment and what “a computer learning to see” really means.
What Is an Embedding? — why a CNN’s penultimate layer is an embedding, and what that unlocks for search and similarity.
The NIPS 2012 paper (ImageNet Classification with Deep Convolutional Neural Networks) and the torchvision.models.alexnet documentation — the primary sources for the layer shapes, parameter counts, and the single-stream reimplementation.

📚 Recommended Resources

Disclosure: some links below are Amazon affiliate links. If you buy through them, I earn a small commission — at no extra cost to you — which helps keep this site running.