Manual Convolution Example

This closing note of the CNN Fundamentals category ties every piece together into one complete, fully worked example — computing an entire feature map by hand, with padding and stride both involved, verified line-for-line against PyTorch's own convolution implementation.

The Setup

A 5×5 single-channel input:

A 3×3 kernel: \(K = \begin{bmatrix}1&0&-1\\1&0&-1\\1&0&-1\end{bmatrix}\), with padding \(P=1\) and stride \(S=2\).

Step 1 — Apply Padding

Adding 1 pixel of zero-padding on every side produces a 7×7 padded input (shown with the padding as 0s around the original 5×5 grid).

Step 2 — Compute the Output Size

A 3×3 output feature map, using the formula from Padding.

Step 3 — Compute the Top-Left Output Value

With padding, the kernel's first position overlaps the padded input's top-left 3×3 corner:

Step 4 — Compute the Remaining Output Values (Stride 2)

Moving the kernel 2 positions right (respecting the stride), then repeating downward, gives the complete 3×3 output feature map. Rather than hand-computing all 9 values here, the code below verifies the entire result — including this first value of \(-3\) — directly against PyTorch's implementation.

Code — Verifying the Complete Example

import torch
import torch.nn.functional as F

I = torch.tensor([[[[1.,2.,3.,0.,1.],
                     [0.,1.,2.,3.,1.],
                     [1.,0.,1.,2.,0.],
                     [2.,1.,0.,1.,1.],
                     [0.,2.,1.,0.,2.]]]])   # shape (1,1,5,5)

K = torch.tensor([[[[1.,0.,-1.],
                     [1.,0.,-1.],
                     [1.,0.,-1.]]]])   # shape (1,1,3,3)

output = F.conv2d(I, K, padding=1, stride=2)
print(output.shape)   # torch.Size([1, 1, 3, 3]) -- matches the computed output size
print(output)
# The top-left value should match -3.0, computed by hand above

Putting It All Together — Every Concept from This Category

Common Mistakes

• Forgetting to account for padding when computing which input values a given output position actually overlaps — always work with the padded input grid when tracing through a manual calculation like this one, not the original unpadded input.
• Losing track of how stride skips positions when manually verifying multiple output values by hand — a systematic position-by-position trace (or, more practically, verifying against code as done here) avoids this class of error.

Interview Relevance

Q: "Walk through computing one value of a convolution's output feature map by hand, including padding and stride." This exact kind of worked example — identifying the padded input, locating the correct overlapping patch for a given stride and output position, and computing the dot product between that patch and the kernel — is precisely the skill that demonstrates genuine understanding of convolution's mechanics, beyond just knowing the formula.

Key Takeaways — CNN Fundamentals

• Convolution's local connectivity and parameter sharing make CNNs vastly more parameter-efficient than fully-connected networks for spatial data like images.
• Kernels/filters detect learned patterns; stacking many filters produces stacked feature maps; stride and padding together control output size precisely.
• Receptive field grows with network depth, letting deep stacks of small kernels "see" large regions of the original input cheaply.
• Pooling (max, average, or global average) downsamples feature maps with zero added parameters; flattening or GAP bridges the spatial convolutional part of a network to its final fully-connected classification head.

Next: CNN Architectures puts all of these building blocks to work, tracing the actual landmark architectures — from LeNet through ConvNeXt — that shaped how modern CNNs are designed.

Practice Question

Using the same 5×5 input and kernel from this note, but with padding \(P=0\) and stride \(S=1\) instead, compute the resulting output size using the formula from Stride.

Related DL Notes

• Convolution Operation
• Padding
• Stride