1×1 Convolution & Bottlenecks in GoogleNet

Interactive 1×1 Convolution

Click any input pixel to see how filters transform it into output channels.

Input Channels

Shape: 3×3×3

Click a pixel to select

Selected Location

Click an input pixel...

Filter 1 Computation

Weights: [0.3, 0.5, 0.2]

Result: -

Filter 2 Computation

Weights: [0.2, 0.4, 0.4]

Result: -

Output Channels

Shape: 3×3×2

Shows applied filters

👇 Click any input pixel below to see how the 1×1 convolution transforms it.

Why Channel Reduction Matters: 3×3 Convolution Comparison

The same 3×3 filter is much cheaper to apply on a thin (reduced) volume than a thick one.

Heavy

192 Channels

3×3 filter must process 192 input channels. Very expensive.

Lightweight

16 Channels (After 1×1)

Same 3×3 filter now sees only 16 channels. 12× faster.

Output

64 Output Channels

Final feature maps after the cheaper convolution.

Heavy path ops: 3 × 3 × 192 = 1,728 per filter → Light path ops: 3 × 3 × 16 = 144 per filter

What Happens Next: 5×5 Convolution

In a real network, feature maps are much larger (e.g., 28×28 spatial size)

(64 filters = 64 output channels. This number is a design choice.)

❌ Without Bottleneck (Direct Path)

Input: 28×28×192 channels

↓

5×5 Conv

(64 filters → 64 output channels)

↓

Output: 28×28×64

Learned Parameters:

Each 5×5 filter: 5 × 5 × 192 = 4,800 weights

× 64 filters: 4,800 × 64 = 307,200

(Parameters = weights the network learns. Same weights reused across 28×28 spatial locations.)

✓ With Bottleneck (Compressed Path)

Input: 28×28×192

↓ 1×1 reduce

(compress to 16 channels)

↓ 5×5 Conv

(64 filters on 16 channels)

Output: 28×28×64

Learned Parameters:

1×1 reduce: 1 × 1 × 192 × 16 = 3,072

Each 5×5 filter: 5 × 5 × 16 = 400 weights

× 64 filters: 400 × 64 = 25,600

Total: 3,072 + 25,600 = 28,672

(Parameters = weights the network learns. The 5×5 filters see only 16 channels, not 192!)

The trick: The expensive 5×5 convolution only sees 16 channels (instead of 192) because the 1×1 reduced it. That's 92% less computation for the expensive part.

1×1 Convolutions & Bottlenecks