Perceptron

The Perceptron, introduced by Frank Rosenblatt in 1958, added the one thing the McCulloch-Pitts neuron lacked: a way to learn its weights and threshold automatically from labeled examples. It's the first trainable artificial neuron, and the direct ancestor of every neural network layer in this hub.

The Model

Structurally, this is exactly the artificial neuron from earlier in this category, using a specific activation function called the step function (formally covered in the Activation Functions category). What makes it a Perceptron rather than just a McCulloch-Pitts neuron is that \(\mathbf{w}\) and \(b\) are no longer fixed by hand — they're learned via the Perceptron Learning Algorithm, covered in the next note.

Geometric Interpretation — A Linear Decision Boundary

The equation \(\mathbf{w}^\top\mathbf{x}+b = 0\) defines a straight line (in 2-D) or a hyperplane (in higher dimensions) that splits the input space into two regions — one predicted class 1, the other class 0. A Perceptron can only correctly classify data where a single straight line can separate the two classes — data that is linearly separable.

Numerical Example

With learned weights \(\mathbf{w}=[1,1]\), \(b=-1.5\): for input \([1,1]\), \(z = 1+1-1.5=0.5 \ge 0 \Rightarrow y=1\). For input \([0,0]\), \(z=0+0-1.5=-1.5<0 \Rightarrow y=0\). This particular weight/bias combination happens to implement logical AND — the same function the earlier McCulloch-Pitts example needed hand-tuning for, but here in principle learnable from labeled examples.

Code

import numpy as np

def perceptron(x, w, b):
    z = np.dot(w, x) + b
    return 1 if z >= 0 else 0

w = np.array([1, 1])
b = -1.5
for x in [[0,0],[0,1],[1,0],[1,1]]:
    print(x, "->", perceptron(np.array(x), w, b))
# implements logical AND

Common Mistakes

• Assuming a Perceptron can learn any pattern given enough training — it fundamentally cannot represent non-linearly-separable functions (like XOR), regardless of how it's trained; see Limitations of Perceptron for the full explanation.
• Confusing the historical single-layer Perceptron with the modern "Perceptron" naming sometimes loosely applied to any single neuron with any activation function — the original, strict definition uses the step function specifically.

Interview Relevance

Q: "What kind of problems can a single Perceptron solve?" Only binary classification problems where the two classes are linearly separable — where a single straight line (or hyperplane, in higher dimensions) can perfectly divide them. It cannot solve problems requiring a non-linear decision boundary, like XOR.

Practice Question

Using \(\mathbf{w}=[1,1]\) and \(b=-0.5\), verify by hand which of the 4 binary input pairs this Perceptron classifies as 1. What logical function does it implement?

Related DL Notes

• McCulloch-Pitts Neuron
• Perceptron Learning Algorithm
• Limitations of Perceptron