Epoch
Mistakes in the last epoch
Weights w₁ · w₂ · b
Converged
  • Class +1
  • Class −1
  • Misclassified
  • Decision boundary
  • Weight vector w

Click adds

More about the perceptron

Perceptron visualizer

The perceptron is the simplest learning machine there is: one artificial neuron that adds up its weighted inputs and answers yes or no. With two inputs its answer changes along a straight line, and learning means moving that line until every point is on its own side. Click points of two classes into the plane or pick a logic gate, then step through Rosenblatt's rule one sample at a time, see each weight before and after the update, and watch the line turn and slide into place, or keep failing forever on XOR. Pick a dataset or click points into the plane: a click adds a point of the class chosen under Click adds, a shift-click or right-click one of the other class, and a click on a point removes it. Then train the neuron. Step shows one sample: the dashed ring marks it, and the table shows the weights before and after the update w ← w + η·(y − ŷ)·x. Epoch runs one pass over all points, and Play keeps going at the chosen speed until a pass makes no mistake or the epoch cap is reached. The line is where w₁x + w₂y + b = 0, the arrow is the weight vector, and a red ring marks a point on the wrong side.

How it works

  1. Choose a dataset, from the logic gates AND, OR, NAND and XOR to random points with a margin or two overlapping clouds, or click your own points into the plane.
  2. Press Step to show the neuron one sample and see the update of each weight, or press Epoch or Play to train on all points.
  3. Watch the decision boundary, the weight vector and the mistakes per epoch until a pass makes no mistake, and try XOR to see a single perceptron fail.

One neuron, one line

A perceptron with inputs x and y computes w₁x + w₂y + b and answers +1 when the sum is positive and −1 when it is negative. The sum is zero exactly on the line w₁x + w₂y + b = 0, so the neuron cuts the plane into two half-planes, one for each class. The weight vector (w₁, w₂) stands at right angles to that line and points into the +1 side, and the bias b shifts the line away from the origin. Treating the bias as the weight w₀ of an input that is always 1 lets one rule train all three numbers.

The learning rule

Training shows the neuron one point at a time. If it answers right, nothing changes. If it answers wrong, every weight moves by η·(y − ŷ) times its input, where y is the true class, ŷ the answer and η the learning rate. That nudges the line towards putting the point on the correct side. One pass over all points is an epoch. If the points can be separated by a straight line, the perceptron convergence theorem guarantees that this simple rule finds such a line after a finite number of mistakes; XOR shows what happens when no such line exists.

Frequently asked questions

Why can't a perceptron learn XOR?

XOR is true for (0, 1) and (1, 0) and false for (0, 0) and (1, 1). The two true points sit on one diagonal of the square and the two false points on the other, and no straight line puts one diagonal on one side and the other on the other side. A single perceptron can only draw straight lines, so on XOR it keeps making mistakes, epoch after epoch, and never converges. A second layer of neurons, or an extra input such as x·y, solves it.

Does the perceptron always converge?

It converges whenever a straight line can separate the two classes: the perceptron convergence theorem limits the number of updates, to at most (R/γ)² for the rule in its classic form, where R is the length of the longest input vector and γ the margin of the best separating line. A small margin can mean many epochs. When no separating line exists, it never converges, which is why the app stops at an epoch cap and checks whether the points are separable.

What does the learning rate do?

It sets the size of each update. Starting from zero weights, it only scales all the weights by the same factor and never changes the line or the number of mistakes, a curious property of the perceptron. With random start weights it matters: a small rate keeps the line close to where it started for longer, a large rate lets the first mistakes dominate.

What is the pocket algorithm?

A small change, published by Stephen Gallant in 1990, for data no line separates. The perceptron runs as usual, but after every update the errors of the new weights on all points are counted, and the best weights so far are kept in the pocket. The pocket line, drawn dashed, is the answer when training stops, rather than whatever line the last update happened to leave.

Is anything sent to a server?

No. The points, the training and the drawing all run in your browser, and your points and weights stay in this browser's local storage.

Ready to try it?

Open PictorX in your browser — no install, no credit card required to start.

Go to Editor