The perceptron, weighted inputs, activation functions, supervised learning, and training — the simplest possible neural network.
We began with inanimate objects. We gave them desires and autonomy. We let them evolve. Now we ask: can they learn?
Some problems are trivial for computers (square root of 964,324) but impossible to explain as rules. Show a toddler a kitten and a puppy — they'll tell you which is which instantly. Teaching a computer to do the same? That took decades of research.
A neural network is a complex adaptive system: it can change its internal structure based on information flowing through it. The mechanism? Adjusting weights on the connections between neurons.
Invented in 1957 by Frank Rosenblatt, the perceptron is the simplest neural network: a single neuron. It follows a "feed-forward" model: inputs come in, get processed, and an output comes out.
The perceptron algorithm has four steps:
Each input is multiplied by a weight — a number (usually between -1 and 1) that controls how much influence that input has. Weights start random and are adjusted during training.
pseudocode inputs = [12, 4] weights = [0.5, -1] sum = 12 * 0.5 + 4 * (-1) = 6 + (-4) = 2
The sum of weighted inputs is passed through an activation function that decides whether the neuron "fires." For a simple perceptron, the activation is just the sign of the sum:
pseudocode function activate(sum): if sum > 0: return +1 else: return -1
This makes the perceptron a binary classifier: it divides the input space into two regions with a straight line. Points on one side get +1, points on the other get -1.
A perceptron with random weights is no better than a coin flip. It must learn. The method: supervised learning.
pseudocode function train(inputs, desired): guess = feedforward(inputs) error = desired - guess for i in weights.length: weights[i] += learningRate * error * inputs[i]
The learning constant (often called learning rate) controls how fast the weights change. It works exactly like maxforce in steering.
| Learning Rate | Effect |
|---|---|
| High (0.1+) | Fast learning, but may overshoot the optimal weights and oscillate |
| Medium (0.01) | Good balance of speed and accuracy |
| Low (0.001) | Slow but precise — tiny, careful adjustments |
A single perceptron can only solve linearly separable problems — problems where a straight line divides the data. AND and OR are linearly separable. XOR is not.
| A | B | AND | OR | XOR |
|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 |
| 0 | 1 | 0 | 1 | 1 |
| 1 | 0 | 0 | 1 | 1 |
| 1 | 1 | 1 | 1 | 0 |
What if we gave our Vehicle a perceptron as a brain? Instead of hardcoding how to weight steering forces, let the brain learn.
pseudocode class Vehicle: brain = new Perceptron(numTargets) function steer(targets): forces = [] for target in targets: forces.add(seek(target)) result = brain.feedforward(forces) # Brain weights the forces applyForce(result) # Reinforcement learning: how far from center? error = center - location brain.train(forces, error)
This is a creative extension — not a rigorous neural network, but a demonstration of how the perceptron concept can be embedded in our autonomous agents to create learning creatures.
Below is a perceptron learning to classify points as above or below a line. Watch it improve as it trains on more data points.
The diagonal line is the true boundary. The perceptron trains point by point. Correct guesses are teal; wrong guesses are orange. Watch accuracy climb over time.
| Topic | Key Takeaway |
|---|---|
| Neural networks | Computational systems inspired by the brain; they learn by adjusting weights |
| Perceptron | Simplest NN: weighted sum of inputs → activation → output |
| Bias | An always-on input that shifts the decision boundary |
| Training | error = desired - guess; weight += error × input × learning_rate |
| Learning rate | Controls adjustment speed; like maxforce in steering |
| Linear separability | One perceptron = one straight line; XOR needs multiple neurons |
| Backpropagation | Feeds error backward through multi-layer networks to adjust all weights |
| Steering brain | Perceptron as a vehicle's brain; reinforcement learning from observation |
What we covered:
• The perceptron (1957)
• Weights, bias, activation
• Supervised learning
• Error = desired - guess
• Learning constant
• Linear separability and XOR
• Perceptron steering brains
The journey:
From vectors to forces to particles to physics libraries to autonomous agents to cellular automata to fractals to evolution to neural networks. We started with a dot moving across a screen and ended with objects that learn. That's the nature of code.