nn-js · MNIST digit classifier
A fully connected network turns 784 pixel brightnesses into 10 probabilities. Each layer multiplies the previous layer by a weight matrix, adds a bias, and applies an activation. Below, real test image #0 (a 7) flows through the trained model. It predicts 7 with 100.00% confidence.
One forward pass
Hidden layers use ReLU (max(0, z)), so a neuron either stays silent or passes its signal on. For any given digit, only part of the network lights up. The output layer uses softmax to turn 10 scores into probabilities that sum to 1.
| x | pixels / 255 | 784 |
| h₁ = ReLU(W₁x + b₁) | first hidden layer | 256 |
| h₂ = ReLU(W₂h₁ + b₂) | second hidden layer | 128 |
| p = softmax(W₃h₂ + b₃) | pᵢ = eᶻⁱ / Σ eᶻʲ | 10 |
| prediction = argmax(p) | most likely digit | 1 |
During training, 20% of hidden neurons are randomly switched off on each step (dropout). This stops the network from relying on any single neuron.
Hidden layer 1: 66 of 256 neurons fire
Hidden layer 2: 51 of 128 neurons fire
Each square is one neuron, in index order. A different digit lights up a different pattern. The output layer reads these patterns to decide which digit it sees.
Inside hidden layer 1
Each neuron in hidden layer 1 has 784 weights, one per pixel, so its weights can be drawn as a 28×28 image. Blue pixels push the neuron to fire when ink is there. Red pixels hold it back. Together they act as stroke and edge detectors. These are the 60 neurons with the largest weights.
Results on 10,000 unseen test images
The network never saw these images during training. It gets 9,817 of 10,000 right (98.17%). Most mistakes are on digits that are genuinely ambiguous or badly written.
Diagonal cells (gray) count correct answers. Darker blue means more errors. For example, the cell at row 4, column 9 shows that 15 images of a 4 were read as a 9.
Training run
One epoch is one pass over the 55,000 training images. Loss keeps falling, but validation accuracy levels off after a few epochs. The model saved to disk is the one with the best validation accuracy (epoch 8).