Section 1 · 10 lessons
Weight Update
Each weight goes down by the input value × the error × the learning rate.
-
The Delivery Time Predictor 01
This network predicts how long a delivery takes. The input value is kilometers driven. The output value is minutes it takes. It also does something new. After each error, it updates the weight to make the error smaller. Observe the weight change over all 8 deliveries, each with its own traffic. This is our first learning network!
Open lesson -
The Beach Day Predictor
This network predicts a day at the beach. The input value is the hours of sunshine. The output values are the swimmers and the sandcastles built. One input value learns two answers at once. Observe the two weights settle as you step through all sixteen days.
Open lesson -
The Fishing Trip Predictor
This network predicts how many fish you will catch on a fishing trip. The input value is the hours spent fishing. Some hours are luckier than others. The weight goes down by the input value × the error × the learning rate. Predict the adjustment to the weight in the first two rounds. The learning rate is 0.1.
Open lesson -
The Pumpkin Patch Predictor
This network predicts how much a pumpkin grows in 2 weeks. The input values are the sunny days and the rainy days. The output value is the kilograms it gained. The network learns what each sunny day and each rainy day is worth. Predict the adjustment to each weight in the first round. The learning rate is 0.02.
Open lesson -
The Test Score Predictor 01
This network predicts a test score. The input values are the hours of study and the hours of sleep the night before. The output value is the score. Both weights start at 0.30. Watch which one grows faster. Predict the adjustment to the sleep weight in the first round. The learning rate is 0.02.
Open lesson -
The Kite Festival Predictor
This network predicts the number of kites at a festival. The input values are the wind and the rain, each scored from 0 to 10. The first output value is too high, so both weights go down. Predict the adjustment to the wind weight in the first two rounds. The learning rate is 0.03.
Open lesson -
The Field Day Predictor
This network predicts the drinks needed at a school field day. The input values are the younger and older classes taking part. Each output value has its own error, and each weight is adjusted by one of them. Predict the adjustment to both weights from the older classes in the first round. The learning rate is 0.02.
Open lesson -
The Bird Song Identifier
This network identifies a robin, an owl and a woodpecker. The input values are the pitch, the hoot length and the tapping speed. The output values are the 3 birds. The answer is whichever output value is largest. Predict the adjustments to the 3 weights from pitch in the first round. The learning rate is 0.1.
Open lesson -
The Drink Identifier 01
This network identifies juice, coffee and soda. The input values are the sugar, the caffeine and the fizz. The output values are juice, coffee and soda. The answer is whichever output value is largest. Predict the adjustments to the weights from sugar to coffee and to soda in the first round. The learning rate is 0.05.
Open lesson -
The Bike Ride Predictor
This network predicts how long a bike ride takes. The input values are the kilometers ridden and the hills climbed. The output value is the minutes it takes. Both weights start at 1, so the first ride's output value is far too low. Predict the adjustment to each weight in the first round. The learning rate is 0.02.
Open lesson
Section 2 · 5 lessons
Bias Update
Each bias goes down too, by the node's own error × the learning rate.
-
The Football Crowd Predictor
This network predicts the crowd at a football game. The input value is the wins in a row. The output value is the crowd. Some fans come whatever happens. The network now updates its bias as well as its weight. Observe the bias climb to about 200, the crowd that comes anyway.
Open lesson -
The Cookie Jar Predictor
This network predicts how many cookies are left in the jar. The input value is the days since baking. The output value is the cookies left. The weight is already about right, but the bias, the cookies in a fresh jar, starts too low. Observe how the bias and the weight both change as you step through all 16 jars.
Open lesson -
The Bus Delay Predictor
This network predicts how late the school bus is. The input value is the rain in centimeters. The output value is the minutes late. The bias, how late it is on a dry morning, is adjusted by the error × the learning rate. Predict the adjustment to the bias in the first two rounds. The learning rate is 0.1.
Open lesson -
The Rainy Day Toy Shop Predictor
This network predicts what a toy shop sells. The input value is the rain in centimeters. The output values are the board games and puzzles sold. Each output node has its own bias, the sales on a dry day, and each is adjusted by its own error. Predict both bias adjustments in the first two rounds. The learning rate is 0.1.
Open lesson -
The Garden Snail Predictor
This network predicts the number of snails in a garden. The input value is the rainy days that week. The output value is the number of snails. The weight is about right, but the bias, the number of snails out in a dry week, starts at 0. Edit the bias to bring every error within 1 of zero. This one is tricky.
Open lesson
Section 3 · 5 lessons
Learning Through a Middle Layer
A weight into a middle node learns the same way, using the error that back propagation carried back to that node.
-
The Blackberry Picking Predictor
This network predicts a blackberry harvest. The input value is the hours spent picking. The output value is the kilograms picked. Every stage runs here, through a middle node. Observe each stage as you step through all 16 afternoons, and watch the errors shrink.
Open lesson -
The Outdoor Movie Predictor
This network predicts the families at an outdoor movie night. The input values are the warmth and the wind, each scored from 0 to 5. Every stage is switched on. Observe all 8 nights a layer at a time, and watch every weight and bias change.
Open lesson -
The Leaf Raking Predictor
This network predicts the bags of leaves raked in a yard. The input value is the hours spent raking. Every stage runs. In the first round, predict both errors, then the adjustments to the first weight and the output bias. The learning rate is 0.1.
Open lesson -
The Playground Predictor
This network predicts the children at a playground. The input values are the sunshine and the warmth, each scored from 0 to 5. Every stage runs. In round 1, predict the output error, the bottom middle node's error, and the adjustments to its bias and its weight from warmth. The learning rate is 0.001. This one is tricky.
Open lesson -
The Recycling Drive Predictor
This network predicts the bags of cans a school recycling drive collects. The input value is the number of classes taking part. The output error comes back to both middle nodes. In round 1, predict the output error, both middle errors, and the adjustment to the weight to the top middle node. The learning rate is 0.02.
Open lesson
Section 4 · 5 lessons
Learning Rate
The learning rate sets how big each adjustment is: a big one takes big steps and can overshoot, while a small one takes tiny, careful steps.
-
The Reading Predictor
This network predicts how many pages a student reads. The input value is the hours spent reading. The output value is the pages read. Each weight update is multiplied by the learning rate, which sets how big a step it takes. Observe how the weight changes as you step through all 8 evenings, then try other learning rates.
Open lesson -
The Delivery Time Predictor 02
This network predicts how long a delivery takes. The input value is kilometers driven. The output value is minutes it takes. Every weight update jumps too far. Find a learning rate that settles near the right time. Figure out which learning rate works, then run it again. It takes some experimenting to get this right.
Open lesson -
The Library Visits Predictor
This network predicts the number of library visits. The input values are the school days and rainy days in a week. The output value is the number of visits. It learns far too slowly. Find a learning rate that gets every output value right by the end. Figure out which learning rate works, then run it again.
Open lesson -
The Test Score Predictor 02
This network predicts a test score. The input values are the hours of study and of sleep. The output value is the score. Its weights blow up as it learns. Find a learning rate that keeps them under control. Figure out which learning rate works, then run it again.
Open lesson -
The Video Game Score Predictor
This network predicts a video game score. The input value is the number of coins collected. The output value is the score. The coins come in big numbers, so every update leaps too far. Try much smaller rates. Figure out which learning rate works, then run it again.
Open lesson
Section 5 · 5 lessons
Training vs. Testing Data
The network learns from its training data, and the testing data then checks it on new examples it has never seen.
-
The Sunflower Growth Predictor
This network predicts how tall a sunflower grows. The input value is the weeks since it was planted. The output value is the height in cm. After it processes all the training data, switch to testing data: 4 weeks it never trained on. Observe whether its predictions for those weeks are right.
Open lesson -
The Fruit Sorter
This network tells apples from oranges. The input values are how bumpy and how red a fruit is. The output values are apple and orange. After it processes all the training data, switch to testing data. Observe whether it sorts the 4 new fruits correctly.
Open lesson -
The Drink Identifier 02
This network identifies juice, coffee and soda. The input values are the sugar, the caffeine and the fizz. The output values are the 3 drinks. After the training data, switch to testing data: 8 drinks it never trained on. Observe how it handles them, and look for one it has no good answer for.
Open lesson -
The Harbor Predictor
This network learns how busy a harbor will be, through a middle layer. The input values are the sunshine and the wind, from 0 to 5. The output values are the swimmers and sailboats. Decide which of the two answers it learns better. Train it on all the days, then switch to testing data and compare.
Open lesson -
The Snack Stand Predictor
This network predicts a park snack stand's sales of ice creams and hot cocoas. The input values are the warmth and the crowd, each scored from 0 to 5. Everything comes together: a middle layer, biases, the activation function and learning. Train it on all 24 days, then switch to testing data: 6 days it never trained on.
Open lesson