> For teachers > High School

Lesson plan · 3 days

The Algorithm Behind Modern AI

Three days, one for each step of training: the forward pass through hidden layers, back propagation of the error, and gradient descent. The third day ends on a network that learns two answers and is judged on data it has never seen.

Grade levels
Grades 9–12
Duration
Three class periods of 45 to 50 minutes
Class activities
3 activities, one a day, using 9 iiNNoo lessons

Learning objectives

Students will be able to:

  1. 01

    Hidden layers

    Trace a forward pass through hidden layers, with weighted sums, biases and an activation function, and predict each middle node's value before the network works it out.

  2. 02

    Back propagation

    Follow the error back across each weight, see it add up at a node that feeds several outputs, and see it stop at a node the activation function held at zero.

  3. 03

    Gradient descent

    Apply the update rule, input value × error × learning rate, and explain why a learning rate that is too large makes training overshoot or blow up.

  4. 04

    Training vs. testing

    Use testing data the network has never seen to judge whether it has truly learned, and compare how well it learned each of its outputs.

Prerequisite knowledge

Before starting, students should be comfortable with:

  • Evaluating an expression such as 2 × 1 + 3 × 2 with the order of operations.
  • Multiplying with negative numbers and decimals, such as 2 × -3 × 0.02.
  • Knowing that a function turns an input into an output. No calculus is needed.
  • Clicking, and typing a number, on a computer or tablet. Students can share one device between two.

Through the hidden layer · 45–50 minutes

Class Activity: Day 1

Introduction for students

Ask the class which AI tools they have used this week, and how they think those tools got so good. Every one was trained with three steps, repeated billions of times: a forward pass, back propagation and a weight update. The class will take one step a day.

Today is the forward pass. Each node works out a weighted sum, each input times its weight, added together, then adds its bias and applies an activation function. The nodes in the middle are a hidden layer: their values are never in the data, only worked out on the way.

Key Vocabulary

Hidden layer
A layer of nodes between the inputs and the outputs. Its values are worked out, never given.
Weighted sum
Each value coming into a node times its connection's weight, added together.
Bias
A number added to a node's weighted sum.
Activation function
A function applied to a node's value. Here, ReLU: any negative value becomes zero.
Forward pass
Working out every node's value, layer by layer, from the inputs to the outputs.

Guiding the activity

Predict the middle nodes

Pairs open The Talent Show Predictor, which asks them to predict each middle node and the output in the first round. With 2 acts and 3 posters, the top middle node is 2 × 1 + 3 × 2 = 8 and the bottom one 2 × 2 + 3 × 1 = 7. The output is 8 × 5 + 7 × 10 = 110.

A bias and an activation

The Birthday Book Predictor has one middle node with a bias of -10, the $10 owed to a friend, and an activation function. Students predict the first two rounds. $5 gives 5 - 10 = -5, held at 0, and so 0 books. $30 gives 20 dollars left, and 20 × 0.2 = 4 books.

Read a bigger network

Pairs then explore The Traffic Light Decider, an example with three inputs, four middle nodes and three outputs, and no task to finish. Each road has its own path through a middle node. Ask them to click on connections to find that path, and to explain what the 0.8 weights mean: 0.8 of green time for each car waiting.

Post-activity discussion

  1. Why does the network need a middle layer at all?

    Listen for Middle nodes can combine the inputs in different ways, and the activation function lets each one switch on only when it is useful. Without them, the network could only draw a straight line.

  2. What did the activation function do in the Birthday Book Predictor?

    Listen for It stopped the network predicting negative books when there was not enough money to pay back the friend.

  3. If a network gives the wrong answer, which numbers would you change?

    Listen for The weights and biases, the only numbers that aren't inputs or worked out from them. That is tomorrow's question.

Sending the error back · 45–50 minutes

Class Activity: Day 2

Introduction for students

Recall the forward pass in one sentence from a student. Today the network gets an answer wrong, and the class follows how it works out who is to blame.

The error is the output value minus the target value. Back propagation sends it back across each weight: a middle node receives the output's error times the weight connecting them. A node that feeds several outputs adds up what comes back from each.

Key Vocabulary

Error
The output value minus the target value.
Back propagation
Sending the error back across each weight, so every node knows its share of the mistake.
Middle error
The error a middle node receives: each output's error times the weight between them, added up.

Guiding the activity

Where the error stops

Pairs open The Street Musician Predictor, which asks them to predict the error that reaches the middle node. In round 1 the network predicts $20 against $23: an error of -3, which reaches the middle node as -3 × 0.5 = -1.5. The input values receive nothing, because nothing can change them.

Errors that add up

The City Park Predictor has two outputs, wrappers and bottles, so each middle node collects two errors. Students predict both middle nodes in the first two rounds: -399 and -199, then 399 and 199. The weights start hidden, and each prediction shows the ones it needs.

An error that can't get through

The Yawn Predictor's middle nodes measure the hours of sleep over and the hours short, so at most one is above zero a round. The activation function held the other at zero, so it played no part, and it receives no error. Students predict 4 and 0 in round 1, then 0 and 3 in round 2.

Show the working

Ask pairs to write each middle error as a sum of products on paper before typing it, such as -3 × 0.5. It makes the second and third lessons much easier to check.

Post-activity discussion

  1. Why does the error travel back across the weights?

    Listen for A weight decides how much a middle node contributed to the output, so it decides how much of the error that node is responsible for.

  2. Why did one yawn node receive no error?

    Listen for The activation function held it at zero, so changing its weights would not have changed the output. In calculus terms, its slope there is zero.

  3. What will the network do with all these errors?

    Listen for Use them to change its weights. That is tomorrow.

Gradient descent · 45–50 minutes

Class Activity: Day 3

Introduction for students

Every connection now knows its share of the error. Today the network uses it to change its weights, the step that makes it learn.

Write the update rule on the board: adjustment = input value × error × learning rate, and the new weight is the old one minus the adjustment. The input value is what came into the connection, the error is the one that reached the node it leads to, and the learning rate keeps each step small. Moving every weight a small step against its error is gradient descent.

Key Vocabulary

Weight update
Subtracting input value × error × learning rate from a weight.
Learning rate
A small number that sets how big each weight update is.
Gradient descent
Training by moving every weight a small step in the direction that makes the error smaller.
Training data
The examples a network learns from.
Testing data
Examples the network never learned from, used to check whether it really learned.

Guiding the activity

One update by hand

Pairs open The Recycling Drive Predictor, which asks for the output error, both middle errors and one weight's adjustment in round 1. 2 classes predicts 3 bags against 6: an error of -3, reaching the middle nodes as -3 and -1.5. The adjustment to the top middle node's weight is 2 × -3 × 0.02 = -0.12, which raises it from 1 to 1.12.

Find the learning rate

The Test Score Predictor 02 learns at a rate of 0.2, and its weights blow up. Pairs try the other rates and record what each one does. The one that works is 0.02; ask them to describe what goes wrong above it and below it.

Train, then test

Bring the class together for The Harbor Predictor, which learns swimmers and sailboats from sunshine and wind through a middle layer. Run it in swift mode across all the training days, then switch to the testing data. Ask the class to decide which of its two answers it learned better: the sailboats, off by about 0.7 a testing day against about 1.1 for the swimmers.

Post-activity discussion

  1. Why multiply by the input value in the update rule?

    Listen for A connection whose input was large had a large effect on the output, so it takes a large share of the fix. One whose input was zero had no effect and doesn't change.

  2. What happens when the learning rate is too large? Too small?

    Listen for Too large and each step overshoots, further each time, until it blows up. Too small and it learns so slowly it never gets there.

  3. Why judge the Harbor Predictor on testing data?

    Listen for A network can fit its training data without learning the pattern. Data it has never seen is the honest test.

Wrap up

As an exit ticket, each student follows one connection through all three steps. "A weight of 4 connects an input of 3 to an output with a target of 10. What is the output, the error, the adjustment at a learning rate of 0.1, and the new weight?" (12; 2; 0.6; 3.4).

Close the unit by asking what an AI model is, after training. Students should be able to answer: a very large set of weights and biases, each found by repeating the forward pass, back propagation and the weight update until the errors are small.

Ready to bring iiNNoo to your classroom?

Questions or ideas? Email us at support@iinnoo.app.

Start Learning How AI Learns