Session 8: Facial Emotion Recognition & Part 2 Synthesis

😊 Lesson 8: Artificial Emotions — A Robot That Smiles and Feels Sad

🤖 Can a robot understand emotions?

In this lesson, we use AI to recognise some facial expressions through the webcam.

A deep neural network analyses the image and predicts the closest matching expression, such as:

• Happy.
• Sad.
• Surprised.
• Angry.
• Neutral.

🎯 Learning objectives

After this lesson, you will be able to:

• Understand how AI classifies facial expressions.
• Turn on the camera and analyse images in PictoBlox.
• Read the expression name and its confidence.
• Make a sprite react to the detection result.
• Recognise some limits of AI systems.

1. How does AI learn to recognise expressions?

To build an expression recognition system, the computer learns from many labelled face images.

For example:

The neural network finds features in the image and compares them with the patterns it has learned.

The basic process:

Webcam image
→ Detect the face
→ Analyse its features
→ Compare with learned data
→ Predict the expression

2. Adding the expression recognition extension

In PictoBlox:

1. 1Click Extension.
2. 2Choose Face Detection or an expression recognition extension.
3. 3Allow PictoBlox to use the webcam.
4. 4Wait for the AI model to finish loading.
5. 5Check that the recognition blocks have appeared.

Extension and block names may differ slightly between PictoBlox versions.

3. Turning on the camera and analysing images

Create this program:

when green flag clicked
turn video on stage with 20% transparency

forever
    analyse image from camera

For better recognition:

• Sit somewhere with good lighting.
• Keep your face in the centre of the camera.
• Don't sit too far from the webcam.
• Avoid having several people in the frame.
• Hold each expression for about one second.

4. Reading the recognition result

Create two variables:

Expression
Confidence

Update the result continuously:

when green flag clicked
turn video on stage with 20% transparency

forever
    analyse image from camera

    set [Expression] to
        [expression of face 1]

    set [Confidence] to
        [expression confidence of face 1]

Expression stores the name of the predicted expression.

Confidence shows how sure the AI is. The higher the confidence, the more reliable the result, but it can still be wrong.

5. Making the sprite react to expressions

if <Expression = [Happy]> then
    switch costume to [Smiling robot]
    say "What a great smile!" for 2 seconds

if <Expression = [Sad]> then
    switch costume to [Encouraging robot]
    say "Let's try something fun together!" for 2 seconds

if <Expression = [Surprised]> then
    switch costume to [Surprised robot]
    start sound [Ooh]

if <Expression = [Angry]> then
    switch costume to [Calm robot]
    say "Let's take a slow, deep breath together!" for 2 seconds

6. Only react when the AI is confident enough

If confidence is low, the sprite shouldn't react straight away.

if <Confidence greater than 70> then
    if <Expression = [Happy]> then
        switch costume to [Smiling robot]
    else
        if <Expression = [Sad]> then
            switch costume to [Encouraging robot]
else
    say "I can't see your face clearly yet."

A threshold stops the sprite from switching states constantly when the AI is unsure.

7. Mini project: the emotion buddy robot

Create a robot that changes its face and words to match the user's expression.

when green flag clicked
turn video on stage with 20% transparency

forever
    analyse image from camera

    set [Expression] to [expression of face 1]
    set [Confidence] to [expression confidence]

    if <Confidence greater than 70> then
        if <Expression = [Happy]> then
            switch costume to [Happy]
            say "Hi! Today is a great day!" for 1 seconds
        else
            if <Expression = [Sad]> then
                switch costume to [Encouraging]
                say "I'm always here for you." for 1 seconds
            else
                if <Expression = [Surprised]> then
                    switch costume to [Surprised]
                    say "What's so interesting?" for 1 seconds
                else
                    if <Expression = [Angry]> then
                        switch costume to [Calm]
                        say "Let's take a slow, deep breath together!" for 1 seconds
                    else
                        switch costume to [Neutral]
    else
        switch costume to [Thinking]

8. Why can AI get it wrong?

Results may be inaccurate when:

• The room is too dark or lit from behind.
• The face is covered by hair, a mask or a hand.
• The user is too far from the camera.
• The face is turned to the side.
• The expression changes too quickly.
• The training data was not diverse enough.

If the AI gets it wrong, change the lighting, look straight at the camera and hold the expression steady.

9. Does AI really know emotions?

The AI is performing facial expression classification, not reading minds.

A person might:

• Smile while still feeling worried.
• Not smile but still feel happy.
• Have natural expressions that differ from the AI's training data.

So AI results should never be used to judge, grade or make important decisions about a person.

🧩 Challenge

Upgrade the project with one idea:

1. 1Add a different sound for each expression.
2. 2Count how many times the AI detects a happy expression.
3. 3Make the sprite change colour with each expression.
4. 4Show a matching emoji.
5. 5Only react when the same expression is detected three times in a row.

Example of counting happy expressions:

if <Expression = [Happy]> then
    change [Happy_Count] by 1

✅ Remember

• AI learns from many labelled face images.
• The neural network uses image features to predict expressions.
• Confidence shows how certain the prediction is.
• Set a confidence threshold before the sprite reacts.
• Lighting and face position affect the result.
• AI recognises expressions; it doesn't know a person's true feelings.

1. 1What does AI use to classify expressions?
2. 2What does confidence mean?
3. 3Why do we need a confidence threshold?
4. 4Can AI know a person's true feelings exactly?