Session 5: Enabling Computer Vision - AI Face Detection Extension Setup

👁️ Lesson 5: Turning a Webcam into the Smart Eyes of AI

🤖 Can a computer see?

Welcome to the world of Computer Vision!

In this lesson, the computer will use a webcam to:

• Detect faces.
• Locate the eyes and nose.
• Follow the user's movements.
• Control a PictoBlox sprite with face movements.

🎯 Learning objectives

After this lesson, you will be able to:

• Add the Face Detection extension to PictoBlox.
• Show the webcam video on the stage.
• Analyse faces from the camera.
• Read the x and y coordinates of the eyes or nose.
• Control a sprite with face movements.

1. Preparing the webcam

Before you start coding:

1. 1Check that the webcam is connected.
2. 2Open PictoBlox.
3. 3Allow PictoBlox to use the camera.
4. 4Sit somewhere with good lighting.
5. 5Place your face in the centre of the camera view.

2. Adding the Face Detection extension

Follow these steps:

1. 1Click the Extension button in the bottom-left corner.
2. 2Find and select Face Detection.
3. 3Wait for PictoBlox to load the detection model.
4. 4Check that the Face Detection blocks have appeared.

Some block names may differ slightly between PictoBlox versions.

3. Showing the camera video

Create this program:

when green flag clicked
turn video on stage with 0% transparency

Transparency controls how visible the video is:

If you can't see the video, check the camera permissions in your browser or operating system.

4. Analysing faces

To let the AI detect faces, use the block:

analyse image from camera

This block must run continuously so the system updates as the face moves:

when green flag clicked
turn video on stage with 0% transparency

forever
    analyse image from camera

The AI can locate features such as:

• The left and right eyes.
• The nose.
• The mouth.
• The position of the face.

5. Reading the nose coordinates

Each point on the face has two coordinates:

• x: the horizontal position.
• y: the vertical position.

Use the blocks:

get [x] position of [nose] of face (1)
get [y] position of [nose] of face (1)

Create two variables:

Nose_X
Nose_Y

Then update the coordinates continuously:

when green flag clicked
turn video on stage with 0% transparency

forever
    analyse image from camera

    set [Nose_X] to
        [x position of nose of face 1]

    set [Nose_Y] to
        [y position of nose of face 1]

When you move your head, the values of Nose_X and Nose_Y change too.

6. Mini project: control a sprite with your face

Make the sprite follow the position of the user's nose.

when green flag clicked
turn video on stage with 20% transparency

forever
    analyse image from camera

    set [Nose_X] to
        [x position of nose of face 1]

    set [Nose_Y] to
        [y position of nose of face 1]

    go to x: [Nose_X] y: [Nose_Y]

Try:

• Leaning your head to the left.
• Leaning your head to the right.
• Moving your head up.
• Moving your head down.

Watch whether the sprite follows your face.

7. Reacting to the face position

You can use conditions so the sprite reacts more intelligently:

if <Nose_X greater than 100> then
    say "You're leaning right!"
else
    if <Nose_X less than -100> then
        say "You're leaning left!"

Or change the costume:

if <Nose_Y greater than 80> then
    switch costume to [Happy]
else
    switch costume to [Normal]

8. If the AI cannot detect a face

Check:

• Is the webcam turned on?
• Has PictoBlox been given camera permission?
• Is your face inside the frame?
• Is the lighting bright enough?
• Is anything covering your eyes or nose?
• Is the analyse image from camera block inside a loop?

🧩 Challenge

Upgrade the project with one of these ideas:

1. 1Steer a spaceship by moving your head.
2. 2Make the sprite wear a virtual hat on the webcam image.
3. 3Play a sound when the user tilts their head.
4. 4Control the sprite with the position of your eyes.
5. 5Create a collecting game that needs no keyboard.

✅ Remember

• The webcam lets the computer capture images.
• Face Detection helps the AI find faces and facial features.
• The analysis block must run continuously to keep the image updated.
• The x coordinate is the horizontal position.
• The y coordinate is the vertical position.
• Face coordinates can be used to control a sprite.

1. 1Which extension is used to detect faces?
2. 2Why must the camera image be analysed continuously?
3. 3What do the x and y coordinates represent?
4. 4How does lighting affect detection?