Session 7: Augmented Reality Game - Wrist-Tracking Fruit Slicer

🍉 Lesson 7: Building an AR Ninja Fruit Game

🥷 Turn your hand into a virtual blade

In this lesson, we combine AI Pose Detection with the webcam to create an AR Ninja Fruit game.

Players don't need a keyboard or mouse. The AI tracks the position of the wrist and turns your hand movement into a virtual blade for slicing fruit.

🎯 Learning objectives

After this lesson, you will be able to:

• Use AI to track the position of your hand.
• Control a sprite with wrist coordinates.
• Create lots of fruit with clones.
• Detect collisions between the hand and objects.
• Build rules for scoring, time and lives.

1. Setting up the game

Create or choose these sprites:

Choose a simple backdrop so the webcam image and sprites are easy to see.

Create the variables:

Score
Lives
Time

Starting values:

when green flag clicked
set [Score] to [0]
set [Lives] to [3]
set [Time] to [60]

2. Turning on AI Pose Detection

Add the Pose Detection extension, then turn on the camera:

when green flag clicked
turn video on stage with 20% transparency

forever
    analyse pose from camera

To play well:

• Stand at a moderate distance from the camera.
• Keep your upper body and both arms in the frame.
• Choose a well-lit place.
• Clear enough space to move safely.

3. Controlling the blade with your hand

Create two variables:

Wrist_X
Wrist_Y

Use the right wrist to control the blade:

when green flag clicked
show

forever
    analyse pose from camera

    set [Wrist_X] to
        [x position of right wrist]

    set [Wrist_Y] to
        [y position of right wrist]

    go to x: [Wrist_X] y: [Wrist_Y]

Now the Blade sprite follows the player's hand.

If left and right are reversed, make the x coordinate negative:

go to x: [0 - Wrist_X] y: [Wrist_Y]

4. Creating a blade trail

To leave a short glowing trail behind the blade, we use fading clones. Each clone stays where the blade was, then fades away, forming a trail behind it.

Add to the Blade sprite:

when green flag clicked
forever
    create clone of myself
    wait 0.03 seconds

when I start as a clone
repeat 10
    change [ghost] effect by 10
delete this clone

After 10 repeats, the clone is fully transparent and is deleted, so the trail doesn't slow the game down.

5. Making fruit appear continuously

Use clones to create lots of fruit:

when green flag clicked
hide

forever
    wait (pick random 1 to 2) seconds
    create clone of myself

6. Slicing fruit and scoring

Each clone flies upwards and checks for a collision inside the same flight loop. That way, the fruit can be sliced anywhere along its path:

when I start as a clone
go to x: (pick random -200 to 200) y: -170
show

repeat until <y position greater than 180>
    change y by 8
    turn right 10 degrees

    if <touching [Blade]?> then
        start sound [Slice]
        change [Score] by 1
        switch costume to [Sliced fruit]
        wait 0.2 seconds
        delete this clone

    wait 0.03 seconds

delete this clone

Each fruit scores only once because the clone is deleted straight after it is sliced. Fruit that flies off the top of the stage is also deleted so the game doesn't slow down.

To make the fruit fly more naturally, you can change both the x and y coordinates on each repeat.

You can give different points for each type:

7. Adding bombs

Bombs appear like fruit but cost you a life:

when I start as a clone
go to a random position at the bottom
show

repeat until <y position greater than 180>
    change y by 7
    turn right 12 degrees

    if <touching [Blade]?> then
        start sound [Explosion]
        change [Lives] by -1
        delete this clone

Keep checking the number of lives in the Blade sprite:

when green flag clicked
forever
    if <Lives ≤ 0> then
        broadcast [Game over]
        stop [this script]

The condition must be inside the forever loop. If it is only checked once at the start, the game will never notice that the player has run out of lives.

8. Building a countdown timer

when green flag clicked
set [Time] to [60]

repeat 60
    wait 1 seconds
    change [Time] by -1

broadcast [Game over]

When the game-over message is received:

when I receive [Game over]
say (join "Your score: " [Score]) for 3 seconds
stop all

9. The complete rules

1. 1The player has 60 seconds.
2. 2Move your hand to control the blade.
3. 3Slice fruit to earn points.
4. 4Avoid touching bombs.
5. 5Touching a bomb costs one life.
6. 6The game ends when time runs out or you lose all your lives.

🧪 Project check

Before you finish, check that:

• The blade follows the wrist correctly.
• The camera isn't flipped by mistake.
• Fruit appears in many different places.
• Each fruit scores only once.
• Touching a bomb reduces lives.
• The timer goes down once every second.
• The game stops when time runs out or lives reach zero.
• Clicking the green flag restarts the game.

🧩 Upgrade challenges

Choose one idea to develop further:

1. 1Make the fruit faster as the score rises.
2. 2Add a golden fruit worth 5 points.
3. 3Use both hands to control two blades.
4. 4Add a split-in-half effect when you slice.
5. 5Create a high-score table.
6. 6Add a start screen and a replay button.

Example of increasing the difficulty:

if <Score greater than 10> then
    reduce the wait between new fruit
    increase the flying speed

✅ Remember

• Pose Detection finds the position of the wrist.
• Wrist coordinates control the blade.
• Clones create lots of fruit without needing many sprites.
• Collision checks add points or take away lives.
• Variables store the score, lives and time.
• The camera needs good light for the AI to track movement reliably.

1. 1Which keypoint controls the blade?
2. 2Why use clones to create the fruit?
3. 3How do you stop one fruit from scoring several times?
4. 4What makes the game end?