Abeona: Souvenirs from a P5.js journey

Description

Abeona is the Greek Goddess of the outward journey, looking over kids when they take their first steps away from home [1]. This project is an overall view of my short journey with P5.js, away from home (as I usually code in a financial/banking context). Along this road, I played with all 3 of the body-related technologies and used hand, face, and body recognition.

When facing a creativity tool as interactive as coding, building a game looks almost inevitable. So, the first study is a tribute to my inner child. I made a candy-eating game using facial recognition. The second study explores other experiences we can have on the 2D screen. This one uses body tracking is to entertain my dream to dive into the deep sea. The third study is my way to step back and reflect on how the digital world is not just affecting us but also defining us [2]. For this study, I employed hand-tracking technology to show how demanding the digital world could be.

The pain points of these implementations were mainly on the body tracking side. The tools introduced are not as accurate or fast-tracking as I would need for most of my ideas. It is essential to consider the limits of each library early in the design process. On the other hand, the P5.js community and immense resources are the Abeona of learning to code. These resources are excellent guides that make any implementation achievable.

 

1&2. The first control in this game is scrolling through head movement from side to side. With this way of scrolling, slowly, the user sees the “mouth” image moving in the same direction as they go.

gamemove

Scrolling through head movement

Secondly, the user has control over when the candies are eaten by opening and closing your “mouth” (this acts as a toggle click).

Eating candy through mouth opening

Eating candy through mouth opening

Only with an open mouth could users eat candies falling their way. The eaten sweets are clustered on the top left side of the canvas in a yellow bucket. This video shows the user scrolling and clicking with face-tracking technology.

Watch the video here

Try it now!

Click here to play this Halloween-themed game. Make sure you are in a well-lit area and have your face in front of a webcam. Give the tracker a few seconds to load, and a couple of seconds after you see your face, you can play.

Edit link

 

3. This project combines the joy of being in nature with coding. For this scroll, the user has to perform a swim-like activity. When arms are opening, the picture rolls. By swimming in front of the camera, users can see the lower parts of the Picture, deeper sea zones.

Watch the video here

I came across a website with a similar idea that has more details on creatures living in different depths of water [3].

Try it now!

Start your adventure by having your arms straight and in front of your chest like this:

Pause scroll with still arms

Pause scroll with still arms

Then start opening them up to sides while holding straight (As you would do in breaststrokes) and watch the image scrolling to a deeper sea area:

Image scrolls down as user opens up their arms

Image scrolls down as user open up their arms

Edit link

 

4. Last work signifies the competition for our attention in the digital world. In this project, a pushbutton follows the user’s index finger and makes itself click. But if the hand posture changes to a fist or high five, the button stops following the user. This game represents both the demanding digital world plus the fact that we can and should control our digital activities. The users gain control by awareness of the time they spend on every digital tool. In this interaction, “No action” is still an action [4]; the button will click if users do not restrain it.

Watch the video here

Try it now!

Put either hand on a well-lit blank surface. The camera could have a top or front-facing view. With a pointing pose like this, you will attract the button:

Pointing pose attracts the button

Pointing pose attracts the button

With other poses (fist, high-five), you can stop the button:

Put fingers together to stop the button from moving

Put fingers together to stop the button from moving

Edit link!

 

 

Bibliography

[1] Took, Thalia. Abeona, Roman Goddess of Journeys, http://www.thaliatook.com/OGOD/abeona.php

[2] Mühleisen, Martin. “F&d Article.” The Impact of Digital Technology on Society and Economic Growth – IMF F&D Magazine – June 2018 | Volume 55 | Number 2, https://www.imf.org/external/pubs/ft/fandd/2018/06/impact-of-digital-technology-on-economic-growth/muhleisen.htm.

[3] Liang, John, et al. “Scroll down to the Bottom of the Sea.” DeeperBlue.com, 17 Jan. 2020, https://www.deeperblue.com/scroll-down-to-the-bottom-of-the-sea/.

[4] Wilson, George, and Samuel Shpall. “Action.” Stanford Encyclopedia of Philosophy, Stanford University, 4 Apr. 2012, https://plato.stanford.edu/entries/action/.

 

Consulted and repurposed code:

Stock images used: