Augmented Alice

Project Description

Augmented Alice is a series of 4 interactive sketches that associated with each other to imagine an augmented journey in near future. In this experiment, I intend to trigger different scenarios in mixed reality settings by tracking body and hand position with p5.js and Posenet.

 

Overall successes/failures

The process of learning creative coding is like speaking a new language for me. At beginning, it took time to get familiar with the programming environment, the syntax and the logic flow before I started creating anything. In the first study, I experimented with replacing mouse clicking with interaction of computer visions, which went smoothly and encouraged me to take a step beyond. Continuing to involve PoseNet and Face API for body and face tracking function, I started off intimidating and a bit tough, but once I stopped staring at my frustrating face in webcam and spent time on examining every line of the code to find breakthrough, it turned out to be promising. The learning section of P5.js is really helpful for me when I hope to systematically understand how the code works. I believe there’s a lot more I can discover and work on creating more engaging responsive media to add playfulness to my interactive work in future.

 

Study 1 Hyper consumerism

Video Link: https://vimeo.com/616672424

Study Description:

The idea of this sketch origins from a guess of how mixed reality will change the concept of user profile and social interaction. I use emoji as an analogical representation of the new profile in near future since emoji is not only as means of expressing emotions, but also have been adopted as tools to express relationally useful roles in conversation. The interaction was triggered using the ml5 face detection and pose estimation model in p5.js. It follows the movement of the viewer’s nose and generate a changing emoji on the face. The key position such as left and right ears is also detected to determine the face angle.

 

Present Link: https://preview.p5js.org/beforespring/present/rASN7bEmF

Edit Link: https://editor.p5js.org/beforespring/sketches/rASN7bEmF

 

Study 2 Future Identity

Video Link: https://vimeo.com/616672424

Study Description:

The idea of this sketch origins from a guess of how mixed reality will change the concept of user profile and social interaction. I use emoji as an analogical representation of the new profile in near future, converting the hexadecimal Emojicode to decimal codes to make them appear. The interaction was triggered using the ml5 face detection and pose estimation model in p5.js. It follows the movement of the viewer’s nose and generate a changing emoji on the face. The key position such as left and right ears is also detected to determine the face angle.

 

Present Link: https://preview.p5js.org/beforespring/present/rASN7bEmF

Edit Link: https://editor.p5js.org/beforespring/sketches/rASN7bEmF

 

 

Study 3 Food Delivery

Video Link: https://vimeo.com/616658482

Study Description:

Playing with visual elements of emoji, I try to apply P5+Clmtrackr for facial tracking to this study, but experienced some technical error during the process, so eventually I uses Face-Api as the reference. This sketch provides an instant “food delivery” for the viewers. When the viewers open their mouths to interact with the computer, the food emoji will suddenly appear and fly into their mouths.

 

Present Link: https://preview.p5js.org/beforespring/present/rXS_tEO24

Edit Link: https://editor.p5js.org/beforespring/sketches/rXS_tEO24

 

Study 4 Tamagotchi

Video Link: https://vimeo.com/616668096

Study Description:

In this study, I created a virtual companion whose color of surface and eyes movement can be controlled with the position of viewers’ nose. With the viewer move left and right, the color varies from apple green to pink and the eyes followed the flow. I use PoseNet to capture the movement of nose and the map function to process the variables of RGB codes. During the process, I found that when I try to apply body as controller to generate 2 immediate feedback in one conditional statement, it didn’t work and shows unknown error. I shall try to figure out the reason in next few days.

Present Link: https://preview.p5js.org/beforespring/present/cQ2X7Twkp

Edit Link: https://editor.p5js.org/beforespring/sketches/cQ2X7Twkp