Author Archive

Space Oddity

By Shuting Zhou & Winter Yan

Experience Video | How It Works Video | Github Code

Key Project Image

img_4391

Project Description

Space Oddity is an interactive installation based on tactile interface and screen-based digital experience. As its name implies, this project is inspired by David Bowie’s song and it attempts to provide a thrilling experience of space travel. It consists of a wood box as spaceship dashboard and four sensors as controller for the adventure. The chosen piece of media is a video clip of wormhole travel. Users are expected to start the exploration of space and the unknown by interacting with the controllers. Each sensor will trigger a sudden change on the screen. Buckle up, space travellers!

The tangible interface is composed of a speed controller with a sliding potentiometer underneath, and decorated with a ball Christmas ornament; a toggle which is functioned with force sensor; two buttons which is a capacitive touch sensor and an ultrasonic sensor to detect the position of the hand of user. Users can choose to interact with one of the four sensors. By playing with the controller, users are able to adjust the travel speed, check the distance to a certain planet, trigger unexpected species in universe or even encounter an unidentified flying object.

Development Images

img_4345  img_4356img_4374  img_4354

Final Project Images

img_4380  img_4383img_4399

Diagram

microsoftteams-image

Experiment 2 Trick and Treat

cover

Project Description

Trick & Treat is an interactive installation designed for trick-or-treaters for Halloween. Before Thanksgiving, Dr. Kieran Moore, Ontario’s chief medical officer of health, released new guidelines on the celebration of Halloween. Trick or treating is officially back! However, social distancing is still accentuated, so I come up with the idea of designing a little monster box that can do tricks and collect candies at the same time.

In this work, I used purple card paper to wrap the box and decorate it with orange LEDs – they will work as blinking “eyes” of the little monster. Since the colors are traditional pick for the Halloween, the ones who see it appear at the doorstep will immediately relate it to trick or treat and interact with it. This installation uses level of luminosity to measure the proximity. Light sensor is placed at the front the box as the nose of the little monster. When a person approaches the box at a short distance, the amount of light received by the sensor will be reduced, which will trigger the LEDs change into 3 different modes. There is also a correlation between proximity and servo motor. As the person get closer and the level of lights goes down, the servo will push the lid of the box up and down, to surprise the person who interact as well as invite him/her to give a treat.

Bibliography:

CBC News (2021). ‘Trick-or-treating back, Thanksgiving dinners on (for the vaccinated), Ontario’s top doctor says. CBC, 7 October 2021

https://www.cbc.ca/news/canada/toronto/covid-19-ontario-october-7-2021-halloween-thanksgiving-1.6203047

 

Discussion of Calm Technology

This project uses one principles of Calm Technology:

Technology can communicate but doesn’t need to speak: The color palette and monster-like figure deliver the Halloween information in a natural way that in a lucid way that the user will probably understand. The change of blinking LEDs when the user approaches conveys a succinct signal that makes user know it reflects on the change of distance between him/her and the object. Through the up and down of the box lid, it indicates that the little monster also displays function of a box.

 

 

Videos

Experience Video | How It Works Video

 

Arduino Code

https://github.com/winteeeryan/Trick-Treat/blob/main/Trick%20Treat%20Code

 

Final Project Images

final-image-001

final-image-002

final-image-003

final-image-004

 

Final Project Images

development-001

development-002

development-003

development-004

 

Circuit diagram

circuit_bb

Augmented Alice

Project Description

Augmented Alice is a series of 4 interactive sketches that associated with each other to imagine an augmented journey in near future. In this experiment, I intend to trigger different scenarios in mixed reality settings by tracking body and hand position with p5.js and Posenet.

 

Overall successes/failures

The process of learning creative coding is like speaking a new language for me. At beginning, it took time to get familiar with the programming environment, the syntax and the logic flow before I started creating anything. In the first study, I experimented with replacing mouse clicking with interaction of computer visions, which went smoothly and encouraged me to take a step beyond. Continuing to involve PoseNet and Face API for body and face tracking function, I started off intimidating and a bit tough, but once I stopped staring at my frustrating face in webcam and spent time on examining every line of the code to find breakthrough, it turned out to be promising. The learning section of P5.js is really helpful for me when I hope to systematically understand how the code works. I believe there’s a lot more I can discover and work on creating more engaging responsive media to add playfulness to my interactive work in future.

 

Study 1 Hyper consumerism

Video Link: https://vimeo.com/616672424

Study Description:

The idea of this sketch origins from a guess of how mixed reality will change the concept of user profile and social interaction. I use emoji as an analogical representation of the new profile in near future since emoji is not only as means of expressing emotions, but also have been adopted as tools to express relationally useful roles in conversation. The interaction was triggered using the ml5 face detection and pose estimation model in p5.js. It follows the movement of the viewer’s nose and generate a changing emoji on the face. The key position such as left and right ears is also detected to determine the face angle.

 

Present Link: https://preview.p5js.org/beforespring/present/rASN7bEmF

Edit Link: https://editor.p5js.org/beforespring/sketches/rASN7bEmF

 

Study 2 Future Identity

Video Link: https://vimeo.com/616672424

Study Description:

The idea of this sketch origins from a guess of how mixed reality will change the concept of user profile and social interaction. I use emoji as an analogical representation of the new profile in near future, converting the hexadecimal Emojicode to decimal codes to make them appear. The interaction was triggered using the ml5 face detection and pose estimation model in p5.js. It follows the movement of the viewer’s nose and generate a changing emoji on the face. The key position such as left and right ears is also detected to determine the face angle.

 

Present Link: https://preview.p5js.org/beforespring/present/rASN7bEmF

Edit Link: https://editor.p5js.org/beforespring/sketches/rASN7bEmF

 

 

Study 3 Food Delivery

Video Link: https://vimeo.com/616658482

Study Description:

Playing with visual elements of emoji, I try to apply P5+Clmtrackr for facial tracking to this study, but experienced some technical error during the process, so eventually I uses Face-Api as the reference. This sketch provides an instant “food delivery” for the viewers. When the viewers open their mouths to interact with the computer, the food emoji will suddenly appear and fly into their mouths.

 

Present Link: https://preview.p5js.org/beforespring/present/rXS_tEO24

Edit Link: https://editor.p5js.org/beforespring/sketches/rXS_tEO24

 

Study 4 Tamagotchi

Video Link: https://vimeo.com/616668096

Study Description:

In this study, I created a virtual companion whose color of surface and eyes movement can be controlled with the position of viewers’ nose. With the viewer move left and right, the color varies from apple green to pink and the eyes followed the flow. I use PoseNet to capture the movement of nose and the map function to process the variables of RGB codes. During the process, I found that when I try to apply body as controller to generate 2 immediate feedback in one conditional statement, it didn’t work and shows unknown error. I shall try to figure out the reason in next few days.

Present Link: https://preview.p5js.org/beforespring/present/cQ2X7Twkp

Edit Link: https://editor.p5js.org/beforespring/sketches/cQ2X7Twkp

Use of this service is governed by the IT Acceptable Use and Web Technologies policies.
Privacy Notice: It is possible for your name, e-mail address, and/or student/staff/faculty UserID to be publicly revealed if you choose to use OCAD University Blogs.