This being my first foray into the world of coding, “Breaking The Ice” is a group of 4 interactive art experiments that have for all intents and purposes, broken the frozen layer of ice between myself and coding in the world of Java script. After a cautious start, I am inspired to delve deeper into the virtual world of multimedia art and interactivity.

Keeping in mind my tag of being a novice, I wanted the art/media I was creating to be fairly straightforward and focus rather on the interactivity between the body movements and the art to see how movements and gestures in the physical world could alter the outcome in the virtual world. The interactive artworks all start from a single shape and respond to changes in movements and gestures of the viewers, such that the viewers can direct the outcome of the piece.

The task at hand was to use our bodies as a controller and to find/create 2 ways to perform a “Click” function and 2 ways to perform a “Scroll” function, both otherwise arbitrary motions we take for granted, carried forward by the mouse. In this instance, we will substitute the action of a mouse “click” or “scroll” with a predefined body movement or gesture. Made using the p5.js web editor, ML5 Library and PoseNet.

Click #1 : Red-tangles

My first piece is a play on words on ‘Rectangle’. In this case, I have moved the background to setup which allows it to draw over itself again and again in a loop. The screen will stay red until both wrists are brought together in front of the face, like a clicking action. Just then small, white rectangles should start to emerge on screen. And when the wrists are separated again, the ‘red-tangles’ will start to fill the screen up again, eventually going back to an all red canvas.

Video Demonstration: https://vimeo.com/616961584

Source Code: https://editor.p5js.org/arjundutt20/sketches/CLlRpep7g

Presentation: https://preview.p5js.org/arjundutt20/present/CLlRpep7g

Click #2 : Vertigo

‘Vertigo’ is intended to recreate the feeling of freefalling. Stimulated by my fear of sudden vertical drops, I wanted to see if I could visualize a long, constant free-fall. In this piece, the constantly changing grayscale of the rectangles is meant to depict motion as you fall past each subsequent rectangle. The grayscale freezes if the viewers right wrist touches their left shoulder. An action that can be considered to be ‘half’ of the full brace position one assumes when they’re about to face impact.

Video Demonstration:  https://vimeo.com/616962516

Source Code: https://editor.p5js.org/arjundutt20/sketches/skWqat5x3

Presentation: https://preview.p5js.org/arjundutt20/present/skWqat5x3

Scroll #1 : Hollipse

‘Hollipse’ or the hollow ellipse was inspired by artists who can draw with both hands simultaneously. Initially, I had given the draw function to the X/Y position of the mouse. So whenever the mouse moved on the canvas it would form a design and its subsequent mirror design would be drawn too. I had later substituted the mouse function for the nose tracking feature in PoseNet. A scroll is a lateral movement in that could move your position on the screen, in this example, I have allowed the scroll to move freely in any direction.

Video Demonstration: https://vimeo.com/616960983

Source Code: https://editor.p5js.org/arjundutt20/sketches/0bPTkezl1

Presentation: https://preview.p5js.org/arjundutt20/present/0bPTkezl1

Scroll #2 : The Blue Ball

The Blue Ball was the first piece I worked on and one that proved the most informative. The Blue Ball is the representation of the nose on the screen in real time. Wherever you move your face, the ball will follow (albeit mirrored). The closer you bring your face to the camera, the bigger the ball will become. If you take a long, sweeping step back- the ball will become smaller. This back and forth motion relative to the placement of the screen is how I wanted to relate it to the ‘scroll’ function.

Video Demonstration: https://vimeo.com/616962613

Source Code: https://editor.p5js.org/arjundutt20/sketches/63bL8aicu

Presentation: https://preview.p5js.org/arjundutt20/present/63bL8aicu

Observations and Reflections:

I was very apprehensive at the start because this was my first time I’d be dabbling in coding. I found the syntax to be particularly difficult to wrap my hear around. The more I watched videos and practiced and asked questions of my classmates, the more familiar it became. I am definitely going to keep tinkering with P5.js and PoseNet, it opens the door to interactivity between the physical and virtual world.