Color Blocking Body Glitches

In this experiment, I designed simple geometrical responses to movements of designated body parts with PoseNet. The glitch-art-like tint filter emphasizes the trippy imaginative space created in my webcam. My collection is designed to discover the beauty happening in the responsive relationship between 3D realistic webcam body captures and graphic shapes in motions.

The overall experience is super rewarding in the most surprising ways. The independent process of modifying existing codes, reading different variables, trying to write my own, repeatedly debugging through the errors, and successfully running it after all is a very hand-on, challenging, yet solid learning process. I find myself remembering the codes I debugged the most, reading the codes I write myself the best. The successful takeaway here is not only the technical knowledge on how to make body responsive art in p5.js, but also the feeling of inspiration and a sense of confidence towards coding and programming.

The hardest part in this experiment for me is understanding the relationship between the p5.js server and external elements like PoseNet. Editing the index.html and understanding codes referring to external variables gets confusing sometimes. But it truly is the best feeling when I navigate myself out of my confusions. And now, I look forward to all the cool things we will get to do together in the very near future.

#1 B&W Nose Stroke Brush:



This black and white nose-tracker tracks the location of the nose in continuous motion. It also tracks the eye coordinates: the closer you are to the webcam, the larger the circle will appear. The further away you are, the smaller. This is the first thing I made. I was very happy to figure out how to replace mouse with body part tracking. Something to add would be another motion that stops the tracking of the nose. Like if the program detects the wrists, the nose tracking motion stops. It would also be cool to see randomized shapes instead of just circles. I wish I figured that out.

#2 Glitching Nose filter:


Building on my #1 piece, I added the webcam and the glitching tint filter. The tracking trace also went away so the dots are more like a moving filter than a paint brush stroke in this one. After doing the Related Work on glitch art, I want to touch upon that topic in the experiment. I really enjoy how the tint filter turned out, adding pixilated color blocks onto the webcam as well a delayed capture time. The nose size shrinks when you are further away and vice versa. The visual effect of delayed capture really adds a vintage and trippy vibe to the work. I wish I figured out how to allow more than one nose detection in this one.

#3 Glitching Performer



I created these four thin triangles hanging from the y axis with coordinates to both wrists and elbows. When you make the triangles very thin, they look like strings. The red ball like the clown’s nose moves in the same left/right direction as the nose unless the nose stays centered in the screen. I love how this is capturing actions happening overtime all into one screen. In action, because of the delayed capturing of multiple actions into one frame, the moving red ball will turn into a thick round stroke instead of a ball every time the nose moves. It appeared as interesting to me to include Marionette string and clown nose together because they share similarities in my opinions. The string and the nose are all signature visual features that in some way define and limit the performers’ freedom on stage: what they do, how they should act, and who they must please.

#4 Tripping:


The last piece is made up with four triangles coming in from the edge of the canvas that all share one common coordinate, the nose. The triangles will all track the nose as the nose move and reshape themselves to keep that common coordinate on the nose. The transparency on white, black, and yellow triangles makes it harder to distinguish them from the blurry and half transparent background of realistic stuff. The pink triangle in full opacity is meant to contrast the rest and remind us that this visual is not real. The fun thing to do here is to place your nose in the center perfectly to turn the pink and black triangles into lines, making them always disappear.

Additional Documentation:






Access codes through Editor Links.