Digital Nose Pen by Unnikrishnan Kalidas


This experiment is a simple project to visualize the body as a controller on a Digital interface like a computer screen. The idea was inspired by the ‘ Nose Scribbler’ by Luisa Pereira, which was shared with us initially as an example pertaining to this particular Experiment. The project involved using tools such as p5.js to draw and ml5 and PoseNet to identify the nose pointer on the video footage that is fed into the code with the help of the webcam on a computer. ml5 and PoseNet identifies the nose pointer skeleton from the video footage and saves it as one set of data, while p5 identifies this data and creates a corresponding dot on the pointer for every frame of the video footage. Both the pieces of information are then overlapped onto one another and presented finally as the nose drawing NosePen.

Present  :

Edit         :

NosePen experience video : final-nose



Brief description of the study:  The project entailed using two different tools to be integrated and used to create a digital pen using a body part. As a further extension to this project other parts that are identified by the ml5 skeleton could be used, also the depth information could be implemented to the interaction. Initially, the size of the dot that follows and the positional difference of the point affected the smoothness of the drawing and hence made it more jittery. This was later on resolved.

Project Context: 

The initial idea was to create a simple project that can be executed well due to the time constraint. The ‘Nose Scribbler’ project presented during the course was the inspiration to this project. A further addition to this experiment in my opinion would be to create a modulating sound device to the system which could then create a digital installation where in artist could move on screen and modulate the sounds simultaeneously.

Another simple and interesting project I came across was ‘Cat face’;  A project  to insert elements of a cat onto a recognized face by By Denisa Marcisovska, where in PoseNet and ml5 detect a face fed into the system and adds ears, nose and whiskers to make a cat face.

Nose Scribbler :

Cat face :

Bibliography :