The Real-Time Virtual Sunglasses

My interest in ML5 is focused on real-time animated effects. Compared to other professional software such as Adobe Character, to make real-time face animation using ML5 is more customizable and simpler. Though the result may not be so highly finished, it is a great choice for designers and coders to produce visual work.

I found it easier for me to just use the p5 editor, however the ML5 script needs to be put in the HTML file in the p5 editor. (the fourth “script”)screenshot-2019-04-17-at-10-35-38

The model used is poseNet. It allows real-time human pose estimation, it can track for example where my eyes, nose, hands are and then build visual work on those positions.


Then I set the canvas and draw functions in the p5 editor, I used the gray filter to add more fun.


Program the poseNet into my coding. When everything is settled, we can see that the ml5 recognizes “object, object, object (which should be my face)…” from the WebCam.


After some research, I learned that nose to feet are coded as 0 to 16 in poseNet. The left eye and the right eye should be 1 and 2.


The first try:



As the gif showed, if I move out of the screen the circles will not be able to track back.

The second try solved it: (if (poses.length > 0))



In fact, I can call my project successful at this point, however, I wanted to make it more finished.

In the third try, I tested the lerp function and instead of a set size, the size of the ellipses are defined by the “distance”, which allows the ellipses to become larger or smaller as I move forward and backward:






The Coding Train


Leave a Reply