The Real-Time Virtual Sunglasses

My interest in ML5 is focused on real-time animated effects. Compared to other professional software such as Adobe Character, to make real-time face animation using ML5 is more customizable and simpler. Though the result may not be so highly finished, it is a great choice for designers and coders to produce visual work.

I found it easier for me to just use the p5 editor, however the ML5 script needs to be put in the HTML file in the p5 editor. (the fourth “script”)screenshot-2019-04-17-at-10-35-38

The model used is poseNet. It allows real-time human pose estimation, it can track for example where my eyes, nose, hands are and then build visual work on those positions.

screenshot-2019-04-17-at-10-50-26

Then I set the canvas and draw functions in the p5 editor, I used the gray filter to add more fun.

screenshot-2019-04-17-at-10-35-50

Program the poseNet into my coding. When everything is settled, we can see that the ml5 recognizes “object, object, object (which should be my face)…” from the WebCam.

screenshot-2019-04-17-at-11-15-11

After some research, I learned that nose to feet are coded as 0 to 16 in poseNet. The left eye and the right eye should be 1 and 2.

screenshot-2019-04-17-at-13-47-03

The first try:

screenshot-2019-04-17-at-14-15-31

01-2019-04-17-17_16_35

As the gif showed, if I move out of the screen the circles will not be able to track back.

The second try solved it: (if (poses.length > 0))

screenshot-2019-04-17-at-14-19-31

02-2019-04-17-17_17_10

In fact, I can call my project successful at this point, however, I wanted to make it more finished.

In the third try, I tested the lerp function and instead of a set size, the size of the ellipses are defined by the “distance”, which allows the ellipses to become larger or smaller as I move forward and backward:

screenshot-2019-04-17-at-16-27-10

03-2019-04-17-17_17_31

04-2019-04-17-17_17_50

 

Reference:

https://ml5js.org/docs/PoseNet

The Coding Train

 

Leave a Reply