Air Printing: Drawing Physical Objects with Leap

Experiment 4: Progress Report:

Air Printing: Drawing Physical Objects with Leap

 

Salisa Jatuweerapong, Sam Sylvester, Melissa Roberts, Mahnoor Shahid

Atelier I: Discovery 001

Kate Hartman, Adam Tindale, Haru Ji

2018-11-27

 

Inspiration

We started with an idea of drawing in the air and transmitting art onto the screen with the movements. At first, we thought of using an accelerometer or conductive paint proximity sensors. We didn’t want any sensors to be attached to the hand. Through research and feedback, we discovered the Leap Motion Controller and a project called “Air Matter”.

“Air Matter” is an interactive installation by Sofia Aranov. The installation takes a new approach on traditional pottering with Leap Motion Controller. The viewer draws a 3D pot in the air which is then 3D printed. An Arduino is also used with potentiometers to control aspects of the model.

 

Context

Related imageThis project is an exploration of alternative interfaces and virtual and physical space.

We took the “Air Matter” installation as our main inspiration. Instead of drawing a vase, we decided to draw a sculpture made of thin rectangles. This idea was based on the disappearing sculptures by Julian Voss-Andreae– which, depending on the point of view, seem to disappear into thin air. Our project “conjures” physical objects from thin air, yet the physical objects it creates disappears back into thin air (conceptually. Our final design isn’t print thin enough for that to actually work). There’s something to be said about the transfer of objects from physical, to virtual, back to physical space, and its permanence and materiality in each layer.

Related interfaces include: webcam motion tracking, Kinect, and a variety of glove interfaces (Captoglove game controller, Mi.Mu). We chose to explore Leap as it seemed an exciting challenge; as well, we wanted to explore extremely non-invasive, non-physical interfaces (no gloves).

Other work that is being done on Leap includes Project Northstar, a new AR interface that aims to redefine the AR experience. Otherwise, the Leap team is focused on creating accurate hand tracking software to be used as a tool for any other projects.

Links to Contextual Work

Air Matter: https://www.sofiaaronov.com/air-matter

Julian Voss-Andreae Sculpture: https://www.youtube.com/watch?v=ukukcQftowk

Mi.Mu Gloves: https://mimugloves.com/

Northstar:https://developer.leapmotion.com/northstar

Images of the work in progress

Progress Timeline Checklist (link).

Thursday 22nd: 

Designing the visuals

11-23-04811-23-047

Friday 23rd:

Getting Started with Leap

received_305280590079371 received_334901954003267 received_347159419428689

Tried out the Leap, ran into some challenges with the different software available for download. Tutorials we found (Research) seem to be for some versions of the software and not others.

Monday 26th:

Processing Sketch with mouseDragged

screen-shot-2018-11-25-at-4-11-45-pm screen-shot-2018-11-25-at-8-34-32-pm screen-shot-2018-11-26-at-1-02-43-pm

According to the sketch, the person would draw a squiggle with their finger as an outline for the sculpture. Thin rectangles should be placed at specific X positions to maintain a consistent gap between them. The height of the rectangles is determined by the Y position of the cursor or the finger of the person.

Processing Sketch with Leap

finger_painting_1003 finger_painting_1010

finger_painting_1102
Wrote code in Processing using the LeapMotion + Arduino Processing library. Used input from the Leap to draw a line. Boxes are drawn centered along the vertical middle of the screen, height and depth of the box are dependent on the y position of the user’s finger, placement along x-axis is dependent on the x position of the user’s finger (box is drawn if the x value is divisible by 50). The width of the box is constant. There is a bit of a lag between the line being drawn and the box is drawn, so the line has to be drawn slowly.

JavaScript finger/hand code reference: https://developer-archive.leapmotion.com/documentation/javascript/api/Leap_Classes.html?proglang=javascript

Tuesday 26th:

Converted Processing Sketch to Javascript

finger_painting-1 finger_painting-2

There was no export as STL file for the processing version we were using, so had to switch to javascript. This was important since Melissa’s STL library code from Experiment 2 had proven to work.

In the Javascript code, we used three libraries

  • Three.js (export STL library)
  • Leap.js (Leap Motion Controller for the javascript library)
  • P5.js
  • Serial Port

Pictured above is the functional p5.js/leap.js code.

Implementing three.js library into the functional p5.js/leap.js code

screenshot-58

This involved getting rid of p5 code, as the three libraries (three, p5, and leap) didn’t work well together. The biggest changes were changing how we created 3D shapes, creating a renderer to replace our canvas, setting up a scene (full of our shapes) to animate and render, and including an STL exporter, which will allow us to print the 3D object drawn on the screen.

The Leap coordinate system seemed to be very different from the Three.js coordinate system, which means the shapes we had created displayed as far larger than originally intended. However, the code technically works. The scene (airPrint) has our shapes in it, and they are being reproduced on the screen. Leap has a coordinate system where the units are millimeters, the origin being the center of the top surface of the Leap.

Further steps involve possibly implementing additional controls with Leap.

Connected Arduino to USB C

screen-shot-2018-11-27-at-12-03-28-pm

Using WebUSB, created a workflow for a physical button push as ‘enter’ on the keyboard.

This push button will download the STL file from the sketch which can then be used to 3D print.

GitHub: https://github.com/SuckerPunchQueen/Atelier-Ex-4?fbclid=IwAR1wH5XmWsn4G-S9e3zezb8yrDMfmp56uRA7xzTVI80JTh3Wj-hnKFjrZ-w 

Previous Experiments

Melissa’s Nameblem provided a starting point for a generative code → .stl → 3D printing workflow. Melissa’s original project combined p5.js with three.js and exported into a .stl file that she would have to manually fix on 3D Builder. While we had hoped to just reuse this code for Air Printing (it is a rather technical workflow), we are having issues interfacing Leap.js with p5.js. As well, something we are hoping we can do is automating the process.

Mahnoor’s work with capacitive sensing on Experiment 3 inspired our original interface for air sensing. Her umbrella had a proximity sensor created by using conductive paint and the CapSense library, and we reasoned we could use two capacitive sensors on two different axis to take an x-position and y-position for a hand. This would not be as accurate as Leap, and since Melissa wanted to buy a Leap anyways, we opted to use that for our project.

We are using p5.js, which Adam introduced us to in Experiment 1 to draw our design.

Haru’s Endless Forms Most Beautiful, specifically the experiments based off William Latham’s work, was our launch point for the visual design. Originally, our code was a bastardized Tinkercad / building blocks game. We felt that we could do more visually, to elevate the project from a tool/workspace to an actual artwork. We looked at the rule-based work we explored in Haru’s unit for inspiration, since we were already restricted by rules as to what would practically be able to print (basic geometry, cubes, connected lines).

Leave a Reply