Absolutely on Music

by Lilian Leung

Project Description

Absolutely on Music is an interactive prototype of a potential living space. The space is composed of a sensor activated light that turns on when a participant sits on the chair and a copy of Haruki Murakami and Seiji Ozawa’s book Absolutely on Music, which plays the music the author and conductor talk about in each chapter of the book.

This experiment expands upon my personal exploration in tangible interfaces, as well as research further into slow-tech and the use of Zero UI (invisible user interfaces). This exploration is meant to re-evaluate our relationship with technology as being able to augment everyday inanimate objects rather that creating alternative screen-based experiences centered around a hand-held device. The audio played beneath the chair is played in context to each chapter of the book, divided between six individual conversations centered around a different topic and part of Ozawa’s career. There are five tracks played due to the sixth chapter being without a musical piece discussed in detail. The auditory feedback playing the music featured in the book creates a multi-sensory experience, and broadens the audience of the book rather than being solely to music experts that don’t require musical reference, to anyone looking to enjoy a light read.

Project Process

November 21 – 22 (Proposal Day)

Early research pointed to using an Arduino UNO instead of an Arduino Nano so I’d be able to use an MP3 shield to play my audio rather than depending on using Processing. For early exploration, I looked into using a combination of flex sensors and pressure sensors on the binding of the book and on the front and back cover to detect when the book was being picked up. This layout was based on inspiration I found by Yining Shi (2015), where they mapped the Jurassic Park novel with the movie.

After having my proposal meeting, I switched to using copper tape instead of flex sensors as switches to make the thing more reliable data. From there I decided on the modes of the experiment and how the book and chair should behave when not being used.


Idle Mode Active
Lamp – Dim Light Lamp – Full Brightness
Book – Orchestra Tuning Book – Play Audio


Having purchased the MP3 Shield, I starting formatting the MicroSD card with the appropriate tracks related to each chapter using ‘track00X’ to be appropriately read by the Arduino and shield. From the shield diagrams, I would only be able to use Analog Pins 0-5 and Digital Pins 0-1, 3-5, and D10. From this, I laid out each switch for each chapter from D0-1,3-5 and kept D10 to be used for the lamp and sensor input and output.


November 23
To create a more natural reading space, I went to purchase a chair and cushion from IKEA. I tried to pick a more laid back chair so that participants would be interested in sitting down rather than repurposing a studio chair. The supporting beam in the back of the chair allowed for a space to safely and discreetly place all my wiring that wouldn’t be seen. 

For the lamp design, initially I had intended to create free standing lamp, but after some thinking, I decided to have it incorporated inside the side table so there would be less clutter in the space. For the design of the side table, I wanted it to be minimal and be able to discreetly hide the light and all the wiring involved.


November 25

To conserve time and memory on the Arduino, all the audio clips for each chapter were compressed to 7 minutes maximum rather than playing the full one hour to two hour performances. I tested out the MP3 shield using external speakers instead of just headphones to check the sound quality.

November 26-27

Early iterations of the code for the Arduino and MP3 shield weren’t working as tracks refused to play with the if/else statements. Some revisions I made with the help of Nick Puckett was adding a statement to always have the track play the default tuning audio (track 7) and to simply change the track number on each switch rather than playing and stopping each track as it played.

In the early production stage of the side table, I cut a set of 7.5” by 10” sheets of ¼ inch plywood with a 4.5” diameter circle in the center and one with a 5” diameter circle to be able to house the LED stripe for the light. A hole was drilled on the bottom to allow for the wiring to be hidden away. To diffuse the light from the LED, a frosted acrylic sheet was cut to be used to securely hold in the lighting. I chose to have the LED light on the bottom side of the table so that the light would be more discreet and so readers wouldn’t have a bright light shining directly up at their faces while reading. 



Once the wiring was complete, I soldered the wiring onto a protoboard to securely hold everything. I used screw terminals for the wiring for the book switches, the chair pressure sensor and the side table light to be able to transport my work easily between locations and to easily troubleshoot wiring problems. From there I mounted a small board I made to the back supporting beam of the chair so the protoboard and Arduino could safely be placed inside.

November 28 – 29

To finish the side table, I put 4 sheets of ½ inch plywood and glued them together to make the legs stronger. For the wiring, I routered one side of the ½ in plywood from the inside so that the wiring could be hidden entirely inside the legs of the table and discreetly come out the leg.


With the side table complete, I brought all the items together to see how the space would look all together.


December 1

To solve the last few problems I was having with the Arduino and getting the chapter working, I updated the code from a if/else statement to an if statement followed by an “else if” for the remaining chapters. Another issue I was having was the difficulting uploading my code as I’d frequently get ths following error in the serial monitor:


I managed to solve an error I was getting for uploading onto the Arduino Uno. From an online forum, a user mentioned it may be due to a pin being wired into pin 0 (RX) which would cause this error, unplugging this pin during upload managed to solve the issue. Another issue I was having was the consistency of the switches turning on and off as participants may hold the book from different angles and might now apply enough pressure for the switches to properly high and low.  Originally the switches were all formatted with all states stated with only the one switch indicated as HIGH. Though due to the inconsistency in pressure, I removed certain states that were inconsistent.

Ch. 1 Switch Ch. 2 Switch Ch. 3 Switch Ch. 4 Switch Ch. 5 Switch

From there the final test was adding in the speakers again with the finished chair and table to make speakers would comfortably fit underneath the chair.


Project Context

Absolutely on Music explores the use of audio and tactile sensors to create a more immersive experience for inanimate objects in the home, rather than creating an augmented screen-based experience. This experiment is based on the philosophy of slow-tech, countering our need to develop tools that work more efficiently to allow use to do more, faster (Slow Living LDN, 2018). The set-up space is meant to re-evaluate our experience technology and potential of creating a multi-sensory and accessible home. 

This work is an example of Zero UI, which involves interacting with a device or application without the use of a touchscreen or visual graphic interface. Zero UI technology allows individuals to communicate with devices through natural means of communication such as voice, movements and glances (Inkbotdesign, 2019). Most Zero UI-based devices are related to the internet of things and are interconnected with a larger network. For this experiment, I wanted to explore creating a multi-sensory experience not requiring any networked communication or quantified data gathered and allow participants a more immersed and mindful experience with an inanimate object.

I choose Absolutely on Music by Murakami and Ozawa purposely for the references to auditory content that readers may be unfamiliar with, and how searching for said music may interrupt the reading experience instead of making it a seamless experience. This makes the content more accessible to a broader spectrum of readers. 

A book was chosen as the object of choice because of the constant discussion between reading from a digital screen versus a physical copy on paper. A physical book is a dumb object and allows a slow more leisurely experience rid of distractions compared to reading on a digital device. 

The book used is Absolutely on Music, a series of six conversations between the Japanese author Haruki Murakami and Japanese conductor Seiji Ozawa. Classical music, like fine art is generally difficult to access and deeply personal. Interest declines as individuals may perceive themselves distrust their own reactions as classical music may feel perceived to more sophisticated folk as mentioned in a New York Times Op-Ed (Hoffman, 2018). By playing the actual audio through speakers below the chair and having the music audible from headphones, any one can follow along the book without any prior musical knowledge of the works described.

Within the book, the author and conductor discuss both of their careers, from key performances in Ozawa’s career and Murakami’s passion for music, as musical pieces are always deeply integral in all his works from the Wind-up Bird Chronicles and opening with Rossini’s The Thieving Magpie or a Hayd concerto within the page of Murakami’s Kafka on the Shore (2002)

To elevate the sensory experience of the book, a set of switches were placed within the first five chapters (conversations) of the book. The audio described in each chapter is played with the use of a switch situated within each chapter of the book to provide context to the works Murakami and Ozawa are discussing.

Table of Contents of Absolutely on Music

1st Conversation – Mostly on the Beethoven 3rd piano concerto
Interlude 1 – On Manic Record Collectors

2nd – Brahms at Carnegie Hall
Interlude 2 – The relationship of writing music

3rd Conversation – What happened in the 1960s
Interlude 3 – Eugene Ormandy’s Baton

4th Conversation – On the music of Gustav Mahler
Interlude 4 – from the Chicago Blues to Shin’inchi mori

5th Conversation – the Joys of Opera
6th Conversation – “There’s no single way to teach, you make it up as you go along”

Based on the contents of the book, I pulled the main musical piece the two individuals spoke about into a tracklist that I would use for the interactive book. 

Timing (Chapter) Tracklist
Idle Mode Orchestral Tuning Audio
Chapter 1 Glenn Gould’s 1962 Brahm’s Piano Concerto in C Minor
Chapter 2 Seiji Ozawa’s Beethoven’s 9th Symphony
Chapter 3 Seiji Ozawa’s Rite of Spring (by Igor Stravinsky)
Chapter 4 Seiji Ozawa’s The Titan / Resurrection (by Gustav Mahler)
Chapter 5 Dieskau; Scotto;  Bergonzi’s Rigoletto
Chapter 6 (No Audio, No Single Musical Piece Focused)





Project Video

Github Code

You can view the github repository here

Circuit Diagram


*Within the actual wiring, the button switches are two piece of copper foil placed on opposite pages acting as the switches

*The schematic displayed is using a Sparkfun VS1053 though I used a geeetech VS1053, the available pins laid out are slightly different where as the Sparkfun version used in the diagram show D3 and D4 being used, they’re available on the geeetech MP3 shield.

Exhibition Reflection

For the Digital Futures Open Show, my piece was exhibited in the entrance of the Experience Media Lab. I set up the space with some plants and an additional light as props to make the area more comfortable. The space was quieter than the Graduate Gallery which worked out for the piece and allowed participants to sit down and experience the piece one at a time without having too much noise in the background. For the seat sensor, I kept the table light on so that participants could see the reading space clearly rather than being seat pressure activated.


My reflection on the experience would be from the participant aspect, where I noticed people were initially hesitant sitting down on the chair, not sure whether they were supposed to touch it, or that the space I created didn’t look like an art piece. I felt that the piece was successfully as it felt like a natural reading space, and don’t mind the confusion as the chair and book were designed in the context of being in a home rather than as an exhibition piece. 

There was some static from the speaker, but I also noticed that participants may have expected a much faster response from the book when the music changed, as many orchestral pieces had a natural slow build up, some participants flipped through the pages to experience the music change faster or couldn’t hear the musical piece.

While the book audio was designed to be for a single reader than can listen while reading rather than flipping through the pages, in hindsight, I’d probably revise the audio to begin likely in the middle of each musical piece when in an exhibition display so that participants could understand the concept faster.

Some helpful feedback I got on how to possibly improve the piece and learn more about invisible interfaces was reading Enchanted Objects: Design, Human Desire, and the Internet of Things by David L. Rose. Other feedback was also possibly exploring using a Maxuino which has more audio capabilities and support with Ableton live in case I wanted more control with my audio files and audio quality compared to using the MP3 shield.



Arduino Library vs1053 for SdFat. (n.d.). Retrieved November 29, 2019, from https://mpflaga.github.io/Arduino_Library-vs1053_for_SdFat/.

Hoffman, M. (2018, April 19). A Note to the Classically Insecure. Retrieved from https://www.nytimes.com/2018/04/18/opinion/classical-music-insecurity.html?rref=collection/sectioncollection/opinion&action=click&contentCollection=opinion®ion&module=stream_unit&version=latest&contentPlacement=4&pgtype=sectionfront.

Inkbotdesign. (2019, August 13). Zero UI: The End Of Screen-based Interfaces And What It Means For Businesses. Retrieved from https://inkbotdesign.com/zero-ui/.

Kwon, R. O. (2016, November 24). Absolutely on Music by Haruki Murakami review – in conversation with Seiji Ozawa. Retrieved from https://www.theguardian.com/books/2016/nov/24/absolutely-on-music-haruki-murakami-review-seiji-ozawa. 

LDN, S. L. (2019, May 25). Embracing Digital Detox and Slow Tech. Retrieved from https://www.slowlivingldn.com/lifestyle/slow-tech-digital-detox/. 

Murakami, H., & Ozawa, S. (2017). Absolutely on music conversations with Seiji Ozawa. London: Vintage. 

Shi, Y. (2015, February 7). Book Remote. Retrieved from https://www.youtube.com/watch?v=M1WrbADjfmM&feature=emb_title. 

Tench, B. (2019, February 11). Some Reflections on Slow Technology. Retrieved November 29, 2019, from https://www.becktench.com/blog/2019/2/11/some-reflections-on-slow-technology.

What Is My Purpose?

Project Title: What Is My Purpose?

By: Nilam Sari

Project Description:

This project is a 5x5x5 inches wooden cube with a 3D printed robotic arm. A hammer head shaped piece is attached at the end of its arm. The arm repeatedly hits the top part of its own body, a sheet of clear acrylic. This robotic piece appears to be self-harming itself.


I started this project by creating a timeline because I thought I should be more organize with this project to meet the tight deadlines.


I modeled my design on Rhino3D to help me visualized the arm that needs to be fabricated with the 3D printer.


At first I created the arm to hold a chisel, but after printing and testing it with a servo, the servos couldn’t handle the weight of the chizel so I compromised with an IKEA knife. That didn’t work either, so I compromised with this 3D printed hammer head that hold a 1in bolt.


At first I had trouble attaching the servo motors into the 3D printed parts, but Arsh suggested that I cut out the little servo extensions and fit it into the 3D printed parts instead of trying to make a hole that fits directly into the servo, and it worked perfectly (Thank you Arsh!).

Next is time to code the movements of the arm.

At first, I achieved the movements that I wanted, but it is too predictable, it feels too ““robotic””. So I randomized the movements within a range and got the results that I wanted.

Then I worked on the wooden body part. This went smoothly and pretty uneventful. The lack of tools in the maker lab makes the process take longer than it needed to, but I managed to do it according to my timeline.


When I was done with the wooden body, I attached the arm onto it.

When I did a test run, it was doing okay. The only problem is the chisel/knife problem I mentioned above. Then I installed the inserts for bolts at the corners of the top of the box to secure the 1/16 inch acrylic sheet.

At first I wanted this piece to be battery run, complete with an on/off button at the bottom of it. But when I tried using 9V battery it wasn’t strong enough to run the two servos. So I asked Kate for help and learned that it’s more about the current rather than the voltage. So I got four AA batteries on series and try to run it again. It was still not enough. Kate suggested to put another four AA batteries and attach it parallel to the other four. And it worked!

However, the battery couldn’t last long enough and the servo started getting weaker after a couple of minutes. It was a hard decision, but I had to compromise and use cable to power the robot from the wall outlet.

For the show, I originally wanted to add delay() so the servo motors get breaks in between and don’t overheat. But when I added delays the motor doesn’t run the same way it did without the delay.

Code: https://github.com/nilampwns/experiment5





Circuit Diagrams: 


Project Context:

We are constantly surrounded by technology that do tasks for us. When machines do not carry out their tasks properly, they are deemed broken. Can we co-exist with machines that are not designed to serve us, humans? What kind of emotion is evoked by seeing a machine hitting itself over and over? Are we projecting our feelings onto non-living objects? These were the questions that lingered in my mind when I decided to create this robot.

I believe that robots have the potential to teach us, humans, how to empathize again. That is basically the belief that I have that drove me into graduate school and life in general. This piece that I created for experiment 5 is in some way a part of my long term research in life. Can robots teach us, humans, to empathize with a non-living being, and ultimately, with each other?

There has been multiple research that ask the participants about the possibility of the robots getting hurt or where participants are asked to hurt the robot. As Kate Darling (2015) wrote on her research report paper, “Subjects were asked how sorry they felt for the individual robots and were asked to chose which robot they would save in an earthquake. Based on the subjects preferring the humanoid robots over the mechanical robots, Riek et al. postulate that anthropomorphism causes empathy towards robotic objects.”

People tend to empathize more with robots that look like them. I want to push how far can I remove my piece from anthropomorphization as much as I can, and push it even further by making a robot that its whole purpose is to hit itself. That’s why I created the body to look like a wooden cube, with visible microcontrollers, the only thing that makes it looks a bit anthropomorphized is the robotic arm.

The act of self-harming is jarring. It’s emotional. It’s a sensitive topic. But what does self harming mean to a robot that cannot feel pain? It does not have sensors that detect pain.

Not so much about self-harming, but Max Dean’s piece, “Robotic Chair” (2008) is a robotic chair that explodes it self into multiple parts, then it slowly search for its missing piece and putting itself back together again autonomously. The viewers’ reactions were positive emotions. “As stated further on the Robotic Chair website, the chair has been shown to elicit “ empathy, compassion and hope ” in its viewers.” (Langill, 2013).

I acknowledge that my approach is very philosophical. The title of the work itself is “What Is My Purpose?” a question that even us humans have not found the answer to yet. I hope to make people think that it’s okay for machines to not have a purpose, to not serve us, still exist around us, and for us to still empathize with them. That way maybe humanity could learn to empathize more with each other.

Exhibition Reflection:

The exhibition was lovely. I got to talk to so many people about my research, receive feedback and reactions, or just simply chat with them. Unfortunately, the servos of my piece got overheated and busted 30 minutes into the show, I thought it was the arduino but I unscrewed the piece to hit the reset button but it still didn’t change anything. I tried to let it cool down for a couple of minutes but it also didn’t work. I have a video of it running so I was showing the videos to people who were interested in seeing it. Thankfully it was running when it was the busiest.

People told me they liked my work. I asked them what they like about it, and a couple of them said that they think it’s funny and silly. Some said they can feel the frustration of the robot. Some felt so empathetic that they felt bad watching the robot hitting itself over and over. One person even said “idiot robot” at it. It was a mixed bag of reactions but most people enjoyed the piece.


Darling, Kate.Empathic concern and the effect of stories in human-robot interaction”. with P. Nandy and C. Breazeal. proceedings of the IEEE international workshop on robot and human communication (roman). 2015. 

Dean, Max. “Robotic Chair”. 2008.

Langill, Caroline Seck. “The Living Effect: Autonomous Behavior in Early Electronic Media Art”. Relive: Media Art Histories. MIT Press. 2013.

Eternal Forms

Experiment 5: Proposal


Catherine Reyto, Jun Li, Rittika Basu, Sananda Dutta, Jevonne Peters

Project Description

“Eternal Forms” is an interactive artwork incorporating geometric elements in motion. The forms accelerate their rotating speeds when a user interacts from ranging proximities. The construction of the elements is highly precise along with accurate measurements to generate an optical illusion of constant collision. The framework establishes from a circular grid system, which is designed by several interlinking ellipses running across a common circumference of the centre ellipse. 

Parts/materials/ technology list 

  • Servo
  • Distance sensor
  • Pulley
  • Acrylic
  • Wood
  • Arduino – distance sensor and servo motor based coding
  • Laser Cutting

Work Plan

Base Circular Grid – Sets the basis of the design constructions.
Design Explorations
  • 24th Nov (Sun): Pattern exploration and design created based on the finalized patterns
  • 25th Nov (Mon): Design a network of connected illusions using the finalized patterns. Also, try out prototype laser cuts of the final pieces. Check out the weight and scaling optimizations.
  • 26th Nov (Tue): Work on the code – Arduino side. Also, try the servo functioning with the prototype model.
  • 27th Nov (Wed): Combine the servo and the sensor part of the experiment and check code functioning.
  • 28th Nov (Thu):  Create the final design laser cuts with the materials finalized – acrylic or wood or both. Additionally, creating the mounting piece that needs to go up on the wall.
  • 29th Nov (Fri): Iterate and work on creating multiple kinetic sculptures and make them interactive. Also, work on the display set-up of the installation.
  • 30th Nov (Sat): Trouble-shooting
  • 1st Dec (Sun): Trouble-shooting
  • 2nd Dec (Mon): Trouble-shooting
  • 3rd Dec (Tue): Final Presentation

Physical installation details

They will be multiple interactive artworks mounted on walls where the user can observe the changing rotating speed based on distance proximities. These artworks will be animated by Servo Motors.

Resource List

10 Nails and hammer – For mounting the artwork.

Extension Cord

Power Plug

Wall space of 5ft x 3ft to mount the piece.


  1. Kvande, Ryan. “Sculpture 40″ – Award Winning Optical Illusion Sculptures”. Spunwheel, 2019, https://www.spunwheel.com/40-kinetic-sculptures1.html.
  2. Whitaker, Lyman. “Whitaker Studio”. Whitakerstudio.Com, 2019, https://www.whitakerstudio.com/.

Experiment 5 — Leave A Window Open For Me

1.Project title: Leave A Window Open For Me

2.Project description: 

For this project, I’m aiming to create an installation that has space within a box build with mirror acrylic sheets and wood. It is meant to be filled with mirrors to form a room that has infinite reflections. It is meant to reflect my own experience in new york when I was going through an emotional break down for a long period of time. I was suffering from insomnia and depression that I had no motivation to do anything, the window was the only thing that I stare at the most, from day to night, from rain to shine.

I managed to step out of the emotional trap eventually, but looking out of the window seemed to become one of my daily routines. Except for this time, things have changed.

There will be a 3D printed bed placed in the middle of the installation to represent my room and my mental status. One side of the board would have a cutout and play the role of the window, which will have the led matrix panel placed behind it.

The audiences are expected to observe the installation from the peek hole at the front board.

3.Parts / materials / technology list:

LED matrix panel, Arduino UNO, acrylic sheets with mirror backing, glue gun, 3D printer, laser cutting machine, transformer adapter, tool power cord, etc.

4.Work plan: 

Nov 21-22: Proposal & Discussion

Nov 23-24: Coding & Researching

Nov 25-26: Purchasing materials & Building small prototype & test

Nov 27-28: Coding & 3D printing & Laser cutting

Nov 29- Dec 3: Debugging & Assembly

Dec 4: Open show

5.Physical installation details: 

The small room in the graduate gallery which is always used to play films because preferably a darker space, a pedestal that has the installation on will be placed against the wall.


Initial sketches:

img_3487 img_3488 img_3489 img_3490 img_3491 img_3492





Project Title – CityBLOCK

Team Member – Rajat Kumar

Project Description
CityBLOCK is a modular city builder experience where the user controls the city building blocks with designed cubes to build their desirable dream city by rearranging the cubes.

What makes Toronto so unique? Being the largest city, most diverse, It’s home to a dynamic mix of tourist attractions, from museums and galleries to the world-famous CN Tower and, just offshore, Toronto Islands. And just a short drive away is Niagara Falls.
This project will let you build your version of Toronto city.

Working flow


  • reacTIVision 1.5.1
  • TUIO Client – Processing
  • OV2710 – Camera
  • Projector – Which supports the short-throw lens with a throw ratio of 0.6 and below.
  • Infrared LED -850nm Illuminator


  • Wooden/Paper blocks.
  • Table 4.5ft*3.5ft
  • Projector Mount

Work Plan

22nd – 23rd -ideation finalizing the concept
24th – 27th -Coding, configurating reacTIVision with processing.
27th – 30th-Testing and making ready for exhibition.
1st – 2nd-Installation/ setup
3rd – 4th-Exhibitions

Space Required

Dark space is required and space should be big enough to accommodate the table.



Experiment 5 – Unexpressed?


Project Title
Unexpressed? (Working Title)
An interactive blank canvas.

Project by Jignesh Gharat

Project Description

On our visit to art galleries and museums, we come across abstract as well as minimal art and try to appreciate and interpret the meaning behind it through gestures, expressions, emotions evoking the unconscious mind with movement and color.

Abstract expression finds its roots in ‘intuition’ (of the artist) and ‘freedom’ (for the artist as well as for the viewer). The artist uses their imagination to look beyond what we can physically see and translate intangible emotions onto the canvas. The audience then tries to connect to the artist’s intention and free their mind of visual restrictions.

Minimalism is getting away from the individual expression it doesn’t always have a connection with the artist.

What is a blank canvas?
An empty canvas with some texture, lines, pattern, and tints of white. Or an expression emptiness, curiosity stories, could be anything.

What if a blank canvas could try to express its self instead of viewers projecting their interpretations, emotions, beliefs, and stories. 


  • Mackbook pro
  • 2 Arduino UNO/ Nano
  • 2/3 Servo motors
  • VL53L0X laser distance sensors 
  • LED Bulb


  • Stretch fabric, Lycra Fabric
  • Face mask, hand model
  • Old/vintage wooden canvas frame 2’ x 3’
  • 4 Plywood 6” x 32”

Work plan

22nd – 23rd _Material procurement, storytelling, and ideation
24th – 27th _Code- Debugging, Calibrating distance sensor, Mockup, Testing
27th – 30th_Iterating, Fabrication, and Assembling
1st – 2nd_Installation/ setup
3rd – 4th_Exhibitions

Physical installation details

As the viewer comes in the range of the distance sensor the blank white canvas will start showing an embossed human figure popping out of the canvas surface.


Exhibition Space

Preferably a silent art gallery space to mount a heavy canvas on the wall. Spotlight on the canvas.

Interactive Canvas

  1. Project Title

Interactive Canvas (Working title) by Arsalan Akhtar


  1. Project Description


Ambient advertising is key to win consumers today which encourages discovery of various story telling tools. The “walls” around us are an empty canvas that could tell a lot of interactive stories. Thus, I would like to make an interactive wall that tells a story through sound or display when in close proximity. The subject of the story is about breach of digital privacy and how we in this digital age have given permissions to mobile apps that we use daily.


  1. Parts, Materials and Technology


  • Arduino micro
  • Resistors
  • Conductive Material
  • Projector
  • Piece of wood
  • Processing and Arduino Software
  • Photoshop for illustration


  1. Work Plan


  • Nov 20-22 : Low fidelity prototyping with arduino, conductive material and resistors
  • Nov 23-24: Work on processing to discover storytelling drawing
  • Nov 25-26: Procure wood and engrave storytelling assets
  • Nov 27-28: debugging of hardware and software
  • Nov 29-Dec1: debugging of physical installation
  • Dec 2: test run
  • Dec 3-4: Display


  1. Physical Installation

A rectangle piece of wood or paper (around 6ftx6ft) attached or hung against the wall with story engraved on it and facing a projector. The visitors could interact with features on the wall and learn about the story. Thus, a flat wall such the DF gallery could be a great place.

  1. Resource list 


  • a bracket to hold the piece of canvas against the wall.


Thank you







Experiment 5 Proposal – OK to Touch!?

Project Title
OK to Touch !?

Team members
Priya Bandodkar & Manisha Laroia

OK to Touch!? is an interactive experience to make the hidden tracking technology i.e. Fingerprinting, visible to the users through interactions with everyday desk objects. The concept uses these tangible interactions in physical objects to convey how users’ private data is being tracked without their consent in the near future.

Project Description
Peering into the digital future, OK to Touch!? is an interactive experience to make the hidden tracking technology Device Fingerprinting, visible to the users, currently used for browsers and apps and soon moving to smart devices. Through simple interactions with everyday desk objects, we will record the physical touch-points IRL, tag them and project them on screen in the backdrop to create an apparent visual of digital fingerprinting data that we as users are sharing unknowingly with each interaction we make.

Physical Installation
The setup is table top display with desk objects, with a large screen or a project wall in the backdrop. As the users interact with the objects, a visual representation of the Fingerprinting data is added on the backdrop to create a visual map and indication to the users that a simple touch or click on a smart object can be tracked and be used to profile them. It is an attempt to create an awareness about the privacy concerns associated with the Internet in today’s knowledge economy.


Setup space:
4 ½  feet x 2 ½ feet table top, set along the wall. The wall serves as the projection screen or a large TV screen could be used.

Parts & Materials
Desk objects will be selected from the following list of items (subject to prototyping results): Table from the DF studio and a chair, a book, an ink stamp, iPad or mobile phone, headphones, an alarm clock, a calculator, a box. These will be either be sourced in their original form or fabricated as indicated below:

Sourced in original form Fabricated
Book Alarm clock (laser-cut wood)
iPad or mobile phone Box (laser-cut wood)

Softwares and Hardware to be used:
p5.js, Processing and Arduino, switches, digitally fabricated objects, DIY switches.

Object Interaction Sensor (Input) Output
Book with a bookmark or pen set to a page Open & close DIY switch Graphic circle or fingerprint icon projected on the wall behind or appears (fade in) on the screen behind the display with supporting fingerprinting text, and stays there the whole time. //particle.system



“Book/5.0 (OS NT 10.0; v64; rv:70.0) User/20177101;Toronto; IP”

Ink Stamp with a paper next to it with a few stamp patterns on it Stamp on paper Velostat/Pressure sensor [Graphic/Icon]


“Stamp/5.0 (OS NT 10.0; v64; rv:70.0) User/20177101;Toronto; IP”

Mobile phone or iPad with text and icons to click

(Privacy Policy)

Multitouch function P5.js multitouch to note the touch on screen [Graphic/Icon]


“Stamp/5.0 (OS NT 10.0; v64; rv:70.0) User/20177101;Toronto; IP”

Headphones hung on a metal stand Remove them from the stand to disconnect the switch and listen to a note on fingerprinting/music DIY switch [Graphic/Icon]


“Headphones/5.0 (OS NT 10.0; v64; rv:70.0) User/20177101;Toronto; IP”

Alarm Clock Open & close DIY switch [Graphic/Icon]


“Alarm Clock/5.0 (OS NT 10.0; v64; rv:70.0) User/20177101;Toronto; IP”

Calculator with a page displaying two numbers to be added Press calculator buttons Velostat/Pressure sensor [Graphic/Icon]


“Calculator/5.0 (OS NT 10.0; v64; rv:70.0) User/20177101;Toronto; IP”



Dates Activities
23rd November – 25th November Material procurement and Quick Prototyping to select the final 4 Objects
26th November – 28th November Writing the code and Digital Fabrication
28th November – 30th November Testing and Bug-Fixing
1st December to 2nd December Installation and Final touches
3rd December to 4th December Presentation

Briz, Nick. “This Is Your Digital Fingerprint.” Internet Citizen, 26 July 2018, https://blog.mozilla.org/internetcitizen/2018/07/26/this-is-your-digital-fingerprint/.

Szymielewicz, Katarzyna, and Bill Budington. “The GDPR and Browser Fingerprinting: How It Changes the Game for the Sneakiest Web Trackers.” Electronic Frontier Foundation, 21 June 2018, https://www.eff.org/deeplinks/2018/06/gdpr-and-browser-fingerprinting-how-it-changes-game-sneakiest-web-trackers.

“Talk to Me: Immaterials: Ghost in the Field.” MoMA, https://www.moma.org/interactives/exhibitions/2011/talktome/objects/145463/.

Chen, Brian X. “’Fingerprinting’ to Track Us Online Is on the Rise. Here’s What to Do.” The New York Times, The New York Times, 3 July 2019, https://www.nytimes.com/2019/07/03/technology/personaltech/fingerprinting-track-devices-what-to-do.html.


Experiment 5 – proposal

by Nadine Valcin


Project description
Much of my work deals with memory and forgotten histories. I am constantly searching for new ways to portray the invisible past that haunts the present. (Un)seen is a video installation about presence/absence that uses proxemics to trigger video and sound. It recreates a ghostly presence appearing on a screen whose voice constantly beckons the viewer to get closer, but whose image recedes into the frame as the viewer tries to engage with it. Ultimately, the viewer is invited to touch the cloth it is projected on, but if they do, the ghost completely disappears from view, leaving an empty black screen.

With permission, I will be using unused footage from a previous project comprised of closeups of a Black woman on a black background and will be recording and mixing a new soudtrack.

Parts / materials / technology list
MacBook Pro
Arduino Nano
Distance sensors (2 or 3) HC-SR04 or KS102 1cm-8M Ultrasonic Sensors
King size bedsheet, hanging from rod
2 speakers (or 4?)

Work plan
22.11.19-24.11.19     Edit 3 video loops
24.11.19-25.11.19     Write ghost dialogue and research sound
26.11.19-27.11.19    Record and edit sound
22.11.19-27.11.19     Program distance sensors and interaction
27.11.19                       Mount bedsheet on rod
28.11.19-02.12.19    Testing and de-bugging
03.12.19-04.12.19    Presentation

Physical installation details
The ideal space for the installation would be Room 118 on the ground floor.
With permission, I will be using footage shot for another project comprised of closeups of a Black women on a Black background. The ideal would be for the image to be projected from the rear onto the sheet. This would require a dark space and enough space behind and in front of the sheet. The mount for the sheet will be kept deliberately light. Metal wire can be used to hang the rod holding the sheet from the ceiling, but would potentially require discrete hooks screwed or attached to the ceiling.

Set-up Option A


Set-up Option B

unseen-setup-option-b1Resource list
Hand drill and other tools TBA
2 (or 4?) speakers
2 pedestals for the sensors (?)
Cables for the speakers
Power bar and electrical extension cords

– Can I have 4 speakers and have the play different sounds in pairs? I.e. the speakers behind the screen wouldn’t play the same sound as the speakers in front of the screen
– Do I actually need 3 distance sensors – 1 behind the screen for the functions triggered by people touching the screen and two mounted (possibly on pedestals) slightly in front of the screen at each side?
– Is it possible to hang things from the ceiling?
– Would a motion sensor also be useful to activate the installation when someone comes into the room?

Experiment 5 Proposal

Zero UI (Working Title)

For Experiment 5, I’d like to expand on tangible interfaces and explore the use of Zero UI (invisible user interfaces) and having technology fully incorporated within a room (or household) with the use of pressure sensitive furniture and sensors with auditory feedback to elevate regular objects (a book) to create a more immersive experiences instead depending on screen based experiences. This experiment is an exploration in creating a multi-sensory reading experience with content catered towards the book’s contents.

The experiment would involve the use of a pressure sensor chair that lights up a nearby lamp when the participant sits down. The pressure sensor may be installed physically on the chair or hidden away with the design of a cushion or lining. The participant can pick up the book and read or flip through the book and hear the music referred in the book playing from a speaker hidden away (possible below or behind the chair). The audio would be mapped depending on what section of the book the participant is on.


The book I’d like to use is still undecided but one with many musical references such as Haruki Murakami’s book, The Wind Up Bird Chronicle, where as the book begins with the protagonist listening to Rossini’s the Thieving Magpie and refers to many other classical musicians. Another possible book could be J.R.R. Tolkien’s The Hobbit with the movie franchise’s music by Howard Shore playing instead.

Project Main Components and Parts

  1. Arduino Nano
  2. Flex Sensor
  3. Pressure Sensor
  4. MP3 Shield (?)
  5. External Speakers
  6. Lightbulb and Wiring (Lamp)

Additional Components and Parts

  1. Chair (Supporting Prop)
  2. Fabric/Cushion (To Hide/Place Sensor)
  3. Book (Prop)
  4. Mat Rug (Prop To Hide Cables)

Workback Schedule

Fri, Nov 22 –  Proposal Presentation
Sat, Nov 23 –  Coding + Gathering Digital Assets + Building Lo-Fi Breadboard Prototype
Sun, Nov 24 – Coding + Gathering Digital Assets + Building Lo-Fi Breadboard Prototype
Mon, Nov 25 –  Coding + Creatron for Final Components
Tues, Nov 26 –  Presenting Progress of Lo-Fi Breadboard Prototype + Revisions
Wed, Nov 27 – Prop Purchasing
Thurs, Nov 28 – Laser Cutting Components and Coding
Friday, Nov 29 – Troubleshooting / Bug Fixes
Sat, Nov 30 – Troubleshooting / Bug Fixes
Sun, Dec 1 –  Troubleshooting / Bug Fixes
Mon, Dec 2 – Troubleshooting / Bug Fixes
Tues, Dec 3 – Final Critique
Wed, Dec 4 – Open Show

Physical Installation

I’d like to ideally place the set up in the corner of a room and with dimmer lighting so the lighting from the lamp is more visible when it turns on. Supporting objects within the set up would be the chair participants can sit on with the sensor attached.



Resource List

  1. Chair and Side table
  2. Will need extension cords for power
  3. External speakers

Info for Open Show

Preferably displayed in the Grad Gallery room. I will just need an electrical outlet nearby or extension cord. We will need to book external speakers from AV.