Experiment 5: Katlin Walsh

Project Description 

While interactive media content displayed within galleries has been updated within the last 5-10 years, presentation formats for tradeshows have not. Digi-Cart brings an adaptive presentation style to the classic concept of a tool cart. Robust building materials and pegboard construction allows corporations to adapt their layout and presentation style to reflect their current corporate event. 

Digi-Cart features a basic controller layout which can be overlayed with a company’s vinyl poster cutout to create an interactive presentation that can be facilitated by an expert or self guided. Corporations are encouraged to update their digital materials & create animated graphics to capture audience attention.  Continue reading “Digi-Cart 2.0”

Ocular
Animated Robotics – Interactive Installation

By Jignesh Gharat

Project Description

An animated robotics motion with a life-like spectrum bringing emotive behaviour into the physical dimension. An object with motion having living qualities such as sentiments, emotions, and awareness that reveals complex inner state, expressions and behavioural patterns.

He is excited to peak outside the box and explore around the surroundings but as soon as he sees a person nearby he panics and hides back in the box as if he is shy or feels unsafe. He doesn’t like attention but enjoys staring at others.

Design Process & Aesthetics

It’s an attempted to create an artificial personality. I wanted to work on an installation which encourages participation rather than spectatorship. So wanted to work on a physical installation that lets people experience familiar objects and interactions in refreshingly humorous ways.

The explorations started with exploring objects and living organisms that can be used as a part of the installation. Where I can implement curious interactions and a moment of surprise and unpredictability. Gophers, Crabs, Snails, Robots were few to mentions. I finally ended up finalising periscope as an object that can have behaviour and emotions and a perfect object to play hide and seek with the visitors.

What is Periscope?
An apparatus consisting of a tube attached to a set of mirrors or prisms, by which an observer (typically in a submerged submarine or behind a high obstacle) can see things that are otherwise out of sight. I started thinking of ideas, object behaviors as well as setup of the installation to come up with engaging and playful interactions.

Step 1 – Designing the bot

The model was developed in Rhino keeping in mind the moving parts that is rotating the head and the spin which is lifted from the bottom.

Step 2 – The mechanism.

Testing the motors stepper (28BYJ-48 5V 4 Phase DC Gear Stepper Motor + ULN2003 Driver Board UNO MEGA R3-Head ) and Servo (SG90 Micro Servo Motor – Head and 3001 HB analog servo – Arm ). Stepper motors the RPM was too less to give that quality and reflex to the bot so decided to go with the servo motor which had better torque.

The 3d printed body was too heavy for the Analog servo motor to lift so finally decided to use paper to model the body and reduce the weight and load on the motor. Surface development was done using Auto CAD software for making 3 to 4 different options based on the mechanism and aesthetics of which I decided to work on the design shown in the below image. The arm was laser cut in acrylic and two options were made to reduce the friction between paper and the lifting arm contact surface.

 

How does it Work?

A micro motor controls the oscillation of the head. A Distance sensor at the front of the wooden  controls the motor 2. When the visitor is in the range of set threshold of the distance sensor  that pulls the lever arm down and the motor 1 (Head)stops rotating.

The installations is minimal and simple with just two materials wood and paper that gives a clean finish to the piece. The head is like a focus with a black sheet and white border as if the sensors are installed inside the head that controls the oscillation. In my further explorations and practice I will make sure that the sensor are not directly visible to the visitors as this some time leads to visitors interacting with the sensor and not actually experiencing the installation.

 

Code :  https://github.com/jigneshgharat/Ocular

 

Project Context

Opto-Isolator (2007), Interactive Eye Robot  -Opto-Isolator (2007: Golan Levin with Greg Baltus) inverts the condition of spectatorship by exploring the questions: “What if artworks could know how we were looking at them? And, given this knowledge, how might they respond to us?” The sculpture presents a solitary mechatronic blinking eye, at human scale, which responds to the gaze of visitors with a variety of psychosocial eye-contact behaviors that are at once familiar and unnerving.

The Illusion of Life in Robotic Characters – Principles of Animation Applied to Robot Animation – Robot animation is a new form of animation that has significant impact on human-robot interaction.
This video relates to and extends a paper we have published at the ACM/IEEE Human-Robot Interaction Conference

Exhibition Reflection

The observations were made and recorded at Open Show Graduate Gallery and Centre for Emerging Artists & Designers which provided me with some good insights and learnings for necessary changes to be made developing the concept further. It was fun to see people interacting and playing around a nonliving object as if it had emotions, feelings and  the way Ocular reacted to the viewer’s actions. The interactions were supposed to be only from the front but as the wall card was placed at the side of the installation placed on a high plinth, people tend to read and start interacting from the side and did not really experience the installation as expected but most of them figured out the working as they read the wall card.
Some interesting comments from the visitors:

• The Installations is making everyone Smile.
• I have 3 cats and all 3 cats will go crazy if they see this.
• She said all the guys react the same way Ocular did so she don’t want to go near him.
• What are the next steps?
• What was the inspiration?
• How does this work?
• Why does he ignore me?

A visitor (Steventon, Mike) referred to Norman White’s interactive robotic project, The Helpless Robot.

Observations and visitors reactions

Reference

• Animated robots and robotic animation
  https://ieeexplore.ieee.org/document/525736
• WALL-E
  https://en.wikipedia.org/wiki/WALL-E
• Behavior-based robotics
  https://en.wikipedia.org/wiki/Behavior-based_robotics
• Simple robots, complex behaviors: A control systems perspective on Braitenberg Vehicles
  https://robohub.org/simple-robots-complex-behaviors-a-control-systems-perspective-on-braitenberg-vehicles/
• Linear Servo Actuators 3D print
  https://www.thingiverse.com/thing:3170748

By Jessie, Liam and Masha

Project Description: 

Antimaterial is an interactive installation that explores abiogenesis and kinesis to articulate a relationship between the human and nonhuman world. Just as we are able to affect our surroundings, the ability of magnetic materials to move matter indicated the presence of life to pre-Socratic philosophers. The rocks beneath our feet were not only the essential catalysts for life, but that microbial life helped give birth to minerals we know and depend on today. In Antimaterial, these concepts are explored as primitive life emerges within the topography of the material in response to human proximity, demonstrating a connection between animate and inanimate matter.

Project Context:

Drawn the freedom and possibility of exploration, from the start of the project,we all agreed to use creative material to create a tangible and experimental interactive art piece. With some of the team member’s previous experience with using water in their personal projects, we thought to use water as it’s a very versatile medium due to it’s free flowing form. Using tools such as a solenoid to tap on water to create ripples, or use speakers in waters to visualize sound waves with water were some of the many initial ideas we thought of. However, with water being a difficult medium to control the form of, we were worried that the final piece would end up looking like a jacuzzi, which led us to finetune to our ideas further. 

After several sessions of brainstorming, we came up with the idea to mix water with some magnetic material to magnetize it which would also give us more control over its form. During our research, we found quite a few interesting projects that have guided us in new directions of exploration. Sachiko Kodama is a Japanese artist that dedicated the majority of her art practice to using ferrofluid and kinetic sculptures to making installations. By combining ferrofluid which can be a messy medium at times with kinetic sculpture which is inherently neatly structured, her works usually create order and harmony amongst chaos and come off as intriguing as well as energetic.  

Sachiko Kodama Studio Website

Inspired by this kind of harmony generated by the juxtaposition of structure and chaos by the use of ferrofluid and kinetic sculpture, we wished to demonstrate it with ferrofluid in our practice for this project too. We purchased a bag of iron(III) oxide and started playing with it by mixing it with different different types of solvents including motor oil and washing up liquid while doing more research how it’s been incorporated to other artists’ works of art. 

 

 

One use of iron(III) oxide that drew our attention was to first mix it with oil and then encase the oily mixture in a container filled with water. For example, the project Ferrolic by Zelf Koelman utilizes the unmixable nature of oil and water and creates a versatile movement of ferrofluid within water with the use of magnets behind the display, and creates different patterns with different modes of operation.

 

We wanted the magnetic material to generate different patterns when visitors interact with it. The mechanism behind the project SnOil inspired us to make an array of electro magnets behind the magnetic material, and different patterns will be programmed to be generated through the activations of electro magnets on certain positions during visitors’ interactions. 

 

However, having talked to Kate and Nick during the proposal meeting, they shared their concerns about whether we would be able to manage an array of more than 10 electro magnets within 2 weeks’ of production period, as well as the cleanliness of the piece on display since ferrofluid tends to get messy. Having reevaluated these essential suggestions, we decided to abort the idea of making ferrofluid and only use iron(III) oxide powder as it generates a fur-like texture when magnetized and still creates a very aesthetically intriguing look. For example, the following project uses furry pom poms as part of the interaction piece to produce a very engaging experience.

 

We were also inspired by the mechanism behind this pom pom piece as it uses servos and turns a circular movement into a linear movement which we later used in our project to simply the movements of magnets. 

 

Process:

This project has undergone a series of experimentations, evaluations and adaptations. Having decided to explore creative material for this project, we chose to use magnets to experiment with. We first started making electro-magnets by wrapping conductive metal wires around a coil, and hoped to create patterns with magnetic material through the activation of an array of electro-magnets. The patterns will change and react to the on and off of the eletro-magnets. 

 

However, the magnetism of the electro-magnet is not strong enough and generates a lot of heat after a longer period of time when it’s activated. We worry that they would cause safety issues during the open show because of overheating and also the visual effect might not be desirable because of its weak magnetism. Eventually, we decided to use actual magnets instead. This change has led us to re-think the reactions of the magnets as they will not have the ability to be turned on and off, and will always have magnetism.

Therefore, we decided instead of making magnets become activated and deactivated, we could change their position through movement actuated by servos motors through a slider. 

It turns out the micro servo motors we have are too weak. One slider weighs around 6kg and the servo we have is only able to move weights up to 1.5 kg. We try to come up with different solutions to solve it. First, we changed the micro servo to a more powerful servo FS5106R which takes up to 6kg in weight. Second, we put the slider vertically instead of horizontally and use gravity to our advantage, and attach the slider to the servo through a piece of fishing wire.

 

 

 

However, it seems the slider is too heavy still, especially now that all the weight is put on the servo motor. The fishing wire also becomes less durable after the serco runs for a longer period of time. Finally we landed on the idea of laying the slider horizontally and using gears to achive the linear movement of magnets through the rotation of servos.

 

Having done that, the mechanism still seems inadequate because the full rotation servo doesn’t seem to have enough torque to move the gear plus the slider. We again had to change to an even more powerful motor, which led us to using stepper motors. We also favor the stepper motors over servos since they are simpler to code as it counts steps instead of time. 

 

Now that all the underlying mechanism is working. It’s time for us to put it all together. The production process begins. We have prototyped what the piece would look like in a digital 3D software and made sure that it would all come together in reality.

We started building the box with the help and guidance of a professional constructor friend. 

 

Finally, this is what our piece looks like.

 

Code and Circuit:

https://github.com/jessiez0810/exp5/blob/master/nptest_2_with_sensor.ino

Reflection:

During the exhibition, the piece successfully achieved creating a sense of mystery and made many visitors to the open show wonder what it was and how we did it. During the critique session, we were suggested by Nick to get gloves to put besides the piece so people would know that they could touch it and maximize the level of interaction. However, having discussed it within our group, we agreed the gloves might temper with the eeriness the piece gave off and wouldn’t fit with the underlying tone. The gloves would also create a barrier for people to feel the texture of the powder, which is an essential part of this interaction experience – to find out what this alien material is through all means of explorations. 

To solve this issue, we decided to keep paper towels with us. After people touched it, we would hand off paper towels to them to wipe the powder off. However, if they didn’t ask us, we wouldn’t say anything about touching. As was expected, many people were tempted to touch the magnetic power on display and we allowed them to do it if they had asked. What’s interesting is that the seemingly dirty powder didn’t hinder people’s curiosity and a lot of people dig their hands into the powder, which seems to be a validation of success in mission of the mystery this piece tries to achieve. This project has taken us a lot of time in all stages of the process. However, it was truly rewarding to see it come together in the end after many trials and errors. 

References:

AccelStepper Class Reference. (n.d.). Retrieved December 8, 2019, from https://www.airspayce.com/mikem/arduino/AccelStepper/classAccelStepper.html.

Brownlee, J. (2018, July 10). MIT Invents A Shapeshifting Display You Can Reach Through And Touch. Retrieved December 9, 2019, from https://www.fastcompany.com/3021522/mit-invents-a-shapeshifting-display-you-can-reach-through-and-touch.

Ferrolic by Zelf Koelman. (n.d.). Retrieved December 9, 2019, from http://www.ferrolic.com/.

Kudama, S. Protrude, Flow (2018) Retrieved December 8, 2019, from http://sachikokodama.com/works/.

PomPom Mirror by Daniel Rozin. (n.d.). Retrieved December 9, 2019, from https://bitforms.art/archives/rozin-2015/pompom-mirror.

SnOil – A Physical 3D Display Based on Ferrofluid. (2009, September 20). Retrieved December 9, 2019, from https://www.youtube.com/watch?v=dXuWGthXKL8.

State Machine. (n.d.). Retrieved December 8, 2019, from http://www.thebox.myzen.co.uk/Tutorial/State_Machine.html.

 

 

 

Project title: Leave a window open for me

By: Neo NuoChen 

Project description This piece is meant to create a space which expands infinitely within a box. It is a reflection of myself and my personal experience of fighting with insomnia and depression. It welcomes the audience to take a peek inside the story and share the feelings through the “window”. 

visuals (photographs & screenshots):

• (3-5) work-in-progress:

• (3-5) portfolio-worthy final images

 

edited video of the interaction: https://vimeo.com/378123696

 

code：https://github.com/NeonChip/NeoC/blob/master/exp5

circuit diagrams https://www.circuito.io/app?components=512,10190,11021,763365

Please note that the servo input is supposed to go to pin 9 and the LED panel input should go to pin 12. There was no breadboard being used for my project, and I was using a 16×16 LED matrix panel.

Project Context (500-1000 words)

I wanted to create this personal space within a box for the public because it is my story and I’m willing to share it with everyone. It is designed to be viewed through the “window” and one person per time so that the audience could have a more intimate and immersive experience.

This is a story about me going through insomnia and depression for several months when I was in New York. When I couldn’t sleep, the window was one of the things that I stared at the most, and I kept thinking about all the things that I shouldn’t have done and could have done. The stress was so real that it constantly haunted me in my head, the slightest light or noise became utterly unbearable. I could feel the bed spinning like a disc, or was it me spinning in my head? It didn’t matter day or night, they were the same, the thoughts kept me awake, the music that played was drowning me.

But I don’t want to drag the audience into a bad mood, it is not what my intention is, ever since I stepped out that situation, I’m proud of myself for doing that and I want to let people know that it is not undefeatable and you are not alone.

The initial idea of building an expanding space within a box was to see how the contrast between the outside and inside could be, the surprise element is always a twist that I want to collaborate in my design. I’ve been to multiple Yayoi Kusama’s exhibitions, and each time I’m amazed by how the glasses are forming these rooms. I was blessed to see Samara Golden’s work, The Meat Grinder’s Iron Clothes, at the Whitney Museum of American Art in New York. It was a multilevel installation built with mirrors to expand the space between where the audiences were standing. Looking up and down when standing there in between, the feeling was oddly satisfying, it made me lost in the thought of an existential crisis.

I was also inspired by Louise Bourgeois’ Untitled (Hologram Suite), 1998 – 2014. I like the way that she picked different items in different images to represent the different stories when creating this series of artworks. That helped me made my decision when I was debating whether to build a whole room full of furniture or go with one specific item. And I do agree that going with fewer items would express the feeling of loneliness more.

“The holographic image is created by laser beams that record the light field reflected from an object, burning it onto a plate of glass. The image is scaled at a one-to-one correspondence with the original sculptural material that was created by Bourgeois. These elements reoccur in many of Bourgeois installations and are related to her interest in physical and emotional isolation and sexuality.”

The servo that I used, at first, I went for the servo in our tool kit and followed a tutorial trying to make it go 360, but the result was not what I was expecting, because it simply turned into a motor without the function of taking input, and the speed was too fast. So I re-soldered everything back and the servo was able to download codes again but the spin was not ideal and made a lot of noise as well as shakes. So I borrowed a servo that could go 360 without being manually altered from Li’s team, which helped me survived this.

As for the choice of LED colors was aiming to create this harmony between itself and the spinning bed, both of them seemed to form a peaceful and harmless space, but it also emphasizes the fact that it’s a space of absolutely no sleep.

Exhibition Reflection (250-500 words)

The open show was a blast, glad to see everyone’s work shine at their spots. It was interesting to receive feedback from the audience, especially when I ask them to take a look inside first without knowing the story behind it, most people described it as “beautiful and amazing”, which was great, because like I said, dragging people into a bad mood was never my intention, and people are free to feel whatever they feel. But after reading/hearing the background, they would be like, “yes, I can totally see that”. It made me wonder, did they actually see that because they’ve felt the same thing in the past? Or did they match the visual with the story? Either way, empathy was created and connected the audience with the piece, and I was more than happy about that.

Towards the end, there was a girl who came looked at my work and told me she knew exactly what I was going through because she went through the same thing before. We sat at the staircase and talked about the experience, and how good it felt talking about it, this made me realized the fact that there are more people than I expected who are facing a similar issue. And they should know that they are not alone, we are on the same boat here, and we will always be each other’s support.

I picked a location where fairly isolated and unnoticeable, which somehow went well with my concept, but not a lot of people went into the darker room where I was in along with the other three works. It would be nice if the audience knew about the room and paid us a visit:)

Reference:

Samara Golden. The Meat Grinder’s Iron Clothes, 2017. Whitney Biennial. The Whitney Museum of American Art, New York https://samaragolden.com/section/450598-THE-MEAT-GRINDER-S-IRON-CLOTHES-2017-WHITNEY-BIENNIAL-WHITNEY-MUSUEM-OF-AMERICAN-ART.html

Heather James Fine Art. Artsy.net. https://www.artsy.net/artwork/louise-bourgeois-untitled-hologram-suite

Yayoi Kusama, Infinity Mirrored Room – The Souls of Millions of Light Years Away, 2013. https://ago.ca/exhibitions/kusama

(Un)seen

Nadine Valcin

(Un)seen is a video installation about presence/absence that relies on proxemics to trigger 3 different processed video loops. It creates a ghostly presence projected on a screen whose image recedes as the viewer gets closer to it despite constantly trying to engage the viewer through its voice.

As visitors enter the room, they see a barely distinguishable extreme closeup of the ghost’s eyes. As they get closer to the screen, the ghost remains unattainable, visible through progressively wider shots. The last loop plays when visitors are in close proximity to the screen. At that distance the array of squares and circles that are layered over the video giving it texture become very distinct making the image becomes more abstract. The rhythm of the images also changes as short glimpses of the ghosts are seen through the progressively longer black sequences.

The video image is treated live by a custom filter created in Processing to give it a dreamy and painterly look.

Context

In terms of content, my recent work and upcoming thesis project deal with memory, erasure and haunting. I am interested in how unacknowledged ghosts from the past haunt the present. As Avery Gordon (1997, p.7) remarks:

“Haunting is a constituent element of modern social life. It is neither premodern superstition nor individual psychosis; it is a generalizable social phenomenon of great import. To study social life one must confront the ghostly aspects of it. This confrontation requires (or produces) a fundamental change in the way we know and make knowledge, in our mode of production.” (2008, p. 7)

This project was a way for me to investigate through image and sound, how a ghostly presence could be evoked. I also wanted to explore how technology could assist me in doing it in an interactive that differed from the linear media production I normally engage with. The video material for (Un)seen comes from an installation piece entitled Emergence that I produced in 2017. I thought the images were strong and minimalist and provided a good canvas for experimentation.

(Un)seen is heavily inspired by the work of Processing co-creator Casey Reas and his exploration of generative art.  I have been interested in his work as it explores the way in which computing can create new images and manipulate existing ones in ways that are not possible in the analog realm. Over the years, Reas has used various custom-built software to manipulate video and photographic images.

Transference, Source: Casey Reas (http://reas.com/transference/)

Transference (2018) is a video that uses frames from Ingmar Begrman’s black and white film Persona (1966). It deliberately distorts the faces represented rendering them unidentifiable and reflecting on contemporary questions around identity and digital media.

Samarra, Source: Casey Reas (http://reas.com/samarra/)

He applies a similar image treatment in the music video Samarra (2016) and in Even the Greatest Stars Discover Themselves in the Looking Glass, An Allegory of the Cave for Three People (2014), an experience in which three audience members interact, mediated through cameras and projected images. In that piece Reas, once again looks at identity as through a technological lens against the backdrop of surveillance.

Even the Greatest Stars Discover Themselves in the Looking Glass, An Allegory of the Cave for Three People, Source: Casey Reas (http://reas.com/cave/)

KNBC, Source: Casey Reas (http://reas.com/knbc/)

In KNBC (2015), Reas pushes his experimentation further, manipulating images to a level of abstraction where they become unrecognizable in the finished product, breaking their visual link to the original source material. The recorded television footage and accompanying sound are processed them into a colourful, pixelated  generative collage.

Surface X, Source: Arduino Project Hub (https://create.arduino.cc/projecthub/Picaroon/surface-x-811e8c)

From the group project Forget Me Not (assignment 2), I retained the idea of working with an Arduino Uno and a distance sensor, this time to control the video on the screen. I wanted to create a meaningful interaction between the image and the distance that separated it from visitors..

The interactive art installation Surface X by Picaroon was cited in assignment 2 and is still relevant to this project because of its use of proxemics to provoke the closure of the umbrellas, revealing the underlying metal structure and mechanism when visitors approach. Whereas the cretors saw the activation of the umbrellas as a metaphor for the way we constantly prefect and control our digital personas and how they collide with reality upon closer inspection in the moments where all our cracks and flaws are revealed.

Surface X, Source: Arduino Project Hub (https://create.arduino.cc/projecthub/Picaroon/surface-x-811e8c)

In (Unseen), the proxemics are used differently, to signify the refusal of the ghost to visually engage with the visitor, or perhaps, signalling that its presence is not quite what it seems.

Process

Still from unprocessed original footage

I started by going through my original footage selecting all the takes from one of the four participants in the shoot for my installation Emergence. I chose this woman because she had the most evocative facial expressions and dramatic poses. I then created 3 video loops between 30 and 60 seconds in duration. The first loop is comprised of extreme closeups focused around the eyes, where the entire face of the character isn’t seen. The second loop consists of closeups where her entire face was visible. The third loop features shots that are a bit wider, but their duration is shorter and there is a significant amount of black between them.

 

(Un)seen – loop 1

(Un)seen – loop 2

(Un)seen – loop 3

I originally thought of manipulating the video image through Adobe After Effects, but I encountered a Coding Train video by Shiffman that showed the potential of extracting the colour of pixels from a photograph (much like it is possible to do in Photoshop) to program filters that would change the appearance of images. It seemed interesting, but I didn’t know if it would be possible to apply those same features to a moving image, given the processing capacity necessary to play and render live video.

Some of the original footage was intentionally shot with a very shallow depth of field, leaving parts of the shots out of focus depending on the movement of the subject being filmed. As I started to experiment with textures, I found that images that were slightly out of focus helped blur the outline of the circles and squares that created were part of the video filter. I used the gaussian blur function in Premiere Pro to get the desired texture. It was a trial and error process, manipulating the footage in Premiere Pro, then in Processing through several iterations.

 

Left: Original source footage, right: blurred footage

Same footage rendered through processing

Left: Original source footage, right: blurred footage

Same footage rendered through processing

 

I recorded the soundtrack, then edited and mixed it. It consisted of a loop with a woman’s heavy breath on which a selection of 13 different short clips play randomly.

The clips are mostly questions that demonstrate the ghost’s desire to engage with the visitor, but also at times challenges them. Examples include: Who are you? Where do you come from? Can you set me free? Do you want to hear my story?

 

The technical set-up was rather simple. The Arduino Nano was used to read the distance data from the LV-MaxSonar EZ ultrasonic distance sensor. The video for the idle state (loop 1) loaded automatically and two different thresholds were set to trigger the playback of loops 2 and 3.

Challenges

The distance sensor gave wildly different distance readings depending on the space and had to be patiently recalibrated several times. Despite the Arduino being set to send readings to Processing at 1500 millis intervals, the liminal distance at the thresholds for the different video loops triggers posed some problems creating rapid  flickering between the different loops. One might say that the system itself was haunted.

The ventilation in the classrooms at OCAD also proved challenging as despite playing at full volume on speakers, the soundtrack wasn’t fully audible except at very close range. The original intent was to have a 360 degree soundscape through a P5.js library to heighten the immersion and feeling of presence. Unfortunately I could not find an equivalent for Processing.

Closeup of image as seen up close, projected on a screen

The Exhibition   

The exhibition was a wonderful opportunity to get members of the OCAD community and of the general public to engage with the work. The fact that (Un)seen was in a separate room was at once an advantage and an inconvenience. Some people missed the piece because they concentrated on the main spaces, but those who ventured into the room focused their entire attention on the installation. Being the sole creator of the piece, left me with all the duties of engaging with visitors and didn’t allow me to visit my colleagues’ pieces, especially those from undergraduates or second year graduate students that I had not seen. I met and spoke with Digital Futures faculty that I hadn’t yet encountered as well as staff and students from other departments. It was a useful and engaging sharing that should happen more often as it created a real sense of community.

People were eager to engage with the piece and the feedback was overwhelmingly positive. Visitors understood the concept and enjoyed the experience. Because of the issues with the distance sensor, they had to be instructed not to move too quickly and to take the time to pause to minimize false triggerings. The only drawback to the room was the extremely noisy ventilation. Despite the sound playing at maximum volume on the room speakers,  the soundtrack and clips were barely audible. The fact that the door was open to entice people to come into the space only added additional din. It would also have been nice to have a totally dark space to present, but I ended up switching spaces with some of my colleagues in order to accommodate their project.

 

CODE: https://github.com/nvalcin/Unseen

 

BIBLIOGRAPHY

Correia, Nico. “Bridging the gap between art and code” in UCLA Newsroom, April 25, 2016 http://newsroom.ucla.edu/stories/bridging-the-gap-between-art-and-code. Accessed on December 6, 2019.

Gordon, Avery F. (2008). Ghostly Matters, Haunting and the Sociological Imagination. Minneapolis: University of Minnesota Press.

Picaroon (2018), Surface X in Arduino Project Hub. https://create.arduino.cc/projecthub/Picaroon/surface-x-811e8c. Accessed on December 6, 2019.

Reas, Casey(2019). Artists website. http://reas.com/ Accessed on December 6, 2019.

Rosenthal, Emerson, “Casey Reas’ Newest Art Is A Coded, Projected ‘Allegory Of The Cave’” in Vice Magazine, March 14, 2014. https://www.vice.com/en_us/article/mgpawn/casey-reas-newest-art-is-a-coded-projected-allegory-of-the-cave-for-thedigital-age  Accessed on December 6, 2019.

 

by Lilian Leung

Project Description

Absolutely on Music is an interactive prototype of a potential living space. The space is composed of a sensor activated light that turns on when a participant sits on the chair and a copy of Haruki Murakami and Seiji Ozawa’s book Absolutely on Music, which plays the music the author and conductor talk about in each chapter of the book.

This experiment expands upon my personal exploration in tangible interfaces, as well as research further into slow-tech and the use of Zero UI (invisible user interfaces). This exploration is meant to re-evaluate our relationship with technology as being able to augment everyday inanimate objects rather that creating alternative screen-based experiences centered around a hand-held device. The audio played beneath the chair is played in context to each chapter of the book, divided between six individual conversations centered around a different topic and part of Ozawa’s career. There are five tracks played due to the sixth chapter being without a musical piece discussed in detail. The auditory feedback playing the music featured in the book creates a multi-sensory experience, and broadens the audience of the book rather than being solely to music experts that don’t require musical reference, to anyone looking to enjoy a light read.

Project Process

November 21 – 22 (Proposal Day)

Early research pointed to using an Arduino UNO instead of an Arduino Nano so I’d be able to use an MP3 shield to play my audio rather than depending on using Processing. For early exploration, I looked into using a combination of flex sensors and pressure sensors on the binding of the book and on the front and back cover to detect when the book was being picked up. This layout was based on inspiration I found by Yining Shi (2015), where they mapped the Jurassic Park novel with the movie.

After having my proposal meeting, I switched to using copper tape instead of flex sensors as switches to make the thing more reliable data. From there I decided on the modes of the experiment and how the book and chair should behave when not being used.

Modes

Idle Mode	Active
Lamp – Dim Light	Lamp – Full Brightness
Book – Orchestra Tuning	Book – Play Audio

 

Having purchased the MP3 Shield, I starting formatting the MicroSD card with the appropriate tracks related to each chapter using ‘track00X’ to be appropriately read by the Arduino and shield. From the shield diagrams, I would only be able to use Analog Pins 0-5 and Digital Pins 0-1, 3-5, and D10. From this, I laid out each switch for each chapter from D0-1,3-5 and kept D10 to be used for the lamp and sensor input and output.

November 23
To create a more natural reading space, I went to purchase a chair and cushion from IKEA. I tried to pick a more laid back chair so that participants would be interested in sitting down rather than repurposing a studio chair. The supporting beam in the back of the chair allowed for a space to safely and discreetly place all my wiring that wouldn’t be seen. 

For the lamp design, initially I had intended to create free standing lamp, but after some thinking, I decided to have it incorporated inside the side table so there would be less clutter in the space. For the design of the side table, I wanted it to be minimal and be able to discreetly hide the light and all the wiring involved.

November 25

To conserve time and memory on the Arduino, all the audio clips for each chapter were compressed to 7 minutes maximum rather than playing the full one hour to two hour performances. I tested out the MP3 shield using external speakers instead of just headphones to check the sound quality.

November 26-27

Early iterations of the code for the Arduino and MP3 shield weren’t working as tracks refused to play with the if/else statements. Some revisions I made with the help of Nick Puckett was adding a statement to always have the track play the default tuning audio (track 7) and to simply change the track number on each switch rather than playing and stopping each track as it played.

In the early production stage of the side table, I cut a set of 7.5” by 10” sheets of ¼ inch plywood with a 4.5” diameter circle in the center and one with a 5” diameter circle to be able to house the LED stripe for the light. A hole was drilled on the bottom to allow for the wiring to be hidden away. To diffuse the light from the LED, a frosted acrylic sheet was cut to be used to securely hold in the lighting. I chose to have the LED light on the bottom side of the table so that the light would be more discreet and so readers wouldn’t have a bright light shining directly up at their faces while reading. 

Once the wiring was complete, I soldered the wiring onto a protoboard to securely hold everything. I used screw terminals for the wiring for the book switches, the chair pressure sensor and the side table light to be able to transport my work easily between locations and to easily troubleshoot wiring problems. From there I mounted a small board I made to the back supporting beam of the chair so the protoboard and Arduino could safely be placed inside.

November 28 – 29

To finish the side table, I put 4 sheets of ½ inch plywood and glued them together to make the legs stronger. For the wiring, I routered one side of the ½ in plywood from the inside so that the wiring could be hidden entirely inside the legs of the table and discreetly come out the leg.

With the side table complete, I brought all the items together to see how the space would look all together.

December 1

To solve the last few problems I was having with the Arduino and getting the chapter working, I updated the code from a if/else statement to an if statement followed by an “else if” for the remaining chapters. Another issue I was having was the difficulting uploading my code as I’d frequently get ths following error in the serial monitor:

AVRDUDE: STK500_GETSYNC<> ATTEMPT 10 OF 10 NOT IN SYNC

I managed to solve an error I was getting for uploading onto the Arduino Uno. From an online forum, a user mentioned it may be due to a pin being wired into pin 0 (RX) which would cause this error, unplugging this pin during upload managed to solve the issue. Another issue I was having was the consistency of the switches turning on and off as participants may hold the book from different angles and might now apply enough pressure for the switches to properly high and low.  Originally the switches were all formatted with all states stated with only the one switch indicated as HIGH. Though due to the inconsistency in pressure, I removed certain states that were inconsistent.

Ch. 1 Switch	Ch. 2 Switch	Ch. 3 Switch	Ch. 4 Switch	Ch. 5 Switch
HIGH				LOW
LOW	HIGH	LOW		LOW
LOW		HIGH	HIGH	LOW
LOW		LOW		HIGH

From there the final test was adding in the speakers again with the finished chair and table to make speakers would comfortably fit underneath the chair.

 

Project Context

Absolutely on Music explores the use of audio and tactile sensors to create a more immersive experience for inanimate objects in the home, rather than creating an augmented screen-based experience. This experiment is based on the philosophy of slow-tech, countering our need to develop tools that work more efficiently to allow use to do more, faster (Slow Living LDN, 2018). The set-up space is meant to re-evaluate our experience technology and potential of creating a multi-sensory and accessible home. 

This work is an example of Zero UI, which involves interacting with a device or application without the use of a touchscreen or visual graphic interface. Zero UI technology allows individuals to communicate with devices through natural means of communication such as voice, movements and glances (Inkbotdesign, 2019). Most Zero UI-based devices are related to the internet of things and are interconnected with a larger network. For this experiment, I wanted to explore creating a multi-sensory experience not requiring any networked communication or quantified data gathered and allow participants a more immersed and mindful experience with an inanimate object.

I choose Absolutely on Music by Murakami and Ozawa purposely for the references to auditory content that readers may be unfamiliar with, and how searching for said music may interrupt the reading experience instead of making it a seamless experience. This makes the content more accessible to a broader spectrum of readers. 

A book was chosen as the object of choice because of the constant discussion between reading from a digital screen versus a physical copy on paper. A physical book is a dumb object and allows a slow more leisurely experience rid of distractions compared to reading on a digital device. 

The book used is Absolutely on Music, a series of six conversations between the Japanese author Haruki Murakami and Japanese conductor Seiji Ozawa. Classical music, like fine art is generally difficult to access and deeply personal. Interest declines as individuals may perceive themselves distrust their own reactions as classical music may feel perceived to more sophisticated folk as mentioned in a New York Times Op-Ed (Hoffman, 2018). By playing the actual audio through speakers below the chair and having the music audible from headphones, any one can follow along the book without any prior musical knowledge of the works described.

Within the book, the author and conductor discuss both of their careers, from key performances in Ozawa’s career and Murakami’s passion for music, as musical pieces are always deeply integral in all his works from the Wind-up Bird Chronicles and opening with Rossini’s The Thieving Magpie or a Hayd concerto within the page of Murakami’s Kafka on the Shore (2002). 

To elevate the sensory experience of the book, a set of switches were placed within the first five chapters (conversations) of the book. The audio described in each chapter is played with the use of a switch situated within each chapter of the book to provide context to the works Murakami and Ozawa are discussing.

Table of Contents of Absolutely on Music

1st Conversation – Mostly on the Beethoven 3rd piano concerto
Interlude 1 – On Manic Record Collectors

2nd – Brahms at Carnegie Hall
Interlude 2 – The relationship of writing music

3rd Conversation – What happened in the 1960s
Interlude 3 – Eugene Ormandy’s Baton

4th Conversation – On the music of Gustav Mahler
Interlude 4 – from the Chicago Blues to Shin’inchi mori

5th Conversation – the Joys of Opera
6th Conversation – “There’s no single way to teach, you make it up as you go along”

Based on the contents of the book, I pulled the main musical piece the two individuals spoke about into a tracklist that I would use for the interactive book. 

Timing (Chapter)	Tracklist
Idle Mode	Orchestral Tuning Audio
Chapter 1	Glenn Gould’s 1962 Brahm’s Piano Concerto in C Minor
Chapter 2	Seiji Ozawa’s Beethoven’s 9th Symphony
Chapter 3	Seiji Ozawa’s Rite of Spring (by Igor Stravinsky)
Chapter 4	Seiji Ozawa’s The Titan / Resurrection (by Gustav Mahler)
Chapter 5	Dieskau; Scotto;  Bergonzi’s Rigoletto
Chapter 6	(No Audio, No Single Musical Piece Focused)

Project Video

Github Code

You can view the github repository here

Circuit Diagram

*Within the actual wiring, the button switches are two piece of copper foil placed on opposite pages acting as the switches

*The schematic displayed is using a Sparkfun VS1053 though I used a geeetech VS1053, the available pins laid out are slightly different where as the Sparkfun version used in the diagram show D3 and D4 being used, they’re available on the geeetech MP3 shield.

Exhibition Reflection

For the Digital Futures Open Show, my piece was exhibited in the entrance of the Experience Media Lab. I set up the space with some plants and an additional light as props to make the area more comfortable. The space was quieter than the Graduate Gallery which worked out for the piece and allowed participants to sit down and experience the piece one at a time without having too much noise in the background. For the seat sensor, I kept the table light on so that participants could see the reading space clearly rather than being seat pressure activated.

My reflection on the experience would be from the participant aspect, where I noticed people were initially hesitant sitting down on the chair, not sure whether they were supposed to touch it, or that the space I created didn’t look like an art piece. I felt that the piece was successfully as it felt like a natural reading space, and don’t mind the confusion as the chair and book were designed in the context of being in a home rather than as an exhibition piece. 

There was some static from the speaker, but I also noticed that participants may have expected a much faster response from the book when the music changed, as many orchestral pieces had a natural slow build up, some participants flipped through the pages to experience the music change faster or couldn’t hear the musical piece.

While the book audio was designed to be for a single reader than can listen while reading rather than flipping through the pages, in hindsight, I’d probably revise the audio to begin likely in the middle of each musical piece when in an exhibition display so that participants could understand the concept faster.

Some helpful feedback I got on how to possibly improve the piece and learn more about invisible interfaces was reading Enchanted Objects: Design, Human Desire, and the Internet of Things by David L. Rose. Other feedback was also possibly exploring using a Maxuino which has more audio capabilities and support with Ableton live in case I wanted more control with my audio files and audio quality compared to using the MP3 shield.

 

Bibliography

Arduino Library vs1053 for SdFat. (n.d.). Retrieved November 29, 2019, from https://mpflaga.github.io/Arduino_Library-vs1053_for_SdFat/.

Hoffman, M. (2018, April 19). A Note to the Classically Insecure. Retrieved from https://www.nytimes.com/2018/04/18/opinion/classical-music-insecurity.html?rref=collection/sectioncollection/opinion&action=click&contentCollection=opinion®ion&module=stream_unit&version=latest&contentPlacement=4&pgtype=sectionfront.

Inkbotdesign. (2019, August 13). Zero UI: The End Of Screen-based Interfaces And What It Means For Businesses. Retrieved from https://inkbotdesign.com/zero-ui/.

Kwon, R. O. (2016, November 24). Absolutely on Music by Haruki Murakami review – in conversation with Seiji Ozawa. Retrieved from https://www.theguardian.com/books/2016/nov/24/absolutely-on-music-haruki-murakami-review-seiji-ozawa. 

LDN, S. L. (2019, May 25). Embracing Digital Detox and Slow Tech. Retrieved from https://www.slowlivingldn.com/lifestyle/slow-tech-digital-detox/. 

Murakami, H., & Ozawa, S. (2017). Absolutely on music conversations with Seiji Ozawa. London: Vintage. 

Shi, Y. (2015, February 7). Book Remote. Retrieved from https://www.youtube.com/watch?v=M1WrbADjfmM&feature=emb_title. 

Tench, B. (2019, February 11). Some Reflections on Slow Technology. Retrieved November 29, 2019, from https://www.becktench.com/blog/2019/2/11/some-reflections-on-slow-technology.