Author Archive

Project 2 Experiment 1 – Recording and Processing Audio and Video

For this first experiment, I took the time to go to two beaches (Rouge Park and Woodbine) to record a few video and audio clips. They will be the basis from where the project will be built upon. Tools used were a Canon dslr camera and a Zoom recording device.
The audio was processed in Ableton Live and I tried both a simple audio compression and equalizing to make it more suitable for later use as well as a more heavy audio processing experiment in order to give a texture like quality to the water sound.
Video was manipulated in Processing and the code used is an adapted version of the code found on this link (p5art). This preliminary test yielded a result close to the aesthetic I want to achieve. I will try different approaches for achieving a similar result and possibly recreate it in Jitter.

Using Arduino to play two tones simultaneously

During the last hack section, Nick introduced us to the concept of using a timer instead of delay() to coordinate the actions executed by arduino. Rida, Glen and I tried to generate multiple tones to be played in sync with LEDs and a servo. But outputing audio from arduino can be a little tricky, so this post is about what went wrong, how to fix the problem and the shortcomings of generating audio in arduino.

Arduino generates audio with its built in tone() function by using an internal timers. The arduino Uno has three available timers and the Mega, six. To generate a second tone, arduino will need to allocate another timer leaving it with only another one available. Each timer already has a preset function: (1) PWM, (2) Servo Library and (3) millis()/delay() functions.

On a standard Arduino board, the first tone will use timer 2 (so PWM on pins 9 and
10 will not be available); the second tone uses timer 1 (preventing the Servo library and
PWM on pins 11 and 12 from working). On a Mega board, each simultaneous tone
will use timers in the following order: 2, 3, 4, 5, 1, 0. (Arduino Cookbook, p.335)

The standard tone() function allows only one sound to be played at a time, and in order to play additional tones, you will need to download and install the Tone Library. This library is, actually, the full featured version of the built in function:

A simplified version of the Tone library has been incorporated into the Arduino core
since 0018. It only provides a single tone (since only one timer is used).
You can find the core documentation here. (Tone Library website)

Once you install the library and import it into a sketch, you will get a compile error. This is due to the fact that the library has not been updated to be conpatible with versions 1.0+ of arduino. To fix this you’ll need to go to “…\Arduino\libraries\Tone\Tone.cpp” and change #include <wiring.h> to #include <Arduino.h>. Now the library will work properly.


Bear in mind that using arduino to generate multiple square wave sounds will affect its functionallity:

Playing three simultaneous notes on a standard Arduino board, or more
than six on a Mega, is possible, but millis and delay will no longer work
properly. It is safest to use only two simultaneous tones (or five on a
Mega). (Arduino Cookbook, p.335)

Given the exercise Nick assigned us involved using the millis() function and servos, what our group attempted to do was not really a good idea (especially when you have around half an hour to figure out how to make things work correctly!). The good news is that we can learn not only from what works, but also from what does not. After fixing the issues, tweaking the code written in class and adding a potentiometer to have real time frequency control, this is how the code looks like. Below you can watch a video of two speakers playing different tones at the same time with a quick mix so the video doesn’t sound like two old video game consoles playing together. And, yes, the arduino tempo is not perfectly in sync with the other instruments.

Just the two arduino square waves:

Broken Space Theremin

1. Introduction / Background

  When Kate unveiled the theme of the first Creation and Computation assignment I realized it would not be an easy one for me the deisgn or come up with ideas for. I have never been really fond of amusement parks or haunted houses. Then our class voted for a space station themed haunted house and although I am by no means a science fiction geek, I do enjoy reading science fiction books (I tend to lean towards the hard scifi subgenre) so I thought I could draw my inspiration from previous reading experiences.

Early this year I read Arthur C. Clarke’s Rendezvous with Rama and the book had a very precise description of the alien environment – Clarke spent a lot of time describing the centrifugal force and trying to render a functional O’Neill Cylinder. I would even say the book focus was more on describing the environment rather than character or plot development. The atmosphere of the alien vessel was extremely eerie and during the first chapters had zero light and sound. The vessel slowly starts to show signs of life and intelligence. What draw my attention was the fact that the astronauts had no clue of what they were investigating nor what to expect to find during their explorations.

Another story I remember reading (this one is included in a collection of robot short stories, but I can’t remember its name or author and unfortunately I left the book at my home in Brasil..) was one of space explorers going to a distant planet investigate a piece of information from an ancient book. The book points to a specific point in the cosmos and as the story unfolds we discover that the place is our planet Earth – now in ruins. The explorers spend the bulk of the short story exploring the remains of the place and engaging with ancient 20th century technology. The reader then learns that before the earth was destroyed, scientists created two human-like robots who were sent to a life bearing planet with the objective of continuing the human race existance. The space travellers journey to Earth mirrors our natural desire to understand our origins and purpose. For this Creation and Computation project, the interesting idea that can be drawn from this story is the one of travellers from an advanced era engaging with techonology from previous generations.

Personlly I believe the atmosphere of the prevously mentioned Clarke’s book has a very creepy nature to itself, one that can create tension to whoever is immersed in it without the need to resort to sudden loud noises, weird monsters from outer space or other devices that create scary moments. I’d say an exception to this rule is the movie Alien and that is due to H. R. Giger creature design that evokes many of our inner fears and plays with mixing organic and non-organic forms – a trait of Giger’s art – in a freightening way. Giger’s work also has a philosophical quality to it as he tries to give us his personal impressions of what the contemporary trend of combining life and technology will eventually lead us to.

2. Design Process

  After the lenghty introduction, time to move into the design process. I knew beforehand I wanted to create something that used sound and also wanted to constrain the design to use only arduino (i.e. no Processing, Max/MSP, ChucK or something else that can produce sound). So having in mind what was introduced in the previous section, my initial idea was an arduino controlled motor/servo/solenoid that would beat a big metal sheet. The hit would result in a sound similat to that of a gong and the idea was to simulate the kind of sound heard if one was in a space station with something going terribly wrong. I did some online research and tried to find a place I could get a metal sheet. Kijiji had a company that sold individual sheets but while looking at the picture I noticed I would need a very strong motor to produce any kind of interesting sound out of a piece of metal that big.

The first idea slowly began to sound impractical and I decided it was time to move into something else. The next idea was to find a way to use arduino to create ambient type of sounds/pads. The first thought was whether it was possible to add reverb to arduino standard sounds (I am refereing here to the sounds generated through the ‘tone()’ function). My search pointed to an arduino library called Mozzi which not only had reverb but promised to make your arduino a fully functional synthesizer. Given my appreciation for synthesizers and the sound generated by oscillators and audio synthesis techniques, it was quickly decided I was going to use the library. Mozzi is very comprehensive and a quick look into its documentation will show the ammount of classes available (special thanks to Hector who helped me figure out the basics of how the library worked so I could start work on writing the code for this project). Basically you can program your arduino to generate any kind of synthesized sound, so now it was a matter of coming up with something interactive rather than composing a song to be played using an arduino and a speaker. After playing with some ideas I thought creating something like a theremin would be a good solution.

The thermin is one of the oldest electronic instruments and it is still being produced today by companies such as Moog – Robert Moog was one one of the people responsible for keeping the interest for the theremin alive after new electronic instruments and technologies were introduced. Moog Music still uses the original circuit as the basis for its product.  Smaller companies are also making their versions of the theremin, one of them being the Electro Faustus Photo Theremin. Many of these cheaper models uses light sensors instead of the antenna from the original Léon Theremin design. A theremin is an instrument in which you can control pitch by moving your hands in empty space – no physical contact with the instrument exists. Theremin’s original version  of the electronic device could control both pitch and volume – each controlled by one hand – while some newer models may use a knob to control the volume and hand movement only controls pitch. The instrument has a very characteristic sound and has been used in some science fiction productions.

To have some materials to build the prototype, I went to Home Depot and looked for something that could be used to simulate something from in a space station, like control boards and panels.


Laminated floorings had an interesting texture to it and the aluminum kickplate was one of the only pieces of metal I could find at the store (it is, actually, probaly just a rigid plastic with some coating to give the impression of a metal surface). By this point I had decided the theremin would be a broken one, and one added interactive function to this version would be a knock on the plate that would trigger a different sound. Knocking was supposed to be the way one would fix a broken circuit in a space station. After all anything broken magically starts working again when you hit then, right? I was going to use a piezzo sensor to detect the knock, but it was not working as planned and it would make the arduino code more complex – which is something you need to avoid with the mozzi library as it is very computationally heavy for the arduino. I changed the piezzo for an accelerometer so instead of knocking the user would tilt the theremin object.

To wrap this section, the final project ended up being a broken theremin played inside a space station. The theremin has an interesting background that can be used to its favour in a project such as this one with a space station theme. It was developed by a Russian in the 20s, was reintroduced into the market by an American in the 50s – at the same time we had an increased interest in space exploration. It was also the Cold War era. The broken theremin could be applied to a Cold War/Space Race period Space Station that used technologies from that time and people from today could explore an empty abandoned eerie place that was once the apex of an era’s technology, but now is nothing else than a pile of malfunctioning equipment.

Below is an image of the prototype made for the class presentation.


3. Circuit


The circuit for the theremin project is very simple. It uses an ultrasonic sensor and an accelerometer to get data from the environment, a speaker to output audio and a mono audio amplifier to amplify the sound generated by arduino. The amplifier breakout has the capacity to adjust volume using its two volume pins. Though this feature was not used on this project, it is nice to point this out given that it allows for volume adjustments outside of arduino – which means less lines of code. And here the ultrasonic sensor assumes the role of the light sensor in other theremin circuits.

One thing to note is how easy it is to setup a speaker with arduino. One ground wire and one wire connected to a PWM pin and you are ready to upload synth sounds to arduino (Update: after some post-presentation research, I’d just like to point out that it is advisable to use a resitor to connect pin 9 to a speaker. If you are using a piezzo transducer, the voltage won’t be a problem, but if using a speaker, some of the available uses of arduino and audio might damage the pin and/or the speaker, so just use a resistor to be on the safe side). Mozzi uses pin 9 as its default output pin.


And a couple pictures of the circuit.

2_ 1_

4. Code

The code used in the prototype of the theremin can be seen at this GitHub link.

The programming environment of arduino is where I spent most of my time during this project. First of all, when you use the Mozzi library, you need to use arduino in a slightly different manner. The loop function is no longer where all the action happens – now it only calls mozzi’s audioHook() function which “is where the sound actually gets synthesised, running as fast as possible to fill the output buffer which gets steadily emptied at Mozzi’s audio rate”, according to Mozzi website. The updateControl() function is now where most of the code goes.

Mozzi, as stated earlier, is a very comprehensive libary and in order to not get overwhelmed by its functionality I chose a few key components I thought would be important for synthesising the thermin sound. The most fundamental class used was Oscil, and it is where you can choose which waveform to use. There are three instances of the Oscil class: one was used for generating audio, the other controls the Low Pass Filter and the third one controls the vibrato. Low Pass Filter is the second class used and is the one who tries to give the theremin its broken quality – its has a degree of randomness to it and the algorithm was taken from the library’s documentation. The last component of the synthesizer, the vibrato, is toggled on and off via the accelerometer. I tried to add an envelope filter to the theremin, but did not had enough time to troubleshoot it and make it work, so it ended up removed from the code.

5. Project Context


What I envision as the ideal context for the broken theremin to be part of is an eerie empty space like the one shown on the first image above. The context should follow what was described on the first section of this post. The broken theremin will be the soundtrack of an era long gone, when technology wasn’t as advanced as today’s and space exploration was a dream cultivated by popular culture.

Use of this service is governed by the IT Acceptable Use and Web Technologies policies.
Privacy Notice: It is possible for your name, e-mail address, and/or student/staff/faculty UserID to be publicly revealed if you choose to use OCAD University Blogs.