Experiment #5: Air DJ
Emilia Mason, Roxanne Henry, & Kristy Boyce
Emilia, Kristy and Roxanne were inspired to create a fun and innovative way to create music and play with your friends. The Air DJ is a gesture-based musical device which allows you to load up your favourite sound effects and assign them to gestures which are intuitive to the user. It runs on the Adafruit Feather M0, combined with the Music Maker Wing. A Sparkfun ZX gesture sensor serves as the input. Then, you take your device with you, and jam wherever and whenever. Up to six sound effects can be loaded per device.
We want to bring fun music to everyone, everywhere.
PROMOTIONAL VIDEO: https://www.youtube.com/watch?v=WxZOndgZ8vM&t=6s
3 SparkFun ZX Distance and Gesture Sensor
3 Adafruit Feather M0
3 Adafruit Music Maker FeatherWing
3 Lithium-Ion Polymer (LiPo) Battery (3.7V 1000mAh)
3 Micro SD Cards
4 Sets of Speakers
Beforehand: Preparations and Testing
What needs to be done ahead of time?
To make this experiment possible it was necessary to get the SparkFun ZX Distance and Gesture Sensor on time so we booked one full day to drive to Markham and buy all the pieces necessaries.
During our conversations, we decided to 3D print all three enclosures, for this it was going to be necessary to go time ahead to 100 McCaul to print them. This meant we needed to have the design ready time ahead.
To test Air Dj we booked Thursday to work together before presenting the experiment on Friday. We also decided to meet during the weekend for our second session of testing and film a promotional video.
Do you need extra batteries?
No, we won’t need extra batteries. We will recharge the 3 Lithium-Ion batteries overnight every time we tested the devices.
What goes into your repair kit?
The assembly of our experiment was not complicated at all. The recurrent issue was to reseat the Music Maker Featherwing, this was necessary every certain amount of swipes (about every 20). Having to reset the Featherwing meant we needed to have easy access to it, for this reason we decided to add velcro to it.
How will you document your own user experiences while using the device? Notes? Journaling? Photos? Video? Audio recording?
We will be using video and photos of us testing Air Dj. We also want other people to interact with this experiment, we will be interviewing them as well.
What will you do to ensure that your data collection methods are consistent for each group member?
Data collection method: Working together to make sure we collect consistent data.
For each group member, what are the dates & times of testing?
Roxanne: One 4 hour session on Thursday
One 4 hour session on Sunday
Emilia: One 4 hour session on Thursday
One 4 hour session on Sunday
Kristy: One 4 hour session on Thursday
One 4 hour session on Sunday
Air Dj is an interactive device that works better when played with others and with a backtrack.
For this reason we have chosen to do two 4 hour sessions in which the three of us would be present, using and testing all three devices at the same time.
Will you be doing your day-to-day activities or doing a special activity?
We will be meeting in the Digital Futures Studio together. The idea is to have all three devices working at the same time and playing a backbeat for us to practice our music skills and test Air Dj.
Deciding what to do:
For experiment 5 we had so many ideas and it took us a few days to days to full decide what our experiment was going to be.
Some of our ideas:
-King of the hill game
-Light up shoes
-Personal/wearable drum set = EL wire
-Wearable Bongo butt, you literally tap that ass and make beautiful music
-AR glove – force feedback, detect collisions w virtual objects -magnetorheological
-Haptic undapants (what?)
-Conductive pen greeting card /copper tape
-Lazer tag phasers – IR sensors
-Virtual dj set, air dj
Emilia mentioned she wanted to make a wearable or something like Behnaz Farahi
When we finally decided to develop Air Dj we made a list of the materials we would be needing:
2 more feather wings (Kristy already had 1 from the Mansplaining project)
3 micro sd cards
Buy 1 battery (Emilia already had 2 from the Love Corner project)
3 Proximity sensors / Gesture sensors
Since we need we were going to need more Music Maker Feather wings, Kristy immediately made an order for two more from the provider she got hers. That was very convenient!
Kristy mentioned it was very easy to use and sent us this tutorial :
Even though we had the Music Maker Feather Wings we were still deciding what sensor to use. At first we thought the proximity sensor we had in our CandC boxes were ok but then we decided to talk to Afaq and consult him which sensor he would recommend for our idea.
The SparkFun ZX Distance and Gesture Sensor is the winner! Afaq recommended other sensors as well but because of price and availability, we chose the SparkFun ZX.
The only “issue” we had is they were only available in Canada Robotix and because of time we knew ordering online was not going to be our best option. We booked Saturday December 2nd for a roadtrip to Markham!!
During Friday’s class Roxanne and Emilia had the chance to speak with Nick regarding the Air Dj idea. Nick recommended to play with the size of our device, instead of making the device small to make it a little bit bigger, this would make the user’s experience more interesting because of the movements. We totally agreed!
That Friday (Dec 1st) we started to look for different enclosure ideas:
After researching possible enclosure ideas we decided our enclosures should be 3D printed turntables. We really liked the idea of 3D printing something that looked like a record for our Air Dj device.
This project was full of adventures! On Saturday Kristy and Emilia went to Markham to get the 3 ZX gesture sensors and 3 micro SD cards.
We also tried to buy these speakers but they were not available anymore and the project was becoming waaaaaaay too expensive. We decided to use regular speakers for the Air Dj.
Saturday night we all met in school and Roxanne started working on the code and explaining how it works.
From the very beginning, we had problems with the pieces. One of the Music Maker Feather Wings was broken and we had to use tape for the micro SD to stay in place.
Programming and Assembly:
Programming for the SparkFun ZX gesture sensor was surprisingly easy. SparkFun was kind enough to provide a comprehensive and well-rounded library which contained useful public methods such as readGesture() and gestureSpeed(). It was fairly trivial to create a case statement which iterated through the available gestures (sort of like an if statement, but cleaner for comparing different possibilities of the same variable) as they were being executed. Then, it was only a matter of determining the speed of the gesture to branch off into different sound effect options.
Following this, the music wing needed to be configured to play the sound clips. It was discovered by Sana, Kylie and Ramona that the music wing only accepted headphones which had a TRS jack. This meant that any speaker cable which also had a microphone would not work. This saved us a lot of time on troubleshooting since we all regularly use iphone headphones, which are TRRS.
One of the first problems we encountered after that was that wav files sounded really distorted. We didn’t have time to test whether file size impacted this, so we decided to simply use mp3 for everything. At this point, we realized that smaller file sizes were optimal, and tried to keep all of our files <= 50kb. The next challenge revolved around the feel of the sound bites we wanted to use. Since the Air DJ is gesture based, which means no haptic feedback, we needed to make sure the sounds associated to each gesture were intuitive and felt good. Once we accomplished this, it was only a matter of implementing the branches based on speed for each gesture.
We had to decide whether or not we wanted to play the full file or allow the feather to continue to operate while the sound bite played. In the end, the sound files were so short and the sensor reactive enough, that we settled on using startPlayingFile() instead of playFullFile(). This allowed the user to play more quickly, but also put the wing at risk of overloading and crashing. Allowing the full file to play wasn’t significantly less at risk of crashing the wing, so the tradeoff was worth it in the end for increasing the experience quality.
Speaking of crashing, boy do those music wings crash a lot! This made troubleshooting especially challenging since the life of the wing before crashing was limited to 5-10 gestures, depending on the sound bites it was trying to load. Some sound bites reacted better than others. In one case, a tiny 40kb mp3 file was crashing the wing almost constantly. We shortened it and compressed it a bit and it was suddenly much more responsive. We have yet to figure out if there was something corrupted about the file before or if the size alone caused the problem (though we had many sound files which were around 50kb, so it’s not likely the size alone which caused the problem). Our theory is that the music wing wasn’t meant to rapidly load and unload as much as we demanded of it.
The problem was always the same: the gesture would register, the sound bite was beginning to load from the SD card, and then the wing would stop working entirely. The loop function would stop executing too, which made it clear that the entire feather crashed. A manual reset of the wing would then be necessary, where the sound bite that was loading off memory would finally play, and then the wing would be good to go for another 5 to 10 gestures. Once every 8 or so resets, though, the feather would need to be powered off and then powered back on. I suspect that at these moments, there is simply too much corruption for it to recover after a reset.
These were some of our worst problems concerning the hardware. A simple, though no less devastating problem was that one of our music wings arrived with a broken SD slot. The pin that holds the SD in place was absent, so the card would have to be taped into its chamber in order for it to read. This was unreliable and frustrating to work with since it seemed that even jostling the card could disconnect it, and the feather required a reset when this happened.
The Enclosure and Branding
After figuring out how to make the sensor and the music maker featherwing we decided to make the coolest enclosure ever made!
From the very beginning we knew we wanted to 3D print the enclosures but the 3D printers in 205 Richmond were broken at the moment, Kristy decided to go to 100 McCaul to get them printed there
The enclosures ended up taking from Mon-Thurs to print due to a failed print on Monday in our lab and the size of the print job. In total we made 3 versions of the enclosures: The 3D printed ones (in grey below, later painted black,) a wooden backup, and then after testing, the melted record top Emilia can be seen working on below.
One of the problems with the casing (more on this later,) is that we didn’t consider that a feather mounted on a square protoboard wouldn’t be able to get close to a circular wall structure. This meant that plugging in headphones wasn’t as easy to do as we envisioned. We also didn’t think of putting in a hole for the charging cable.
Otherwise, the physical design of the product looked and felt really great. The rippled look of the vinyls gave a kind of unique affordance to the device which we might not have been able to convey otherwise. A small adjustment we should have considered sooner in the production, however, was the placement of the feather.
Since we had to open the case so often to reset the feather, and putting the lid back on to align with the sensor wasn’t a trivial task, we should have placed the entire device itself on the lid so we could adhere the sensor to the lid. With the feather and sensor adhered to the bottom of the body, it was easy to jostle the solid core cables a bit too much, and several of the connections ended up snapping at certain points during the testing.
Survey and User Testing
Survey Link: https://www.surveymonkey.com/r/QKB5MWB
After Testing Session:
We tested the Air Dj amongst ourselves and students at OCAD. In general, everyone was very excited to interact with the device and seemed to enjoy using it. There is a steep learning curve in terms of actually making anything that resembles music. We found that it felt more or less intuitive and organic to use depending on the beat of the backing track that was playing.
The most frustrating part (touched on earlier) was that the microprocessor in the featherwing would stop functioning within approx 20 swipes. This affected its portability and caused us to change the design of the top of the enclosure(mentioned earlier) as the 3D printed enclosure was incredibly tight which we thought would be fine if you were just opening it occasionally to charge the battery, but was not feasible if it needs to be opened after a minute or two of use.
Again, to resolve this, we switch to the melted record top option which we then secured via velcro.
Kristy preferred the older design, while literally, everyone else liked the look of the new melted record top. Regardless of design, functionality needed to prevail. Most people (ourselves included) really enjoyed interacting with the Air DJ, we just wish it would last longer each session. We were also testing using large external speakers, though we had originally envisioned the Air DJ as an all in one enclosed product. This was a strictly financial choice as the project had become quite expensive. Our ideal iteration (especially after testing,) would have an enclosed speaker along with the battery with a small port added on the side to plug the battery in.
Final Product and Photos From Class Show:
Project Context and References:
We wanted this product to be fun and portable, so we researched other musical devices, things DJs are currently doing and toys.
After already deciding on the Air DJ, we came across this great device that creates a MIDI interface out of a pizza box:
Keith McMillen is also an interesting musician that’s creating different sounds and instruments:
A look at what’s currently on the market: