MVP Console is an interactive installation based on the tangible interface experience. Players are invited to play with three digital activities created in Processing: Core ball, Spinning Music, and Starry Drum. MVP Console is a handmade cardboard box with 5 aluminum foil buttons that attached the 5 pins from MPR121 touch sensor. In this project, MVP Console stands for Musical, Visual, Physical Console; it makes the players enjoy physical touching, music creating, and music visualization. The players could switch the card paper upon the box to play the matching activities.
The first activity is Starry Drum. Once the player touches the buttons, they would hear constant drum beats and the starry dots would appear on the screen. Every button represents a different part of the drum and the different sizes of the starry dots. The dots would fade away once the new dots appear. In this activity, the players are creating an image of a starry sky and play with the drum at the same time.
The second activity is Spinning Music. The players could touch the button to play the songs and watching the spinning pattern, which is the music visualization. There are 5 songs overall, the spinning pattern would change color and speed that matches for each song. The songs I used are Woodkid’s music: “L’aérogramme de Los Angeles”, “Iron (Quintet Version)”, “I Love You”, “Iron” and “Run Boy Run”. The spinning pattern and song would stop once the player releases the button, it is a unique way to touch with music and visuals.
The last activity clones a popular phone game, Core ball. The players need to shoot the given needles to the rotating circle. Every shot needle would be stuck and rotate with the circle. If your needle hits an existing needle on the circle, it will be automatically judged lose by the system, you need to restart to continue. The screen would turn blue when you win the level, and red if you lose the game. The players need to press three aluminum button that labeled C, S, and SHOOT! to play, everything is drawn and labeled on the card paper. The players can see their level on the screen, and the number of needles left in the center of the circle. The higher the level, the more needles you need to shoot.
How it works
Final project images
Link to the Arduino code hosted on Github
MVP Console is inspired by my favourite entertainment and learning device in my childhood, an electronic dictionary. It is like a small size laptop, it was heavy but convenient. Other than look up the vocabulary on it, it is my “all-purpose media player”. It could be the music player, and once the music is playing, the screen would show the music visualization, which is the spinning firework pattern. Although it has a dull green dot-matrix screen and it has only one song, its music visualization was like magic to me in my childhood. It is also my game console, it has one game, Snake, which always made me lose time track. Finally, it is also my musical instrument. Every button has a different sound effect, I used that to cover all popular songs in that period, I even used that to perform in the music class. The electronic dictionary is now obsolete, its functions are simple, but it gave me countless happy memories. I want to recreate my childhood memories.
Once I know the theme of Experiment 3, Tangible interfaces, the first thing pop into my mind is about music. It reminded me of the tap-tap games, they have the perfect combination of touching, visual, and music. Other than that, it also reminded me of the electronic devices that could imitate the sound of real musical instruments, which could be a more convenient way of composing music. It could also be about music visualization, combining the physical interaction with the seem music could be an inspiring experience. Music is important in my life. It affects my emotion, and it is completely connected with my vision. I composed with my vision and I listen to create images.
For this project, I want to explore the relationship between music, visual, and physical. I want to make an “all-purposed media player”, an “old game console”. Inspired by Tyler Crumpton’s Capacitive-Touch Arduino Keyboard Piano, I want to make the button with aluminum foil, tapes, and cardboard as my material. I made a cardboard box with 5 square holes on the top, and I sticked aluminum foil under each hole. There are three card papers with my drawing and holes that fit each game. The players could switch the card paper on the box, like switching the disk of the game console. The touch of aluminum foil buttons is the Physical part of my MVP console.
I used the sound effect of the button from my electronic dictionary to make music. Even the musical instrument are not with me, I could make music. It is kind of like the calculator cover music, which is now very popular. It is a creative new way to make music with unconventional methods. Each button of the musical calculators corresponding to a musical note. With two calculators, the players could try the classic piano-playing method of using both hands. (Lee) It is quite inspiring, as to play music with 5 buttons, I want to recreate it with the sound of drumbeats. When I was learning drums, with the practicing with the speed of beating, the countless beats in a second make me imagine a thousand-star sparkle, the stars blink with the rhythm. I want to combine those visual imaginations with the sound of the drumbeat. I purposely make the drumbeat unstoppable if the touch isn’t released, there is no single beat. To fit the speed of the drumbeat and the endless stars. Repeating the same sound could be beautiful, same as visuals.
Spinning Music is a musical visualization of the rotating pattern. The looping rotating pattern always makes me lost the time track for just staring at it. After watching the tutorial of the rotating pattern on Youtube, I think it could fit the music well, it could be the animation of music visualization. By changing the speed and color of it, it could work with music in different styles. That is one of the reasons I chose Woodkid’s music, he has various styles. His songs get to my emotions and speak to me a lot. The visual and the music reflects what he states “I like to translate sounds into images and images into sounds”(Dellarciprete) The pattern would keep rotating when we touch the button, when we stop it, even the pattern is static, we would feel like it’s rotating backward. It creates an optical illusion, which also indicated the melodies was stuck in our minds. Music affects our mood, and we are exploring 5 different emotions and memories in our mind in this activity.
Coreball is a popular phone game, I fit it with the shooting sound effect. As there are 5 buttons overall and we only need 3, so I cut 3 holes on the card paper. After we are familiar with the game, we could form beautiful patterns, like the dandelion. The game rule is simple but quite addictive. Every shooting needle combines the human touch and sound effect.
Overall, with the topic of the tangible interface, I make the viewers explore 3 types of touching. In Core ball, we used a single click to have one shoot and sound effect. In Starry Drum, the drum sound is constantly beating after we touch the button, we would not hear a single beat. In Spinning Music, the music start when we touch it, and we need to hold the button to keep the music playing, it would only stop when we release it. I want to make it like a real game console, with only a few buttons, but you can play multiple games.
Crumpton, Tyler. “Capacitive-Touch Arduino Keyboard Piano.” Instructables, Instructables, 23 Oct. 2017, www.instructables.com/Capacitive-Touch-Arduino-Keyboard-Piano/.
Dellarciprete, Alex. “Woodkid’s The Golden Age – An Analysis of an Audiovisual Experience.” Medium, Medium, 12 Dec. 2017, medium.com/@alexdellarciprete/woodkids-the-golden-age-an-analysis-of-an-audiovisual-experience-44ec7d73047.
Lee, Dami. “Calculators Are the Latest Instrument Used in Creative New Ways to Make Electronic Music.” The Verge, The Verge, 11 Oct. 2017, www.theverge.com/tldr/2017/10/11/16459502/musical-calculator-youtube-super-mario-theme.