The Soundscape Experience
Tyra D’Costa | Shiloh Barnes | Francisco Samayoa
Our project seeks to encompass elements of sound, sensory perception, and nature in a installation based artwork. Our process began by discussing the types of sensors we wanted to explore, eventually we decided on the MPU 9250 Accelerometer and Gyroscope. Our proposed idea, is to integrate the sensor into headphones, which can then change the sound and visual experiences of the user. As the user navigates the room and moves their head about, the world in which they are digitally immersed in will enhance the real world.
- Engaging UX Design: We want to solidify the conceptual ideas of the soundscape experience to create an immersive UI/UX design.We want this to include, elements of storytelling and gamification.
- Sound-Sensory Experience: Moving forward we will have to test that the sensor can allow smooth transition between sound files. The media assets will consist of ethereal soundtracks, data visualizations (natural algorithms and patterns), and nature related images.
- Integrate wearables: Also, we need design a way to integrate the sensors into the wearable technology ( the headphones).
- Discuss concepts of nature, ecosystems and natural algorithms: Lastly, we want to think about how these concepts can work together to create a narrative and gameplay element.
So far, we have able to get the MPU sensor working with Arduino so that we receive X,Y,Z coordinates. This required, scouring the web for solutions to library related issues, and calibrating the code in order to receive accurate data readings. Additionally, we were able to take these coordinates and map them to a camera in Unity, so that the sensors orientation changes the perspective of the viewer.
Links To Contextual Work
How to play multiple audio files
Research on Arduino + Gyroscope
Good Link explaining how IMU sensor works
Tutorial for IMU and Unity set up
How to set up MPU and Arduino Library
Unity + Arduino Library
- User starts at any location on the map
- They put on the headphones
- Depending on the position of the head, the headphones will play a particular sound.
- If a particular sound is playing then the screen will display an associated image or video
- The user must experience all the audio and visual data in order to try and make a connection between the media and a place that exists in the real world before their opponent.