Salisa Jatuweerapong | Tyra D’Costa | Mahnoor Shahid
In this Augmented Reality focused project our team sought out to create a magical experience using the technology we have learned in the last design challenge. We all enjoyed working with Unity collectively decided that we wanted to build on our AR development skills by using Vuforia. Some main target goals for this project were to include elements of storytelling, narrative and original artwork for all the assets used. We settled on developing an app that allows users to combine our designed artwork to create ‘magical powers’ which were designed as shaders in Unity. The artwork was printed on temporary tattoo paper and stickers so that users could apply ‘powers’ to their own bodies and interact with others in a playful and engaging way.
We derived our inspiration from a 2017 art exhibition called Mirages and Miracles by Claire Bardainne & Adrien Mondot. This work gave us the idea to integrate particle systems into our designs in a way that brought the artwork to life.
Miracles and Mirages Exhibition Pieces
Our challenge for this piece was to create a collaborative AR experience: inspire social collaboration and movement from a medium that unfortunately often ends up socially isolating. Our thought process to solving this challenge was engineering interactions off the screen. We decided this could be done through interactions with the tracking images. Thematically, we were inspired by alchemy and magical girl themes: the idea of mixing different elements and powers (including those of friendship!) to create something bigger and better.
In the process sketches pictured below we have outlined two different foundations for the user experience design. The one on the left, design idea one, shows a scenario where users are meant to place their hand in the middle of the plinth and use their phone to activate the visual effects. The image on the right, design idea two, shows a scenario in which users place their hand into the plinth under and iPad to active the visual effects. We also considered using wearable jewelry like bracelets or rings instead of body art, however we opted for the body art as a more intuitive design feature. In the end, we decided to go with design idea one because because the iPad camera we were using was inconstant and laggy with image tracking.
Design Idea One Design Idea Two
Art Direction: Tattoos & Stickers
The idea behind the three tattoos was that when they overlap on top of each other, they would create the design seen on the acrylic plinth. Hence we decided on a geometric design for our tattoos as basic shapes can combine to create interesting designs. To keep a common theme through our tattoos, we repeated an identical circular outline, arrows, triangles, squares and dashed lines.
One of our challenges was to keep the tattoos simple enough to easily transfer on the skin without ripping the design. Another aspect we had to keep in mind was the Vuforia image tracking rating. The tattoos had to be detailed yet asymmetrical to have a high feature count. Consequently, we had to edit two of our tattoo be more asymmetrical to increase their rating to 5 stars. As a result, our original idea of the tattoos overlapping to show the exact design on the acrylic was cancelled.
Art Direction: Plinth
The artwork pictured on the acrylic plinth is a combination of all three tattoo designs layered on top of one another. We used this layered design to create a laser cut friendly template, then we rasterized, scored and cut into a large acrylic sheet. The last step was to carefully glue the led light strip onto the sides of the plinth so that the light would diffuse aesthetically and evenly.
Tech Design: Shaders and Particle Systems
The visual of the shaders would define the “magic” of the experience. We wanted our experience to elicit a sense of play, above all–that was our rational for choosing more “cartoony” effects. While early tests were done with VFX cinema-style light bursts and photo-realistic flames, we wanted our final “powers” to be less realistic and closer resembling low-poly video game style. This was our first time working with shaders. Mannie explored using shader graph early on, while Salisa experimented with the code route. Salisa’s shaders made it to the final and the code was created by closely following this tutorial. Each magic power is two shaders combined: they are made of two stacked balls to create a more interesting affect. As well, an animation code was written to make each “power” bloom to life upon activation. To show that each tattoo had latent power, and something would happen, a generic particle effect was applied to each tattoo so it would sparkle when detected by Vuforia. Only when another one was detected though would a power bloom to life.
Tech Design: Image Tracking and Distance Tracking
Using Unity we created a script in C# that would accurately track the distance between three image targets. This was an integral part of the system because we wanted users to be close enough together in order to activate their ‘power combinations’. When users view their tattoos through our AR app a weak particle system is activated displaying to the user their unique power element (green core, blue flame or fire ball). The script allows for the system to detect when more than one image is being tracked. If two different image target are being tracked, and the total distance between the two images is small enough, then the system will activate a stronger shader effect depending on the image target combination. When three image targets are tracked at an optimal distance then the users will activate the largest shader effect called ‘Big Blast’.
Scripting Distance Tracking – In this process video you will see the first stages of creating the distance tracking script
Activating Shaders and Particle Systems – In this process video we solidly our proof of concept by successfully activating shaders and particle systems with distance tracking scripting
We created a brief narrative surrounding the installation to tie the visual elements and artwork together. In the future we would like to do some more character building by creating character models and personas that associate with the different tattoo types.
Narrative – Story Telling Presentation Video – In this video we have detailed the story of the three brave archers as part of the installation.
CLCK HERE TO SEE VIDEO OF WORKING FINAL INSTALLATION – see the final product in action.
In this installation users are drawn in by the narrative video and encouraged to take on a character persona. Each character has a specific tattoo associated with a unique elemental super power. Users come can apply the tattoo of their choice to their hands, and finally enter the plinth to find two other users with different tattoo markings. By connecting with other users at the plinth the user can activate the full potential of their powers. As they combine tattoos with other they will unlock new stronger powers with greater effects. In the future we hope to incorporate what we have done with this project into a gameplay setting. This could include anything from a VR game environment to a table to a table top board game, ideation is still in the works. There’s potential for this to grow as a commercial game and ARG. along the lines of trading card games and LARP-ing.