Alchamation : Making Magic with Augmented Reality

 

Alchamation

Salisa Jatuweerapong | Tyra D’Costa | Mahnoor Shahid

Context 

In this Augmented Reality focused project our team sought out to create a magical experience using the technology we have learned in the last design challenge. We all enjoyed working with Unity collectively decided that we wanted to build on our AR development skills by using Vuforia. Some main target goals for this project were to include elements of storytelling, narrative and original artwork for all the assets used. We settled on developing an app that allows users to combine our designed artwork to create ‘magical powers’ which were designed as shaders in Unity. The artwork was printed on temporary tattoo paper and stickers so that users could apply ‘powers’ to their own bodies and interact with others in a playful and engaging way.

Inspiration 

We derived our inspiration from a 2017 art exhibition called Mirages and Miracles by Claire Bardainne & Adrien Mondot. This work gave us the idea to integrate particle systems into our designs in a way that brought the artwork to life.

screen-shot-2019-04-11-at-10-50-31-pm screen-shot-2019-04-11-at-10-49-49-pm screen-shot-2019-04-11-at-10-49-30-pm

Miracles and Mirages Exhibition Pieces

Our challenge for this piece was to create a collaborative AR experience: inspire social collaboration and movement from a medium that unfortunately often ends up socially isolating. Our thought process to solving this challenge was engineering interactions off the screen. We decided this could be done through interactions with the tracking images. Thematically, we were inspired by alchemy and magical girl themes: the idea of mixing different elements and powers (including those of friendship!) to create something bigger and better.

Process

In the process sketches pictured below we have outlined two different foundations for the user experience design. The one on the left, design idea one, shows a scenario where users are meant to place their hand in the middle of the plinth and use their phone to activate the visual effects. The image on the right, design idea two, shows a scenario in which users place their hand into the plinth under and iPad to active the visual effects. We also considered using wearable jewelry like bracelets or rings instead of body art, however we opted for the body art as a more intuitive design feature. In the end, we decided to go with design idea one because because the iPad camera we were using  was inconstant and laggy with image tracking.

sketch        55625733_2264748886880174_8888595064301813760_n

            Design Idea One                                      Design Idea Two 

 

Final Design Idea
Final Design Idea

 

Art Direction: Tattoos & Stickers 

Unused Tattoo Design Iterations
Unused Tattoo Design Iterations

The idea behind the three tattoos was that when they overlap on top of each other, they would create the design seen on the acrylic plinth. Hence we decided on a geometric design for our tattoos as basic shapes can combine to create interesting designs. To keep a common theme through our tattoos, we repeated an identical circular outline, arrows, triangles, squares and dashed lines. 

screen-shot-2019-04-11-at-11-38-16-pm screen-shot-2019-04-11-at-11-38-09-pm screen-shot-2019-04-11-at-11-38-02-pm

One of our challenges was to keep the tattoos simple enough to easily transfer on the skin without ripping the design. Another aspect we had to keep in mind was the Vuforia image tracking rating. The tattoos had to be detailed yet asymmetrical to have a high feature count. Consequently, we had to edit two of our tattoo be more asymmetrical to increase their rating to 5 stars. As a result, our original idea of the tattoos overlapping to show the exact design on the acrylic was cancelled.

Iteration Process for best quality Image Tracking
Iteration Process for best quality Image Tracking

Art Direction: Plinth

The artwork pictured on the acrylic plinth is a combination of all three tattoo designs layered on top of one another. We used this layered design to create a laser cut friendly template, then we rasterized, scored and cut into a large acrylic sheet. The last step was to carefully glue the led light strip onto the sides of the plinth so that the light would diffuse aesthetically and evenly.

lasercuttemplateforplinthredandbluetemplateforlasercut56711214_1301739379977108_2917642842622918656_n

Tech Design: Shaders and Particle Systems 

screen-shot-2019-04-16-at-8-53-56-pm

The visual of the shaders would define the “magic” of the experience. We wanted our experience to elicit a sense of play, above all–that was our rational for choosing more “cartoony” effects. While early tests were done with  VFX cinema-style light bursts and photo-realistic flames, we wanted our final “powers” to be less realistic and closer resembling low-poly video game style. This was our first time working with shaders. Mannie explored using shader graph early on, while Salisa experimented with the code route. Salisa’s shaders made it to the final and the code was created by closely following this tutorial. Each magic power is two shaders combined: they are made of two stacked balls to create a more interesting affect. As well, an animation code was written to make each “power” bloom to life upon activation. To show that each tattoo had latent power, and something would happen, a generic particle effect was applied to each tattoo so it would sparkle when detected by Vuforia. Only when another one was detected though would a power bloom to life.

56744578_443984636163464_411708929270087680_n

Tech Design: Image Tracking and Distance Tracking

Thought Process involved in designing the distance tracking script
Thought Process involved in designing the distance tracking script

Using Unity we created a script in C# that would accurately track the distance between three image targets. This was an integral part of the system because we wanted users to be close enough together in order to activate their ‘power combinations’. When users view their tattoos through our AR app a weak particle system is activated displaying to the user their unique power element (green core, blue flame or fire ball). The script allows for the system to detect when more than one image is being tracked. If two different image target are being tracked, and the total distance between the two images is small enough, then the system will activate a stronger shader effect depending on the image target combination. When three image targets are tracked at an optimal distance then the users will activate the largest shader effect called ‘Big Blast’.

Testing Proof of concept with Game Objects
Testing Proof of concept with Game Objects

Scripting Distance Tracking – In this process video you will see the first stages of creating the distance tracking script

Testing proof on concept on Body
Testing proof on concept on Body

Activating Shaders and Particle Systems – In this process video we solidly our proof of concept by successfully activating shaders and particle systems with distance tracking scripting

Narrative Design

We created a brief narrative surrounding the installation to tie the visual elements and artwork together. In the future we would like to do some more character building by creating character models and personas that associate with the different tattoo types.

Narrative Concept
Narrative Concept

Narrative – Story Telling Presentation Video – In this video we have detailed the story of the three brave archers as part of the installation.

Finished Installation

img_5557 img_5561

CLCK HERE TO SEE VIDEO OF WORKING FINAL INSTALLATION –  see the final product in action.

57183845_722992521429753_4641635671490953216_n

In this installation users are drawn in by the narrative video and encouraged to take on a character persona. Each character has a specific tattoo associated with a unique elemental super power. Users come can apply the tattoo of their choice to their hands, and finally enter the plinth to find two other users with different tattoo markings. By connecting with other users at the plinth the user can activate the full potential of their powers. As they combine tattoos with other they will unlock new stronger powers with greater effects.  In the future we hope to incorporate what we have done with this project into a gameplay setting. This could include anything from a VR game environment to a table to a table top board game, ideation is still in the works. There’s potential for this to grow as a commercial game and ARG. along the lines of trading card games and LARP-ing.

 

 

Toronto & CO: AR Tour Book

Documentation: Atelier Unit 3: Final Project: Ambient Sound

Kiana Romeo | Dimitra Grovestine | Tyra D’Costa | Sam Sylvester |Ola Soszynski

Concept

Our concept for our final project was developed off of the idea of tourism and everything that the city of Toronto has to offer. Originally, we thought about having our set space as the entire city, where tourists could walk around and discover history through images and audio. We then decided to make an interactive promotional piece for Toronto which could immerse tourists here, and elsewhere in the world, in our city’s culture.

54515251_334714937155178_1038404701399285760_n

Context

Being such a diverse and ever-changing city, there’s always something new and exciting to learn about and explore here in Toronto. Although experiencing what our city has to offer firsthand is ideal, it is not always as simple as jumping on a plane and going across seas and even if one wanted to do just that, it is definitely a good idea to explore what the destination has to offer in terms of tourism and entertainment first. By creating an immersive and interactive tour pamphlet, we have given people the opportunity to experience Toronto from the comfort of their own home before their visit to the city.

Project Breakdown

  • Promotional Piece (a graphically designed pamphlet for tourists to access)
  • Representative Blender objects for each highlighted Toronto element
  • A general ambient sound
  • A corresponding sound or story through audio for each highlighted Toronto element
  • An app created through Unity

screen-shot-2019-03-27-at-11-25-44-am

Division of Roles

Unity App Development  – Tyra

Blender 3D Modelling – Kiana, Dimitra & Tyra

Audio Design – Sam, Tyra & Dimitra 

Graphic Design – Tyra & Ola

Execution

  1. Research
  2. Develop App in Unity
  3. Model Objects in Blender
  4. Record and Mix Sound in Reaper
  5. Design and Print Magazine in Photoshop

Unity Testing

One of the first tests that Tyra conducted for unity was the initial use of combining the camera with the 3D objects. Tyra did this by first building out an AR business card, and attaching video, sound and images to the AR object in Unity. 

screen-shot-2019-03-27-at-11-15-41-am

Next, Tyra started to build the final application, she built  five Image Tracking  Anchors. Nested within the anchors  are five different prefabs containing the modeled objects from Blender. In each Prefab modifications were made to include ambient sound, and touch based interaction. Using Lean Touch Tyra was able to incorporate an interactive aspect between the user and the AR object. The Lean Touch Script was built to allow users to scale, rotate and move the objects within the AR space. The last step was to replace the test objects, images and videos for the final project assets.

CLICK HERE TO SEE THE APP IN ACTION

CLICK HERE TO SEE THE APP IN ACTION # 2

Blender

For many of us, the models created in this project were our first attempts at working in Blender without the help of an instructor. Our modeling team was able to put together five models: A coffee cup (Tyra), a timbit (Dimitra), a hockey puck (Dimitra), a CD (Dimitra) , and a Teepee (Kiana). These models were they exported and sent to Tyra to be uploaded into the Unity AR application. 

With Materials

screen-shot-2019-03-27-at-11-19-43-am screen-shot-2019-03-27-at-11-19-54-am

screen-shot-2019-03-27-at-11-20-34-am screen-shot-2019-03-27-at-11-20-48-am screen-shot-2019-03-27-at-11-20-04-am

Without Materials

screen-shot-2019-03-27-at-11-22-57-am

Graphic Design

The booklet was centered around being a visual aid for the rest of the experience. As such, Tyra  designed the layout to be quite simple, with a focus on photos and headlines of the topics covered. We debated on having either filler text or full information, and eventually settled on filler text, due to time constraints and to not distract from the core of the simulation. The pages are laid out in InDesign, with a modern layout and format. This process was relatively simple, as InDesign is something Ola was worked in for several years, with the biggest dilemma being with the print layout of the pages, which Ola settled after changing the page setup and order. The booklet itself is hand-bound, to have the product itself last longer and seem a bit more solid, as it were.

screenshot_2-1screenshot_4screenshot_6screenshot_3screenshot_5user_scoped_temp_data_msgr_photo_for_upload_1552954397434-jpg_1552954399745

Audio

The Audio team brainstormed and mixed together a variety of Canadian inspired artwork such as: poems, songs and field recordings using Reaper. These were then sent to Tyra and uploaded into the Unity AR application.

  • Bad Canadian Songs(Sound Sourcing done by Dimitra, Editing done by Sam)
  • Hockey Arena Foley (Sound Sourcing done by Dimitra, Editing done by Sam)
  • Hey Trudeau Poem  (Sound Sourcing done by Dimitra, Editing done by Sam)
  • Crickets (Field Recording done by Tyra, Editing done by Tyra)
  • Crowd Sounds (Field Recording done by Tyra, Editing done by Tyra)

screen-shot-2019-03-27-at-11-24-28-am screen-shot-2019-03-27-at-11-24-33-am

screenshot_27

 

The Dating Game

Tyra D’Costa | Ola Soszynski | Samantha Sylvester

The Dating game is indented to be an interactive storytelling table top interface. Our inspiration came from the short story “Cat Person” by Kristen Roupenian that recently went virally after being published in the New York Times. We wanted to mold Roupenian’s short story into what we recognized as the dreaded, and inevitable stages of dating of in today’s day and age. The work that we have creates is meant to discuss the drastically changing landscape of romance in the 21st century in comparison to the age old sexual motivators found in human nature.

CLICK HERE TO READ THE SHORT STORY THAT INSPIRED THE WORK

How it Works

The Board itself is connected to an Arduino UNO and MAXUINO, it has eight wired buttons and two conductive switches that can be activated by touch. When activated in order the buttons tell a story through audio and visual assets that we created with After Effects and Audition. In between the video sequences the user will be prompted to make decisions about the story and characters, however regardless of what they choose the story will end the same way. The user interaction is intended to reflect the frustrated, sometimes powerless feeling that Roupenian’s main character Margot feels in the short story “Cat Person”.

Our Process

Materials:

  • Copper tape
  • 8 sensors(button, sound, mic, pressure, etc)
  • Speaker
  • Thin Plate of Wood

Equipment:

  • Projector
  • Arduino Uno
  • Soldering iron
  • Speakers

Software Used:

  • Adobe Premiere
  • Arduino
  • Maxuino
  • Touchdesigner
  • Adobe Illustrator

1.         Brainstorming

We came up with a lot of ideas, and potential topics to explore by simply making a flowchart to help interrogate various elements of the UX/UI design. One of the main things we wanted to be part of the experience was the ability for the user to leave input.

dsc_0193

dsc_0194

dsc_0196

image_123923953

image_123923953-1

2.         Voice Acting and Recording

Ola was able to take the original narrative and construct a new script that was simplified and more compatible with the aims of our project. Using the new and improved script we headed to the recording studio where we worked together to record the voice over narration. Next, we made a list of background sounds we wanted add to the background narration and we recorded those too. Sam took the raw audio files and mixed them together to create 8 separate audio tracks for the final output.

BEHIND THE SCENES FOOTAGE

3.         Visuals and Animation

Ola worked in After Effects to create 8 individual animation files that would play with the push of the buttons. However, all of the files were corrupted and the work was lost resulting in a very frustrated Ola. Luckily, Sam came up with the idea of using colors rather than visual scenarios to communicate the ideas and emotions in the story. Together Ola and Sam did some research on Color Theory and applied the knowledge to the visual element of the project.

 

mood-color-chart

4.         Technical Work

Tyra put together the user experience design framework and the Max patches required to activate the functionality of the board. Tyra began by laying out the exact flow and structure of the user experience via pen and paper sketch wire framing. The process began with designing the layout of the board itself, as well as the content and syntax of prompted user interactions. The main Max patch consists of eight buttons that open and play media files when activated, each button has a different video and sound that becomes part of the overall story. Within the main patch is a sub patch, this sub patch is connected to the ‘swipe left’ and ‘swipe right’ feature of the board that makes it interactive.

The sub patch is responsible for opening and playing video files from a two separate lists. When the user is prompted they will have to choose to ‘swipe left’ or ‘swipe right’, whatever they choose will activate the next video file from the corresponding list. At the same time, the sub patch will eliminate the video file that was not played because it was not chosen by the user. This was an attempt to sequence the movie files and make sure they only play given the right conditions. However, the interactive functionality of this project was not developed as well as it could have been. Unfortunately, the Max subscription Tyra was using was unlicensed and ran out before she could debug the problem with the sub patch. Overall, the sub patch does achieve its base function of movie sequencing, however it does not eliminate the unselected video files which is necessary to maintain the syntax of the story progression.

img_5221

5.         Rapid Prototyping

Tyra  designed the physical board that holds the electronic components using Illustrator and a Laser Cutter from the Rapid Prototyping Center. Ola designed the circuit and put it all together. 

screen-shot-2019-03-11-at-8-56-31-pm

 

img_5225

20190228_142502

6.         Putting it all together

Together we integrated the circuit into the physical prototype board, then we plugged in the Arduino and connected to the Max patcher. Finally, we hooked up the projector and speakers so that the visual and audio experience could be shifted from the centrality of screen to the openness of the physical world.

Reflection

The story takes place over time, but is told over the course of a few button presses from the user, filling and concentrating the space with feelings of nostalgia, relatability and humour. Through these fragmented experiences the interpretation of the work can vary from view point to view point making each interaction with the space unique. For example, someone who relates to the story might understand the place in which it is situated as somewhere where they belong and feel heard. Whereas, someone who is confused or puzzled by the story might interpret the space as a distant, maybe even emotionally attacking.

dsc_0199

Pub Trivia

Tyra D’Costa | Shiloh Light – Barnes | Omar Qureshi

Pub trivia is a decentralised trivia game that is targeted towards groups of friends or pub – goers who want to play trivia, where trivia is not being offered. This means there is no single command screen that  has exclusive control over the data or processes of the game, the game will b dynamic as players communicate data between each other. Our trivia game requires each member to Join a team, they can do this individually or collaboratively. Once all the players have joined the game the questions will begin and the console will log the players score, the top three players, and which players have guessed wrong or correctly. The trivia game we put together focuses on music and orchestral music facts from the early 1800’s to the late 1900’s.  Ideally, the player would have a multitude of topics and themes to choose from, but for the sake of demonstration our project we chose to focus on the topic of a random non fiction book we were given. This documentation seeks to display the key concepts and functionality required to develop a decentralised trivia style game app, or webpage.

See the code here: https://github.com/qureshiomar/atelier-ii-book

We decided to use Pubnub to develop the backend because its Data Stream Network (DSN) and real time data transfer capabilities are particularly useful when trying to get the players device to communicate effectively between each other. We also used P5.js to develop the backend functionality because it is what we are most familiar with and it is fairly simple to build the functions and structures that were necessary. For the front end development we formatted with HTML and CSS. We did this because we wanted to separate the  the visual aspects of the code and the functional aspects to avoid errors, and organise the code more effectively. We started with front end development, which consisted of Illustrator mockups, and the p5.js DOM library. We then began to develop a loose wireframe of pure HTML buttons, labels, and inputs – which would serve as a playground environment to test whether PubNub would be a viable option to move forward.

 

screen-shot-2019-02-12-at-7-22-05-pm

We concluded with a simple PubNub system of sending messages on one channel to join a game, and another channel to answer questions. This meant that each channel could receive a different object storing specific information. Each time a player answers a question, for example, a message is sent to all other users containing the players name, and the letter that they have selected (A,B,C,D). The computers will then continue to receive answers until the total number of answers is equal to the total number of players. Once the last answer arrives, the computers will then remove all the elements from the screen and generate the so-called results screen. From there, a simple three second timer returns all players to the question screen. The computer will continue to select questions from the array until the number of questions is equal to the variable. Overall the the development process went relatively smoothly, except for one problem we faced which restricted the ability to have a dynamic player number. We wanted the ability to invite a dynamic number of players into the game depending on the circumstance, but due to us not being able to integrate with PubNub “presence” features, we ended up having to go with a fixed “numberOfPlayers” variable. If we decided to continue with this project, we will focus on integrating a dynamic player account feature, as well as improving General bug fixes and UI elements.

CSS Reference:  ferrysdayoff et al,. Jun 11, 2018, 10:16:47 PM. “Styling HTML Forms”. [Web Article] Retrieved from:https://developer.mozilla.org/en-US/docs/Learn/HTML/Forms/Styling_HTML_forms

  • Explains how most of the CSS was done. Including how to align buttons and input bars, edit button properties and add svg files to code.

Javascript Reference: Berhanu, Yaphi. May 23 2017. “How to Make a Simple Javascript Quiz. [Web Article]. Retrieved from: https://www.sitepoint.com/simple-javascript-quiz/

  • Explains the basic functionality, problem solving tasks, and features that need to be developed in order to create a trivia game. For example, we learned that we would need to use a loop to fill in the answer choices for the current question.

CSS Reference:  ferrysdayoff et al,. Jun 11, 2018, 10:16:47 PM. “Styling HTML Forms”. [Web Article] Retrieved from:https://developer.mozilla.org/en-US/docs/Learn/HTML/Forms/Styling_HTML_forms

Javascript Reference: Berhanu, Yaphi. May 23 2017. “How to Make a Simple Javascript Quiz. [Web Article]. Retrieved from: https://www.sitepoint.com/simple-javascript-quiz/
Pub nub Reference: 
Pub nub. Aug 30th, 2016 at 4:57PM. “Solutions Home”. [Webpage]. Retrieved from: https://support.pubnub.com/support/solutions/articles/14000043857-here-now-
screen-shot-2019-01-23-at-7-03-57-am
Inspiration + Ideating   
kahoot pubtrvia

Once we chose “The story of Orchestral Music and its Times” as our story to base the project on, we contemplated the different interactive forms in which players could express their knowledge of orchestral history. Inspired by quiz games such as Pub Trivia, Kahoot, and HQ Trivia, we decided on a real-time multiplayer trivia game. From there we went back-and-forth between simply asking question after question and displaying result at the end, or showing a continuous real-time results screen containing information about players current score. We ended up going with the continuous option to keep players engaged and competitive with one another.

 

The Soundscape Experience

The Soundscape Experience


Tyra D’Costa | Shiloh Barnes | Francisco Samayoa

screen-shot-2018-11-26-at-6-19-54-pm

Project Inspiration

Our project seeks to encompass elements of sound, sensory perception, and nature in a installation based artwork. Our process began by discussing the types of sensors we wanted to explore, eventually we decided on the MPU 9250 Accelerometer and Gyroscope. Our proposed idea, is to integrate the sensor into headphones, which can then change the sound and visual experiences of the user. As the user navigates the room and moves their head about, the world in which they are digitally immersed in will enhance the real world.

Goals

  1. Engaging UX Design: We want to solidify the conceptual ideas of the soundscape experience to create an immersive UI/UX design.We want this to include, elements of storytelling and gamification.
  2. Sound-Sensory Experience: Moving forward we will have to test that the sensor can allow smooth transition between sound files. The media assets will consist of ethereal soundtracks, data visualizations (natural algorithms and patterns), and nature related images.
  3. Integrate wearables: Also, we need design a way to integrate the sensors into the wearable technology ( the headphones).
  4. Discuss concepts of nature, ecosystems and natural algorithms: Lastly, we want to think about how these concepts can work together to create a narrative and gameplay element.

Background Work

So far, we have able to get the MPU sensor working with Arduino so that we receive X,Y,Z coordinates. This required, scouring the web for solutions to library related issues, and calibrating the code in order to receive accurate data readings. Additionally, we were able to take these coordinates and map them to a camera in Unity, so that the sensors orientation changes the perspective of the viewer.

Links To Contextual Work

How to play multiple audio files

Research on Arduino + Gyroscope

Good Link explaining how IMU sensor works

Tutorial for IMU and Unity set up

How to set up MPU and Arduino Library

Unity + Arduino Library

Project Details

  1. User starts at any location on the map
  2. They put on the headphones
  3. Depending on the position of the head, the headphones will play a particular sound.
  4. If a particular sound is playing then the screen will display an associated image or video
  5. The user must experience all the audio and visual data in order to try and make a connection between the media and a place that exists in the real world before their opponent. 

Sketches

img_4916img_4924img_4924img_4926

Process

46837114_564283070696082_6498849018557235200_n 46812922_473012876557775_1587641447014727680_n

46801407_337187233779982_3353290089944842240_n 46854703_644056072656401_7774253524338081792_n 46796952_337902460325535_4724015759863316480_n

Blast From The Past Arcade

 

screen-shot-2018-11-19-at-1-36-18-pm


Tyra D’Costa | Ola Soszynski | Kiana Romeo | Dimitra Grovestine

Project Description 

For the third experiment, our team studied a variety of conductive materials that could be used as conductive sensors. We applied these sensors to wearable based controllers, in order to create a larger purpose for them in terms of their use in gameplay, and design for health. Our process began with a discussion on a topic we all found nostalgic- our childhood. We all remembered playing  Miniclip and Disney games when they first came out. Being one of the first generations to grow up with technology, this topic seemed both fascinating and relevant to us. As we reminisced on our favorite childhood games, we also began to talk about the very real effects they had on social health of the millennial generation. Many of us remember rejecting the park or our friends to stay inside and play computer games or watch Netflix. Soon after, social media would further this drastic change in how children play, interact and socialise.

screen-shot-2018-11-19-at-1-39-12-pm

We also discussed some of the stereotypes and harmful effects associated with gaming for long periods of time. These effects may include:  anti-social behaviour, stress in the tendons of the hands and forearms, reduced mobility etc. Our focus was to think about how we could reintroduce the games we loved so much as children, while addressing some of these issues.  We wanted to create buttons that either engaged a group of individuals together, or encouraged an individual to engage in physical activity. However, we wanted to ensure that these purposed engagements did not have negative longterm effects on the body, and that the controllers were ergonomically designed to  enhance the users experience.

img-4901

Materials and Design

Using a Makey Makey micro controller eliminated the need for any code based work,  we really wanted to focus our attention to building and designing the conceptual aesthetic  of our project. Instead of making a single game controller we decided to make serval and host a 2000’s themed arcade. This involved analysing various elements of the games we chose. We observed the movements, directions and positions of each game in order to design controllers that were intuitive to the player and met the standard of our design goals.Our team concluded that it would useful to replace the traditional controller with wearable controllers in order to optimise the fluidity and interactivity of the designs. We thought it would be fun to make the controllers funny outfits from the early 2000’s, so we created mock ups of potential outfits and brainstormed design ideas for how they could be transformed into game controllers. The conductive thread became very useful when we wanted to hide wires and create a more seamless look.  To contribute to the overall theme of childhood fun and nostalgia we decided to add prizes and a ticket booth. This added a whole new dimension to the arcade, bringing a sense of good spirited competition and challenge to the user experience.

Design Goals 

  • Economic Design
  • Addresses social issues
  • Design for health
  • Interactive user experience
  • Kinetic user experience
  • Childhood Fun and Nostalgic Themes

Materials

  • Used clothing ,
  • Makey- Makey micro controller,
  • Iron on conductive fabric,
  • Conductive thread,
  • Aligator Clips

Outfit Mockups

2003 - Rocker Look
2003 – Rocker Look
2006 - The Dude
2006 – The Dude
2010 - The Hispter
2010 – The Hipster
First Sketch of the Arcade User Interaction Design
First Sketch of the Arcade User Interaction Design

How it all Works

In the Blast from the Past Arcade, every game we had used the Makey Makey, a small board that works like a USB and sends WASD/ left, right, up, down and spacebar signals to the computer. When the circuit is closed for any of these parts of the board, it sends the command to the computer being used as if the button was pressed on the keyboard.

Makey Makey Pluggeded into our Happy Wheels game contollers.
Makey Makey Pluggeded into our Happy Wheels game contollers.

Using the Makey Makey was quite easy since when connected to a computer it takes over the keyboard controls that would have been used otherwise. This allows for more movement in playing video games because the player is not required to sit directly in front of a screen and push buttons, they can get up and move around depending on how it is hooked up. As seen in the photo of the Makey Makey below, it can be seen how simple alligator clips and wires at various lengths can be attached in order to make more interesting controllers.

Makey-Makey Reference page

When performing the first test for the Makey Makey, we used it on the games we performed our research on, games that required an assortment of keyboard controls. The first game we tested was Super Mario Bros as this is a classic keyboard/ controller game. 

Video Documentation – Learning to use the Makey-Makey: Our First Success

Finally, when we put together the games, we had to ensure the controls on the Makey Makey corresponded with those for the specific game. For example, with the Sandwich Stacker game, the “left” and “right” arrow controls on the Makey Makey were connected to the two players wearing the suspenders and glasses while “ground” was connected to the player wearing the bow tie. Therefore the person who was grounded would simply have to make skin contact with the other players to activate the left and right controls.

glasses controller ( far left ), Bowtie contoller (middle), and Suspenders contoller (far right) in use.
glasses controller ( far left ), Bowtie contoller (middle), and Suspenders contoller (far right) in use.

With regards to the Club Penguin Dance Game, the player was connected to ground on each of their hands through the edgy gloves. The player wore a rock’n’roll t-shirt with two conductive iron on shoulder pads. These two shoulder pads were connected to the up and down arrows. Finally, the side movements were controlled by iron on conductible fabric on the hips of the belt. The gloves, not only were grounding themselves, but they also grounded the entire human body. Prior to getting the gloves to work, we had tested out using a single wristband. Because the wristband grounded the entire body, players could still use either hand to complete the switch and produce the proper game movement.

Edgy gloves is used as a ground to complete the circuit.
  Edgy gloves is used as a ground to complete the circuit.
The full Edgy Club Penguin Dance game controllers outfit
The full Edgy Club Penguin Dance game controllers outfit

For the Happy wheels game we  made two floor based controllers modelled after car pedals, we thought this was an inherently intuitive design for the acceleration and break.  If the user is wearing the conductive ground wrist bands they can touch the left and right sides of the hat, as well as the centre of the bandana to front flip, back flip and reverse. While designing these game controllers we paid close attention to our research to ensure that the design was safe for long periods of game time. We made sure to keep the neck aligned, the bean bag chair came in handy for this as we were able to adjust the game screen as needed. Secondly, the design allows the elbows to always be bent at either sides of the player when playing. Lastly,  the hand of the player is always in the same plane as the forearm which is key to reliving stress points in the hands and tendons.

Game controllers for the Happy Wheel Game
Game controllers for the Happy Wheel Game
Floor pedals for Happy Wheels game
Floor pedals for Happy Wheels game

Research

When conducting research for our Blast from the Past Arcade, we had to consider many factors. Firstly and possibly the most important part of our project we had to carefully select games that we would use within it. These games had to have certain criteria:

Nostalgic games from different eras of our childhood:

We had to ensure that the games would resonate with everyone who tested our arcade and also with us in particular. We tested many games and had to dig deep in the internet to find these games (Some of them are over a decade old!) furthermore, since the arcade was about revisiting the past, we had to exclude some of our beloved games as they coincided with the same era. Research on the game release dates was necessary so we could successfully spread our arcade across many years (around 2005- 2012).

Games that use any combination of keyboard controls (spacebar, left, right, up, down buttons):

Although we had many games in mind, the Makey Makey requires keyboard controls to work. Therefore, a game with mouse movements would not work at all while a platform game using the left and right keys would. Furthermore, while researching, our group decided that instead of looking for games that could potentially use mouse movements, it was more authentic to use games with keyboard controls.

Links to games we had tested with the Makey Makey (games highlighted are the ones we used in the final prototype):

Ergonomics

The next step in our research was to figure out the science and implications of the way movements of the body affect our health and also how different game controllers and the way we hold them effects game play. The most logical way to go to find info on this was to look for research conducted by e-sports specialists. Our research found that posture and ergonomics are directly related to successful game play. Certain postures and movements provide the body with relief from cramps and muscle pain. Furthermore, when playing video games, watching the screen is a very important part of the experience but it’s also known that staring at screens for long periods of time can strain the eyes. By making the placement of the game eyelevel and the screen at least an arm length away from the player, the gaming experience will be a lot better. As well, wrist and finger positioning should be in such a way that it makes it easy and intuitive for someone to use.

When we designed our interfaces, we took all of these facts into account. For example, with the Happy Wheels game, one would sit on a comfortable bean bag in which they could position it to their liking. The controls included intuitive foot pedals with one used to step on the gas and move the character on the screen forward and one used to break. This freed up the players hands so that they could reach up to the hat and press it to control certain things in the game and lastly the bandana on the chest of the player one would simply touch to activate. We thought carefully about the controls with all of our games, both relating the functionality with the theme of the game itself and making sure the controls were comfortable and ergonomic enough to make playing more fun.

Research sources:

http://www.1-hp.org/2016/10/28/esports-health-it-starts-with-ergonomics-and-posture/

https://medium.com/what-the-tech/ergonomics-of-gaming-gear-a07058f88bf3

Documentation

Click to see Club Penguin Dance + Rocker Look

Activates the body and encourages users to move and dance in order to gain pints, win tickets and get prizes!

Creating the 'Edgy Gloves'
Creating the ‘Edgy Gloves’

Click to see Happy Wheels + The Dude

Encourages the user to step outside of the traditional hand positions found in most game controllers. This design relives stress that could build in the the tendons and muscles of the hands.

Creating the floor pedals
Creating the Happy Wheels floor pedals

Click to see Sandwich Stacker + The Hipster

In this game users have to interact and work as a team to keep the game going, beat high scores and win big!

Wiring up the Sandwich Stacker game with Conductive Thread
Wiring up the Sandwich Stacker game with Conductive Thread

*Click to See the Full Arcade Experience *

A group of 3 working together to set a high score on sandwich stacker
A group of 3 working together to set a high score on sandwich stacker
Nick all ready and wired up to play club penguin dance.
Nick all ready and wired up to play club penguin dance.
Explaining the how to use the game controllers.
Explaining the how to use the game controllers.
Ola explains how to use the game controllers for the sandwich stacker game
Ola explains how to use the game controllers for the sandwich stacker game