Mika Hirata

3154546

• Luna

Luna was a calm and relaxing experience with amazing sound effects. The visual was extremely beautiful and the details made me feel like I was in a painting. At first, I found the control a little bit hard to understand, since the grabbing motion only worked on some objects. Also I started from the middle of the game, so it was hard to understand I had to clear the puzzles to move on the to the next level. Since the game has puzzles, it got more depth of storytelling rather than just a pretty settle environment. Particle effects are used in many objects, so that the whole experience was very magical. Also as Max pointed out, this experience is made for one person only who is viewing the game in the VR headset. Unlike the other games, the people surrounding the player were not able to understand or join the magical feelings, so it was an isolated experience.

• Beat Saver

I really enjoyed playing the beat saver, because I have watched a girl playing this game on Facebook and I have always wanted to try this game. The game itself is very simple and easy to understand. The user experience and user interface were both very friendly, so I was able to play it without any directions. Also the sound effects were amazing, which made game very exciting. I realized that while the player is playing this game, people around the player were also having fun and they were engaged to the game. Moreover, the scoring system made the game very competitive and enjoyable for many people, so it was not an isolated experience like Luna.

• Lone Echo

Lone Echo was a very interesting and unique experience, which allowed me to feel like I was actually in the space. The 3D models that were used in the game were extremely impressive and the quality was amazing. I really liked the dialogue of the story, so I was able to become the captain of the space shuttle. The movement was non-gravitational, so it made the whole experience more realistic and exciting. I would love to create something like this for the next project, and it was differently very inspirational in terms of having a clear concept for the environment.

By Erika Davis, Ziyi Wang, and Natalie Le Huenen

This project is open source. View the entire project on GitHub:
https://github.com/natalielh/unity_window_house

Project Description:

Window House is an interactive digital world where viewers explore surreal fictional environments. Our aim was to create something that’s timeless and can be experienced universally. Free of any form of language, all interactions involve visual and auditory symbols. From there, viewers are encouraged to create their own interpretations. Visual aesthetics and pleasant ambient music are emphasized to create a sense of harmony. The environmental design and architecture also reflects the dreamlike quality of existing fantasy worlds found in digital games. The lighting is exaggerated to make the environment feel more dynamic, and also gives the viewer a hint as to where to proceed to the next area. Glowing ‘portals’ fill the screen with light and transport the viewer upon interaction.

One particular theme we wanted to look at is adding unexpected movement to objects that are always static in the real world. Experimenting with digital tools allows us to explore interactions that are physically impossible. Building our world using Unity allowed us to adapt modified 3d game physics and mechanics to our custom use.

Window House aims to create a deep feeling of mystery and curiosity. We decided to continue developing upon our original project, which was a single room filled with mysterious pots, acting as “windows” allowing you to peer into different worlds. Images on pottery (something that is always still in the real world) move and come to life when you approach or look at them.

Process:

3D modelling the first scene

Experimenting with lighting and post-processing effects

Experimenting with Unity’s “Post-Processing Stack” allowed us to boost the look of ambient lighting in the scenes and push the detailing to its highest potential. Creating a filmic look, adding a ‘glow’ to the lights, creating a subtle lens-dirt effect and adding a slight vignette helps add immersion.

Using baked lighting involves rendering lighting effects and applying them as textures to models in the scene. We used baked lighting and light probes for a more realistic effect.

Light cookies are black-and-white textures used to represent the light cast by emitters onto objects in the scene. We created custom light cookies to tweak the appearance of all window and portal lights in the scenes.

Programming movement into the textures: Adding movement to the art on the pottery involved modifying the ‘offset’ value of each texture every frame. Achieving a ‘parallax scrolling’ effect involves setting the scrolling speed value of each texture at different values, giving the illusion of 3d movement.

Parallax scrolling in a texture

Creating transparent rectangular textures to wrap around meshes

Programming player interaction: The pottery reacts to the player’s movement. The texture-offsetting algorithm stores all the data about the player’s 3d position, the player’s viewing angle, and the 3d position of each pot. To simplify calculations, this 3d data is projected onto a 2d surface where all the vector calculations happen.

The custom script accepts values to tweak the calculations to create a result that feels the most intuitive for viewers.

View the texture offsetter script:
https://github.com/natalielh/unity_window_house/blob/master/unityproj_windowhouse/Assets/Scripts/PotTextureOffsetter.cs

A glowing portal

Transitioning the player between the three scenes involves using a ‘portal’ system. Upon colliding with hitboxes designated as portal colliders, the next scene is loaded.

View the scene transition script:
https://github.com/natalielh/unity_window_house/blob/master/unityproj_windowhouse/Assets/Scripts/LoadOnCollision.cs

The animated window light

Creating the animated window light involved making a keyframe animation for the light’s rotation and colour. This lets players witness the passage of time more dynamically. The “Volumetric Fog” package was also added to add to the effect.

Creating water

We used a modified version of our texture offsetter script to animate the fountain water. A simple water texture is wrapped around a cylinder model with the top and bottom faces removed, and is scrolled to give the illusion of falling water. A water splashing sound effect is also present in the center of the fountain. The closer to the fountain the viewer is, the more prominent the sound effect is.

Creating the second scene:

Final images of the second scene

Studying and testing the structure of different vaults in Rhino

Creating the staircase

Putting little parts together

Modeling columns

The second scene is a very open space allowing you to look at the stars outside. They move across the sky very quickly, similar to the rotating light in the first scene. This is accomplished by offsetting the skybox’s texture every frame.

An early view of the starry skybox outdoors

Creating the third scene:

The skybox of the third scene was hand-painted and also rotates. The scene uses light and fog tweaked to match the skybox, creating an intense sense of depth for the monolithic structure.

A view of the final scene

Walking on the bottom creates a feeling of ‘floating’ as you can float over empty holes in the architecture.

Builds:

Builds for Windows and Mac:
https://drive.google.com/open?id=1yfnGI5cHmBL-O3G3Oee-MWvk4zVo-hDF

By Harit Lad, Sanmeet Chahil and William Selviz

Abstract

D E S K, a VR experience showcasing the life of a former office worker packing up their most dearest, yet mundane, belongings with a V A P O R W A V E twist.

Project Description

We developed this VR experiment after deconstructing the human experience we call nostalgia. This feeling needs a host, whether its an object, aesthetics or art piece, in order to remain meaningful over time. The team was inspired by the lo-fi or “vaporwave” aesthetic. This internet subculture uses nostalgia as a tool to evoke satirical melancholy, combining 80s and 90s imagery with current trends, events, and technology. The idea of reminiscing about the past with an existentially neutral perspective seems more appealing than living through our painful realities. Nostalgia has the power to take us back to a moment in time, and even make us say “those were the good times”. Combining this concept with VR technology allowed us to construct a paradigm-shifting interpretation of this unlucky, yet relatable, event. Vaporwave is dead, long live vaporwave.

Builds

For Mac OS – Click here

For Windows – Click here

Documentation

Our Final Documentation can be found here

Mika Hirata 3154546, Vivian Wong 3158686, Anran Zhou 3157820

Concept

HOKUSAI is an interactive art installation experience of culturally influenced art illustrating the evolution of the famous Japanese artist, Katsushika Hokusai.

Projected on a paper-like hanging screen, we bring Hokusai’s artwork to life, in an inviting, open, interactive space. Layered in a parallax formation to display depth, animations are activated with motion detected by a Kinect. The installation features reworked and refined digital paintings, special sound effects, the artist’s quotes, and self-composed, culturally-influenced music.

Beginning with a looping animated scene of the artist’s name, the audience is invited to approach the piece. Once a user is detected by the Kinect, the screen reveals a dynamic representation of Hokusai’s painting, The Great Wave of Kanagawa. The viewer can interact with the painting, as the waves and particles follow the viewer’s horizontal location. The closer the viewer gets to the painting, the louder the accompanying music and waves become. The audience is directly invited to experience the culture of the artist. As an interactive installation, the viewer is no longer only the audience, rather, the viewer becomes a part of the piece.

Inspiration

For our final project, we decided to continue our previous VR project using a completely different medium. Due to our lack of knowledge with Unity and VR, we struggled greatly with developing the previous project. As a result, we decided to take this project into Processing, a program that our team has much more experience with. We also wanted to represent our concept in a more digital way.

Process

We originally had six different scenes in the VR version, but decided to only work with one scene for the final project. We dissected the scene into separate layers to be manipulated by user interactions using Photoshop. We determined our interactions and visuals in a storyboard initially.

Separated layers of the bottom and high waves in the scene.

Initial Storyboard

In After Effects, we first attempted to create videos with quotes in a calligraphy-like style. The writing animations ended up taking too long to produce for each scene, so we used special ink-like effects to recreate it.

We started putting the scene together by importing libraries, adding the layers of images one at a time. Once we had a basic setup of waves figured out, we included the SimpleOpenNI user3D example template. To get the interaction to work, we replaced the x-axis locations of specific images with the userX center of mass variable. We used mouseX to prototype our sketch motion animations when we did not have the Kinect at hand. We learned to calibrate the Kinect with the map function.

We created functions to oscillate boats back and forth, to play videos, and to fade in scenes and assets. We also created separate sketches for the Processing particles animation. To clean up the sketch, everything was organized to have its own function.

Code on Github

Sparkles on the drawing: https://github.com/0102mika/sparkles/blob/master/mouseInteraction.pde

Main Sketch: https://github.com/vee97/HOKUSAI/blob/master/Hokusai_New_O.pde

Testing

Testing and determining the best way to setup the piece. Found it effective to project onto a large hanging sheet of mylar with a mirrored display, so that viewers can interact with the piece up close, without blocking the projection.

https://drive.google.com/file/d/1wsXkSNpNQ6wLVsPeB1TGPiNgNeOZ2Ozq/view?usp=sharing

Challenges

Hand Detection

Initially, we planned to add sparkle animation following the hand waving position. We were trying to use User3D example file for hand detection but it doesn’t work properly when we combine the code to our sketch, so in the end we decided to make the sparkle animation follow body movement.

We also often had trouble determining how to use the Kinect Processing library, SimpleOpenNI. While it was simple to use with the default example sketches, it became more challenging when combining it with our own scene, due to the several translate() and perspective() functions it came with. The scene resolution and display was often altered due to these issues. Sometimes the scene was mirrored or displayed a zoomed in version of the original image unintentionally. We had to start the sketch from scratch, testing everytime we changed or added new code several times to identify the issue.

We had attempted to add a third interaction to our installation. Originally, the number of users detected by the Kinect could affect the scene as well. Depending on the number of users detected, the boats would oscillate with different speeds. The more users detected, the faster the boats would oscillate. Unfortunately it did not work, the boats spun around in 360 unrealistically. With the final sketch, the boats would also stop oscillating after the sketch resets, which we could not debug.

Sketch of the final outcome:

How we locate the kinect, projector and user interaction.

On the day of presentation, we noticed that our sketch ran with a lag. However, during our testing and development sessions, we never encountered this error. This could have been due to the resolution change, or too much going on in the sketch that Processing could no longer handle it. We lowered the count of the particles in the sparkle animation, which helped a bit.

Final Product

Final Product Videos:

https://drive.google.com/file/d/1lRQGLSMWRStfo08hYR2y8f85hx84Uwrw/view?usp=sharing

https://drive.google.com/file/d/1IaM0RFK8OZ57deaPZS6VQRpmNBUxQ_RG/view?usp=sharing

https://drive.google.com/file/d/1mEz7g0c5xiDzYZxxuEtnHwqETpMbQjAx/view?usp=sharing

Master Folder: https://drive.google.com/open?id=1CKJDuclJwnhc52yPIAcPBisZTPIJI8v-

Conclusion

The interactive art installation approach proved to be more effective than the VR piece. By simplifying and reducing the project to one scene, we could focus on making the scene’s interactions and visuals more appealing. Taking the project further, we would refine the animations, continue to debug the code, remove the lag, and focus on improving the viewer relation to the installation. We gained valuable knowledge and experience from working with the Kinect and Processing throughout the development of this project.

Sydney Pallister, Julianne Quiday, Pandy Ma, Vivian Fu, Tania Samokhvalova

Project Concept: The focus was to create a small indoor space that players could explore and find a narrative within. The game followed two key concepts: storytelling and the transition of object agency. Together, we designed a room to tell the personal story of two people and their relationship with death and grief. Players would be able to explore this ordinary room and learn along what happened. We decided to use voice lines to guide the story forward. This allowed us to create a second character while leaving the distinction of the characters unclear.

Altogether, When You’re Gone explores the passage of grief and how it can transform the outlook of life. The player, alongside the main character eventually experiences survivor’s guilt and the inability to move on.

In order to manage in the given time frame, we created a list of tasks and split them into weeks. Items of priority would be completed in the first week to allow for more time in the final week. Each week we were able to complete a new build of the game and finally reach our final product.

Maya modeling/assets: Initially, the plan was to create an ordinary room with various trinkets and objects to narrate the story. Together, we created a list of possible things to be found in a room and the meaning that could be tied to them. From there, we decided how much we could accomplish in the given time and how many objects would be needed to successfully tell the story. Finally, we decided on seven significant objects.

The second half of modelling was about creating the room. First, we decided on a floor plan and took furniture references from Ikea catalogs. Then we assigned one person to work on the furniture, while another worked on the interactive objects. The room that we all decided on was the bedroom, which allowed multiple objects with backstories. The build consisted of complete models with found royalty free textures. As we progressed to the final stages of the second iteration, the majority of objects have had their textures UV mapped in order to ensure there would be no texture glitches as all as having custom textures. All works models were then moved into Unity to assemble the room.

2D assets/drawings: The overall goal was to move away from using stock photos and having our own artworks. To accomplish this, we assigned to people to work on painting photos and the skybox. These drawings included newspaper prints, photographs, book covers, and posters. By creating our own works, it allowed us to have more personal pieces that tied together with the story.

Unity UI/GUI: The first version of the game started out as just the game itself; the player starts in the room and ends with the final line of the game. There were no clear signs as to when the game would begin and conclude. As a result, we decided to add UI menus throughout the game in order to create a better flow for the players experience.

The Start menu was added first to give the player options to actually play the game, learn the story behind it, or exit the game. The Controls and Credits tabs were added to provide guidance for players who have never used a controller before (a fact we learned at the TSV open show). We chose from two different backgrounds, each an image of the outdoor terrain of the game.

The next menu added was the Pause menu which was a gateway for the players to momentarily leave the game if desired. The Pause menu would appear when the player hit escape, and the options would be to resume or quit the game.

The last menu added was the Restart menu to give a clear indication of when the game ended. A black screen would fade in after the final audio line of the game and the player was able to go back to the main menu or quit the game.

There were definitely some obstacles when creating these menus, some of which were unavoidable. With two people running Unity for the coding aspect, it was a challenge trying to send the menus back and forth and then directing the other on how to add them in from scratch. Converting from mouse and keyboard to controller also took some time because of player options that were not enabled in the Unity inspector. However, being able to finally add these menus was definitely an accomplishment and also created a natural flow to the game.
Unity Builds/Coding: This project started with a coding base that we made for our project The Biography of Things. The initial code simply triggered the voicelines to play when players clicked on the correct object. Voicelines corresponded with specific objects and were tagged accordingly, and after a certain amount of objects had been clicked, a counter was set to trigger a new set of voice lines after each object clicked in succession.

For the improvements of this project, a main issue was that there was no way of stopping a player from clicking on all the other objects and having the the voicelines all play all at once. This issue was fixed by creating a counter that would add a one while the line was playing, and if the counter was equal to one then players couldn’t click on any other objects. Once the line was finished, the counter subtracted back to zero, allowing players to click on other objects and trigger the cycle once again.

Early into testing it became obvious that players required an indicator to know when the cursor was pointed at an object. Many different scripts were tested to change the material color of the object but had issues with the compatibility of the raycast script. Instead, we worked with particle effects. Once the player had the cursor focused on the object and the raycast hit, the particle effect would change to a blue color. This was done by referencing the particle effect with the corresponding object name and changing its start color to blue when it was looked at, and white when it was not. Once it was clicked, the particle effect was set to stop.

The next steps were to customize the post processing settings as well as the FPS controller. Many people when they played the game found that the player controller moved and looked around the room way too quickly, and that it took you out of the game. The settings were then adjusted accordingly to these suggestions. The head bob was set to be less dramatic, and various walk speeds and sensitivity were tested until it was better suited to the game itself.

Research was done on the post processing stack settings, what each one accomplished and what feel they gave the room. The various values and options were tweaked until a suitable color palette and lens filtering system matched with the art of the game and accented its theme.

The custom skybox we made presented a unique challenge, as none of our group members have experience with making them. First attempts were to make a cubemap, and set the material as a six sided image. This worked, and the skybox could be rotated with a simple script to make it seem like the clouds were moving. The issue with this way of doing it, however, very clearly made the sky look like an actual box.

There was a very clear distinction in the vertices in the cubemap, which ruined the immersion of the game and looked very unclean. The next step was looking into having it mapped onto an inverted sphere and then once Tania, the game’s background artist, had repainted to fix the distortion this would cause, was to take the material and have it read as a cubemap latitudinal and longitudinal. Initially the cubemap was repainted and attached to a sphere and then inverted in a 3D software (Blender). This left no seams, but left the clouds very stretched.

Finally we used photoshop and converted the cubemap to a horizontal format, and converted that into a panorama which we then imported into Unity as a texture. This solved the issue. Putting an actual inverted sphere into the scene would have as well, except this gave strange coloring issues.

To give the setting a bit more character, people had suggested have ocean sounds and background noises. To do this, two different audio sources were placed outside of the room which allowed them to have their own 3D sound, so they get louder when the player gets closer to the window and have stereo effects that go from ear to ear when headphones are plugged in.

One final step was to make the game input compatible with both mouse and keyboard as well as an Xbox controller. Unity has built in settings that support some of the controller mapping, but the rest must be done manually. Controller input mapping was put into the unity input menu and then coded accordingly. “ || “ Or notation in the code was used so that the game would accept both mouse click and the a button as input.

Progression of Each Build:

Code Gist:

https://gist.github.com/sydney4529/6dc9e0fadf4fbe2403cf12b18cb3afea

https://gist.github.com/sydney4529/47f05638b74c6928fb4c7385e7eb3c64

https://gist.github.com/sydney4529/a72c10e8d6fd74ba05dff11544e09c75

Game Executable:

PC/Windows: https://drive.google.com/open?id=1YJpqllheQBO3sXAF00wPz5sDPzkBM5CO

Mac: https://drive.google.com/open?id=1k2UM8PvaUGikbxwnLuD4uMWYSiF9RoIi

Credits: https://docs.google.com/document/d/170GkBt-zAPrhOrFbaOBiCeSzH-uZG6uAO_oxicuQQ40/edit?usp=sharing

Mustafa Abdel Fattah

Atelier II: Collaboration

Project #4

Project Description

FRAGMENTS is a projection installation created by Mustafa Abdel Fattah. It illustrates the concept of faded experiences and raises the question; if objects had memories what would they be? Following the perspective of a children’s toy, a loop of the toy’s distorted memories is revealed and it’s up to the viewer to interpret them.

FRAGMENTS began as an exploration into the significance of childhood memories to adults and how memories shape adults to become who they are. Associated with one’s childhood are children’s toys; which are known to be linked to some of the deepest of memories for every individual. This piece aimed to show a glimpse of the memories that accompanies a children’s toy and subtly commented on the subject of passing time and growing age. As the concept became refined, FRAGMENTS grew to showcase the passing of time and the diminishing significance of children’s toys in one’s life as toys become just another memory.

Click to View Final PDF Documentation

Wiring Team

DIGF-2005-001

January 25, 2018

Wiring

Process:

https://docs.google.com/presentation/d/1Rf3LAA2yxtv8NxzLapPuzdjaUsADUxSeE42De0c6RFM/edit#slide=id.g2edd56b021_0_44

Things to look in to:

1. Communicating between boards
2. Multiple nanos and multiple screens
3. Connecting the uno with sensors
4. Photoresistor, potentiometer, pressure sensor (just one of each)
5. Communication of sensor nano to all other nanos
6. Redundancy
7. Labelling system
   1. On screens
   2. On Nanos
   3. On wires
   4. Everything
8. Troubleshooting system

Materials Needed:

• Wires
• Photoresistor (1)
• Potentiometer (1)
• Pressure sensor (1)
• Protoboard (1)
• Headers (60)
• Breadboard (1)

What if we used the PWM pins for float and int information (photoresistor, potentiometer, pressure sensor) and Digital out pins for boolean information (such is on, thus is not)

Wiring Legend

Final Documentation:

Initial wiring test:

One of the first steps was to start soldering power headers into protoboards, allowing for the large amount of connections that would be needed. We left a few extra pins, partly for inter-board connections, and partly in case of faulty connections (this came in handy).

At first, we planned to power the OLEDs using the 5V and GND pins on the Arduinos. This was scrapped in favour of keeping separate power headers on the protoboards, making for denser boards, but sturdier connections.

On the initial test day after these three separate protoboards were put together we tested each OLED screen and arduino to make sure that they were functioning properly, and that the connections from the boards were correct. There was one faulty protoboard which meant having to fix the short so that the connections were no longer bridged.

It was also at this time that we added a third header to transfer state information from the central Arduino in the control panel.

Everyone was sent home with an individual screen and nano so they final wiring job besides the protoboard power connections couldn’t be started until the morning of, which was the real challenge.

Wiring to the Domes

Labour for dome-wiring was divided in two sections. One team dealt with the base, and the other simultaneously dealt with the internals of each dome.

We started by drilling an extra hole in the case for the 5V power cable.

The first problem that came up when attempting to wire the Nanos to the OLEDS in each dome, was figuring out the organization of the wires. We only had to colours (red and black) to work with, and this immediately led to some confusion to where everything belonged and connected. We used black-red-black-red as a mnemonic for the OLED wiring pattern. In some instances, this pattern was reversed, leading to a substantial number of burnt-out screens.

The wires got tangled easily and when testing each individual screen that wasn’t working this made it difficult to find the correct connections. This issue was accelerated by our decision to dispense of numeric labelling conventions, connecting any Arduino to any OLED in any dome. Such a choice made troubleshooting less a matter of, “Screen 24 is down, something’s up with Nano 24,” and more a matter of, “This screen is down, check all of the Nanos.”

Troubleshooting

With each individual dome hooked up, we tested if the nanos and screens would power on. Initially, none of the nanos had any power. The DC power jack’s positive and negative terminals had been bridged by spare solder, bypassing the assembly entirely. This was solved by de-soldering the jack completely, cleaning the site of its connection, then re-soldering it with extra care taken.

With that fixed, we realized most of the screens had no power or just would not turn on. In part, this was due to the aforementioned wiring reversals in the OLEDs. The screens which had their wires reversed were burnt out, and needed replacing. This largely happened in the desert dome; the final one to be installed.

Problems in the desert dome were compounded by a lack of distinction between power and signal cables, which was not resolved due to time constraints.

Some screens resisted all troubleshooting. In some cases, swapping out either Arduino or the screen itself solved the problem. In other instances, the issue could be resolved by finding loose jumpers. However, a rare few simply resisted all solutions, and so remained unsolved.

Thoughts on the Prototype

For next time a different organization system of wiring would have made this quicker and more simplified to go through, labelling each wire and OLED with a specific number so we could tell what wasn’t plugged in properly and which ones weren’t functioning correctly and make a note of that. As it was, when we opening up the wiring box when there was clearly a couple of shorts going on, there was no way to sort through them besides the labels that we had put on all of the data cords. At that point we had to individually go through each wire and arduino to see where the problem was and if anything had been fried or plugged in wrong.

As for the OLED screens and creatures themselves, another organization system that would have been beneficial would have been assigning numbers and domes to each person in order to properly know which OLEDs hadn’t been turned into us or were having difficulties.

Wiring Images/ Materials:

https://docs.google.com/document/d/1rawrmO6kliL9iuReL6c0J6kpSfLHgz1fyNdyetbeMzw/edit?usp=sharing

Process photos:

https://drive.google.com/open?id=1U09chxDF3O6l_l2RF31EH_TZpDpDrJty