The Ripples Playground is a fun interactive sketch meant to mesmerize the user by generating colorful, aesthetically pleasing, expanding ‘ripples’ on the screen from the users mouse and input. The ripples speed and color are randomized between a range. There is also a second variant of the ripple called a background ripple that is created on right click which is opaque and has a fill of either white or black. I added this because it gave a kind of wiping the screen look and it looked pretty trippy.
Alongside this, the ripples generate their own unique oscillation frequency, based off their speed. This gives them a slight resemblance of sound or radio waves.
I tried to add a radio button switcher for the oscillation type but since I got the idea last minute experimenting with the types I couldn’t get it fully implemented or cleaned up. You can view the sine and triangle buttons at the bottom of the page.
I chose p5.js for this project because it allowed the sketch to be accessible to lots of devices by not requiring them to download any software and running smoothly as well on many devices. For example, if I used Processing for this I would need to run it in the Processing editor. As well, it allows me to build UI using HTML elements.
The context and idea for this project came from me wanting to experiment and expand on my Sketch_1 with drawing colorful ellipse on the page. After I got them to ripple and expand on the page I thought they kind of looked like sound/radio waves and experimented with the oscillator.
For this project, I wanted to create something that would interact with sound, but I did not want to create the same generic DJ type sound reactive system that has been seen so many times before. Instead, I wanted to make it so that whatever was going on on the screen would be controlled by the user, and the sound made by the user would directly correlate to the resulting animation. I wanted the project to focus on loud sounds so I started brainstorming things that a clap would be able to imitate. Instead of the action causing the sound, I wanted the sound to cause the action!
At first, I had debated on whether making lightning would be a better option. With this I would make an environment in which random lightning strikes would be created when the user clapped their hands, thus mimicking the sound of lightning. But as it was a simulation of nature, it would have been too predictable so I instead chose to make fireworks instead. This way, I could change colour, size, velocity and shape as they are manmade and the way they look does not need to be a certain way in order to recognize what it is.
I relied heavily on these three videos to create my project. Many hours were spent simply watching the videos until the code made sense, and then typing out any code that was given to help with my project. Through watching these videos I learned a lot about particle systems, arrays and functions as well as using the p5 sound library to make interesting, interactive artwork.
For our concept, we wanted to tackle the possibility of text-to-speech through a representation of a synthetic being speaking to the audience. We drew influence from futurists who perceive AI as a possible threat. To represent this, we decided to visualize a face with unsettling features which would speak in monotone similar to previous fantasy representations of a personified AI. Essentially, the prototype runs like this, the user inputs anything from numbers to words and sentences into the text box and then after pressing the enter key, the face would speak through animation. For the face, eyes, and mouth movement, we the p5.play library to visualize the AI and used the p5.play library for the audio aspect and the text-to-speech.
The project itself went through many phases and alterations. A text-to-speech code wasn’t our initial starting position. We still started off with the concept of a series of animated faces reacting to the tune of music. If the music was mellow the face would be different to a song that is upbeat. We had to scrap this concept after encountering difficulties with the microphone as it is limited to picking up on specific frequencies and tunes.
Rationale of the Programming Language
Our group decided to use the famous programming language known as p5.js for our project since we were introduced to this language in the first day of class. Since then, we found out that the p5 language is very flexible and excels at animating objects. Our idea for the final project was based on the 5 experiments assignments where we discovered the p5 library and the vast possibility of features that it unlocks for the canvas. Therefore, we decided to use those add-ons to animate a AI interface. Our code is based on two major add-ons known as p5.play.js and p5.speech.js.
(Issues running in chrome, run HTML file in Firefox or MS.Edge)
For this project, I had decided to expand upon the Atelier tutorials, focusing on the implementation of p5.js. I goal was to create an interactive audio visualizer where the user can input their own tracks and interact with the visuals.
The very beginning of the project focused on the in-class tutorial based around circle visualizer and a circular graph tutorial I found on Youtube: https://www.youtube.com/watch?v=h_aTgOl9J5I&list=PLRqwX-V7Uu6aFcVjlDAkkGIixw70s7jpW&index=10
What led to the final design was based on a tutorial by Yannis Yannakopoulos, where he further explains the p5.sound library and the use of the FFT (Fast Fourier Transform) algorithm.
This eventually leads to the current iteration of the Interactive Audio Visualizer, which is composed of three rings of dots of various sizes that focus on the bass, mid, and treble of the audio track. Much like the circle visualizer, the three rings change in size according to the level of bass, mid, or treble in the audio track. On top of this, a fourth circle composed of lines focused on the bass which is interactive based on the user’s mouse position on the canvas. The lines mimic a camera shutter motion that rotates in place. Below the audio visualizer is a play and pause button, along with an upload button to allow the user to upload their own track to have visualized. The design theme around the visualization is based around a spring bloom concept with regards to colors and floral motifs.
Untitled is a series of VR project experiments of varying degrees of success, with the original aim of creating a VR music video and learning a VR workflow. While the former was not accomplished, the latter was somewhat achieved; I researched several types of VR workflows and semi-successfully implemented two on Google Cardboard: big bang (p5.vr), and AYUTTHAYA (Unity). big bang takes you to an indefinite point in surrealistic low-poly space, while the eponymous AYUTTHAYA places you in a cloudy day in Thailand.
My initial idea was to create a VR music video for Halsey’s Gasoline (Tripled Layered) (https://www.youtube.com/watch?v=fEk-9bOqvoc ). Working with glowing, smoky, audio-reactive spheres (probably would’ve created a particle system and then brought in p5 sound libraries), I wanted to create one animated sphere, giving it personality, then make it grow and rush at the viewer, crowding them in as the audio grows louder. This would then be reflected three times around the viewer, with each sphere following each layer of the audio track. At one point in the song, they’d all transform into snake women (I was going to model them on Blender but now I’m inspired by Mellisa’s snakes to try something in p5). I also wanted to explore 3D sound (which would’ve been related to bringing in the triple-layered audio, but did not have enough time for that. I feel like 3D sound is an essential for VR spaces.
My second concept, born as my time dwindled, was a world where all objects were substituted by its corresponding noun (for example, a sky, but instead of blue and clouds, just the word sky floating in the air), that would be read out to you once they were in your line of site. This was a half-formed idea inspired by accessible blind-friendly image descriptions on the internet; though rather than designing for the blind, I suppose it would be more to show how blind people “saw” the world—through a disembodied computer voice telling them what the view was.
Before I could execute these concepts, however, I needed to just get a VR workflow HAPPENING. This ended up being rather difficult and took up the majority of my time.
I initially planned to use the p5.vr library (https://github.com/bmoren/p5.vr ), but upon testing, it was incompatible with Android (VRorbitcontrol didn’t work in X axis and display would not show up properly). I also had trouble hosting it on the chrome webserver and webspace, but shelved that.
I started searching for other ways to code VR, or turn a 3D environment into VR, and stumbled upon webVR. I researched that further and liked its concept; so I also downloaded that and looked through how to create an app in webVR. I also read up on webVR polyfill. Following this tutorial (https://developers.google.com/web/fundamentals/vr/getting-started-with-webvr/ ), I tried to integrate webVR into an existing p5.js WEBGL sketch I had. Didn’t work due to incompatibility between webVR polyfill & p5.js.
When I was researching webVR I also found three.js, and really loved the project examples hosted on their site, especially this one (https://demos.littleworkshop.fr/track ). Trying this one out (after figuring out I needed to disable some chrome flags first) was what convinced me to try out webVR.
At this point I was running out of time and I switched back to p5.vr since it was supposed to work on Apple and I figured I could borrow an IPhone in class. Spoiler: it wasn’t working still. I don’t have an IPhone with me so I wasn’t able to investigate the issue further after class, but for some reason, even though it works fine on desktop, the mobile VR shows up with a large gap in the stereocanvas.
big bang notes:
the p5.vr library doesn’t open a lot of doors for interaction in its VR environment, something I was disappointed by as I value interaction a lot. I tried to counter that by position a directional light at your POV that would be adjusted towards whatever direction one was looking at, and then placing in planar materials that would disappear without proper lights. This created some sort of pseudo-interaction where viewers had to work to see the plane.
I created a simple space environment with stars, then was inspired by walking codes I’d seen on three.js and star wars and space operas to create some sort of warp drive effect by translating the stars. While the feel I created reflects the stars moving rather than the viewer moving, I still though it was sort of cool how they collected at a single point and that inspired the idea of the big bang.
Finally, I reset the code every 50 secs, because the big bang doesn’t happen just once. There’s probably a more contained, seamless way to do it than my goOut() code, but it worked.
I was also inspired by this year’s Nuit Blanche theme of You Are Here, and Daniel Iregui’s response to it with their installation, Forward. The sped up time in my VR work and the looping animation alludes to the presence of time and how the future is always out of reach. That’s also reflected by the half-present planar window, always too far ahead of you.
“Wow, it feels like I’m actually there!” – anonymous OCAD student
I’m really, really fond of my home country, and that shows quite frequently in my work. Ayutthaya, dubbed (by me) as Thailand’s collection of mini Leaning Towers of Pisa, is one of Thailand’s oldest historic site. I recently visited this past summer (2018), and the sense of history in the air is palpable. I’m not sure this VR experience replicates that by any means, but it at least shows people that Ayutthaya exists.
Honestly, this was more of a test, than anything, I’d need to revisit this and create some interaction or movement. I believe Unity’s the right way to go with VR environments and would continue using Unity now that I’ve got it to work. It has a lot of functionality and I’d be able to easily place 3D objects in the environment. One thing I’m having trouble with is that the video won’t loop/play properly.
big bang files: https://github.com/salisajat/e1-big-bang *I’d accidentally worked straight inside a cloned repository of p5.vr and was unable to push that code or keep any of my commits when I remade the folder :/
Track Tiles is a one-week game jam that I created for our class prototype assignment. I wrote the game in C# using the Unity3D game engine and designed the art assets in Illustrator CC. The game is an endless runner themed, single tap mobile game that takes place over an infinity long span of train tracks. Once the game begins, squares of these tracks spawn in a predetermined path along the screen. However, the track pieces are rotated randomly – leaving it up to the player to keep the train from crashing. As the game progresses the player must tap on track pieces to rotate them in such a way that the train will continue to ride forward. As of now, players can collect stars along the way and notice their score increase. In the future, I would hope to add additional track types, power-ups, more obstacles, and cosmetic items.
In terms of context, my assignment was very much a solo venture. The Idea came to me while I was thinking of a game that is complex enough to keep the player stimulated yet simple enough that It can be played with a single tap. Track Tiles was also inspired by a previous (nameless) game that I have in development. Some of the code was actually directly used – the ninety-degree turn mechanic and tap detection code mainly. However, a majority of Track Tiles was inspired by my own thoughts and ideas, and online forums that helped me move past some difficult aspects of the project – All of which I have linked below.
Post Presentation Thoughts:
After showing the project to the class, first impressions were mostly positive. Many players found the game much too hard at the beginning, and never saw a score that they were proud of. However, almost everyone that played immediately hit the “retry” button. This was interesting because I’ve made and demoed games before, but never received this behavior so strongly. I believe the higher difficulty actually drew the player in, rather than repelling them. Interesting as that may be, I will be lowering the difficulty of the game at the beginning, and then increasing train speed and adding double tracks later on. Overall, I’m very happy with how the project turned out and I will surely continue to work on it into the future.
The project I created is a colour tracking audio visualizer. The program takes the pixel information of a webcam/computers camera to track the presence of specific colours (magenta, cyan, and yellow). The tracking information/data (which includes x and y coordinates, tracked colour, and dimensions of the coloured object) is stored in an array as values. Within the script, I was able to call those values to create parameters for the representative ellipses and sounds for the tracked colours. I used p5.js’s capabilities in oscillation to create a sound output. The tracked cyan colour controls the oscillator’s frequency and the tracked yellow colour controls the amplitude. The magenta colour adds a Bandpass filter over the sound and controls the filter’s resonance and frequency.
Note: In the future, I would like to create a glove of sorts with colourful fingers instead of individual objects. I think it would be easier for the user to create more controlled and finesse sounds. For example, one would only have to hold a finger down if they do not want to track a particular colour, instead of having to completely put it down while juggling the objects for other colours.
My goal for this project was to create something that was fun to interact with. I wanted to go beyond simple mouse events and into an interactivity that went beyond just a gesture of your fingers. Because I decide to use colour tracking and computer vision, the user can interact with the program without even touching the computer. All they need is colour to control the sound and visuals of the project.
I have always been fascinated with the installations and projects that use technology to track the movements of the human body to create generative works of art or immersive experiences. My use of colour tracking in my project is just a stripped down way of implementing movement analysis within p5.js. Originally, I thought of using a Kinect, like many other of the projects I’ve seen, for its built-in motion tracking abilities, but I decided against it. Instead of using the form of the human body I wanted to use colour because I felt it would be easier to implement with my present skills.
My father and I are movie buffs, and the Matrix has stood the test of time as one of our shared picks. I’ve always linked the movie to computer programming, and now that I’m learning it in school I wanted to explore the possibility of emulating something from it. When I began to learn p5, I instantly sought out ways to recreate the digital rain. I came across a tutorial on YouTube that showed you exactly how to do it. Actually, it was on The Coding Train, a channel I am quite familiar with. But before that, I wanted to create as much of it as I could on my own. I ended up creating the Symbol class on my own, using the String fromCharCode() method and displayed one symbol on the screen. From there I populated the screen with symbols using an array, with much dissatisfaction. After that, it became increasingly more difficult, even with my knowledge and some help from a friend. I ended up referencing the tutorial but – including the code corruption aspect – most the final code is original. One aspect I wish I improved on was making the canvas full screen.
What I created was a playable piano within p5.js. I got my inspiration from Liszt – La Campanella which shows notes falling onto as Synthesia and in the video as you pressed the note a spark of small particles fly off and I thought that I was a really nice visualization so I tried to attempt it. Also I wanted to try the opposite of what the video did which showed the notes falling and made it so that when you click the notes a note would spawn and the size would depend on the duration you held the note down. For example https://vimeo.com/190474651.
How I started this project was I first looked at the song example on p5.js and tried to replicate its way of making the keys play the note. As I created the code I ran into a couple of problems when playing the notes. When the notes were played at the same time or in rapid succession the note won’t fully play. So to solve that the notes had to be created with different variables. For the keys itself I mapped it to the keyboard and showed the letters that corresponded to the keys on the piano.
So then after that I started to work on the particle system at first I started by having some particles spawn randomly near the note then I wanted to have it shoot out into a cone. Video of the piano and particle system (https://www.youtube.com/watch?v=c0Y9VgjQCVA). At first I just made the particles to just keep on bouncing within the canvas, but after some feedback I changed it to have them go off screen.
So from here problems started to occur when I tried to work on the note spawning system. It was a difficult task to do because I needed to find a way to make the note growth = the duration the note was held for and release it going off screen. I got the note spawning idea to somewhat work (https://www.youtube.com/watch?v=tFAufhJ3axg) but then another problem occurred which was that when I clicked another note after it had spawned the previous note stopped and kept growing. After many attempts of trying to fix the problem I just scraped that idea due to how much time I had left to do the project.