Author: Jun Li

Experiment 5: Eternal Forms

Names of the Group Members:

Catherine Reyto, Jun Li, Rittika Basu, Sananda Dutta

Project Description:

“Eternal Forms” is an interactive artwork incorporating geometric elements in motion. The construction of the elements is highly precise in order to generate an optical illusion of constant collision. The illusion is a result of linear patterns overlapping in motion between geometric forms. The foreground square will be firmly stabilised while the background circle will be in rotating constantly. While display lights change their chromatic values when participants interact from ranging proximities.

The artwork takes inspiration from various light and form installation projects by Nonotak, an artist duo consisting of Illustrator Noemi Schipfer and architect-musician Takami Nakamoto. Nonotak works with sound, light and patterns achieved with repeating geometric forms. The installation work aims to immerse the viewer by enveloping them in the space with dreamlike kinetic visuals. The duo is also known for embedding custom-built technology in their installations, as well as conventional technology to achieve desired effects in unconventional ways.

Visuals:

Final Images:

 

Circuit Diagrams:

https://www.circuito.io/app?components=512,11021

Project Context

Initial Proposal

Originally we had the intention to continue our explorations with RGB displays. Four out of five of the group members had come a long way while working together on Experiment 4 (Eggsistential Workspace), only to have our communicating displays malfunction on account of the unexpected fragility the pressure sensors. We had hoped to pick up where we had left off, by disassembling our previous RGB displays and revamping the project into an elaborate interactive installation for the Open Show. We designed a four-panel display, each one showcasing a pattern of birds from our respective countries (Canada, India and China). The birds would be laser-cut and lit by effect patterns with the RGBs. After many hours of strategizing, we found we were facing too many challenges in the RGB code that, given our time constraints, became overly risky. For example, we intended to isolate specific lights within the RGB strips, thereby designating the neighbouring lights on the string to be turned off. Once we broke down how complex doing this would prove to be (each message sent to an LED would involve sending messages to all preceding LEDs in the string), it became clear that the desired codebase was out of scope. We returned to the drawing board and began restrategizing a plan that could work within the restraints of our busy schedules, deadlines and combined skills. Having five people in the group meant a lot of conflicting ideas, making it tricky to move out of the brainstorming process and into prototype iteration. But we were all interested in kinetic sculptures and the more examples we came across, the more potential we saw in devising one of our own. It seemed like an effective way of keeping us equally involved in the strategy as well as the code. Having minimal experience working with gears (only Jun Li had used them previously) we were intrigued by the challenge of constructing them. We came across this example and began to deconstruct it, replacing the hand-spun propulsion with a motor and controlling the speed and direction by means of proximity sensors.
(show video) : https://www.youtube.com/watch?v=–O9eyKIubY

Though we aimed to keep the design as simple as possible, we weren’t able to gauge the complexities of the assembly until we had really started to dig into the design. We thought a pulley system could be built, where a mechanism surrounding the motor could trigger motion in another part of the structure by way of gears. We were mesmerized by the rhythmic patterns we came across, in particular, the work of Nonotok studio. They primarily work with light and sound installations. Taking inspiration from their pieces and work, we decided to create visual illusions based on the concept of pattern overlap. We also planned to make use of light and distance sensor to make the piece an interactive light display.

https://www.nonotak.com/_MASK

Tools and Software

  • Distance sensor
  • RGB lights
  • Servo motor (360)
  • Nano and Uno board
  • Acrylic sheets – black and diffused
  • ¼’ and ⅛’ Baltic Birch sheets
  • Laser cutting techniques
  • Illustrator

Ideation

Our previous ideas seemed complicated in terms of implementation. Hence, we sat for a second round of brainstorming on the several outcomes with the given time frame and resources. We commenced on browsing on existing projects of ‘Kinetic Installations of NONOTAk Studio, ‘The Twister Star Huge ‘by Lyman Whitaker, ‘Namibia’ by Uli Aschenborn and Spunwheel Award-winning sculptures made from circular grids. We proceeded with the creation of our circular grid. We designed a circular grid system, which is constructed by several interlinking ellipses running across a common circumference of the centre ellipse. This grid served the base our proceeding designs.

We derive inspiration from Félix González-Torres, American visual artist from Cuba who created minimal installations from common objects like lightbulb strings, clocks, paper, photographs, printed texts or hard candies. Being a member of ‘Group material’, a New-York based artist organisation formed to promote collaboration projects with regard to cultural activism and community education.

Process

Several constructions and geometrical formations were explored. We studied how to create an optical illusion with forms in motion. We tried to simplify the curvatures into straight lines since we had no idea on the feasibility and reliability of complicated junctions. Thus one simple circle and one simple square were included.

As you can see in the above diagrams, a layout was created to give us an idea of the entire frame, its size, materials to be used as well the complications or hindrances that may befall upon our way.

After the finalization of the entire setup, we can up with a list of different layers that would be encased in an open wooden box (20’by20’). The list is as follows from top to bottom:

  1. Square Black acrylic sheet with laser cut patterns – This will be the front view (covering of the open wooden box ) and will remain stationary – size 20’by20’
  2. Circular Black acrylic sheet with laser cut patterns – This will be in motion as the centre will be connected to the 360 servo motor – size 18.5’by18.5’
  3. Diffused white acrylic sheet with a cut outline in the centre to fix the base of the servo motor
  4. RGB lights + Nano and Uno board – These are stuck to the base of the wooden box
  5. A small wooden strip with a distance sensor holding area to be attached in front of the installation – This will change the pattern lighting based on distance

Image: The 2 layers of forms that were laser cut to include in our final setup.

Image: The RGB bulbs set up to create an even distribution of light across the 20”x 20” board.

Image: After the setup was done, above are a couple of effects created using lights an motion of the overlapping layers.

Prototyping

We created numerous samples of our design in miniatures and overlapped them. We experimented with black and white colours by playing with the following arrangements-

  • White Square rotating on the White Square (Stabilised)
  • Black Square rotating on the White Square (Stabilised)
  • Black Square rotating on the Black Square (Stabilised)
  • White Square rotating on the Black Square (Stabilised)
  • White circle rotating on the white square (Stabilised)

Coding

Motor — the motor is set to run slowly counterclockwise at the optimum speed to give the interplay of with the geometry. It’s important to get the speed exactly right or the lines will not show the desired effect.

Lights — the distance sensor reads in the value and includes it the running average of the distance (last 10 readings), it then maps that distance to a value that will be used for the brightness of the lights, and the speed of the effects. The closer the brighter, but slower the effects. The distance is also used to determine what light effect is shown. When very close, it breathes, a little further away, it blinks quickly, and at the standard distance it paints the colour to the background. Each effect adds to the effect of the illusion.

Final Project & Exhibition Reflection

For the exhibition, we were given an entire room to display the piece. We projected a video of the manufacturing, on to one wall, and on the opposite wall, we solved the concern of the empty space by projecting artwork that was appropriate for the display : a generative mandala formation of various altering forms (coded by Sananda in her individual project). The work allows participants to create their patterns with varying colours using manual alterations by potentiometers. We also had some calming tunes that played along with the laser cutting video which was being projected.

Many in attendance commented that they couldn’t pull their eyes away from the piece, and that it was meditative, mesmerising and calming. We also received three offers for the purchase of the installation. One participant analysed the piece praising the use of colour, lines, geometry and interaction that made it very aesthetically pleasing, and we noticed many leaving and returning with their friends to have them experience the illusion themselves, and to interact with the distance with great delight. Overall the experience of light and subtle motion in a dark room created some beautiful visual illusions and that became the limelight of our experiment.

References

  1. SCHIPFER, NOEMI, and TAKAMI NAKAMOTO. “MASKS, Kinetic Object Series”. Nonotak.Com, 2014, https://www.nonotak.com/_MASKS.
  2. Kvande, Ryan. “Sculpture 40″ – Award Winning Optical Illusion Sculptures”. Spunwheel, 2019, https://www.spunwheel.com/40-kinetic-sculptures1.html
  3. Whitaker, Lyman. “Whitaker Studio”. Whitakerstudio.Com, 2019, https://www.whitakerstudio.com/

Experiment 4: Influence Over Distance

Eggsistential Workspace

Course: Creation & Computation
Digital Futures, 2019

Jun Li,  Catherine Reyto, Rittika Basu, Arsalan Akhtar

Project Description

Eggsistential workspace is a somatosensorial platform intended to communicate motivation and companionship between two participants communicating from distant spaces through transitional chromatic values. This personalised bonding experience enables participants to convey their activity while in their individual workspaces, and more specifically, their laptop or desktop PC. With the motion of their wrists rising and falling while in the act of typing, pressure sensors (nestled in a mousepad made from a balaclava and light stuffing) activate the RGB patterns of the corresponding participant’s set of lights. The faster they type, the greater the activity of the lights, and the slower they type, the more their inertia is echoed by the decreased activity of the light patterns. We have all experienced that feeling of being strapped to one’s desk under the pressure of a deadline, as well as the lack of community if working alone is a frequent occurrence. We thought it made for a fun, expressive but non-intrusive way of keeping one another company while working at home or in a solitary-feeling workspace.

One example of telepresence that inspired our ideation process was the project titled The Trace6 from El Rastro. In this installation, two people in remote rooms common space with the help of light visuals and sound which is triggered by a sensor when one individual enters a room. This results in them occupying the exact same position in the space.
In addition to that, another great use of ultraviolet lights for telepresence art installation could be observed from the project titled “Miscible “7. This work from Manuel Chantre and Mathieu Le Sourd used sensory, light and principles of chemistry to make two liquids homogeneously while participants were in remote locations. In this performance, participants from remote locations are expected to mix the liquid with UV lights in a way to mix them perfectly. Each UV light mixes to create a perfect blend of liquid and colour.

Ideation & PROcESS

In our first brainstorming session, we agreed from collective past experience that it would be wise to keep the idea simple. The very first topic we discussed was the language of colour, and how hues are interpreted differently in different countries. But we struggled to find a tangible means of working with colour translation, given the complexity of networking. We brainstormed several ideas, explored online examples and struggled to procure elements from previous projects. One of the dismissed proposals involved creating a winter scene (via graphics in Processing) wherein participants could collectively monitor parameters of day-to-night transition (changing the background colour), intensify the snow-storm via wind (rain/snow particle code), wavering opacity, amplify audio effects etc.

Through the iterations of our project, our common interests aligned and a concept began to take shape. We were inspired by the concept of ‘Telepresence’ from Kate’s class, especially the ‘LovMeLovU’ project by Yiyi Shao. (“Shao”) [1]. We were all drawn to the idea of remote interaction between two individuals, in two separate spaces, by means of two displays of light. We finalised on an output of 50 RGB LEDs set for 2 rooms. (bitluni’s lab) [2] It did mean upping the ante in a big way, but we had two group members with experience in our corner. It meant they had a chance to fulfil some of the objectives from a previous experiment and gave the rest of us the opportunity to learn about working with RGBs. We also recognized that there was a gap between the code capabilities of some group members compared to others, and it meant a lot to us to all have a hand in writing code in some capacity. Since Arduino has by now become a fairly comfortable language, it further emphasized the desire to add an extra layer to the project requirements, in that we could all work, code and test together, learning from one another along the way.

Because of the visual appeal and chromatic range offered by RGBs, we were determined from the outset to incorporate them in the output design. We were really taken by the idea of being able to illuminate a friend’s room from a remote location, and it was important that the display interaction feel emotive and intuitive. At first, we imagined this action taking place with hugs and squeezes (by way of a stuffed toy or pillow), sensor-driven to create an effective response in the corresponding display. A light squeeze of a pillow could light up a friend’s bedroom on a dreary and perhaps lonely evening, feeling like a hug and a small gift of an uplifting atmosphere. A hard squeeze, by contrast, might generate a bright and panicked effect in my friend’s room, letting them know I’m feeling anxious.

Knowing that we had our work cut out for us, we made a list of benchmarks on a timeline. We had laboured away for 9 hours on Saturday, learning how PubNub worked and by early evening, sending switch values through PubNub to Arduino and finally to Processing. That was the big hurdle we had been unsure about, but thanks to clear communication, we were able to build a clear path that everyone was able to understand and work with. The achievement gave us confidence, and we set about storyboarding an ideal setup: two modes, one using Arduino and another using Touch Designer. We were very interested in trying out both systems; Arduino because we all knew our way around a bit, and Touch Designer for the added benefit of effects as well as what Jun Li could show us. By the next session though, when testing the LEDs out across the network, we encountered a major issue before even getting that far. To our surprise, the Nano could only power a small portion (about 5 to 10) of the RGB strips (out of 50 per strip). This wasn’t enough for a significant display. We were able to resolve some but not all of the issue by using Megas instead.

As the days past, the stuffed toy or pillow took on various forms… eventually landing on wristwarmers. We moved in this direction because for one thing, we were apprehensive about the surprise challenges that the pressure sensors might present. It seemed logical that the enclosure is easily accessible, and that the sensors have as much contact with the point of pressure as possible. With wrist-warmers, we could control the variable resistance by gripping and relaxing our hands. It felt like a very natural use and appropriate for the (very sudden) change in season. The fact that wrist-warmers are less common / expected than gloves was a bonus. We eventually settled our design on an ergonomic support pad for typing. There was a more refined simplicity in this concept. No coding language (in the form of squeeze strength of the number of squeezes to communicate) was needed, in fact, no thought on the user’s end was required at all. Instead, they would carry on as they normally would, typing away at their work.

It took some time to work out the display for the lights. We had decided on using the two south-facing adjacent walls in the DF studio, modelled as bedroom settings. We became invested in the notion of adjustable displays (and it still seems like a cool idea), where individual strips could be attached at hinges while supporting the LED strip. We envisioned participants configuring the display themselves, and hanging it from hooks we’d fashioned in the window sill. Ultimately this plan proved unfeasible and we set it aside as a divergent. We settled on the hexagon-shaped mounts because they made practical sense: they were good housing for the narrowly-spaced LED strips and were less time-consuming to produce. But we opted for the hexagons because in the meantime we had run into a major issue with the LEDs: even with the use of Megas, we could only generate power into the full strip if they were lit at 50% opacity, max. Thus having the lights configured in a cluster meant optimizing the light effect.
We had early on opted for using ping pong balls as diffusing shells for the bulbs, and had wisely ordered them in just enough time from Amazon. We went through a lengthy process of making holes in 100 balls, then fastening them to the hexacon rigs.
Meanwhile, we had devised a system for the sensors, fastening them snuggly into sewn-felt pouches intact with velcro openings for easy removal. These were designed to fit in the openings of the wrist-warmers, but after running into some unexpected complications with this system, we placed them instead inside mousepads improvised from balaclavas.

SOFTWARE IMPLEMENTATION

image13

Arduino + Processing + Pubnub Implementation
For the default implementation, we used two channels to communication between the rooms — each room published and listen to seperate channels corresponding with their rooms. We had issues with the port detection of the Arduino on certain computers but the root cause was never determined. Once the port had connected and maintained a stable connection, the communication from Arduino to Processing, to Pubnub and back could be made.

TD + Arduino Implementation
In this experiment, We brought more possibilities to this project with the experience of Jun Li. We tried to challenge ourselves by utilizing different techniques to achieve the same effect. In setup, we utilizing TCP/IP internet protocol instead of the PubNub to send the same data to control the LED lights. After testing Processing, we found the colour didn’t appear as what we designed, we tried and debugged a lot of ways to fix that colour problem. We thought it might be an issue with the hardware. After researching, we realized the model of LED lights is a bit different from the one that had been used in Li and Arsalan’s Experiment 2. The first set was WSB2811, whereas the sets purchased for Experiment 4 are WSB2811. It turned out the Red, Green channels are switched. After editing the data, its effect and it worked the same as the way of using PubNub. All the interactive effects and settings were performed in Touchdeisgner and send through TCP/IP internet to Arudino in real-time.

Arduino + Processing + Pubnub + TD Implementation
Because of the powerful functions of Touchdesigner, which can easily perform and design a lot of effects, we had tried and brought TD into the PubNub way to meet the project requirement. So the workflow became more complicated and difficult in this implement. We tried to bring 2 Arduino on each side and 2 serial communication happened on the same side. One for receiving the data from processing coming from the other side, another for sending date the received data to TD and send it back to the LED light. Theoretically and Technically it can be done, however, we found it was difficult to send and receive data among much software and we don’t have enough time to achieve the same effect like above 2 ways in the end.

Reflections

It seemed clear from our first brain-storming session that we were going to work well together as a group. We had a diverse set of skills that we were eager to pool together to come up with something especially creative. The challenge inspired us and we were ready to put in the work.

The early success we’d achieved after putting in long hours on the weekend might have given us a false sense of confidence. It put us ahead of the game and made us feel that since we’d overcome the biggest obstacle (networking from Arduino to Processing to Pubnub, end-to-end), the rest seemed easy enough to achieve. We set into production of the materials while devising a library of messages that users could communicate to one another, by means of pulses on the pressure-sensors and and lighting. What we failed to foresee was assembly issues with the sensors. We took it for granted that they were made at least a little durably (seeing as they are commonly used in wearables projects), but that turned out being far from the case. In spite of protecting the soldering with heat-shrunk tubing, encasing the a wires with cardboard backing, and harnessing them as securely as possible in the hand-sewn pouches, we went through one sensor after another, the plastic ends shredding with the slightest movement. We didn’t yet know we could repair them on our own (and have yet to try although it was investigated after we broke the collective bank), resulting in trip after trip to Criterion throughout that snow-filled week. After our early successes, and especially once we saw the extent to which we could actually communicate messages via lighting effects (we had even devised our own list of messages), it was so disheartening to have the project fall apart each time we tested on account of the extreme fragility of the sensors. It was a major factor in pivoting the concept from wrist-warmers to a mouse-pad (which involved no movement of the sensor), a decision that unfortunately took place too late in the game and didn’t allow us sufficient time for proper testing with the rest of our setup.

 

image30
List of messages derived from a library of Arduino lighting effects

Hindsight is always 20/20, and there is no way in this case we could have anticipated this problem unless we had researched “How fragile are variable resistor pressure sensors when placed in clothing?”. But we will know to be that specific about troubleshooting and testing well beforehand next time around.
It was overwhelmingly frustrating to have our demo fall apart to the extent it did right before the presentations. We’d had such a strong start, and working on this project had been an invigorating, devoted process for our group. Overall it was a great experience, one that involved a great deal of learning and collaboration, creativity. In spite of falling a little short in the demo, we had succeeded in some pretty amazing results along the way.

 

References

  1. Shao, Yiyi. “Lovmelovu”. Yiyishao.Org, 2018, https://yiyishao.org/LovMeLovU.html. Accessed 8 Nov 2019.
  2. bitluni’s lab. DIY Ping Pong LED Wall V2.0. 2019, https://www.youtube.com/watch?v=xFh8uiw7UiY. Accessed 8 Nov 2019.
  3. Hartman, Kate, and Nicholas Puckett. “Exp3_Lab2_Arduinotoprocessing_ASCII_3Analogvalues”. 2019.
  4. Hartman, Kate, and Nicholas Puckett. November 5 Videos. 2019, https://canvas.ocadu.ca/courses/30331/pages/november-5-videos. Accessed 8 Nov 2019.
  5. Kac, E (1994). Teleporting An Unknown State. Article. Retrieved from: http://www.ekac.org/teleporting_%20an_unknown_state.html
  6. Rastro, E. (1995). The Trace. Retrieved from: http://www.lozano-hemmer.com/artworks/the_trace.php
  7. Chantre, M. (2014). Miscible. Retrieved from: http://www.manuelchantre.com/miscible/

Live VJ Performance Show

Project Title: Live VJ Performance Show

-An interactive live performance that explored audio visualization

By Jun Li – Individual Project

wechatimg2308

Abstract

This project was a live performance show based on the concept of ‘Audio Visualization’, duration of 11 mins-long show for the audience. Also, it’s my first attempt at being a VJ. All 8 of the different effects were generated and interacted with music input in real-time. It was built on the Arduino controllers, TouchDesigner in addition to projection of the video output on the background. The purpose of this experiment was to create a very simple user-interface with different switches and sliders to manipulate the effects.

This dynamic experience allows the every participant to have an opportunity to become a ‘Visual Jockey’ as, they can operate and furthermore, change each parameter of the audio resulting in creation of the energetic graphics in the background. 

Keywords:  VJ, Live Performance Show, Audio Visualization, Interaction.

Requirements

The goal of this experiment was to create a tangible or tactile interface for a screen-based interaction that uses Arduino and Processing. Since, l came from a very similar undergraduate program called ‘Technoetic Art’ and l had a lot of experience working with these software beforehand. l am always eager to challenge myself for a high-tech level and create various fascinating projects. After discussing my idea with Kate and Nick, I received their permission. I used the technical logic of Processing and applied it to TouchDesigner. Thus, retaining the knowledge of serial communication in these 2 softwares.

Description

Music visualization, refers to a popular way of communication that combines audio visual and audio with the core of vision, music as the carrier and various media technologies such as new media technologies to interpret music content through pictures and images. It provides an intuitive visual presentation technique for understanding, analyzing and comparing the expressiveness of internal and external structures of musical art forms.

Vision and hearing are the most important channels for human beings to perceive the outside world. They are the most natural and most common behaviors of human beings and are irreplaceable for the activities of the human cognitive world. “Watching” and “listening” are the most natural, direct, and most important means of recognizing the outside world through a variety of audiovisual senses. Sound and video, hearing and vision, in contemporary society, the two agree on aesthetic trends and dominate the aesthetic form of mass culture. Vision provides many conveniences for people to see and understand musical works and music culture. People will increasingly rely on visual forms to understand audio content. The specific application of music visualization is very wide. For example, live music, exhibition site and other music visualization systems, combined with special images and customized immersive content development works, can give people a strong visual impact while enjoying music.

Process & Techniques

Research question

First, l have explored and researched what kind of parameters from audio can affect the visual part and how can l utilize these to manipulate the shape or the transformation of the visual design. Usually, audio visualization is based on the high, mid, low frequency, beat and volume of the music. So, one of the most important techniques is how to get these data or parameters from the audio itself and how to convert it to audio visualization.

Arduino Implementation

Most of my project program was built in TouchDesigner, for the Arduino part, the code was not difficult, just sent the data of a switch and 8 sliders to TD for manipulating the visual effect through serial communication.

TouchDesigner Implementation

  1. How to generate visual effects in real-time based on the different parameters of audio?
  2. How to utilize the data coming from Arduino to affect the visual effects that already had been generated in real-time?
  3. How to arrange and switch different visual effects and what’s the key and special point for VJ performance.
  4. How to optimize the program to avoid affecting the real-time output.
  5. How to output the whole project (both visual and sound part) to large displays (2 screen monitors, 1 projection) and 2 speakers to let VJ performer monitor and change the effects in real-time and show it to the audiences.

For the first and second questions, l had created 8 effects, utilizing different parameters from audio.

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-15-43-03

1.1 – Audio analyzing (high, mid, low frequency)

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-22-26-11

1.2 – Utilize the data from Arduino to manipulate visual effects

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-22-15-40

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-22-15-59

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-22-16-13

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-15-49-53

1.3 – Opening introduction (particle writing system)

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-47-48

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-22-05-51

1.4 – 8 different visual effects (Initial state & Altered state, Same below)

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-48-32

％e5％b1％8f％e5％b9％95％e5％bf％ab％e7％85％a7-2019-11-04-22-06-06

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-47-33

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-22-06-25

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-47-14

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-22-06-43

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-48-47

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-22-06-59

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-49-15

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-22-07-22

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-48-18

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-22-07-38

For the third question, I used the data from the slider and added an additional layer in black to switching and transited to the next effect. At the same time, l explored a lot of VJ will add strobe visual effect while the music reached its climax. So, there was another additional layer based on the low frequency of the audio, it increased the strobe of the video, which could improve and enhance the live atmosphere.

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-39-30

1.5 – The interface of switching 8 effects

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-41-13

1.6 – Different function in layers

For the fourth question, in the beginning, functionally l could utilize one data from the slider to control all 8 visual effects. However, l realized it was a huge challenge for my laptop to process it on the GPU & CPU at the same time. So, l had to separate the data into 8 sliders to control each effect and test it carefully. Eventually, it worked and did optimize my program successfully.

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-37-08

1.7 – The data flow in Touchdesigner 

wechatimg2306

1.8 – Arudino interface

For the last question, l did explore the function in TD for project output, it was powerful to allow the creators to create a super convenience interface to supervise the whole process and outputted it to any screen and speaker.

%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-34-21

1.9 – Output to display

Challenge & Learning

  1. The due time of this project was super short. It required us to submit in 9 days, including having ideas, coding, testing, debugging and final setup. Especially, l used a more difficult software – TouchDeisgner. Although l had experience working with TD before, not as familiar as Processing. At the same time, l didn’t have much project experience. l barely got any help and reference from my classmates. It’s really an individual work and quite challenging for me, which l am excited as well.
  1. Set a goal to a live VJ performance show means too few visual effects won’t be accepted and not powerful enough in visual and hearing. I am a perfectionist. So, I had to create 10 mins-long show at least, that’s the extra requirement and pressure for me. So, to achieve that in the end, l kept testing and creating different visual effects. Eventually, l chose 8 powerfully and great enough from 26 effects l created in TD. %e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-16-55-12                                       2.1 – Reference library l created
  1. Processing data and having these effects in real-time were a super heavy job on my GPU and CPU. it also challenged the performance of my computer. l almost getting closer to ruin my GPU and unfortunately, l still lost my 3.5mm audio output in the end. After coding, l had to work really hard to optimize the program to get a better, stable 60 FPS output. Because my computer was not powerful enough, especially the GPU. Eventually，there still were some frame droppings during live performances.
  1. To my satisfaction, this project was successful been made and the response from others were very great. l perfectly achieved the goal l set in the beginning, compared the initial goal, NOTHING had been changed. I learnt a lot of project experience and techniques in TD, including analyzing audio, visual designing, data integration, manipulation and analysis, project optimization, output and management.
  1. The next step l will do is to keep improving and optimizing my program and create a simpler operation interface, which can let users manipulate it easily.%e5%b1%8f%e5%b9%95%e5%bf%ab%e7%85%a7-2019-11-04-15-20-28

2.2- VJ setup in TouchDesigner

Code Link:

https://github.com/LLLeeee/Experiment3_C-C_JUN-LI

Conclusion

Currently, we use technology-based new media art as a way out. These forms of new media art are often the forerunners of future art. Artists work in the area of “art and technology” to create collaborative artworks. What’s more, in the era of such a big technological transition in modern times, the way of life has changed accordingly, and it has become necessary to fundamentally grasp the essence of art. The cross-border nature of music visualization art is very obvious. It involves music art, visual art and the artistic integration of the two.

When this project was exhibited for the exhibition, I was the VJ and presented it myself and able to personally observe the interaction and experience of the participants with it. To my satisfaction，I achieved my goal of becoming a VJ, bringing a show which challenging me incredibly.

wechatimg2315

wechatimg2314

Reference

  1. vjfader (2017). VJ Set – Sikdope live @ Mad House – ShenZhen China.Available at: https://www.youtube.com/watch?v=uG1GrD7VQOs&t=902s.
  2. Transmission Festival (2017). Armin van Buuren & Vini Vici ft. Hilight Tribe – Great Spirit (Live at Transmission Prague 2016). Available at: https://www.youtube.com/watch?v=0ohuSUNHePA 
  3. Ragan, M. (2015). THP 494 & 598 | Simple Live Set Up | TouchDesigner.Available at: https://www.youtube.com/watch?v=O-CyWhN4ivI
  4. Ragan, M. (2015). THP 494 & 598 | Large Display Arrangement | TouchDesigner. Available at: https://www.youtube.com/watch?time_continue=1&v=RVqNjJfE9Lg 
  5. Ragan, M. (2015). Advanced Instancing | Pixel Mapping Geometry | TouchDesigner.  Available at: https://matthewragan.com/2015/08/18/advanced-instancing-pixel-mapping-geometry-touchdesigner/
  6. Ragan, M. (2013). The Feedback TOP | TouchDesigner.  Available at: https://matthewragan.com/2013/06/16/the-feedback-top-touchdesigner/

Experiment 2: Promexics Study/Interactive Infinity Mirror

Interactive Infinity Mirror

An interactive & LED-Light project that explored ‘Proxemics’
By Arsalan Akhtar, Jun Li

Proxemics – the study of human use of space and the effects that population density has on behaviour, communication, and social interaction.

Abstract

This experiment is an interactive, LED-light project, exploring in a critique of the concept of proxemics – the study of human use of space and the effects that population density has on behaviour, communication, and social interaction. In our interpretation, we attempt to show a visual representation of various reactions to “personal space” that humans create around them, in the form of various interactions of light, and represent the idea of an ideal level of social interaction amongst multiple parties.

The purpose of this study is to visually demonstrate the effect people can have on each other through the use of different colour effects. The goal is to deconstruct the relationship between behaviours and colour, and reshape this relationship to be presented in a new form. Throughout this process, the parties control the effects based on the states they are in.

Keywords: Colour, Behaviour, Communications, Visualization, Proxemics, Interaction

Table of Contents

1.0 Requirements
2.0 Planning & Context
3.0 Implementation
3.1 Hardware
3.2 Software 
3.2.1 Arduino-Only Implementation
3.2.2 TouchDesigner + Arduino Implementation
4.0 Reflections
5.0 Photos
6.0 References

1.0 Requirements

The goal of this experiment is to creatively use a microcontroller, up to 3 rangefinders, and actuators: any number of individual LEDs, or up to 4 servos, to create a minimum of 3 distinct behaviours in response to the environmental conditions.

2.0 Planning & Context

ca_c_exp2_sketches_compiled-min
Sketches and Brainstorming from our planning phase

 

In the planning phase concepts of anxiety, one’s “personal bubble/space”, and ideal desired interactions were examined. We determined that the most effective way to illustrate these changes would be by using light and incorporated the interaction of the primary colours of light (R, B, G) to illustrate this. We used the three distance sensors to capture the location of three participants within the space of the installation.

We completed planning the final design and concept, and began purchasing materials locally and abroad on October 4th.

2.1 Distances

Four ranges were plotted for each of the three sensors that read in the distances. This is reflective of the personal preferences of the three parties:

  • Too close: this is a distance considered to be too close for comfort. This can vary from person to person, but for our study, we fixed this distance.
  • Comfort zone: the general region of comfort, where one isn’t out of touch, and not too close.
  • Out of touch: so far the interaction is not possible.
  • Ideal: the preferred level or region of interaction.

 

c_and_c_range_diagram2x
Diagram of Our Distance Ranges for ‘Green’, ‘Red’ and ‘Blue’.

In our code, these were called states:

Out of touch, also called idle -> 0
Comfort zone -> 1
Ideal Zone -> 2
Too close -> 3

2.2 Interactions

Based on the state of the system, different behaviours were manifested. For example, if the red and blue are in their respective comfort zones, the two zones meshed to form their additive colour, which is magenta. The table below shows the various interactions that were planned for both implementations.

Interactions
State All Parties Some Parties One Party
Out of Touch/Idle (0) Colour wipe Off (Black Static) Off (Black Static)
Comfort (1) Sparkle Glow Blink in the secondary colour blend  Blackout chase of the respective colour
Ideal Zone (2) Rainbow Rainbow Rainbow
Too Close (3) Blink white

if, parties remain in the zone, blink orange, if they stay longer, rapidly blink red.*

 Blink in the respective colour Blink in the respective colour

* The time-based interaction was added from a suggestion from Nick.

The colour wipe when all parties are idle represents a state of receptiveness from everyone. No boundaries are being pushed, but no positive interactions are being made either. The colours cycle through without interacting with each other, and forgetting the previous state. If one or more sides are idle, and the other side(s) are in different state(s), the side goes ‘off’. The ‘off’ state indicates that it is not currently participating in the interaction that is happening. It is ‘Out of Touch’.

Once the comfort zone is entered by a  side, that side performs a ‘blackout chase’ of the colour it represents, i.e. the red side will have a red ‘blackout chase’ effect. When two sides enter the comfort zone, their secondary colour is shown. This represents the potential for ideal communication between the parties. Three in the comfort zone results in a Sparkle Glow, a combination of red, green and blue.

The rainbow, a common symbol of happiness and bridging, is used when one or more sides are in the ideal state. When within the installation, the desire is to remain in that state, and hope others can also find their ideal. Once everyone has it, the rainbow effect runs in sync, simulating an ideal flow of information and ideas.

There is of course, the potential for one to feel overwhelmed by a presence, and be ‘too close’ for comfort. When this happens, the corresponding side(s) flashes its colour repeatedly as a warning. If all sides are experiencing this, the additive colour (white), flashes for all sides. If this warning is ignored, the colour changes from white, to orange, then to ‘danger’ as red, speeding up as the warning continues to be ignored.

3.0 Implementation

3.1 Hardware

polaroidmoodboard-polaroidwall-min
3.1.1 LEDs

In this project, we were required to use individual LEDs. After considering the desired final effect of the project, we quickly abandoned the use of domed single colour LEDs, and decided to use WS2812 Neopixel LEDs. These have a full range of colours and can be individually addressed. This decision came with the challenge of soldering all the individual LEDs. Nonetheless, we were at an advantage as Jun Li had previous experience soldering, and was determined to take on the ambitious task.

As a first step, we measured and performed calculations on the box, and deduced that 23 LEDs were to be placed on each side, amounting to a total of 92 LEDs in all. Next was to create the LED strips to be placed inside the box. This involved first adding solder to the 6 connections, cutting the wires to the measured lengths, and soldering on each of the connections. A total of 1104 soldered connections were made by our team, and this figure did not include the several failures that occurred during the process. It was very important to ensure that each connection was sound and functioned impeccably. This was definitely a tough challenge for the team, and we relied heavily on the expertise of Jun Li as guidance, and who was aptly nicknamed ‘Solder King’. The entire soldering process took approximately 30 hours over the course of a few days.

3.1.2 Body

The body of the installation was a shadow box spray painted black. A hole was cut on the side to feed the wires from the LEDs to the Arduino. A mirror was placed at the base, and the LEDs around the inside of the frame. To cover the frame, we cut an acrylic sheet to size, and added a layer of reflective film to it. Arsalan and Jun Li used their knowledge of fabrication and workshopping to make precise cuts and measurements for the holes and the acrylic. Adding the film was a group effort as a smooth and reflective surface was key to creating the desired effect.

Finally, the three rangefinders were hidden under the lip of the shadow box, and the hidden wires fed to the breadboard at the back of the installation. The front and back of the installation were controlled by the single rangefinder located at the front, and the sides were controlled by their attached rangefinders.

img_6815

3.2 Software

3.2.1 Arduino-Only Implementation

The Arduino-only implementation was built with the backing of the Adafruit_Neopixel, and WS2812FX libraries. Several other libraries were tested, including FASTLED, Neo Patterns, and NeoPixel Painter, but ultimately, WS2812FX was selected due to its simplicity, and wide range of built-in effects. The loop of the code ran the service function for the WS2812FX library, and then the distance readings of the rangefinders taken. If the difference between the current and the last distance value measured was above the noise threshold, the new state of the section(s) that rangefinder controlled was determined, and the function to display that light effect was activated.

3.2.2 TouchDesigner Implementation

The effects were replicated using a TouchDesigner + Arduino implementation. In this setup, Arduino was used as a communication tool and bridge between TouchDesigner and the WS2812FX LEDs. All the interactive effects and settings were performed in TouchDesigner, and sent to the LED strip through Arduino in real-time.

4.0 Reflections

Implementation Explorations and Outcomes

The light effects in the TouchDesigner and Arduino implementation proved to be easier to create, as it is a node-based and artist-friendly software. The transitions were smoother, and more visually appealing. However, this came with several drawbacks. The Arduino Nano and Uno were both not powerful enough to provide stable performance as a result of the two-way real-time communication, and the bridge (the Arduino) required to send and receive a large amount of data per second.

There were two possible solutions to this problem: (1) increasing the processing power by using an Arduino Mega, and (2) lowering the frequency of data transfer which would affect the real-time interaction and cause a delay. These two solutions were combined, and the parameters adjusted optimally to give the best performance, and mimic the effect in the Arduino-Only implementation.

The Arduino-Only implementation had similar issues with both processing and electrical power. To solve this, the brightness was lowered, and the rangefinder values were only read in every 5 seconds, as that was found to be the bottleneck in the code. This unfortunately made the installation less real-time, but the benefit was an increase in reliability and performance.

The only major obstacles during the hardware build, was the time required to complete, and skill required to ensure that the connections were true. Although buying an LED strip, would achieve the same effect and solve both of these obstacles, it would have been in violation of the limitations of this experiment.

Concluding Thoughts

This artistic experiment was meant to allow participants to critically consider the ideas of personal space in the context of proxemic behaviour. In a related study, “Proxemic Behavior: A Study of Extrusion”, the cultural group and sex were held constant and introduced the problem of the interviewer’s movement from the original comfortable distance established by the subject. In all cases within that study, the subject re-established a new comfortable distance, which in our study we called the “ideal” zone. The study surmised that this new state of comfort was a compromise between the one originally chosen, and the distance assumed by the interviewer. In our case, a retreat from the ideal resulting in a warning signal, and a new state of comfort was not sought.

Technology-based new media art is one of the forerunners of future art, and allows the creation of collaborative, and interactive artworks. This experiment serves as an example of such.

5.0 Photos

6.0 References

  1. https://www.tandfonline.com/doi/abs/10.1080/00224545.1991.9924653
  2. https://www.youtube.com/watch?v=sAPGw0SD1DE
  3. https://www.youtube.com/watch?v=b2bvWArORSc