Coding Team

Jerez Bain 3152579
Vivian Wong 3158686
Anran Zhou 3157820
Mika Hirata 3154546
Natalie Le Huenen 3156341
Katrina Larson 3159249

CONCEPT

Neuro (Subject to Change) // SanDome

An interactive art installation, in which the audience is given the power to change the environmental conditions of three different worlds: Forest, Desert, Water, by adding heat, poison, or water. Virtual creatures inhabiting these worlds will respond and react to the changes according to their traits in either a positive or negative way. Neuro // SanDome focuses on the connection between these worlds and creatures, similar to how neurons in the brain connect and react to inputs.

As part of the coding team, we were responsible for the functionality aspect of the project, rather than the design. However, the choices we made helped shape the design and limitations of the project. Our challenge… the code has to function, or the whole project falls apart.

It was absolutely vital that we did everything we could to prevent that.

THE PLAN

In order to successfully develop the code with minimal error, we needed to make sure that we had a plan. Referring to the initial design of the project, we listed out all of the tasks we had to complete, and produced a timeline. We gave priority to completing the animation base template first, as the entire class would need it to complete their individual parts.

We wrote pseudocode to understand and determine the logic behind our code. We communicated our ideas with the Enclosure and Wiring teams and made changes based on their input. We divided our team into three parts: Animation, Communication, and Control Panel/Central Brain, and got straight to work.

ANIMATION

Preparing a ‘template’ which houses each set of animations loaded on each Arduino Nano was crucial for having all the components synchronized.

We began by making an early working template for shape-based animations to be placed in. Using the shape functions provided by the u8g2 library (found here: https://github.com/olikraus/u8g2/wiki/u8g2reference) simple movements created by shifting coordinates of shapes and lines can be combined to create animated creatures.

We chose a set timing for the animations, three frames per second. The template is set to loop through animation frames in the order of frame 1, frame 2, frame 3, and frame 2 again. This creates a ‘breathing’ or ‘pingpong’ effect, maximizing the use of six frames per creature (three ‘happy’ frames and three ‘sad’ frames).

hframe1photo hframe2photo hframe3photo
Testing the timing and frame order of an example ‘happy animation’ on an OLED screen

We found that creating animations with this method works but is time-consuming. Verifying that the animations are working as intended involves re-uploading the template to the Arduino Nano every time, which takes several minutes.

Experimenting with using drawn images as animation frames involved designing a process where image information can easily be added to the existing animation template. We created instructions which guide the rest of the crew in using the template. There are many steps involved to ensure that the image information is compatible with the OLED screens.

drawinginps

hframe001 hframe002 hframe003 sframe001 sframe002 sframe003

Drawn images use black-and-white pixels to tell the OLED screen which pixels are on and which are off. Exporting the image as an xbitmap (.xbm file) and opening it with a text editor displays the image information which can be moved into the animation template using the Arduino IDE.

screen-shot-2018-01-23-at-12-50-31-pm
screen-shot-2018-01-23-at-1-06-39-pm

As the animation template was developed independently of the communication system, merging the code involved ensuring that there was enough onboard memory in each Arduino Nano. Fortunately, drawn xbitmap animation frames can be stored within the Nano’s static memory instead of the RAM, which the communication system uses more heavily.

COMMUNICATION

For the communication, we started off by using SoftwareSerial library on Arduino, which allows us to use any pin on the board as RX and TX. SoftwareSerial library allowed serial communication on other digital pins of the Arduino, using software to replicate the functionality.  So that we were able to have multiple softwares serial ports( Arduino Uno and Arduino Nano).

Process Images:

First trial of the Arduino communication:
Since we did not have the same GND for both of the Arduino Unos, we were not able to get the Arduino communicating each other.

%e3%82%b9%e3%82%af%e3%83%aa%e3%83%bc%e3%83%b3%e3%82%b7%e3%83%a7%e3%83%83%e3%83%88-2018-01-29-14-27-01

img_3487

Second trial of the Arduino communication:
Second failure of connecting two Arduino Nanos with using OLED. We still did not have same GNDs for both of the Arduinos.

img_3500

Third trial of the Arduino communication:
We started to use SoftwareSerial library on the same computer by plugging in two Arduinos in the same laptops.

img_3494

Fourth trial of the Arduino communication:
Successfully connected the Arduino Nanos by using one laptop. We could light up a green LED to test if the Arduinos are successfully communicating.

img_3499 img_3498

CONTROL PANEL & CENTRAL BRAIN

Before we could even start writing the control panel code, we had to have a working prototype of the control panel inputs. Using sample code, we tested it out each sensor individually, to make sure they were working and that the wiring was correct.

20180118_092945 design-proposal-circuit-control-panel-a_bb
screen-shot-2018-01-20-at-1-06-01-pm

We then rewrote the code so that all sensors could be used at the same time.

screen-shot-2018-01-20-at-4-13-35-pm

screen-shot-2018-01-20-at-4-22-38-pm

We worked individually on the code at first, testing out our own versions. We added checking functions with if-statements that referred to the range of each sensor value.

There were several ways to code the language of the control panel code. In one version, we used booleans to store the input information according to the if-statements. In another version, we replaced the boolean type with int types defined as either 0 or 1.

Working in collaboration with the communication code team, we merged their working code with the control panel code to test. Unfortunately we kept getting strange readings in the Serial Port. The Serial Port should have sent ‘1’s but kept sending ‘-1’s, and the occasional ‘49’. Through research and trial and error, the error was fixed by replacing Serial.print() with Serial.write() and boolean variables.

screen-shot-2018-01-21-at-10-21-53-pm
screen-shot-2018-01-28-at-7-53-29-pm

Test 1: https://drive.google.com/file/d/1em31ZhJvwfBosZ0zZIY7pm5Txupnn0un/view?usp=sharing

We replaced the original animation template images with text that signified which input was on when testing initially.

However, when we switched out the Uno to the Nano, our original code stopped working. To fix this, we had to switch back Serial.write() to Serial.print() and integer variables of 1, 2, or 3 according to the input.

Central Brain OLED Animation:

Since we decided to create three types of conditions which are fire, water, and poison, we created three patterns of animations on the OLED display.

  • If the FSR pressure sensor is pressed, then poison animation is on.
  • If the photoresistor detects a certain amount of light, then fire animation is on.
  • If the potentiometer is twisted over a level, then water animation is on.

We used the If() statement to declare each type of condition and made sure the animation property switches based on the value which the sensor detects.

URLS for the code and videos:

Vimeo link for condition animations(water, poison and fire animations):
https://vimeo.com/252647599

Github link for condition codes:
https://github.com/0102mika/conditions

Images of animations:

img_3547  img_3550 img_3553

When we were ready, we merged the communication code and the central brain condition animation code with the control panel code.

Final Test video: https://vimeo.com/252927841

DISCOVERIES & CHALLENGES

Throughout our experience, we have learned that good instructions come a long way. The animation template and instructions made it easy for anyone to successfully create their creature. During Communication and Control Panel coding, there were multiple times where we uploaded code to the wrong Arduino. It became a nuisance to ensure that the ports and Arduino build were correct each time. It was also interesting how the Arduino Uno and Nano required different methods to send data through the Serial Port. While the Uno worked better with Serial.write in bytes and booleans, the Nano worked best with Serial.print in integers. 

Out of all of the tasks, Merging everyone’s code together was certainly one of the most challenging. We attempted to merge the code several times, only to be met with no working results. When we slowly merged the code step by step and continued to test each time, we finally succeeded. Coding always comes with a constant need of research, debugging, and troubleshooting. With luck, we are able to stumble upon solutions for the obstacles that we face. Good commenting on everyone’s code was extremely helpful to avoid misunderstandings and confusion.

Delay & Millis

At the beginning, we were using delay() to control each function, such as checkLight, checkWater, checkPressure functions for the prototype. But it turns out that these functions cannot be played at the same time since the delay function works in a linear way. Thus, we replaced Delay function to Millis function, which allows different sensor and functions to be triggered at the same time based on different rate.

Code:
Create intervals, timers for each function and use Millis() to calculate time

screen-shot-2018-01-28-at-6-40-37-pm

FINALIZATION

Since we had the control panel wiring ready, we took responsibility for attaching the wired components inside the control panel dome. There were a few challenges with fitting everything inside the dome, and making sure that the wires did not disconnect. We used a half breadboard for the Nano and replaced long jumper wires with manually cut, colour-coded wires. The potentiometer and FSR sensor were the most difficult to keep in place. We had to raise the height of the pressure sensor by using an extra breadboard as a platform. We used alligator clips for the pressure sensor as the jumper wires did not fit, and taped everything into place.

Image inside the dome for central brain:

img_6095 img_6096

Final outcome images:

img_3569

img_3577

REFLECTION

All things considered, developing the code for this project was a huge undertaking and quite challenging. A huge part of this journey was teaching ourselves and each other, and learning as a group and as individuals. As individuals, we taught and shared our knowledge with one another. As a team, we learned to communicate and work together, both in our small groups and with different teams.

The success of the entire project depended heavily on how well we could organize time and communicate our progress and required tasks with each other. We developed an effective way of communicating  and sharing files with one another that made this possible considering the size of the project team.

The journey was nerve-wracking, frustrating, challenging, but well-worth the result. The project helped us all see how many people working on their individual parts can come together and make a fantastic project. Success would never have been possible without everyone’s hard work and commitment.

All project code and video links below.

ALL LINKS:

Master Folder:
https://drive.google.com/open?id=1wNW3k21vQDh0-6DpUJAr7AAKEDJe3gyn

Control Panel Code & Central Brain Code:

Control Panel & Central Brain Final Code: https://github.com/vee97/SUPERCOMPUTER-FINAL/blob/master/Control_Panel_and_Brain_Animations.ino

Test video: https://vimeo.com/252927841

Animation Template & Communication Code:

OLEDscreenguide shared google doc:
https://docs.google.com/document/d/1ihCFUBChDTq37hIRFhA47UVAZowFMD6z4WTmEagy6Iw/edit?usp=sharing

Animation Template (Using shape drawing functions):
https://github.com/jgiovanni/SUPERCOMPUTER-Slave-Code/blob/master/slave_serial-oled.ino

Animation Template (With xbitmap example code):
https://github.com/natalielh/SUPERCOMPUTER-Slave-Code/blob/master/anim_xbitmap_001a.ino

Central Brain OLED Animation Code:

Vimeo link for condition animations(water, poison and fire animations):
https://vimeo.com/252647599

Github link for condition codes:
https://github.com/0102mika/conditions

References

https://www.arduino.cc/en/Tutorial/SoftwareSerialExample Software Serial example
http://forum.arduino.cc/index.php?topic=42603.0 Serial.write forum
https://forum.arduino.cc/index.php?topic=288234.0 Serial Port Communication
http://forum.arduino.cc/index.php?topic=380946.0 GND forum
https://github.com/olikraus/u8g2/wiki/u8g2reference u8g2 reference
http://coconauts.net/blog/2015/01/19/easy-draw-bitmaps-arduino/ drawing bitmaps to arduino
https://www.gammon.com.au/blink millis function tutorial

This entry was posted in Experiment 5. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

Post a Comment

You must be logged in to post a comment.

Use of this service is governed by the IT Acceptable Use and Web Technologies policies.
Privacy Notice: It is possible for your name, e-mail address, and/or student/staff/faculty UserID to be publicly revealed if you choose to use OCAD University Blogs.