As a response to my interest in the interconnecting senses I wanted to explore the relation between audio and visual stimulation. Since last semester I had the intention of creating a device that would translate audio signals into visuals as a form of synthesia. For this project I wanted to convert conversational language into visual language through the RGB leds. Using the intensity of the led colours provided I translated soft conversation to a sequential blue led pattern, normal conversational levels with a green led pattern and loud conversational levels with red led patterns. I find the interaction of individuals through conversation to be quite interesting as there are many aspects to interpret including the characteristics of one’s voice which is what the device is intended to translate.

As with any wearable project I pursue I started off with the electronic components of the project to make sure I knew how to hook up everything properly and to go through any trouble shooting before I made the final product. I purchased my first Mini Ardunio Pro which I am now in love with! As it is a great tool to use for prototyping and for the final product. I used right-angled headers with the mini arduino to create a great device to prototype using a wire wrapping approach.

I then proceeded to connect a electret mic along with a 103 capacitator and diode that would help to smoothen out the values the mic provided. I also started to fool around with RGB leds which were interesting to work with as I had experience using them previously but instead of having the 3-coloured pins connected to positive they all had to be connected to ground which made the process a but more complex and messy as I had to connect each Led to 3 resistors for each leg to make sure that the circuit would not blow up.

The schematic provided describes all the connections and calculations made and to make the schematic less complex I just indicated the amount of RGB leds used in two of the led patches. I also created my own version of an RGB led symbol and electret mic.


I wanted to encase the leds into a structure the wearer would put on their head and angle it in a way so that the patterns would be visible so I decided to use a face shield as it seemed to possess the qualities I needed but was lacking the headphones to cancel out the wearer’s hearing. I also proceeded to use a technique I used last semester to diffuse the light coming from the leds by painting them with white nail polish. As I connected the circuit together I would use a power source aka battery pack to make sure the connections were all properly hooked up before soldering everything together.
The video documentation is essentially how I would like the sequence of leds to correspond with one’s voice using a calibration coding.
Arduino Coding-
When coding the RGB leds I had to remind myself that to turn on the colour you would have to initialize a ‘LOW’ statement instead of a ‘HIGH’ as the led pins that trigger the colour are ground pins.
Fade RGB with colour correspondence:
int fadeAmount = 5; //how many points to fade the LED by
int brightness =0; // how bright LED is
int SensorPin = 0; //Sensor connected to analog pin 0
int BlueLedone= 3;
int BlueLedtwo= 9;
int BlueLedthree= 10;
int GreenLedone= 2;
int GreenLedtwo= 7;
int GreenLedthree= 11;
int RedLedone= 5;
int RedLedtwo= 6;
int RedLedthree= 12;
int SensorValue;
void setup () {
Serial.begin (9600);
pinMode (SensorPin, INPUT);
pinMode (BlueLedone, OUTPUT);
pinMode (BlueLedtwo, OUTPUT);
pinMode (BlueLedthree, OUTPUT);
pinMode (GreenLedone, OUTPUT);
pinMode (GreenLedtwo, OUTPUT);
pinMode (GreenLedthree, OUTPUT);
pinMode (RedLedone, OUTPUT);
pinMode (RedLedtwo, OUTPUT);
pinMode (RedLedthree, OUTPUT);
}
void loop () {
SensorValue= analogRead(SensorPin);
if (SensorValue-800 >= 45 & SensorValue-800 <=48 ){
analogWrite (BlueLedone, brightness);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (BlueLedone, HIGH);
}
else if (SensorValue-800 >=48 & SensorValue-800 <=51 ){ //range of values that fade in
analogWrite (BlueLedone, brightness); // LedPatchone & LedPatchtwo
analogWrite (BlueLedtwo, brightness);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (BlueLedone, HIGH); //turn off LedPatchone & LedPatch two
digitalWrite (BlueLedtwo, HIGH);
}
else if (SensorValue-800 >= 52 & SensorValue-800 <=55 ){
analogWrite (BlueLedone, brightness);
analogWrite (BlueLedtwo, brightness);
digitalWrite (BlueLedthree, LOW);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (BlueLedone, HIGH);
digitalWrite (BlueLedtwo, HIGH);
digitalWrite (BlueLedthree, HIGH);
}
else if (SensorValue-800 >= 56 & SensorValue-800 <= 66 ){
analogWrite (GreenLedthree, brightness);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (GreenLedthree, HIGH);
}
else if (SensorValue-800 >= 67 & SensorValue-800 <=77 ){
analogWrite (GreenLedthree, brightness);
analogWrite (GreenLedtwo, brightness);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (GreenLedthree, HIGH);
digitalWrite (GreenLedtwo, HIGH);
}
else if (SensorValue-800 >= 78 & SensorValue-800 <= 88 ){
analogWrite (GreenLedthree, brightness);
analogWrite (GreenLedtwo, brightness);
digitalWrite (GreenLedone, LOW);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (GreenLedthree, HIGH);
digitalWrite (GreenLedtwo, HIGH);
digitalWrite (GreenLedone, HIGH);
}
else if (SensorValue-800 >= 89 & SensorValue-800 <= 99 ){
analogWrite (RedLedone, brightness);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (RedLedone, HIGH);
}
else if (SensorValue-800 >= 100 & SensorValue-800 <=110 ){
analogWrite (RedLedone, brightness);
analogWrite (RedLedtwo,brightness);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (RedLedone, HIGH);
digitalWrite (RedLedtwo, HIGH);
}
else if (SensorValue-800 >= 111 & SensorValue-800 <=130 ){
analogWrite (RedLedone, brightness);
analogWrite (RedLedtwo,brightness);
digitalWrite (RedLedthree, LOW);
delay (500);
brightness = brightness + fadeAmount;
digitalWrite (RedLedone, HIGH);
digitalWrite (RedLedtwo, HIGH);
digitalWrite (RedLedthree, HIGH);
}
Serial.println (SensorValue-800);
}
Calibration RGB-Mic:
const int sensorPin = A0; // pin that the sensor is attached to
const int ledPin = 9; // pin that the LED is attached to
// variables:
int sensorValue = 0; // the sensor value
int sensorMin = 1023; // minimum sensor value
int sensorMax = 0; // maximum sensor value
void setup() {
// turn on LED to signal the start of the calibration period:
pinMode(13, OUTPUT);
digitalWrite(13, HIGH);
// calibrate during the first five seconds
while (millis() < 5000) {
sensorValue = analogRead(sensorPin);
// record the maximum sensor value
if (sensorValue > sensorMax) {
sensorMax = sensorValue;
}
// record the minimum sensor value
if (sensorValue < sensorMin) {
sensorMin = sensorValue;
}
}
// signal the end of the calibration period
digitalWrite(13, LOW);
}
void loop() {
// read the sensor:
sensorValue = analogRead(sensorPin);
// apply the calibration to the sensor reading
sensorValue = map(sensorValue, sensorMax, sensorMin, 0, 255);
// in case the sensor value is outside the range seen during calibration
sensorValue = constrain(sensorValue, 0, 255);
// fade the LED using the calibrated value:
analogWrite(ledPin, sensorValue);
}
Relatable/Inspirational Projects Include:
Isophone- By Auger-Lorizeau Isolates the body into a tank of water and encases the head into a dome formation so that all the wearer is concentrated on is the voice of the individual they are conversing with.
Veasyble- By GAIA isolating yourself from the environment using origami-like forms.
