Assignment 8 Update: Video Interaction with Ultrasonic sensor

One of the things I wanted to explore in this class was the relationship between sensors and video playback, and what kinds of content would work in an installation with sensors. I did a simple experiment to update my week 8 assignment, which was to create an interaction between Arduino and Processing via a serial connection.

I used an Ultrasonic sensor to try out an interaction with a video that zooms in and out on a small note. Your proximity to the sensor makes the video scroll along its timeline, in relationship to sensor values. The video itself zooms in and out creating a relationship of distance with the viewer’s distance. I wanted to reverse the interaction, so that the closer you get to the screen and sensor, the further away the object/subject in the video gets. On a conceptual level, instead of facilitating your viewing experience, it problematizes it, in a way that may frustrate the viewer or depending on content, make it fun to try to read the message. This is the basic interaction:

The quick video I tested it with is here. What needs some adjustment is the sensor value code, but that would require testing in an empty space. It works well enough to express the interaction outlined.

Arduino Code
const int analogInPin = A0;  // Analog input pin that the potentiometer is attached to

int sensorValue = 0;        // value read from the sensor

void setup() {
Serial.begin(9600);
}

void loop() {
// read the analog in value:
sensorValue = analogRead(analogInPin);

// print the results to the serial monitor:
//Serial.print(“sensor = ” );
Serial.write(sensorValue);

delay(10);
}

Processing Code

// Creation and Computation Assignment Week 8
// Based on Exercise 16-5 from Learning Processing by Daniel Shiffman

import processing.video.*;
import processing.serial.*;

Serial myPort;        // The serial port
int videoControl;

import processing.video.*;

Movie myMovie;

void setup() {
size(320, 240);
myMovie = new Movie(this, “readmerev.mov”);
myMovie.loop();
println(Serial.list());
String portName = Serial.list()[0];  //initializing serial port
myPort = new Serial(this, portName, 9600);
}

void draw() {
float ratio = videoControl / (float) width;
myMovie.read();
image(myMovie, 0, 0);
}

void serialEvent(Serial myPort) {  // serial analog read
int inByte = myPort.read();
println(inByte);
videoControl=inByte;
float ratio = videoControl / (float) width;
myMovie.jump(ratio*myMovie.duration());
}

Assignment 8 – Error / Video Brightness Controlled by Potentiometer

 

Here’s my Processing sketch with the printed sensor data at the bottom. It’s just a test to see how the potentiometer in Arduino would control video brightness. It should make the video controllable from dark to bright in a smooth predictable way, but instead it’s random amounts of brightness with no control. This isn’t my final assignment, but I can’t get past this point without some idea what the error is. I’m confused as to why a potentiometer with accurate 0-255 control in Arduino would produce this effect – erratic data input. Any help would be very much appreciated. Thanks!

 

 

 

Serial Communication

Missing last class made things a bit more challenging for assignment 8. I started with connecting the button to the arduino for a simple read. Using the example code I managed to control the color of the processing square:

 

IMG_2696

IMG_2697

IMG_2698

 

 

 

 

 

 

 

 

I tried using the second code example posted but kept getting a port error in processing. I moved on to the second exercise and played with the potentiometer and graph exercise. I replaced the potentiometer with the photo light sensor, and managed to get some interesting graph patterns.

IMG_2701

IMG_2702
IMG_2703
IMG_2706

I

 

 

 

 

 

 

 

 

 

 

 

 

I worked with the dimmer tutorial to control the brightness of the LED as you move the mouse. I then switched the LED with a servo motor. The motor spin in relation to where I moved my mouse. It worked for a couple of times and then in processing I saw a warning about the serial port. Processing stopped working, I couldn’t close the sketches and when I got back to Arduino, it was also and I couldn’t upload anything to the board. I reboot and try again and got the same problem. Not sure what went wrong exactly. Below is the code that I used and the images of the connections:

Arduino
const int ledPin = 9; // the pin that the LED is attached to

void setup()
{
// initialize the serial communication:
Serial.begin(9600);
// initialize the ledPin as an output:
pinMode(ledPin, OUTPUT);
}

void loop() {
byte brightness;

// check if data has been sent from the computer:
if (Serial.available()) {
// read the most recent byte (which will be from 0 to 255):
brightness = Serial.read();
// set the brightness of the LED:
analogWrite(ledPin, brightness);
}
}
Processing:

import processing.serial.*;
Serial port;

void setup() {
size(256, 150);

println(“Available serial ports:”);
println(Serial.list());
port = new Serial(this, Serial.list()[0], 9600);
}

void draw() {
// draw a gradient from black to white
for (int i = 0; i < 256; i++) {
stroke(i);
line(i, 0, i, 150);
}

port.write(mouseX);
}

 

IMG_2708

IMG_2709

IMG_2713

IMG_2712

Sensor Control

To combine processing and arduino, I want to make some work relate to my previous processing assignment.

Then, I decide to improve my second assignment, I want to use arduino to control my processing game.

After I looking at the tutorial I understand the value from arduino can be used as a variable value in processing.

First I start the project from the simple circuit.

Here is the Arduino Code

int analogPin = A0;
float temp = 0;

void setup() {
  Serial.begin(9600);
} 

void loop() {
  temp=analogRead(analogPin);
  temp=(map(temp,0,1024,0,5000)/10.0);
  Serial.println(temp); // println add Linefeed to my float
  delay(10);
}
here here is the processing code
import processing.serial.*;

int lf = 10;    // Linefeed in ASCII
String myString = null;
Serial myPort;  // Serial port you are using
float num;
float carx=0;
float x=300;
float y=300;
float speed = 2;
float gravity = -0.009;
int r;
int g;

void setup() {
  myPort = new Serial(this, Serial.list()[0], 9600);
  myPort.clear();
    size(600,600);
}

void draw() {
  background(255,0,0);
  smooth();

 //Draw a car
 stroke(0);
 strokeWeight(1.5);
 fill(100,100,255);
 rectMode(CENTER);
 rect(carx,500,100,50);
 fill(255,255,0);
 ellipse(carx-30,530,30,30);
 fill(255,255,0);
 ellipse(carx+30,530,30,30);
 carx=carx+4;

 if (carx>width+50) {
   carx=0;
 }

 //set CENTER mode
 ellipseMode(CENTER);

 //Draw Frog's feet
 stroke(110);
 strokeWeight(10);
 line(x-20,num+310,x-20,num+240);
 line(x+20,num+310,x+20,num+240);

 //Draw Frog's body
 noStroke();
 fill(r,g,0);
 ellipse(x,num+230,100,110);
 fill(255);
 ellipse(x,num+210,70,120);

 //Draw Frog's head
  noStroke();
  fill(r,g,0);
 ellipse(x,num+160,200,100);

 //Draw Frog's eyes
  noStroke();
   fill(r,g,0);
 ellipse(x-100,num+130,70,70);
 ellipse(x+100,num+130,70,70);

 //Draw Frog's eyes' ball
  stroke(110);
  strokeWeight(1.5);
  fill(255);
 ellipse(x-100,num+130,50,50);
 ellipse(x+100,num+130,50,50);
 fill(110);
 ellipse(x-90,num+130,20,20);
 ellipse(x+90,num+130,20,20);

 //Draw Frog's highlight
  noStroke();
  fill(255);
 ellipse(x,num+150,40,20);

 //Draw Frog's mouth
 stroke(110);
 strokeWeight(3);
 noFill();
 arc(x,num+170,100,50,0,PI);

if((carx>200)&&(carx<400) && (num>220)){
  r=int(random(255));
  g=int(random(255));

  }  else{
    r=225;
    g=252;}

  while (myPort.available() > 0) {
 myString = myPort.readStringUntil(lf);
 if (myString != null) {
 print(myString); // Prints String
 num=float(myString); // Converts and prints float
 println(num);
    }
  }

}
Now I can play the game by controlling the light sensor.

Assignment 8 when Processing contacts with Arduino

Thanks for Hajor’s sharing of video tutoring, I finally know what the serial contact means and what are the code means, it become much easier for me now.

It is just like the arduino with sensor can provide processing another variable, then use some code to play with changeable value, it’s like how we use with mouseX, mouseY.

I’ve try the button control example , but button only can provide 2 value, true or false, it is much more fun to play with the sensor or twist switch.

The following 2 project are all using twist switch and mouse to control what’s going on on screen.

The code is very easy ,it’s use the basic processing code ,but when it is arduino, it provide more random effect, which can create cool work.

 

Project 1 Art work

import processing.serial.*;
Serial port;
float a=0;
float val;
void setup()
{
size(600,600);
background(255);
smooth();
colorMode(HSB,360,100,100);
port = new Serial(this, “COM5”, 9600);
port.bufferUntil(‘\n’);
}

void draw()
{fill(0,0,100,2);
rect(0,0,600,600);
a+=0.1;
translate(mouseX,mouseY);
rotate(a);
fill(random(360),100,100);
line(-70,0,70,0);
ellipse(-70,0,val/2,val/2);
ellipse(70,0,val/2,val/2);
}

void serialEvent (Serial port)
{
val = float(port.readStringUntil(‘\n’));
}

 

Project 2 Disco Ball

 

 

 

 

 

 

 

 

 

 

 

import processing.serial.*;
Serial port;
int y=100;
float val;
void setup()
{colorMode(HSB,360,100,100);
size(600,600,P3D);
port = new Serial(this, “COM5”, 9600);
port.bufferUntil(‘\n’);
frameRate(10);
}

void draw()
{background(50);
translate(mouseX,mouseY,20);
rotateY(PI*frameCount/125);
lights();
fill(random(360),100,100);
stroke(255);
sphereDetail(10);
sphere(val);
}

void serialEvent (Serial port)
{
val = float(port.readStringUntil(‘\n’));
}

Arduino code (same)

//Define Pins
int potPin = 0;

void setup()
{
//Create Serial Object (9600 Baud)
Serial.begin(9600);
}

void loop()
{
int val = map(analogRead(potPin), 0, 1023, 0, 255);
Serial.println(val);
delay(50);

}

Serial Lab

lots of fun working through the tutorials:

I changed the look of the graph, adding colour, limiting the scope of the bars movements, and a smoothing function (not sure that is working)…

 

DATA TYPES

I went through the tutorials _ priting the  table of characters and  ASCII values was OK….

using two sensors to push a ball around on the screen was excellent. Again, it sparks the imagination…I had trouble getting the ball to conform to the height and width of the screen (using the light sensor as the input). I spent some time in Arduino, using a mapping function, but couldn’t get it…

OK – final experiment, I had a tough time … but I ended up with a colourful , albeit SIMPLE, sketch.

by linking the ellipse’s size and fill to xpos and ypos – which are linked to sensor input, I was able to create a cool sketching tool, that I think my little girl might appreciate. but I need to better learn the syntax / vocabulary of serial communication if I want to go any further.

 

here’s the code.

 

Assignment #8 – combining Arduino & Processing

I found most of the lab exercises (graphing input from the Arduino board, dimming an LED on the arduino using input from Processing, Arduino shaking hands with Processing) pretty straightforward.

After getting the handshake tutorial with 2 analog sensors and one digital switch to work as per original code, I played around with using the sensors to control ball size, background colour etc.

I had to modify the input and/or output ranges in the map command to suit the actual range of data and the animation.  Of the two handshake, call and response examples, I prefered the second one that allowed you to control the ranges for the map command in Processing.  It gave a lot more fine control of the animation.

Then I tried to use the arduino controls to control a sketch that is an experiment for our group project.  The idea is that the arduino sensors or controls control the width of the slot aperture in the proessing sketch and, therefore, the amount of the image behind that is visible.  In the interests of taking things one step at a time, I first got the sketch to work smoothly in Processing alone using mouseY to control aperture width.  Then I adapted this week’s exercises to use the arduino sensors and controls.  It can work with either the potentiometer knob or the photocell.  I prefer the latter b/c I can make the slot narrower when light levels are higher and vice versa (just like your eye’s iris closes in bright light conditions and opens in dark).

Unfortunately, the results were very choppy and unstable (unlike the smooth results from the Processing-alone version).   I played around with delays, frameRate to no avail.  I tried to add capacitor to the breadboard to smooth out the current as per Jim’s suggestion, but with no change – but I wasn’t sure I was installing it correctly.  I will ask him to show me what he had in mind this evening.

The Arduino code is straight out of this week’s tutorials.  Here’s the Processing code at this stage:

// This program takes ASCII-encoded strings
// from the serial port at 9600 baud and uses them to control the width of
// a slot aperture in the foreground of the Processing sketch.
// The width of the slot determines how much of the image behind is visible
// range 0 to 1023, followed by a newline, or newline and carriage return

// Created 8 November, 2011
// by Anne Stevens

PImage bg;
//PImage shadow;

import processing.serial.*;

Serial myPort;        // The serial port
int xPos = 1;         // horizontal position of the graph

void setup () {
  // set the window size:
  //size(400, 300);
  size(800, 600);
  frameRate(30);
  smooth();
  bg = loadImage("lamb.jpg");
  // set inital background:
  //background(0);
  //background(loadImage("lamb.jpg"));
  //shadow = loadImage("black.jpg");

  // List all the available serial ports
  println(Serial.list());
  // I know that the first port in the serial list on my mac
  // is always my  Arduino, so I open Serial.list()[0].
  // Open whatever port is the one you're using.
  myPort = new Serial(this, Serial.list()[2], 9600);
  // don't generate a serialEvent() unless you get a newline character:
  myPort.bufferUntil('\n');
}

void draw () {
  // everything happens in the serialEvent()
}

void serialEvent (Serial myPort) {
  // get the ASCII string:
  String inString = myPort.readStringUntil('\n');
        println(inString);
  if (inString != null) {
    // trim off any whitespace:
    inString = trim(inString);
    // convert to an int and map to the screen height:
    float inByte = float(inString);

    // map low input value to height and high to 0 so that aperture closes when
    // light on the photosensor is brightest and opens widest when
    // light levels are lowest, as the eye behaves
    //  The input range is adjusted to match the actual input range of my
    // light sensor.  Change the range as req'd to suit local light conditions.
    inByte = map(inByte, 425, 750, height, 0);

    // draw the photo image
    noStroke();
    image(bg,0,0,width,height);

    //draw two black rectangles on top of the photo
    fill(0);
    noStroke();
    rect(0, 0, width, height/2 - inByte/2);
    rect(0, height/2 + inByte/2, width, height);
    //delay(6);
  }
}

Jim also suggested I might need to average out the readings using an array.  I assumed that he meant in Processing rather than Arduino.  Here’s my attempt, though it doesn’t work.  I would like some input if possible.  As I said, this is a working idea for our Group project, so we need a solution.

// This program takes ASCII-encoded strings
// from the serial port at 9600 baud and uses them to control the width of
// a slot aperture in the foreground of the Processing sketch.
// The width of the slot determines how much of the image behind is visible
// range 0 to 1023, followed by a newline, or newline and carriage return

// Created 8 November, 2011
// by Anne Stevens

PImage bg;
//PImage shadow;

import processing.serial.*;

Serial myPort;        // The serial port
int xPos = 1;         // horizontal position of the graph

// for the array
int numReadings = 5;
float [] inByteArray = new float [numReadings];
int inByteIndex = 0;
float inByteTotal = 0;
float inByteAve = 0;

void setup () {
  // set the window size:
  //size(400, 300);
  size(800, 600);
  frameRate(30);
  smooth();
  bg = loadImage("lamb.jpg");
  // set inital background:
  //background(0);

  // List all the available serial ports
  println(Serial.list());
  // I know that the first port in the serial list on my mac
  // is always my  Arduino, so I open Serial.list()[0].
  // Open whatever port is the one you're using.
  myPort = new Serial(this, Serial.list()[2], 9600);
  // don't generate a serialEvent() unless you get a newline character:
  myPort.bufferUntil('\n');
}

void draw () {
  // everything happens in the serialEvent()
  //draw two black rectangles on top of the photo
}

void serialEvent (Serial myPort) {
  /*
  // get the ASCII string:
   String inString = myPort.readStringUntil('\n');
   //println(inString);
   if (inString != null) {
   // trim off any whitespace:
   inString = trim(inString);
   // convert to an int and map to the screen height:
   float inByte = float(inString);
   */

  //load readings into the array
  for (int i = 0; i < numReadings; i++) {
    // get the ASCII string:
    String inString = myPort.readStringUntil('\n');
    //println(inString);
    if (inString != null) {
      // trim off any whitespace:
      //inString = trim(inString);
      float inByte = float(trim(inString));

      inByteArray[i] = inByte;
      inByteTotal = inByteTotal + inByteArray[i];
    }
    inByteAve = inByteTotal / numReadings;

    // map low input value to height and high to 0 so that aperture closes when
    // light on the photosensor is brightest and opens widest when
    // light levels are lowest, as the eye behaves
    //  The input range is adjusted to match the actual input range of my
    // light sensor.  Change the range as req'd to suit local light conditions.
    float aperture = map(inByteAve, 0, 1023, height, 0);
    println(aperture);

    // draw the photo image
    noStroke();
    image(bg, 0, 0, width, height);

    //draw two black rectangles on top of the photo
    fill(0);
    noStroke();
    rect(0, 0, width, height/2 - inByteAve/2);
    rect(0, height/2 + inByteAve/2, width, height);
    //delay(6);
  }
}

On a general note, it may be old school, but I find it really helps to write code out long hand, even code that you plan to copy (left column) and modify to your purposes (right column).

Chicken Scratch

Stopping Time & Freezing Gravity

This week, I wanted to experiment with time and space. I thought that using the information we learned in Processing regarding ellipses that drop because of programmed gravity would be the simplest way to demonstrate this. I combined simple code for Processing with simple code for Arduino (so they would be able to talk to each other).

When I ran into some problems with the ball being visible in Processing, I went back to the print line code to see what vales the photo resistor was picking up. This helped clear up my first problem and assisted me in being able to trouble shoot the second issue with the program code. After all the kinks were worked out, I added a second ball for kicks and visual interest.

My attempt to freeze, trap or stop time and gravity comes into play when the photo resistor receives less light–when this happens, time and movement stop. I like using your hand to stop time in this scenario because the body becomes an interactive and controlling factor.

At the end of this post, there is a video of my project in motion. After testing out the video in the post, it seems the video is nearly illegible / un-viewable because it is so small. Sorry, if it’s difficult to interpret.

Here is a picture of how I set up my Arduino & breadboard:

 

 

 

 

 

 

 

 

 

Processing Code:

import processing.serial.*;

int lf =10;
String myString = null;
Serial myPort;
float num;
float x = 100; // x location of ellipse
float y = 0; // y location of ellipse
float speed = 0; // speed of ellipse
float gravity = 0.1;

void setup() {
myPort = new Serial(this, Serial.list()[0], 9600);
myPort.clear();
size(500,700);
smooth ();

}
void draw() {
background(255);

fill (20, 75, 125); // first circle
noStroke ();
ellipseMode (CENTER);
ellipse (x, y, 20, 20);

fill (120, 75, 125); //second circle
noStroke ();
ellipseMode (CENTER);
ellipse (30, y+10, 20, 20);

while (myPort.available() > 0) {
myString = myPort.readStringUntil(lf);
if (myString != null) {
print(myString);  // Prints String
num=float(myString);  // Converts and prints float
println(num);
}
}
myPort.clear();

if(num<450)
//Display the square
{

y = y + speed;
speed = speed + gravity;

if (y > height) {
speed = speed * -0.95;
}

}
}

Arduino Code:

int analogPin = A1;
float temp = 0;

void setup() {
Serial.begin(9600);
}

void loop() {
temp=analogRead(analogPin);
temp=(map(temp,0,1024,0,5000)/10.0);
Serial.println(temp); // println add Linefeed to my float
delay(10);
}

Stopping Time, Freezing Gravity Video

 

Assingment 8: Arduino and Processing (Serial)

I wanted to use one sensor to control two sets of data within processing. I decided to use the potentiometer, and send this data serial to processing. I wrote a processing sketch that would map the pot values between 0 and 5 to control which image filter is applied to a live video stream. I also mapped the pot values to use within certain filters parameter value.

Here is a video of the setup:
CreatCompAss8

Video also Here

arduino code:

and processing code:

import processing.opengl.*;
import processing.video.*;
import processing.serial.*;

Capture grabber;
PImage camImg;

Serial serial;
int filterSerial = 0;
float filterValue = 0;

void setup() {

size(640, 480);
frameRate(30);
grabber = new Capture(this, width, height, 30);
camImg = createImage(grabber.width, grabber.height, RGB);

serial = new Serial(this, Serial.list()[0], 9600);
}

void update() {

if (grabber.available() ) {
grabber.read();
camImg.copy(grabber, 0, 0, width, height, 0, 0, width, height);
}

}

void draw() {
background(0);
update();
image(camImg, 0, 0);
switch(filterSerial) {
case 0:
filter(GRAY);
break;
case 1:
filter(INVERT);
break;
case 2:
filter(POSTERIZE, filterValue);
break;
case 3:
filter(ERODE);
break;
case 4:
filter(BLUR, filterValue);
break;
case 5:
filter(DILATE);
break;
}

}

void serialEvent(Serial port) {
int p = port.read();
filterSerial = (int) map(p, 0, 255, 0, 5);
filterValue = map(p, 0, 255, 0, 20);
println(“Filter Serial: ” + filterSerial);
}

Assignment 8 – Serial Communication Lab

Working thru the lab – I had no problem with graphing my potentiometer. Then switched it up using a photo sensor. The hardest part of this one was using my left hand to change the light reading while doing an image capture with my right hand. The graph illustrates a rapid tapping of the sensor to block the light.

 


 

 

 

 

 

 

The dimmer tutorial was straight forward. I did not have any 220 ohm resistors so used a 330. It didn’t work but I think it had more to do with how I wired the Arduino. Anyway, I did replace the resistor and re wired the board (looking at an earlier lab using an LED) and it worked really well. Switching up the LED with the Servo Motor was a bit more challenging. I looked at both the Code examples in the Servo Example folder and chose the Knob example to emulate. It “seemed” to work but I felt there was very little control using the Processing gradient. Perhaps my code is wrong. The servo would sweep back and forth if I moved the mouse quite quickly and beyond the borders of the screen? And it also jumped around a bit when I first opened the Processing sketch.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now for data types; The tutorial for serial call/response was a bit challenging. I see how “doing” is “learning” because I often make mistakes wiring and have to figure that out. The Processing sketch worked fine but the fun began when I tried to change the ball colour and background. Changing size is easy. I think that I have to use an if/else statement in the draw function, and so tried a lot of different things. At first I tried an if statement based on the boolean variable in the code. Now I think that I need to look carefully at the Arduino code and perhaps write an if/else statement based on information received from the button sensor. But that still doesn’t work. I think I need to move on. If I do figure it out, I will post the new code!