Processing Fishing


For the water park project I decided to use colour tracking to affect a processing animation. I have created a “fish tank” with various coloured laser cut fish and “barnacles” that the user fishes out of the tank using a magnetic fishing rod. When the fish or barnacle comes into the webcam’s view, the colour is detected and the swimming fish in my processing animation change colour based on the fish or barnacle you have caught. This project is interactive for the user because they are physically affecting the change in the animation and the user can see this change. It fits the theme of water in three ways. First, thematically, the visual elements are related to fish and fishing. Second, the fish and barnacles sit in a tank of water waiting to be caught. Third, the processing animation illustrates swimming, undulating fish. I modified code from:



photo 3 (16)

First I decided to have “fish” as my colour tracking objects so I had them laser cut from transparent blue, orange and green plexi. My plan was to use these fish alone and combine them to create mixed colours. (explained in experiments)


photo 4 (15) photo 5 (9)


Once I abandoned my original idea of having the fish on sliders, I decided to change my vessel to a fish tank and have a magnetic fishing rod to retrieve the coloured pieces in the tank so the webcam could view them as the came out of the tank. I made this fishing rod by drilling a hole in both ends of a square dowel and feeding a string through both holes. I then glued a magnet to the long end of the string.

photo 4 (14)photo 2 (18)


The laser cut fish outfitted with magnets. All the pieces ready to go into the tank.

Project Context:

This is an example of a project using Processing and OpenCV. This project is similar to mine because it is also a one player game using the same software tools. Instead of colour tracking, this project uses motion detection by comparing the current frame and the last frame. If there was movement, a bubble pops, if not, a bubble in drawn. The goal is to pop all the bubbles. His code also has to update constantly to look for new movement whereas my code constantly updates to look for new colours. Like my game, there is no set ending or something that happens in the code once all the bubbles have been popped as his project is also a basic first start. I would have liked to add possibly a different processing animation that occurs once all fish and barnacles have been caught.

Project Experiments

1: This is my first assignment from Body Centrics using Processing

For this assignment I chose to use a sensor I had never used before, the flex sensor. The first human movement that came to mind with regards to bending or flexing was exercise. This set up of sensors determines how effectively someone is performing specific exercises; in this case, a squat.

Screen shot 2014-10-05 at 11.08.34 PM


(click to enlarge)

Flex sensors are attached behind each knee and a light sensor is placed under the person performing the exercise. The maximum is reached when both flex sensors are fully bent by the knee and when the light sensor reads 105 (in this case). At this point, the knees are fully bent in a squatting position and the body is low to the ground covering available light to the light sensor. I modified your code by changing the layout, colours, and circle positions. I then made it so that the circles grow once they reach their respective maximum so as the person performs the exercise the circles will grow and shrink accordingly with a large circle for each sensor being the goal.

Screen shot 2014-10-05 at 10.29.48 PM

Screen shot 2014-10-05 at 10.29.41 PM

Screen shot 2014-10-05 at 10.29.53 PM

Screen shot 2014-10-05 at 10.19.55 PMScreen shot 2014-10-05 at 10.22.49 PM


These are screen shots of the Processing animation. The first image shows the resting body where both flex sensors are straight and flat and full ambient light is reaching the light sensor. The second image shows the sensors reaching their maximums. All maximums and minimums in the code have to be altered and calibrated in each new room due to differing light sources. I chose a yellow circle for the light sensor and two pink circles for the flex sensors.

photo 1 (13) photo (18)


For the set up, the breadboard and arduino sit on the floor underneath the person. The two flex sensors are affixed to velcro bands that wrap around above the knee. The flex sensor is then stuck onto the leg underneath the knee bend. This set up could be modified to test the efficacy of a push up (with flex sensors at the elbow bend) or a sit up (with the flex sensors along your stomach muscles).

2: Processing sketch of stick figure me

This is the second time I used Processing in class. I made a stick figure of myself that shows when the mouse is pressed. This was a good way for me to learn how to create a drawing in Processing from scratch. I started with a 100,200 size and centred the middle line, triangle skirt and ellipse head. I then added two arcs for bangs with a gap in the middle for a part and two long rectangles for hair. I then added triangle shoes and two small ellipses for eyes. I then added and if statement for mousePressed so the function: drawTegan will appear when the mouse is pressed or not if else.

3: Face tracking Body Centrics assignment


image (50)

You will need white eyeshadow or face paint, black eyeshadow or face paint, foundation, tape and an optional  curling iron.

photo 1 (18)

First I did my hair. I curled all my hair to give it some body. I then added two buns on either side of my head to obscure the shape of my head. I also used a large front section to swoop over one of my eyes and pinned it to the side. The long curls also obscure the shape of my jawline.

photo 3 (15)

I then wanted to remove dimension from my other ocular region. I used white eyeshadow all around my eye and on my eyelashes. I then used foundation to remove any darkness from my eyebrow.

image (44)

My next step was obscuring the bridge of my nose. I taped a large triangular shape across my nose and filled it in with black eyeshadow.

image (47)

To remove dimension from my mouth region, I taped another shape onto my lips and filled the lips in with foundation and the skin part with black eyeshadow to give the illusion of opposite light and dark areas.

image (48)

Final look

This anti face tutorial is based on Adam Harvey’s CV Dazzle project and follows his tips for reclaiming privacy.

1. Makeup

For my look,  I avoid enhancing or amplifying any of my facial features. Instead I use dark make up on light areas and light make up on dark areas to confuse face tracking technology.

2. Nose Bridge

The nose bridge is a key element that face tracking looks for. I obscured it by using dark makeup in an irregular shape over the nose bridge area.

3. Eyes

The position and darkness of the eyes are key features for face tracking. Both ocular regions are dealt with here in different ways. A large section of hair completely obscured one and light makeup conceals the dimension of the other ocular region.

4. Masks

No masks or partial masks are used to conceal my face.

5. Head

Face detection can also be blocked by obscuring the elliptical shape of one’s head. I used buns on either side of my head, and long curls to break up the roundness of my head and jaw.

6. Asymmetry

Most faces are symmetrical and face detection software relies on this. All hair and makeup elements are different on each side of my face.

 4: Colour Tracking

This is me experimenting with colour tracking using the gesture based colour tracker Nick provided us with. When I mouse press something in view, the colour is recorded and the tracker draws a blob of that colour over the object and anything else that it that colour. Here I learned that the tracker is not as specific as I would like and picks up on things that you don’t want it to because of lighting. This issue came up a lot when I was testing my project. Anything I was wearing, the colour of my skin, the colour of the wall behind the tank or even low lighting would make the colour tracker pick up on things I didn’t want it to.

5: Modification of fish_sketch

At this point I had decided to modify this sketch for my game project. Here I try changing variables like size, position, fill colour and stroke weight. When I make the background colour (0,0) the fish do not update their position. Instead, you see the track of where the fish has been the whole time within the sketch and therefore shows the colours that have been tracked throughout the game. This could become part of the game in the future potentially. It could become a multi-player game where whoever’s colour is of the highest percentage on the screen at the end would win. It reminds me of this:


6: Fish sliders

photo 2 (19) photo 1 (20)


My first idea was to have this plastic container as my tank. I laser cut rectangles of plexi that I glued together to create channels for a slider that the fish would be attached to. The player would change the position of the fish to change the colour of the Processing fish like the final game, but in this set up, the fish could be placed in front or behind each other. Then the colour tracker would be able to pick up specific colour mixes as the plexi is transparent and would create new colours (two fish together or all three together). In the end I abandoned this idea because the sliders I made didn’t work very well and weren’t sliding properly. Also the fish would not stay attached to their sliders.

7: Fish transparency

The next issue with my original idea was that the colour tracker had trouble identifying the colour mixes as they ended up to dark. In this video I attempt to have the colour tracker identify orangeblueFish, bluegreenFish and allFish. They all just end up a dark purplish colour. Another issue is that the surface of the fish reflect back the computer screen and confuses the tracker. At this point I decided to switch to my final “Processing Fishing” idea. Beyond the laser cut fish I also wanted other coloured items that the tracker could identify.

8: Foam sea creatures

photo 3 (17) photo 1 (19)


At this point I decided to try tracking these foam sticker sea creatures. I first tried gluing magnets to them so they could sit in the tank with the laser cut fish but no amount of weight would make them sink to the bottom. I then decided to attach them to a stick so the player could add them into webcam view but this didn’t really fit with the fishing idea I had come up with. I then decided to make coloured “barnacles” as extra tank items that do sink to the bottom. In this video I show the tracker identifying all the colours of the extra foam creatures. Some were too close to the colours of the laser cut fish so this confused the colour tracker.

Comments are closed.

Use of this service is governed by the IT Acceptable Use and Web Technologies policies.
Privacy Notice: It is possible for your name, e-mail address, and/or student/staff/faculty UserID to be publicly revealed if you choose to use OCAD University Blogs.