Author Archive

Tinker Box

Abstract

Tinker Boxes are physical manipulatives designed for digital interaction. Based on the concept of MiMs (Zuckerman, O., Arida, S., & Resnick, M. (2005)) which are “Montessori-inspired Manipulatives”. The boxes are low fidelity devices used to bridge the physical interaction with the digital world. They are aimed at children from 5 to 7yrs but can be extended to any age depending on the frontend software designed to fit the interaction. This iteration of the software looks at using it as a scaffolding tool to teach kids how to recreate or understand the making of a physical object, Lego toys in this instance. The plan to extend it to build other educational games and interactions to explore concrete and abstract concepts.

img_3288

Introduction

What can I tinker with next, the goal of Experiment 5 was to take a project from the past weeks and add to it in some meaningful way? This could be anything but “anything” is a very large canvas given 1.2 Weeks from concept to completion. I would like to say it was all clear from the start but it was murky on what this next step would look like. I tried to think about what I liked about the old project and what I did not it was quite clear the biggest pain point was the potentiometer breaking the interaction and dictating how far the kids could make the character chase each other before having to reverse and go backward to add more interaction.

This whole experiment would be to figure out how to use the Rotary Encoder, the initial idea was to change the whole first version of the box and add the rotary encoder to it, this would mean reverse engineering the hardware to fit in the new, it was not really worth it as the first version worked quite well as proof of concept and I wanted to keep it like that.

I then decided to use the RE[Rotary Encoder] to create a new kind of interaction but also examine critically what it is I was building, I used the case study assignment to dive deeper into what the interaction meant and how I could position the work in a meaningful way. In the paper “Extending Tangible Interfaces for Education: Digital Montessori-inspired Manipulatives”

Methodology

So what can I make interactive and make it meaningful to children? This was the question I kept asking myself, it’s easy to build an interaction but to have it meaningful and pedagogical is where the challenge is. I looked at my kids for inspiration, where I usually start, they are learning through play every single day but we seem to miss it.

I’m jumping ahead a bit because before I could even imagine what kind of interaction I wanted I needed to get the rotary encoder working and sending data, this may seem like a no-brainer for a coder, but for person new to the coding world of p5 and Arduino it was a critical first step else I have No Dice!

The base code for the project was a mix from the class code from Nick and Kate and also from Atkinson, multiwingspan.co.uk. I got the encoder sending a signal and then used the JSON library to parse the code so I could read it in p5.js. This is not the best way to do it I realized in retrospect and as I need to map different variables based on the length of the sprite animation I will be controlling. the better way to do it is to set a large range in Arduino and then map that range down to what I need based on each individual interaction. This is a bit technical in understanding but if you do venture into using my code it is something to keep in mind when you need to modify it to your needs.

img_3250

OK now that I had that and the button working I assembled all of the hardware even before I could get the software working, why would I do that, well basically because time was running out and once you write software, debugging and refining is a rabbit hole you can spend all of your time doing ill the cows come home, and I may never get to finish the physical hardware. I had this happen on other projects where the software take precedent and the hardware ends up being presented on a proto-board as there is no tie for refinement and fabrication.

img_3266

The circuit is pretty simple as you can see in this Fritz diagram below, it uses:

  • 1 Rotary encoder
  • 1 button
  • 1 Arduino Micro Original

That’s it, the circuit is also very clean so I could get to the main task of creating the interaction.

fritz

Once the circuit was done, I built the housing and the soldered all the components on to the PCB board.

img_3265

Software

I now looked at all the possible interactions I could create, using this rotary dial.

The basic idea is Turing up or down a value, you can then map this value to anything you like, in my case I decided to use sprite animations.

Coming back to observing my son play with lego, he would iterate and create new creations, cars, safe’s, vending machines, the list was exhaustive, he would look through youtube videos to follow along or just iterate, he would then share these creations with us at home and take them to school to show his friends. The thing is they could see the completed work not the process of getting there or even the individual parts that made the whole, this sparked an idea based on other stop-motion projects I had seen, with my son’s permission I broke apart his creations brick by brick and shot them using an iPhone and a tripod, I then used that to create sprite sheets which were controlled by the Think boxes rotary encoder. It took a bit of time to figure out how the sprite sheets worked and what was possible but it worked and the end result was satisfying, I then used the button to change the sprite and show another sprite animation, in this way I could get the user to scroll through the different creations I had created animations for.

img_3291

The interaction was automatic, there were no instructions needed, people turned the knob and clicked the button, I had built on their past experience of what buttons and Knobs do, it was now just a matter f changing the software to create a pedagogical experience for the child.

Some ideas I came up with based on this interaction are:

  • Simple Machines: where the box could be turned on its side and the knob is replaced with a crank, lending itself to simple machines, like cranes, fishing poles, ratchets, pulleys etc
  • The process of folding and unfolding has numerous pedagogical uses, least of which is just the wonder of seeing what is inside something, like for instance the layers of the earth’s crust, making planets move and just rotating objects on a different axis.
  • This makes the MiM very versatile yet simple in its interaction, the triangles complete when the software fits the user and the interaction.

Feedback

Some of the feedback was I should use this as an educational tool for products building IKEA furniture, pitch it to the company to create stop-motion videos for showing the different steps.

There was also interest in seeing how two of these devices could change the interaction if they controlled different aspects of the same Object/Interaction.

Summary

I would like to explore this prototype further and build more tinker boxes, which network or are even wireless, I had an early idea of building a wireless interaction but Nick said it might be a delayed interaction because of using a server like pubNub, I will look to see if there is any way to directly interface with the Mac/pc without the use of a third party software.

References

Zuckerman, O., Arida, S., & Resnick, M. (2005). Extending tangible interfaces for education.
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI 05. doi:10.1145/1054972.1055093

Atkinson, M. (n.d.). MultiWingSpan. Retrieved from http://www.multiwingspan.co.uk/arduino.php?page=rotary2

GitHub code can be found here: https://github.com/imaginere/Experiment5DF

Tinker Theatre


Overview

Tinker Theatre is a small interactive controller which mimics an old-school puppet box, the ones where you move around things with strings or hand puppets.  That along with a recent game I made at my company a few months back, were the primary inspiration for the project. The interaction is fairly simple and it was designed for an age group  of 3-7 yr olds, the knobs and buttons are ways to interact with a screen, kids at the age of 3-5 might have trouble using a mouse or even figuring out touch controls, this is much like a toy they can tinker with and get instant feedback on the screen, also it is meant tool from kids to tell stories and imaginative play. This is version 1.0, which is more of a proof of concept, if taken further it can have more sophisticated interactions, animations, sound effects and also can be ported to other games much like a joystick, the game would need to have hooks for the sensors and controllers. The project can also extend as a DIY project for older kids 8+ to build one these simple controllers and test them with the game which could be placed online.

Demo Video:

img_3058

Process Journal

This has been a fun project to work on but it did not start that way, having worked as a group on the previous projects this one seemed daunting as it included code on the Arduino side and the p5 side of things.

The initial code we were learned seemed like it would be really intensive to setup each sensor on both sides and then program it to do something interesting.

I went straight to what Kate said about one of the projects in Experiment 2, which was “This project was well scoped”. This was something I tried to keep in mind when working on the project scope.

I also leveraged a game I made a few months back for design assets, I got clearance from my employer to repurpose some of the assets so they could be used on this project.

The idea really came together after we got the code from Nick about using Jason to connect the sensors and the potentiometer. the original game had already some functionality I wanted to mimic, but it had a lot more animation, p5 is kind of limited when it comes to being and game engine, everything you need to be interactive has to be coded in to function. a simple thing like loading a complex sprite sheet is not built into the system, there is a generic sprite sheet loader but that has not been updated and it uses rectangular blocks to load a sequence of images which can be very resource intensive and make your sprite sheets very large.

Improvise
So I had to improvise instead of using animated assets, I choose assets to be tinkered with which create a sense of play and animation.

I looked at old school toys which has simple controls which made things move back and forth, much like the original games inspiration.

This can also be seen in shadow puppet boxes kids make in grade 3/4 by kids for science projects, my son had made one of these at the start of this year with a little help from me 😉 Here you use the sticks to move the fish back and forth creating a sense of movement and play.

shadowbox
F, Andre.(2018)

Arduino side of things:

Potentiometers, Light Sensors, and Dials, Oh My!
With this in mind I had to figure out the following:

Arduino:
This took a little bit of studying Nick’s and Kate’s code on getting the json to add more sensors and then adding the appropriate mapping on the p5 Side.

Mapping the light sensor was a real challenge and I realized I did not really have to map it after all as the readings were in the range I needed, its been a challenge to figure out the mapping function like most p5 functions the code reference shows only the base syntax but in reality there are many other parameters to the function.

The one useful thing I found in debugging the code is that you can get to the base function in the p5.js library and look at how the function is written and what parameters it is looking for when called, this takes time but it will broaden your scope for using the inbuilt functions.

I also found getting a javascript reference book for, things like debugging using the console is very useful.

Basically, you can pause your code and step through your code till the code breaks and you can figure out why from that point, easier said than done I know but it a start.

Design and Code the frontend using p5

I then started making the prototype on p5 end with some crude graphics. The reason I did not use final graphics to prototype because it gets distracting, I tend to get fixated on making things align correctly and look pretty instead of working on the functionality when I have crude graphics I know the goal is and I’m focusing on is UX and not design.

This is what it looked like:

proto

Once I got these moving back and forth, I knew I had a starting point for the functionality, I have to mention Omid Ettehadi at this point as he helped me immensely in understanding the code and also showed me how I could cycle the character back on the other side of the screen, when I saw that was possible I wanted the potentiometer to keep going but that is not how a potentiometer is built.

I asked Nick if there was something else I could use to get the constant rotation and he pointed me to a Rotary Encoder, which could do that but the code for that is different and so is the setup of it.
[https://www.creatroninc.com/category/rotary-encoder/component/switch]
Given I had three other projects to deliver in the same week, this was out of scope and I did go and buy one to add to the board but did not get time to figure it out.

It might be something I will add to experiment 5 if I end up using this project to come back to.

Here is a list of things I decided to add to the front end functionality:
– 2 Characters which can change expressions so 8 in total
– Parallax scrolling background with 3 layers
– DarK background for night
– 2 sun characters one that changes when the light gets more intense
– Music that can be raised or lowers to add mood

I then moved into getting the final detail in place, I worked out a small story arc you could create with the characters:

02a01a

Hardware

Part List:
3 Potentiometers = 2x5K and 1x1K
3 Rotary Knobs 2xA2 and 1GTE Knob- Medium
1 7mm light dependent resistor – optre-000007
1 Yellow Push Button momentary – 48VDC 2A

*Light Sensor
I decided to add a light sensor and a button instead.

Functionality breakdown:
– Potentiometer: Used to make the characters back and forth
– Rotary Knobs: Make it easy to differentiate the volume control and the character control
– Light Resistor: Used to change the background and the sun character
– Yellow momentary push button: Used for parallax scrolling of the backgrounds

The circuit

Breadboard

The breadboard was pretty simple and I kept it clean by using flat wires instead of the prototyping wires which get in the way of looking at the circuit.

I then removed each component and soldered it on a half size prototyping board. This PCB has the basic layout of a half-sized breadboard. This helps you replicate it exactly as your plastic protoboard, with some exceptions of wire length.
protoboard
working-only-with-potentiometer_bb

Fabrication

The next step was the Fabrication, again I looked at the scope and decided to reuse the laser cut box we built for our last project but did not use.

I did a quick sketch of how I wanted the box to function and the holes I would need to drill in the maker lab. Also helped me finalize the button I would need to make it feel more tactile and kid-friendly. I also use it to plan how big the protoboard should be and the length of the wires in the box.
02p01p

03p

04p

Troubleshooting

These were problems I had to solve for:

  • The Light sensor was too sensitive and the background kept flickering when I turned the dial
    Solution: Raise the sensitivity level and add three levels of sensitivity in the code
  • The characters were not responsive:
    Solution: Disable the console.log() function which was tracking all the sensor data
  • The hole on the side of the box was not working for the potentiometer as it was too thick.
    Solution: Move the volume to the front

Github Code:

https://github.com/imaginere/Experiment3_TinkerTheatre

Reference:

Ferrao, A. 2018 [Shadow Box]

Game Art Usage thanks to https://mpower.tvo.org/  TVO

Code Help: Omid Ettehadhi and Mark Moyes

Fabrication guidance: Reza Safaei (Maker Lab OCAD U)

Illustrations by: Frank Ferrao

Humans Vs Zombies – Hack, Slash, Bang Bang!


An experiment with 20 screens and adventures in p5.js

– Omid Ettehadi and Frank Ferrao

cautiontape

Project Scope:
Humans Vs Zombies is a human interaction game that uses smartphones based on the classic playground game, Rock Paper Scissors. Initially, our goal was to make an entirely digital experience, but the project evolved to a simpler version which focused more on using the smartphones as tools to enhance the user’s experience in playing the game. The change was due to time and technology constraints, in addition to a better understanding of the assignment after learning more about multiscreen projects and their nature to create an interactive experience for their audience.

Code:

Github: https://github.com/Omid-Ettehadi/Hack-Slash-BangBang

Working URL: https://webspace.ocad.ca/~3170557/Creation&Computation/Experiment1/index.html

We have two version of the code for the final project.

The chocolate eclair version was written mainly by Omid where Frank contributed to the User testing and the overall user testing, iterating with the flow of the experience by mostly removing features we had planned on but was adding complexity to the usability.

Frank worked on a simple version, which included no dynamic switching of Human and Zombies in the interface, there are two urls with identical code but the graphics and sound effects have been changed in for each of those url’s.

The other main difference is that in Frank’s version you can just keep shaking the device to bring up the next random card, in the advanced version you have to reload the page to start a new game.

This has its advantages if you are playing solo, but because ours was group experience we did not want people to keep playing solo games as we would have liked them to engage in the group experience of winning as a team.

cautiontape

Design and UX:

There are two phases to the designs we went through, in the initial part we wanted to make it a full-fledged app and Frank went down the rabbit hole of getting a lot of art done for the project, he asked is his son to make some Zombie drawings, that he could use in the game. The initial concept did not have humans in the mix, we were calling it **Self Actualized Zombies**, where your goal was to become a zombie first and then become a mega-zombie as a group of 5. The game would have people collect Zombie body parts till they were a complete Zombie and then join body parts to become an S.A.Z as a group.

Planning with a complicated set of ideas and multiple UI screens:

flowuiold

Angry Zombie on the Right by Emily Age 7

We spoke with Nick after the first week of planning for this style of game and he steered us away from such a complex endeavour. We then went into our PHASE 2 planning, where we got rid of all in the in-app Scorekeeping, Multiple UI screens, and the final payout with tons of in-app messaging which would hard to read and understand in the dark while playing the game anyway.

We tried hard to simplify UX and UI. In doing this we looked at several criteria for the design

Character Design and UI Design:

Frank asked his 9yr old to design the Zombies for the game, we did not have any human characters at this point and below is what he came up with which Frank cleaned-up to fit into the game.

chardesign

We also did a sample UI for the Interface, which Omid put into our first prototype of the game, it was based off the mock-up image below, we had things like an in-game economy, point collection and a timer with 3 lives.

verone

The initial concept also called for a mix and match of Zombie parts on several screens, we experimented on how these would fit together for the final experience:

prototype01

Game Genre:
A chance based game, much like a casino-style game, where there is a luck of the draw on what shows up, games of chance reward the user with money/virtual money or points in our case it was the latter. We used an analogue scoreboard to keep track of who was playing the leading.

An example of an analogue version of our game mechanic can be seen in this video:

rockpaper

Source: “Rock Paper Scissors Play Machine DIY from Cardboard.” YouTube, La Mamka Creative Club, 30 Oct. 2017, www.youtube.com/watch?v=vubTGCcW4D8.

Pacing:
The pacing of the game was real-time and turn-based, where different people in the group took turns and the rest would cheer them on.

Humans Vs Zombies the Experience:

gameplay

We landed on a mix of a role-playing game with a digital component which was the luck based draw of the cards, we used music, lighting and prop’s to enhance the experience and user engagement, the app on its own can be mundane and glitchy as taking a device can lead to unexpected results depending on processor speeds and devices specifications.

The main question we kept asking ourselves: IS THIS FUN!? and if it will keep people engaged!

We did not want to use the words ROCK, PAPER, SCISSORS on the cards. We went through different iterations of objects but we always came back to interpretation issues, as nothing was clear enough to be identified as a rock, paper or Scissors, we knew we wanted separate objects for Zombies and Humans so we decided to use illustrations and Symbols to represent the type of Card you had down, the result was a card with R,P or S on them, we found during gameplay that it still took time to identify a winner as our past experience with the game had to make sense of the current graphics and the presented outcome.

Here are some of the features of p5.js we used in the final digital prototype, we dare call it a full-fledged app as it could do with a lot of refinement to become a well-polished app.

– The p5 Play library to make animated sprites
– The shake and random function to give users the resulting card
– The sound library to trigger sounds on shake.

What we found out that on the iOs device the silence button disables browser sounds even if the phone is at full volume. This was a last minute discovery and was causing lots of frustration on many other projects which depended on sound for their implementation.

This was the final loop of the game:

1. Chose a Character -> 2. Click to start -> 3. Ready to shake -> 4. The result of Shake.

Step 2 was very confusing, as players started shaking the device after they clicked the character card and were expecting the result to show up.

 

Process Journal:

Day One (Game/User Experience Design):

The first day was used to write down the design of the experience and see in which parts could the tool be of help. After much talk, we realized that the best thing to do is to keep the design simple and use it mainly as a random tool generator. The process started with the creation of the initial frame. We were sure that we needed two groups for the game so, we decided to make two simple buttons so that the users could choose which team they are playing. By selection of the button, the user will be shown different tools based on their character.

choice01

Day Two (Game Functionality):

Our goal for our second day was to make sure the website works completely, and it is fully functional. We developed a random generator to choose a card at random for players. We used the deviceShake ability so that after the team has been chosen, the player can shake their mobile phone so that a card is chosen at random as is displayed on the screen.

itteration01

Day Three (Re-design of the User Experience):

After knowing that the design was fully functional, we focused on making the interface more like a game. We replaced the pictures with animations so that when the device is shaken, the animation is played. The animation shows a card rotating around and then revealing the card that was chosen at random. We added sound for each card and for each button as feedback to keep the experience more entertaining.

ezgif-com-video-to-gif

Day Four (Final Touches):

We changed the structure of the initial buttons to two cards, where now the users could select their team by choosing a card. This way, the experience seems much more like a game. In addition, we added a delay for the audio files so that no matter the phone that is being used, the audio file is played after the card has been revealed.

finalscreen

 

Coding Challenges:

  1. Difficulty dealing with animation

We faced two problems when we decided to work with animations. First was the speed of the animations. The speed was different on different devices based on the processing power and their frame rate. In order to overcome the problem, we added a delay to the audio file so that they would be played after the card has been revealed. The second problem was the size of the animation. We had no control over the size of the animation, the only control we found was to change the size of the pictures in the animation. We chose a dimension that would be perfect for most mobile devices.

  1. Audio file not playing on all devices

No all phones or browsers supported the audio file format. In order to fix this issue, we had to add different formats of the audio file so that most browsers would support them.


Some of the Feedback we got:

  • There should have been more of player choice as in the classic game of ROCK PAPER SCISSORS where you choose the hand you play.
  • The game could be used as is a party game for kids.
  • The build-up of the game environment was appreciated and well received.
  • If we could have more of group interaction as most players got to play it only once.

We have tried to respond to some of this feedback in the post above.


Gallery:

imag0572 imag0569 imag0568 imag0567

imag0566 imag0565 imag0564


Bibliography:

Overcoming IOS HTML5 Audio Limitations. Retrieved October 8, 2018,
from https://www.ibm.com/developerworks/library/wa-ioshtml5/index.html

Pedercini, P. P5.play. Retrieved October 8, 2018,
from http://molleindustria.github.io/p5.play/docs/index.html

Reyes, A. (2017, March 29). p5.Play Game Development – Part. 1 and 2: Setting Up. Retrieved October 8, 2018,
from https://www.youtube.com/watch?v=YJlovfYsp1Q

Shiffman, D. (2018, July 30). Coding Challenge #111: Animated Sprites. Retrieved October 8, 2018,
from https://www.youtube.com/watch?v=3noMeuufLZY&t=2s

blogend

.Game On!.

 

Use of this service is governed by the IT Acceptable Use and Web Technologies policies.
Privacy Notice: It is possible for your name, e-mail address, and/or student/staff/faculty UserID to be publicly revealed if you choose to use OCAD University Blogs.