# EmojiBall

The Emoji Ball

Creation & Computation : Experiment 2 (Multiperson)

by Carisa Antariksa, Nick Alexander, and Maria Yala

EmojiBall is a multiplayer physical ball game where the game pieces exert influence on the players. The components of the game are three EmojiBalls, three hoops, and an environment that affects the game’s outcome. The game is intended for a minimum of three players, and there is no necessary maximum number of players; however, each player must be assigned a hoop and an EmojiBall. The balls are reactive and have “moods” which will change based on how they are manipulated. Balls can be shaken, rolled, thrown, or otherwise handled in any way as the player decides. When a player scores with a ball in a “positive” mood they win a point, while scoring with a ball in a “negative” mood leads to an opponent losing points. The game is intended to be played in any space, regardless of size or surface. The distinct features of whatever space the game is played in will influence the available options players have with the balls, resulting in a unique game scenario every time.

The Project:

Process

We began by brainstorming by ideas; each team member gave ideas about what kinds of experiments would interest them, and then presented to the group why they were interested in the idea. We considered whether it would be a game or more of an art installation, and included ideas on what inputs and outputs the product would need.

We then proceeded to come up with various products stemming from the set of ideas.

After the brainstorming session, we were interested in two ideas; what wold become EmojiBall (a ball game that would let players manipulate the mood of a ball and score points when they made the ball happy) and a Wearable Glove that would allow someone to send signals and secret messages to another person.

We ended up choosing EmojiBalls as it was the idea we were most excited about; It combined elements from each of our initial ideas – mood (Maria), emojis (Carisa), and ball games (Nick). EmojiBalls would also allow us to explore a larger combination of inputs and outputs. Additionally, EmojiBalls would provide the most interaction for the players that wasn’t completely tied to the device as it incorporated a spatial component whereby the environment affected the balls’ state. We planned on using the light, sound, and motion sensors for inputs and the vibration sensors, LED lights, and speakers as outputs.

To further the collective concept we had another brainstorming session in which we went over the project and defined what would be needed to accomplish our task. During this session we also began to discuss possible game mechanics and form of the ball.

Our initial idea for EmojiBalls was this – Emojiball: a game where you must manage the mood of three balls in order to score points. The game encourages players to use tactics that usually don’t come into play in game spaces – such as manipulating the room’s brightness, shouting, and even cuddling the game pieces.

Once we had a general idea of what our project was we decided to break down tasks into chunks so that each of us would explore the various working of the project and teach that to the rest of the group members. We ended up deciding to use Light Sensor & Motion Sensor as our input and (Green, Yellow, Red) LEDs & Speaker as our outputs.

Lights & Points

We decided that the ball would have 3 main mood states for out minimum viable product. Happy, Calm and Angry. Each mood state would have a corresponding color – Happy (green LED), Calm (yellow LED) and Angry (red LED). Once we connected all the lights to the breadboard, we began by testing cycling through the three moods.

First, we tested the light sensor, setting up our breadboard so that when the light was bright ( a reading of 500 and above) the green LEDs would turn on and if the room was dim ( a sensor reading of under 500) then the red LEDs would go on indicating an angry ball whereas the green LEDs indicated a happy ball.

Once we got the light sensor functioning to switch the ball’s mood depending on the intensity of light, we wrote code that would alternate lighting each of the LEDs mimicking the idea that the ball’s mood would change over time. I.e. It would move from calm towards happy then back to calm and then down to angry in a looping fashion.

To affect the balls mood changing, we created a global variable, moodpoints , that would affect the ball’s mood. We decided that the ball’s moodpoints would max out at 1000 points and would be at least 0. Between 1000 moodpoints and 750 moodpoints, the ball would be happy. If the moodpoints were between 749 and 250, the ball would be in a calm state. And if the moodpoints were between 249 and 0, the ball would be in an angry state.

Input from the light sensor was then used to affect the moodpoints – If the player moves the ball towards light they score 10 moodpoints, if the player moves the ball to a darker environment they lose 10 moodpoints. A function – checkLight() – is called once per loop, where the light reading (dim or bright light) determines whether the player will score moodpoints or lose moodpoints. If light reading is under 500, 10 points are lost. If the light reading is over 500, 10 points are scored. Then depending on the value of the moodpoints a function is called i.e. getHappy(), getCalm(), or getAngry(). Each of these function will turn on a corresponding light and play an accompanying melody.

Orientation Sensor

After assembling the Adafruit BNO055 and downloading the libraries and driver, we installed and tested the sensorapi example supplied by Adafruit at their website. We experimented with the hardware and software to get a sense of what kind of readings it could supply and how they might be used for our purposes. We worked with the sensor for some time before being informed by our instructors that it required a pair of 4.7KΩ pullup resistors in order to be used safely – as it happened, we had been having trouble with our computers consistently detecting ports while the sensor was enabled, and installing the pullup resistors solved this issue.

Our instructors also supplied us with demo code for the sensor which we experimented with but eschewed at that time as we were already comfortable with the code supplied by Adafruit.

Our next hurdle was determining how best to put the raw data returned by the orientation sensor into use. The sensor can return multiple types of data but we looked at it primarily for data it could return as a gyroscope and accelerometer. We noted that the sensor could return temperature and considered exploring temperature as an input to our device and as a game mechanic, but ultimately shelved it as being out of scope at this time.

In considering the application of the sensory data we considered the intended “personality” of the ball. We wanted a ball that likes to be moved gently, and gets angry if it is kicked, hit or thrown. For our purposes measuring acceleration seemed like the better choice over measuring rate of rotation. Acceleration could serve as a catch-all for fast movement, especially if the ball was tossed or otherwise moved in a way that prevented rotation. We noted that measuring acceleration would only return good data over very short periods – if the ball was, for example, thrown a great distance, it would accelerate at the beginning of the throw and at the end, but the period of time in the middle of the throw would likely not have enough of a rate of change in velocity to trigger our code. However, considering that we were working on a short timetable and demonstrating on a small scale, most interactions with the ball involving its velocity were likely to be brief – thus returning beneficial data nearly the entire time the ball would be in motion.

We instituted a threshold within the code of + or -2 m/s^2 on any axis as triggering a change in moodpoints. The threshold itself is a variable for quick adjusting during testing. Using the if(millis()-lastRead>=sampleRate) loop suggested by Nick Puckett we set the ball to test its acceleration twice per second. It was our intention that this low sample rate would discourage players from simply shaking the ball in order to anger it and instead encourage them to throw or roll the ball. As discussed below, this did not pan out in testing.

Melodies

Aside from the LED output, we decided that using the 8 ohm speaker to convey different tunes for happy, calm and angry would be essential in enhancing the experience. Moods are often quickly associated with sound, and the strong connection between the two would play a large part in creating more effective game mechanics. It also would bring these items to “life” as the sound output shares an important indication of “feeling a certain way” as it is a personification to express feelings through spoken words, opinions or sounds.

To start with, identifying how to read the notes with the pitches.h library for the toneMelody code would help in creating custom melodies. There was also a difference in volume that we would need to note between the lower notes and the higher ones. The much lower notes would be much “quieter” than the higher ones. In fully utilising the potential in the default library, research was done by looking up different musical scales, from major to minor. We quickly realized that in this library, the major scale with sharps in the notes were used. Once the middle C in the major scale was found (NOTE_C4) playing around with the notes for happy and calm were easier to do.

Initially, we had a blue LED placed in the circuit for a “sad” mood and a sad tune was intended, but it was later scrapped. The choice to implement only happy, calm and angry was arbitrary and other moods could be implemented in future iterations of the game.

There were a lot of instances tested with the toneMelody examples, such as:

1. Altering the note durations between 2-16 and 1500-2000 overall to either speed up or prolong the note in the melody sequence,
2. Experimenting with the delay() within the tune itself by either placing in a value or calling a ‘pauseBetweenNotes’ action
3. Placing in a ‘0’ between the notes (e.g. In the calm tune, the sequence became “NOTE_C4, 0, NOTE_C4, NOTE_G4, 0, NOTE_G4, NOTE_A4, 0, NOTE_A4, NOTE_G4” to pause briefly between the melody to create a more musical tune.

These were then verified and uploaded in a simple button and speaker circuit, to repeatedly test the tunes and confirm that they are suitable in the happy, calm and angry outputs. These would activate as the color of the LED changes into the respective colors.

Testing different combinations of note sequences.

For each respective mood, we decided that we would go for the standard beeping alarm for the angry as opposed to a custom one, a moderate tune for the calm mood, and a kept a custom tune for the happy sound. With the high pitched alarm sound, there came a sense of urgency that can affect player motivation as opposed to what could be a more robotic, angry melody. The calm tune was a familiar sound that stopped just the right timing to anticipate the change in the mood that you are after. The happy tune was a custom sound that indicated the EmojiBall is communicating a “I’m Happy!” sound by using a higher pitch at the end of the sequence (This is well reflected in the main video above.)

Final note sequences used in the code.

These are then placed into the respective if statements that can be triggered once they have reached a certain threshold for happy and angry. By defining the “makeASound” function, each melody is called upon based on the mood changes. Some minor problems we encountered in embedding this code was making sure to define the total amount of notes in the sequence to make sure they all played within the for() equation. Within the “for (int thisNote = 0; thisNote < x; thisNote++)” the x would need to be replaced with the right amount of notes for the specific mood for the speaker to play, otherwise it would come out as a stifled sound. In the final code, the melodies synced well with the LED output, which allowed for a more “living” EmojiBall to be used in the game.

Initial Build

First complete circuit board. This one includes a voltage regulator (lower left) while subsequent builds did not.

The physical build of the balls was an exercise in trial and error. We felt it was important to solder our boards due to the physically demanding way they would necessarily be treated. None of us had soldered before and the first board, after being built and tested on a breadboard, took the better part of two days. After becoming familiar with the materials and apparatuses, making mistakes and undoing them, the first board was complete.

Early exploration of the final circuitry

Power management was an immediate issue. We explored several options for power before consulting with a peer, Omid Ettehadi, who gave us excellent advice and pointed us toward a simple 12V battery. He explained to us that the Arduino contains a built-in capacitor to keep regulate incoming voltage. He also suggested building a voltage regulator into our circuitry to regulate the voltage and take some strain off the chip – Omid warned us that it might overheat without an external regulator.

12V power source with voltage regulator

We built one prototype with the regulator and two without. While early testing showed no practical difference between the circuits that lacked a regulator and the other, it eventually became clear that the balls without regulators drained battery power at a much higher rate than the one that did.

Whatever we used as a case had to be tough enough to withstand rough play but supportive enough to keep our components safe. We considered hamster balls for housing early on but were dissuaded by their expense, lack of versatility, and advice from Reza Safaei, our fabrication guru. After discussing with Reza we settled on foam for our case, which could be easily shaped and hollowed to accommodate our components but prevent them from being jostled or moved out of place. An added advantage of foam was that it is light, therefore doesn’t affect the balls portability or the player’s ease of use.

As a final touch, we covered our balls in a soft felt material, which gave them a pleasant tactile feel and encouraged players to be gentle with them. This had the added benefit of interacting well with the wood floor of our demonstration surface, as the juxtaposition of hard vs soft objects in the play space and amplification of the balls’ sounds by the wood made for a pleasant experience that we did not explicitly design for.

The three EmojiBalls

Bill of materials per unit

User Testing

Our user testing suffered as a result of our inexperience with fabrication. Our team’s familiarity with code allowed us to build a game engine with parameters that could be easily tweaked, and we planned through testing to be able to adjust the parameters to get a sense of reactivity and personality of the balls. However we took longer than anticipated to complete a testable prototype and did not have enough time to test the product or the game to our satisfaction. Early iterative testing of the code using breadboards instead of soldered components pointed us in the right direction. We adjusted our thresholds and parameters to give the ball a sense of agency within the game and deliver immediate feedback to players. Moodpoint gain and loss from interaction was increased greatly from the values we had initially set for them, and the balls instantly felt more dynamic.

Testing the game itself was another matter. Our intention was to the keep the stated rules of the game simple and brief in order to allow interactions between players to flow naturally from the behavior of the balls rather than following arbitrary rules. We found that players did not intrinsically understand the consequences of interacting with the balls in certain ways and had to have the parameters of the mood-shifting mechanic explained to them. At that point, they tended to discover a single action – shaking the ball or holding the ball under light – that most reliably caused the mood-shifting outcome they wanted. We hoped that with more adjustment or the addition of extra sensors this might be mitigated, but we had prioritized fabrication over user testing and we were not able to tune the ball and the rules of the game to the extent we had hoped.

Product Design + Next Steps

The goal of the EmojiBall project was to create game pieces that felt as if they had emotions and exerted their will on the players of the game.

The actual rules of the game were secondary to the interactions with the ball. We considered adding rules such as “no touching the ball you last scored with” to prevent continuous scoring, but we decided against adding new rules in order to keep players’ minds clear and encourage interactions arising organically from play. We wanted to keep the rule set broad to ensure players not feel constrained and encourage them to try unusual things.

There are capabilities of the orientation sensor that we are not making use of that could be explored in further iterations. In addition to allowing us to tweak gameplay through the effects of physical interaction with the ball by measuring its current g-force or rate of rotation, the sensor is capable of measuring ambient temperature. Affecting the temperature of the ball or the game space might make for an interesting game mechanic.

Foam was chosen for its lightness, ease of use as a prototyping material and its protective qualities. Early in testing affixing the two foam cases together with dowels was adequate, but the rough nature of play as we drew closer to the due date necessitated us finding a more secure way to close the balls. We settled on tape in the interest of time. Layering felt material on the surface of the foam would also improve the mobility of these objects in the game. There was also a great advantage to having a felt ball to further emphasize the concept of a ball of emotion, where the idea of it becoming a sentient “feeling” object can be realized. The change in the tactile aspect could affect the interactions that can occur between the player and the object, which can bring about a sense of care for these balls that you are about to throw or roll into the hoop. With the code and core components now complete, another round of exploration and design is warranted to settle on the perfect material and closure system for the ball itself. Perhaps the ball can be designed using two spherical halves with grooves that fit into each other or even have one piece screw over the other, locking the components in place.

To refine our existing prototypes, we would add voltage converters to conserve battery life on the two prototypes that lack them. We would continue to explore options for casings and ball sizes – how might the gameplay change if the balls bounced like basketballs or were as heavy as bowling balls? How might the gameplay change if the ball texture was further altered?

We also see a likely expansion of the project as involving testing more sensors. This is not necessarily a priority, but experimenting with different kinds of inputs (e.g. a microphone to allow coaxing into different moods) might serve to refine the feeling of the ball as a discrete organism. Additionally, we would add RFID sensors to the goals and balls so that players are released from the burden of having to keep track of who scored where and by how much and who lost points where and by how much. This way, players can focus on enjoying the game and interacting with each other. The ball could be networked to cause it to react to stimuli outside the control of the players – for example, as suggested during the critique, it could change moods according to the weather.

Resources

Critique Presentation Slides

EmojiBall Code on GitHub