Project 3: Tweet Monitor- Created by Heather Simmons, Laura Stavro & Areen Salam

 

logo3

 

 

 

 

 

 

 

Inspiration:

artie at the vet copy copy copy2Slide02

 

 

 

 

 

 

 

 

 

Tweet Monitor:

Our concept for Tweet Monitor is a 2-way communication product that creates a meaningful connection between users across space and time.   The product enables users, via Twitter and SMS, to send and receive messages correlated to physical activities.  In the first application, tweets (via force sensor and microphone input) notify a dog owner every three minutes when the dog has jumped or barked.  A tweet from owner to dog can also be sent, activating a green light signal and a voice message to the dog. One of the key features of the Tweet Monitor is that you are able to stream a live video transmission through a website at all times from anywhere in the world.  Hourly summary tweets and a daily update graph allow owners to view summary information at their leisure.

For our project Tweet Monitor, we used Arthur the dog as our inspiration and created our final project for the Creation & Computation course.

User Requirements and Design Considerations:

  • Two way communications – dog to owner and owner to dog
  • Both visual and audible communications
  • Direct text message to owner to make monitoring easy
  • Frequent updates, every 5 minutes at least, in the early days
  • Hourly summary in case we eventually want to shut off the more frequent updates
  • Daily graph of activity
  • Durable – able to withstand dog jumping on it
  • Calibrated to the barks and weight of a very small dog

Project Demo:

 

Process:

Visualizer/Screensaver:

Following a few simple steps that we found on this blog we created a visualizer. We made a few changes according to our project and integrated it with the rest of the code.

The first step was to download and install Twitter4J in processing. The next step was installing the authentication with Twitter API by visiting the website https://dev.twitter.com/ and generating a consumer key, consumer secret, access token and token secret.

We thought it would be interesting to create a visualizer and embed that in our code to serve as a screen saver. We built an Array List to hold all of the words that we got from the imported tweets. The Twitter object gets built by something
called the TwitterFactory, which needs our configuration information that we set using the access tokens and keys generated.

Now that we had a Twitter object, we built a query to search via the Twitter API for a specific term or phrase. This code  will not always work – sometimes the Twitter API might be down,  our search might not return any results or we might not be connected to the internet. The Twitter object in twitter4j handles those types of conditions by throwing back an exception to us; we need to have a try/catch structure ready to deal with that if it happens.

After that, we made a query request and broke the tweets into words. We put each word into the words Arraylist.  In order to make the words appear to fade over time,  we drew a faint black rectangle continuously over the window. We then randomly set the size and color of the words to appear in random order in the black window.

Screen shot 2013-12-02 at 12.44.25 AM

Microphone

Figuring out the microphone was a bit of a cut and paste. Though we’d seen Kate use the microphone as a trigger we weren’t sure which library she used so we had a look at a number of different examples. The most useful of examples was on creativecoding.org. The code was in German, Stefan Fiedler’s example used a sonar-like visualizer, but gave us the basis to isolate the linein and volume codes.

We then added in the println portion to allow us to properly test and calibrate the volume. Arthur’s barks vary widely in volume, which gave us an opportunity to include a range of tweets triggered by the different levels of his voice. As the volume rises so does the urgent tone of the messages. Not sure this is an accurate translation but it’s fun to give a voice to a puppy!

Sending a Tweet from Dog to Owner

We used a force sensor connected to the Arduino and the mic input via the minim library and compared those values to thresholds set in the code.  The force sensor reading was sent over the serial port to Processing – this is in our serialEvent(Serial port) method.  If the noise/force exceeded the threshold, a tweet was sent using twitter4j’s updateStatus() method.    The code to send the tweet is contained in our postMsg() function.

Screen shot 2013-12-02 at 12.44.40 AM

 

Twitter to text/SMS

After the twitter feed connection was established using the twitter4j library we were good to use Ifttt to create the recipe:

Untitled-1

This made the project more useable in the real world. Though of course one could constantly monitor @artiethewestie’s twitter feed that’s not really conducive to leading a normal life.

Using the SMS feature makes it easy and streamlined – no checking Twitter, instead updates are routed to your cellphone and vibrate or jingle when the dog barks or pushes on the gate. This gives the owner an understanding of how often, how intense and how long their puppy is barking for.

IMG_5583

Two Way Communications – Sending Tweet from Owner to Dog

The function getLastTweet() allows the owner to send a Tweet with a coded keyword in it (in this case, the word Artiehome), which sets off a light on the dog’s gate and plays a sound file telling the dog that the owner is coming home soon.  (We modified some code from http://lucidtronix.com/tutorials/32 to get started).  getLastTweet() causes Twitter to search for all Tweets containing the word Artiehome.  If it finds one, and if that Tweet is not exactly the same as the one it previously pulled, it uses splitTokens() to grab the first word in that tweet.  If that first word is Artiehome,  it sends the code “1” through Processing to Arduino via the serial port.  Arduino reads that code from the serial port, and causes the light to flash.  Processing then plays the sound file (this function is playSoundClip().

Livefeed to Website

We wanted to create a site that would allow for visual monitoring of the puppy and the tweets as only one phone can be connected using ifttt. By using ustream.tv and a webcam, we were able to get a good angle of Arthur’s space. Ustream.tv is a great system and we were able to embed it into a wordpress site with the twitter feed in the sidebar. The web page gives a single visual representation of the previously invisible trigger and response action that allows for the puppy and its tweets to be monitored remotely by many users. It is accessed using wifi which makes it available at those times when there is no cell phone reception and only internet or wifi connection.

Screen shot 2013-12-02 at 12.59.13 AM

Hourly Update

Not all owners want a Twitter update every three minutes, so we created the function hourlyUpdate(), which, each hour, sends a summary of Arthur’s total barks and jumps in that hour to Twitter.

Graph

We then created a summary graph, tied to the keyPressed() function (up key to graph, down key to return to main), which shows Arthur’s total barks and jumps since the program was last run.

Screen shot 2013-12-02 at 12.45.17 AM

Fabrication:

To create a Tweet Monitor product, we used a long force sensor, which we pasted on a hard cardboard strip specified to the length and width of the sensor. In order to conceal the force sensor, we used canvas to wrap it around the force sensor and  cardboard. Once the force sensor was concealed properly we mounted it on the baby gate making sure it was at an appropriate height for Arthur the dog to jump and touch the force sensor on the gate.  The force senor was soldered with additional wires to extend it and then wired into Arduino. The Arduino along with the breadboard was safely placed in a canvas pouch behind the baby gate.

images

 System Diagram

Slide05

 

Twitter Monitor Code (Functionality):

  • Wireless – Bluetooth Smirf or Blue Mate Gold.
  • Uses Twitter4j library, video library, and minim audio library.
  • 2 modes – screensaver and readout.  Toggle between them by moving cursor into or out of upper right corner of screen.
  • Screensaver mode displays random words from Arthur’s tweets.
  • 4 inputs – force sensor in a gate, microphone in laptop, webcam, and Tweets using a keyword.
  • 4 outputs – audio, LEDs, streamed video, and Tweets.
  • Readout displays different messages depending on whether dog is barking, barking loudly, jumping on the gate, or both.
  • Once every two minutes (time threshold is adjustable) , sends a Tweet saying whether the dog is barking, jumping, or both.  Owner can retrieve Tweets from anywhere and also gets a text message on cellphone (IFTTT).
  • Once an hour, sends a summary Tweet saying how many times the dog has barked or jumped this hour.
  • Pressing the up arrow key reveals a graph of the number of times the dog has barked and jumped today.  Pressing the down arrow key returns to readout mode.
  • The dog owner can send a keyword in a Tweet, from anywhere and from any account, which will cause green LEDs on the gate to flash 3 times (from Arduino), and a sound file to play (from Processing) telling Arthur that his owner will be home soon.
  • In other applications, such as eldercare, the flashing green lights and audio file could be used to indicate “help is on the way.”  In a baby monitoring application, the lights could be nightlights turned on and off via Tweet by a parent, and the sound file could be a lullaby.

Key Coding Challenges:

Avoid Crashing Twitter:

  • Twitter limits the number of Tweets to 1,000 per day.  To avoid crashing Twitter while still continuously monitoring for barks or jumps, a timer was used.  If the time threshold since the last Tweet has passed when the dog jumps or barks, a Tweet is sent.  If not, the reading is passed to a “temporary” variable whose release is triggered when the threshold has passed.
  • Twitter also prevents users from posting duplicate messages.  All bark or jump messages include a time stamp to prevent duplicates from being recorded.  The keycode for inbound tweets used to set off the LED and sound file is the word “Artiehome.”  The owner, when tweeting to the dog, tweets the word Artiehome, then space, then anything.  The reason for “space, then anything” is again to avoid a duplicate message.  The splitTokens() function is then used to parse out the first word in the inbound tweet, Artiehome, which is then compared to the required key specified in the program (Artiehome), to ensure a match.
  • Twitter also limits the number of queries an application can make per 15 minute window.  The code which compares keywords in inbound Tweets to the key in the application is therefore also nested in the timer.

The key to comparing strings:

Use “equals()” function, not ==, because == may return a  false value when in fact the strings are equal.  This is because == sometimes compares the location of String objects in memory, not the words themselves.

if (homeKey.equals(twitterKey)) {

playSoundClip(“artiesound.wav”);

twitterKey=””;

}

Fritzing:

Breadboard View:                                                    Schematic View:

tweeting dog (1)_bb        tweeting dog (1)_schem

Next Steps:

  • Build a data set over time for training purposes.  We could send daily data to a text file (similar to our classmates who completed the “Hush” project) and from there display daily graphs of Arthur’s barks and jumps, perhaps with hourly breakdowns within the day to identify “problem times” for the dog.
  • Consider integration of other sensors for different applications (motion sensors to detect falls in seniors, for example).
  • Consider building a durable, mobile rubber robot to increase interaction between owner and dog, similar to the one built by the Pawly team.  This robot could be controlled via Twitter “commands”.

Similar Products:

i) Angel Care

Top_baby_digital_video_movement__sound_monitor_AC1001

 

video2  Under-the-Mattress Movement Sensor Pad

video3  Color video Transmission

v4   LCD Touch Screen

v5  2-Way Talk-Back Feature

v6  Audio “Tic” Feature

v7  Adjustable Camera Angle

 

ii) GrandCare System

new-homebase-with-sensors-jpg

 

Motion Sensors

Daily and weekly graphing – Detect patterns of motion

If there is No motion in kitchen from 8am to noon, email daughter.

If Any motion is detected at the foot of bed at night, turn on the bathroom light.

If there is Excessive motion in bathroom for more than 15 minutes between 10pm and 6am, contact Emergency Call List.

Door Sensors

Daily graphing – Used for doors, windows, cabinets, drawers, refrigerators

If door is Opened from 10pm to 6am, call neighbor.

If the pill cabinet is Not Opened between 8-10am, send a reminder call to the Resident.

If door is Not Opened between 10-11am call Supervisor (the Caregiver has not arrived).

Bed and Chair Sensors

Nightly graphing – Sleeping patterns or when a loved one leaves bed at night

If nobody is In Bed for more than 45 minutes between 10pm and 6am, text night nurse
(Resident got up and didn’t return to bed).

————————————————————————————————————————————————-

Project Code:

In order to run the following Processing sketch, please add this photo to the sketch and name it “arthur.jpg”

arthur

In order to run the Processing sketch, please download and add the sound file “artiesound.wav” to your sketch from the website link attached below:

Github Arduino Code: https://github.com/heathersimmons/Creation2/blob/master/tweetDogArduino

Project Demo (C&C Group)

Vimeo: https://vimeo.com/groups/209883/videos/80624483

References:

Visualizer/screensaver ref: http://saglamdeniz.com/blog/?p=124

Two-way communications – keycoded Tweets from owner to dog http://lucidtronix.com/tutorials/32

Microphone Code:

Steffen Fiedler http://www.creativecoding.org/lesson/topics/audio/sound-in-processing

Minim Manual – http://code.compartmental.net/tools/minim/manual-minim/

 

 

Comments are closed.