Crossing obstacles while walking is a daily challenge for the visually impaired. Often depending on their walking tick alone, obstacles are only sensed when too close to the person therefore risking accidents. The Sense Stick is an affordable haptic sensor that detects obstacles from a wide range and changes the vibrating rhythm to notify the wearer of the approaching proximity of any obstacle. While the sensor itself is placed on the base of the wearer’s walking stick, the feedback mechanism is placed on a small ring located on the middle or index finger of the wearer.
The reason for the feedback mechanism being placed on the finger isthat there is a significantly wide range of vibrations that need to be sensed and the thin sensitive skin around the finger (right under the nail) allows this. The wearer should wear the ring on the hand that is not holding the stick to maximize its efficiency. However, if they choose to hold things or partake in any activity using this hand, the device will not interfere with either activity due to its compact size and specific placement.
– Arduino Micro
– Ultrasound sensor
– Vibration motor
– Jumper wires
– USB cable
– Adafruit Haptic Driver DRV 2650
Figure 1: Adafruit DRV2650L
Figure 2: Vibration motor
Figure 3: Arduino Micro
In class experiments:
We conducted three experiments in class, that helped us understand haptic mechanisms and functions. We began with basic analog experiments with the vibration motor and moved on to trying variations with the Adafruit Haptic Driver. For each of the experiments we took the following into consideration:
– Intensity of vibration
– Frequency of vibration
– Rhythym (Incase of multiple motors)
– Is the motor placed on the body or the body on the motor?
– Position on the body (How sensitive is the area of skin you are using for the motor, why have you placed it there, is it under apparel or above apparel, what are the ergonomics of the motor on the skin.
Figure 4: Clint Zeagler’s map for motion impedance in males and females.
Using this diagram to understand sensitivity levels for different parts of the human body was helpful. I experimented with motors on the calves, palms wrist and fingers. Since the fingers were the most sensitive to low pulses I decided to focus on them as the area to experiment and place my prototype.
The initial experiments conducted without the haptic driver had varying results. We used the Blink and Fade defaults available in Arduino as the base code for our experiments. The Fade set gave us a lot of versatility and allowed us to play with speeds, intensity and rhythm. Some of the findings from the fade set are listed below:
At default full range, the vibrations from the motor are too harsh and slightly painful on the finger. However, the same when placed on the cheek feels pleasant and therapeutic. On the ears as well it is painful and disturbing. When placed on the wrist, its intensity is felt much lesser.
The more we increased the delay, the more rigorous the vibrations became, and the longer they lasted. For eg., at delay(5000) the intensity was too high for the fingers. I began playing with reducing the intensity to understand how it changed.
At delay(200) the vibrations were short and felt more like a pulse than an actual rigorous vibration. These subtle motions could be clearly felt on the fingers.
At delay(50) the vibrations were soothing and felt like a subtle heartbeat. This reminded me of stress busters and other calming techniques.
The two delays mentioned above were changed for both highs and lows, the next 2 experiments I conducted were with simultaneously contrasting high’s and lows in the delays.
delay(50) High and delay(1000) Low gave me a soft subtle long duration ambient pulse, which was also an interesting motion to sense.
Then I swapped the two to delay(50) Low and delay(1000) and this motion was too rigorous for too long.
After conducting individual experiments, we moved into study groups and conducted experiments together with multiple vibration motors. For this experiment, we focused on playing with the “fade” function in Arduino. We played with different fade intensities between 5-55. We noticed a change in intervals and intensity of pulsation during this process. The maximum range of vibration is reduced in this process as well, and thereby makes the movement much softer and bearable on sensitive areas of the body such as ears, lips, cheeks and fingertips.
Haptic Driver Experiment:
We also tested the Adafruit DRV2650L Haptic Driver. The driver comes with a corresponding library that needs to be loaded into Arduino prior to use. The default code ‘basic’ available for the driver under examples runs each vibrating effect in sequence with a corresponding number that shows up on the serial monitor. Since the driver comes with over 100 vibration rhythms, once I set up the basic breakout (Circuit diagram below) I focused on understanding the differences in the rhythm, by going through the library a few times. I picked a set of rhythms that I thought made the most sense when used with this particular device.
Figure 5: Basic circuit diagram to test the vibration motor with a haptic driver.
Considerations for ergonomics of the concept:
– Weather Conditions: The sensor placed on the walking stick should be between 15-25 cm above ground level to minimize intervention from snow during the winter.
– Location of the vibration motor: The vibration motor should be placed on the area of the body that will not be deterred by apparel or jewellery. But at the same time it has to be sensitive enough to sense a single pulse. Therefore the area that I picked was on the fingers, under the nails. It will be worn as a ring (as shown in the sketches above).
Once my experiments were complete I shifted focus to the prototype
The circuit for the prototype followed the same output pattern as the circuit for the haptic driver experiment. For the input, I added an ultrasound sensor and connected it as per usual practices.
Figure 6: Final circuit fritz for the prototype.
For the coding process, I used base code from the Creation and Computation course for the ultrasonic sensor input (created by Kate Hartman and Nick Puckett). Thereafter I used if and nested if statements to add the vibration effects from the driver into the input code. I spent most of my time adjusting the distance threshold values and finding the right series of vibrations that were not too harsh, but not too soft. For the vibrations, they increase in intensity as the obstacle gets closer, and when inactive the motor emits a slight pulse to allow the user to know that it is powered and working. As an obstacle gets too close the motor stops vibrating to indicate that it is time to step over or stop or ensure that the obstacle is out of the way.
The workspace can be very stressful for most, and often we need messages of security and reassurance to keep us going. These inspirational messages usually form part of our ambient environment in the form of postcards, or cut outs. But I believe that the experience of motivating oneself, is more intimate and private and it can be awkward to have strangers at your desk space analysing and staring at your personal messages, often asking questions that can make the user uncomfortable. The Invisible motivator aims to resolve this issue, with an ambient system that only displays messages to its user on the user’s demand at the push of a button, else camouflages with their environment. It blends into other artefacts that the user collects and displays in their personal desk space.
Figure 1: Completed prototype
Figure 2: Closeup of invisible message patch
Figure3: In class worksheet
We learnt how to work with multiple forms of thermocromic pigments in class, which was an insightful process. We learnt how to make dye, paint and screen print with the medium. This helped us with using the pigment in a versatile manner. I found that these pigments are extremely useful when dealing with temperature sensitive projects, especially wearables. They can also be used alongside other devices like heat pads, as a feedback indicator.
After class I, conducted research on different projects with thermocromic inks to better understand how the medium works and what it can do.
After class I proceeded to test some of the patches we had made with two different stimuli, the first was a hairdryer, and the second was our e-textile tester with 3 batteries of 3v each, which is 9v of total power. I found that the material reacts to the extreme heat of the hairdryer much more instanteously and effectively as compared to that of the e-textile controller.
Figure 4 and 5: Testing patches at home with the e-textile tester
For my prototype I used the following materials
– Thermocromic ink in blue
– Acrylic paints both white and blue
– A paintbrush
– A patch of cloth
– Resistive thread
– Basic sowing supplies
– Arduino Micro
– Push Button
– Alligator clips
– E-textile tester with 3 batteries of 3V or battery pack
I began my fabrication process by mixing the blue thermorcromic ink with white acrylic paint, and making a mixture of blue and white acrylic paint in the same colour. Once both the mixtures were ready I painted the purely acrylic mixture onto a patch of cloth and let it dry. This part of the prototype is the non-reactive one.
Thereafter I took the mixture with the thermocromic ink and painted a motivational word onto the patch that I had painted previously, and tried my best to ensure that it camouflages into the base. I painted 2 coats of the paint to make sure that it is reactive enough.
Once this mixture was dry, I ran a running stitch of resistive thread across the word. I ensured that it was equally distributed and aesthetically pleasing.
Figure 6: Mixing and matching paints (both thermocromic and regular)
Figure 7, 8, 9: Painting the base patch, painting the thermocromic message, stitching the resistive thread onto the patch.
I built a circuit with a push button, since I only wanted the message to activate on the user’s demand (or auto display it at fixed intervals of time in the next version of this prototype). For the circuit I took inspiration from a project on instructables.com by uzarate (link in citations)
Prior to building the circuit with the painted patch, I ran a test at home with a regular patch to see if it works.
Figure 10: Cirtuit
Figure 11: Testing the circuit at home with a regular patch
(Circuit reference by uzarate on instructables.com)
The only difference between this circuit and mine is that I used the e-textile tester as the second power source with 3 batteries of 3v each within it.
For the coding process, I used a simple I/O code for Arduino. Screenshot below.
While the circuit did work, and the e-textile controller lit up, I feel like the power supple was insufficient, as the ink reacted perfectly to heat from hands and a hair dryer but took sufficiently longer and was slower with this circuit. I am to fix this in the next version of this prototype.
Figure 13: Cold patch
Figure 14: Activated patch
This patch can be placed inside a simple photo frame , even be developed further into a camouflaging patch on their desk wall to minimise notciability for non-users. wWith a push button hidden under the user’s desk at a convenient distance, I envision the user using this prototype as a stress buster during a hectic work day.
Stress, anxiety and anger are all emotions or states that most of us deal with in today’s day and age. All of them, are also contagious and easily spread amongst people. It’s best to attack and deal with these emotions. It’s best to nip these emotions in the bud using technology to facilitate the process.
The idea for our low fidelity prototype is to create a device that can calm its wearer during the moments that they feel like everything is the worst. Using biofeedback for self-reflection or somaesthetics, the device senses the heart rate of the wearer and provides them with instant motivational feedback in stressful situations (as opposed to just raw data). The purpose of this device is to induce instant calmness, promote self-efficacy, help keep the user motivated and empower them to tackle stressful situations that they otherwise may avoid in their day to day lives.
Subtleness: The vibration is soothing and gradual, pleasant and relaxing.
Intimate correspondence: The vibration frequency and motivational interface react to the heartbeat rate captured by the pulse sensor.
Making space: The device is extremely light, easily portable and can give the user a few moments of personal headspace even in the most crowded gatherings.
Articulation: The rhythmic vibration, along with the encouraging interface helps a user relax and calm down. Research shows that positive quotes can promote self-efficacy and it is our hope to encourage that through this prototype.
Fig 1: Low fidelity prototype placed in a glove, shown here with the wearer and without.
In Class reflection
Something that really stuck with us during the class was the discussion on the different approaches to biofeedback. When dealing with Biofeedback as a form for self-reflection, you need to consider the way the device provides you with more insight and awareness about your own body. Something that was challenging in our project was to find the ideal method of communicating with the user without publicly sharing their data.
Our research process led us to discover that there are several products in the market that detect stress, very few of them actually help the wearer actionably deal with the stress. Most wearable devices currently also detect heart rate but rarely give the wearer any feedback/ insight or call to action on the data gathered. This tends to make the wearable experience dull since the wearer cannot act upon the data and improve or amend things as they might the desire to do.
Fig 2: From left to right: Our vision for the prototype was inspired by HeartMath sensor, Stress chips, and a vibrating watch(Doppel)
Idea and worksheet
Having worked out the sheet in class, we had a clear picture of the prototype idea. We wanted to create a device that regulates stress, anxiety and other forms of negative emotions that trigger elevated heartbeat. It comprises of minute sensors on the wearer’s fingertips, and a stylish wrist band to enclose the microcontroller seamlessly.
Fig 3: Idea sketch for high fidelity prototype
Fig 4: In class worksheet
The device consist of a pulse sensor and an output component in the form of vibration. It also contains an interface output on a screen that corresponds with the user’s heart rate.
1 x Arduino Micro
1 x USB cable
1 x 2mm Mini Vibrating Disk Motor
1 x Pulse Sensor
1 x Glove
Ancillary material to sow sensors, and a computer to process the code
As shown in the ideation section, we turned the glove inside out and sowed the heart rate sensor onto the pinky finger and the vibration motor onto the ring finger. Thereafter we connected them using cables to the Arduino micro, which we fastened to the wearer”s wrist using the glove itself.
Fig 5: Inside out glove few for fabrication of components. They were sewn onto the glove for this prototype.
To get a better understanding of the sensor and its capabilities, we started experimenting with the examples from the PulseSensor Playground. After going through all the examples, We Chose the Getting_BPM_to_Monitor example as the base for our code and made our modifications to that file. We initially tried using the Adafruit ESP32 Feather for the project, so that we could connect the glove using the internet to the user’s mobile phone. Due to the limitations of this board in dealing with Analogwrite, we were not able to run any of the examples on the Feather and therefore decided to stick with the Arduino micro for this stage of the project and use the Serial Control to communicate with the computer.
In the Getting_BPM_to_Monitor example, the device already calculates the beat per minute of the user and prints on the Serial monitor, in addition to turning onboard LED one every time it detected a heartbeat. We added another pin to control the vibrating disk motor. We set the code, such that if the heart rate of the person went higher than the threshold set by the user, the vibrator would turn on. We experimented with leaving the vibrator on for the duration of the time the heart rate of the user was higher than the preset value, but after some testing, we saw that the constant vibration leads to an increase in the heart rate of the user. We made some changes to the program so that if the user’s BPM passed the threshold level, the device would vibrate at a 50 per minute rate to mimic a slow heart rate and using the body’s natural reaction to the frequency to lower the user’s heart rate.
The next step was to connect the Arduino to P5. We used the serial control, so that every 200 ms, the Arduino would send the status of the LED (indicating heartbeat) and the BPM to the P5. We wanted to use this information and further assist the user in their attempt to lower their heart rate. We decided to stick with something simple. We used a drawing of a heart to indicate the user’s heartbeat and to print out positive thoughts when the user’s BPM passed their preset threshold. This was to make sure the data is perfectly transmitted between the devices. By finding the right conditions, we can now set the P5 to do any actions we want in order to lower the user’s heart rate, such as play relaxing music or etc.
Fig 6: Circuit
Fig 7a and b: 7a shows the calm state, and 7b shows the stressful state with calming motivational messages.
Fig 8: WIP CODE with preview > Arduino and p5.
As a team, we were fairly clear about a realistic scope for the timeframe we were given. Our priority was to get the sensor working to the vibration and only thereafter did we incorporate the interface functionality.
The vision for the final product
If we were to develop a higher fidelity prototype, we envision gaining more accuracy with data for thresholds of stress, anger, anxiety or any other negative emotion that is accompanied by a change in heartbeat patterns.
We would also focus on improving the ergonomics of the sensors, and making them more wearer friendly and unintrusive. The vibrations on the current prototype are fairly simple, however, we would love to customize the vibration to the emotion in a rhythmic way, so as so as to maximize its effectiveness in tackling the stressful situation. We would even consider personalizing this motion to its user depending on how they react since every user is unique and might respond differently to different emotions.
It would also benefit us to help track and save the data that we are able to gather for the wearer to determine their emotional patterns and help them tackle their issues in a sustainable long-term manner.
Running, as a form of exercise can help reduce anxiety and stress, but running in urban areas can often be a source of stress as well. Loud honking cars, pollution, careless pedestrians, and unexpected weather conditions can weigh a runner down. The Relaxed Runner is a set of wearable devices aimed at helping runners address some of the stressful situations they encounter on the go. A controller, worn on the runner’s fingers is connected to a scarf around the runner’s neck and triggers specific features. The scarf contains a speaker, to help cut noise when needed, LED lights to help runners signal pedestrians and traffic (especially when running in the dark), and the occasional mist spray for some hydration. For this workshop, the focus was on building a prototype for the controller.
The Relaxed Runners controller is worn as a single device with two controlling functions within it.
Controller 1: Knitted finger socks
Controller 2: Woven band
Both controllers function as push buttons, that activate upon circuit completion, i.e. a combination of the power and ground lines. In the case of the finger socks, the mechanism is activated when the user joins both his fingers together. For the woven band, a user has to pinch two ends of the band together, with their other hand to activate the circuit. Both mechanisms work easily and are focused at users who are on the move with minimum interference.
A thin scarf work on the neck is aimed at giving the runner a sense of security and not obstructing their movement. The functionality on the scarf works with the LED Lights and the mist spray. These are the most critical triggers for the controllers. The speaker can be activated before the run depending on the wearer’s decision to wear headphones or not.
The idea for this controller was the outcome of an in-class exercise. We were asked to individually write on word cards, names of different types of clothing, verbs, and adverbs. After that, we had to combine these cards in groups of four and pick up random combinations in ballots. Then we had to sketch eight ideas each based on these combinations, and pick our favourite. Some of the combinations I got were softly-reading-necklace and exciting-gymanstics-headband. I chose a DESTRESSING – JOGGING – SCARF, and that formed the base of my idea.
Given that building a high fidelity prototype for a scarf is unachievable in a short period is unrealistic, I opted to build the controller instead. The fact that this concept is targeted towards runners, any wearable device has to be light, easy to wear and extremely comfortable. It cannot interfere with the runner’s movement. With this in mind, I moved into execution.
As a first timer, I began the knitting process with a few sample patches. This experiment helped me get comfortable with the technique, the size of needles and the nature of knitting I wanted to execute. Usually, beginners are recommended thick needles because it makes the process easier to learn, but due to the tight timeline and nature of my prototype (it had to look good and fit on a finger), I consciously opted for thin needles. This decision worked in my favour because it gave me the right size of knit stitch, and the correct tension I needed for it to fit perfectly on any finger size.
NOTE: Knitting requires a stitch count. I used a twelve-stitch count to fit the height of the more extended finger (middle finger) and an eight-stitch count for the thumb.
I began the knitting process just with yarn and added the conductive thread halfway through. I wanted only a particular patch of the finger sock to be conductive, so the runner doesn’t accidentally trigger anything while moving. I just knotted the conductive thread onto the yarn and kept knitting. Once the thread is added, it might feel rougher and make the knitting process slightly harder, but continue as planned.
CAUTION; The conductive thread can tangle easily, and knots cannot be undone. Avoid cutting it off the roll you are working from and also avoid taking too much out at once.
Once the conductive patch is complete, add another knot, cut off the edge and continue working with the yarn. Once completed, proceed with a bind off and trim off all extra thread.
Once the patch is ready, hold the two edges together to form a cylinder, and insert a sponge in the middle to help maintain the shape of the sock. Use a regular needle and thread (preferably in a similar colour) to sew the two edges of the patch together to form the sock. Trim off all extra thread. Complete both fingers.
Once both the finger socks are ready, proceed to weave the band that joins the two together and also works as a controller. Start by making six same size incisions along the vertical edges of the cardboard patch. After that tie a knot at the beginning of your yarn and align it vertically along the incisions. The front should have them laid down vertically, and the back should have them laid out horizontally (pictures show how). Conclude this part with another knot at the end of the yarn and trim off any extras.
Next, cut a long piece of yarn and tie it to your needle. Begin moving perpendicularly to the vertical threads in an alternate pattern across threads and rows (pictures show how). Leave approx 1 inch on each end before you start the horizontal weave. Once a row ends move back in the opposite direction, and continue to do so till the end of the patch. Use the fork to tighten each row while weaving. Linearly make multiple knots along the ends and trim off any extras.
Once the weave is complete proceed to sew on the conductive thread onto it. Sew two same size patches closer to the ends of the patch, that can make contact with ease. You can choose any pattern you like for this.
Once the three components are ready, sow the edges of the patch to the bottom end of the finger socks ensuring that the conductive areas on the socks align on the inside of the hand, and the patch outside (See picture).
To test the finger socks, connect the edges of conductive areas to the edges of your e-textile tester with alligator clips. Joining both fingers should trigger the LED.
To test the band, connect the edges of the conductive patches to the edges of your e-textile tester with alligator clips. Joining both patches should trigger the LED.
NOTE: Ensure that your battery is in the right poles prior to testing. This often causes the test to fail.
Before this workshop, I had never knitted or woven in my life. This was a fascinating new world to venture into, and I’m so glad that I had the opportunity to learn both techniques. While felting also seemed like a good skill to acquire, because weaving and knitting both were more complex an technically demanding I wanted to use this opportunity to learn both. While the weaving process is agnostic to the hand of its user, knitting was an extremely challenging start for me as a left-hander. With limited resources available online it took me a few days to grasp the technique accurately, and that was the most challenging aspect of this project for me. Designing the controllers was not so much of an issue as much as mastering the technique with high-quality output. I almost gave up after day two, but I knew that this was the only opportunity I would have to pick up the skill, so I pushed myself, and I am pleased with the outcome. Next steps :
I would improve the quality of the sewing on the weave and perhaps make the band longer, so there’s more space to play. I would also like to try creating a more complex controller with all five fingers, using each finger to trigger a different kind of reaction.
I would create a wrist band for my Arduino and attach it to the finger controllers, to improve the functionality of the prototype.