Touch Music – Work from Home Edition

Project Description

Since the beginning of the COVID-19 Pandemic the new norm became working from home which led to monotony during the lockdown period. Despite the slight semblance of normalcy of being able to go to work or school in-person we have slowly adopted the work from home notion as it has been popularised over the past two years. Touch Music – Work from Home Edition is the extension of a previous project I did in the Creation and Computation studio class back in 2020. The project explored capacitive sensing using and Arduino Nano 33 IoT connected to a capacitive sensor which was attached to conductive material wrapped around a wine bottle. Once connected it was activated by touch and would display a visualizer that reacted to the beat of a song.

touch-music-2touch-music

In this version of Touch Music – Work from Home Edition, I further explore the concept by designing a wearable sweater that is worn while someone is working from home. The wearable sweater has two capacitive sensors one on the left cuff of the sweater and the other on the back of the sweater and the idea is to have the sensor play different types of music when someone is studying or working and when they are taking a break. The capacitive sensor on the cuff is activated when someone is placing their hand on their computer while they are typing and plays focus music. Whereas the capacitive sensor on the back is activated when the user is taking a break and leans back completely on their chair and the music genre changes to the users’ preferences. The wearable is intended for use by people who work or study from home and enjoy playing music while they work.

Project Context

The Bose SoundWear Companion Wearable Speaker (Bose n.d.) is one of the inspirations of the Touch Music project. The wearable speaker is designed to be worn on the neck and offers the best of both worlds’ portability and great sound output. The speaker is well designed and sits comfortably on the neck and is great for people who work from home. The wearable has two speakers that are upward-facing and point toward the ears with good sound quality which is an alternative solution for hands free use such as answering calls while driving. While the wearable is not exactly designed for use in public spaces as it is a speaker and can get loud it has other potential uses which include use at home or in the backyard, taking a walk, speaker calls, virtual reality, and driving. Touch Music is designed for use at home specifically for people who work or study from home and spend most of the day using their computers. Like the Bose speaker the wearable sweater’s microcontroller is designed to sit on the shoulders of the user as it is unobtrusive and not distracting to the wearer.

Liquid MIDI is an experimental modular textile interface for sonic interactions, exploring aesthetics and morphology on contemporary interaction design (‘Liquid MIDI’ n.d.). It uses experimental textiles and conductive ink for the sonic interaction and is a tangible interface. The piece consists of textiles, screen printed with conductive paint. The paint creates a network of intersecting lines with pronounced circles at specific junctions. The lines are connected to an Arduino through alligator cables which helps it to communicate with a desired software, using midi protocol. Trigger pads and fader board are screen printed modules focused on AV performances, allowing the performer to build its set up regarding its needs. Sound is a medium that has been increasingly gaining ground in the visual arts during recent decades, despite this seeming contradictory (‘Liquid MIDI’ n.d.). Liquid MIDI relates to Touch Music which uses the concept of fusing conductive material on a fabric in this case a sweater to trigger a sound output when touched. The unique interaction of Liquid MIDI is foldable and morphable which allows for interesting uses and interactions where the interface becomes a part of the process of creation itself.

liquid-midi_002

Photo Credit: Liquid MIDI

Woojer Vest is one of the most powerful haptic experiences it delivers high fidelity tactile sensation which reproduce the rich emotion of sound(‘Vest’ n.d.). The Vest Edge gives 360 degrees of immersion, delivering powerful and accurate, detailed sensations. The Woojer Vest works by pumping low frequencies of sound into the body. It is best experienced when playing games, watching movies, or listening to music. Thus, it would give you a one-of-a-kind audio experience that cannot be experienced with just headphones. It makes the experience of watching movies and playing games, especially with VR, much better (Sutton 2021). The vest is designed to be worn and plugged into a gaming console, headphones, or any other device that you want to use with the vest. It’s the perfect companion for at-home gaming, movies, VR and music. It’ll pump the low frequencies through your body, delivering a unique and mesmerizing audio experience. The idea of Touch Music is to introduce a different experience of working from home by adding an interesting interaction to clothing that allows the wearer to control music using the sense of touch. As it can be monotonous to see the same four walls of someone’s workspace the idea of Touch Music offers a way to liven up the work from home experience with the potential of exploration into other interesting interactions.

Parts List

Electronics

  1. (1) Arduino Nano 33 IoT – ARDNN-032333 (Arduino)
  2. (1) MPR121 12 Key Capacitive Touch Sensor – PROTS-001982 (Adafruit)
  3. (2) 6″ (M-F) Jumper Wire – CONJU-062319
  4. (2) Ribbon Cable 22AWG 40 way – 1 Meter
  5. (2) Mini Breadboard Red – PCBBA-120442
  6. (1) USB (A) to Micro (B) Cable – 5ft – ZCABL-010215
  7. (4) 22AWG Hookup Wire – WIREJ-000140

Materials

  1. Copper Tape – COPER-010567
  2. Sweater
  3. Velcro
  4. Thread
  5. Aluminium Foil

Circuit Diagram on Fritzing

touch-music-circuit-diagram

Link To Code

Code on GitHub

Wearability Assessment

The wearable sweater was comfortable to wear for its intended purpose while seated it was comfortable and unobtrusive. While moving around or walking the microcontroller felt slightly heavy but was not very noticeable, the wires sewn through the sweater were not felt and stayed in place even while moving around. Based on Gemperle’s argument “The weight  of  a  wearable  should  not  hinder  the  body’s movement or balance (1998).” The construction of the wearable sweater is not bulky or heavy however it would be beneficial to use more compact electronic components in the next iteration.

Final Photos

img_3423img_3424img_3426img_3436img_3425img_3431

Construction Photos

img_3413img_3420 img_3414 img_3415img_3419 img_3416img_3417img_3418img_3455 img_3458img_3457img_3456

Video

Link to Demo Video – https://ocadu.techsmithrelay.com/b1LE?tab=Details

Link to Process Video – https://ocadu.techsmithrelay.com/apDM?tab=Details

Challenges and Successes

The wearable sweater was a fun project to work on some of the successful aspects was the code worked well and the construction of the wearable was easy to assemble. It was comfortable to wear and unobtrusive as I sometimes forgot I was wearing the device. The progress made from the first Touch Music project which focused more on physical computing let to further exploration of wearable technology in this iteration.

Some of the challenges faced during this project was time constraint trying to balance thesis work. As the wearable sweater incorporated capacitive touch it sometimes lagged to respond or would cut of the music midway and reconnect after some time. The capacitive sensors are hidden from the line of sight of the wearer which could potentially be jarring if the user forgets, and music automatically starts playing when the sensor is activated.

Next Steps

The next step of the project would be to design a more compact wearable that can be worn and removed by the wearer as they please. Presently the design of the wearable is designed to be fixed on the sweater which is not feasible if it is intended to be worn over an extended period as not everyone enjoys listening to music as they work. The possibility of exploring different form factors and ideas of incorporating the device into a wearable has the potential to be further developed and looking at different concept ideas beyond playing music while working from home.

 

Bibliography

Bose. n.d. ‘SoundWear Companion Wearable Speaker – Bose Product Support’. Accessed 2 May 2022. https://www.bose.ca/en_ca/support/products/bose_wearables_support/soundwear-companion.html.

Gemperle, F., C. Kasabach, J. Stivoric, Malcolm Bauer, and R. Martin. 1998. ‘Design for Wearability’. In , 116–22. https://doi.org/10.1109/ISWC.1998.729537.

‘Liquid MIDI’. n.d. E J T E C H. Accessed 2 May 2022. http://ejtech.cc/?page_id=790.

Sutton, Robert. 2021. ‘Woojer Vest – Everything You Need To Know’. Teckers® Tech (blog). 31 August 2021. https://teckers.com/woojer-vest/.

‘Vest’. n.d. Woojer. Accessed 2 May 2022. https://www.woojer.com/pages/vest.

Getting Stronger – By Trish

2237fe1b-5bdd-4fca-9795-1e89d7c57b40

Project Description

Getting stronger is a project inspired by my healing journey post ankle surgery, one of the most important processes on the road to recovery is physiotherapy. Physiotherapy recommendations is usually 2-3 times a week depending on the type of surgery. Through out my journey I have been interested in tracking my own progress hence this project explores wearable technology as a way to keep track of different physiotherapy exercises.

Getting stronger takes a look at two specific exercises I have been doing for the past year now one is ankle AROM – Inversion and Eversion. Inversion entails moving the ankle so that the foot faces towards the body while eversion involves moving the ankle so that the foot faces away from the body. This exercise is done to improve range of motion of the ankle from left to right position. The second exercise is toe towel curls which is performed using a towel or other material to scrunch. This exercise works well standing up or sitting down. This exercise is a good foot strengthening workout.

Getting stronger uses the accelerometer on the circuit playground express to keep track of the ankle AROM – Inversion and Eversion stretch to monitor the progress of range of motion on the ankle. For the toe towel curls a pressure sensor is attached to the circuit playground express to keep track of the foot strength progression.

Final Photos

img_3265 img_3264 img_3263img_3247 img_3248 img_3249

img_3246 img_3243 img_3242img_3237 img_3244 img_3245

 

Parts List

Electronics

  1. (1) Circuit Playground Express
  2. (1) Lipo Battery
  3. (3) Alligator Clips
  4. (1) 220ohm resistor

Materials

  1. Elastic strap
  2. Velcro
  3. 3D printed case for the CPX
  4. Felt
  5. Velostat
  6. Conductive Fabric

Circuit Diagram On Fritzing

getting-stronger

Link to code

Code on GitHub

Project Context

Getting stronger is inspired by another project I did for the Body Centric Technologies Studio Class in winter 2021 called Vibrating Knee Brace. The project documented my physiotherapy journey for my runner’s knee at the time which would cause pain and discomfort when I would run. For the project I designed a wearable knee brace that had four vibrating motors two at the top and two at the bottom connected to a CPX. The brace had four modes that were activated using the one of the on-board buttons on the CPX. When the button is pressed it would shift through the modes as follows:

  1. The CPX Neo Pixels lights up red to indicate the motors are off
  2. The CPX Neo Pixels lights up yellow to indicate the top motors are on
  3. The CPX Neo Pixels lights up orange to indicate the bottom motors are on
  4. The CPX Neo Pixels lights up green to indicate all the motors are on

The wearable was designed to be worn while someone is either running, jogging, or walking to relieve pain felt on the knee while doing these activities. The project borrowed its idea from the TENS machine which is used for physiotherapy. Getting stronger relates to this project as it also borrows ideas from existing physiotherapy exercises to keep track of progress throughout the healing process.

Sensoria® Smart Socks are a smart textile wearable designed to improve running form by keeping track of speed, pace, cadence, and foot landing. It helps a user learn how to run to avoid injury prone running styles. Sensoria smart socks are infused with comfortable, textile pressure sensors (Sensoria Fitness). They offer real-time feedback when someone striking with the heel or the ball of your foot. They help monitor foot-landing technique and the data is visualized on the Sensoria Fitness mobile app (Sensoria Fitness). The idea of getting stronger links to Sensoria as it incorporates pressure sensing with an output of the Neo Pixels on the CPX to indicate the pressure exerted during the toe towel curls exercise.

Orpyx SI® Sensory Insoles is a wearable designed to help prevent foot complications (‘Orpyx Medical Technologies Inc.’). The wearable devices offer pressure monitoring for preventing foot complications and provides physiological data that can guide patient care. It also helps to gain an understanding of a user’s activity for remote patient monitoring services and the Orpyx SI® Flex Sensory Insole System is designed to help reduce the risk of plantar foot complications. As my project looks at monitoring patient progress throughout the physiotherapy process the Orpyx Insole offers inspiration as it also uses pressure sensing for patient monitoring.

Bibliography

‘Circuit Playground’s Motion Sensor’. Adafruit Learning System, https://learn.adafruit.com/circuit-playgrounds-motion-sensor/twinkle. Accessed 29 Apr. 2022.

‘Orpyx Medical Technologies Inc.’ Orpyx Medical Technologies, https://www.orpyx.com. Accessed 2 May 2022.

Sensoria Fitness. https://www.sensoriafitness.com/smartsocks/. Accessed 29 Apr. 2022.

Social Body Lab. How to Make an E-Textile Analog Sensor. 2020. YouTube, https://www.youtube.com/watch?v=tA37mGEnPes.

 

Wait!

wechatimg710

Project title: Wait!

Project Description: 

This project targets on facilitating walking activity for users with vision impairment by incorporating TENS (transcutaneous electrical nerve stimulation) device as a form of haptics. People with blindsight—the ability to detect things in the environment without being aware of seeing them, are faced with the obstacles of perform daily activities without bumping into erect objects or surfaces. They need to use a stick to track obstacles or require guidance by another person when walking around buildings.

Wait! is a piece of wearable that addressed the demand of users with vision issues by providing audio and haptic feedback to alert users from being too close to impediments. It consisted of an ultrasonic sensor to detect real-time distance from the user to the object/surface in front of. When user has walked up too close towards the object/surface, the ultrasonic sensor will trigger the switch in the relay, generating the sound of a click and send low-voltage electrical pulses via TENS device to human body. Only when user stepped back, the electrical pulses will be put on halt. Henceforth, user are better supported and assisted in navigating indoor or outdoor spaces.

By incorporating an established TENS device, users are able to adjust the strength and frequency of pulse signals according to their own preference and level of comfortableness. The value could be set beforehand and saved for future uses with customization.

Project Context:  

EMS(Electrical Muscle Stimulation) and TENS has been widely employed in the field of medicine. They can be considered as supplements to conventional muscle training, particularly for therapeutic treatment and physical rehabilitation. TENS is more specifically used in pain treatment that it stimulates the nerves exclusively by delivering electrical signals that do not trigger muscle movement. Applying TENS signals to painful spots at the body can reduce discomfort and relieve pain (Gibson et al., 2019). 

Despite an emphasis on the field of medicine, there are emerging studies of using EMS as a form of haptics in the creative industry. Studies have been conducted around the cross-section of wearable devices, mixed reality, human-computer interaction and experience design. EMS has been used as a haptic input and output technology in wearable and textile-based computing through crafting comfortable textile electrodes (Pfeiffer & Rohs, 2017). EMS has also been incorporated into haptic interfaces to simulate the force feedback effect caused by a collision, in generating a mixed reality tennis game (Farbiz et al., 2007). With all the applications of EMS in non-traditional settings, TENS however, is yet to discovered and applied in other disciplines from an innovative perspective.

One case study that informed this project is Using Electrical Muscle Stimulation Haptics for VR. While traditional approaches in VR focus on lightweight objects via skin receptors such as vibro-tactile gloves, simulating heavy objects is still confronted with limitations by traditional methods of physical props or hand tethering (Burdea, 2000). This study explores how to better render heavy objects in VR via EMS in the form of wearables. Its main concept is to prevent the user’s hands from penetrating virtual objects by means of EMS. Tension is created in the user’s bicep, tricep, pectoralis, and shoulder muscles. The system stimulates up to four muscle groups to generate the desired tension, thereby constructing the desired realistic experience of touching the wall or lifting up heavy objects. 

The system is encapsulated into a small backpack, which could be worn by the user. The backpack contains a medical compliant 8-channel muscle stimulator, which is controlled from within the VR simulators via USB. Other components include a typical VR system consisting of a head-worn display (Samsung/Oculus GearVR) and a motion capture system (eight OptiTrack 17W cameras). 

This project informs the design of Wait! in the way that in a VR environment, users are less capable of sensing the surrounding while moving around – similar to be under the vision-impaired circumstance. Using EMS to generate muscle tension and henceforth simulate the weight of object is intriguing and very informative in terms of possible applications of muscle stimulation.

Other related works include vibration-based tactile haptics such as CyberTouch by Virtual Technologies (Burdea, 2000), pneumatic gloves using air pockets (Tarvainen & Yu, 2015), tethers and exoskeletons for fingers or upper body muscles. Passive haptics are also oftentimes used in VR to simulate the existence of objects such as still props and props placed by human or robot. Even though tactile haptic could potentially support a better texture rendering, it does not deliver a directional force that acts upon the user’s hands or muscle groups. Pneumatic gloves are also confronted with the similar challenge of representing heavyweight objects. Hence the utilization of EMS/TENS to activate muscles is of great value and importance to be further investigated.

Another study that I have looked into is Remote Controlled Human project which uses Spark Core, a TENS unit, and a relay to remotely control a human minion over WiFi. The Spark Core connects the TENS unit and user it attached to with the Internet of Things. With the TENS unit stimulating involuntary muscle movements, the relay acts as a bridge between the signal provider and the receiver. This project is a great reference for circuit construction, specifically how to bridge a commercialized healthcare product with computer softwares via a simple relay. It also extends the future possibility for hooking up sensors and writing one’s own sketches via the Spark IDE to generate outputs based on all kinds of conditions, such as rhythmic music input, light levels and et cetera.

Parts List:

  • 1 Arduino UNO
  • 1 TENS unit (TENS 3000)
  • 1 Ultrasonic Sensor HC-SR04
  • 1 relay unit
  • 1 Felt hat
  • Breakout cables
  • 3 Male-to-Male jumper cables
  • 2 Male headers
  • Wire cutters/strippers
  • Sewing kit
  • Small slotted screwdriver

Circuit Diagram:

circuit-diagram

Github Code | Link to Video

Wearability Assessment/Priorities:

  1. Unobtrusive placement. (The hat works at the appropriate area of human body without impeding dynamic body movements.)
  2. Design for human perception of size. (This wearable is designed to minimize thickness and weight as much as possible. The sensor is within the range of normal hat weight to be unobtrusive and transparent.)
  3. Containment. (This piece of wearable contains materials of all electronic components, wires, electrodes, etc. While some of these things are malleable in form, there are many with fixed volume that one needs to consider how the ‘insides’ bring to the outer form.)

Final Photos:

wechatimg714 wechatimg711 wechatimg712

Challenges & Successes:

Figuring out a way to incorporate TENS signal that is non-harmful yet effective on human bodies was one of the biggest challenges during the early stage of this project. As electrical muscle stimulation has a safe range of usage and a varied level of comfortableness upon users across different demographics, utilizing TENS needs thorough preliminary preparation and physiological backup. Out of the safety concern, I decided to proceed with a relatively safe and minimal side effect TENS device which has been clinically tested and commercially employed. 

Reference to Remote Controlled Human project, I was able to hack into TENS 3000 unit and control its digital switch using a relay and Arduino UNO. The analog control of TENS unit would be much more complicated with an established device. 

The construction of this project was also one big challenge in terms of fitting the whole circuit into the interior of a hat while maintaining enough space for user’s head to fit in. I mapped the circuit onto the inner surface and tried to design an optimal route for wires to connect to each component without taking extra space. The Ultrasonic sensor was placed in front of the hat with the other part of sensor hidden in the hat brim by poking out two holes. The wires were attached underneath with another cut to hide the connection. However, it still leaves visible wires connected to battery or electrodes that users might find complicated and troublesome when attaching TENS pads onto the body.

Next Steps:

Furthering this project, I would like to experiment with controlling TENS/EMS signals in an analog way rather than digital. The wearable could be elevated if it can sense body signal as inputs and adjust the output automatically with the assist of Arduino programming. However, figuring the way of connection and making sure the safety range of pulse stimulation are going to be the two biggest challenges.

In order to achieve that, it might require the abandonment of established TENS units but to figure out a way to manually set frequency/strength of electrical stimulations. It adds to the level of insecurity and danger as it is not monitored under a controlled environment.

Incorporating EMS into other creative disciplines as an innovative medium would be interesting to be explored, such as with the cross-section in music, performance art, writing, education and et cetera.

Bibliography:

Farbiz, F., Yu, Z., Manders, C., & Ahmad, W. (2007). An electrical muscle stimulation haptic feedback for mixed reality tennis game. SIGGRAPH ’07.

Follmer, S., Leithinger, D., Olwal, A., Cheng, N., & Ishii, H. (2012). Jamming user interfaces. Proceedings Of The 25Th Annual ACM Symposium On User Interface Software And Technology – UIST ’12. doi: 10.1145/2380116.2380181

Gibson, W., Wand, B., Meads, C., Catley, M., & O’Connell, N. (2019). Transcutaneous electrical nerve stimulation (TENS) for chronic pain – an overview of Cochrane Reviews. Cochrane Database Of Systematic Reviews. doi: 10.1002/14651858.cd011890.pub3

Janczyk, M., Skirde, S., Weigelt, M., and Kunde, W. (2009). Visual and tactile action effects determine bimanual coordination performance. Hum. Mov. Sci. 28, 437–449. doi: 10.1016/j.humov.2009.02.006

Jones, S., Man, W., Gao, W., Higginson, I., Wilcock, A., & Maddocks, M. (2016). Neuromuscular electrical stimulation for muscle weakness in adults with advanced disease. Cochrane Database Of Systematic Reviews, 2016(10). doi: 10.1002/14651858.cd009419.pub3 

Lee, J., Kim, Y., & Jung, H. (2020). Electrically Elicited Force Response Characteristics of Forearm Extensor Muscles for Electrical Muscle Stimulation-Based Haptic Rendering. Sensors (Basel, Switzerland), 20(19), 5669. https://doi.org/10.3390/s20195669

Lopes, P., You, S., Cheng, L.-P., Marwecki, S. & Baudisch, P. (2017). Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation.. In G. Mark, S. R. Fussell, C. Lampe, m. c. schraefel, J. P. Hourcade, C. Appert & D. Wigdor (eds.), CHI (p./pp. 1471-1482), : ACM. ISBN: 978-1-4503-4655-9

Pavani, F., Spence, C., and Driver, J. (1999). Visual capture of touch (tactile ventriloquism); out-of-the-body experiences with rubber gloves. J. Cogn. Neurosci. 11:14.

Pfeiffer, M., Schneegass, S., & Alt, F. (2013). Supporting interaction in public space with electrical muscle stimulation. Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication.

Robertson, A. (2021). Meta’s sci-fi haptic glove prototype lets you feel VR objects using air pockets. Retrieved 21 April 2022, from https://www.theverge.com/2021/11/16/22782860/meta-facebook-reality-labs-soft-robotics-haptic-glove-prototype

HEART RATE SENSORS – By Trish

Description

Heart rate is the measure of Beats Per Minute (BPM) which is the number of heartbeats detected in one minute. Normal resting heart rate range from 60 to 100 beats per minute (Laskowski). Resting heart rate is described as the heart pumping the lowest amount of blood needed when one is not exerting a lot of energy. Heart rate data can be collected from various parts of the body some of which include the wrist, fingertip, chest, thigh etc. Heart rate is measured using a monitoring device that allows one to get heart rate data in real time.

The two common medical applications that measure heart rate are optical based heart rate sensors known as photoplethysmography (PPG) sensors and electrical heart rate sensors known as electrocardiography (ECG) sensors. Optical heart rate sensor “uses a light-based technology to sense the rate of blood flow as controlled by the heart’s pumping action”(NeuroSky). While an electrical heart rate sensor “measures the bio-potential generated by electrical signals that control the expansion and contraction of heart chambers”(NeuroSky).

 

PPG Sensors

PPG is an optical heart rate sensor that is often used for heart rate monitoring purposes (Castaneda et al.). It is often non-invasive and uses a light source and a photodetector at the surface of skin to measure the rate of blood flow as controlled by the heart’s pumping action (Valencell). Optical blood flow sensors were first innovated in the late 1800s this was done by having someone place their hand over a candle in a dark to view blood flow and vascular structure(Valencell). Since then, the evolution of PPG sensors have gotten smaller and more accurate with modern technology.

There are two types of PPG sensors one is the pulse sensor that uses a green LED which emits light and hits surface of the skin, and a photodetector to measure incident light. The pulse sensor is designed to measure pulse waves when the heart pumps blood. The application of pulse sensors is widely used in wearable devices for health and fitness tracking. The second type of PPG sensor is the pulse oximeter that uses a red LED, and an infrared LED, both measured by a single, shared photodetector used to measure blood oxygen levels. In hospitals pulse oximeters are commonly used to measure pulse rate and blood oxygen.

ECG Sensors

ECG is an electrical heart rate sensor that measures and records the electrical activity that passes through the heart. “With each heartbeat, an electrical wave travels through the heart. This wave causes the muscle to squeeze and pump blood from the heart (Heart and Stroke Foundation).” The first practical ECG machine was invented in 1903 by Willem Einthoven who was a Dutch Doctor (Barold). EGC machines work using lead wires connected to electrode sensor pads placed on specific parts of the chest, arms, and legs. The electrical activity of the heart is then measured, interpreted, and printed out either on paper or digitally on a screen(Electrocardiogram).

There are 3 main types of ECG monitoring which include:

  • Resting ECG – conducted while someone is lying down in a comfortable position.
  • Stress or exercise ECG – conducted while someone is using an exercise bike or treadmill.
  • Ambulatory ECG (sometimes called a Holter monitor) – the electrodes are connected to a small portable machine worn at someone’s waist so their heart can be monitored at home for 1 or more days.

The type of ECG monitor depends on symptoms or heart problem suspected (‘Electrocardiogram (ECG)’).

Comparison of PPG vs ECG

The comparison between PPG and ECG sensors include:

PPG ECG
Uses electrical signal derived from light reflected due to changes in blood flow during heart activity Uses electrical signal produced by heart activity
Can measure heart rate but only suitable for average measurements Measures heart rate accurately
Uses electrical signal derived from light reflected due to changes in blood flow during heart activity Uses electrical signal produced by heart activity
Requires a longer settling time due to the need to measure ambient light Meaningful readings can be obtained within a brief time

 

Wearable Devices with Heart Rate Sensors

Consumer wearables such as smartwatches and fitness trackers commonly have PPG sensors as they are cheaper than ECG sensors.         In this research I compare two wearable devices that use PPG, ECG and a variation of both sensors.

Xiaomi Mi Band 6

The Mi Band 6 is the latest series of the Xiaomi smart band, it has two main sensors one is a high precision 6-axis sensor (3-axis accelerometer and 3-axis gyroscope) and the other a PPG heart rate sensor. It is used for both health and fitness tracking and the features include:

  • Heart rate monitoring: Whole-day heart rate manual heart rate, resting heart rate and heart rate curve.
  • Sleep monitoring: Deep sleep, light sleep, rapid eye movement (REM), naps.
  • Women’s health tracking: Provides recording and reminders for the menstrual cycle and ovulation phases.
  • Stress monitoring, breathing exercises, PAI vitality index assessment, idle alerts, step counter, goal setting (Mi Smart Band).

Apple Watch Series 7

The Apple Watch Series 7 has both PPG and ECG sensors which measures blood oxygen level and allows a user to take an ECG anytime, anywhere respectively. Some of the features include heart rate sensing, mindfulness, and sleep tracking for health monitoring. The blood oxygen sensor and app allow someone to take on-demand readings of their blood oxygen as well as background readings, day and night (‘Apple Watch Series 7’).

The ECG sensor works using the ECG app, Apple Watch Series 7 can generate an ECG similar to a single-lead electrocardiogram. Electrodes built into the Digital Crown and the back crystal work together with the ECG app to read someone’s heart electrical signals. It works by placing a fingertip on the Digital Crown to generate an ECG waveform in just 30 seconds. “The ECG app can indicate whether a heart rhythm shows signs of atrial fibrillation — a serious form of irregular heart rhythm — or sinus rhythm, which means your heart is beating in a normal pattern(‘Apple Watch Series 7’).”

 

Conclusion

Overall, both PPG and ECG have their advantages and disadvantages and have become an integral part of designing health and fitness wearable devices. Each offer their own benefits to the type of device depending on factors such as placement on the body, type of heart rate data being collected, fitness tracking, health monitoring and so on.

 

Bibliography

‘Apple Watch Series 7’. Apple (CA), https://www.apple.com/ca/apple-watch-series-7/. Accessed 27 Apr. 2022.

Barold, S. Serge. ‘Willem Einthoven and the Birth of Clinical Electrocardiography a Hundred Years Ago’. Cardiac Electrophysiology Review, vol. 7, no. 1, Jan. 2003, pp. 99–104. PubMed, https://doi.org/10.1023/a:1023667812925.

Castaneda, Denisse, et al. ‘A Review on Wearable Photoplethysmography Sensors and Their Potential Future Applications in Health Care’. International Journal of Biosensors & Bioelectronics, vol. 4, no. 4, 2018, pp. 195–202. PubMed Central, https://doi.org/10.15406/ijbsbe.2018.04.00125.

Electrocardiogram. 8 Aug. 2021, https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/electrocardiogram.

‘Electrocardiogram (ECG)’. Nhs.Uk, 18 Oct. 2017, https://www.nhs.uk/conditions/electrocardiogram/.

Heart and Stroke Foundation. Electrocardiogram | Heart and Stroke Foundation. https://www.heartandstroke.ca/heart-disease/tests/electrocardiogram. Accessed 30 Mar. 2022.

Laskowski, Edward. ‘2 Easy, Accurate Ways to Measure Your Heart Rate’. Mayo Clinic, https://www.mayoclinic.org/healthy-lifestyle/fitness/expert-answers/heart-rate/faq-20057979. Accessed 31 Mar. 2022.

Mi Smart Band. Mi Smart Band 6 – No.1 Wearable Band Brand in the World – Xiaomi Global Official. https://www.mi.com/global/product/mi-smart-band-6/. Accessed 27 Apr. 2022.

NeuroSky. ECG vs PPG for Heart Rate Monitoring: Which Is Best? 28 Jan. 2015, http://neurosky.com/2015/01/ecg-vs-ppg-for-heart-rate-monitoring-which-is-best/.

Valencell. ‘Optical Heart Rate Monitoring Technology: What You Need to Know’. Valencell, 15 Oct. 2015, https://valencell.com/blog/optical-heart-rate-monitoring-what-you-need-to-know/.

 

 

 

 

Shoulder Bird

Project Description
“Shoulder bird” is a wearable accessory I made for artistic purposes. It is supposed to imitate a bird on your shoulder, much like the pirate trope. The main purpose of the wearable is for cosplay and for fun. The project features two interactable components: A pressure sensor on the beak of the plush bird and an accelerometer on the circuit playground. The project was made with car proximity sensor in mind and how the beeps have a shorter interval and higher tone when they are too close to another object. The pressure sensor alters the tone of the speaker while the accelerometer’s Y-axis readings alters the interval between each beep. The X-axis readings alter the delay between each loop. I made the project with car sensors in mind, specifically when it beeps when reversing and how the beeps display urgency and danger through the tone and interval. I also chose the icon of a bird because I feel that the beeping of the speakers are reminiscent of a bird’s song/ tweet.
20220427_011322screenshot_20220427-021643_galleryscreenshot_20220427-021614_gallery
(The bird goes on your shoulder, it id fastened there by a safety pin)
Parts List:
-Minky fabric (Black, Grey, and White for plush)
-Black Spandes (For Base)
-Circuit Playground Classic
-Conductive Fabric
-Conductive Thread
-1k Resistor
-Velostat
-Foam
-Polyester Beads

Videos:
Video 1 (fastened on the Shoulder): https://drive.google.com/file/d/1XctzX1guXIVyzLaTN0zDIRTNDSqj4Cl-/view?usp=sharing
Video 2 (Unattached): https://drive.google.com/file/d/1XM_RO2UDMMxHkLhXONC3iyTdgvLGU85l/view?usp=sharing

Circuit Diagram
wearablesassignment2_bb

Code
Here is the link to the code on github

Project Context
https://rippleskip-art.tumblr.com/post/675544751937830912/guys-its-making-me-so-happy-to-see-people-in-the
This pattern of an axolotl plush is my largest inspiration for the project as I wanted to use a similar pattern to make a plush. I incorporated the plush before anything else into the project.
https://youtu.be/SCfUaYKF5DI
Though this project isn’t a wearable project it inspired me to got for a more whimsical/ fantastical direction. THis helped me decide to make something just for the sake of art and fun.

Process: Plush
I took reference from the pattern but adapted it a bit for my own purposes. The plush was hand sewn and full of plastic beads to make it floppier. I feel that this design choice worked out well as it makes it perfect to be situated on the shoulder.
screenshot_20220427-023603_gallery screenshot_20220427-023626_gallery screenshot_20220427-023538_gallery

Process: Electronics
The circuit features a voltage divider with a pressure sensor attached to it.The whole circuit isn’t placed directly on the plush, instead  it is sewn onto a Black Spandex “housing”. The rest of the electronics are housed in the circuit playground.
screenshot_20220427-023949_gallery

Stretch Controller

Project Description
“Stretch Controller” is a wearable controller that interfaces with Unity. It takes the form of a glove and has two stretch sensors that work as the inputs for the project. The inputs are used alongside other modifiers in the Unity Engine. In this way the users can interact with the project by flexing and relaxing their fingers. I chose to use a stretch sensor for the inputs because I thought it would be interesting to take a simple concept and play around with it to make something interesting and functional. Additionally, I chose to create a wearable controller because I like creating video games and feel that creating my own version of a controller would allow new and interesting design conventions and affordances that are different with an ordinary controller. While I do enjoy the tactility of ordinary controllers and the physical feedback they provide with buttons, I feel that a wearable controller can offer more immersion when playing a game. This project serves more of as exploration of alternative controllers rather than a replacement for ordinary ones.
screenshot_20220426-191744_gallery20220426_19544520220426_19545420220426_195057

Interactions/ Controls
The main interactions of the piece is the stretch sensors. They are placed along the pinky and index fingers. This is so that when the fingers bend/ ball up the sensor would be stretched and change the input value. The value is then run through a C# script on Unity so that it is remapped to be more suitable for interactions. The stretch sensor controls the rotation of a cube. The screen also displays the maximum and minimum values (important for calibration),  the mapped value, as well as the raw value. (The full range of the stretch sensor is 0-1024, it doesn’t go through the full range and must be calibrated to be more suitable. The cube rotation also only has a range of 360.)
The other interaction is holding down the on-board button (pin 4). While the button is held down, the minimum and maximum values will be recalibrated.

Videos
Video1(Shows the glove and hand flexing): https://drive.google.com/file/d/1WSHZCvftgVV4atS4N2ZegyDyHGLOKVX0/view?usp=sharing

Video2(Shows the computer inputs in the computer screen):https://drive.google.com/file/d/1v6HMORIKVE0vf1LTWKLBslA4rtk5ZEKm/view?usp=sharing

Materials:
-Adafruit Circuitplayground Classic
-Conductive Rubber (Stretch Sensor) 350ohm per inch in relax (From Creatron)
-Black Spandex (From Kings Textiles)
-1k Ohm Resistor * 2
-Conductive Thread

Fritzing Schematics

untitled-sketch_2

Code
Here is the link to the code on github
The repository features 4 files: 2 Unity files and 2 Arduino Files.
-The Unity files respectively remap the input for the pinky and index. They also use the remapped data to change the x (pinky) and y (index) rotation values of a cube.
-The Arduino file titled “Final_Test” was a test I used at the beginning of the process to make sure all the analog inputs are correct
-The Arduino file titled “Uduino_Final” is the example code from the uduino library so that the Arduino (Circuit Playground Classic in my case) and Unity can interface with each other. The code is only altered in the beginning to allow the circuit playground to be read.
#include <Adafruit_CircuitPlayground.h> is the only addition to this. The rest of the code is by  Marc Teyssier.

Project Inspiration
The project was inspired by the Nintendo Power Glove and its predecessors, the VPL DataGlove, and Z-Glove made by Thomas Zimmerman.

The Z-Glove inspired me because of the Optical flex Sensor. I really like how it uses optical flex sensors which uses a basic physics concept that light travels in a straight line to create a sensor to detect how much the finger bends. Similarly, I wanted to use a simple concept of stretching and flexing for my own glove-based controller.

Process: Electronics
The electronics were rather difficult to construct and wire up. Most of the issues comes from the conductive thread. Because of how stiff it is it tended to come undone quite often. From there was often the risk of the thread coming into contact with everything the rest of the circuit. This was especially a problem at the end of the circuit, when connecting everything to the circuit playground. To fix this, I put the loose ends down and placed a small piece of conductive fabric on top of it. This is because the fabric has a heat activated adhesive on it.
screenshot_20220426-191858_gallery
Besides that, the circuit features two voltage dividers with the stretch sensor assigned on each of them.
screenshot_20220426-191925_gallery
Process Video: https://drive.google.com/file/d/1WaJUJ78CmuAfbBZdRBp_SrrjEqhqX_it/view?usp=sharing

 Challenges and Successes
Overall, I really liked how tidy the wiring is for the project. I’m also really glad that it does work and that the fabric isn’t pulled b the conductive rubber. I also learnt from testing that adding another strip of the conductive rubber would average out the values a little bit. I’m really glad that I was able to interface the circuit playground with unity. This knowledge will serve me very well in the future.
There were a lot of challenges for the project. Though I wanted it to be modular with 3d printed pieces, I could not get them printed in time so I sewed the whole thing onto the glove. The circuit playground also tended to overheat so I’m a bit wary about keeping it plugged in for too long. The playground also seems to be able to read the analog inputs one at a time, so that will be a difficult challenge to overcome in the future. Another challenge was that everything came off a lot, as mentioned earlier with the wires often coming undone.

Next Steps:
There is a whole lot I want to improve on with the project, I probably want to use a different material/ make a bigger glove as the spandex is very tight, while this is good in detecting the stretch, it makes taking the glove off rather hard, especially when taken into account how fragile it is and how the components can easily come off it. I also really want to focus on making it modular. Most of all I really want to make a better game to interface with the glove. This works well as a demo but there are so man other thing I want to make with it.

 

Web Futures

img_6326

 

img_6273

 

My art style, especially in the context of technology, is heavily influenced by juxtaposing seemingly opposite realm: physical-digital, technologic-organic. By uniting these realms, technology departs from being a mere utility or novel, into a dynamic and stimulating extension dimension to our own. 

Web Future is a conceptual project that further explores this artistic interpretation in a variety of techniques. The project features a glove that connects via bluetooth to a TouchDesigner animation, endeavouring to bridge digital and physical realities with a seamless and dynamic interaction. As the user moves their hand, an animation will likewise respond to these motions, serving an experience akin to physically reaching into the digital world.

Additionally, the physical makeup of the glove is an exploration into demonstrating technology through a seemingly opposing medium. The glove was crocheted very loosely from raw cotton twine, delicately covering the hand through a very organic weblike structure. The BLE board is plainly displayed atop and is beautifully striking against the organic cotton, its wires and battery connection nicely hidden in the user’s sleeve. Technology is often considered to be something highly engineered, so I found it particularly interesting and even futuristic to move beyond that and display technology in such a way that appears to be falling apart.

img_6330 img_6312

 

Wearability:

The wearability of my final project drastically diverged from the wearable criteria I outlined in my project proposal. Initially, I imagined an installation piece available for all sorts of people to try, and therefor required a simple and easily adjustable glove. However when it actually came to creating the glove I felt more driven to create an artistic piece that explored my interest in smart textiles.

Materials:

  • 1x Arduino Nano 33 BLE Sense
  • 1x Arduino Nano 33 BLE
  • 1x D Battery
  • 1x D Battery Clip
  • Conductive Thread
  • Conductive Yarn
  • Raw Cotten Twine
  • Regular Thread

Circuit Diagram:

screen-shot-2022-04-25-at-8-31-50-pm

Code:

https://github.com/kahaniploessl/web-future

Final Video:

https://youtu.be/wqDIlZn0i2g

Challenges/Successes:

Overall this project was a success and I was able to create what I sought out to do. Incorporating the bluetooth was the biggest challenge and I’m so pleased it worked because the project would not have felt as cool without it.

Next Steps:

I’m very excited for what comes next in this project, which entails adding finger switched by crocheting conductive yarn. I think introducing this element will really elevate the project as it will serve an excellent bridge between the circuit board and the crocheted glove, excellently displaying the idea of “loose/deconstructed technology”. While it’s unfortunate I was not able to include that this semester it’s something I’d like to work on right away and so I should have an update relatively soon. 

Fairy Dust – Angelina Do & Valeria Suing

Project Title: “Fairy Dust”

Project Description

“Fairy Dust” is a wearable electronic device exhibited in the form of a vest mounted with moving wings controlled by the movement of the body. It also has an installation/interactive piece where the movement triggers on-screen animated fairy dust. “Fairy Dust” was an exploration by use of wearable technology as a platform for self-expression through fashion or style. This wearable was created for those who dare to dream and imagine!

Project Context

Parts, Materials, Technical Assets

  • Circuit Playground Express 
  • Micro USB Cable 
  • Micro Servo Motors (with 3D printed servo 90 degree)
  • Needle and Thread 
  • Fabric (Flannel, Cotton, Tule) 
  • Fairy Wings 
  • Decorative Elements (Flower Trim, Rose Embellishments etc.) 
  • Alligator Clips 
  • TouchDesigner 
  • Power Source (Powerbank used)

diagram_servomotors

Link To Code

Link To MakeCode

Wearability Assessment

“Fairy Dust” was designed to create an engaging experience. It’s meant to be presented as an interactive art piece that would impress, surprise and delight the user and audience. Throughout the process, we followed the Design Framework For Social Wearables (2019) as we considered how and why we would create our desired outcome. 

  • Sensing 

Our wearables project consists of using an accelerometer to sense body movement. The placement of the accelerometer was discussed by using Clint Zeagler’s Movement Sensor Placement Body Map. After taking into consideration garment manufacturing and wire placement, we decided to place the sensor in the back. We also considered that according to the accelerometer’s height, there would be different values for the y and z axis that would have to be addressed via calibration. After several tests, we saw that z axis values gave us consistent results in relationship to the wearable’s behaviour and environment. 

  • Actuating 

“Fairy Dust” has two expressive visual outputs that are meant to capture the attention of the public. As a kinetic wearable project, we made use of servo motors to activate movement. Motors were attached to fairy wings in the back to create a flapping motion. As part of the immersive experience, the other actuator was interactive fairy dust visuals using a particle system. 

  • Sensing-Actuating Interplay 

Since fairies are mythical creatures, our perception of their wing behavior is heavily influenced by childhood characters and stories. In our interpretation, we decided to use an upright position to make the wings move. When the accelerometer senses a change in this position, the wings will stop moving. 

At first, this interaction may be a surprise for the user, creating a sense of magic. As the user keeps interacting with the wearable, the input may become more apparent. At this point, the user can start expressing through their body movement while changing the behavior of the outcomes. 

  • Personal and Social Requirements 

This immersive experience requires attention from the wearer. A big decision to incorporate visuals was to keep the user’s engagement and focus. This physical immersion allows participants to step into a unique interaction and let their imaginations flow. We wanted to create an experience where the user feels like part of the story. This wearable is meant to create an audience and allows for different interpretations. 

Photos

  • Being Worn & Displayed

img_0763img_0764img_0783img_0761img_0770

Video of Interaction

Interaction of body movement (CPX accelerometer values attached to serial monitor) controlling fairy dust animation on TouchDesigner:

finalvisual

Supporting Sketches, Diagrams, Models, Renderings

  • Visuals Process

To decide the look of the visuals we decided to make a moodboard. This helped us choose a colour palette and aesthetics.

visualsmoodboard

The next step was to allow serial communication between MakeCode and TouchDesigner. After some research we realized that this connection was only possible on Windows. We were able to make it work by switching computers and using functions to reference these values. 

To make the visuals reactive we used a sphere geometry function. The position of the sphere referenced the serial monitor. We then used the sphere as an input in the particlesGPU and customized the behavior of the particles.

After testing the behaviour of the particle system, we customized the look to match the desired aesthetics of our immersive experience.

finalvisual

  • Wings Process

Being our first time working with micro servo motors, or any motors as a matter of fact it was important to run many trial and error sessions. In this first video here, one half of the wings were roughly attached to view the potential motions the motor could create.

After seeing how just the motor on its own made more of a “wiggle” motion rather than “flapping”, we were lent a 3D printed 90 degree servo attachment to allow for a more seamless flapping motion. At this point in time, the wings were moving seperately, rather than together.

The final step of the motor mechanics was placement inside the vest. We had to ensure they were sewn in at an appropriate height and distance from each other for a more accurate emulation of “flapping” wings”. At this height, the wings do not obstruct the wearer’s movement or offer any discomfort which is exactly what we were aiming for!

  • Vest Process

Since this project was more of an exploration for self-expression, it was important that we created a wearable that captured the majestic style and mood that was envisioned. We decided to create a vest from scratch using materials sourced from FabricLand. I started out by using a form-fitted shirt as a guide since the wearable would be fitted to my (Angelina’s) body. The final wearable ended up being a hand-sewn double lined vest made of a flannel fabric, with tule sleeves and a tule trim, finished off with rose embellishments. The wings were the only thing that were pre-made (purchased from Dollarama). This was my first time doing something like this and I am very proud of the outcome!

Challenges & Successes

Being a collaborative project, we depended on meetings and constant communication. As per safety precautions due one of us contracting COVID-19, we were forced to work remotely on the last week before the due date. This was challenging since only one of us had access to the wings and vest. We also needed access to Windows to connect the serial monitor from MakeCode and, unfortunately, this led us to not see both finished pieces work together. 

In spite of all circumstances we are both very proud of the outcome. We followed a schedule and we communicated really well throughout the process. We set goals for each team meeting and we assigned responsibilities to work on our own. We were able to successfully work as a team and individually. 

Next Steps

We would definitely want to keep working on this project as a collaboration. Something  that we want to incorporate is another output such as lightning. We had some trouble incorporating neopixels into our design, we also discussed the possibility of adding fibre optics and the design possibilities that this would bring. 

Our goal was to see body movement interact with the wings and visuals. Nonetheless, another exploration that we are curious about is making the visuals interact to the wing’s movement. We’re both curious to keep exploring this little world that we created and the endless opportunities for different unique interactions. 

References:

Dagan, E., Márquez Segura, E., Altarriba Bertran, F., Flores, M., Mitchell, R., & Isbister, K. (2019). Design Framework for Social Wearables. Proceedings of the 2019 on Designing Interactive Systems Conference. https://doi.org/10.1145/3322276.3322291

Zeagler, C. (2017). Where to wear it. Proceedings of the 2017 ACM International Symposium on Wearable Computers. https://doi.org/10.1145/3123021.3123042