Project title: Wait!
This project targets on facilitating walking activity for users with vision impairment by incorporating TENS (transcutaneous electrical nerve stimulation) device as a form of haptics. People with blindsight—the ability to detect things in the environment without being aware of seeing them, are faced with the obstacles of perform daily activities without bumping into erect objects or surfaces. They need to use a stick to track obstacles or require guidance by another person when walking around buildings.
Wait! is a piece of wearable that addressed the demand of users with vision issues by providing audio and haptic feedback to alert users from being too close to impediments. It consisted of an ultrasonic sensor to detect real-time distance from the user to the object/surface in front of. When user has walked up too close towards the object/surface, the ultrasonic sensor will trigger the switch in the relay, generating the sound of a click and send low-voltage electrical pulses via TENS device to human body. Only when user stepped back, the electrical pulses will be put on halt. Henceforth, user are better supported and assisted in navigating indoor or outdoor spaces.
By incorporating an established TENS device, users are able to adjust the strength and frequency of pulse signals according to their own preference and level of comfortableness. The value could be set beforehand and saved for future uses with customization.
EMS(Electrical Muscle Stimulation) and TENS has been widely employed in the field of medicine. They can be considered as supplements to conventional muscle training, particularly for therapeutic treatment and physical rehabilitation. TENS is more specifically used in pain treatment that it stimulates the nerves exclusively by delivering electrical signals that do not trigger muscle movement. Applying TENS signals to painful spots at the body can reduce discomfort and relieve pain (Gibson et al., 2019).
Despite an emphasis on the field of medicine, there are emerging studies of using EMS as a form of haptics in the creative industry. Studies have been conducted around the cross-section of wearable devices, mixed reality, human-computer interaction and experience design. EMS has been used as a haptic input and output technology in wearable and textile-based computing through crafting comfortable textile electrodes (Pfeiffer & Rohs, 2017). EMS has also been incorporated into haptic interfaces to simulate the force feedback effect caused by a collision, in generating a mixed reality tennis game (Farbiz et al., 2007). With all the applications of EMS in non-traditional settings, TENS however, is yet to discovered and applied in other disciplines from an innovative perspective.
One case study that informed this project is Using Electrical Muscle Stimulation Haptics for VR. While traditional approaches in VR focus on lightweight objects via skin receptors such as vibro-tactile gloves, simulating heavy objects is still confronted with limitations by traditional methods of physical props or hand tethering (Burdea, 2000). This study explores how to better render heavy objects in VR via EMS in the form of wearables. Its main concept is to prevent the user’s hands from penetrating virtual objects by means of EMS. Tension is created in the user’s bicep, tricep, pectoralis, and shoulder muscles. The system stimulates up to four muscle groups to generate the desired tension, thereby constructing the desired realistic experience of touching the wall or lifting up heavy objects.
The system is encapsulated into a small backpack, which could be worn by the user. The backpack contains a medical compliant 8-channel muscle stimulator, which is controlled from within the VR simulators via USB. Other components include a typical VR system consisting of a head-worn display (Samsung/Oculus GearVR) and a motion capture system (eight OptiTrack 17W cameras).
This project informs the design of Wait! in the way that in a VR environment, users are less capable of sensing the surrounding while moving around – similar to be under the vision-impaired circumstance. Using EMS to generate muscle tension and henceforth simulate the weight of object is intriguing and very informative in terms of possible applications of muscle stimulation.
Other related works include vibration-based tactile haptics such as CyberTouch by Virtual Technologies (Burdea, 2000), pneumatic gloves using air pockets (Tarvainen & Yu, 2015), tethers and exoskeletons for fingers or upper body muscles. Passive haptics are also oftentimes used in VR to simulate the existence of objects such as still props and props placed by human or robot. Even though tactile haptic could potentially support a better texture rendering, it does not deliver a directional force that acts upon the user’s hands or muscle groups. Pneumatic gloves are also confronted with the similar challenge of representing heavyweight objects. Hence the utilization of EMS/TENS to activate muscles is of great value and importance to be further investigated.
Another study that I have looked into is Remote Controlled Human project which uses Spark Core, a TENS unit, and a relay to remotely control a human minion over WiFi. The Spark Core connects the TENS unit and user it attached to with the Internet of Things. With the TENS unit stimulating involuntary muscle movements, the relay acts as a bridge between the signal provider and the receiver. This project is a great reference for circuit construction, specifically how to bridge a commercialized healthcare product with computer softwares via a simple relay. It also extends the future possibility for hooking up sensors and writing one’s own sketches via the Spark IDE to generate outputs based on all kinds of conditions, such as rhythmic music input, light levels and et cetera.
- 1 Arduino UNO
- 1 TENS unit (TENS 3000)
- 1 Ultrasonic Sensor HC-SR04
- 1 relay unit
- 1 Felt hat
- Breakout cables
- 3 Male-to-Male jumper cables
- 2 Male headers
- Wire cutters/strippers
- Sewing kit
- Small slotted screwdriver
- Unobtrusive placement. (The hat works at the appropriate area of human body without impeding dynamic body movements.)
- Design for human perception of size. (This wearable is designed to minimize thickness and weight as much as possible. The sensor is within the range of normal hat weight to be unobtrusive and transparent.)
- Containment. (This piece of wearable contains materials of all electronic components, wires, electrodes, etc. While some of these things are malleable in form, there are many with fixed volume that one needs to consider how the ‘insides’ bring to the outer form.)
Challenges & Successes:
Figuring out a way to incorporate TENS signal that is non-harmful yet effective on human bodies was one of the biggest challenges during the early stage of this project. As electrical muscle stimulation has a safe range of usage and a varied level of comfortableness upon users across different demographics, utilizing TENS needs thorough preliminary preparation and physiological backup. Out of the safety concern, I decided to proceed with a relatively safe and minimal side effect TENS device which has been clinically tested and commercially employed.
Reference to Remote Controlled Human project, I was able to hack into TENS 3000 unit and control its digital switch using a relay and Arduino UNO. The analog control of TENS unit would be much more complicated with an established device.
The construction of this project was also one big challenge in terms of fitting the whole circuit into the interior of a hat while maintaining enough space for user’s head to fit in. I mapped the circuit onto the inner surface and tried to design an optimal route for wires to connect to each component without taking extra space. The Ultrasonic sensor was placed in front of the hat with the other part of sensor hidden in the hat brim by poking out two holes. The wires were attached underneath with another cut to hide the connection. However, it still leaves visible wires connected to battery or electrodes that users might find complicated and troublesome when attaching TENS pads onto the body.
Furthering this project, I would like to experiment with controlling TENS/EMS signals in an analog way rather than digital. The wearable could be elevated if it can sense body signal as inputs and adjust the output automatically with the assist of Arduino programming. However, figuring the way of connection and making sure the safety range of pulse stimulation are going to be the two biggest challenges.
In order to achieve that, it might require the abandonment of established TENS units but to figure out a way to manually set frequency/strength of electrical stimulations. It adds to the level of insecurity and danger as it is not monitored under a controlled environment.
Incorporating EMS into other creative disciplines as an innovative medium would be interesting to be explored, such as with the cross-section in music, performance art, writing, education and et cetera.
Farbiz, F., Yu, Z., Manders, C., & Ahmad, W. (2007). An electrical muscle stimulation haptic feedback for mixed reality tennis game. SIGGRAPH ’07.
Follmer, S., Leithinger, D., Olwal, A., Cheng, N., & Ishii, H. (2012). Jamming user interfaces. Proceedings Of The 25Th Annual ACM Symposium On User Interface Software And Technology – UIST ’12. doi: 10.1145/2380116.2380181
Gibson, W., Wand, B., Meads, C., Catley, M., & O’Connell, N. (2019). Transcutaneous electrical nerve stimulation (TENS) for chronic pain – an overview of Cochrane Reviews. Cochrane Database Of Systematic Reviews. doi: 10.1002/14651858.cd011890.pub3
Janczyk, M., Skirde, S., Weigelt, M., and Kunde, W. (2009). Visual and tactile action effects determine bimanual coordination performance. Hum. Mov. Sci. 28, 437–449. doi: 10.1016/j.humov.2009.02.006
Jones, S., Man, W., Gao, W., Higginson, I., Wilcock, A., & Maddocks, M. (2016). Neuromuscular electrical stimulation for muscle weakness in adults with advanced disease. Cochrane Database Of Systematic Reviews, 2016(10). doi: 10.1002/14651858.cd009419.pub3
Lee, J., Kim, Y., & Jung, H. (2020). Electrically Elicited Force Response Characteristics of Forearm Extensor Muscles for Electrical Muscle Stimulation-Based Haptic Rendering. Sensors (Basel, Switzerland), 20(19), 5669. https://doi.org/10.3390/s20195669
Lopes, P., You, S., Cheng, L.-P., Marwecki, S. & Baudisch, P. (2017). Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation.. In G. Mark, S. R. Fussell, C. Lampe, m. c. schraefel, J. P. Hourcade, C. Appert & D. Wigdor (eds.), CHI (p./pp. 1471-1482), : ACM. ISBN: 978-1-4503-4655-9
Pavani, F., Spence, C., and Driver, J. (1999). Visual capture of touch (tactile ventriloquism); out-of-the-body experiences with rubber gloves. J. Cogn. Neurosci. 11:14.
Pfeiffer, M., Schneegass, S., & Alt, F. (2013). Supporting interaction in public space with electrical muscle stimulation. Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication.
Robertson, A. (2021). Meta’s sci-fi haptic glove prototype lets you feel VR objects using air pockets. Retrieved 21 April 2022, from https://www.theverge.com/2021/11/16/22782860/meta-facebook-reality-labs-soft-robotics-haptic-glove-prototype