Shreeya Tyagi, Thoreau Bakker and Jeffin Philip
inFORM is a Dynamic Shape Display that can render 3D content physically, so users can interact with digital information in a tangible way. This project was done by Tangible Media Group in MIT Media Lab. The purpose of this case study was to understand the design process and the research that was involved in the project.
MIT media lab created an interface based on inFORM that was able to give urban planners more control to be able to shape and view entire cities. Dynamic shape displays changes how we virtually collaborate from a display. We can touch and manipulate objects from a distance and also collaborate on 3D data sets.
MIT Media Lab
The MIT Media Lab is an interdisciplinary research laboratory at the MIT devoted to projects at the meeting of technology, multimedia, sciences, art and design.”Inventing a better future” is the theme of the Media Lab’s work. A current emphasis of Media Lab research, which encompasses the work of several research groups, is on human adaptability. Other Research groups at Media Labs are focused on .
Overview of the Tangible Media Group
The tangible media group was formed by Professor Hiroshi Ishii, explores the Tangible Bits & Radical Atoms visions to seamlessly couple the dual world of bits and atoms by giving dynamic physical form to digital information and computation.
Vision driven design research of the Tangible Media Group
“Looking back on the history of HCI, they notice that quantum leaps have rarely resulted from studies on users’ needs; they have instead stemmed from the passion and dreams of visionaries like Douglas Engelbart. By looking beyond current limitations, the group believes that vision-driven design is critical to foster these quantum leaps, while also complementing needs-driven and technology-driven design. From Tangible Bits, an early example of their vision-driven research, they are shifting to Radical Atoms, which seeks out new guiding principles and concepts to view the world of bits and atoms with new eyes, with the goal of trailblazing a new realm in interaction design.
From the three approaches in design research: technology-driven, needs-driven, and vision-driven, they focus on the vision-driven approach due to its lifespan. They know that technologies become obsolete in ~1 year, users’ needs change quickly and dramatically in ~10 years. However, they believe that a clear vision can last beyond our lifespan. While they might need to wait decades before atom hackers (like material scientists or self-organizing nano-robot engineers) can invent the necessary enabling technologies for Radical Atoms, we strongly believe the exploration of interaction design should begin from today.” Tangible Media Group
Context, Significance, Related Works
Hiroshi Ishii, head of the Tangible Media Group (TMG) published an interesting paper in 2008, contextualizing the inForm project in terms of human evolution. In it, he notes that humans have developed “sophisticated skills for sensing and manipulating our physical environment”, yet most of these skills are “not used when interacting with the digital world where interaction is largely confined to graphical user interfaces“ (Ishii 32). He argues that, despite the ubiquity of graphical user interfaces (GUIs) as championed by Microsoft and Apple, there is something greatly lacking in these methods of interaction — that they do not allow us to “take advantage of our evolved dexterity or utilize our skills in manipulating physical objects” (32). That this project addresses these challenges and presents an alternative way to interact with digital content is both fascinating and valuable. It is not the first or only technology to interact with computers in a novel way, but the effect created is almost magical.
If there are other potentially more intuitive technologies that exist, why have they not been adopted? Perhaps the current dominant paradigm of keyboards, mice and increasingly, touchscreens, is to some extent influenced by the market. The assumption that it has to do, at least in part, with economies of scale seems plausible, and perhaps when demand increases enough, more alternative interaction technologies will become available. The tremendous potential of technologies like inForm to harness the incredible ‘touch’ skills humans already possess, speaks to the importance of research in these fields. While inForm was both groundbreaking and unlike anything seen before, there are a number of other projects that relate to the project conceptually.
The following examples will highlight other technologies that deal with interacting with data and virtual objects in unconventional ways.
Haptic Sculpting Device
There is a research lab at the University of Guelph called the Digital Haptic Lab, run by Dr. John Phillips and Christian Giroux. Their lab takes its name from a sculpting device, that provides haptic feedback through small motors, embedded in a multi axis pen type device. As a user uses the tool to ‘sculpt’ an onscreen three dimensional object, the haptic device varies the motor feedback to give the feeling of interacting with a real world object. The effect is almost startling, as one is able to ‘feel’ with the muscles of the hand, a virtual object that is not actually there. This device has a number of research and commercial applications, one of which is the design / sculpting of coins for general circulation.
This device is quite different from inForm in that it represents a virtual object in 3D space without the object actually being there, whereas inForm, as a “shape display”, actually represents virtual objects as real objects (albeit in a low resolution) with its extendable pin blocks. Although very different, the underlying issues they address are related: how we make virtual and remote objects tangible.
Automotive Design: Still Using Clay
Another striking example of the relationship between technology and tangibility, is the automotive industry’s continued use of clay to model vehicles. Despite access to the best 3D software packages, holography, VR and other cutting edge technology it is still common, even a mandatory in many design labs to build clay models. Take the following quote from an Wall Street Journal article, one example of many discussing this fascinating phenomena:
“Indeed, despite Ford’s use of three-dimensional imaging technology that allows executives to don headsets and see a virtual vehicle in a computer-generated cityscape, the top brass won’t sign off on producing a new car—a decision that can involve spending a billion dollars or more—until they see full-size physical models” (see reference below).
The full-size models the article describes, are made of clay built on an armature and refined using hand held scrapers. For all the utility afforded by new technology, there is something missing in these tools requiring a real human touch, and the ability to see a full size model in the real world. The article notes that it is not a one or the other scenario however, and that 3D modeling is used extensively in the design process. The workflow goes back and forth, working together, and both tools afford special abilities and perceptions.
Again, this example is very different from the inFOrm project, even more so than the haptic devie. It is presented however, as an example of the importance of tangibility, and real world object. That it is essential even for multinational corporations with huge budgets and access to the latest technology, speaks to the value of what inForm is doing. The inForm project incorporates some of the best of both worlds in a way: The reproducibility and flexibility of the digital world, with the intuitive qualities and space of the analog / ‘real’ world objects.
Other related project:
Kinetic blocks: http://www.theverge.com/2015/10/14/9529947/mit-kinetic-blocks-shape-display-video
Technical overview: How does it work? sensing method, actuation method, materials, relationship to user and audience
InForm is a dynamic shape display and at the same time it is also a tangible user interface. The key principles behind the interactivity of inForm are dynamic affordances and constraints. This is implemented through haptics, actuated affordances, actuation of physical objects etc. By mimicking familiar interfaces from tangible and physical domains, the inFOrm interface invites and encourage the user to interact with it. By providing constraints on these interactions the user input can be measured and used to control other actions.
The device consists of a shape display with rectangular pins, each controlled by a linear actuator, a kinect to sense depth information, a projector to display data. When used as a communication device both end users will have a set of theses components.
Each pin of the shape display can be individually moved using the linear actuators. The movement of these actuators are controlled using arduino chips. But the depth calculation and other calculations are done on an external computer. The linear actuator mechanism also employ PID controllers to detect and keep track of the position of pins and to provide accurate motion with continuous error correction.
The depth information is calculated from the depth image stream of kinect and is mapped to the movement range of the shape display pins. The projector is used for the visual feedbacks.
@wsjeyesonroad. “One Thing Isn’t New in Car Design: Clay Prototypes.” The Wall Street Journal. Dow Jones & Company, 2014. Web. 13 Dec. 2016.
Ishii, Hiroshi. “The Tangible User Interface and Its Evolution.” Communications of the ACM 51.6 (2008): 32. Web.
Ishii, Hiroshi. “Materiable: Rendering Dynamic Material Properties in Response to Direct Physical Touch with Shape Changing Interfaces.” http://tmg-trackr.media.mit.edu/publishedmedia/Papers/598-Materiable%20Rendering%20Dynamic%20Material/Published/PDF
Ishii, Hiroshi. “TRANSFORM: Embodiment of “Radical Atoms” at Milano Design Week.” http://tmg-trackr.media.mit.edu/publishedmedia/Papers/554-TRANSFORM%20Embodiment%20of%20Radical/Published/PDF
Ishii, Hiroshi. “Radical Atoms: Beyond Tangible Bits, Toward Transformable Materials.” http://tmg-trackr.media.mit.edu/publishedmedia/Papers/485-Radical%20Atoms%20Beyond%20Tangible/Published/PDF