Fundamentals of Immersion

May 29

Annotated Bibliography

Geertz, Clifford. “Deep play: notes on the Balinese cockfight. Daedalus. Fall 2005, 134, 4: Research Library. pg 56

  • Largely an anthropological treatise on the Balinese. There is some exploration of cultural play and on the imperatives informing “deep” vs “shallow” play
    • “status” play is correlated to “deep” play while “money” play is correlated to shallow play
    • Does immersive play necessitate a certain amount of status, cultural cache or class? Is this why, at a glance, (and to be highly general) “blue-collar” types seem less interested or even baffled by immersivity?
    • Geertz lays out a series of facts that influence the depth of a given cockfight. This taxonomy is intriguing as a method for categorizing the depth of immersion
    • “It is a story they tell themselves about themselves”
  • Deep Play: from Bentham The Theory of Legislation.
    • play in which the stakes are so high that it is irrational to engage in it at all
    • In “genuine” deep play this is true for both parties
      • The play will net pain, collectively
    • Therefore Deep Play is immoral, says Bentham
  • Such play (to the Balinese, argues Geertz) is a symbol of moral import
  • “status gambling” is correlated to deep fights and “money gambling” is correlated to shallow fights
  • the cockfight in Bali is a dramatization of social concerns
    • expected to bet on a bird owned by kin or friends, eg
  • Social clout is correlated with the deep play cock fights
    • Balinese cock fighting is deep b/c of  the potential loss of status on the line
      • is Immersion deep play because of the removal of social safety mechanisms? The necessary excision of defense mechanisms in order to allow new experiences to take root?
    • Betting activates village and kingroup rivalries but in “play” form
    • no ones’ status really changes
    • “It is a story they tell themselves about themselves”

Konzack, Lars. “The Wunderkammer-Gesamtkunstwerk Model: A Framework for Role-Playing Game Analysis and Design.” Digra 2015, 14 May 2015.

  • Describes the Wunderkammer-Gesamtkunstwerk (Wu-Ge) Model
    • Subcreation
      • Worldbuilding
    • Ludus
      • Game mechanics and rules
    • Performance
      • RPG performed by players
    • Narrative
      • Storyline
    • Cabinet of Curiosities (Wunderkammer, “chamber of wonders”)
  • Structure based on two dichotomies:
    • Concrete vs Abstract
      • Performance & Narrative are Concrete, Ludus & Subcreation are Abstract
    • Action vs Contemplation
      • Performance and Ludus are Action, Subcreation and Narrative are Contemplation
  • A matrix of physicality/action/concretion vs abstraction/contemplation/conceptualized, similar to the direction I am exploring my study.
  • LARP compared to improv or performance theatre (Choy, Fatland, Flood, Lampo)
  • Tabletop RP is unique in that there is no external audience
    • a distinct literary/gaming tradition of their own
  • GNS Theory (Ron Edwards)
    • Gamism
      • competition
    • Simulationism
      • exploration of the fictional world
    • Narrativism
      • creation of story via roleplaying
  • Three Way Model (Bockman)
    • player behavior, style of play
    • Dramatists
      • storyline
      • argues this is the same as Narrationist
    • Gamists
      • solving
    • Immersionists
      • Living the role in a 360 illusion
      • vicariously live out characters (argues that this is the same as Simulationism)
  • Prior research. Largely focused on player behavior, not on conceptual theory. Possibly useful when designing immersive experiences, but not likely useful in the context of developing a theory of immersion
  • Wunderkammer term is taken from proto-museums,
    • “a microcosm of wonders that trigger imagination and ingenious thoughts as to the greater macrocosmos of which it is a representation”
    • “a macrocosmos not only of fictional laws, but ideal and fictional entities”
  • Proto-museums are an interesting avenue into pre-industrial worldbuilding and immersion!
  • WK means “tabletop RPGs have the ability to insert features into the setting and genre and make it somehow fit the feel or mood of the rest of the collection of ideas.”
  • Every item becomes a natural part of the collection
  • An artefact dropped into a RPG world becomes appropriate and contextualized by the setting (eg a Crystal Skull in D&D vs Call of Cthulhu)
  • Everything need not add up in the Aristotelian Theatre manner, where all parts must serve the performance.
  • The Wunderkammer is applicable to transmedia storyworlds – arguably, every storyworld milieu serves as a Wunderkammer as well
  • Gesamtkunstwerk (“all-embracing art form”)
    • term from Wagner (1850)
    • “Every part of the role playing game works together as a complete experience with as many powerful effects as possible
    • Only when each of Performance Narrative Ludus & Subcreation work together that the experience becomes a “proper” tabletop RPG one
    • Wagner wanted opera to have all its components work as one
    • “The aim is to get as much as possible out of what the medium… can offer”
  • Performance
    • Defined as pure drama analyzed through performance studies
    • central is the process of playing a character
    • an essential part of RPGs
    • does not assume a separate audience
      • necessitates a greater degree of introspection
      • “more literary”
    • about narrative and play – play-like routines
    • the true content is the presentation and exhibition of the performer
      • narrative and play are support of the performance
  • Narrative
    • the narrative structure through which the action unfolds
    • presented in an orderly fashion, but not a fixed narrative
    • Berger Rognli & Westlund narratologists
    • interactive narrative is made dynamic through player activity
  • Ludus
    • game-features
      • game mechanics
      • “deep play” (Geertz) <– check this out
      • Game classification (Caillois)
      • games as culture (Huizinga)
  • Sub-Creation
    • From Tolkien
    • refers to the fictional world of the roleplaying
    • few researchers go into detail about how shared worlds work
  • Action
    • Performance and Ludus
    • Ludus is applying abstract rules to adjudicate the results of Performance
  • Contemplation
    • Sub-creation and Narrative
    • designer must avoid suspension of disbelief in which anything can happen without reason or consequence
    • instead build on the concept of Inner Consistency of Reality
    • everything is there for a reason
  • Concrete vs Abstract
  • The factors in Konzack’s framework can be applied to immersive experiences as well as traditional RPGs. In Konzack’s terms, the immersive experience (like the RPG) is a Gesamtkunstwerk whose Wunderkammer of objects can be explored by player-audiences. Arguably the Wunderkammer of the immersive experience is less than that in the traditional RPG, as the RPG’s flexibility allows any game object to become immediately and seamlessly contextualized within the world at any time. This may not be so easy in the world of the immersive experience, whose suspension of disbelief is dependent chiefly on sense.

Williams, J. Patrick. Gaming as Culture: Essays on Reality, Identity, and Experience in Fantasy Games. McFarland & Co., 2006.

  • This work is less concerned with the process of immersing within games than it is in exploring the cultures that games generate. It does however touch on roleplaying as pedagogy, so I will be returning to this book when I look at Immersion as pedagogy.
  • ludology, the study of play
  • a ludologist is somebody who wants to have a better understanding of games
  • RPG historical basis is in war gaming
  • Researched as a pedagogical tool
    • games, sims, RP in the classroom
  • Fantasy gaming involves the creation of and interaction with/in social realities
    • the fantasy reality + the real world
  • virtual identities emerge within the social reality of the game – the border between these identities is fuzzy
May 21

Annotated Bibliography

Machon, Josephine. (2013) Immersive Theatres: Intimacy and Immediacy in Contemporary Performance, Basingstoke: Palgrave Macmillan.


  • Immerse: “be submerged in a liquid”, “to involve oneself deeply in a particular activity or interest”
  • Immersive: provides stimulation for a number of senses beyond sight and sound
  • Experience means “to undergo” for the purpose of this discourse
  • One-on-One or One-to-One is designed for one audience member
  • worlds can last longer than the event; one example is even that the proposal/grant is written in-world
  • points out that the space between the show’s end and the return to the world can be “theatricalized”
  • the principle of minimum fiction
    • coined by Stevens
    • “minimum fiction required to play with the real world and… recast it as otherworldly in the imagination and experience of the beholder”
    • in other words, the least that must be done to turn the mundane into something exciting to the audience’s imagination
  • growing fan-bases are said to come about from a growing desire for “genuine physical connection”
  • reduced opportunities for “sentient human interaction” in the face of digital communication
  • immersive theatre requires bodily engagement, tactility
  • “sensually stimulates the imagination”
  • “epic”: grand in execution and profound in appreciation, spectacular but with powerful meaning
  • Immersive theatre means new environment and context every time – no familiar theatrical lights down/lights up presence/lack of presence
  • Techniques and sensibilities attached to immersive theatre can be traced to Modernist Theatre (1905-59) owing to the growing interdisciplinarity of the time
  • the mix of “fun, flux and furore” of immersive practice can be traced the theatre traditions of Commedia Del’Arte and European street theatre, which influenced the so-called “Assemblages, Environments and Happenings” of the mid-20th century
  • Antonin Artaud’s “total theatre” – “an intensive mustering of objects, gestures and signs used in a new spirit” – immersive theatre puts much of his manifesto into practice with its “sensual practice”
  •  Happenings – Allan Kaprow
    • The line between art and life should be kept as fluid, and perhaps indistinct, as possible”
    • viewers are fused with the space-time of the performance and lose their identities
    • should take place over multiple locations and avoid the static
    • Should not be repeated
    • Duration perceived as real and experienced as opposed to the conceptual time of traditional theatre
  • Installation Art
    • “art in to which the viewer physically enters”
    • the effect is that the viewer (“actor”) is “submerged”, “engrossed”
  • Immersive experience arise where medium and message are fused
    • the perceiver is within the art
  • Immersive technologies
    • seek to heighten sensual experience
    • seeks control in order to get at the experiential intent of the work
  • impact of duration + space and human interrelationships within the space establish an environmental sensibility that allow wider ecological concerns to exist within the work
  • Communitas
    • place/space/art as living experience facilitates communal recreation
  • Immersive theatre requires
    • the letting down of boundaries
    • giving in to experience
    • willingly engaging in a truly embodied fashion
  • Requires liveness and “live(d)ness
    • the live performance is fleeting, never can be repeated, but lasts in the receiver’s memory of the event
    • praesent – being at hand
    • “live(d)”: the performing/perceiving body is living, tactile, haptic material
    • live(d) bodies establish a constant praesence

Part One: Defining Immersive Theatres

Definitions and Details

  • Traditional theatre vs Immersive Theatre
  • The conventional relationship with theatre asks us to forget our physical bodies. immersive reminds us of our physical presence. Perception is more important than attending.
  • traditional theatre is audio/visual and offers nothing to other senses
  • “Immersive” first applied to computer technologies in the 1980s
  • In digital, “immersive” is applied to displays that generate a 3d image that surround the user.
    • or, deeply involve the senses or create an altered mental state
  • Grau: “Immersion arises when the artwork and technical apparatus, the message and medium of perception, converge into an inseparable whole.. maximize the intensity of the message being transported”
  • In gaming theory and game studies “immersion” is used interchangeably with “presence”
  • Immersive theatre must be a practice in and of itself, and not defer to any technological practice
  • a sense of presence is always related to form (Calleja)
  • the live(d) experience is a tangible fact and a pivotal element of the immersive experience
  • Immersion as Absorption
    • the event is a total activity that engrosses the participant within its form
  • Immersion as Transportation
    • the participant is imaginatively and scenographically reoriented in another place, requiring navigation according to its own rules
    • in games this occurs in a conceptual space, in immersive theatre this is a conceptual space and an inhabited physical space
    • affords actual physical cohabitation and contact with human bodies
  • Total Immersion
    • Involving both of the above and leading to an uncanny recognition of the participant’s own praesence within the experience
    • the participant can fashion their own narrative or journey
  • All immersive events exist at some point between these three criteria
  • Earliest claimed usage of “immersive” in relation to theatre is attributable to Artangel; Morris uses it to describe La Fura Dels Baus in 1983.
  • Oily Cart using immersive sensory explorations as educational and therapeutic purposes, such as being immersed in hydrotherapy pools, sensory explorations using scent, touch, feeling of momentum using trampolines
  • Immersive theatre begins to be defined as a genre between 2005-2010
  • Immersive theatre is the practice which allows participants to be in “the playing area” with the performers

Features and Finer Details: A Scale of Immersivity

  • Venue, architectural details & design, landscapes, are of concern
    • can incorporate a focus on geographic location, community, culture, history politics
  • “Audience involvement, audience evolvement”
  • Threefold agenda of “activation; authorship; community” (Bishop)
  • “theatre experienced from within… part of it rather than fundamentally distinct”
  • Audiences are keen for visceral experiences that can remind an individual what it is to feel alive; esp as people spend more time online
  • Machines like ATMs now perform work that once represented “many opportunities for exchanges, pleasure, or conflict”
    • “art is a state of encounter” (Bourriaud 2006)
    • Immersive practice creates a space for reinvigorating human interaction and exchange
  • Difficult to come up with a term for audience members (I’ve been saying “participants” mostly)
    • immersants vs immersees
  • Mercuriali: “everyday technology use can serve to distance human interaction and destroy a sense of personal connection… The immersive practice can employ technology to return the user to the personal relationship that is more precious than anything”
  • A central feature of immersive theatre is related to the sensual construction of the world
  • Banes taxonomy of odour: 1) to illustrate words, characters, places, or action. 2) to evoke a mood or ambience. 3) to complement or contrast with aural/visual signs. 4) to summon specific memories. 5) to frame the performance as ritual. 6) to serve as a distancing device
    • Olfactory sense has the capacity to “summon up memories”, argues that it in some way fuses past and present in time
  • Haptics (meaning skin to surface, skin against skin, kinaesthetics and proprioception – not necessarily the digital definition of haptics) are “often crucial” to immersive performance experience
    • Haptics are “whole body” experience
  • touch is an opportunity for “sensate involvement” – 78
  • The prioritization of all human senses opens up a new taxonomy for appreciation – 80
    • Articulated by Lundahl and Seitl as the “sixth sense”.
    • The “(syn)aesthetic sense”
    • draws on the full cognition of the body to make sense and sense the inarticulable
    • This is unusual due to the process of becoming aware of the fusion of senses
  • “the play of the senses… allows for an immediate and intimate interaction within the performance event.
  • Immersive practice is, and must be, an embodied event. – 83
  • Immersive practice can encourage individual to invest in each other as well as the work
    • a palpable sense of communitas – 85
  • Higgin: the storytelling environment bleeds into the real world – 87
  • collaborate with communities to encourage them to find an artistic access to their local environment + reignite their relationship to their community (Holdsworth) – 88

Immersive Perspectives

  • Theories pulled from multiple disciplines
    • philosophy and art
  • (syn)aesthetics – 104
    • making sense/sense-making
    • understanding through embodied, somatic perception via feeling
  • aesthetics
    • subjective creation, experience, and criticism of the art
  • corporeal memory and embodied knowledge – 105
    • primordial human impulses
    • the emotional and physiological capabilities of the physical body
  • Deleuze: “Immanence” – 108
    • fusion between immanence (a state of being, within the material/physical qualities of human experience) and transcendence
    • transcendence is supressing the materially immanent and entering int oan otherworldly state. “Transcendence is always a product of immanence”
    • (syn)aesthetically pulled between the plausible and implausible – 109
    • “sensation is in the body and not in the air”
    • Deleuze aligns with Artaud’s theories for theatre
    • “artworks just are sensation” and allow us to enter into a “pure presence” or a “pure plane of immanence”
  • Kathryn Linn Geurts: “Seselelame” – 110
    • Anlo-Ewe peoples’ word
    • embodied consciousness for which in some non-western cultures there already exists an expressive vocabulary (Geurts, 2003, 2006)
    • social and cultural experience is bound in sensory order established by early education
    • Seselelame: “perceive-perceive-at-flesh-inside”
    • Anlo sensory does not divide itself by five senses (a Western ideal) but by a generalized feeling within the body that includes internal senses (such as balance and proprioception) and external ones, as well as perceptual, emotional, intuitive dimensions of experience – 111
    • both a specific sense and a descriptor for human sentience in general
    • The embodied sentience innate in all humans, which exists in everyday pursuits
      • (intention is not to appropriate the Anlo terms a theoretical concept and academic jargon, but to show how Geurts’s study is useful to prioritizing embodied knowledge) – 112
  • Umberto Eco
    • wrote of the active judgment of the audience member of the audience member when interpreting and experiencing artistic works – 113
    • “art can only represent and express the experience of human existence by employing equally open and complex forms” – 114
    • where works are open, the performer can “impose his judgement on the form of the piece” which forms “fresh dialectics between the work of art and the performer” – openness is “indefiniteness” – audience is an active collaborator in the work, resulting in an outcome where the work is constantly being formulated and interpreted
    • the audience and artist must work together
    • the form of the artwork gains an “aesthetic validity” in proportion to the perspectives from which it can be viewed and understood
    • Eco notes that his perspective comes from contemporary theory and practice and cannot be applied to work prior to the twentieth century – 115
  • Jacques Ranciere
    • theorizes the relationship between aesthetics and politics
    • his perspective is central to the collaborative nature of immersive events
      • suggests a democratic community within these events – 117
    • refers to Plato’s “choreographic community” where everyone moves to the “community rhythm”
    • emancipation: “the blurring of the boundary between those who act and those who look; between individuals and members of a collective body”
    • “viewing” is an action that encapsulates “observes, selects, compares, interprets”
  • Nicolas Borriaud
    • “art is a state of encounter” – 120
  • Juhani Pallasmaa
    • the sensuality of space – 123
  • Doreen Massey
    • The politics of space and place – 130
  • Gaston Bachelard
    • the poetics of space – 137
May 21

Contextual Analysis – DR. SILVER: A Celebration of Life

DR. SILVER: A Celebration of Life

DR SILVER: A Celebration of Life is a joint production of Outside the March and The Musical Stage Co. Billed as an “immersive musical”, it is the brainchild of librettist siblings Anika and Britta Johnson who, partnered with director Mitchell Cushman, developed an ambitious musical presented in a unique way, with many feints toward total immersion. However, due perhaps to the creators’ comfort with traditional musical theatre conventions, the production falls short of achieving the kind of immersion striven for by Machon and her contributing artists.

Held at Heliconian Hall in Toronto, a small gothic building that looks and feels uncannily like the kind of hall in a small town that a small cult might rent for a gathering, DR SILVER is a musical about a family who is at the centre of a cult whose messianic patriarch figure, the titular Dr Silver, has recently passed away. It explores the grief of each of the family members and their desires for the future. Much of the drama comes from each of the family’s feelings about the absent Gordon, who forsake the cult some time ago and may or may not be dead in the present. Aside from the family, who represent the entirety of the speaking cast, a chorus of young cult members fill out the space and serve as ushers, dancers, and singers.

The space is decorated and lit in a clever fashion, allowing the production to pivot from the feel of a mundane meeting hall to the rapturous wonder of a church congregation. A projection of the late Dr Silver rendered as stained class changes subtly as the production goes on. Audience-participants, who are treated as welcomed visitors and (it is presumed) are cultists as well, are seated on pews along four walls of the space and invited to paw through a gorgeously printed chapbook that serves as the holy book of the cult.

The chapbook, like a bible, also contains hymns, and early in the production audience-participants are invited to sing along with a number.

The first fifteen minutes or so of the production are deeply immersive. From the moment the audience-participant arrives on the site (which, as mentioned, is an imposing but out-of-the-way building) are are greeted by signs advertising the funeral of the late Doctor, they are treated as guests of the cult. Ushers/chorus members are dressed in identical unsettling blue scrubs and welcome participants with euphoric words and wide smiles while led to places on the pews. The performance takes place in full light, so every audience-participant is equally present as the performers. After they are invited to sing, the audience-participants are offered a drink of what looks like blue Kool-Aid that, it is suggested, contains a hallucinogen. This is perhaps meant as an explanation for why the remainder of the play is staged as it is, but sadly it is the last feint toward immersion that the play makes.

Shortly after the Kool-Aid sequence the show pivots from immersivity into traditional musical theatre-in-the-round. The remainder of the show is rendered as a series of flashbacks featuring the family, catching us up to how and why they find themselves relating to one another as they do in the present. These flashbacks are staged under spotlights, removing them from the present/presence of the immersive space, and the audience-participants are darkened, endistancing/alienating (as Brecht would say) them from the moment being staged. All pretenses of immersivity are gone – from this point on, the audience-participants are participants no more.

It is particularly disappointing because DR SILVER had, until this point, been an exemplar of Machon’s total immersion. The greeting and setting, as well as the opportunity to chat with other audience-participants, establish a firm external form through which to experience immersion as absorption, and the book of hymns, performers, and especially the drink of Kool-Aid engage all five senses, succeeding at immersion as transportation.  Not totally realized is the social communitas discussed by Machon, though it is arguably attempted through the invitation to sing with the rest of the group.

The first act of DR SILVER delivers an embodied praesence (defined by Machon as being “at hand”) for audience-participants. The remainder is a familiar piece of musical theatre. Immersive elements remain as the chorus of cult members stalk the pews and the projections on the walls metamorphize as the plot continues, but there are no further invitations to engage – in other words, to realize one’s own praesence – and no further sense that the event possesses live(d)ness – that is, a uniqueness, and the knowledge that it will never be repeated as experienced.

May 19

Process Journal 3

Posted in Process Journal           No Comments »

Process Journal Week 3: Early Layout


Inspired by my exploration with AR, I envision a short immersive experience hinging largely on simple AR interactions using EyeJack.

Briefly, the experience as I envision it involves a character who uncovered a dark secret about the nature of reality before disappearing mysteriously. Participants retrace her steps using her research, uncover an artefact I’m thinking of as “The Lens” (incorporating EyeJack) which reveals secret truths, and realize that they too are infected by the same organism that led the character to catastrophe.

Elements I’d like to experiment with and/or include:

  • AR on the body (through a sticker or temporary tattoo)
  • Non-gamified puzzle elements
  • Unclear start moment – metatext (including the “artist’s statement”) are part of the immersion
  • Utilizing EyeJack’s scan-to-unlock feature as a checkpoint mechanic
  • Out-of-room elements (for example, a dead drop or telephone call)

I mapped out the room’s flow above in order to help visualize the necessary pieces and begin to consider the writing and fabrication that it might require. Currently I conceive of it as being mostly paper (perhaps mountable on 2 or fewer walls or bulletin boards) plus a locked box and some light fabrication.

Any writing and my thoughts are being kept on a live working document, which you can access here.

May 13

Pre-Cinema Immersion Contextual Analysis: The Pseudoscope

img_20190513_134316 img_20190513_142307

As a contextual analysis for a series of artefacts that are no longer being produced and experienced at scale, I thought it would be interesting to build one from scratch. I have encountered most of the artefacts discussed at some point in some form or another, but never a pseudoscope. I thought it would be interesting to build and experiment with one.

A pseudoscope is a device that uses a series of mirrors to swap the inputs of each eye.

pseudoscopeImage from

Looking through the pseudoscope makes the viewer confront and consider the minute differences between what is glimpsed through each eye. It is said to cause fascinating optical illusions. For example, looking through a pseudoscope at a spinning sphere with a stick in it is said to appear as if if the stick and sphere are spinning in opposite direction.

I found several sources online for building a pseudoscope quickly and for cheap. After gathering materials and overcoming some logistical problems with the maker lab, I was able to assemble a working pseudoscope in about half an hour.


The experience of looking through a pseudoscope without an illusion ready is lackluster. It takes some time to adjust it for the spacing of your eyes, and it takes time for your eyes to adjust. The minute imperfections of my pseudoscope are pronounced – the base is not perfectly flat so it must be held a certain way, or else the images do not line up and result in double vision. It also does not “auto-focus” for distance in the way our eyes do, requiring manual adjustment of the reflectors if you move from viewing something close up to something far away.

Once these issues are settled, though, the pseudoscope proves fascinating. It feels like it requires focus and alertness to view images through it, and although the images are familiar it feels like it is more work, physically, to view them.

The next steps would be to research and create a series of illusions for the pseudoscope, as well as tweak the current design for user-friendliness.

Works Consulted:

Make: “Weekend Project: $10 Pseudoscope.” YouTube, YouTube, 7 May 2009,

Make a Pseudoscope,

May 12

Process Journal 2

Posted in Process Journal           No Comments »

Process Journal Week 2: Experiments with AR

This week I have been considering story world artifacts and the potential for using Augmented Reality (AR) for hiding additional information. The “lens” of AR serves as an additional perspective and, to my mind, begs to be included as a storyworld artifact itself. How and why does the act of viewing through an AR lens modify the nature of the information revealed? How can the mechanism of AR be activated to engage critically with itself?

This week, though, I have focused on learning how to use EyeJack.

I started by making an animation using the simplest components at hand: myself and my phone’s camera. I took a few photos to serve as frames in a simple animation.


Then I opened up Photoshop, applied a “Reticulation” filter to each image to give it a bit of a mysterious feeling, and created a simple GIF animation using Photoshop’s tools.


I imagined this image and animation being somewhat like the moving pictures in Harry Potter, so I printed out a simple document with the trigger image.

EyeJack proved simple to use: the desktop EyeJack Creator app is used to upload the trigger image and the animation. The potential for setting up a series of AR interactions using EyeJack in a gallery or immersive happening space is exciting, especially with its ease of use.

May 12

Contextual Analysis

Speculative Design in She-Ra and the Princesses of Power.

The 2018 TV show She-Ra and the Princesses of Power (created by Noelle Stevenson and produced by DreamWorks Animation, a re-imagining of and not to be confused with 1985’s She-Ra: Princess of Power) demonstrates strong understanding of the speculative design tradition. Originating from a series of tight constraints – that is, being required to keep the same core characters, setting, naming conventions and basic plot from a quarter-century old show that famously was not very concerned with logic or narrative consistency – She-Ra and the Princesses of Power proves itself a work of speculative futurism masquerading as space fantasy.

She-Ra and the Princesses of Power is about Adora, a soldier from the army of the fascist Horde that has invaded and has been terrorizing the planet Etherea, who defects when she is chosen by the Sword of Protection. Using the sword she can transform into She-Ra, a powerful being dedicated to protecting Etherea. She makes friends with Glimmer, one of several Princesses of Power who are granted supernatural powers by ancient Runestones, and Bow, a talented archer. Together they travel Etherera with the goal of to reuniting the Princesses and defeating the Horde.

While the show has the trappings of a fantasy (and indeed the various powers of the Princesses are referred to as “magic”) the show contains details that suggest the show may be a science fiction instead.

Presumably the show’s creators were mandated to keep details from the original show – for example, the lead character and her iconic transformation into the titular She-Ra. These details would serve as entry points into the world.

The Sword of Protection, arguably the defining artifact carried over from the 1985 show, is described at one point in the 2018 version as a “portable Runestone”. The Runestone, then, is the source of She-Ra’s power. When Adora is removed from the sword she is unable to transform and access the various powers of the sword. Another clear requirement is the presence of the various Princesses of Power, who are holdovers in name and power from the original show. It is explained that each of the Princesses gain their powers from a Runestone as well, and that if their Runestones are destroyed or tampered with their powers can suffer.

So now we have the Runestones as artefacts of the world, and requisites for the existence of the princesses. This is further extrapolated by the existence of the First Ones, a highly advanced but mysteriously vanished progenitor species that left their technology (usually referred to in short hand as “First Ones’ Tech”) scattered around the planet. It is eventually revealed that the First Ones created the Runestones and that they are all connected through First Ones Tech that runs throughout the planet. The Runestones are technology, not magic, and dictate the nature of the communities that they belong to (the Runestone that affects nature is surrounded by a great forest and is tended by a community of survivalists, while the Runestone that affects cold is suspended inside a mountain of ice and is tended by Inuk-inspired arctic-dwelling people). They can even be hacked by computer, which happens several times in the show, at one point causing She-Ra’s behavior to change (suggesting that She-Ra’s transformation is technological in nature).

This further lends context to the question of why only Princesses can access the power of the Runestones (perhaps they are descended from First Ones?) and the question of why the world of Eternia is in such a bizarre state  —  why it seems to be divided by ecological biomes (the elemental influence of the Runestones, which aside from Adora’s are immobile) and lacks stars in its sky. None of these questions are the focus of the show, which tends to focus its plots on brief adventures and the interpersonal relationships of the protagonists and antagonists.

They serve as worldmaking details that belie the absurdity of the show’s presentation. Stevenson and the team behind She-Ra and the Princesses of Power used details from a defunct and little-cared for story world as entry points (or “core samples” if considering McDowell’s Mandala framework) to extrapolate an internally consistent story world that holds up against scrutiny – that is, stand up to suspension of disbelief – and ask the question “how would the world be if an ancient progenitor race distributed supernatural runestones on the planet?” with a straight face.

May 11

Annotated Bibliography

Coulton, P., Lindley, J., Sturdee, M., & Stead, M. 2017. ‘Design Fiction as World Building’. In: Proceedings of the 3rd Biennial Research Through Design Conference, 22-24 March 2017, Edinburgh, UK, Article 11, pp. 163-179. DOI: 10.6084/m9.figshare.4746964.

  • worldbuilding
  • Bruce Sterling: Design fiction is more “practical” than science fiction
  • “Design Fiction is the deliberate use of diegetic prototypes to suspend disbelief about change”
  • There is a diversity of mediums in which to create design fictions
  • What is the diegesis?
  • what is the “story” in “story world”?
  • “It tells worlds not stories”
  • “The creation of rhetoric within a world rather than through a story allows those interacting with the world to explore the rhetoric of that world rather than being forced down a prescribed path”
  • Diegetic prototypes that exist within the story world are part of everyday life and are real in that world
  • The world is the principle task of the designer when creating a design fiction
  • they propose a framework where the design fiction world is a distinct entity, with a distinct shape, but with a complex hidden inner structure
    • a series of “entry points” are visible
    • this is an artefact that contributes to making up the design fiction and can be extrapolated from (see Figure 1)
  • “Powers of 10” inspired by the film of the same name
  • start with an entry point and increase the “visible area”
  • world building applied to design fiction moves the focus away from storytelling and instead it is placed on the cohesion of the world – how things and people interact
  • Design fictions are “collections of artefacts, that, when viewed together build a fictional world”
  • “The artificially built world is a prototyping platform for the very designs that define it, meanwhile these designs reciprocate in kind and prototype the world”

Dunne, Anthony, and Fiona Raby. Speculative Everything: Design, Fiction, and Social Dreaming. MIT Press, 2013. pp 69 -100

  • Chapter 5 – A Methodological Playground: Fictional Worlds and Thought Experiments
    • Shift the focus from how the world is now to how the world could be
    • Lubomir Dolezel: speculative design is “experimental laboratory of the world-constructing enterprise”
    • Speculation = imagining other worlds and alternatives
    • tools for reflection, critique, provocation, inspiration
    • build a whole world rather than traditional starting points
    • actual vs fictional is key distinction
    • literature & fine art offer the best sources of inspiration
    • a fictional world can be impossible; a possible world needs to be plausible
    • examples of non-fictional worlds in fashion, fine art Crewmaster Cycle and The Fantasy Collection
    • Game design is the area where fictional worlds is most developed
    • Utopia: literary utopia, utopian practice, utopian social theory
    • Nazism, Fascism, Stalinism are said to be the fruits of utopian thinking <- attempts to realize utopias from the top-down
    • utopia is “far more interesting when used as a stimulus to keep idealism alive” “something to aim for rather than build”
    • cognitive estrangement, a development of Brecht’s Alienation Effect, describes how alternate realities can aid critique of our world through contrast
    • Limitation of cinema: can deliver a powerful story and immersive experience but “requires a degree of passivity in the viewer reinforced by easily recognized and understood visual cues”
    • As designers we are between literature and cinema
    • start with designs that the viewer can use to imagine the society/world that would have produced them
    • China Mieville’s The City and the City is poli-science fiction, everything is familiar; the reconceptualizing of the common concept “the border” is the crux of the fictional world (and “makes it relevant to design”)
    • Reducto Ad Absurdum thought experiment: take one argument to the extreme
    • Counterfactual thought experiment: change a historical fact and consider what might have happened
    • What-ifs thought experiment: what might happen in a society of extreme circumstances
    • Speculative design is fictional, disconnected from industry and market
    • Speculation & imagination is a part of everyday design
    • The purpose of speculation is to “unsettle the present rather than predict the future”
  • Chapter 6 – Physical Fictions: Invitations of Make-Believe
    • Fictional objects exist outside of art and design; such as patents and failed inventions. Fictional objects but accidental fictions
    • Film props and design speculations have different requirements; film props must be legible and serve a greater purpose & use a visual language
    • functions in design speculations can be wider and more varied
    • Kendall L Walton: props are objects that “prescribe imaginings” and “generate fictional truths”
    • Being “caught up in a story” means “to participate psychologically in a game in which the story is a prop”
    • Speculative design props do not carry a behavioral schema with predetermined roles to fill
    • A Prop’s purpose is to “facilitate imagining”
    • Props are synecdoches of the fictional world, designed to elicit a response to the world the prop belongs to
    • Suspension of disbelief = asking people to participate in make-believe, not belief
    • belief leads to fakery, trickery, hoaxes
    • for Dunne & Raby “fooling the viewer into believing something is real is cheating”
    • A speculative object does not have to be realistic, just plausible
    • Objects must appear real but subtly signal that they are not
    • Differences between design fiction and speculative design, including that design fiction is intended mostly for video & internet

Von Stackelberg, Peter, and Alex McDowell. “What in the World? Storyworlds, Science Fiction, and Futures Studies.” Journal of Futures Studies, 20 Dec. 2015, pp. 25–46.

  • Universes provide rulesets that develop larger realities
  • Theories have found their way into foresight methodologies
  • Foresight professionals should understand storyworlds
  • Also read: Floyd (2008), Fuller and Loogma (2009), Inayatullah (2010), Slaughter (2011) for analyses of foresight methods
  • Causal Layered Analysis (CLA): delve into subjectivity, interpretation, and cultural context. Multiple levels to make sense of reality and the future
  • Social Constructionism: Whenever we employ words or symbols to refer to objects we are constructing them. Re: Future Studies, each member of a group contributes to the development of a world using currently available information.
  • Integral Theory: Subjective Perspective (individuals interior world), Objective Perspective (individual’s exterior world), Interobjective Perspective (collective exterior world or physical world), Intersubjective Perspective (collective interior world or shared meaning of groups as expressed in culture)
  • Critical Futures Studies (CFS) and Integral Futures (IF): examines “social interiors” (language, worldviews, paradigms, values) of the future (equivalent to Intersubjective Perspective of Integral Theory)
  • Sense-Making: Social activity where plausible stories are shared, retained, and preserved. Vital to the processing of information.
  • Storytelling exists to help us make sense of the world around us
  • Narratives support strategic decision making and critical reflection, help comprehend uncertainties
  • Futures narrative creative process by Schultz, Crews, and Lum (2012)
    • 1) Create a participatory, integrated futures process that digs into organizational cultural assumptions and blind spots
    • 2) produce scenarios inductively by interconnecting impacts of multiple variables to mimic more closely the turbulence of real-world change
    • 3) engage participants in creating their own richly detailed, vivid, dramatic stories about possible futures
  • Design fiction prototypes possible future outcomes of contemporary life
  • science fiction and future studies can be confused and “each side worries about being confused with one another”
  • Emotional processes can affect attention and information processing, which science fiction provides
    • “the future is felt as well as imagined and considered”
  • Literary metaphors of future-oriented science fiction (Levin, 2010)
    • Cautionary Tales
    • Thought Experiments
    • Literalized Metaphors
    • Explorations of new science and technology
  • A formal framework for science fiction prototyping (Graham, Greenhill, Dymski, Coles, & Hennelly 2015)
    • narratives based on facts as a design tool in the development of a technology
  • Storyworld: “The place and time in which a narrative happens”
    • or Chronotope (“time-space”)
  • This ability to immerse participants in a persistent coherent storyworld is one of the key strengths of the worldbuilding process and is one that holds promise for futures oriented projects”
  • Future Reality as opposed to science fiction in Minority Report
  • Alex McDowell’s Mandala as a world building process framework
    • First stage: “What If And Why Not”mandala
  • organic evolutionary collaborative process centred in storytelling allowed the emergence of the holistic fictional world (called “precognitive”) of Minority Report
  • Foresight professionals should “set the future in a story” so the audience is better able to experience it
  • Peter von Stackelberg Alfred State College of Technology (SUNY) USA E-mail:
  • Alex McDowellSchool of Cinematic Arts Media Arts + Practice University of Southern California USA E-mail:
May 07

Process Journal 1

Posted in Process Journal           No Comments »

Wed May 8
Early Ideation

As an early concept I have envisioned a combination interactive installation/storyworld artefact immersive experience.

Expanding somewhat on the Synth concept from the Transmedia Storytelling class, this concept revolves around a digital assistant search engine called Cassi. Cassi is realized as an animated face projection mapped onto a bust in a gallery setting.


Participants enter and ask her questions or converse with her before receiving their conversation summarized in a printed receipt before moving on.

img_20190506_132005The second part of the experience takes place “behind the curtain” in an adjoining room themed as the control centre. Players move through this office space and can look through computers and documents for information on Cassi’s construction and programming. They can enter information from their receipt to read details from their own conversation.

The crux of the experience is that Cassi, the all-knowing search engine, is manipulating the responses and saving the details of the conversations for the benefit of the organization that created her.

img_20190506_132018Practically I envision the Cassi interaction as being a chatbot utilizing text to speech and Wolfram Alpha API connection.img_20190506_132021The “behind the curtain” room I envision as being a work of set-dec and worldmaking, along with a means of storing and playing back key data from each interaction – perhaps logged as a number referenced on each participant’s receipt.

May 06

Annotated Bibliography

Comment, Bernard. The Panorama. Reaktion Books, 1999. pp. 1-27

  • Crowd-pleasing, lowbrow entertainment – a “motley crowd in search of wanton, enigmatic  and rarely denied pleasure”
  • Patented by Robert Barker 1787
  • means “to see all”
  • “so true to life that they could be confused with reality”
  • Viewers walk through a dark corridor and staircase before entering
  • paradox: enclosed area representing a space “free of all worldly restrictions”
  • Context: industrial revolution + first metropolises. People wanted to control & escape from collective space + mass culture
  • First subjects were the towns in which they were exhibited
  • Panorama became a propaganda machine, showing battles & historical events
  • Travel to distant lands & historic sites
  • Painted in conjunction with camera obscura and early photography to ensure accuracy
  • very heavy painted canvas, vulnerable to fluctuations in temperature
  • Multiple painters required which caused workflow + quality problems
  • No panoramas after 1861

Holmes, Oliver Wendell. “The Stereoscope and the Stereograph.” The Atlantic, June 1859, pp. 1–8. Accessed from

  • A comparison to “film” as an organic material – say, in the eye of an animal – is made to “film” as it applies to photography
  • Written in response to the daguerreotype and photograph (Daguerreotypes made publically available in 1839, photographs made it obsolete in the 1860s)
  • modern inventions are given a mythical import, compared to the magical artefacts of an Arabian Nights story
  • A layperson’s explanation of how daguerreotype and photograph are created
  • Stereoscope: invented by Wheatstone in 1838, contemporaneous with Daguerre’s invention
  • “A stereoscope is an instrument which makes surfaces look solid. …by this instrument the effect is so heightened as to produce an appearance of reality which cheats the senses with its seeming truth”
  • Description of the laws of perspective in such detail that it’s almost laughable – to us “things look flat with one eye closed” is something we learn as little children
  • “we see something with the second eye which we did not see with the first… the two eyes see different pictures of the same thing”
  • the brain works with the eyes to produce an image. If they are not in concert (damaged, moved, drunk) the image will be distorted
  • An artist could not make a stereoscope, but a photograph can
  • Simply take a picture, move the camera “the distance between the human eyes” and take another – or “better yet” take two pictures in a double camera
  • How do we “slide them together?” squint, or build glasses that “squint for us”
  • “a frightful amount of detail” upon looking through a stereoscope (device) at a stereograph (dual-image print)
  • Holmes argues that we can see more details and inferences in a stereoscope, as there is a suggestion of motion and intention in the difference between the two images
  • Holmes offers suggestions of how to best consume stereographs, with tips like “beware of investing largely in groups” and avoid high-detail images of people as they may be flawed and are of less interest than architectural views
  • “Form is henceforth divorced from matter” <- very resonant in the digital world, especially with a mind to VR

Prince, Stephen. “Through the Looking Glass: Philosophical Toys and Digital Visual Effects.” Projections, vol. 4, no. 2, 2010, pp. 19–40., doi:10.3167/proj.2010.040203.

  • Argues that the meeting of art and science have been and continue to be necessary for advancement of cinema.
  • cinema-centric but details the Camera Obscura and other pre-cinema visual spectacles.
  • intense collaboration with scientists allows for the development of convincing special effects in film
  • “Perceptual Realism” is the replication via digital means of contextual clues designating a 3d world
  • It is a major goal of VFX artists
  • Visual effects = spectacle. Spectacle is anti-narrative
  • vision is culturally coded and therefore relative to social formations in a given period
  • Early films emphasized spectacular views – attractions – rather than stories
  • Function ruled cinema until 1907 when narrative became dominant
  • Techniques and instruments originating in cinema are now used by scientists
  • Camera Obscura – known in the 11th century to Ibn al-Haytham
  • This + the zoetrope/phenakistoscope/thaumatrope were “philosophical toys” – both scientific investigation and popular amusement
  • stereoscope – first exploration into binocular vision
  • pseudoscope – played with the stereoscope to induce visions of impossible geometry – inspired computer scientists who would simulate the phenomenal world