Fundamentals of Immersion

Jul 09

Pohjola, Mike (2004): “Autonomous Identities: Immersion as a Tool for Exploring, Empowering and Emancipating Identities” in Montola & Stenros (ed.)  Beyond Role and Play: tools, toys and theory for harnessing imagination

  • New key words: immediated/immediacy/immediate art; diegetic frame; inter-immersion; Temporary Autonomous Zone & Temporary Autonomous Identity”
  • Inter-immersion will be a key concept for developing the community space inherent in immersive interactions
  • New definition of immersion: “being in character or becoming the character”
  • “Immersion is the player assuming the identity of the character by pretending to believe her identity only consists of the diegetic roles” 
  • An examination of the nordic-style school of LARPing. I’m less interested in the roleplay aspect and more interested in the notion that immersion can serve some purpose in and of itself. This paper covers the exploration of personal identity through immersive techniques, the factors that make roleplaying/immersion a unique art form, and the potential for developing spaces of temporary reality that allow for us to take on temporary identities
  • Name checks the Turku school of immersion that is trying to achieve “character immersion”
    • considered impossible by the author – “based on faulty premise of character that originates with traditional fiction and cannot be applied to immersive, immediated artforms like roleplaying”
  • “This essay is written partly as an attempt to update and post-modernise the ideas of the Manifesto of the Turku School (Pohjola 1999), specifically those concerning character and immersion.”
  • “the internal processes and interpretations of the player are for the game as a whole until they are expressed and become part of the diegetic frame. Before that they are merely “individual narrative readings”
  • Stuart Hall (1996) beleives that seeing self as narrative is the essential part of identity creation
    • Turku manifesto saw this the opposite way: character identiy can be created by seeing the narrative as the self
  • J Thomas Harvianinen: Three kinds of immersion: Character Immersion, Reality Immersion, Narrative Immersion
  •  Impossible to roleplay alone. Roleplaying should include interactivity
    • It follows that immersion requires interactivity
  • Roleplaying games are experienced as they are created – a unique feature of them
    • could also apply to immersivity
  • media is generally divided into three loose categories: passive, active and interactive
    • passive are recorded and cannot be affected (like film, literature, recorded music)
    • active has possibility of interaction, such as karaoke
    • interactive requires participation, such as computer game, hypertext
      • fourth “transcendent” category is immediate art, “experienced as it is created and has no use for the division between performers and audience”. Examples include roleplaying games burt also parties, communal storytelling or improvised musical jams
    • “An outside audience cannot understand a role-playing game, although it can seem like an interesting performance. Role-playing games take place in the present moment and are transmitted directly from person to person. This makes them immediate”
  • “immersion without action is daydreaming”
  • roleplaying is not necessarily an interactive act
  • the character must exist within a diegetic frame
  • Inter-immersion: the collective experience of immersion shared and strengthened through interaction
    • strengthens the identity of the character rather than the player
    • staying in character helps to stay in character
    • “as the player reaches the inter-immersive state she starts to forget she is just pretending to believe it is all real. She acts as if she really believes the diegesis, and when everybody else does the same and reacts to each other’s beliefs (instead of the pretensions), they forget they are just pretending and start to really believe.”
  • “There is a pattern, and a very clear one when you know where to look. Each new generation of games is less abstract. Where Go is about capturing and re-capturing land, Chess is about a war between two nations, Chainmail is about commanding armies in battle and Dungeons & Dragons is about directing a singular adventurer in a dungeon, modern role-playing games are about acting as any individuals in any setting”
  • Perceiving a game as having a reality is difficult when the game is abstracted (like Chess)
    • as games get more complex and less abstract gaming reality is stronger & more fulfilling.
    • “The next logical step is to lose the barrier separating games and reality once and for all.”
  • Pretend belief moves toward real belief and subjective diegesis becomes subjective reality, temporarily. In RPGs the diegesis is temporarily the player’s identity.
  • Temporary Autonomous Zones can be created to invite players to take on a Temporary Autonomous Identity (TAI).
    • If we can constantly carry a TAI with us, we can always be in a roleplay.

Case Studies:

Masquerade by Kit Williams

Masquerade | The Hankerer

  • Masquerade was a 1979 picture book that contained clues to a real-world treasure hunt, the prize to which was a golden jewelled hare figurine. The puzzle was first solved and the hare claimed by a person who, arguably, cheated.

Mystery on Fifth Avenue by Eric Clough

  • Clough was a young architect who, unbeknownst to the owners, hid a series of puzzles inside the architecture of an apartment he renovated. His intention was to supply the children growing up in the space with an ongoing sense of wonder.
Jul 03

Process Journal 8

Posted in Process Journal           No Comments »

This week I have been focusing more on experiments with incorporating DepthKit into Vuforia. I used a Kinect for Windows (graciously provided by my supervisor) to record some test footage using Depthkit, and experimented with post-processing and green screen to get a sharper image. (I will have to reshoot the test footage with a better green screen setup next time, as my rather slapdash studio setup had too many wrinkles for a clear mask.) I was able to learn how to implement a green screen as well as play with Depthkit’s settings to reduce spikes.

My first new interaction is a hand reaching out of a surface, which is one of the first images I envisioned when I first considered using volumetric capture.

docudepth2Test footage with a cylindrical trigger:docudepth1I’ve also begun playing with Spark AR by Facebook. Facebook’s usage policy seems highly constraining, as it can only generate content for Facebook platforms, and thus may not be all that useful for an artistic practice. Still, it couldn’t hurt to become familiar with the tool.barf-docu Other experiments still to come:

  • Depthkit AR interaction using an object as trigger, incorporating the object in the film.
  • Learning how to maximize the efficiency of the available processing power, as experiments suggests the Android phone has trouble rendering multiple volumetric videos simultaneously.
  • Volumetric videos generated on a surface and in midair.
Jul 02

Theme Parks 2/2

Posted in Contextual Analysis           No Comments »

I visited the Wizarding World of Harry Potter (WWHP) at Universal Studios Orlando in 2012, several years after opening but before the Hogsmeade expansion. I was familiar with the Harry Potter storyworld and accompanied by an avowed super-fan I found the place captivating. Since the two of us were familiar with the Harry Potter world we found no impediment to understanding or enjoyment of the park – it was designed for us.

There is no meaningful onboarding for non-fans, and unlike some of the other rides at Universal Studios that remediate the plot of the source material, there is no retreading of known plot lines. WWHP stands alone as a transmediality, existing as a distinct artefact separate from any of the films or books (although it is explicitly based on the films, as it uses actors from the films and is designed after the look of the films). Interestingly, it seems to exist at a specific point in the Harry Potter timeline, being set before the events of the last few books, as evidenced by the presence and age of the young stars of the films.

The sense, then, is of inhabiting the Harry Potter franchise at a pivotal time in the plot – before the climax and denouement of the final books but after most of the world is established by the first few books. This is a distinction that would not be noticed or appreciated by those who are not familiar with the story world.

Embodiment in the space translates to embodiment within the narrative. It is those for whom the cult geography resonates that are the core audience of WWHP. These people become embodied within the plot by their presence at the park, and their presence at the park generates the sense of communitas that brings the park to life.

Jun 28

Waysdorf, Abby, and Stijn Reijnders. “Immersion, Authenticity and the Theme Park as Social Space: Experiencing the Wizarding World of Harry Potter.” International Journal of Cultural Studies, vol. 21, no. 2, 2016, pp. 173–188., doi:10.1177/1367877916674751

  • The functioning of a park like Wizarding World of Harry Potter (WWOHP) is reliant on sensory detail as well as the inherited communitas of fans who feel like the space was prepared for them. It invites embodiment within the fiction and encourages engagement with the existing communitas of the HP fandom. It is perhaps designed to be the ultimate expression of fandom – the place that is as close as possible to habitation inside the fiction, designed to be experienced in a way as honest as possible to the way it would be engaged with were it “real” – as a place where you can not stay forever, but merely visit as a tourist.
  • New key terms: Habitus, Ironic Imagination
  • Returning key terms: simulation, communitas
  • a new definition of immersion: immersive because it feels inhabitable – as detailed as the real world and shared with others as an imaginary habitus
  • emblematic of a trend in the industry toward environments promising immersion into a favourite text
  • Limited understanding of why theme parks appeal to people, as most of the theory has focused on form and simulation
  • Paper is a visitor-centered work
  • small research base – only 15 interviews
  • Ironic Imagination (Saler)
  • Visitors can explore the storyworld in an embodied manner
  • Eco: “Disneyland tells us that technology can give us more reality than nature can
    • In choosing Disney over actual locations “the tourist has preferred the simulation to the reality” (Sorkin)
  • Success of the theme park points to the postmodern preference for simulation, safety and entertainment over the “real” experience of landscapes and environments
    • “a special place full of communitas not found anywhere in the mundane, consumerist habitus” (Aden)
  • Cult Geography: “fan attachment to non-commodified space, or… to space/place which has been indirectly or unintentionally constructed” (Hills)
    • eg tour of X-Files filming locations in Vancouver – constructed through fans meaning-making process rather than the media or tourist industry
  • Most critiques of theme parks is that they are inauthentic
  • Lukas: a themed environment becomes authentic when it “is sensory available”
    • That is, those visiting know it is a simulation but it becomes authentic when it feels correct on all sensory levels
  • Fantasies, mythologies and cultural icons can be enacted and played with
  • The visitor is encouraged the engage with the fictionality of the theme, and to imagine on a body level
  • This is the same kind of performance asked for and required for Total Immersion as suggested by Machon
  • Immersion, for this article: “The feeling, through the medium, that the audience member is part of the artistic or narrative world”
  • Saler: Virtual worlds: “acknowledged imaginary spaces that are communally inhabited for prolonged periods of time by rational individuals”
    • A storyworld becomes virtual when it is adopted and discussed by many individuals who group together in order to fill its details and make it more real
  • Becomes immersive because it feels inhabitable – as detailed as the real world and shared with others as an imaginary habitus
  • Embodiment (Crouch) : “A process of experiencing, making sense, knowing through practice as a sensual human subject in the world”
  • Ironic Imagination: A “double-consciousness” that allows the subject to be emotionally invested in and contemplative about a fictional world while maintaining knowledge that it is fictional.
  • There is no more “real” version so the simulation suffices as the closest thing to actual experience
  • diegetic expansions: products that are, or could be, from the series
    • “Like you are vacationing in Diagon Alley”
  • In WWOHP there is a sense of communitas inherent in arriving with a shared experience and camaraderie coming from Harry Potter. For example, recognizing another park-goer as being from the same HP House as yourself and using that to strike up a conversation

“‘Like Walking into a Movie’: Intermedial Relations between Disney Theme Parks and Movies.” The Journal of Popular Culture 50.4 (2017): 704-22. Print.

  • Provides some insights into the cinematographic inspiration behind theme park and ride design. More importantly the article provides some language and taxonomic tools for discussing elements and techniques borrowed from other media.
  • The organization of space in Disneyland follows cinematic principles rather than architectural ones
  • Wolf and Rajewsky’s typology of intermediality.
    • distinguishes between phenomena of transmediality and intermediality
    • each of their categories can be illustrated with an example of Disney parks “borrowing” from movies
  • parks are distinct from museums in that they create self-contained worlds that are separated from the rest of the world, fusing media that have been historically and culturally viewed as distinct
    • Can thus be classified as a “hybrid” medium, “composite” medium, or “meta-medium”
  • transmediality refers to elements that are nonspecific to individual media and whose origin is unimportant. Intermedial transposition is concerned with specific artefacts that have been “translated” from one medium to another
  • Intended to provide critics with a set of conceptual and terminological tools to describe intermedial phenomena
  • In many Disney rides vehicles frame the visitors’ view like a movie camera.
    • The ride vehicles, writes Lainsbury,were designed to function like movie cameras by twisting, turning, and directing the gaze of passengers from scene to scene. Their high backs and sight-restricting sides, not to mention the metal lap bars that held guests firmly in their seats, guided the experience further by erasing from view anything that might spoil the illusion. 
    • a Transmedial phenomenon as it is also employed in photography and painting, for example
  • Remediation is the retelling of a story in a new medium – in theme parks the line between remediation and intermedial transportation can be blurred
  • “Magic Wand” principle :a corridor links the entrance to a central square, from which different themed areas radiate out like the spokes of a wheel
Jun 20

Process Journal 7

Posted in Process Journal           No Comments »

This week I have been continuing my AR explorations. My most noteworthy development is the incorporation of volumetric video into AR, as seen above.


Situating the volumetric video clip is as simple as nesting it as a GameObject child of the intended ImageTarget. Scaling and placing it is slightly more complicated as the GameObject seems to exist at a different origin than the video itself.

More experimentation is required. I have done some experiments with regular video, and virtual buttons – that is, areas on trigger images that can be occluded to cause events to trigger in Unity.

img_20190620_155103_452 screenshot_20190620-154017 screenshot_20190620-153959

The app builds and runs, but it is very slow to generate a video once the button is pressed. I suspect that the code is turning each video off and on every time a new one is called. I experimented with having all three videos running offscreen, as well as switching the renderers on and off, but succeeded only in crashing unity. Still, it’s a promising proof of concept.

Jun 20

Contextual Analysis

Where Dark Things Dwell poster

Where Dark Things Dwell at Black Creek Pioneer Village is a unique escape game experience at Toronto’s Black Creek Pioneer Village. It is intended a showcase of the historic buildings at Black Creek, using the escape game format as a means of bringing players who might not otherwise have come to the village on a tour of the site.

The game uses environmental storytelling to explore the true history of the site as well as the fictionalized story of a curse trapping the players in the village. Each building has a set of puzzles themed to the building, which can have Nicholson’s “Ask Why” paradigm applied to them to explore their placement in the village. The mill has a puzzle involving sacks of flour, for example, while the building containing drying racks of herbs contains puzzles involving potions.

It is interesting to note that the puzzles forms are often unrelated to their content – the flour sack puzzle, for example, could be easily replicated with paper or wooden tiles. The potion puzzle is simply a series of riddles. While thematically linked to their sites (or by content – for example, the potion riddle puzzles are themed after spells) there is rarely a physical necessity or affordance that causes a puzzle to be situated in a physical form or space.

Each space’s puzzles are contextualized by the space they occupy, bringing narrative synchronicity to what would otherwise be unrelated puzzles. By couching puzzles in the trappings of their space narrative coherence is strengthened, even when there is no deeper relationship.

Jun 20

Nicholson, S. (2012, June). A User-Centered Theoretical Framework for Meaningful Gamification. Paper Presented at Games+Learning+Society 8.0, Madison, WI.

  • My hunch is that immersion need be largely self-directed (or internally motivated). The Organismic Integration Theory may be useful for helping me define my taxonomy
  • Deterding on gamification: “use of game design elements in non-game contexts”
  • can remove the internal motivation a user has for an activity, replaced with external motivation
  • suggests “pointsificiation” for adding scoring to a non-game activity
    • focuses on  goals without play
  • suggests “exploitationware” as a better description of what is going on
  • almost all forms of rewards reduce internal motivation
  • once you start giving someone a reward “you have to keep her in that reward loop forever”
  • the goal of the paper is to explore theories in user-centered gamification that is meaningful to the user and therefore does not depend upon external rewards
  • Organismic Integration Theory: a continuum of motivations based upon how much external control is integrated along with the desire to perform the activity
    • an activity is more likely to be integrated if it is meaningful to the user. If too many external controls re integrated the user can have negative feelings about engaging in the activity
  • In order to be meaningful the activity must be relevant to the user
  • by involving the user in the creation and customization of the gamification system the user can select or create meaningful game elements and goals
  • Situated Motivational Affordance: a user is motivated only when there is a match between htat aspect and the backgorund ofthe user. example: if an element of gamification in a company is tied to financial reward, the perception of this as a controlling activity is greater than if the element is tied to a badge
  • Universal Design for Learning: courses should be designed so students can demonstrate learning in a number of ways.
  • Gamification needs to allow different ways for users to achieve goals so that users can be involved in the ways most meaningful to them
  • external rewards are not user-centred

Nicholson, Scott. (2016). Ask Why: Creating a Better Player Experience Through Environmental Storytelling and Consistency in Escape Room Design. Paper presented at Meaningful Play 2016, Lansing, Michigan.

  • Nicholson is primarily concerned with narrative consistency within escape games and the associated player experience. His “Ask Why” paradigm is valuable for designers of narrative-driven experiences.
  • embed the challenges in the game along with the narrative into the environment using the concept of Narrative Architecture
  • consistent genre, setting, world, challenges make more engaged player experience
  • player is their own avatar and are thus more sensitive to inconsistencies
  • cognitive dissonance between who they are supposed to be in the game and what they are doing in it
  • game designers should start with the experience they are trying to create, then every design decision should move players closer to that experience.
    • question a decision which does not move players toward the core experience
  • Narrative Architecture (Henry Jenkins, from the theme park industry) gmae mechanisms are embedded in a larger world. players explore that world as they explore game mechanisms, the story is woven into the space the player is exploring. they are also exploring the story as they explore the mechanisms.
  • Ask Why: ask “why is this here” when looking at every element of a game space.
  • Lee Sheldon: audiences want 3 things from storytellers:
    • Take me to a place I’ve never been
    • Make me into something I could never be
    • Let me do things I could never do
  • players should have a meaningful reason for taking on a task other than “its in the room”
  • Meaningful Play: for play to be meaningful, the actions a player take have to be discernible, meaning the player understands the result of what they are doing, and integrated, meaning the actions the player takes makes a difference.
  • Meaningful challenges:
    • engage with an element of the narrative that has been presented to the player
    • interact with the world in which the player exists
    • have a direct impact on the player or other characters
  • When the world is incoherent, and immersive elements are not put into place, players can’t psychologically step into the world and imagine navigating it
  • a good puzzle is about creating frustration for the player
    • the player experiences some frustration but also provides checkpoints so the player knows they are on the right path
    • they have a clear solution
    • a puzzle with no choices is a task
    • a puzzle must balance the requirement for effort and inspiration to solve
    • Escape rooms can be categorized by technology
      • Gen 1: mostly mechanical and require human engagement and human power
      • Gen 2: more electronic sensors, mag locks, remote controls, but still human triggered
      • Gen 3: technology and computer control is integrated so the room can respond to the actions of the players without human involvement
      • Gen 4: automated clue systems, control the flow of players, change the game space based on the players needs
  • “Round Up To Fun”
    • sometimes something needs to be adjusted away from realism to make an enjoyable player experience
Jun 12

Process Journal 6

Posted in Process Journal           No Comments »

This week I have been exploring AR through Vuforia and Unity with the help of Udemy course Augmented reality (AR) tutorials on Markerless tracking, Cloud AR, 3D Object detection, + more with Unity & Vuforia (instructor Dinesh Tavasalkar).

Through a series of guided experiments I have made a sort of living sketchbook, each page of which has a different set of interactions and code.

After finishing the course (I’m only half done) I would like to explore volumetric video capture in AR.

screenshot_20190611-175204 img_20190611_175956_032 screenshot_20190611-175458 screenshot_20190611-175421 screenshot_20190611-175533 screenshot_20190611-175510 screenshot_20190611-175700

Jun 12

ARGs 1/2

Posted in Annotated Bibliography           No Comments »
Gouveia, P. (2009). Narrative paradox and the design of alternate reality games (ARGs) and blogsIEEE Consumer Electronics Society’s Games Innovation Conference 2009 (ICE – GIC 09Proceedings. Imperial College, South Kensington, London, pp. 231-38. In
  • An exploration of games that argues that the simultaneity of narrative, combined with the growing desire of players to perform as actors in other narratives, is expanding the potential for game development. This paper is a little unfocused for my purposes but it provides links to other thinkers, such as Eskelinen and Salen & Zimmerman, whose work might be useful for my taxonomy of immersion.
  • “pervasive games”, “playable fiction”
  • Mourao: “hyperfiction”
  • Games are non-linear but each players’ actions imply linearity w/ a beginning, middle, and end
  • Torben Grodal: videogames are stories for the eyes, ears, and muscles
  • the protagonist is simultaneously and the agent of character actions
  • “it is suggested that the trick for developing a working base for interactive drama is to integrate the phenomenological aspect of first-person experiences with structural aspects from well designed, third-person stories”
  • Player has four functions (Eskelinen): interpretative, exploratory, configurative, textonic
  • the human-machine interface allows players to create their own personal experiences where they perform as actors
    • this relationship is performative
  • “the pleasures of becoming actors” is being discovered by players
  • “as game-actors they become masters of interpretive embodiment; they accept as their mission the real world incarnation of a digital design”
  • Salen & Zimmerman: every game is a simulation and a cybernetic system of control
  • implosive stories: where things happen simultaneously

Rose, Frank. The Art of Immersion: How the Digital Generation Is Remaking Hollywood, Madison Avenue, and the Way We Tell Stories. W.W. Norton & Company, 2012. 

Introduction through Chapter 2

  • An effective history of the origin of the ARG. Highlights some problems the ARG faces but doesn’t engage with my main question: how do you get players if you don’t have a huge budget or a valuable property to attach it to?
  • The internet is a chameleon – it can act like all media
  • it is inherently participatory
  • immersive: “you can drill down as deeply as you like about anything you care to”
  • “Deep media” eg Lost, Dr Who, Avatar – media that asks more engagement than its runtime
  • Why So Serious ARG implicated the players as accomplices with the Joker’s heist that starts the Dark Knight film.
  • nonlinear, collaborative
  • Jordan Weisman, credited with inventing the format
    • “on the net we sift through information we don’t care about like an archaeologist sifting through dirt”
  • Game The Beast for AI: Artificial Intelligence in 2001 was the first
  • unlimited players have unlimited resources, unlimited time, unlimited money – they solved everything in a day
  • attempt at subscription ARG called Majestic
  • Otaku: passion, obsession, yearning to immerse oneself in stories. the desire to experience the universe through as many different media as possible. A need to expand that universe by telling new stories within it.
Jun 05

Process Journal 5

Posted in Process Journal           No Comments »

Peggy showed me some of her primary sources in her research. She has an old folio of translated Greek writings – apparently it’s the only copy that exists. She showed me some of the translations and her notes on them, as well as the notes made by the fellow who originally made the translations.

I’ll be uploading Peggy’s annotations to the translations to her website. In the meantime, here is the abstract to her dissertation.

The first part reverse-engineers the theories of Therodius (c.250 BCE) who was the first to posit the literal existence of a realm of fiction. Therodius, who was primarily concerned with the Grecian theatre of his day, wrote that the act of Conjuration (what we today would call sub-creation, or simply writing fiction) was impossible, as human beings lacked the perspective to imagine anything that did not already exist. He pointed to the fantastical stories of the Grecian mythology, arguing that such outlandish tales widely accepted as gospel suggested a common origin. His last surviving writings imagine a literal realm of the gods where Conjuration of new stories occurs and are transmitted “askance” (λοξώς) to human minds. He further suggests that the Conjured material can “leak” (διαρροή) should there be no minds open to receive them, and points to such phenomena as the Bacchian orgia as examples of this.

The second part explores contemporary communication modes through the lens of Therodius’ conjectures. It explores the semiotics of conversation, with an eye to what must remain unsaid in the name of clear communication. It examines the calculus of communication, exploring the range of information we are able to draw from when making decisions, how our decisions are rendered into reality, and how the discarded information continues to exert an influence on us, unbeknownst.

The third part explores known methods for transcending the brain and achieving a higher mode of consciousness, including hypnotism, psychotropic medication, and sensory deprivation. It also explores and summarizes the results of practical research.

The fourth part summarizes the core taxonomy of the research. Our perception is like a bubble, formed of the decisions we made using the information available to us, immersed in an endless ocean of discarded or incorrect information. Our “non-fictional” world exists in contrast and opposition to the “fiction”. Our world is finite while the fiction is endless. Reality, non-fiction, truth, it all must define itself against what it is not. Therefore, that which does not exist must exist in some form, somewhere.