Fundamentals of Immersion

Aug 07

The process of researching immersive design has been crucial in defining exactly what elements are key to the establishment of the art form. It has allowed me to sift through definitions and clarify what I find most exciting about immersive experiences, and why there is confusion between disciplines that claim ownership of immersion.

In brief: I propose using the term “immersion” sparingly in discourse on immersive design. What most people think of as “immersion” is better thought of as “engrossment/absorption” –  or simply giving full attention to what is at hand. True immersion can be described with the term “embodied praesence” which is shorthand for the sublime experience of transportation from the real world where real-world rules apply to an alternate world where different rules apply, while simultaneously choosing to believe, for the duration of the experience, that this is the way the world has always been.

I have submitted a paper as a summation of my research, which can be accessed at the link below:

Link to Summative Paper

Further research should be conducted into the actual psychological mechanism of embodied praesence: if everyone is capable of experiencing it, how it can be reliably generated, and which forms of media best support it.

Jul 23

Process Journal 9

Posted in Process Journal           No Comments »

I have decided to put a freeze on the development of an immersive experience as the summative portion of this study.

This study began with the intention of exploring practices from every corner of the immersive design discipline in order to highlight effective design methods and, eventually, synthesize them into a singular “one-size-fits-all” practice-based approach. When that was the intention of the study, it made sense for the final evaluation to be a demonstration of that practice.

As the study progressed it became clear that the discourse that I found most engaging and useful was the theoretical/conceptual writings on the nature of immersion as it is understood by the various quadrants of design practitioners who utilize it.  The conceptual research was highly useful in pinpointing and the phychosociological phenomenon at the core of what I call immersion that I am keen to explore. Through this process it became clear to me that before I could begin to codify a design practice I needed to conduct a thorough context review and lay out a set of definitions – the myriad disciplines developing immersive experiences share many commonalities in terms of the mechanisms they utilize, but lack a common terminology or even definition.

It is more useful to me, then, for the summation of this study to be, rather than a piece of experiential design, a comprehensive contextual analysis laying out my understanding of the terms, phenomena, and mechanisms at play in immersion. The explorations in practical design undertaken as part of this study are being put to use in the CFC Prototyping assignment.

Jul 23

Grau, Oliver. Virtual Art: from Illusion to Immersion. MIT Press, 2007.

  • Grau warns that the diminishing interface has the potential to remove critical engagement. Fully embodied immersion may cause problems we have not yet encountered or even considered. Grau’s warning is central to my interest in immersion. Immersion’s potential for misuse and manipulation is, to me, related to the trend of social media manipulation and “fake news” and the disintegration of the interface. We are being asked to undergo immersion frequently – social media platforms, it can be argued, have the goal of immersing the user so they become engaged in a virtual space, and the interface is miniscule. Users of social media are immersed to a degree, and Grau’s warning and rationale are relevant.
  • New Keywords: Natural Interface,
  • The observer – the “immersant” (Char Davies)
  • Osmose controlled by a vest full of sensors
    • float up when lungs are full etc
  • sfumato deceives the eye, a mechanism of suggestion present in all media of illusion
  • Osmose presents a Natural Interface (an ideology) (201)
  • As interface dissolves and become more natural and intuitive, the more psychological detachment, the more distance from the work vanishes (202)
  • Without distance a work cannot be perceived as an autonomous aesthetic object
  • distancing is a prerequisite for critical reflection
  • ‘‘All happiness is immersion in flesh and cancels
    the history of the subject. All consciousness is emancipation from the flesh to which nature subjects us.’’ (Hartmut Boehme)
  • the more “natural” the interface the greater the danger – the “technological iceberg” will be inaccessible to the user as they are unaware of it.
  • “Against the backdrop of virtual reality’s illusionism, which targets all the senses for illusion, the dissolution of the interface is a political issue.”
  • The more powerful computer the more suggestive potential of virtuality
    • has a psychological and manipulative influence
  • “In conjunction with the attempt to generate the feeling of real presence, these impressions, which run counter to habitual perception according to nature’s laws, may result in problems of perception that should not be underestimated. The serious contradiction between corporeal reality and artificial image illusion is likely to be at a level that almost precludes rational access.”
  • “For Allan Kaprow, who gave the word ‘‘happening’’ its new meaning in the late 1950s, art is an ever-changing ‘‘work in progress,’’ a narrative created by audience participation.”
  • “such works are defined increasingly in terms of their processual nature, which stresses their unfinished or open quality and  locates art within a framework of communicative social relations.”

Soloski, Alexis. “The Problem with Immersive Theatre: Why Actors Need Extra Protection from Sexual Assault.” The Guardian, 12 Feb. 2018, www.theguardian.com/stage/2018/feb/12/immersive-theatre-punchdrunk-sleep-no-more.

  • Performers and participant safety is at risk in immersive experiences, as participant behavior can’t be controlled or effectively predicted. Corporate policies are not an effective deterrent against misconduct. Context and in-the-moment investiture (such as audience costume, crowd, or even the endistancing of the self via immersed transportation) have an effect on participants and may tend to encourage misconduct and unsafe practices. Technology, especially experimental or untested designs, may break or even harm users. Corporate culture may tend to discourage complaints from employees. Instituting best practices to ensure participant safety should be the highest priority when developing an experience, even if it is to the detriment of the production.
  • Actors are in danger in immersive performances
  • Companies often do not have effective policy to manage audience misconduct
  • No physical divider between performer and audience
    • no stage
  • in Sleep No More audiences are masked
    • The things an audience is invested with affect their behavior, and what they understand is appropriate.
  • crowd congestion means anonymity is high
Jul 18

de Souza e Silva, A. & Girlie C Delacruz. “Hybrid Reality Games Reframed: Potential Uses in Educational Contexts.” Games and Culture, vol. 1, no. 3, 2006, pp. 231–251., doi:10.1177/1555412006290443.

  • new keywords: hybrid reality games, hybrid space, street sociability, Sociocultural Learning Theory, Experiential Learning
  • The accessibility and affordances of the mobile phone present opportunities for experiential learning, which requires exposure to what is being studied. Simulated exposure is not as effective as actual exposure for the purposes of education
  • The paper proposes that HRGs are uniquely engaging and fun, fostering skills by making them part of the task at subject in the game
  • Introduces pedagogical concepts that I can explore when attempting to apply immersion design concepts to educational ones
  • Case Study: Frequency 1550 (2005) a Hybrid Reality Game (HRG)
    • descendent of Multi User Domains (MUDs)
    • Three main characteristics:
      • use mobile and location-aware interfaces
      • bridge physical and digital spaces
      • transform the city space into the game board rather than be solely simulated
  • The paper posits that bridging digital and physical spaces can make learning more meaningful by anchoring information in concrete physically accesibile situations
    • In HRGs information is distributed in
      • local physical spaces
      • digital spaces
      • players’ prior knowledge
  • Sociocultural Learning Theory is the framework
  • Research questions:
    • what are the affordances of HRGs for potential uses in educational contexts?
    • why are learning tecnnologies shifting from fixed interfaces to mobile ones?
    • How can HRGs benefit discovery and learning differently from traditional video games and Multiuser Virtual Environments (MUVEs)
  • HRGs have 3 main design elements:
    • mobile and location=based activities
    • multiuser games; social activities
    • expand the game environment outside traditional game space into physical space
  • 3 learning principles: social, experimental, and situated learning via a new relationship to space
  • experiential learning occurs because all game players participate actively
  • sitauted learning is made possible by mobility of users and location-aware interfaces emphasizing the notion that learning occurs as a function of its context
  • The central mechanic of the MMORPG is social storytelling/collaborative fiction
    • MUDs are narrative spaces
    • HRGs are RPGs brought into real space; players need to be moving around to play
    • most of the story takes place in the players’ minds
  • Street Sociability: “the particular public form of sociality, of being at once both interested and yet indifferent and anonymous”
    • “It is a question of a similar anticipatory expectation as in games of chance: something might happen”
  • mobile phone is a culturally-dependent technology;
  • the general belief is that mobile phones will be the technology that helps bridge the digital divide in developing countries
  • Cell phone as a gaming platform is best utilized when informed by the affordances of the cell phone
  • “the most popular [mobile] games will most likely be based on game play rather than on graphics. They will also incorporate the true nature of the mobile phones: communication and location”
  • HRGs can be distinguished from others in educational settings: they take advantage of the user’s mobility in the digital world instead of placing them in a simulated environment
  • Because experiential learning emphasizes that learning involves a direct encounter with the phenomenon being studied, the paper suggests that HRGs take advantage of this emphasis by situating the game in the physical space (such as setting a game on the site where historical buildings stood)
  • HRGs inhabit both physical and digital worlds, and neither is privileged over another
Jul 09

Pohjola, Mike (2004): “Autonomous Identities: Immersion as a Tool for Exploring, Empowering and Emancipating Identities” in Montola & Stenros (ed.)  Beyond Role and Play: tools, toys and theory for harnessing imagination

  • New key words: immediated/immediacy/immediate art; diegetic frame; inter-immersion; Temporary Autonomous Zone & Temporary Autonomous Identity”
  • Inter-immersion will be a key concept for developing the community space inherent in immersive interactions
  • New definition of immersion: “being in character or becoming the character”
  • “Immersion is the player assuming the identity of the character by pretending to believe her identity only consists of the diegetic roles” 
  • An examination of the nordic-style school of LARPing. I’m less interested in the roleplay aspect and more interested in the notion that immersion can serve some purpose in and of itself. This paper covers the exploration of personal identity through immersive techniques, the factors that make roleplaying/immersion a unique art form, and the potential for developing spaces of temporary reality that allow for us to take on temporary identities
  • Name checks the Turku school of immersion that is trying to achieve “character immersion”
    • considered impossible by the author – “based on faulty premise of character that originates with traditional fiction and cannot be applied to immersive, immediated artforms like roleplaying”
  • “This essay is written partly as an attempt to update and post-modernise the ideas of the Manifesto of the Turku School (Pohjola 1999), specifically those concerning character and immersion.”
  • “the internal processes and interpretations of the player are for the game as a whole until they are expressed and become part of the diegetic frame. Before that they are merely “individual narrative readings”
  • Stuart Hall (1996) beleives that seeing self as narrative is the essential part of identity creation
    • Turku manifesto saw this the opposite way: character identiy can be created by seeing the narrative as the self
  • J Thomas Harvianinen: Three kinds of immersion: Character Immersion, Reality Immersion, Narrative Immersion
  •  Impossible to roleplay alone. Roleplaying should include interactivity
    • It follows that immersion requires interactivity
  • Roleplaying games are experienced as they are created – a unique feature of them
    • could also apply to immersivity
  • media is generally divided into three loose categories: passive, active and interactive
    • passive are recorded and cannot be affected (like film, literature, recorded music)
    • active has possibility of interaction, such as karaoke
    • interactive requires participation, such as computer game, hypertext
      • fourth “transcendent” category is immediate art, “experienced as it is created and has no use for the division between performers and audience”. Examples include roleplaying games burt also parties, communal storytelling or improvised musical jams
    • “An outside audience cannot understand a role-playing game, although it can seem like an interesting performance. Role-playing games take place in the present moment and are transmitted directly from person to person. This makes them immediate”
  • “immersion without action is daydreaming”
  • roleplaying is not necessarily an interactive act
  • the character must exist within a diegetic frame
  • Inter-immersion: the collective experience of immersion shared and strengthened through interaction
    • strengthens the identity of the character rather than the player
    • staying in character helps to stay in character
    • “as the player reaches the inter-immersive state she starts to forget she is just pretending to believe it is all real. She acts as if she really believes the diegesis, and when everybody else does the same and reacts to each other’s beliefs (instead of the pretensions), they forget they are just pretending and start to really believe.”
  • “There is a pattern, and a very clear one when you know where to look. Each new generation of games is less abstract. Where Go is about capturing and re-capturing land, Chess is about a war between two nations, Chainmail is about commanding armies in battle and Dungeons & Dragons is about directing a singular adventurer in a dungeon, modern role-playing games are about acting as any individuals in any setting”
  • Perceiving a game as having a reality is difficult when the game is abstracted (like Chess)
    • as games get more complex and less abstract gaming reality is stronger & more fulfilling.
    • “The next logical step is to lose the barrier separating games and reality once and for all.”
  • Pretend belief moves toward real belief and subjective diegesis becomes subjective reality, temporarily. In RPGs the diegesis is temporarily the player’s identity.
  • Temporary Autonomous Zones can be created to invite players to take on a Temporary Autonomous Identity (TAI).
    • If we can constantly carry a TAI with us, we can always be in a roleplay.

Case Studies:

Masquerade by Kit Williams

Masquerade | The Hankerer

  • Masquerade was a 1979 picture book that contained clues to a real-world treasure hunt, the prize to which was a golden jewelled hare figurine. The puzzle was first solved and the hare claimed by a person who, arguably, cheated.

Mystery on Fifth Avenue by Eric Clough

  • Clough was a young architect who, unbeknownst to the owners, hid a series of puzzles inside the architecture of an apartment he renovated. His intention was to supply the children growing up in the space with an ongoing sense of wonder.
Jul 03

Process Journal 8

Posted in Process Journal           No Comments »

This week I have been focusing more on experiments with incorporating DepthKit into Vuforia. I used a Kinect for Windows (graciously provided by my supervisor) to record some test footage using Depthkit, and experimented with post-processing and green screen to get a sharper image. (I will have to reshoot the test footage with a better green screen setup next time, as my rather slapdash studio setup had too many wrinkles for a clear mask.) I was able to learn how to implement a green screen as well as play with Depthkit’s settings to reduce spikes.

My first new interaction is a hand reaching out of a surface, which is one of the first images I envisioned when I first considered using volumetric capture.

docudepth2Test footage with a cylindrical trigger:docudepth1I’ve also begun playing with Spark AR by Facebook. Facebook’s usage policy seems highly constraining, as it can only generate content for Facebook platforms, and thus may not be all that useful for an artistic practice. Still, it couldn’t hurt to become familiar with the tool.barf-docu Other experiments still to come:

  • Depthkit AR interaction using an object as trigger, incorporating the object in the film.
  • Learning how to maximize the efficiency of the available processing power, as experiments suggests the Android phone has trouble rendering multiple volumetric videos simultaneously.
  • Volumetric videos generated on a surface and in midair.
Jul 02

Theme Parks 2/2

Posted in Contextual Analysis           No Comments »

I visited the Wizarding World of Harry Potter (WWHP) at Universal Studios Orlando in 2012, several years after opening but before the Hogsmeade expansion. I was familiar with the Harry Potter storyworld and accompanied by an avowed super-fan I found the place captivating. Since the two of us were familiar with the Harry Potter world we found no impediment to understanding or enjoyment of the park – it was designed for us.

There is no meaningful onboarding for non-fans, and unlike some of the other rides at Universal Studios that remediate the plot of the source material, there is no retreading of known plot lines. WWHP stands alone as a transmediality, existing as a distinct artefact separate from any of the films or books (although it is explicitly based on the films, as it uses actors from the films and is designed after the look of the films). Interestingly, it seems to exist at a specific point in the Harry Potter timeline, being set before the events of the last few books, as evidenced by the presence and age of the young stars of the films.

The sense, then, is of inhabiting the Harry Potter franchise at a pivotal time in the plot – before the climax and denouement of the final books but after most of the world is established by the first few books. This is a distinction that would not be noticed or appreciated by those who are not familiar with the story world.

Embodiment in the space translates to embodiment within the narrative. It is those for whom the cult geography resonates that are the core audience of WWHP. These people become embodied within the plot by their presence at the park, and their presence at the park generates the sense of communitas that brings the park to life.

Jun 28

Waysdorf, Abby, and Stijn Reijnders. “Immersion, Authenticity and the Theme Park as Social Space: Experiencing the Wizarding World of Harry Potter.” International Journal of Cultural Studies, vol. 21, no. 2, 2016, pp. 173–188., doi:10.1177/1367877916674751

  • The functioning of a park like Wizarding World of Harry Potter (WWOHP) is reliant on sensory detail as well as the inherited communitas of fans who feel like the space was prepared for them. It invites embodiment within the fiction and encourages engagement with the existing communitas of the HP fandom. It is perhaps designed to be the ultimate expression of fandom – the place that is as close as possible to habitation inside the fiction, designed to be experienced in a way as honest as possible to the way it would be engaged with were it “real” – as a place where you can not stay forever, but merely visit as a tourist.
  • New key terms: Habitus, Ironic Imagination
  • Returning key terms: simulation, communitas
  • a new definition of immersion: immersive because it feels inhabitable – as detailed as the real world and shared with others as an imaginary habitus
  • emblematic of a trend in the industry toward environments promising immersion into a favourite text
  • Limited understanding of why theme parks appeal to people, as most of the theory has focused on form and simulation
  • Paper is a visitor-centered work
  • small research base – only 15 interviews
  • Ironic Imagination (Saler)
  • Visitors can explore the storyworld in an embodied manner
  • Eco: “Disneyland tells us that technology can give us more reality than nature can
    • In choosing Disney over actual locations “the tourist has preferred the simulation to the reality” (Sorkin)
  • Success of the theme park points to the postmodern preference for simulation, safety and entertainment over the “real” experience of landscapes and environments
    • “a special place full of communitas not found anywhere in the mundane, consumerist habitus” (Aden)
  • Cult Geography: “fan attachment to non-commodified space, or… to space/place which has been indirectly or unintentionally constructed” (Hills)
    • eg tour of X-Files filming locations in Vancouver – constructed through fans meaning-making process rather than the media or tourist industry
  • Most critiques of theme parks is that they are inauthentic
  • Lukas: a themed environment becomes authentic when it “is sensory available”
    • That is, those visiting know it is a simulation but it becomes authentic when it feels correct on all sensory levels
  • Fantasies, mythologies and cultural icons can be enacted and played with
  • The visitor is encouraged the engage with the fictionality of the theme, and to imagine on a body level
  • This is the same kind of performance asked for and required for Total Immersion as suggested by Machon
  • Immersion, for this article: “The feeling, through the medium, that the audience member is part of the artistic or narrative world”
  • Saler: Virtual worlds: “acknowledged imaginary spaces that are communally inhabited for prolonged periods of time by rational individuals”
    • A storyworld becomes virtual when it is adopted and discussed by many individuals who group together in order to fill its details and make it more real
  • Becomes immersive because it feels inhabitable – as detailed as the real world and shared with others as an imaginary habitus
  • Embodiment (Crouch) : “A process of experiencing, making sense, knowing through practice as a sensual human subject in the world”
  • Ironic Imagination: A “double-consciousness” that allows the subject to be emotionally invested in and contemplative about a fictional world while maintaining knowledge that it is fictional.
  • There is no more “real” version so the simulation suffices as the closest thing to actual experience
  • diegetic expansions: products that are, or could be, from the series
    • “Like you are vacationing in Diagon Alley”
  • In WWOHP there is a sense of communitas inherent in arriving with a shared experience and camaraderie coming from Harry Potter. For example, recognizing another park-goer as being from the same HP House as yourself and using that to strike up a conversation

“‘Like Walking into a Movie’: Intermedial Relations between Disney Theme Parks and Movies.” The Journal of Popular Culture 50.4 (2017): 704-22. Print.

  • Provides some insights into the cinematographic inspiration behind theme park and ride design. More importantly the article provides some language and taxonomic tools for discussing elements and techniques borrowed from other media.
  • The organization of space in Disneyland follows cinematic principles rather than architectural ones
  • Wolf and Rajewsky’s typology of intermediality.
    • distinguishes between phenomena of transmediality and intermediality
    • each of their categories can be illustrated with an example of Disney parks “borrowing” from movies
  • parks are distinct from museums in that they create self-contained worlds that are separated from the rest of the world, fusing media that have been historically and culturally viewed as distinct
    • Can thus be classified as a “hybrid” medium, “composite” medium, or “meta-medium”
  • transmediality refers to elements that are nonspecific to individual media and whose origin is unimportant. Intermedial transposition is concerned with specific artefacts that have been “translated” from one medium to another
  • Intended to provide critics with a set of conceptual and terminological tools to describe intermedial phenomena
  • In many Disney rides vehicles frame the visitors’ view like a movie camera.
    • The ride vehicles, writes Lainsbury,were designed to function like movie cameras by twisting, turning, and directing the gaze of passengers from scene to scene. Their high backs and sight-restricting sides, not to mention the metal lap bars that held guests firmly in their seats, guided the experience further by erasing from view anything that might spoil the illusion. 
    • a Transmedial phenomenon as it is also employed in photography and painting, for example
  • Remediation is the retelling of a story in a new medium – in theme parks the line between remediation and intermedial transportation can be blurred
  • “Magic Wand” principle :a corridor links the entrance to a central square, from which different themed areas radiate out like the spokes of a wheel
Jun 20

Process Journal 7

Posted in Process Journal           No Comments »

This week I have been continuing my AR explorations. My most noteworthy development is the incorporation of volumetric video into AR, as seen above.

ardocu

Situating the volumetric video clip is as simple as nesting it as a GameObject child of the intended ImageTarget. Scaling and placing it is slightly more complicated as the GameObject seems to exist at a different origin than the video itself.

More experimentation is required. I have done some experiments with regular video, and virtual buttons – that is, areas on trigger images that can be occluded to cause events to trigger in Unity.

img_20190620_155103_452 screenshot_20190620-154017 screenshot_20190620-153959

The app builds and runs, but it is very slow to generate a video once the button is pressed. I suspect that the code is turning each video off and on every time a new one is called. I experimented with having all three videos running offscreen, as well as switching the renderers on and off, but succeeded only in crashing unity. Still, it’s a promising proof of concept.

Jun 20

Contextual Analysis

Where Dark Things Dwell poster

Where Dark Things Dwell at Black Creek Pioneer Village is a unique escape game experience at Toronto’s Black Creek Pioneer Village. It is intended a showcase of the historic buildings at Black Creek, using the escape game format as a means of bringing players who might not otherwise have come to the village on a tour of the site.

The game uses environmental storytelling to explore the true history of the site as well as the fictionalized story of a curse trapping the players in the village. Each building has a set of puzzles themed to the building, which can have Nicholson’s “Ask Why” paradigm applied to them to explore their placement in the village. The mill has a puzzle involving sacks of flour, for example, while the building containing drying racks of herbs contains puzzles involving potions.

It is interesting to note that the puzzles forms are often unrelated to their content – the flour sack puzzle, for example, could be easily replicated with paper or wooden tiles. The potion puzzle is simply a series of riddles. While thematically linked to their sites (or by content – for example, the potion riddle puzzles are themed after spells) there is rarely a physical necessity or affordance that causes a puzzle to be situated in a physical form or space.

Each space’s puzzles are contextualized by the space they occupy, bringing narrative synchronicity to what would otherwise be unrelated puzzles. By couching puzzles in the trappings of their space narrative coherence is strengthened, even when there is no deeper relationship.