Final Project Documentation
Nicaragua’s 2018 Revolution – Sharing experiences and stories through an Augmented Reality interactive documentary
For my research project, I am interested in creating and developing an Augmented Reality interactive documentary that tells some of the stories of people living the current revolution in Nicaragua. This interactive documentary will allow people outside Nicaragua to experience through Cultural Augmentation real-life experiences, bringing these stories to Canadian locations and context.
I am half Nicaraguan and half Canadian, I grew up in Central America, and most of the people I know and love are from Nicaragua. Living in Canada while this revolution is happening has led me to realize how little people in North America now about what is happening outside the developed world. My previous work background is in activism, civic engagement, and journalism. For this reason, it comes naturally for me to want to tell the stories of people, share their experiences in the current Nicaraguan context. It is important for people outside Nicaragua to know these stories and to be aware they belong to “normal”, “regular” individuals, who have found themselves living under an oppressive government, therefore, have decided to risk their own lives for their freedom and to protect others.
For my final prototype, I want people to experience these stories through Augmented Reality. Using this type of technology as a vehicle will allow the users to keep experiencing their current context, in this case, the streets of the city of Toronto, but augmenting that context with holograms, audio and the stories of people from Nicaragua.
This research project and prototype are relevant because the people need to know about what is happening in other countries. Allowing users to be aware of specific situations in other countries and experiencing those stories by contrasting both contexts will make them question how they would react if they were in a similar situation.
For the ideation and development of my prototypes for this course, I used the research questions for my thesis:
- How can an Augmented Reality interactive documentary provide awareness and a deeper understanding of the different experiences of the same event?
- How can Augmented Reality be used for storytelling purposes in an interactive documentary to tell the experiences of people living the current revolution in Nicaragua?
- How can an Augmented Reality interactive documentary on the Nicaraguan current revolution be designed for people who are not familiar with those stories and reality?
- What are the affordances of Augmented Reality as a technological resource to tell the stories of the Nicaraguan people in Toronto?
- How can decolonizing methodologies be used to conduct ethnographic research and share the stories of the Nicaraguan people experiencing the 2018 Revolution to people abroad?
- How can this research project support the struggles of the Nicaraguan people experiencing the 2018 Revolution?
- How can (In what ways) 2D content, low-resolution content, and social media posts be used and translated for AR content?
- How will I approach the ethical part of the anonymity while using people’s social media content as raw data for my research project?
Related Digital Projects:
Reference 1: Terminal 3 – Asad J. Malik (2018)
Terminal 3 is a Cultural Augmentation interactive experience. It tells the stories of Muslims who face additional screenings when they enter the United States. Users take the role of a customs officer interviewing and questioning each “character. The experience takes place across two rooms, in the first room one, a user is given a Microsoft HoloLens headset and uses voice commands to ask one of six holographic characters increasingly personal questions. Once the users finish the interrogation process, they proceed to go to the second room, where they meet the actual people the characters are based on.
The artist uses Cultural Augmentation for interactive storytelling, allowing the users to learn more about each story not only through an interview process. During this interrogation/interview process, the users learn personal details of each character, which makes it very personal and insightful. This allows users to understand what the characters go through. One feature I found extremely interesting is how the hologram, at first, looks low-res but becomes more real as the character tells you about his or her life and journey, I feel that is compelling visual feedback.
This project puts the users in the position of an interrogator with airport security. An interrogation room is built each time the experience is presented. The characters were filmed using Depthkit; the volumetric capture was transferred to Unity and experienced with Microsoft HoloLens.
Asad J. Malik is using storytelling AR, and his projects understand how to use context for evoking emotions. I think his work is powerful, as it based on his own experiences. Malik is from Pakistan and moved to the U.S. a few years ago, and he has experienced this type of situations in the past. Terminal 3 is all about context, it refers to the travel ban forced by the U.S. Government and putting these experiences into a context is part of the reason why Malik decided to create it into AR instead of VR, he didn’t want to remove the users from reality. He has mentioned in several interviews how his work could not be displayed in China, the situation there is different from the political context from the U.S.
Terminal 3 – Asad J. Malik https://youtu.be/8B70uYZFO3Y
I also want to reference another one of his works that also inspired me: Holograms from Syria, an AR Installation which allows users to experience hologram images of the Syrian war in different locations across the US.
Reference 2: Pivot the World (2015)
Pivot the World is an archival database app and web-based platform which uses physical locations where users can open the app and click their ways through historical images and data.
The app’s original goal was to provide a platform for historical preservation. With its video-game-like AR features allowing the users to have access to historical information in an entertaining way. I am interested in this reference because it is all about location, what it was and what it currently is.
The app operates on a system of physical locations where users can open the app and click their ways through a miniature time machine of historical images and data. The archival database grows daily through open-sourced online archives and crowdsourced image libraries. Through these available resources and the app’s AR technologies, users are led on a virtual tour through the past.
What I like about this project is the fact it is an open-sourced online archive and a crowdsourced library that allows people to make their versions of historic preservation. It could empower communities and towns that most likely won’t have the opportunity for digital historical preservation. Today, there are different companies and even non-for-profit NGOs, like CyARk that specialize in travelling the world to make archives and databases of monuments, But this is no available for every town or city as it is costly. This project also allows users to compare how locations and areas have changed through time and development.
Reference 3: Fusun Uzun- OCAD U – “Very frustrating Mexican Removal”, An immersive 360° verbatim documentary. Master of Fine Arts – Digital Futures – April 2017
This immersive verbatim 360° documentary about Canadian immigration detention system. It presents the events surrounding the death in 2013 of migrant Lucia Vega Jimenez while in detention with the Canada Border Services Agency.
The main objective of the project is to raise questions about the transparency of the Canadian migrant detention system. Videos available online demonstrated the architecture of the detention center, which served as the initial core evidence representing the constraints that detained migrants face. Using the coroner’s inquest transcriptions allowed the many traces left behind in Lucia’s case to appear and build the story.
This documentary uses the practice of Verbatim Theatre, techniques and technologies of 360° Cinema. I had never heard of Verbatim theatre and I am very interested in possibly using this as a possibility for translating some of my stories.
It is necessary to tell the stories that make people feel uncomfortable. It is important, to tell the truth of what happens around us, even if we don’t like what it is, it is necessary to address our own privileges. This case was a complicated and very delicate topic, as it presents the story of a woman who died seeking asylum in Canada, and it also addresses the discrepancies in the Canadian system. I am interested in the research process Uzun went through, as she used several references and translated those into this piece.
Prototype process documentation
At the moment of the ideation for this first prototype, my thesis consisted of two sets of story archives, one in Nicaragua and another one in Toronto. Since I had access to Toronto users, I focused on that part of the thesis project.
My first prototype consisted in two maps, a map of a specific location in Managua, a map of downtown Toronto around 100 McCaul, and an augmented reality feature with icons with specific meanings of the current Nicaraguan revolution. The idea was to first allow the users to get familiar with both map, then present the augmented reality sheet on top of the Managua map for them to understand the situation, and then bring the augmented reality sheet to the Toronto map. The purpose was to examine if the users would be able to transport the unfamiliar stories from the Nicaraguan context to the streets of Toronto.
For both maps, I used iMaps using basically the same scale. One of my first suggestions was to use Google Street Views but that option is only available in the Toronto maps and not in any of the Nicaraguan cities.
During the ideation process for Prototype 1.0, I was still debating if it was going to be an interactive archive or an interactive documentary. During the prototyping process, I realized the project should have a structure and story, not just random assets placed together. Developing the prototype map of people, location, objects and interactions were incredibly helpful. Thanks to this step I was able to understand how to focus my prototype.
With the map, I understood I wanted to test the interaction and the reactions of those interactions in with the Toronto Users.
Next step was developing the User Flow:
Step 1: See map of the location in Managua, Nicaragua.
Step 2: Place acrylic see-through sheet with icons representing experiences/stories on top of the map of the location.
Step 3: Users must touch with their fingers the icons they are interested in, for every icon they press I will explain what it means. What happened there.
Step 4: Repeat step 3 as many times as the user is interested in.
Step 5: Have users transfer acrylic see-through sheet on top of the map of Toronto (area around 100 McCaul).
Step 6: Users must touch with their fingers the icons they are interested in, I will translate the story of each icon to Toronto locations.
Step 7: Ask them questions about what type of interaction they would prefer to experience here in Toronto
This was the user flow steps I presented to Emma and Ana. I received one suggestion, to run two tests, first with the google map view I had already envisioned, and the second one using Google Maps street view, as that would represent real Augmented Reality, as it is the view the users would truly have.
I proceeded to work on the captures from Google Maps only to realize that Street view is one of those privileges I didn’t know was a privilege. Nicaragua does not have Street view available and the maps on iMaps and Google Maps are from several years ago and have not been updated in a long time.
That day, after CFC I already had an appointment scheduled with my thesis principal advisor, Judith Doyle. She suggested adding an additional step to the user flow as it felt incomplete. The new user flow included the next steps:
Step 6: Once the user wants to stop from learning about the icons placed on the Managua Map, tell the user to transfer acrylic see-through sheet on top of the map of Toronto.
Step 7: Explain to the user they can point with their finger to whichever icon they want, and I will translate the story from Managua as if it had happened in Toronto. Tell user they didn’t have to point at any icon if it would make them feel uncomfortable.
Step 8: Ask them questions about what type of interaction they would prefer to experience here in Toronto.
Ana suggested the street view maps, which were not available from Nicaragua, so I decided to print satellite views to see if people would find them more appealing than the regular map. Nope, no they don’t. Users found them extremely confusing and were not able to identify the streets of Toronto they are familiar with.
The user flow was extremely helpful to understand how the interaction with the users and the maps would work, it also helped me imagine how the final project, could go about.
During my first user test round, I was able to test on Savaya, Dave, and Kristy. Emma Britto was my note taker.
Notes by Emma Britto:
There are 2 maps laid on the floor, one of Managua’s Core and one of Downtown Toronto
- Savaya places projection sheet/AR stand-in on top of Managua
- S touches a tree icon which is revealed to be a “Tree of Life”
- Represents propaganda of the Government and were chosen by the First Lady.
- Cost between $20,000 and 40,000 USD
- Location and icon represents the physical monument which was destroyed during protests.
- S touches “X” icon
- Represents location of a protestor who was killed at that location by police and paramilitary forces.
- Flowers icon shows were the mothers of students were killed made a peaceful march called “the march of flowers”.
- S tuches gun which represents a sniper.
- Each icon touched fills in more of the revolution’s story
- S caught on very quickly to the process
- S switches AR/Projector sheet to Toronto
- These atrocities are now relocated to a Toronto context
- Water icon which marks where water bottles were handed out to students during protests is now relocated to McCaul st.
- S comments that the switch to Toronto was jarring, made it personal
- From watching S, D was able to quickly understand the process
- D touches motorcycle icon first, followed by stick figures and an “X”
- D seems very interested in prototype
- Street view offers even more details
- Makes it even more “real”
- The switch is made to Toronto
- Death of student relocated to McCaul and march of flowers would be down St. Patrick
- Body language alters slightly, gets closer to image
- Death of student relocated to McCaul and march of flowers would be down St. Patrick
- D says the switch to Toronto is eery, that there is a cognitive dissonance
- Also caught on fast
- Touches icons which represent tear gas and another one killed
- Translated to Toronto
- Tree of Life on MacCaul
- Tear Gas at OcadU
- Kristy asks about evocation
- Thinks that the fact she saw the process/user testing happen twice before numbed her to it
- Prototype easy to understand
- User friendly
- Simple but effective interaction
- Projection sheets lines up well with both maps
- Effective at personalizing foreign events
On my second round of testing I was able to interview Feng, Sean, Finley, Ramona, and Kylie.
Sadly, ran out of battery, and had no note taker but managed to record the interviews of Feng and Sean.
I am very surprised at the results, I honestly wasn’t expecting people who grew up in Canada and people from other countries to have such strong reactions to the prototype. What I found more interesting is that the reaction in their faces was not as strong when I would explain the situation in Nicaragua, the moment they switched the augmented reality see-through sheet to the Toronto Map everything changed.
Reflections Prototype 1.0:
- My user interface was easy to understand but needed to contextualize the stories to the Toronto map, sometimes it was confusing.
- I realized I can only test on Canadian user testers. This made me reconsider if the Nicaraguan part was truly answering my research questions.
- It is emotional for Canadian testers, more than I anticipated.
- It is emotional and exhausting for me, more than I anticipated.
- The prototype had 3 types of users:
- Moved by stories in Nicaragua.
- Moved only by the Nicaraguan stories adapted to Toronto Context.
- Moved by A and B.
- I realized an interactive archive doesn’t really answer my research questions as specific items give information about the context but don’t really tell a story.
- Learned Users in Toronto prefer to follow a story map at an outdoor location instead of recreating a space indoors.
- Storytelling through user’s stories and intersectionalities to connect between Nicaraguan users and Canadian users.
During the Methodologies workshop, David recommended working on a map of the stories to understand what stories already exist, and are available for me to use. This idea made me question if this should be a collaborative or participatory project. Although I am Nicaraguan and have a past as an activist, I consider myself an outsider as I am not there and my life is not in danger. I don’t know what are the stories that people living this current revolution truly want to pass to other people.
Since I can’t start interviewing people on what stories from Nicaragua they would like for folks in Canada to know about, I decided to map some of the stories I have access to right now. I went online to see what stories I could map out and found several different posts of people who experienced the 19-hour siege attack that started at the National Autonomous University (UNAN), the Church of the Divine Mercy, finishing at the Cathedral of Managua. I selected one story for people to learn about what that person lived during a 24-hour period. Researching what the stories are, where the stories are taking place, and how the stories are told sure helped understand how difficult, both timewise, technically, and emotionally this thesis and thesis project will be.
The story I chose for the prototype is the twitter feed of “Concepción Palacios,” from July 13th to July 14th, 2018. She is a doctor who was volunteering to attend in a medical stand at UNAN. The reason I chose her story is that it was the easiest to translate and wasn’t as graphic as the other stories I have available.
Using the Chicana/Latina Testimonios methodology, I translated the doctor’s testimony not word by word, but the context of each tweet. The methodology suggests one must translate the meaning and context as a priority, especially because most of the times, testimonies will be regionalized.
Twitter feed – Concepción Palacios – @Medica_vandalic
Doctor – Public profile – 3,279 followers
July 13th – 12:11 pm
A helicopter has been flying since yesterday above the university.
July 13th -1:06 pm
They drove by and shot at us about 15 minutes ago. So far, no one has been injured or killed.
July 13th -1:11 pm
We are alert, on maximum alert.
July 13th -1:14 pm
If the paramilitary forces enter the university they will destroy it.
July 13th -2:26 pm
They are shooting at us, a sniper is shooting at us.
July 13th -2:40 pm
They just stopped shooting. They are reorganizing and reloading.
July 13th -2:48 pm
They are attacking us.
July 13th -2:55 pm
Heeeelp! Please, send some help! We are under attack!
July 13th -3:18 pm
I already made it inside the university, but the paramilitary forces are still shooting at the people outside.
July 13th – 3:30 pm
I don’t want anyone to die 🙁
July 13th -3:33 pm
Direct attack on the university with Dragunov, AK 47, and other weapons being used.
July 13th -3:53pm
They want to kill us all!
July 13th -4:15 pm
People are injured everywhere. It is so painful to see them like that.
July 13th -4:30 pm
They are killing us! The attack on the university is still going.
July 13th -5:34 pm
They are attacking the university from the entrance near the church. This is a massacre.
July 13th -6:14 pm
I am still alive. This regime will not kill me.
July 13th -7:08 pm
They burned down a building inside the university. Damned paramilitaries, damned genocide government.
July 13th -7:18 pm
I don’t want to see anyone taking advantage of this situation. No politicians, no one.
July 13th -8:03 pm
They are attacking the church.
July 13th -8:08 pm
I am at the church and they are attacking us. I love you.
July 13th -10:23 pm
I am not injured.
July 13th -10:26 pm
We are surrounded.
July 13th -10:33 pm
We need to evacuate, we are surrounded, they have orders to kill us all!
July 13th -10:46 pm
They are still shooting, non-stop. They want us dead.
July 13th -11:32 pm
We are still under attack.
July 13th -11:48 pm
Urgent! We need to evacuate! We must follow the caravan to evacuate!
July 14th -12:13 am
The priest is with us, these men of God have not abandoned us. Thank you!
July 14th – 12:15 am
I told you! They were going to destroy the university and everything else!
July 14th – 12:19 am
Hurry up people! We need the caravan to arrive to the church. PLEASE HURRY!
July 14th – 1:42 am
The church is destroyed 🙁 We are still under attack.
July 14th – 2:14 am
They just cut the power off 🙁
July 14th – 3:12 am
They are still shooting at us 🙁 What a day, night, and early morning of terror the one we are living.
July 14th – 10:10 am
I made it to the Cathedral of Managua, we are here hugging each other and crying. We lost two of us but this battle must go on. No one is giving up!
July 14th – 10:31 am
We resisted 19 hours of gunfire by the paramilitary forces and the Ortega police. Long Life for Nicaragua, fuckers!
July 14th – 12:46 pm
I am in a safe place. I love you all.
July 14th – 2:13 pm
Why are we the ones who have to run and hide? They are the murderers. 🙁
July 15th – 10:08 am
They are kidnapping doctors and medical help but you should know I am willing to keep going in this battle. They have weapons, we have courage and honour. This fight must go on! Freedom to our country to live!
July 16th – 5:21 pm
When I close my eyes I still hear the murdering bullets flying through the air.
Doctor Vampire – Medical stand UNAN
July 20th – 12:54 pm
Hi everyone, due to safety reasons I don’t have internet access, I’m disconnecting myself for a while. I love you all and this fight must go on. We want Freedom. Hugs.
July 21st – 9:12 am
At this same time, one week ago, an older lady hugged me at the Cathedral, her eyes were filled with tears. In between tears, fear, and desperation, I realized I had just survived 19 hours of attack. Bullets and more bullets sent by the Ortega Dictatorship.
July 21st – 9:15 am
We can’t go back to normality without justice, there is no peace without freedom of expression, you can’t go back and study in Universities where there is the blood of our brothers and sisters. We can’t go back to a normal schedule when we fear for our lives.
July 21st – 9:19 am
I could hear the bullets so close, but so, so close that I told myself “You’re going to die here, in this medical stand”, but you grabbed my hand and told me “Doctor, I don’t want to die.” I proceeded to cover your bullet wound, give you an IV, and I caressed your face saying “my love, don’t worry about a thing, everything will be ok. Just breath.”
User Testing – Tuesday, July 24th
User Flow for the second iteration of the prototype:
Step 1: Give the scripted introduction to users.
Step 2: Give disclaimer to users. Remind them they can stop at any time.
Step 3: Presented map of Downtown Toronto. Point where Dundas Street is, Where OCAD University is located, Grange Park, and Queen Street.
Step 4: Request users to place an acrylic see-through sheet on top of Map of Toronto.
Step 5: Ask users to place the icons for University and Church in the right place of the map.
Step 6: Give users the printed tweets and tell them to read the text, not the date or time.
Step 7: As users read the tweets I will place the icons on the map.
Step 8: Once users are done present them the map of Managua with the items already placed. Explained the event how it happened in Nicaragua.
Step 9: Ask questions.
Questions for users right after testing:
-What was your first impression of the prototype?
-What do you think of the experience?
-How do you feel about narrating the story of this person?
-What do you feel when listening to the story and picturing the visuals on Toronto streets?
-Is there anything thing that you consider like too graphic or that you think I should change?
Reflections Prototype 2.0:
This prototype was different than the previous one, it evolved into the stories. My initial idea was to present both maps, I would continue working as the interface for leading the users into the story and also what the visuals and audio (the icons) would be. I presented my prototype 2.0 to 2 students in a group testing…it went well…I found they were very moved and uncomfortable by the test and the story, this made me second guess my entire process and prototype. Am I pushing it? Am I going to make people too uncomfortable with this project? Also, is this too much for me? Every time I go through the prototype, the stories, or the raw material I get very stressed, and it affects me. I had been going to see the school counsellor but I stopped. This entire process made me reconsider staying on counselling/therapy during my entire thesis, as the situation in Nicaragua affects my personal life but also my work at school.
Overall, the user testing gave me feedback regarding how the icons should be presented. All three testers were very specific regarding sound, even though they were looking at only the icons on the maps they were also imagining the sounds (helicopter and gunshots). After the test I realized I had some issues with the icons, I didn’t practice enough and all my icons were not in order, I spent some time looking for the right icon at the time of the right tweet.
After the test, I was stressed because it makes me very uncomfortable to cause negative emotions in people. I mean, I do want to tell the stories and the reality that is happening in Nicaragua but I feel like I break something every time I do. This is something I have to work on and also should write more about my feelings and how this process affects me, it both drives me and stops me.
Anyway, when I was going to present the prototype 2.0 to Kate and Emma I was so stressed and uncomfortable about the entire prototype I made last minute changes to the entire thing and I also forgot to present the entire story and events on the Managua Map. I bombed!
Their feedback was extremely valuable, they recommended to make a graphic content warning and tell users they can stop/withdraw at any moment they want. I don’t know how I didn’t think of this before. Also, I would be allowing the users to read the tweets themselves, this would allow them to feel the story in the first person but also let me have enough focus to select and post the right icon at the right time.
I also realized that the entire prototype affects me so much that I make last minute changes, I needed to more firm in my performance as the interface but also stop doubting my prototype and thesis.
Prototype 2.2 for CFC demo:
The first thing I did when I got back to my apartment was to google what is considered Graphic Content, to my surprise, all my stories, fall under this description. Then I proceeded to look into waiver examples and ended up having a mini script about the story and prototype.
Script used for Introduction and Disclaimer – Prototype 2.2:
In this prototype, I will present you the story of Concepción Palacios, MD. All we know is she is a doctor and she live-tweeted her experience during a 19-hour siege attack that started at the National Autonomous University (UNAN), the Church of the Divine Mercy, finishing at the Cathedral of Managua.
These tweets are public on her Twitter handle @Medica_Vandalic and shared publicly with her 3,279 followers.
I used the Chicana-Latina Testimonios methodology for the translation of the text, this consists of translating the words but more importantly the context. Some words had to be changed because Nicaraguan Spanish is very regionalized and the phrases cannot be directly translated into English.
As a disclaimer:
This prototype has sensitive and graphic content. Feel free to stop reading the tweets at any time you want.
Siege (noun): A military operation in which enemy forces surround a town or building, cutting off essential supplies, with the aim of compelling the surrender of those inside.
Then assigned a proper icon for each tweet:
Reflections on prototype 2.2:
From the very beginning, I found that having a graphic content disclaimer was extremely useful, not just for the user to be aware of the type of content but also for me, I felt I didn’t have to worry about the story and the prototype.
I printed several copies of the script and the tweets translation for the CFC people to read and be able to understand the dates, times, and event. Ana read a couple of the first tweets as we didn’t have enough time to go over the entire prototype.
After watching the demo video Jad so kindly recorded for me, I feel the demo went smoothly, I feel I talk way too much and repeated myself quite a lot.
During the Demo, one of the suggestions that were made was not to “desensitize” myself from the stories and reality, but instead use my feelings into the project. I still need to figure out how to do this.
Some of the questions were regarding how I am going to narrow down the stories and locations, of that I am still not a 100% sure, as events keep happening and I haven’t decided what to do yet. Meanwhile, I will keep an archive of the stories and profiles of people telling their stories and situation. This affects the narrowing down of a location, this example was all about the similarities in both locations: The University and the church. Ana suggested mapping the same spaces because it makes the story much stronger and I completely agree with her, but I feel I still need to explore other ways to play with the context of locations and stories to decide what to do.
For the next steps, I feel I need to continue to archive as much information as possible, also, apply for REB. After completing the TCPS2 certification and having Cris in the classroom explaining further information about the REB, I was informed that even if the tweets are public and the videos were made for sharing and reporting an event, I still need to apply for REB approval. I will probably have to make the stories anonymous.
During the summer I will start developing content in Augmented Reality to start prototyping a digital version of the stories.
I plan on using Unity and AR Kit for mobile AR to begin with. I also want to experiment with the North Star Project by Leap Motion.
USER TEST FEEDBACK FOR: