AMA: Constructing Memory

"They thought that they killed you when they shouted ‘Fire!’
They thought they buried you
But what they did
was bury a seed"

– Ernesto Cardenal“Epitaph for the grave of Adolfo Baéz Bone,” 1948

AMA Constructing Memory is an Interactive Art Book that Commemorates the victims of the Massacre perpetuated by the Nicaraguan Government in April of 2018. The Book utilizes easy to access AR technology that allows the readers to place a replica of the altars erected by the families of the victims creating a space for grieve or remembrance.

Augmented Reality •Interactive Art Book • Social Justice Project
Experience Designer  •  Developer  •  Animator
Credits
Director:
Emilia Yang Rappaccioli
SparkAR design and Artwork
Andy Cao
‍Ana Carolina Estarita Guerrero
SparkAR development assistance
Alanna Cassias Foilose
Augmented Reality Creative production
Emilia Mason
Experience Design
Ana Carolina Estarita Guerrero
Photogrammetry process
Emilia Mason
3D modeling and textures
Juanita Rubio
MY ROLE

I worked on the team that designed the Interactive Art Book format.  I researched user flows and designed the AR User Experience. I was also on charge of the Development and Implementation of the experience inside the AR Studio Tool

THE TASK AT HAND

AMA AR interactive Art book, makes part of a larger ecosystem within AMA, an organization and collaborative process that seeks to dignify the victims of state violence in Nicaragua as of April 2018, and to honor their memory. The purpose behind the Interactive Art Book was to create an itinerant platform that work as a stand alone experience that allow the readers to get full context of what happened in Nicaragua during April of 2018.

OUR APPROACH

One of our main goals when developing the AR component of the Interactive Art Book, was to make sure that in every interaction with these AR Altars due respect was given to the victims and their families. Unlike a VR Experience in which the designer has total control over the input the user is receiving in any given moment, in AR the user’s context interacts with the digital environment, leaving the designer very little control over the final experience. To work around that limitation we decided to use Target or Marker Tracking as our experience trigger. This solution served multiple purposes

  • By using target tracking we made sure that whoever comes in contact with the Altars has an understanding on the context surrounding them: Either because they find them while reading the Interactive Art Book, in a Museum Exhibit or because they downloaded them from the AMA website, every user that sees an altar has a baseline knowledge that allows them to interact with the altars with respect.
  • The Target tracking allow us to guarantee consistent Scale for each use case: We wanted the Altars triggered from the Art Book to be of a scale consistent with a desk or a similar surface where a user would come in contact with the Experience.
  • Using Target Tracking made it possible for us to design for other use cases besides the Art Book page:  For example, In Museums, instead of using the smaller book prints, we’ve used a A2 print of the tracker, which gives as a result a bigger Altar that is coherent with the distance and flow of museums.
  • Targets allowed us to create a physical piece that works as a reminder no matter the circumstances. For a lot of the families and communities that were directly impacted by the violence the AR component was a visceral and powerful experience, but is not a thing they could have on display on their living room at all times. Using an image tracker created the opportunity to design beautiful posters that commemorate the victims, that can be easily displayer, that look unique to the Computer vision system.
  • Target Tracking scan and find process is more intuitive and immediate that the one of ground plane tracking. This is a particularly important consideration when the experience is targeted to sectors of the population that usually don’t come in contact with this sort of technologies and might have short patience for the “scan your surrounding” prompt in loop. This project is at its heart a homage to the victims and their families, who a lot of times come from low income families, we wanted to make sure the experience felt as natural and comfortable to them as possible.

SOCIAL AR FOR SOCIAL JUSTICE

Once we figured out Target Tracking was the way to go, we needed to decide where would the ARE live. Creating a custom App was not only outside the scope but would add an extra level of friction we wanted to avoid. WebAR was an attractive solution, but it was cost prohibiting. For that reason we decided to use Social AR due its familiarity for a most users. Usage data reveals that Facebook is still the most popular Social Media Platform in Latin America, while Instagram targets to another subset of expats or scholars outside the region. With that in mind, we decided to use SparkAR as our creation tool.After deciding we wanted to host the experience in IG/FB, the biggest challenge ahead was optimization. Each altar is made by more than 20 objects captured using photogrammetry. To make sure the Altars fit within the 4 MB constrains on IG Filters, we had to aggressively optimize the 3D models as well as the textures and make all animated features driven using visual scripting instead of baking.

THE USER EXPERIENCE

Unlike Stand Alone AR Experiences, AMA’s AR user experience starts offline either once they pick the book or once the enter museum experience. Added to that, both ways of approaching AMA are very self directed, meaning we needed to make sure the user’s knew how to use and interact with the Altars, way before they reach the AR component itself.
Drawing of flow showing the transition from Offline to Online Interactions for the user
The first component that primes the user to the Interactive component of the Art Book is on the cover. A line that states clearly that this is “An Interactive Art Book”. That same Piece of information appears in the interior cover.
Right after the table of contents, we included a spread that details how to access the Altars. This section explains in detail what is a QR code and how to read it and gives the user a QR Code for “testing” so they can familiarize themselves with the use of the technology. The users are also prompted to share the content they capture with the #YoNoOlvidoAbril hashtag.
  • Scanning the one of the codes takes the user to the target platform where they are Prompted to Scan image: We decided to use the default test hint instead of a custom one, as the default hints are displayed in the user’s device language dynamically. Additionally, multiple tests revealed that keeping the text for as long as the image wasn’t scanned gave best results, as a lot of the users on the target audience lacked experience with AR.
    ‍
    Though each map QR code redirects to the ARE that is activated by that specific image we included a “reference ghost image”. This image serves not only as reminder of what to scan, but through testing we noticed it prompted the users to try to match the real image with the ghost image. If well this is actually not necessary for the experience to work, it served as a good marker for the users of being at the right distance from the target image to have consistent tracking
  • Once the Image Target Tracker is found the reference image and the text hint disappear, An a sinusoidal animation that shows the Altar scaling up starts.
    The animation timing is set on a middle point where it doesn’t feel too jarring for users that have little experience and might be surprised by the altar separation, Neither too slow to make it feel the target wasn’t found.
  • The user can use pinch, drag to change the scale and the position of the altars. We decided to leave instructions for those two actions out of the experienced. We figured advanced users would try that on their own, and for more inexperienced users, scaling up and down wouldn’t add much to the experience, and instead adding the instructions would contribute to a confusing experience.
  • Finally, if the user taps the Altar once, the candle to the left lights up. If the user taps the altar once more, the candle to the right lights up. This last interaction is thought almost as an easter egg, aimed to allowing users to pay respects to those commemorated in the Altar
  • User’s can share their experience using the #YoNoOlvidoAbril hashtag.