Exploring Immersivity In-Home and Virtually

Initial Thoughts & Project Impetus

Currently, global pandemic in the form of COVID-19 has transformed the ways humans relate to each other. With most of the world in some form of quarantine and observing social-distancing guidelines, most people are remaining in their homes and must turn to technology in order to engage in work and play, as well as to connect with colleagues, friends and loved ones. Whether socializing with another person or escaping reality via Netflix, screens provide the most safe and convenient way to be with other people at this time.

The inability to gather in groups has severely impacted all traditional theatres from Broadway on down the line. These businesses have been forced to close and move their content into online formats in order to continue community engagement, cultivate their audience, and keep their companies alive. Necessity being the mother of invention, this world-wide shift to social gathering online through platforms such as Google Meet, Hangout, and Zoom, has awoken many theatre practitioners to a hybrid art form that straddles the lines between theatre and film.

While the opportunity to watch the National Theatre of London’s broadcasts of famous productions or to see a Zoom play or stage reading within the comfort of my own home has merits, I miss the impact and ephemerality of sharing the experience of theatre with a stranger. Knowing that only those present in this place and time can claim to share a particular moment of any live performance bonds me in a small way to another human being who I might otherwise have never come across in my life. This project stems directly from a desire to make that connection a reality in the midst of the coronavirus crisis.

My previous PAR work has focused on audience engagement, participation, and interaction through various means and in different sites. Most recently, my project Within These Walls sought to coach an audience participant in role as an “Adventurer” in their own home through the process of creating a small “crumb” of creative material (a piece of art, writing, music, movement, et cetera). Building off of the site of the audience-participant’s home (or current dwelling), I wondered how I as a deviser could use Zoom to unite audience strangers together and immerse them into the world of a participatory play in their own homes. This mid-range project, Immersive Zoom Play Lab, focuses on exploring the software and user-interface of Zoom and experimenting with various activities of audience interaction in order to eventually devise an in-home immersive experience that relies on audience participation and engagement to tell the story.

Research Questions

In what ways can Zoom be used as a platform for an in-home immersive theatre event? What interactions (between strangers, the virtual space, and the physical space) are most satisfying to an audience participant? How can an audience participant become oriented, disoriented, and re-oriented within their experience of a virtual video encounter?

I ask these questions in order to determine the feasibility and possibilities of devising an immersive theatre experience that capitalizes on performers and audience participants’ immediate surroundings to support a highly responsive narrative theatrical experience.

Terminology

In order to keep my mind straight on the language of this project, I find it necessary to take a moment and clarify some of my language as it relates to this line of inquiry:

  • Narrative: the story or stories that will be devised and developed for an immersive, in-home virtual theatrical experience.
  • Prototype performance: a performance structure that is explored with a test audience in which the narrative is used solely as a means to play with the form of the performance structure
  • Play Lab: a facilitated experimental session in which 2-6 participants work with a facilitator to try out different devised activities virtually.
  • Activity: a structured form of participation explored in the play lab
  • Workshop: a facilitated experience of a prototype performance with a test audience or narrative devising session completed in collaboration with potential performers.
  • Synchronous experience: a live interaction between people (performers, audience-participants, or performers/audience-participants) that takes place virtually
  • Asynchronous experience: a virtual interaction between people that occurs at each person’s convenience
  • Physical Space: the environment and objects a person can encounter in real life (IRL) and through at least three of their five senses.
  • Virtual Space: the digital landscape of collective presence, whether in a Zoom call or Marco Polo chat (for the purposes of this project) that is shared through a screen.

Literature Review

Figure 1. Sketches on orientation as they relate to the gateway from physical to virtual space (Friedgen 2020).

To prepare for this project, I sought to gain a better understanding and knowledge of several different frameworks through a variety of sources. The vision for this project stems directly from immersive theatre experiences, such as Sleep No More, and a desire to recreate the delight this style of performance elicits within the realm of an audience’s own home or personal space. Therefore, it became necessary to explore the phenomenology of orientation as a way to ground an audience in both the virtual and physical space and disorient them or skew their perspective of space virtually. Sara Ahmed’s Queer Phenomenology provided (and continues to provide) great insight into understanding how to orient and disorient audience members within both iterations of this project. Ahmed’s explanation of concepts such as familiarity, recognition of sides, and the body’s relationship to its surroundings contributed directly to play lab activities and design. Early on, I defined the concepts of physical and virtual space, as I initially sought to bring strangers together and create a collective experience. The following images illustrate some of the ways in which I conceptualized both spaces for a play lab session.

Figure 2. Envisioning the physical space of play lab participants (Friedgen 2020).
Figure 3. Defining and visualizing the parameters of space and control within the interface of Zoom (Friedgen 2020).

In addition to Ahmed’s writings, the work of Wrights + Sights called to me as I began to think about applying play lab learning into a prototype performance experience. They collective calls their work intentionally “porous; for others to read into and connect from it and for the specificities and temporalities of sites to fracture, erode and distress it” (Hodge 2018). This appealed to me as a way for an audience to take multiple openings for connection through innumerable paths. My multiple experiences with Sleep No More kept calling to me as I seek ways to create both a shared experience and an individual, personal one through this project. The exploration of both iterations (Play Lab series and Avatar Adventure) attempted to build on the idea of a crafting a porous experience while playing with the benefits of both a synchronous and asynchronous design.

In addition to written knowledge, I turned to some examples of performance that shared a connection with my envisioned project. I began by viewing/experiencing theatrical events produced and distributed through Zoom or another digital format. Notably, three experiences immediately fed into my design for the play labs and prototype performance: Hiawatha Project’s My Traveling Song, Not A Theatre Company’s Play in Your Bathtub, and a Zoom performance of Rabbit Hole in which Neal performed as Jason.

My Traveling Song

While originally a live TYA (theatre for young audiences) performance that included a tactile element in performance, in the wake of the pandemic the producers, The Hiawatha Project, released an at-home viewing of a filmed performance accompanied by a set of instructions to recreate the tactile elements at home. I experienced this performance alongside my nephews (ages 5 and 2.5) who rejoiced in the tactile parts of the performance and quickly lost interest in the songs that filled the moments in between participatory cues. The strength of this piece lay in the various textures of the practice elements which sought to recreate the feel of wind, rain, falling leaves, and earth. Because I was facilitating my nephews’ experience of the performance, I turned often to the props that we were encouraged to use to keep them engaged. However, this actually distracted them from the formal aspect of performance as the narrative carried us from element to object/prop.

Learning from this performance, I sought to incorporate tactile elements of participation into the play labs through activities such as Narrative Drawing and Storied Objects. I also noted that once participation begins it is important to craft any ebb or flow in a way that scaffolds up or down elegantly in order to not disrupt either the performance or the participant’s experience of it.

Promotional video for My Traveling Song.

Play in Your Bathtub

Produced by This Is Not A Theatre Company, this “performance” is described as a “site-specific interactive audio play that takes place in your own bathtub” (“Home” 2019). The company’s work overall seeks to engage on a multi-sensory level and created fully embodied experiences for their audience. In their manifesto, founders Erin Mee and Jessie Bear write, “I don’t love linear theatre that builds to a single climax followed by a cigarette. I like non-linear theatre with multiple orgasms” (Mee and Bear 2019). As far as participatory theatre is concerned, I would agree with Mee and Bear. Porous and non-linear theatre with many opportunities for an audience member to climax on their own, as it were, imprints as an experience on the participant.

While Play in Your Bathtub was more a meditative experience for me, it provided some interesting ideas for asynchronous participatory performance. The invitation strategy and structure of participatory opportunities intrigued me. Every activity or prompt provided was obviously optional – no one else was in the room with me to ensure that I completed every invitation extended. However, if I opted out of an invitation, I was still embedded within the experience of the performance – feeling the water caress my body, smelling the scents of my candles, and tasting my tea all grounded me in the performance site as I let the spoken prompt on the audio recording go unanswered. Sitting out of the activity and inside of the experience, I began to notice a form emerge within the participatory activities. This form reminded me of how jazz music is structured – the melody plays through, perhaps is repeated, then solo improvisations come on top of the form of the melody, perhaps an improvised duet occurs, and then we return to the original melody which now is informed by the discoveries made throughout the journey of the song. Reflecting on this discovery, I noted this idea as a way to scaffold in participation through an immersive performance.

Rabbit Hole “Zoomie”

Figure 4. A screen capture of the opening scene from Rabbit Hole in a performance adapted for Zoom.

Just before Neal and I met for the first time to experiment with activities for the play lab, he happened to be performing in a version of Rabbit Hole that was adapted for Zoom. This performance re-envisioned the play as if all of the characters were connecting on Zoom rather than in real life. I found myself viewing this production from the perspective of a director/choreographer. The sparseness of the visual landscape and the flow and choreography of webcams and Zoom squares made me curious as to how one might stage-manage or control the appearance of a Zoom call for an audience member. Neal’s experience as a performer on the piece provided him great knowledge in both the technical side of things and performance and presentation strategies which he brought to both our planning sessions and play labs.

Figure 5. Neal (at bottom left) captures his experience of performing in Rabbit Hole on Zoom from multiple perspectives.

Finally, my personal and professional interactions with Zoom helped me to build knowledge of the platform itself and that knowledge directly impacted the orientation and use of the play lab space. Zoom’s design includes the use of space associate language – “room”, “open/closed”, “Waiting Room”, etc. – which allowed me to visualize the participants’ experiences within the context of a virtual space.

Project Design

This project has two main components two it – a series of play labs and a prototype narrative performance. The play labs seek to explore and experiment with different strategies for audience participation on Zoom. While the prototype performance will be conducted via Marco Polo for the ease of the test audience, the goal will be to use techniques and discoveries from the play labs in the course of that prototype performance. Working with collaborator and performer, Neal Davidson, during the play labs and applying the methods garnered on my own in the prototype performance, I hope to determine what engagement strategies are most effective at capturing attention and delight while balancing the technological experience of a diverse audience.

Developing the Play Labs

Neal and I met three times on Zoom to develop several ideas for initial audience participation and interaction on Zoom. Our brainstorms led us to generate activities that we categorized into five types of audience interaction:

  • Game/Competition
  • Collaboration
  • Viewer to Participant Transformation
  • Sense Play
  • Power Transfer

To test out our ideas and further devise more strategies, I set up play labs over Zoom which would serve as generative and experimental workshops. Each 90-minute play lab included 3-6 participants, Neal and myself. I developed a script for these workshops, which we would refine and adapted with each iteration of the play lab. This allowed us to be responsive to feedback between sessions and refine our methods of pursuing the research question.

Assessment Design

To assess our progress and process, I will embed several opportunities for feedback from both participants and researchers. I want to gather qualitative feedback in the moment as well as after the experience for both the play labs and the performance prototype adventure, therefore I will incorporate reflection opportunities at certain points of both play lab and performance prototype. I plan to analyze this feedback for trends to help guide my hunches as I further develop this approach to participatory theatre-making. I intend to cultivate feedback that will help me to narrow my scope of inquiry as I learn what audience-participants respond to through a virtual format of engagement.

Ultimately, primary assessment data came from participants in the form of open reflection, prompted discussion questions, surveys, and video voicemails on Marco Polo. In addition to the data collected from participants, Neal and I engaged in reflective dialogue during key moments of the play lab, before play labs, and after the entire series of play labs completed. I have recorded all of our play labs and planning and reflection sessions on Zoom and have stored the video footage, audio recording, and transcripts as a record of this project. I also kept a journal record of each play lab (pictured below) where I kept a record of the schedule for the play labs, the participants, detailed comments from participants’ experiences and further ideas for continued iteration on the activities.

Play Lab Discoveries & Reflections

About the Play Labs

For the purposes of this midrange project, I outlined a series of play labs based on the initial discussions and meetings Neal and I had into three broad lines of inquiry:

  • Play Lab A: game play that utilized Zoom’s software and the participants’ physical space
  • Play Lab B: avatar experiences that explored camera manipulation
  • Play Lab C: sensory play and power transmission

We primarily executed activities from the Play Lab A outline, however, in our final play lab we explored some of the activities initially scripted for
Play Lab B. We conducted three play labs (Play Labs A version 1-3) over the course of a week and included 16 participants total. We had a fourth play lab scheduled but elected to push this play lab to the next phase of development due to too few participants.

Drawing Activities

Each play lab group played some of the same activities, particularly Pen Fight and Narrative Drawing. Both of these games experimented with sketching as a means of communication. In Pen Fight, two participants battled for control of the pen as they sought to draw an object in their own space. Narrative Drawing brought participants back to their kinesthetic spheres by asking participants to draw (on paper with pen) something that a participant described to them. While these two games had similarities, the two platforms of gameplay helped Neal and me to discover some key desires from participants and potential audiences.

Video of Pen Fight being played in its second iteration (Zoom Play Lab Av2).

Our first two play labs included all of the same activities (Pen Fight, Narrative Drawing, and Storied Objects), but we played with slightly different facilitation of the activities. Some notable differences included:

  • exploring the dynamic between pairs & trios
  • experimenting with the concept of team-building and competition
  • adjusting parameters for the found objects from general to specific
  • incorporating formal written instructions vs. use of the chat

The third play lab included the most variation. We played Narrative Drawing and eliminated Pen Fight. However, we let the participants pick the winner of Narrative Drawing. Additionally, we substituted Avatar Space Explorer and Hidden Objects for Storied Objects in order to explore using a phone camera, a suggestion that was shared in Play Lab Av2. The structure of welcome, introduction, and reflection remained the same throughout the three play labs, although the final reflection questions in Play Lab Av3 changed to reflect the nature of that play lab’s activities.

Almost every participant who turned in the post-play lab survey identified Narrative Drawing as a highlight of their experience, 78.8% noted that they enjoyed this game as a participant, while 85.7% enjoyed it as a spectator. Conversely, Pen Fight was the least enjoyed game as either a participant (28.8%) or a spectator (14.3%). This feedback leads me to believe that a connection to the physical world could be vital to the audience’s experience of this medium, particularly as a means to orient a participant within their own physical space or someone else’s.

Another important thing to note is the concept of community over competition. Throughout each iteration of the play lab, we played these two games first. The goal was to scaffold the group up to a competition. Some play labs facilitated this build-up better than others, but no matter how well facilitated or not, we consistently got feedback from participants that they did not enjoy the moments of competition (usually through Pen Fight) nearly as much as the moments of collaboration and community building (that usually occurred through narrative drawing). There could be many factors for this – the current crisis moment, a lack of familiarity with others in the play lab, a dislike for competition in general, no clear objective. Upon further reflection, but Neal and I agreed that without a clear objective or reward attached to it, competition should be avoided. Several participants felt some level of insecurity or awkwardness of not knowing who to cheer for or how to cheer over one another.

However, people overwhelmingly responded positively to sharing or being included in a moment of sharing a personal item. One participant noted, “Any inclusion of objects in my personal surroundings felt very engaging and was a great way to bridge the gap to our online world. Thank you for taking the time to facilitate this!” (“A IZPL Feedback” 2020). In all of the play labs participants commented on a shared enjoyment of bringing their own spaces or belongings into the experience and sharing them:

“Being able to interact with our space and things in our space was really nice. Sharing those and being able to write something and share it versus doing it on the white board… there is something that is very engaging about looking around you and bringing your space into this virtual world.”

“I love that you were taking advantage of the features of zoom, like the whiteboard, but again it’s very tangible to hold onto something in your environment. Using the objects in our space was very tangible and ground.”

“Bridging physical and virtual is interesting. What I wanted to see someone do was take advantage of other zoom features of the whiteboard to make a set [for their Storied Object performance].”

“I love the way that an in-person, in the same room experience was part of this. That was really satisfying [to see] other people interacting with each other.”

Some excerpts of feedback from the play labs.

On a personal note, I found myself drawn to the evocative nature of the images created by Pen Fight. While each player worked at cross purposes, the explosion of digital ink and any given moment into shapes and lines and textures looked more collaborative than competitive when viewed in isolation. However, the surrounding elements (sound, dialogue, etc.) kept it firmly in that competitive place.

Storied Objects

Storied Objects was an activity Neal and I formulated as a way to come out of a found object challenge in the workshops. It essentially asks participants to devise a 1-2 minute story using found objects as props, puppets, dancers, etc. The discoveries generated during this activity have led to major findings in how Zoom and other digital video synchronous performance might be utilized by performers to create a more immersive experience and further connect collaborators who are physically apart, but virtually together. From shared objects, to camera angles, reflective surfaces to depth of field, the creative solutions that the participants found while navigating Zoom as a platform for storytelling directly impacted my experience as performer/director while developing the prototype performance, Avatar Adventure.

I found the collaborative and generous spirit of the participants in each play lab extremely helpful in refining my own knowledge and thinking about strategies for audience participation in the context of Zoom. As we are currently at week six of quarantine in the United States, most people have some familiarity with Zoom from business meetings. This proved useful when navigating the app, but also allowed for quite a bit of surprise and delight as we embraced different uses for Zoom’s features. In fact, all of the participants who completed the post-play lab survey indicated that they felt surprise during the course of the play lab, as well as excitement, curiosity, and amusement (“A IZPL Feedback” 2020).

As I reflect on the relationship between the virtual and physical worlds, I personally feel great delight when these worlds can align and “kiss” as Tectonic Theatre Project might refer to it. The almost kismet discover of a connection with another person across time and space has a strong sense of satisfaction. One such moment for me occurred during Play Lab Av2, when two members of a team working on storied objects had the same object (see Figures #-#). This created continuity in their performance of Storied Objects that filled me with joy.

In addition to sharing objects, people also explored how objects might meld at the perimeters of their screens. Two participants in Play Lab Av1 (Figure #), incorporated movement, depth of field with their camera, and physical alignment to try to marry their objects in the same world. While this intended viewing experience sometimes aligned during the play lab, we discovered some challenges to unifying the experience for everyone. For instance, Zoom always places the individual’s view of themself as the second window screen when looking at Gallery View. So while one of the performers in this session viewed himself in one position on his own screen, the rest of us saw him in a different place. This discovery identified a potential obstacle toward utilizing object passing (between screens). However, the trio in Av2 overcame this issue by using the center of the screen as a point of transference for the object.

Reflection and depth of field were two elements utilized in each of the play labs in a variety of ways. During Play Lab Av1, two groups utilized depth of field with reflective objects. This created an eerie sensation that the other player was physically in the same room with the manipulator of the reflective object. For future iterations of Play Lab A, I requested each group to try to locate a reflective or transparent object as part of their devising work.

Depth of field perhaps is the most obvious manipulation of space as it is the easiest to achieve – simply move a part of your body closer to or further from the camera and the image on the screen appears more three-dimensional. This tool hit me most profoundly after Play Lab Av2, when a participants’ daughter jumped on the call and shared a puppet show she had created in minutes with paper cut out (image below). This young girls’ work coupled with the “fate” moment of two participants sharing an objecting inspired me to design another iteration of workshops in which I design and send out paper cut-out figures that participants can use to create continuity in storytelling.

Figure 15. Depth of field through puppets.

To present each group’s performance, we experimented with the presence of the audience on Zoom in Play Lab Av1. Initially, viewers muted their microphones and turned their cameras off. Neal and I accidentally had left our microphones on though and laughed at one point during the first group’s performance. The laughter from the audience felt familiar and reminded me of sitting in a darkened theatre sharing this experience with a stranger. The next performance we tried with viewer’s cameras off, but their microphones on. I found this effective for bringing the audience into the communal experience of the performance via zoom in the moment. However, this protocol could present challenges in a larger setting. In future play labs, at one point I experienced an echo from someone’s microphone picking up my voice and sharing it back over zoom. Similarly background noise in one viewer’s location could become distracting and upstage the intended experience. During the third performance in Play Lab Av1, we played with turning microphones off and having all the viewers’ cameras on in order to witness the audience watching. We took a moment to reorder the videos so that the performers would be proximal on the viewer’s screens when viewed in Gallery Mode. While the visual information of seeing the audience faces generates an interesting barometer for feedback, viewers collectively agreed that this created some distraction from the performance.

Figure 16. Storied Objects in performance during Play Lab Av1. Both the performers and viewers are visible.

Additionally, the amount of screen real estate that the viewers’ squares took up made it difficult to see certain details of the performance. Below is a screen capture from the second time this group performed. Having this group perform again allowed the viewers to see and understand their piece in greater detail, notably the work with the mirror.

Figure 17. Storied Objects in performance during Play Lab Av1.

Play Lab Av2 also introduced the concept of a faceless narrator or disembodied voice as an element of performance that can be effectively implemented over this medium. The first group in this play lab included three performers. The two on-screen served as puppeteers and performed the visual half of the story while the third member of the group narrated the story. The second group also used three performers, but two of the performers (a mother and daughter) were in the same room. However, the daughter hid under a table out of frame and provided the voice for a helmet puppeteered by the third member of the group. In the video below, an excerpt from each example described here can be viewed.

Video excerpts from Play Lab Av2’s Storied Object performances. Example 1 from 0:00-1:04. Example 2 from 1:05-2:06.

These groups also both utilized green screen in their performances. Personally, I find this feature to be somewhat disruptive to the viewing experience most of the time. Particularly because Zoom really requires some sort of actual green screen to be truly effective. Neal and I played with this using a green notebook that I had using both image and video in as a virtual background. Interestingly enough, the effect seems to work better when foregrounded and manipulated in this way. The video below illustrates some of the ways in which I explored proximity and spatial relationship with the green screen.

Video exploring efficacy of green screen on Zoom.

Overall, Storied Objects proved an effective activity for generating ideas and strategies that could be utilized in devising engaging performances over Zoom. Some of these ideas Neal and I had hit upon in our early brainstorming sessions, but much of the participant-generated ideas were new or improved variations on our initial ideas.

Avatar Space Explorer and Hidden Objects

One of the notes Neal and I received in Play Lab Av2 was a curiosity to utilize phone cameras which have more agility than a web cam to move about and change angles. The Play Lab B activities were more centered around this style of engagement, so I pilfered some of those ideas and brought them into Play Lab Av3. Avatar Space Explorer asks a participant to walk us through their space with their camera facing out so that we see what they see while another player guides them or directs them through the space. Hidden Objects centers around an imagined circumstance that a third-party has hidden an object somewhere in your space and a narrator guides you through a provided script to find the object. Playing with both of these activities, along with an experiment to mix the two that developed from immediate feedback in the play lab, provided a wealth of information on the possibilities and challenges of pursuing avatar activities.

In the pre-play lab reminder email, I asked participants who already had a phone with the Zoom app on it available to bring those to our Zoom meeting. Only two participants had this at the beginning of the meeting in addition to Neal and I, so we both took part in exploring the use of the phone camera through zoom at different phases of this play lab.

We began by demonstrating how the phone camera should function with my own camera. One participant volunteered to tour us through her space as Neal (in the role of Guide) prompted her. Other participants were invited to privately chat Neal with suggestions of how to direct the avatar participant through their space. I performed the role of “God” in this example and determined when the activity would come to an end. A video excerpt from this activity is included below.

Video of Avatar Space Explorer’s initial launch.

We garnered quite a bit of feedback about this activity. The participant in the role of avatar shared that it “helped [her] to see [her] space in a whole new light” while viewing participants shared that they enjoyed indulging in the “curiosity of seeing other’s spaces” (Play Lab Av3 notes). One participant remarked that she “felt like everyone could participate [through providing prompts in the chat], but it felt like a mystery of how your contribution could add [to the experience].” In later discussing this activity, Neal shared that this participant was the only one to send him prompts in the chat. I believe this speaks to some confusion over what to ask or prompt and a general concern shared by participants in the avatar volunteer being “put on the spot” to share their space.

Hidden Objects encountered some kinks in the facilitation. We were short of players as one participant who was signed up never showed and I had to step in as a player. The group that Neal facilitated finished rather quickly, while my group met with several challenges and took much longer to complete the activity. Confusion between the purpose and responsibility of roles within the activity made it difficult for participants to enjoy themselves and several noted that they felt a clear objective was missing from this activity. Given that Hidden Objects was born out of a narrative concept exploration I had done while brainstorming for the prototype performance portion of this project, the frustration over a lack of objective makes a lot of sense to me. I am not sure that this particular activity works well divorced from a narrative structure, but implementing it in this play lab allowed participants to provide us with great ideas for the next series of play labs. Some notable ideas include:

  • scaffolding in tutorial on ways to manipulate and use the camera
  • build up to sharing commands for the avatar
  • providing each participant a particular score to perform with the camera

These games also integrated the chat feature interestingly through private messaging which many participants enjoyed. Building off of the feedback from Avatar Space Explorer and Hidden Objects we explored a mix of the two using Neal as the avatar in search of something “hidden” or perhaps “lost” or “forgotten”. A participant served as the guide while other participants used the chat function to message her ideas for how to direct Neal through his space. Again, I returned to the role of God, but this time I facilitated the ending which was determined democratically by asking the viewing audience to send me a private message if they thought Neal had “found” the object.

Video excerpt of our attempt to combine Avatar Space Explorer and Hidden Objects in Play Lab Av3.

Neal’s facility and experience with film and live video helped to make this variation of the activity successful. He later reflected on the importance of “giving the audience an experience” as he manipulated the camera – choosing when and what of his body to include in the shot or how to use proximity and depth of field to change the viewer’s perspective of his space or objects. This makes me wonder whether facility with the technology or its artful implementation can be transferred to an audience participant and how I might account for this within the scope of devised participatory performance.

The use of camera phones through the avatar activities furthered understanding along my orientation inquiry. One participant shared that “moments of orientation for me [occurred] when I understood where you were in space and I related with my body to how you were with your body. …When you had to shift and move around and … didn’t see the transition moment that [prompted] disorientation.” Each time we explored another person’s space, I learned more about camera manipulation to elicit a perspective change as well. These understandings were directly implemented in the prototype performance I created in the second iteration of this project.

Utilizing the Chat Subversively

In previous Play Labs, we had made use of the Chat as a place to include directions for those who prefer a visual cue for the activities. In Play Lab Av3, however we explored private messaging throughout the play lab in various ways. One participant mentioned that they felt like they were sharing a secret through the private chat feature. A perceived sense of intimacy or connection developed between Neal and the participants, “I really liked being in the chat, messaging, and throwing out random ideas. I really enjoyed when Neal would choose something that I would say, I was like ‘Oh he chose mine!'” Playing with this idea, I encouraged Neal to send the participants private messages commanding them to perform certain actions as we continued a group discussion reflecting on the previous activities. The image below shows one moment of three participants collectively performing a command to “look at the ceiling”.

Figure 18. Neal directs some participants through use of the private chat feature.

The private space of the chat opens up interesting opportunities for subversion and coalition-building within a performance structure. It could allow performers to cue audience participation through a secret signal or create a depth of experience as one character builds subtext through text for an audience (collectively or individually) in juxtaposition to the moment being viewed. The chat feature, along with the breakout room feature, provides opportunities to explore virtual space and intimate connections in the context of a narrative in similar ways to the experience of being in a live immersive performance such as Sleep No More or Then She Fell.

Assessing the Performance of the Avatar Adventure Experience

Following up on some of the participatory strategies and techniques explored in the Play Labs, I decided to create a performance prototype for members of my PaR class as a proof-of-concept experiment. Because the format of presentation needed to be asynchronous and this particular group already had knowledge of Marco Polo, I used that app as the platform for performance. While I normally prefer to approach the invitation to participate fully in role, I found it necessary to provide some clear instructions to clarify my research aims and let participants know the expectations if they chose to participate. Because I wanted to see if this style of performance could hold an audience’s attention, I allowed participants to choose not to engage in the whole performance if they preferred not to. Following the end of their experience, I sent each participant a survey to collect information about the experience overall, their responsibility and perception of their role within the experience, and why they elected not to complete the experience if they did not finish.

Video introduction to the prototype performance, Avatar Adventure.

This video provided the set up of the narrative and included two options for roles that they audience could choose to fulfill – either assist in helping Kristina (the avatar character) escape and reach Margot or direct Kristina back to her room in her place of confinement. Both roles allowed for the player to elect to play honestly or covertly as they directed me through the game. Perhaps because every player knew me or perhaps because they are all caring and empathetic people, every participant (to my knowledge) elected to help Kristina escape. However, I experienced a sense of mystery in wondering how honest each player was with me in their aims. The first participant managed to lead me into my bedroom and down to the deepest levels of the house, which made me wonder if she was truly an ally in helping Kristina escape. The dissonance in my firm orientation in this space and what could be translated through videos helped to keep participants on their toes to a degree.

Exploring Orientation Virtually

Interestingly, Marco Polo’s format helped participants confirm their orientation by allowing them to review videos that had been sent and check the surroundings. One player (A) remarked, “I developed a strong familiarity with the main spaces we explored and did go back to prior videos to try to find clues and orient myself” (Avatar Adventure Experience Feedback 2020). Out of the five participants who responded to my survey, they seemed to indicate a mostly moderate orientation with my space. However, Player A, the one outlier who felt very oriented also had the most contact with my space. We exchanged 81 video messages through the course of the game. In addition to the extensive play likely contributing to Player A’s strong sense of orientation was some previous knowledge of seeing the space, “knowing this space some from prior interactions was interesting though I spent my time almost exclusively in other rooms” (Avatar Adventure Experience Feedback 2020).

Figure 19. Participant responses that demonstrate a range of orientation experiences.

Of the three other participants who responded to my survey, one (Player B) did not make it out of the “Mess Hall” (the room where Kristina initially began the adventure) and the other, “didn’t get to finish. [she] was confused by the last video [she] received and didn’t know if the experience was over” (Avatar Adventure Experience Feedback 2020). Player B and I only exchanged six messages, while Player C and I exchanged 17. Player D, who did finish the game was the six-year-old daughter of one of my classmates, but from her feedback, she seemed to orient best in the interiors of the game. When we finally made it outside, she struggled to direct me to go “left” or “right” and provided more vague instructions like, “which one looks more safe.” While more open-ended, these responses actually made it easier to facilitate her experience as I was able to set her on the course I filmed for Player A.

Producing and Performing the Adventure

To create the Avatar adventure I generated videos either in Marco Polo or through my phone’s camera. Anything that required multiple angles was edited using TikTok and then uploaded to Marco Polo for viewing. Player A’s experience was fully crafted from original content and therefore was completely responsive to her suggestions. For other participants, I used a mix of original content and prior video. If necessary, I would record a new voiceover for a prior video to be responsive aurally to the player. While this mostly worked well, I ran into some technical issues for the videos that I originally recorded in Marco Polo. When saving these videos to my phone the video quality degraded too much for me to edit some using TikTok.

Video that features some of the experiences of performing and producing an asynchronous participatory performance.

Filming content outside of Marco Polo is also necessary to not disrupt another player’s experience of the game. While navigating Player A’s feed to find and forward a video to another player, she happened to see me in the chat and was eager to see if there was more to the adventure. Also, while reviewing some videos that I had created solely in Marco Polo, I noticed that I made some consistency errors – leaving a door open that should have been closed in one shot, forgetting the “veil” for one shot of the mess hall, and in the wrong costume for a shot that needed to have my face in it, for instance. I was able to catch this mistake with one of the videos I filmed for the ending sequence – I had left Margot’s sunglasses on while filming a shot of Kristina taking a certain path. Luckily because I didn’t shoot it live for Marco Polo, I was able to catch this inconsistency and not further confuse the player.

I met with some challenges of responding to participants at their convenience. Because the adventure began in a place that connects to one nephew’s room and requires going through his bedroom to get to a portal with four possible exits out of the house I did not always have access to this crucial space. The participants who didn’t finish the experience all began it during the evening hours right before this nephew went to bed. At that point, I lost access to this space and couldn’t continue facilitating their experience until the next morning. Despite the necessary interruption to game play, most participants did not mind the lag time between communication, with the exception of Player D.

Occasionally, the lag time was necessary to film and edit the content together. These were the moments that I lived in fear that the participant would change their mind while I was editing their previous suggestion for view. Luckily, this did not happen, but it emphasized the need to film ahead of time if I continue this style of performance in the future. Ideally, the performance could be set up to be extremely responsive and a new video would be sent within seconds or minutes of a direction from a participant. However, I wonder if the lag time impacted the suspense of wondering what would happen next? All participants reported feeling excitement and surprise during play, and 3/5 reported feeling impatient. It merits some further investigation into lag time as useful or distracting from the participant’s experience.

Overall, I received robust feedback from all five participants which has me wondering about the variety of uses for a virtual format of theatrical experience in a post-COVID world. The video included below features one player’s engagement and strong connection with the performance. While my perspective of her play was a solo play, she had been sharing her experience of the performance with her partner and a friend via Marco Polo.

Video featuring one player’s response to the ending sequence.

Reflections on Proof of Concept

Each phase of this project, from research to experimentation to implementation, helped me to understand more about the landscape and feasibility of creating in-home immersive theatrical events. Furthermore, the work thus far has piqued my curiosity about the implications this format of participatory engagement might have to connect and affect change in a theatre for social change context. As I reflect upon my initial research questions, I have made inroads toward further understanding and knowledge along each line of inquiry that directly inform my plan for a long-range PaR project.

Reflections on Research Questions

In what ways can Zoom be used as a platform for an in-home immersive theatre event?  

The virtual space of Zoom and its software features create interesting opportunities to collaborate with audience participants virtually, but also present some limitations in presentation that devisers and experience designers should be aware of. Through our play labs, we discovered how Zoom’s group chat and private chat features might be used to subvert the main action of a performance over Zoom as well as create suspense and delight by creating a space to communicate one-on-one with performers and individuals. Neal and I have also discussed how we might orchestrate secret pockets within the experiences through the use of breakout rooms. Interestingly, once a participant is in a breakout room, they can choose to exit back to the main room and return to that breakout room (as long as it remains open), but if they are assigned to a different breakout room, they lose control of their navigation until their screen has re-oriented within the new breakout room. In testing this out during a post-feedback session in Play Lab Av1, a “down-the-rabbit-hole” sensation emerged for the participants as my commands whisked them between breakout rooms.

While Zoom excels at creating virtual space for connection between people on different levels, the user experience is highly dependent on the participant’s access point. Participants who used the browser version of Zoom or who called in through an iPad noted an extremely different experience of the play labs than those using the app on a computer. It is also not possible to entirely control the viewing experience of participants and align it with the performer’s view. Technology also created a hindrance in an activity like Pen Fight which used the whiteboard. The Whiteboard seemed to be more responsive to players who either ‘owned’ the whiteboard (meaning it was their screen we were sharing) or who used a stylus to draw. Lag time also affected participant’s ability to succeed at whiteboard activities.

Zoom provides an opportunity for a larger group of people to share an experience synchronously that seems to be consisted across the majority of users who participated in our play lab. While we did not explore every feature of Zoom, the A series of play labs allowed me to better understand how to elegantly facilitate Zoom for performance and provided me with many tools for further research and experimentation.

What interactions (between strangers, the virtual space, and the physical space) are most satisfying to an audience participant?  

Most of the play lab participants seemed to enjoy opportunities of collaboration and intimacy building between people over moments of competition or conflict. Whether working together to create a performance or trying to draw something described to them, participants overwhelming agreed that the interaction and participation strategies elicited amusement (100%), excitement (92.9%), and surprise (92.9%). As we further began to explore the chat function in the final play lab, we discovered another avenue to engage participants individually that felt satisfying to the participants, Neal and I.

As I pivot the findings from the play labs into performance form, I begin to wonder about the individual vs. the communal experience in relationship to a protagonist or an antagonist. Through the Avatar Adventure prototype performance, I have become even more curious about the grayness that might surround a role – could an audience participant be enlisted to further both the antagonist and protagonists’ objectives or must they choose (or be guided into serving) only one side of the conflict?

How can an audience participant become oriented, disoriented, and re-oriented within their experience of a virtual video encounter?

Virtual video encounters, whether over Zoom or Marco Polo in this project, can manipulate an audience’s perception of orientation and disorientation. The play labs generated many strategies for creating the sensation of disorientation:

  • use of repeated/shared objects
  • depth of field (like the tunnel/tube)
  • green screen effect
  • reflective and transparent objects
  • camera movement

which I then utilized in the performance prototype on Marco Polo. In both platforms, orientation was achieved by sympathetically sensing the viewed body’s relationship to space and then orienting in one’s own body. One player in Avatar Adventure performance shared that she would move along with Kristina in the videos to help get a better sense of where to go next. Additionally, some participants found connection with their own surroundings. One shared that she “could go on an adventure like this at my house” (Avatar Adventure Experience Feedback 2020).

In a future iteration, it could be interesting to explore multiple perspectives within the world of the play. One player suggested the option to do “Marco Polo exchanges with Margot or the jailer” which could provide more opportunities to virtually orient the player in the physical space. Other possibilities for improved orientation would be to repeat the objective of the game in Marco Polo or note mini-objectives for each phase of the game to the participants. A few remarks from participants lead me to believe that a struggle to orient in the space actually turned them off from playing, such as “The camera moved quickly, so I had to play the videos more than once. I tried to locate objects that would be useful in moving forward, but I didn’t get very far” (Avatar Adventure Experience Feedback 2020). Because I am so familiar with the environment, I thought it would be too easy to “escape” the first room and get out of the house (there were only three physical doors between me and the outside). This belief led me to utilize added levels of disorientation, such as the “veil” or color filter that wouldn’t be lifted until the player led Kristina out of the mess hall. It took all but one player between 3-5 exchanges just to get me to the outer room of the mess hall. Only Player A provided enough instruction to get me to the door. Interestingly, Player A’s first video exchange was edited with TikTok, which means it had multiple angles. Perhaps if I can guide the participant to giving me enough ideas in the first few exchanges, I can compile those into one video response that gets me out of the mess hall quicker and into an option with a door sooner?

Implications for this Form of Performance

Perhaps what excites me most about the results of both the play labs and the performance prototype is the potential for continued audience or community engagement outside of the bounds of a live-theatrical performance. Especially in relation to a theatre for social change piece, I believe that a play by itself cannot provoke an audience into enacting societal change. However, a theatre company, community, or organization might utilize a play or a theatrical experience to highlight, expose, present, or communicate about a social or community issue, experience, or perspective. Working along these lines, an immersive, virtual performance form could allow the audience to explore the possibilities that proposed social change might present, or experience the world through another’s perspective, or test out a line of action and safely experience the consequences that might occur.

Of those participants who interacted with the Avatar Adventure, 80% reported feeling a sense of responsibility and concern throughout the game while 60% reported empathizing with the avatar. Fifty percent of play lab participants who completed the survey noted feeling compassion during the play lab. While these are not conclusive findings, these numbers prompt me to wonder how this performance form might be further explored and shaped to steer into an experience of communal empathy or open up possibilities for sharing in someone’s lived experience through the privacy and convenience of the viewer’s own home.

Forms response chart. Question title: What level of control did you feel that you had over the course of the story?. Number of responses: 5 responses.
Figure 21. Participant’s experience of control over the performance.

One key element to this format is designing in a level of player control and autonomy that generates a sense that the play experience is fully responsive while keeping the performer safe. Overall Avatar Adventure participants reported a sense of extreme or strong responsivity, but two participants noticed potential threat levels that they had to navigate around:

“I was nervous I might ask Kristina to do something that might go too far or cause her new problems in the storyline (such as an entanglement with the police) because she was so responsive to my ideas.” – Player A

“I felt it was [responsive] but I didn’t feel like I could really help as much as I wanted to because I wasn’t sure I knew all the rules! Did land mines mean we were both gone forever?” – Player B

(Avatar Adventure Experience Feedback 2020).

I find the investment of care and concern for me as a real person carrying out these actions to be crucial to the resulting care and compassion that participants felt during play. This might also create opportunities for building communal empathy by allowing audience members who are outside of a particular group experience to sympathetically embody the perspective and lived experience as someone from within that community.

For more on the future of this project, read on.

Figure 22. Play Lab participants working to beat the clock in Narrative Drawing