Learning by doing: Seeking best practices for immersive journalism

Ernest Wilson, the dean of the Annenberg School of Communications and Journalism, put it like this: What if, after receiving the home and garden section in the morning, the reader could walk right into the section and visit a garden? This bucolic vision reflects one potential scenario for what we are calling at Annenberg “immersive journalism,” a new genre that utilizes gaming platforms and virtual environments to convey news, documentary and non-fiction stories. As a senior research fellow, I am prototyping immersive journalism stories, hoping to discover and create best practices for a burgeoning filed that can capture audiences increasingly accustomed to experiencing digital worlds.

The fundamental idea of immersive journalism is to allow the audience to actually enter a virtually recreated scenario representing the news story. The pieces can be built in online virtual worlds, such as Second Life, or produced using a head-tracked head-mounted display system, or HMD. An HMD is a lightweight helmet that has screens covering the eyes and tracks head movement so ensure digital imagery on the screens stays in perspective to create a sensation of having a virtual body in a virtual location. Immersive journalism can also be constructed in a Cave, which uses full body-tracking technologies in a small room so that individuals can move their bodies around the space.

Visual and audio primary source material from the physical world reinforce the concept that participants are experiencing a nonfiction story, with the video, sounds or photographs acting on the narrative. For example, video triggers at key points in the virtual landscape to remind a participant that the computer generated environment is grounded in the physical world. Scripted events that create a first person interaction with the reportage can also help create a feeling of “being there.” Also, participants can query or interact with the elements around them to learn more about the details or context of the news story.

In general, participants travel through the story as a digital representation of themselves, or as one of the subjects in the news piece. Whether visiting the space as oneself or as a subject in the narrative, immersive journalism aims to afford the participant unprecedented access to the sights and sounds, and possibly, the feelings and emotions that accompany the news.

Well-crafted journalism always aims to elicit a connection between the audience and the news story. Creating that connection via different kinds of ‘immersion’ has long been considered ideal. Describing her reporting during World War II, Martha Gellhorn called it “the view from the ground.” Writer George Plimpton famously joined the Detroit Lions American football team in order to give his readers the most intimate sense of playing on this team. Television news correspondent Walter Cronkite made a series of documentaries recreating historical events in which he would offer a brief introduction before an announcer would give the date and the event, proclaiming, “You Are There!” More recently, attempts to combine audio, video and photographs on the Internet have created what some journalists call “immersive storytelling.” As technology editor at MSNBC, Jonathan Dube (now Vice President at ABCNews.com) said that he believes this can bring the reader or viewer “closer to the truth.”

In collaboration with digital media designer Peggy Weil, we have built several prototypes which reflect my interest in covering human rights issues. Gone Gitmo, a virtual Guantanamo Bay prison built in Second Life, allows participants to explore a place that is inaccessible to the average American citizen and press. (In fact, the Pentagon just expelled four reporters who have been covering the prison for years.) Gone Gitmo includes an experience on what it might be like to be detained, hooded and then imprisoned in Camp X-Ray. It also examines the ramifications of losing habeas corpus rights.

Another Second Life prototype, Cap & Trade, is a news report on the carbon market that sends people on a journey to follow the money in order to try to better understand the complexities and human consequences of trading carbon credits. Cap & Trade was built in partnership with the Center for Investigative Reporting and Frontline World and is particularly reliant on the excellent reporting by Mark Schapiro that appears on Frontline and in Mother Jones and Harpers Magazine.

A third prototype is based on the interrogation logs of Detainee 063, Mohammed Al Qahtani, who had been declared tortured by the Bush administration. Built at the Event Lab in Barcelona with Mel Slater and his team, we use an HMD to put participants into the virtual body of a detainee who is held in what is referred to as a “stress position.” When participants look around, they see a virtual mirror with a digital figure in that mirror who looks like a detainee and moves in unison with the participant. Participants also wear a breathing strap that programs the avatar to breathe at the same time as they do, further enhancing the sense of virtual body ownership. Throughout, the sounds of the Al Qahtani interrogation play as if coming from the next room. While research data was not collected on this particular prototype, every participant anecdotally reported that their body was hunched over in a stress position when in fact they were sitting upright.

Immersive journalism is distinct from news games in that news games embrace gaming protocols. With news games, the player undertakes a task or pursues a goal, voluntarily constrained by agreed upon rules, and must take action to advance position. Progress is often measured by indicators such as levels or points.  In contrast, a participant in immersive journalism isn’t playing a game, but is put into an experience where she is participating and affected by events but may or may not have agency to change a situation. Immersive journalism also parallels a news narrative playing out in the physical world much like a piece in a newspaper or segment on television and while one might experience the story from different starting points, the story itself should not shift.

When the record industry refused to consider experience, i.e. how their audience was going to interact with music, they gave Apple the right of way to build iTunes. The result was an extremely successful and robust environment that offers an entertaining, multilayered way to access music while also supporting Apple’s iPod music device. No doubt immersive journalism is nascent, but we hope to learn from the mistakes of the music industry which, unfortunately, legacy media seems well on its way to repeating. With iTunes as our model, we are concentrating on experience, and hope that in the near future we will support an offline platform as well.

You can see videos about the prototypes mentioned in this piece and learn more about this burgeoning avenue of journalism at www.ImmersiveJournalism.com.

About nonnydlp

Nonny de la Pe