Learning by doing: Seeking best practices for immersive journalism

Ernest Wilson, the dean of the Annenberg School of Communications and Journalism, put it like this: What if, after receiving the home and garden section in the morning, the reader could walk right into the section and visit a garden? This bucolic vision reflects one potential scenario for what we are calling at Annenberg “immersive journalism,” a new genre that utilizes gaming platforms and virtual environments to convey news, documentary and non-fiction stories. As a senior research fellow, I am prototyping immersive journalism stories, hoping to discover and create best practices for a burgeoning filed that can capture audiences increasingly accustomed to experiencing digital worlds.

The fundamental idea of immersive journalism is to allow the audience to actually enter a virtually recreated scenario representing the news story. The pieces can be built in online virtual worlds, such as Second Life, or produced using a head-tracked head-mounted display system, or HMD. An HMD is a lightweight helmet that has screens covering the eyes and tracks head movement so ensure digital imagery on the screens stays in perspective to create a sensation of having a virtual body in a virtual location. Immersive journalism can also be constructed in a Cave, which uses full body-tracking technologies in a small room so that individuals can move their bodies around the space.

Visual and audio primary source material from the physical world reinforce the concept that participants are experiencing a nonfiction story, with the video, sounds or photographs acting on the narrative. For example, video triggers at key points in the virtual landscape to remind a participant that the computer generated environment is grounded in the physical world. Scripted events that create a first person interaction with the reportage can also help create a feeling of “being there.” Also, participants can query or interact with the elements around them to learn more about the details or context of the news story.

In general, participants travel through the story as a digital representation of themselves, or as one of the subjects in the news piece. Whether visiting the space as oneself or as a subject in the narrative, immersive journalism aims to afford the participant unprecedented access to the sights and sounds, and possibly, the feelings and emotions that accompany the news.

Well-crafted journalism always aims to elicit a connection between the audience and the news story. Creating that connection via different kinds of ‘immersion’ has long been considered ideal. Describing her reporting during World War II, Martha Gellhorn called it “the view from the ground.” Writer George Plimpton famously joined the Detroit Lions American football team in order to give his readers the most intimate sense of playing on this team. Television news correspondent Walter Cronkite made a series of documentaries recreating historical events in which he would offer a brief introduction before an announcer would give the date and the event, proclaiming, “You Are There!” More recently, attempts to combine audio, video and photographs on the Internet have created what some journalists call “immersive storytelling.” As technology editor at MSNBC, Jonathan Dube (now Vice President at ABCNews.com) said that he believes this can bring the reader or viewer “closer to the truth.”

In collaboration with digital media designer Peggy Weil, we have built several prototypes which reflect my interest in covering human rights issues. Gone Gitmo, a virtual Guantanamo Bay prison built in Second Life, allows participants to explore a place that is inaccessible to the average American citizen and press. (In fact, the Pentagon just expelled four reporters who have been covering the prison for years.) Gone Gitmo includes an experience on what it might be like to be detained, hooded and then imprisoned in Camp X-Ray. It also examines the ramifications of losing habeas corpus rights.

Another Second Life prototype, Cap & Trade, is a news report on the carbon market that sends people on a journey to follow the money in order to try to better understand the complexities and human consequences of trading carbon credits. Cap & Trade was built in partnership with the Center for Investigative Reporting and Frontline World and is particularly reliant on the excellent reporting by Mark Schapiro that appears on Frontline and in Mother Jones and Harpers Magazine.

A third prototype is based on the interrogation logs of Detainee 063, Mohammed Al Qahtani, who had been declared tortured by the Bush administration. Built at the Event Lab in Barcelona with Mel Slater and his team, we use an HMD to put participants into the virtual body of a detainee who is held in what is referred to as a “stress position.” When participants look around, they see a virtual mirror with a digital figure in that mirror who looks like a detainee and moves in unison with the participant. Participants also wear a breathing strap that programs the avatar to breathe at the same time as they do, further enhancing the sense of virtual body ownership. Throughout, the sounds of the Al Qahtani interrogation play as if coming from the next room. While research data was not collected on this particular prototype, every participant anecdotally reported that their body was hunched over in a stress position when in fact they were sitting upright.

Immersive journalism is distinct from news games in that news games embrace gaming protocols. With news games, the player undertakes a task or pursues a goal, voluntarily constrained by agreed upon rules, and must take action to advance position. Progress is often measured by indicators such as levels or points.  In contrast, a participant in immersive journalism isn’t playing a game, but is put into an experience where she is participating and affected by events but may or may not have agency to change a situation. Immersive journalism also parallels a news narrative playing out in the physical world much like a piece in a newspaper or segment on television and while one might experience the story from different starting points, the story itself should not shift.

When the record industry refused to consider experience, i.e. how their audience was going to interact with music, they gave Apple the right of way to build iTunes. The result was an extremely successful and robust environment that offers an entertaining, multilayered way to access music while also supporting Apple’s iPod music device. No doubt immersive journalism is nascent, but we hope to learn from the mistakes of the music industry which, unfortunately, legacy media seems well on its way to repeating. With iTunes as our model, we are concentrating on experience, and hope that in the near future we will support an offline platform as well.

You can see videos about the prototypes mentioned in this piece and learn more about this burgeoning avenue of journalism at www.ImmersiveJournalism.com.

Mapping technology provides journalists a new medium for storytelling online

Cartography is undergoing a renaissance that is opening new opportunities for journalists. For example, the Los Angeles Times’ successfully updates and moves the police blotter onto the Web by using Google maps to pinpoint homicides. However, mapping technologies offer even more robust data mining possibilities.

Hypercities, a mapping project out of UCLA, connects time and geographical spaces. The site allows users to put historical layers on to maps, such as overlaying John Snow’s nineteenth century work tracing the cholera epidemic on to a map of present-day London. A much more impressive undertaking on Hypercities was created by Xarene Eskandar, a graduate student at UCLA. She consolidated content on the Iranian election to create a geo-located reportage of more than 800 YouTube videos, Twitter feeds, Flickr photographs, and other forms of documentation. Hypercities says that, “The result is the largest, day-by-day, hour-by-hour, and sometimes even minute-by-minute web documentation of the election protests in Iran.”

FloatingSheep.org, a collaboration between the University of Kentucky and Oxford University, uses comparative maps to show places in the U.S. where bars outnumber grocery stores, which country creates the most web-based references to god around the globe, the geographical face of the abortion battle and, more recently, a map of Wikipedia that takes time into account. Using this time feature, Floating Sheep was able to point out the many biases in Wikipedia including the “lack of pre-16th century biographies in locations (Fertile Crescent, China, Indian subcontinent, etc.) with the longest histories of civilization.” The time element was key in teasing out Wikipedia’s focus on western cultures.

Interestingly, the connection between time and geography does not need to be confined to the physical space, says Kevin Leander, associate professor of language and literacy at Vanderbilt University. His studies on childhood learning take into account not only where children spend their day but also how they move through virtual locations. “Kids have become constrained in their physical mobility,” he says, pointing out the possible role of increased fear over children’s safety as helping create the issue. “But a digital mobility has arisen in concurrence with this back seat child who is not outside of adult supervision.” Simply put, he is finding that activity in the virtual space is increasingly significant for children and needs to be considered when looking at temporal mapping of their lives and interests.

Leander’s work on the relevance of our digital existence to the physical world is part of a larger question just beginning to be asked. Should stories that occur in those spaces, such as virtual worlds, be covered by reporters? How often will they resonate in the physical world? Time will tell — and mapping techniques may help be a guide.