USC Annenberg Online Journalism ReviewUSC

Immersive News Technology: Beyond Convergence
Adapted from a paper given at the Radio-Television News Directors Association convention at Minneapolis, September 16, 2000

Presenting news over the Internet holds special problems because digital technology changes so rapidly. As soon as news outlets adjust to developments, the Next Big Thing surfaces, such as XML or wireless Internet. Suddenly, major investments become obsolete and new competition arises. This uncertain environment prevails today as telecommunications companies make unprecedented investments to install broadband capacity. This fundamental change in the delivery of data ? by cable modems, high-speed phone lines and satellites ? has opened both opportunities and pitfalls for news organizations.

Media empires that own print, broadcast and Internet properties have been quick to embrace the concept of convergence. Corporations such as the Tribune Company, Media General and A.H. Belo have made substantial efforts recently to combine their various newsrooms. Stories at more and more news organizations now go out on multiple platforms, sometimes simultaneously. Newspaper photographers carry digital video cameras, print reporters go on television, audio feeds go to both radio stations and Web sites and news writers struggle to convert stories from one medium to the other.

This is bold, ambitious, pioneering work that requires large investments, versatile news teams and a powerful digital delivery infrastructure. Not all of the elements of convergence are in place yet and many vexing problems need to be addressed, such as increased compensation to employees for working on multiple platforms that reach greatly expanded audiences. Newsroom chains of command and procedures for covering events remain a work in progress. Writers must experiment to find new ways to present information in a way that combines text, hypertext, audio, video and graphics. These writers become more like producers ? assemblers of news packages with multiple dimensions for use on diverse platforms, such as cell phones, digital assistants, in-car computers and e-mail.

Add to these complications the need for some news organizations to reach a worldwide audience. The BBC, for example, can produce Internet news in 41 languages. This means that news must be cross-cultural if it is to reach a global audience. At the other end of the spectrum, news aimed at local audiences must be tailored to reach individual communities and even neighborhoods.

But nowhere has the introduction of broadband technology offered a greater opportunity ? and challenge ? than in the development of immersive news. The word ?immersive? is best known within engineering schools, although it now makes appearances at advanced trade shows, such as SIGGRAPH. Elements of it can be seen on more ambitious Web sites, such as Quokka Sports immersion. But for the most part, this technology is still in the research box, awaiting the Broadband Genie to fully unleash it.

Immersive digital technology combines elements of enhanced multimedia: spatially-placed sound and 3-D video and graphics, plus haptic technology capable of conveying a sense of touch, texture and temperature. It employs data compression techniques and ?surrogate? images, such as avatars or 3-D representations of humans that exist on a host computer and change or animate with only a fraction of the data required by a video stream. Immersive technology has complex databases and multiple dimensions. It is thoroughly interactive and customizable, a virtual world that extends the real world with simulated experience.

Some of this technology will not be leaving engineering labs for a decade or more. Key elements are still in the basic research phase, heavily financed by government and industries in many of the developed nations. In the United States, for example, the leading immersive lab is here at the University of Southern California. The Engineering School?s Integrated Media Systems Center (IMSC) receives substantial funding from the National Science Foundation and additional support from industry partners, such as Hewlett Packard, Motorola and Raytheon.

Parts of immersive technology are starting to enter the news marketplace. Our laboratory at the Annenberg School for Communication, working with IMSC, has begun to test how this technology can be used and what its introduction will mean to news organizations and audiences.

Simply put, in some situations, the viewer can be immersed in a news story. The most dramatic examples would be an urban riot, an unfolding natural disaster or a major spectacle, such as the opening of the Olympics or a Presidential Inauguration. The real event would be digitally re-created as a virtual event that surrounds the viewer with a visual, aural and even tactile experience.

Immersive technology puts control of news coverage in the hands of viewers. They can accept a default perspective picked by a director or they can opt for a different perspective or news experience, placing themselves in alternative parts of an event or even requesting a reporter to get added information.

Viewers might first choose which level of presentation they want ? video, audio, interactive graphics and text, depending on the platform they are using. The same story could be presented as audio, perhaps to a voice-activated computer in a car, or in a fully immersive, multimedia presentation in a home theater, or as bare-bones text for a handheld wireless device. The full-blown immersive experience would attempt to insert the news consumer into the environment the reporter, film crew and field producer inhabit ? wrapping the person in 3-D sound and video, as well as offering background information and context, with different versions of the event customized for specific users.

This technology would also have important applications for the performing arts and for sports. Do you want to be in a mosh pit or behind the drummer? In the loge seats or sitting among the woodwinds? Standing at the 50-yard line, out with the right tackle or up in the press box?

These possibilities put a heavy emphasis on the creation of tools future journalists will need to produce the immersive news experience. Such a news team must be able to assemble the elements of a multi-layered story. And the field reporters and producer will need a studio team that can organize and categorize all of the elements of the emerging story.

Immersive stories are like any other story; they need structure, a space within which both the reporter and viewer can operate in without getting lost. The rich alternative information surrounding the spine of the story has to be easily navigated and quickly presented. This puts a huge burden on database software, a key area of further research.

The degree to which immersive news will succeed depends a lot on how well the news presentation process can be automated, another area of research. Many of the background elements ? photos, bios, polls, statistics, audio and video clips ? will have to be automatically assembled and inserted into the story. Internet news consumers demand speed; smart authoring tools will be essential to converting breaking news into immersive, open-ended news experiences.

This will also require the training of a new breed of reporters, ones capable of operating in the field with complicated equipment but also able to get the story in the old-fashioned, street-smart way. They will also have to know the potential of immersive technology in order to exploit it. Producers and directors will have to be even more tech-savvy.

The goal is to fulfill the Internet?s strength as a ?pull? medium, as contrasted with a traditional ?push? medium. Interactivity allows viewers to be part of the scene as news happens and get any slice of the story they choose. This ?democratic? concept of news coverage, with its possibilities of distortion and selective bias, raises tough ethical issues. Traditional journalists tend to see so many drawbacks that some reject interactive news outright. But as the technology leaves the laboratories, it has to be dealt with, preferably by concerned, informed journalists.

New technology enhances the role of the journalist. Not only do they have to sort through thorny ethical problems and work with engineers to solve them, but they have a new role on the Internet as helpful guides to viewers ? forward scouts and navigators through a new world of experience. News organizations that can gain the public?s trust will earn the biggest audience.

The selective nature of new technology means that images easily get ?out of synch? with reality. In the digital world, data is sorted, metered, chopped and channeled. To save valuable bandwidth, for example, computers can be programmed to only record changes in a scene, not the entire perspective, as in analogue video. This opens Internet news (and digital TV) to charges of distortion and even visual fraud. Is the viewer looking at a news event or only pieces of it? When is the depiction of a news event no longer ?valid?? What can a viewer trust in this malleable world? Which artificial depictions make a difference? (We know that yellow stripe at the 42nd yard is not actually marked on the field but shows us where the ball has to go to make a first down. Is that deceptive or just convenient?)

Such questions inevitably lead back to the importance of branding and the recognized integrity of certain news organizations. Traditional journalism is inherently selective, from the winnowing process the reporter uses to assemble a story to the editing or direction needed to deliver it. Readers and viewers trust news teams that can do this fairly and without causing distortion. Highly digitized news will be no different, except complex tools will require reporters, producers and directors to be even more diligent. As always with computers, the margin for error gets smaller.

And how viewers use ?pull? technology to create their own view of the world raises another set of social problems. Does customized news mean that viewers in advanced industrial nations will become even more insular and arrogant than some people believe they now are? Not necessarily. Internet news, whether it be today?s hypertext version or tomorrow?s immersive news, has one powerful characteristic that goes against overly-narrow personal news: the ability to form links.

Connections between pieces of information, which computers can form so powerfully, lead viewers to choose unexpected paths to further information, to explore inviting links to new experiences. The human being is genetically disposed to curiosity and computers play to that strength. Internet news lets the brain wander, seek and find. And the more powerful the technology, the more experiences will be available for viewers to enter and share. Stumbling onto a Web cam that shows elephants cavorting at an African water hole can be a thrilling moment for a Web surfer. Before long, you will be able to become an elephant for a minute or a day. The possibilities will be limitless.