It feels relevant: biological tactility in news media

Thanks to the Internet, we know a lot more about how news is used. Traffic records and data analysis give us the “what,” “where” and “how” consumers take information from a website; we also know “when” it’s accessed and somewhat less about for how long. The “why,” however, is still largely a mystery. Nor do we know much about how the senses absorb online news, how the brain sifts and orders it and how it affects the body, moods, emotions and decisions.

What happens when users receive news? More to the point, why do Internet users not consume what is traditionally defined as news? Why do millions head to YouTube, MySpace and online games, including serious ones? Why to Petopia, Second Life or video blogs like Crooks and Liars?

If online journalists knew the answer, they might be offering more attractive and informative news sites. Neurobiologists and cognitive psychologists—only two of many disciplines that give us insights on how digital technology impacts the senses—have conducted recent research and crafted theories, many of them tentative, on how the brain reacts to information. (For a dated yet excellent overview, see “Nature’s Mind; The biological roots of thinking, emotions, sexuality, language and intelligence,” by Michael S. Gazzaniga, 1992.)

These findings can help us to understand how digital data is used—how the brain rejects or absorbs it, then meters it into the neural system. Researchers are looking at how online content can trigger emotions, including visceral ones, how the nervous and limbic systems, the reflexes, blood circulation and sexual organs all respond to the signs and icons of new media.

The latest research points to a general conclusion: online digital worlds like YouTube appeal to the whole body, from frontal lobe to the toes. This payoff from multimedia may be unique in communications history. The question is how can journalists put that understanding of a mind-body connection to good use.

But scientists have no monopoly on making sense of the rapid rise—the unprecedented global acceptance—of new media. A rich legacy of the study of theater, narrative and visual culture has already provided the groundwork for new media theory. An understanding of theories of art and art history and basic differences in presentation can help those who work in the digital world to know who they are and what traditions they draw from while engaging in the practice of digital convergence. In the words of one new media critic, Mark B. N Hansen at The University of Chicago, it enables us to grasp “the aesthetic newness” of digital media and “its resistance to capture by now dated, historical forms of art and media criticism.”

If a journalist deals with a 3D graphic, an immersive multimedia news environment or GIS mapping mash-p, he or she has reached fundamentally new territory. Hansen and others, drawing from scientific research, conclude that the way a person receives and absorbs mediated digital information is a mind-body process. And the online multimedia experience is more complete, more biologically compelling than previous forms of media, including cinema. As Hansen puts it, the new media experience is “qualitatively different from …the ‘verisimilitude’ and ‘illusion’ of the cinematic image.”

This also differentiates online news video from broadcast TV news practices, as journalists who work with online video photography have found through trial and error. This difference becomes more pronounced with the use of panoramic cameras and immersive perspectives.

But whiz-bang devices are only the experimental edge or mega-toys of the Internet. The medium’s unique tactile experience can easily be appreciated by clicking a mouse, tapping the keys or interacting with audio-visual displays. This is another world from turning pages or flipping through channels.

From a historical approach, the push to expand new media over the last decade to meet the demand of a voracious and adoptive audience can be looked at as the joining together of rival ways of creating illusions that have developed over many centuries.

For more detailed discussion of art and theater traditions, readers can go to the works of theorists such as Henri Bergson and Walter Benjamin. These trailblazers have helped today’s media critics conceive of a multimedia family tree that has two main branches: One starts with Greek drama and wends through Tudor theater and the rich tradition of outdoor spectacles and illusions that invite audience participation. The second branches off from Baroque theater into increasingly sophisticated indoor presentations aimed at passive audiences.

Both Greek amphitheaters and the open-air Tudor theaters of the 1590s are believed to have offered an intense and pleasurable communal experience. London’s theaters at Shakespeare’s time are considered to have been the most popular form of entertainment of that era, drawing people of every class to form enthusiastic and often rowdy crowds of up to 2,500. The Shakespearean-era theater experience had multiple layers, from the cerebral to the hair-raising. The narrative was propelled by magical effects – trap doors and winches, painted canvases, fake hangings and beheadings, fireworks, thunder, drums, gunshots, hoof beats and lots of pigs’ blood.

This is a tradition of outdoors public spectacle—a lineage of fairs, markets, freak shows, street performances and exhibitions, parades, bandstands, songfests, dances and sporting events. Opportunities for audience interaction expanded in the 18th and 19th centuries with panoramas representing famous battles, museum tableaux, expositions and world’s fairs. Viewer-run motion-picture and three-dimensional photographic inventions in the 19th Century required manual production of movement, such as spinning a stroboscope, flipping a flip book, or changing slides in a stereoscope – and debating among friends about which slide should go next. These pre-cinematic devices provided hands-on, shared, communal entertainment.
The rival tradition of the immobile audience began in the more politically correct indoor theaters of Europe’s 16th and 17th centuries where architectural controls divided performers from the audience. Histories of drama indicate that the use of intimate playing spaces on stage emphasized “actorly effects.” Political and social satire displaced the spectacular and magical. Illusion became tightly framed, emotional manipulation more structured and audiences consigned to immobility, if not censorship, both state- and self-imposed. (For an overview of that transition, see “The Theatrical World,” a forward to the plays in the Pelican Shakespeare Series.

It’s possible to see how these controlling practices led to the industry of the Silver Screen, Broadway producers, Big Media, and teams of screenwriters, studio vice presidents for creativity, ego-driven directors and superstar actors. (Not to mention a commercial cult of personality driven by advertising, marketing and public relations.)

The tradition of audience mobility went in another direction, leading to the development of the all-enveloping panorama in 18th Century England and its subsequent use at national exhibitions and for morale-boosting propaganda. Heads of state and entrepreneurs created large panoramic battle and other patriotic scenes and some were taken on tour in Europe and later in the United States.
Needless to say, the concept of outdoor illusions, life-like tableaux and thrill rides became the staple of 20th Century amusement parks and traveling carnivals. The middle of the century saw media corporations bend the free-wheeling, bordering on outlaw tradition of amusement parks and “carnie shows” back into branded commercial control with the advent of theme parks.

Technology—the use of electricity, applied engineering skills and lens developments—drove much of this growth in both traditions towards more sophisticated applications. But media theorists avoid notions of determinism. They observe that participants in websites like YouTube take over the technology and use it in ways that can’t be extrapolated or predicted. Computers empower the creation of online virtual spaces, which, by themselves, are not the medium of communication. Virtual environments like those proliferating now on the Internet, are “the context within which a variety of image and sound-based media operate,” says Vancouver media critic Ron Burnett.

At one level, this seems quite straight-forward: Build an electronic field of dreams and the videocam fanatics and their audience will show up. But the research indicates something much more profound is going on at the YouTubes and MySpaces.

New technology enables unique multimedia perspectives that, in turn, open up new possibilities for story telling and may even be changing the way that humans process information. Digital technology, Burnett says, enables humans to “create the foundations for different ways of thinking. … Technology is as much about cognitive change as it is about the invention and the creation of physical devices.” (102)
Virtual reality has a “hallucinatory” dimension, Hansen says, that “explains the capacity for the VR interface to couple our bodies with (almost) any arbitrary space, and not just spaces that are contiguous with the physical space we happen to occupy or even spaces that we typically occupy.”
According to recent research on perception, this capacity of computer imagery to “make it real” occurs at a deeper, more biologically based level of human experience, one in which, to use Hansen’s words, “the embodied mind actually creates what it sees.”

The history of visual culture and the new findings of neuroscience, when combined, help us gain a better understanding of consciousness when a viewer clicks on video or enters a 3D or panoramic environment. How do these electronic spaces function? What is the connection, if any, between the physical and virtual world?

Researchers who work with advanced digital interfaces like “fog screens” and 3D helmets or high-speed game displays say the participants exist in both spaces simultaneously – what Burnett calls a “third space.” Others, such as Luciano Floridi, define this space as a mental zone between past and future.

Media critic Brenda Laurel calls it a shared or common ground, “a space of mutual knowledge, mutual beliefs and mutual assumptions,” an alternative reality that gets updated or revised moment to moment: In other words, a “whole” experience that extends the physical world, gives individuals an identity and invites entry into online communities, including virtual newsrooms, if editors would permit.

Once we enter the common ground of YouTube, MySpace or Second Life, we are empowered to live in another dimension, a psychological plane created by a combination of the cognitive ingenuity of software, the quality of content and the participating audience. Deep levels of code and data and the converting algorithms create the illusion of “being there.”

But the next step that researchers in various disciplines and by phenomenologists such as Hansen take is a reversal of perspective of almost Copernican proportions, one that could have profound implications for journalists. Researchers are finding that the human brain does not take in digital imagery as if it were an external geometrical space. Instead, visual sense-making is located within the body. Various sensory processes “generate a ‘haptic spatiality,’ an internally grounded image independent of geometrical space,” as Timothy Lenoir at Stanford explains in a forward to Hansen’s latest book, “New Philosophy for New Media.”

This body-brain connection has profound implications for new media because it downplays “an abstracted sense of vision as the primary sense in favor of the internal bodily senses of touch and self-improvement.” Hansen calls it “haptic vision,” or vision that is engaged with the sense of touch. It accounts for the sensation of flying through 3D environments, diving into satellite-generated images, the belly laugh from a Flash graphic or arousal from the erotic. Some applications are well known (infamous); others have just emerged. Therapists, for example, are beginning to use this tactile dimension to help stroke victims regain mobility and speech functions.

Instead of separating us from our senses by projecting virtual worlds, computers forge an internal body-brain link. “The source of the virtual is thus not technological, but rather a biologically grounded adaptation to newly acquired technological extensions provided by new media,” says Lenoir.

The body-brain experience inspires the user to act, since he or she is now at the center of the universe, as opposed to sitting passively in an audience. Multimedia presentations, especially versions that display with panoramic perspectives or 3D devices such as HMDs, or head-mounted displays , PanoChambers or CAVE virtual reality systems, place the spectator in a single, coherent space. The virtual world continues the physical space surrounding the spectator.

This is the opposite of the Renaissance perspective, which came down to us through photography, cinema and television. While this tradition emphasizes the realism of what is observed, it also splits the viewer’s identity between the physical space and the space of representation. Both cinema and TV confine the viewer to seeing “reality” through a rectangular frame. This is efficient and, as media critic Lev Manovich at the University of California, San Diego, has noted, gives us images that “are easily processed by the brain.” But it also restricts mobility, confines perspective and eliminates the experience of touch.

Hansen identifies the tactile or haptic dimension as the distinguishing feature of new media, requiring more involvement on the part of the viewer than the representational tradition provides. The goal of new media technology is not just to make the image more believable but “to bring into play a supplementary element of bodily stimulation.” Recent physiological research, he notes, shows that tactile stimulation functions as “reality-conferring.” It is an essential element of presence, which Kwan Min Lee at the USC Annenberg School for Communication calls “a psychological state in which the virtuality of experience is unnoticed.”

This bodily activity can be as simple as passing a mouse over a Flash button or as crucial as wearing a “digital glove” to perform surgery. Flight simulators and arcade games have long provided tactile feedback. Whatever the level of engagement, the research indicates that this body-mind link allows the virtual world to be synchronized with the physical world in a way that is grounded in the biological potential of human beings.

Other areas of research—such as biological anthropology, neurophysiology and zoology—deal with building a factual floor under a developing theory called mimetics. A collection of disciplines looks at thoughts as being not necessarily self-generated within the brain but as being acquired through the thoughts of others.

This topic is perhaps best articulated in the pre-Internet work of anthropologist Gregory Bateson in the 1970s, and by the recent work of British psychologist Susan Blackmore (“The Meme Machine”) and anthropologist Robert Aunger (“The Electric Meme”). The word “meme” has been popularized by Oxford zoologist Richard Dawkins (“The Selfish Gene”) to mean a unit of information that plays a social role analogous to genes. Aunger argues that once inside us, “these thoughts (memes) then go to work for themselves, pursuing goals that may be in conflict with our best interests. These ideas have their own interests by virtue of having qualities that make them like biological viruses.”

Aunger warns that the existence of memes remains to be established, like theorized subatomic particles or unseen planets. The concept also faces opposition from other disciplines, such as sociobiologists and evolutionary psychologists, who tend to equate memes with mysticism. From a journalists’ perspective, for now, it’s worth noting that some rather bright scientists believe that the transmission of news may function like computer viruses. The messages or memes—for example, “Islamofascists,” “NASCAR,” urban legends or Microsoft chimes—may replicate and move from one brain to another by means of signals or icons that initiate “the reconstruction of the relevant meme from materials located there.”

Note that this still-nascent theory seems to fit well with the work of Burnett, Hansen and other new media theorists. Mimetics and related disciplines may help identify how news engages the brain, becomes shared online and how it might influence public discourse, as well as subsequent voting behavior. If Aunger and others are right, daily news conferences, duplicated in thousands of newsrooms each day worldwide, may be acting like Petri dishes, assembling and unleashing digital signals over the Internet that can then replicate in billions of brains, sometimes almost instantaneously. Many is the virus that would envy this infection rate. (For a discussion of the “technology of memory” and how the memory functions in bodies, see “Tangled Memories” by Marita Sturken.)

Equally intriguing is the study of how a large percentage of incoming signals get rejected or filtered by the brain. The sensory input often fails to find an instant fit with an individual’s meme-building materials, such as stored memories, competitive instincts, survival strategies and the potential for empathy. If journalists understood that process better, they might be in a position to offer stronger news that is both intellectually and biologically relevant.

Online newsroom wisdom argues for more interactivity, rich local databases, concierge-like services, blog columns and user-generated content. But that may not be what’s called for. Often, a superficial fix substitutes for fundamental reform, such as arming notebook-carrying print journalists and SLR-equipped photography staffs with video cameras, or setting up a 24-hour rewrite desk run by people who can both write text and edit audio and video content rapidly as it is sent from reporters in the field.

The audience demand for both instant news and deeper forms of interactivity on websites can be seen in the online gaming world, with its forays into online competition, inexpensive pay-per-download services, low-resolution online games that owners can upgrade, personalized karaoke and controllers like batons that allow the user to lead an orchestra or ones shaped like tennis rackets.

Participants demand the tools for interaction, more controls and the ability to assemble forms of reality that matter to them. But Web traffic and extensive use of e-mail indicates that they want access to, and the ability to share, the reality of trained, experienced journalists who do the hard digging, ask the tough questions and shoot professional video, sometimes under hazardous circumstances.

No doubt, the more convincing forms of “presence” and body-mind involvement open new possibilities for telling news in compelling ways. Combining 3D immersive technology with GIS mapping techniques, for example, would offer content to compete with and draw audiences from the YouTubes and MySpaces.

Manovich says that the language of digitization is in an early stage, where cinema was 100 years ago. “We don’t know what the final result will be, or even if it will ever stabilize. … We are witnessing the emergence of a new metalanguage, something that will be at least as significant as the printed word and cinema before it.”

Larry Pryor is an Associate Professor at the USC Annenberg School of Journalism. He’s currently researching the haptics and epistemology of digital news media.