One of the greatest opportunities of multimedia journalism is the ability to make different design choices. Although most online organizations present digital derivatives of their “parent” products – newspaper sites present columns of text, radio sites feature audio files, and TV sites provide video – we are seeing an increase in the number of sites embracing all design options. Radio sites are complementing their audio with photos and/or text, newspaper sites are presenting video and audio slide shows along with their text, and TV stations are supplementing their video pieces with text stories.
Increasingly, news organizations are challenging themselves and their staffs with stepping outside of their format expertise and trying to produce news packages that take full advantage of the array of media formats available. Online news sites are trying to integrate different media types into the story package – creating rich multimedia experiences for their audience. Exploration in the use of Flash helps designers create a common interface that transitions easily from graphics, to video to photos to audio without interrupting the user.
Creating these rich media experiences is a commitment of time and specialized talent that news organizations cannot – and should not – afford for every story. This is the biggest challenge for news designers: Given all the design options now available, how does one evaluate effort over return? When does an integrated, interactive story work best in terms of users’ enjoyment and/or comprehension? When is it warranted to help with understanding of the topic? Bottom line, when is it worth it?
In this column we will find and report on the beginning efforts to research and evaluate story design effects on news audiences. In this month’s column we discuss the findings for the first project of our research consortium – DiSEL – the Digital Story Effects Lab.
Comparing Static / Passive Text and Dynamic / Active Multimedia Stories
DiSEL study: Overview
In 2002, working with a graduate student at the University of Minnesota, we attempted to “catalog” the areas where design decisions could be made when crafting stories online. These “Elements of Digital Storytelling” (www.inms.umn.edu/elements) looked at a variety of attributes of digital stories.
In our first DiSEL study, we looked at two of these attributes, both related to “action.” We wanted to compare the impact on user attitude and experience between different approaches to content and user action.
Stories can be designed with either static content (the material just sits there, there is no movement) or dynamic content (the material moves.) In terms of how the user must engage with the content, stories can be designed to be passive (once the user has clicked to the page they can sit back, there is no action to take) or active (the content is designed so that the user must engage with it in order to fully experience the full set through selection of options or clicking to see the next portion.) Dynamic / active content is the type that is typically crafted using Flash. There is motion and choice. Static / passive content describes HTML coded, there is no motion and what you see is all you get.
We found perfect pieces to test these two presentation styles in the BBC’s material on health effects of recreational drugs. They had created two packages – the static, encyclopedia-type page display here and the dynamic interactive package here.
The integrated, interactive piece about recreational drug use creates a scenario where users give various drug combinations to a dancing clubber and witness the effects on his body.
The encyclopedic-type presentation lists various drugs, describes their effects with text and an image.
In addition, we decided to see if the motivation for going to the site would change the user’s perception, attitude, and information retention.
In late 2005 we collected data from 63 subjects (the target audience for this content was young adults so we tested 18-29 year olds), using four different testing conditions, resulting in a total count of 15 subjects tested under each condition. (Three of participants had unusable data.)
The conditions were:
- Condition one: Users who viewed the dynamic / active Flash site and were told they had received the link from a professor telling them to explore it to complete a research paper. This was the “information” motivation scenario.
- Condition two: Users who viewed the Flash site and were told a friend e-mailed them the link as something interesting to check out. This was the “entertainment” motivation scenario.
- Condition three: Users who viewed the static / passive HTML site with the “information” motivation.
- Condition four: Users who viewed the HTML site with the “entertainment” motivation.
These conditions allowed insight into the effectiveness of each form based on what the users’ motives were in seeing the presentation, and also allowed for enough subjects to generate statistically reliable results in some areas.
Research participants filled in a pre-exposure survey intended to gauge their overall use of the Internet, their preference for certain styles of presentation, the use of news sites, and their attitude toward drugs. Then their movements around the page they were sent to were “eye-tracked”. A post-exposure survey provided feedback on their attitudes toward the experience and the news organization that presented it, the ease of navigation, and their retention and recall of information presented.
The challenge in this type of research is determining what it is you want to test. There is a variety of hoped for outcomes when a news organization creates and presents an online news package. Which is most important? Effective presentation of information as seen in greater retention and recall of facts? Stickiness as seen in length of time spent with the content and greater depth of examination of the material? Brand enhancement as seen in reported enjoyment or appreciation of the organization presenting the information? With this study, we tried to get at the impacts of the presentation form on a variety of these areas.
DiSEL study: Findings
This comparative study showed that for the two sites tested:
Interactive presentations work best when you want users to…
- spend more time with the presentation;
- describe the experience as “enjoyable;”
- recall more of the information;
- recall your brand;
- feel entertained.
Static presentation work best when you want users to…
- “click to” all the of the presentation’s materials;
- perceive the site navigation as easy.
Either form is equally effective if you want to…
- increase the likelihood a user would return to the site.
In terms of the motivation for going to the site, there were some interesting differences in people’s responses to the two presentations.
If users are seeking information…
- They will spend an average of two minutes longer on the site than if they are looking to be entertained;
- They will have greater recall and comprehension of the information than those seeking entertainment.
Motivation did not matter in terms of…
- How enjoyable a user found the site.
DiSEL study: So what?
This research shows that the choices made in presenting information will have significantly different impacts on the audience. No one presentation form is going to be the most effective by all measures that you have in your newsroom for determining successful design. What the research does seem to reveal is that the highly interactive content results in more time spent online with the material and a greater level of reported “enjoyment.” In addition (and counter to some other studies which show a negative impact) the Flash version seemed to help people recall the information being presented. So, if your goal in presenting a story – particularly one that has potential for a long “shelf-life” – is to entertain, inform, and keep people online longer, then investing in a creative, interactive presentation could be well worth the effort.
Supporting Research
A portion of Poynter’s Eyetrack III study tested similar situations. In this study, two distinct story designs were considered. With the help of NYTimes.com, text versions of two news stories were edited to 3-5 minute reads. Then, existing multimedia presentations were condensed to 3-5 minute experiences.
The text version of the story “Dangerous Business” that was used for the study
The multimedia version ofo the story “Dangerous Business” that was used for the study.
The text version Al Hirschfeld’s obituary that was used for the study.
The multimedia version Al Hirschfeld’s obituary that was used for the study.
Half of the test participants (approximately 25 people) experienced one of the stories in text and the other in multimedia. The other half experienced the opposite formats. (All participants saw a control article beforehand.)
After they read or viewed the stories, participants were given the same recall quizzes.
Eyetrack III: Findings
This study shows that:
Interactive presentations work best when you want users to…
- recall unfamiliar terms and processes/procedures more effectively.
In one test story an animated graphic showed how cast iron pipes are made – an essential component to understanding the overall story content. Those who received this graphic had better recall of the terms and processes involved than those who received the same information in text form.
Static text works best when you want users to…
- correctly recall specific factual information, such as information about names and places.
It was found that with both stories, individuals had better recall of the names of people involved and the locations of specific story events if they read the text version.
How this research can help: the checklists
Common threads from findings in this work can help guide multimedia editors and designers to make more effective decisions. Here are lists of questions that can help. (Also available in a printable PDF.)
Should we present this story as an interactive?
Before undertaking any large story project be sure to ask:
- Who is the target audience for this story?
- What do we hope to accomplish in telling this story to them?
Then use this decision-tool to see which approach to storytelling is best supported by the research in these studies:
- Does the story concern elaborate or unfamiliar processes / procedures?
- Yes – 1 point
- No – no points
- Is the level of interest in the topic high enough that people would be willing to figure out story navigation?
- Yes – 1 point
- No – no points
- Does the story have value beyond the first few weeks? Is it likely to be a topic in the news again?
- Yes – 1 point
- No – no points
- Is entertaining the audience more important than simply informing?
- Yes – 1 point
- No – no points
- Is it important that the audience be able to recall specific facts from the story?
- Yes – no points
- No – 1 point
- If the story is told in separate components, it is essential that all the components be viewed by the audience?
- Yes – no points
- No – 1 point
- Do you hope the audience recalls where they saw the information?
- Yes – 1 point
- No – no points
If you get five or more points, then you should strongly consider an interactive story approach.
Coming in March: Journalism-applicable results from the Nielsen/Norman Group’s first eyetracking study.
Coming in April: An interview with Poynter Eyetrack ‘07’s researchers.
Coming in May: DiSEL research results about:
- the design and placement of “Breaking News” and supplemental links
- how people move through different slide show designs.
