Continuous Updates: Design decisions when designating breaking news

This is one in a series of reports on DiSEL (Digital Story Effects Lab) Research projects conducted in 2007 through a research grant from the University of Minnesota. First in the series was on Navigation through Slide Shows

Why we did the study

One of the great strengths of the Web is the ability to keep news updated and to alert readers immediately to stories they need to know about. This is also one of the biggest organizational changes the Web has brought to newsrooms. Shifting from daily to constant deadlines has caused a rethinking of work flow, editing, and reporting responsibilities.

But questions remain about the best way to ensure that these updated or breaking news items are presented on the page for greatest visibility. Judging from the wide variety of design techniques newsrooms use to designate breaking news, there is no consensus on the best approach.

In May 2007 the top 102 US newspapers’ websites were analyzed to catalog the different ways “breaking” news was being displayed. We looked at labels used to indicate news was updated or new and the design techniques for differentiating “breaking” news from other news items on the homepage.

Labeling: Thirty-four of the online news sites examined had no designation of “breaking” news. Of the 68 sites that did:

  • 31% used some version of “Breaking”
  • 30% used some version of “Latest”
  • 14% used some version of “Update”
  • 25% used miscellaneous labels including “Developing News”, “News Flash”, “News Bulletin”, “News Alert”, “Up to the Minute.”

Design: The methods used to designate, design-wise, the “freshest” news items on the page varied, and were often combined.

  • 12% tagged individual stories with “New” or “Updated’, usually in a bold color
  • 57% put “updated” stories in a box
  • 62% timestamped the entire page and / or individual stories

It was clear that no conventions had been established for designating those news items that were freshest or most recently updated.

We designed the study to get at the following questions:

  • Did the design choices made to designate updated content affect the user’s recognition of which items were new or updated?
  • Would the design aid, or impede, the likelihood that the news user would find, read, and remember news items most recently added to the site?

How we did the study

We worked with the Minneapolis Star Tribune to get daily feeds of their homepage. We inserted into the “live” page a fake “breaking news” story (about a tank truck accident and subsequent chemical spill shutting down a major highway in town.) Each day of the testing, our designer created three test “home pages” using the top three ways updated news is being designated on websites – timestamp, labeled, boxed – to distinguish this “breaking news” story from the others on the page.

Version 1: Timestamped

Version 2: “New” placed next to updated story

Version 3: Updated story headlines in a separate box

Participants: We wanted to study a wide array of online users in this research so we took the eyetracking equipment to two locations in Minneapolis: the student union at the University of Minnesota (where the demographics were largely young, Anglo adults), and the Midtown Global Market Downtown (where the participants represented a wide range of demographics – age, race, education.) In all, 96 participants were tested, divided into each of the three conditions.

After the eyetrack calibration, research participants were asked to go to the homepage and “just look around.” They were told they could look however long they wanted and could click on whatever they wanted. When they indicated they were done, they were asked a couple of questions:

  • What are the main stories you recall from the website?
    The response to this question could be checked against the eyetrack video. Was it clear they “saw” the updated story yet did not list it as one of the stories they remembered?
  • What was it about these stories that made you remember them?
    The response to this open-response question helped catalog the attributes of the news story that made it memorable.
  • Which of the stories on the website was identified as a “news update” // or which of the news stories was the most recently updated?
    This question sought to discover if people, in fact, recognized that there was a story that was designated differently than the others.
  • Do you try to find the most current story when you go to a news website?
    This question would allow the researchers to see if there was a difference in response from users who are self-proclaimed “fresh” news seekers from those who are not.
  • On a scale of 1 to 5 (with 1 not at all interested and 5 very interested) how interested are you in: Politics, Crime, Traffic Reports, Sports
    This question could help researchers to see if interest in a news topic (in this case traffic) resulted in a higher degree of recall.

Findings

Both the eyetracking videos and the post-exposure survey responses were analyzed and matched – the videos to see whether participants eyes “fixated” on the breaking news visual cue and the survey to see their responses to the post-exposure questions. Here are some of the key findings:

Finding 1: Bigger is Better – or is it? In terms of visual cues, it was clear from the results of the eyetrack sessions that the larger the cue, the more likely it was to be noticed. By visual cue, the percent of participants exposed to that style of display who fixated:

Headline box: 89%
Timestamp: 48%
“New”: 49%

But when participants were asked in the post-exposure survey to say which story on the homepage was the most recent or which was the breaking news item, the participants exposed to the timestamp (35%) and “new” pages (32%) had greater recognition of the freshest news than those exposed to the headline box (20%). Even though 89% of the people exposed to the headline box page clearly looked at it, only 20% of them recalled any breaking news story.

What might this mean? Although the headline box drew more eyes, the headline text size in the box was smaller than the other two display styles. Headline text size may be an important factor in user’s memory of a story. So, although boxing the headlines made that visual element on the page more broadly seen, the small size of the headline type within the box made the recall of the story weaker.

Finding 2: Story Attributes Participants were asked in the post-exposure survey “What was it about the stories you recalled on the page that made you remember them?” Their open responses were analyzed and categorized (for example, if they said “I remembered that story because it happened near where I live.” Or “I drive that highway every day.” the response would have been coded as proximity or familiarity. If they said “I remembered it because of the color photo next to it” it would have been coded as “photo.” Here are the categories and findings:

I have a personal interest in this story 41.0%
I’m familiar with the topic or focus of the story 9.4%
There was something surprising or emotional 9.4%
Size / position of the story on the page 10.5%
Photo 8.4%
They had clicked on the story 5.2%

The attributes of the story and what made people recall them fell into two categories: personal and design. By far (66% vs 24%) it was the personal triggers (interest in the topic, proximity, familiarity, emotional response) rather than design cues (size / position on page, photo, hyperlinked headline) that were given as reasons for recalling the story.

Observations: The old adage “You can lead a horse to water but you can’t make him drink” probably fits here. If there isn’t interest, the design may well not have any impact.

There is a good deal more observations we can make based on this research, look for more results in future OJR columns.

Online usability questions that need answers

If your home page has a rotating menu of featured stories at the top, do users look at it, understand how it works and use it to navigate the site? How about a slideshow with text, sound and images: Do users attend to all three of these items at once? How about the number of links or other informational items on a home page: What number is ideal? When do we cross the line between being informative and becoming overwhelming?

If you’ve ever wondered about these questions, you are not alone. Some of the sharpest minds in online journalism gathered at the University of Minnesota’s Institute for New Media Studies about a month ago to discuss some of their pressing research questions and to set the research agenda for DiSEL: The Digital Storytelling Effects Lab.


Tyson Evans, LasVegasSun.com, discusses the value of research

The group is part of DiSEL’s first official consortium. Eleven news organizations made financial and time commitments to help Laura Ruel and Nora Paul (authors of this column) determine what to test with eyetracking, usability, and effects research methods.

It was no surprise to anyone that there were many more questions than we could answer in the first round of research. We had to prioritize. In this column we’ll let you know what’s on the horizon. We also invite you to add your thoughts to the mix by completing the DiSEL survey.

Riding a carousel; wearing a belt

No, it wasn’t a circus (well maybe!) or a fashion show. The terms “carousel” and “belt” were the focus of some big design questions. Carousels, the group determined, are the pieces at the top of a home page that rotate images and/or headlines to promote stories. Martha Stewart’s site, http://marthastewart.com, was one that the group referenced as having an interesting form for its “carousel.”

Graphic
Martha Stewart’s site directs users to content with a carousel.

In this case the carousel can be rotated in an autoplay mode (user clicks the “play” button) or manually (user clicks the section name).

“Carousels are coming up a lot,” said Jaime Hutt from startribune.com. “Are there standards?”

Others also wanted to know what works with this form.

Chris Snider from desmoinesregister.com was curious if it is more effective to use a carousel or have a user scroll a panel of story images to navigate to content that is not on the home page.

“What works in carousels and what doesn’t?” he asked.

Similar to carousels are belts. These navigational items usually appear mid-page or towards the bottom of the home page and also direct users to content that is not on the home page.

Belt questions centered around how users navigate and interact with them. Feilding Cage from Time.com discussed some of the questions his organization ponders.

Graphic
Time.com has a belt on every page.

“We are curious about the functionality of our belt,” said Cage. “Are users reading items on the second scroll and beyond? It may be more of a usability issue, but could be an eyetracking one as well.”

Cage went on to explain that there are 10 items on the belt, and there is a belt on every page. He said the top five items (the ones seen without clicking on the arrows in the upper right or left) are huge drivers to the stories they promote. There is a question of how to drive users to the other five featured stories.

“Should we use a ‘next’ button or is it fine that the belt moves from the first five items to the next five on one click?” he asked. “People most often click on the photos to get into the story.”

When is it TMI?

What is the ideal amount of information to present, and how should it be labeled so viewers notice it? How many supplemental links are useful, and when do they become too much?

“I was always struck with the plethora of information entry points on NYTimes.com,” said Nora Paul. “Can we test for impact on behavior? What about cognitive overload?”

The multimedia producers agreed that the issues of how much information to include – and how to successfully present it – are crucial.


Josh Hatch, USAToday.com, addresses labeling links and page density

“We want to know what images people are looking at on our home page,” said Josha Hatch of USAToday.com. “How much chatter do people read? What else are people looking at?”

Hatch also noted concern about whether users notice items such as interstitial ads, sharing buttons, polls and Flash interactives. He and others were curious about whether handcrafted story content (that looks similar to print design) is effective in capturing users’ attention.

Video viewing

Online video has become an expected media form for all news sites – not just those with broadcast stations as their parent.

YouTube.com also has influenced how users interact with video online. (Notice where the video with this piece resides?)

The questions about video presentation were numerous, but the group narrowed it down to three.

What is the best way to let a user know that video is available? Should video be labeled with an icon of a camera, with words, with a player that already is on the page or with a combination of these elements. A local news site in Raleigh recently won a regional Emmy award for its player that is a combination.

Graphic
WRAL.com recently won a regional Emmy Award for its video player that combines methods of interactivity.

What is the best way to present the video once a user clicks on it? Members of the consortium want to know if video should be embedded on the page and play in a specified spot, or should it emerge in a pop-up window at a larger size?

“On CNN, if I see a video icon, I don’t click on it because I don’t want to go to the video player,” said Hatch.

He added that if video is placed in a Flash player, many are not aware it is an option.

Cage added that Time.com is experimenting with different player interfaces, some similar to YouTube.com.

“Has YouTube trained people?” Hatch asked.

What is the best way to handle advertising with video? Advertising combined with online video is creating revenue for news sites. Consortium members decided to look at this issue by proposing experiments that assess the effects of the frequency and positioning of the ads. They also would like to discover if pre-video ads make users leave a site.

How to show your slides

Similar to video, audio slide shows are another basic – and expected – mainstay for news sites. Many questions arose.

“To what degree are users reading captions or script to audio?” Hatch asked. “Also, if ads are swapping out beside the slide show, how does that affect the user experience?’

“What is a good size photo to put on a galleries page?” said Amish Desai from yahoonews.com.


Gabriel Dance, NYTimes.com, discusses slide show usability issues

Consortium members agreed that the questions above should be studied. In addition, they listed these other important areas to research:

  • Should users be provided with the length of time for an auto play slide show, or does this information make them choose not to invest the time in the presentation at all?
  • What is the best way to present a slide show with multiple chapters?
  • Where is the best place to put cutlines?
  • What are the differences in user behavior when ads are placed next to slide shows vs. above slide shows?

    This meeting was a true collaboration among an energetic group of multimedia thinkers. Partner news organizations sent the following individuals to participate:

  • Dallas Morning News: Noel Gross
  • The Des Moines Register (Gannett): Chris Snider
  • Las Vegas Sun: Tyson Evans
  • The New York Times: Torben Brooks, Gabriel Dance
  • San Jose Mercury News: Randall Keith
  • Star-Tribune: Jamie Hutt, Jason Erdahl, Matt Thompson, Will Tacy
  • Time: Feilding Cage
  • USA Today: Joshua Hatch
  • The Washington Post: Nelson Hsu
  • Yahoo! News: Amish Desai

    Watch this column for updates as DiSEL embarks on answering the questions this group introduced. Columns on the DiSEL research projects completed last year will also be appearing soon.

  • Navigating slide shows: What do people choose when every choice is possible?

    During the month of May we rented the Tobii eyetracker to conduct a variety of studies about online news design decisions. Different designs for displaying “breaking news” and supplemental links were tested. We also looked at three variations of New York Times story level pages (the difference was the intensity and variety of supplemental information links available.) All three of these studies need some time to digest the data (from both the eyetracking behavior recorded and the survey responses by the participants.) They will be reported on in later columns.

    But as a little “add-on” study, we asked 34 of the people who did one of the other tests to also take a look at the Washington Post’s “Cuba by Korda” slide presentation.

    Image

    We were interested in seeing how people decided to navigate through this package which included every possible option for moving through the slides.

    There was a thumbnail view:

    Image

    You could click on an arrow next to the photo to go forward or back. Or you could use the “Next” button.

    Image

    There was an “autoplay” option that let you change the speed of the slide transitions.

    Image

    Or you could click on the individual numbers lined along the bottom which would reveal a thumbnail of the image associated with that number.

    Image

    We had a number of questions about use of this complicated navigational suite.

  • Given all these options – which one(s) did the user select?
  • Did one navigation style result in more complete viewing of the images?
  • Did people move linearly or non-linearly through the set of slides?
  • Did one navigation style result in more complete reading of the associated narrative?

    With this study we simply sent people to the site and asked them to look through the package as they would if a friend had sent them the link. There were no instructions about how long to look, just to go through the site until they had had enough. We did not ask them any questions about the experience or their preferences, we just recorded their eye-movements on the screen. Here’s what we found in an analysis of the eyetrack recordings:

    Navigation choice

    Of the 34 participants, their first navigational choice:

  • Next 19 (56%)
  • Numbers 8 (23%)
  • Arrow 5 (15%)
  • Autoplay 2 ( 6%)
  • Thumbnail 0

    11 of the 34 switched between two different navigation methods, and 3 of those 11 used three methods (not repeating any of them.)

    Of the 19 that started with the “Next” button:

  • 13 used “Next” the entire time
  • 4 used “Next” for an average of 7 slides then went to autoplay
  • 1 went to the thumbnails, looked at a few, then clicked on numbers
  • 1 clicked on numbers

    Of the 8 that started with the Numbers

  • 7 clicked through the Numbers the entire time
  • 1 went to “autoplay” after clicking on five numbers

    Number of slides viewed

    The average number of the 40 slides in the package viewed by those who used one method the whole time:

  • Next – 28 (70%)
  • Arrow – 25 (62%)
  • Numbers – 12 (30%)

    Nine of the 34 participants viewed all 40 of the slides – all of them started with the “next” method of navigating the slides. Seven of those nine used “next” the whole time, the other two went to “autoplay” to view the rest of the stack.

    For all the participants – the average number of slides viewed was 23.

    Time spent

    The average time spent with the slide show package was 2:55. The longest time was 8:17 (a young woman of Hispanic background – carefully read all the slide information). The shortest was 0:48. With these outliers removed, the average time spent was 2:49.

    For people who stayed with one method, here’s the amount of time they spent with the slides:

  • Next – 2:34
  • Arrow – 3:31
  • Numbers – 2:16

    Linearity

    One of the possibilities in designing online presentation is the option of moving through material linearly or non-linearly. Two of the navigation options facilitated non-linear exploration of the material – the “numbers” and the “thumbnails.” No one started with the “thumbnails” and of the eight who started with the “numbers” half of them clicked the numbers in order (linearly), the other half clicked around in random order. Of the half that clicked linearly, the average number of slides viewed was 20.75. Of the half that clicked randomly, the average number of slides viewed was only 6.5.

    Reading

    We viewed all the eyetrack recordings to see whether the participant read the related text about each slide.

  • Eleven (33%) of the participants carefully read the slide text
  • Sixteen (47%) skimmed or read the text sporadically
  • Seven (20%) did not look at the slide captions

    There was no predominant method of viewing the slides that resulted in a more careful reading of the text. Of the eleven seen as carefully reading the text, 4 used the “number”, 3 used the “next”, 3 used the “arrow”, and one used “autoplay”

    Observations

    In terms of practical advice, this observation of navigational methods use makes clear that if you can only have one navigation method – the “next” method, moving linearly through the set of slides is the one to use. It was the primary choice of the majority of the participants and resulted in viewing the most slides.

    However, if amount of time spent with the story package is your primary goal, people who clicked from slide to slide using the “arrow” spent almost a minute longer than the “next” users.

    The reason for some of the other observations (for example, why no one selected the “thumbnail” view as an initial navigation method and why so few (2) selected autoplay) is not known – we did not ask people about their choices (or about their possible confusion about the choices.) This would be an interesting project for a future time – to do more of a “think aloud” session about people’s navigational choices. But this observational study does provide some insights into the choices made by people faced with a variety of methods for navigating to through rich and deep slide shows.

    But perhaps the most interesting observation was the very low level usage of the non-linear approach (and when it was used, how few slides were observed.) Is the linear orientation to looking through material so hard-wired into our media usage that it is, and will continue to be, the preferred way to take in media? Even when it was visual information – as this was – and did not logically need to follow a narrative thread – people preferred to move through in the order it was presented. What does this observation tell us about innovation in digital storytelling and our audience’s tolerance for new design paradigms.