Laura Ruel and Nora Paul on OJR

Taking a ride with carousels

Wed, 04 Feb 2009 08:19:00 MST

By Laura Ruel and Nora Paul: Are rotating displays of Web content an effective way to promote news stories? This is the second in a series of articles about findings from the studies conducted for the member of the DiSEL's Eyetracking Research Consortium.

One challenge that faces all of us who have a wealth of content on our Web sites is how to best promote it. Unlike a print magazine or newspaper there is no big stack of paper to provide a clear physical indicator that there is much to read and experience beyond the front page.

Although we can debate the effectiveness of using a Web site�s front page when it comes to promoting content¹, our industry is trying a variety of methods to tackle this challenge. One method is the �carousel� � or a rotating display of a site�s content that appears in a dominant spot on the front of the page.

You can see a variety of carousel styles on sites such as MarthaStewart.com, aol.com, msn.com and even the newly redesigned whitehouse.gov. This past October, the Yahoo! Developer network launched the �carousel control� in their user interface library. They describe it as a widget that provides a means for �browsing among a set of like objects arrayed vertically or horizontally in an overloaded page region.�

So, the obvious question from the DiSEL research consortium was: Do carousels work on news sites?� Also, is there is a preferred design style that is most effective? With the help of page prototypes created by USAToday.com ² we put some carousels to the test. Here�s an overview of some of our findings.

What we did

In July of 2008 we tested 54 people with a mean age of 31. About three-fourths of the participants were women and one-fourth were men. Most had some college education.

This study was conducted in conjunction with two others that explored the ideal number of links and images on home pages. (More on these studies in future columns.) Test subjects were solicited via Craigslist, local newspaper sites and blogs. Each participant was given a $20 Target gift card for participating. The entire test took about 40 minutes per participant.

The third test conducted was the carousel test. Users were directed to one of three home pages and asked to �browse the site as you normally would� and to �tell the experimenter when you have seen what you would like of the site.�� If test subjects browsed beyond five minutes, they were asked to stop.

While users browsed the site, we tracked their eye movements using the Tobii eyetracker. �

After the browse time, users completed a questionnaire that assessed their thoughts about the site�s effectiveness and collected demographic information.

Here are the three different home pages viewed, with links to the actual test sites.

Arrow version (Link to test site)

This site�s home page contains an automatically changing carousel that allows the user to control the rotation of the stories by clicking on navigational arrows in the upper right of the carousel element. Seventeen of the 54 people tested saw this version of the carousel.

Dot version (Link to test site)

This site�s home page contains an automatically changing carousel that allows the user to control the rotation of the stories by clicking on navigational dots in the upper left of the carousel element. Nineteen of the 54 people tested saw this version of the carousel.

Thumbnail version (Link to test site)

This site�s home page contains an automatically changing carousel that allows the user to control the rotation of the stories by clicking on thumbnail images to the left of the main image in the carousel element. Eighteen of the 54 people tested saw this version of the carousel.

What we found

To make these results useful to those thinking about carousel use as a promotional tool, we examined the users� eyetracking and also asked them survey questions to provide us with some of their overall impressions.

Survey says�.

As the chart below indicates, we found that the type of carousel used did not seem affect how engaging users found the home page. Interestingly, though, we did see that users who viewed the home page with the Thumbnail version of the carousel, felt more strongly that the Web site was easier to navigate.

Those viewing the Arrows version also had a stronger desire for more stories on the home page. This could be due to the fact that the arrows navigation did not clearly indicated the number of stories available in the carousel.

Another result worth noting is that users viewing the Thumbnail version of the carousel seemed to indicate more than the other two groups that the overall site was easy to navigate.

All three groups were fairly neutral when it came determining which stories were most important. � The carousel style did not seem to provide them with a clear indication of story hierarchy.

In light of that, we also wanted to know how did users would respond to this question:� What was the headline for the main news story on the site?

As the chart above indicates, is seems that users who viewed the Arrow version of the carousel were more apt to say the first story appearing in the carousel was the �main� story. Those with the other two versions were more apt to say that it either all or some of the stories that appeared in the carousel element.

All these survey results � while purely observational � may suggest that editors ask themselves what their goals are with one type of carousel-style presentation over another. Obvious navigation � such as thumbnails � seems to encourage users to view the site navigation as easy. It also may be an indicator to users that all stories within the carousel are of equal importance.

But, this is what users SAID. Let�s take a look at the hotspots to see what they DID.

�Eyetracking shows�

Eyetracking offers researchers valuable information about where users actually look on a site. It is accomplished by calibrating a user�s pupils with a small camera hidden in the base of the computer monitor. Calibration takes only a few seconds, but the tracking results allow researchers to know where users� eyes went within a centimeter of accuracy. (For more on how eyetracking works, click here.)

We can get a variety of data forms from eyetracking. An aggregate view of what viewers saw is displayed in a �hotspot� or �heatmap� of the Web page studied. The chart to the left details how to read this data.

We generated hotspots for users� fixations on each version of the carousel. Here are hotspots from the top portions of the page, which seemed to show us the most interesting data about the pages. See the actual hotspots from the pages below.

Past results ring true

As eyetracking researchers, we always will spend some time observing what trends are similar to results we found in past studies. In this case, we found two things worth mentioning:

The areas of the photos that got the most fixations were faces.

Top navigation gets the most use.

Past studies have shown that faces in photos are the areas where eyes tend to fixate. From observation during this study (and the heatmaps below) you can see that this is true again. No matter what carousel navigation style was used � and regardless of the size of the photo or the size of the person in the photo � users tended to look for and find that human element to relate to. (Stay tuned: In a future column, we will go results of a study conducted with prototype pages from the DesMoinesRegister.com that experimented with size and quantity of images on a home page.)

We also saw that the carousels that employed top navigation elements (Arrows version and Dots version) got more clicks than the Thumbnail version. Other factors (discussed below) contributed to this observation, but it is interesting we consistently find users viewing and using navigation more if it runs along the top of a page or page element.

Attention to change

One clear observation is that the Dots version had a higher percentage of user fixations on all elements on the top part of the page than the other two versions (Arrows and Thumbnails).

Why? Well, one clear difference here is that the navigational dots changed color when story changed. The change had high color contrast as well � from a blue dot to a white dot. The moving dots seemed to draw attention to the page overall.

The Arrows version had no visible change in when stories changed in the carousel. In the Thumbnail version the box around the image changed when the story changed, but the movement and the change in color contrast was not as marked or clear as the dot changes.

So it seems that the moving navigational dots in the Dots version encouraged a higher percentage of users to view all aspects of the top portion of the page. For example, only about half of the users looked at the page headlines on the arrows and thumbnail pages, but closer to 70 percent looked at this list when the dot navigation existed.

Interestingly, though, even though there was a higher percentage of eyes on the headlines in the Dots version, there were less total clicks in this area of the page than on the other two. In addition there were more �dead� areas � or areas with zero eye fixations � on the tops of the pages with the Arrows version and the Thumbnail version than on the page with the Dots version.

The type of carousel used did not seem to affect the number of clicks on the carousel photo and the corresponding headline and blurb.

These observations seem to suggest that if you are going to use a carousel element it may be best to place navigational elements above the display elements and to utilize a navigational design that clearly indicates the change from one story to the next.

But � what do you think?� Below are the hotspots from each carousel version. We look forward to your thoughts and comments about what you see � and why. � Post your questions here. We�ll respond.

Arrows version�

Automatically changing carousel that allows the user to control the rotation of the stories with navigational arrows in the upper right of the carousel element.

Dots version

Automatically changing carousel that allows the user to control the rotation of the stories with navigational dots in the upper left of the carousel element.

Thumbnails version

Automatically changing carousel that allows the user to control the rotation of the stories with the use of thumbnail images to the left of the main image in the carousel element.

¹ Some news sites editors have mentioned �off-the-record� that less than 20 percent of their story traffic comes from people linking to pieces from the home page. The rest comes from blog posts, Google searches, Delicious links and other forms of online promotion.

² Many thanks to DiSEL Research Consortium members Josh Hatch and Kristin Novak at USAToday.com for creating the testing materials. Also, thanks to UNC-CH Ph.D. candidate Bart Wojdynski for his assistance with designing this study and running test subjects.

Eyetracking research shows how younger readers view news websites

Wed, 10 Dec 2008 04:29:00 MST

By Nora Paul and Laura Ruel: In January 2008 a group of interactive producers from news websites gathered at the University of Minnesota for the first Eyetracking Research Consortium, part of the Digital Story Effects Lab project run by Nora Paul and Laura Ruel. Following is the first in a series of articles about findings from the studies conducted for the Consortium members. For more information about the Eyetracking Research Consortium, go to www.disel-project.org

San Jose Mercury News

Some of the eyetracking studies conducted with the consortium members were comparisons of different design approaches or navigational schemes and their impact on user behavior. Other members just asked for feedback from users about their experience on the website. With the eyetracking we could record not just what someone said about what they did on the site, but to actually see what they did.

The San Jose Mercury News wanted to see how their site compared with the Contra Costa Times site � a sister site with a slightly different emphasis on visuals on the home page. They wanted an unstructured study of how people engage with the two sites and then to hear reactions.

Between April 29 and May 1, 2008 fifteen undergraduate journalism and mass communication students were brought in for the eyetracking session. All were between the ages of 19 and 22. All self-identified as being very comfortable with the web. They were set-up with the following scenario: "You are considering moving to northern California for a job and decide to look at two regional news websites. Look at the sites as if you were sitting in your own room. Go where you want to on the site, stay for as long as you would normally. Let us know when you are done." Half of the participants saw the Contra Costa site first then were sent to the San Jose site, the other half did it in reverse order.

San Jose was also interested in the usability of their calendar function. We asked a few of the participants specifically to go to the Things to Do function and search for particular items: a concert on Friday night, the movie "Forgetting Sarah Marshall," sushi restaurants. We also asked them to look at the My News / My Blogs function. This video shows one of the participants' use of those two functions:

Comments / Observations: In the post-exposure discussion about the sites we asked participants if they remembered a "Things To Do" feature. Seventy-two percent did not recall anything like that. Some of that can be attributed to the fact that the features is below the scroll and people didn't scroll down the page:

Click to see full image

Other times they did scroll but they simply didn't fixate on it.

Additional studies / observations: Contra Costa and San Jose use a left hand embedded ad style on their story level pages. We looked at all of the stories that were linked to by the participants to see if the left-embed ad was fixated on. Ninety-nine stories were clicked on by participants within the two sites, an average of 6.6 stories per participant or 3,3 stories per site. The most number of stories clicked by one participant was 11, the least was three.

We looked at the information that was available about the stories that were clicked on (headline only, headline and a blurb, headline and photo and blurb, or headline and photo only.) Fifty-nine of the 99 stories clicked on were headline only, with 17% headline / blurb and 21% headline / blurb / photo. (Two were headline / photo links � they were ARA "stories", essentially advertisements.)

Of the 99 news stories clicked on 63 were stories with the left-embed ad. We looked at the gaze plot for each of these participants and saw that for 23 of the 63 embedded ads was there some level of "fixation" or about 36%. Sixty-four percent of the time embedded ads were ignored. You could see that the people read right around the ad.

We also looked at whether people went to the right rail ads on the page. Only 22% of the story level viewings indicated any looking at the right rail advertising.

By contrast, though, 56% looked at the left rail. In the left rail was the "most viewed / most blogged" listing and a "top classifieds" box. Sixty-nine percent of the viewings of left rail content was for the "most viewed", 30% looked at both of the content features in the left rail.

Click to see full image

The other content on story level pages is sidebar or related material boxes. Fifty-two of the stories looked at contained sidebar material. When people went to those stories with sidebars, 75% of the time the material was looked at � mostly when there was a box in the upper right with a photo or slide show.

Click to see full image

The participants were interviewed after they went through both of the sites. Following is the transcript of one of the interviews, some of the insights into how young adults use news sites and different features and functions are interesting.

Interviewer: You looked at two websites. The first one was the San Jose Mercury News. Let's start with that. Was there anything that stood out?

Interviewee: I noticed that there were columns. I paid more attention to the middle column.

Interviewer: Which was?

Interviewee: I don't know. I think it was headlines. There was a most viewed section too. I think I clicked on something there. I kind of skipped over all the political stuff because I blocked those out. That's not a good attitude.

And I think there was some kind of moving ad that caught my attention because there were women in it moving and laughing.

The weather at the top was nice.

Interviewer: There are different ways you can navigate the site. So what did you prefer and what did you use?

Interviewee: I just remember clicking the titles of stories in all three columns. But they were kind of up at the top a little bit more. I tend to think of stuff down at the bottom as just like useless. There's not usually very many links down there.

Interviewer: The second website was the Contra Costa site. And the same question here. What kind of features do you remember?

Interviewee: I remember that the most viewed / most sent was way down at the bottom. And it's usually like way up at the top. But they had celebrity news. Like where I usually look right in the center column with all the most important, I don't know, bad celebrity stuff in there which I usually don't see on anything other than AOL. And I thought that was a little funny. Although I never heard of the Contra Costa before. Is that like a location?

Interviewer: Yep, a city in northern California.

Interviewee: Oh, okay. Well, I thought they were both very jumbled. Both websites were very busy. They had lots of columns. And text wasn't very large. It was very small. And not a very readable typeface. So I didn't really like the layout of either one.

I'm used to � I spend a lot of time on like AOL. So they've got like a box with images and then text. And it kind of switches, too. I don't know. That's not really a news website though. It's kind of like an entertainment thing. I don't know. I could have done with a little less pictures, a little more text.

Interviewer: In both of them or-

Interviewee: Mostly in the second one, the Contra Costa.

Interviewer: So which of the two do you prefer?

Interviewee: The first one. The San Jose Times.

Interviewer: Okay. What was in it that-

Interviewee: I don't know it just seemed more informative. But I saw some of the same stories echo between both websites. So I thought the first one was a little more interesting because by the time I got to the second, the Contra Costa one, I had already seen the story. San Jose Times, I just liked it because it seemed more professional.

Interviewer: In what sense? What made you feel it was more professional?

Interviewee: I guess just the layout. There's a little less jumble. Like the headlines were larger. I don't think it buried the stories. And they started with topics that were a little more sober. They didn't go right off the bat with like � stars. I can't remember.

I kind of look at � the movies were good. Local news was good. And stuff that affects me, I guess.

Interviewer: Okay. Now there were a few places that you could do things - for example, you had an option to create a section for my news. Did you notice those?

Interviewee: No. I noticed that I could personalize the weather.

Interviewer: The weather is important.

Interviewee: I like the weather.

Interviewer: In which of the news websites was that?

Interviewee: San Jose Times had a weather thing up at the top. No, I didn't really notice any of the interactive stuff I guess.

Interviewer: Did you notice the photo slideshows?

Interviewee: No, just that one ad with the moving, laughing women.

Interviewer: Which website was that on?

Interviewee: I think it was the first one. The San Jose Times. It was over on the right.

Interviewer: Did you notice the web polls or web vote? You know they ask you questions-

Interviewee: No, I never vote on those. They always try to set you up.

Interviewer: In what way?

Interviewee: Like on AOL it'll ask you the stupidest questions. Like I spend most of the time on AOL. And every single news story that you view, it'll have some kind of � I think it's some kind of trick to get people to interact more with the story. But they'll have a little question at the bottom like, do you agree? Do you think this is right? It could be like a � like the kid who was like drowning the swimming pool. It would be like, do you think the conditions could have been stricter? And I just get so tired of those things.

Interviewer: Now did you notice a bar with Digg and Delicious and Facebook on it?

Interviewee: No, but I don't share stories. I don't email stuff out unless it's about dogs. 'Cause my mom really likes dogs. And she's pretty much the only one I'd send that to.

Even without eyetracking, this kind of session with casual users of the site can give valuable feedback into what works, and what doesn't, about your site's design and features.

Continuous Updates: Design decisions when designating breaking news

Wed, 24 Sep 2008 10:02:00 MST

By Nora Paul and Laura Ruel: This is one in a series of reports on DiSEL (Digital Story Effects Lab) Research projects conducted in 2007 through a research grant from the University of Minnesota. First in the series was on Navigation through Slide Shows

Why we did the study

One of the great strengths of the Web is the ability to keep news updated and to alert readers immediately to stories they need to know about. This is also one of the biggest organizational changes the Web has brought to newsrooms. Shifting from daily to constant deadlines has caused a rethinking of work flow, editing, and reporting responsibilities.

But questions remain about the best way to ensure that these updated or breaking news items are presented on the page for greatest visibility. Judging from the wide variety of design techniques newsrooms use to designate breaking news, there is no consensus on the best approach.

In May 2007 the top 102 US newspapers' websites were analyzed to catalog the different ways "breaking" news was being displayed. We looked at labels used to indicate news was updated or new and the design techniques for differentiating "breaking" news from other news items on the homepage.

Labeling: Thirty-four of the online news sites examined had no designation of "breaking" news. Of the 68 sites that did:

31% used some version of "Breaking"
30% used some version of "Latest"
14% used some version of "Update"
25% used miscellaneous labels including "Developing News", "News Flash", "News Bulletin", "News Alert", "Up to the Minute."

Design: The methods used to designate, design-wise, the "freshest" news items on the page varied, and were often combined.

12% tagged individual stories with "New" or "Updated', usually in a bold color
57% put "updated" stories in a box
62% timestamped the entire page and / or individual stories

It was clear that no conventions had been established for designating those news items that were freshest or most recently updated.

We designed the study to get at the following questions:

Did the design choices made to designate updated content affect the user's recognition of which items were new or updated?
Would the design aid, or impede, the likelihood that the news user would find, read, and remember news items most recently added to the site?

How we did the study

We worked with the Minneapolis Star Tribune to get daily feeds of their homepage. We inserted into the "live" page a fake "breaking news" story (about a tank truck accident and subsequent chemical spill shutting down a major highway in town.) Each day of the testing, our designer created three test "home pages" using the top three ways updated news is being designated on websites - timestamp, labeled, boxed � to distinguish this "breaking news" story from the others on the page.

Version 1: Timestamped

Version 2: "New" placed next to updated story

Version 3: Updated story headlines in a separate box

Participants: We wanted to study a wide array of online users in this research so we took the eyetracking equipment to two locations in Minneapolis: the student union at the University of Minnesota (where the demographics were largely young, Anglo adults), and the Midtown Global Market Downtown (where the participants represented a wide range of demographics � age, race, education.) In all, 96 participants were tested, divided into each of the three conditions.

After the eyetrack calibration, research participants were asked to go to the homepage and "just look around." They were told they could look however long they wanted and could click on whatever they wanted. When they indicated they were done, they were asked a couple of questions:

What are the main stories you recall from the website?
The response to this question could be checked against the eyetrack video. Was it clear they "saw" the updated story yet did not list it as one of the stories they remembered?
What was it about these stories that made you remember them?
The response to this open-response question helped catalog the attributes of the news story that made it memorable.
Which of the stories on the website was identified as a "news update" // or which of the news stories was the most recently updated?
This question sought to discover if people, in fact, recognized that there was a story that was designated differently than the others.
Do you try to find the most current story when you go to a news website?
This question would allow the researchers to see if there was a difference in response from users who are self-proclaimed "fresh" news seekers from those who are not.
On a scale of 1 to 5 (with 1 not at all interested and 5 very interested) how interested are you in: Politics, Crime, Traffic Reports, Sports
This question could help researchers to see if interest in a news topic (in this case traffic) resulted in a higher degree of recall.

Findings

Both the eyetracking videos and the post-exposure survey responses were analyzed and matched � the videos to see whether participants eyes "fixated" on the breaking news visual cue and the survey to see their responses to the post-exposure questions. Here are some of the key findings:

Finding 1: Bigger is Better � or is it? In terms of visual cues, it was clear from the results of the eyetrack sessions that the larger the cue, the more likely it was to be noticed. By visual cue, the percent of participants exposed to that style of display who fixated:

Headline box: 89%

Timestamp: 48%

"New": 49%

But when participants were asked in the post-exposure survey to say which story on the homepage was the most recent or which was the breaking news item, the participants exposed to the timestamp (35%) and "new" pages (32%) had greater recognition of the freshest news than those exposed to the headline box (20%). Even though 89% of the people exposed to the headline box page clearly looked at it, only 20% of them recalled any breaking news story.

What might this mean? Although the headline box drew more eyes, the headline text size in the box was smaller than the other two display styles. Headline text size may be an important factor in user's memory of a story. So, although boxing the headlines made that visual element on the page more broadly seen, the small size of the headline type within the box made the recall of the story weaker.

Finding 2: Story Attributes Participants were asked in the post-exposure survey "What was it about the stories you recalled on the page that made you remember them?" Their open responses were analyzed and categorized (for example, if they said "I remembered that story because it happened near where I live." Or "I drive that highway every day." the response would have been coded as proximity or familiarity. If they said "I remembered it because of the color photo next to it" it would have been coded as "photo." Here are the categories and findings:

I have a personal interest in this story 41.0%

I'm familiar with the topic or focus of the story 9.4%

There was something surprising or emotional 9.4%

Size / position of the story on the page 10.5%

Photo 8.4%

They had clicked on the story 5.2%

The attributes of the story and what made people recall them fell into two categories: personal and design. By far (66% vs 24%) it was the personal triggers (interest in the topic, proximity, familiarity, emotional response) rather than design cues (size / position on page, photo, hyperlinked headline) that were given as reasons for recalling the story.

Observations: The old adage "You can lead a horse to water but you can't make him drink" probably fits here. If there isn't interest, the design may well not have any impact.

There is a good deal more observations we can make based on this research, look for more results in future OJR columns.