When BYU business student Richard Miller was assigned to read the print Wall Street Journal earlier this year, he did what a lot of people trying to manage reading material do these days -- he traded online for print, then went looking for an RSS feed that would deliver headlines to his desktop and let him know when new stories posted.
Miller checked a syndication feed directory, Syndic8.com, for links to feeds but couldn't find any provided by WSJ.com. The links he found in Syndic8's directory to old "scraped feeds" -- scripts pieced together by scraping through pages for the coding surrounding headlines -- didn't work. Other users might have shrugged and moved on. Miller, class of 2005, went to work.
In a few hours spread over several nights, he diligently scraped from six WSJ.com index pages the HTML codes used for the site's headlines, wrote a script using his own cookie to harvest the headlines, set up a Web page to host the script and posted that link to Syndic8's directory. "On Syndic8.com," Miller explains, "they call this process scraping because you don't have a perfect feed but you scrape off what you can."
By writing this script, he yanked from Dow Jones control over his information gathering, as did the more than 8,500 unique users who found his feed through Syndic8 -- for a few months anyway. From February 17 until mid May, when an apparently unrelated change in the way the Journal set its cookies shut down his feed, Miller's makeshift effort had over 170,000 hits. Headlines were available to anyone with an RSS reader, but only WSJ.com subscribers could access the articles.
Richard Miller's creative use of script is not unique, especially given that RSS relies on open standards using XML. The Web is as littered with scraped feeds as it is with the orange XML and RSS boxes widely used to identify that a site offers its own feeds. Miller and his fellow readers aren't stressed about business models or strategic planning. They know content feeds can be done. They think it makes their lives easier. They either find a way to do it or they move on, abandoning sites that don't make the effort to provide feeds.
Today's online news universe might be divided into outlets that have joined the RSS ranks, ones that have declined and those that continue to take a close look. Without doubt, a mechanism that drives traffic to a site would be beneficial, but what are the downsides? The RSS fence sitters point to several potential problems, specifically, that the technology can:
- Replicate indexes at off-site locations and divert potential ad audiences;
- Offer another opening for content aggregators;
- Confuse mainstream users because it lacks standardization and is still emerging from an early-adopter phase.
Low barrier to entry
Until now, it probably caused little damage to ignore or postpone syndicating Web feeds. It was more of an opportunity cost than an actual loss. Sites that bought in early got some "tech cred" and a spot in the RSS readers. They also expanded their reach considerably across the blogging community and their traffic increased. The Christian Science Monitor's csmonitor.com started in October 2002 with 1,000 RSS files served; last March it served nearly three million. Sites adding it recently see mounting interest, particularly as more articles about RSS make it into the consumer press.
But news sites that don't offer even a front-page headline feed in this online universe risk becoming irrelevant not only to bloggers who can drive traffic with a mention of a story but to increasingly savvy news consumers like Miller who want control.
That's especially true given the low cost of entry for news outlets using a content management system -- usually a few hours of developer time per site for coding.
That low barrier opens the door for people like Miller. "I thought about the ethicality of using my own cookie to supply headlines but I thought it was OK," Miller, said during a telephone interview. "The cookie that I send is in my script in my server; it never even comes to their (other users') news reader program, let alone their browser."
He added: "It's so convenient. I just can't go to the Web site every day and make it a priority to look for stories that interest me. I just won't use it if it's not an RSS feed."
Even an imperfect feed was better than none to him and to the users he's heard from who are disappointed about losing his WSJ.com feed. Fortunately for them, Dow Jones has caught on to the interest in RSS and plans to launch an RSS feed "in coming weeks" for the main sections of the Online Journal, Jessica Perry, VP-business development, said via e-mail. Like Miller's set-up, full stories will be available only to subscribers.
Perry said she wasn't familiar with Miller but "we're aware that there are a couple of subscribers who have built rogue feeds in violation of the WSJ.com Subscriber Agreement." She added, "Interestingly, any impact on the RSS feeds from our recent change in policy re concurrent usage was merely a by-product. We weren't trying to shut anything down."
Why is Dow Jones adding feeds? "We expect this will be a traffic-driver that serves our most news-hungry users," Perry wrote. "We have had a few inquiries from subscribers seeking to gateway into WSJ.com through RSS, and it may attract some new subscribers as well. Our WSJ.com Network free sites, such as CareerJournal.com will be included in the feed, and those stories will be accessible to non-subscribers as well."
That should give Dave Winer, coiner of the phrase "really simple syndication" as an alternative to "rich site summary" and developer of several versions of RSS, a reason to try the Journal again. Winer only reads RSS-enabled sites (he subscribes to 300-plus feeds) and hasn't visited WSJ.com in years. Winer also developed Radio UserLand, a pioneer Web publishing and blogging tool with one of the first feed readers and news aggregators.
"I'm not everybody," Winer admits. "I'm not a huge number of people. But I'm a growing number of people." How many is nearly impossible to gauge. There are no reliable Web-wide measurements of RSS use. Some sites don't even track RSS traffic.
"They should support it because their users are probably exactly the people supporting RSS," says Winer. "What do they have to lose by doing it?" Teasing him could push him into subscribing, he adds. "They're not in my face often enough for me to get the idea that I should subscribe."
He didn't have to give up nytimes.com. Instead, he convinced New York Times Digital CEO Martin Nisenholtz that RSS was worth a try. Nytimes.com joined Winer's Radio UserLand in 2002 as a content partner with Winer doing the set-up. Don't bother looking for an orange icon or even the letters RSS on the front page. Only Radio 8 users can link to the Times feeds from UserLand's site; others can link from a feed directory Winer maintains as a fellow at the Berkman Center for Internet & Society at Harvard Law School.
"I haven't been to the New York Times home page in years and yet I read 20 articles a day in the New York Times," says Winer. "I remember when it was first available in RSS how much more I saw."
"We're always looking at it"
That early entry gave nytimes.com an advantage, but the site now has to move past the legacy of the deal with Radio UserLand -- a lack of on-site promotion and some user confusion about whether nytimes.com even has feeds. "We're always looking at it," says product manager Eliot Pierce, adding that the orange buttons can be confusing, too.
Now that washingtonpost.com offers RSS, the next step is making it more accessible to users, says Executive Editor Douglas Feaver. The site mentions RSS on the front page but hasn't promoted it. Feaver doesn't feel pressed to use the feeds as anything but traffic drivers for now. He puts it in "the category of things we all need to know how to do."
"You need to know what it is you're trying to get out of having an RSS feed," cautions Eric Bauer, information architect for Boston.com. "Early adopters looked at RSS as something cool that adds a little wow to its site. That phase is over. A lot of people have RSS. It's still not universal by any means but not unique." Boston.com is getting 10,000-15,000 page views a week from RSS feeds. Without marketing beyond the RSS link on the front page, traffic has increased 10-fold since the feeds were introduced earlier this year -- partly in response to reader requests.
"I think fundamentally for us it's a way of getting your content out among a group of users who are very heavy users of the Internet and I think you would find very heavy users of news."
What would Bauer tell counterparts at non-RSS sites? "To me the thing that sort of speaks to doing it if you're a newspaper site is I think news content is particularly well suited to RSS, the fact that it auto updates and the fact that it allows the user to sort of control the clutter."
But there is a downside compared to e-mail. "You don't have much control over how it's presented so you can't feature an article or a package in the same way you can in email and say it's the single most important thing of the day," Bauer said.
In RSS feeds, all headlines look the same and are given equal weight. Some sites are learning to pay particular attention to the blurbs that can accompany headlines knowing that may be the key to enticing readers.
At nytimes.com, Pierce, who is responsible for the site's e-mail products, is looking at the possible use of RSS as a spam killer. Like his counterparts at other news sites, he doesn't think that will be very practical until it's as easy for the average reader to subscribe to RSS as it is to sign up for e-mail or click on a browser link.
"From my perspective the next level is to make it so easy any user could sign up," says Pierce."I don't feel the technology or the players are there to expose it to our whole audience."
The Opera Web browser includes RSS capability in its most recent version, 7.50. A lesser-known Internet Explorer act-alike called Slim Browser also includes a built-in RSS reader. Potentially more important to RSS adoption, Yahoo is testing RSS though its personalized My Yahoo feature. Users can add up to 50 RSS feeds to their customized front page by checking boxes on lists provided by Yahoo, including a list of recommendations based on the user's current subscriptions. Users can also subscribe to a site that isn't on Yahoo's lists by pasting in the RSS feed's URL.
Scott Moore, recently promoted to general manager of Microsoft's MSN Network Experience, wants to include RSS readability in that portal's personalized product. "One of the things I would like to see in the next version of My MSN is RSS integration," said Moore. MSN is also creating its own feed for video on the site. Moore said MSNBC.com will be adding RSS "quite soon."
"It's a good tool to have for bloggers to have access," Moore added. "I don't think the average consumer cares a heck of lot." That won't change, Moore said, "until the technology is packaged and delivered in a manner that's super easy for a consumer to understand the value of it. ... We've just said over and over and over that unless you make things dead simple in terms of a media offering, people just don't bother."
Microsoft's integration of RSS into Internet Explorer or Outlook has repeatedly been described -- often ruefully -- as one of the steps, if not the step, that would move RSS from niche tool to wide use. For now, Microsoft's official use -- including feeds for 600-plus blogs by employees, the Microsoft Developer's Network and Slate -- doesn't come close to mass integration.
During a speech at Microsoft's CEO Summit 2004 last month, Microsoft Chairman Bill Gates devoted seven paragraphs to his thoughts on RSS. He didn't announce that RSS or some other form of Web syndication would be integrated into Internet Explorer or Outlook. But by speaking about the value of alerting users when content has changed and calling RSS a standard, he sent a signal that he believes it is worth a look.
Salon, available only by subscription or ad-sponsored day pass, takes the teasing approach with a headline-only feed; it's being redesigned to include more information. Non-subscribers who click on a headline go to a page that offers one of those two options to gain access to the full story, while subscribers with enabled cookies are sent directly to the story.
Executives at Belo Interactive are concerned that the company's registration requirements could confuse readers who expect to move from a click to a story. Jay Small explained via email that this "could be especially disconcerting for out-of-market visitors who just happened upon a headline of interest." Of course, bloggers don't need RSS to post a link, but Small's concern illustrates the myriad details now being addressed by some news outlets.
In addition to usability, several issues often come up when news executives offer reasons for delaying RSS, including too many news readers, the rivalry between RSS models and an alternative called Atom, and the possibility that Web syndication could undermine the ad rates based on driving traffic through a site's front page.
Morris Communications is "playing with it, which I think is appropriate," says Steve Yelvington, vice president, strategy and content for Morris Digital Works. "The numbers are not large enough to make it important. We're monitoring it to determine if it's meaningful from a business perspective."
Yelvington wonders about making sure the feeds match the capabilities of various RSS readers. "There is so little predictability of whether this stuff is going to work from one reader to another," says Yelvington. "Some readers ignore images, others include them."
Others worry that multiple versions of RSS readers with varying subscription methods and viewing experiences can make it more difficult to promote RSS.
As for the battle between RSS and Atom, sites that already have RSS expect their versions to be supported by whichever standard reigns -- so the competing formats aren't holding them back. Google uses Atom for its blog service and in its beta test for news groups, but so far most news sites pick from the variety of RSS versions.
|Online News and RSS
The advertising question
Belo Interactive has done what Jay Small, director of product development, calls "proof of concept" testing "just to show ourselves we could provide feeds automatically from our content systems." He offers ProJo.com's feeds as an example, adding "the technology itself is definitely not standing in our way."
But "pure business" is a factor keeping Belo from embracing RSS. "Our own index pages display ads; that's one way we pay the bills," explains Small. "Despite some clunky initial examples, I've yet to be impressed by any demonstrated customer experience for advertising in RSS. So if we go out and expose feeds that replicate our own indexes, ad-free, we risk diluting potential audience for the ads we serve. That may sound defensive and reactionary, but what it amounts to is: We're waiting as fast as we can."
Yelvington sees potential in classified ad RSS feeds. A rough prototype is in the works at Jacksonville.com, and he expects to try that before deploying RSS news feeds.
Scott Rosenberg, Salon senior vice president of editorial operations/managing editor, in an e-mail interview compared the site's RSS feed to an e-mail newsletter in that both are "a way of dangling headlines in front of interested readers to encourage them to visit the site."
But Rosenberg doesn't see RSS as a potential solo revenue stream, although he says the company is closely watching RSS advertising experiments like the text ads-in-feed at InfoWorld. "Plainly, ads in feeds would need to be relevant to the user -- perceived to have some value -- otherwise, the user will just unsubscribe," Rosenberg wrote. "My hunch is that ads-in-feeds is not the future; instead, the business value of the feed is in building traffic to the site. With the ad market picking up again, everyone is back to thinking about ways to build traffic."
Winer doesn't like the in-feed concept. "I can only comment as a user. I'm not a visionary when it comes to advertising on the Internet," he said. "On the other hand if they said pay me some money to get rid of it I wouldn't do it."
Csmonitor.com actually offers an RSS text ad on its rate card but nobody has bought it yet, according to partnership development specialist Joel Abrams. "I suspect it's a hard sell for our ad reps," Abrams wrote in an e-mail. "It's difficult enough to sell a banner ad, which everyone pretty much understands at this point."
As for front-page advertising, sites with successful e-mail products already get a heavy amount of traffic that doesn't come in the front door. At nytimes.com, Christine Mohan, a former company spokesperson now on the product development side and working on RSS, says: "Frankly, our e-mails are driving as much traffic to our site as the front page." At one point last summer, the e-mail referrals exceeded front-page referrals to stories. They want to grow RSS in a way that doesn't detract from already successful products like the newsletters and the news tracker.
MSN's Moore, who is still responsible for Slate and MSNBC.com until a successor is chosen, acknowledges the risk RSS poses for publishers. "I think the interesting question is how is the technology of RSS is going to affect the business models of publishing," Moore explained. "I've heard people say it's sort of like TiVo. If people have the ability to disintermediate the publisher ... then we have a problem." But that problem can be avoided as long as users still need to visit the site to retrieve content beyond the headline.
Adds Moore: "In that case, it's not particularly troubling. In some ways, it's a benefit for publishers who don't have a portal attached to them."
After exploring various RSS readers, Jay Small at Belo is softening his resistance to RSS as a possible substitute for e-mail newsletters. "I used to say RSS doesn't represent 'distribution' in a sense as meaningful as e-mail. But I do think RSS reader software has evolved to be nearly as intuitive as the concept of an e-mail inbox."
Changing news consumption
CNET Networks' News.com launched its first feeds in 2000. John Roberts, assistant vice president of product development for news, estimates the company now hosts hundreds of feeds. He hasn't been able to get a good sense of how the feeds alter user behavior on CNET, but his own habits shifted considerably with the advent of RSS.
"Personally, I find I pay attention to a lot more information at a lot more sites," Roberts wrote in an e-mail. "I was never a big bookmark user, mostly typing in URLs of sites it occurred to me to check. I think this is what aggregation always wanted to be, and I love that it's under my control. I wouldn't go back." Does the trend extend to his users? "I am always curious whether this transformation is common or not. TiVo altered my TV consumption habits dramatically. RSS has done the same for my online consumption. TiVo fans usually are more rabid than RSS fans, but I wonder."
CNET is testing that theory right now with the recent removal of summaries from its News.com feeds. "We're watching our logs and listening to readers to see how it changes usage, positively or negatively," Roberts wrote. "No decision yet... The experiment continues." So far, several subscribers have announced their decision to drop the site's feeds in protest.
CNET views RSS as a way to make new connections with users, which has kept adding advertising on feeds off Roberts' to-do list. They're open to the idea, although Roberts thinks feeds already are "advertising." "To make money," says Roberts, "you need to build an audience -- the same as other media. The actual advertising model is still forming, but as the audience grows, the incentive to find the right solution grows."
During a follow-up phone interview, Roberts said he tracks CNET's standing at sites that rank feeds and is thrilled when he sees results such as coming in second on Yahoo's list. "Is that changing our businesses? Not at this point."
But it is changing news consumption. "My thought is this just accelerates the explosion of content in its atomic units," he said. "And I'm not referring to Atom. This was already happening. You're facing presentation of your content in dozens, hundreds of contexts you have no control over. RSS accelerates that."
That brings the conversation back to the issue of control. If the user wants more control and the publisher can provide it while maintaining context and possibly increasing reach, why wait for a perfect solution or some kind of mythical critical mass?
Despite his evangelical status, RSS developer Dave Winer isn't displeased with the pace. "I think the adoption rate is great," Winer says. "I don't want an avalanche. Technologies that go with that way generally don't stick that well."
But Winer cheers every time a big name hits the list, making Time.com's mid-May entry a milestone of sorts for him. "For one thing, it was brain-dead simple. They're a huge organization so they must have talked this thing to death. They deliberate, they meet, they second-guess -- I wasn't privy to any of that. All I know is out the other end came the right result. Absolutely perfect."
Mark Coatney, deputy editor of Time.com, said RSS had been under discussion for a while but didn't start coming together until a month before the launch. Within an hour and without any publicity, the page listing the nine RSS feeds had 160 page views.
"We had that debate -- this is going to be for the five percent of people who do this. But it's going to grow," says Coatney. "Only 10 percent of Web users read blogs but it's an influential group. We wanted to be in on the ground floor. It was really a matter of allocating the resources to do it."
A week later, at a very rough count the site was getting 500-1,000 RSS feed users a day. Coatney says he's happy with that, especially given that Time.com has yet to announce the launch or promote the feeds beyond the front-page link. He's also enjoying positive reviews; one blogger complimented the site for explaining RSS in non-geek language.
By comparison, at AJC.com, Editorial Director Hyde Post says he's pushing RSS as a way of gaining "that persistent desktop presence." But, he adds, "if we're going to do it, then we have to have our own reader or our own branded reader." That could make the user experience easier to manage; it also could give the publisher a higher profile with the user.
Belo's Small worries that RSS continues a trend of dissipating the publisher's control over the way news is shared: "The major search engines, with their news portals, already shift some of that responsibility onto themselves, whether we like it or not. If the news industry chooses not to fight against that shift, we will have to adapt to a world where we put up news accounts but allow the rest of the Net to determine how ours play against all the others."
He admits "quite a few" think that's already the case, but he thinks "it's still important to preserve and enhance the consumer value proposition of editorial judgment, as in, "we believe this story is more important and relevant to you than that story."
True. That's why it's up to the news industry to play an active role in reshaping the way people get their news -- and to admit how significant it is that this debate is over how people get news, not how to get them to care about it.
Editor's note: OJR intends to include an RSS feed as part of an overhaul of our production software and a light redesign. The change hopefully will take place this summer. In the meantime, we've been scraped.