By John Dodge
Posting in Technology
In two concessions, Google has made it easier for newspaper publishers to better control their own content. Is this too little too late?
I have mixed feelings about Google's two latest concessions to newspapers that want to wall off paid or subscription content or restrict what content Google News crawls. What do you think?
On one hand, this old print hand has adapted well to the online world and spent the last five years figuring out how Google can best find my content. I'm used to living in a world of tallied page views and my compensation at various stages has been tied to how many of those I generate.
Without Google, I'm toast on that score as readers who find my work have mostly looked for it on Google. Where else do people go searching for stories? Remember the archaic notion of factory piece work? That's where factory workers were paid on how many pieces or products they produced as opposed to punching a clock. Being paid on page views is the blogging equivalent to factory piecework. Under such a system, one has to be creative and fast lest he or she quickly get weeded out.
On the other hand, Google is second biggest reason media companies are laying off scores of talented journalists and shuttering magazines and newspapers. The first reason is the Internet which by itself had newspapers in a slower decline. Google has put their demise on the fact track. Indeed, a substantial chunk of Google's $20 plus billion used to go into the pockets of traditional media companies for better or worse. New York Times columnist David Carr wrote a terrific column Sunday on the demise of tradiitional media and who's filling the void.
What did Google do exactly? This morning in one blog post, Google described how it has made it easier for publishers to opt out of being found by Google News and listed in its search results. It was something publishers could do all along by filling out an exclusion contact form and submitting it to Google. Now publishers can block out the crawler themselves by adding instructions to a .txt file embedded in something called a Robots Exclusion Protocol or REP for short.
When I first learned about these concessions, where did I go? Google News, of course where I came up with 854 related stories. For research, I use Google News multiple times every day although 20 stories on any single news event usually suffices.
The other concession would limit the number of free stories on subscription or paid sites a user could access through Google. It's called the First Click Free program which has been around for a while. Google in separate posst announced modifications to the program yesterday.
"As most users are generally happy to be able to access just a few pages from these premium content providers, we've decided to allow publishers to limit the number of accesses under the First Click Free policy to five free accesses per user each day," Google wrote in its Webmaster Central Blog yesterday.
The original First Click Free program apparently emerged a couple of years ago and gave users unrestricted access to subscription-based or paid content without subscribing or paying for it. In the case of the Wall Street Journal, that is a problem because it is one of few newspaper sites to charge for premium content.
"If you aren’t familiar [with it], First click free is a way for publishers to share their subscription-only content with Google News readers. All articles that are accessed from Google News are allowed to skip over the subscription page," Google posted in September, 2007. Translation: You can have the WSJ's content for free through Google News while everyone else has to pay for it.
"Now, we've updated the program so that publishers can limit users to no more than five pages per day without registering or subscribing. If you're a Google user, this means that you may start to see a registration page after you've clicked through to more than five articles on the website of a publisher using First Click Free in a day," Google wrote yesterday.
Still doesn't seem like cricket, does it Ruppert?
That's Ruppert as in News Corp. media magnate Ruppert Murdoch who's been threatening to do a deal content-licensing deal with Microsoft and cut Google completely off from News Corp content. WSJ managing editor Robert Thomson has warned if WSJ.com did not charge for content, News Corp. would have to fire 290 reporters (there's plenty of that going around, anyway).
My main paper, The Boston Globe, has been mulling ways to charge for content without losing droves of readers. It may not be possible, but something has to pay for great journalism. There is something unfair about the way Google capitalizes on content created and paid for by others. if I understand this correctly, Google News users can only circumvent the WSJ's subscription wall only five times a day, now.
Indeed, the WSJ has the right to establish the rules of its business. But the choice for publishers is difficult. You can either exclude your content from Google News entirely through the REP so almost no one finds it unless they are on your site. Or you can use First Click Free and have your content indexed so the Google search engine can find it. The exchange is give away a few pages of premium content free for all important content indexing.
All that said on behalf of fairness, I have moved deeply into Google apps, News, dashboard...you name it. Google is an enormously powerful and everyday tool for me. I've moved on, but I like it that newspapers are fighting back.
Follow me on Twitter.
Dec 2, 2009
The real problem is that few if any "news" agencies these days are in the business of reporting news. They have been "corporatized" and now serve only to mold public perception and limit the exposure of the corporate fat assess from legal prosecution. They are as controlled as any state sponsored media could ever hope to be, except for one thing. With google archiving every thing they can't make previous stories disappear down the memory hole as they so choose. Just like when a lier can't keep his lies straight he gets in trouble when they start to contradict each other. At present we can quarry google for the proof and call them on their unethical practices. They want this little thorn in their side gone. They are well aware that the attention span of their viewers and readers is about as long as a 30 second sound bite and it needs to be repeated many times if it is to be remembered. It's no wonder the important stuff is always in the sub text.
You raise a good point and I have edited my post accordingly. I also thank you for reading the post to the end.
John, there is an important point missed in your analysis. The "First Click Free" program was only made available it the publishers wanted it to be. Your statement below is simply wrong. "That Google ever got around the WSJ subscription or payment page is akin to what Naptser did with music." If the WSJ, or any other publisher did not want to utilize "First Click Free", their content would not be available through the program. Google did not circumvent the pay wall. However, if there is a pay wall to the content, Google can not index it and make it available via search. This means lost traffic. So, there was the option for the First Click Free program. Using this program, you can have your content indexed by Google and available in search. Users that found the article on Google could view it unregistered, but to go to other articles they would need to register. This is the program that has been updated to allow only 5 clicks per user today. This way, a user can't just go back to Google and search for the next article to continue reading on your site (at least not for long). The other option is to simply use robots.txt to prevent content from appearing in Google at all. Any web publisher can do this as well, which does not even require a paid registration. If you don't want to be in Google results, you don't have to. Google could have made another option, which is to include your paid content in the search results but still allow the site to require a paid registration before any access. This might be a preferred option for many publishers. However, for a Google user this seems like a bad experience. The first link you click doesn't take you to the article you wanted to read, but to a request for money. Google thinks that is a bad user experience, so it does not offer that option. The publishers are free to choose to either leverage the First Click Free program or not based on that or other factors. It is a decision about how they want to manage their business, just as the choice to offer First Click Free was a choice for Google.