(This is all my personal opinion.)
To be completely honest, I was a little worried about Saul Hansell, a journalist for the New York Times, sitting in on some of our confidential quality meetings at Google. Even though everything was off-the-record, you can’t help but be slightly nervous talking about evaluation methodologies and confidential projects with a reporter in the room. You can read the article now, and in my opinion it does a good job of describing search quality at Google.
I think it was worth the risk of letting a journalist attend our quality meetings. To see why, I’ll highlight five things from the article that you might not have known:
Google continues to have a strong focus on search
All the time I hear things like “If Google doesn’t pay attention to search…” or “If Google loses its focus on search.” That’s not likely to happen, but let me explain why people might worry that Google will lose our focus on search.
- Something like Street View is splashy, cool, and easy to understand, so launches like that tend to get more coverage. It’s much easier for someone to write about a new product or feature than about how Google has improved its semantic understanding of the web, or when we get better at scoring documents. I love Street view, Google Gears, and mobile Calendar, by the way. I’m just using them as examples because they’re easy to understand and recent.
- We don’t always talk a ton about core search quality. Part of the reason is that some reporters are less interested in changes that can’t even be seen (”Google’s search just got a little better in Thai. You can’t see it, but it did!”). Sometimes core search is hard to get other people excited about — kinda like it’s hard to make a picture of someone working on a computer exciting. And sometimes as a business you don’t want to give hints to competitors about how you do things. I’ve got a funny story about “url.host” that I’ll tell someday. Maybe someone will ask me about it in the Q&A tomorrow at the conference.
What happens when you put these two trends together? People see media coverage on neat/wild/fun things that Google does, and they don’t read many stories about core search quality. From those two facts, they extrapolate to what seems like a reasonable conclusion: Google is focusing less on search. But that’s just not true. Hundreds of engineers pay attention to our search quality in ways big and small. Google is practically designed from the ground up so that we can’t lose that search focus. It’s natural to combine these two trends and come to the wrong conclusion. By giving a glimpse at what our search quality engineers do on a daily basis, this article dispels that misconception.
Google makes lots of improvements that most people never notice.
Some people think that Google changes a few things every few months. At least in search quality, it’s more like a few things every week. From the article: “the search-quality team makes about a half-dozen major and minor changes a week to the vast nest of mathematical formulas that power the search engine.” I don’t think we’ve discussed our pace of search quality changes before.
Getting search right is really hard
The article quotes John Battelle:
“People still think that Google is the gold standard of search,†Mr. Battelle says. “Their secret sauce is how these guys are doing it all in aggregate. There are 1,000 little tunings they do.â€
In my experience that’s correct. Running a search engine at Google-scale means that you have to get lots of big things and hundreds of little things right. Missing even a few of those things will annoy users (sometimes subconsciously) and they won’t use your search engine as much. I would never claim that we get all of those hundreds of things right ourselves, but we try to. I read a quote from someone from a different search engine last year. They essentially said that “there was no more secret sauce left” in search. After reading that claim, I walked around happy for days. 
Google has some good internal tools
This article was the first time that I know of that we’ve mentioned our internal debug tools. When you get hundreds of millions of queries a day, it’s inevitable that some queries won’t return the ideal set of results. At Google, we love hearing about those queries because we can dissect them and plan how to improve our algorithms.
There are a lot of people “behind the curtain” at Google that improve search
I think it’s important to get more Googlers out into the spotlight. Sometimes search engine optimizers attribute (say) some crawl change to me when the most I might have done was relay a problem report to the experts in the crawl/index team, who then do the real work of deciding how to tackle an issue and implementing that idea.
So I’m glad that the article sheds light on some new people in search. The article discusses Amit Singhal, who is a wonderful guy and a strong influence at Google. The newspaper article also includes a picture of Jianfei Zhu. Jianfei is a colleague that works with me and others on Chinese webspam; Jianfei also spoke at SES China recently and has done interviews about SEO and Chinese search.
Most importantly, the article mentions that there are hundreds of engineers that pay attention to search and quality at Google. These are phenomenal people who work on everything from international issues to evaluating our quality to crawling/indexing to personalization to fixing bugs to new quality initiatives. (Not to mention all the other people who make a difference at Google in hundreds of ways outside of search.) I know that Saul Hansell talked to several other engineers when he visited Google, so over time I believe we’ll get even more Googlers out into the spotlight.
So, five things you might not have known about Google’s search until you read this article:
- Just because Google doesn’t always talk about search and journalists don’t always write about core search doesn’t mean stuff isn’t happening. Google devotes a ton of effort to improving our search in many different ways.
- Google makes a go/no-go decision on several different quality changes each week.
- If you want to build search loyalty, you have to get a lot of different things right.
- Google has many ways to prioritize feedback and tools to look at how to improve search.
- I’m glad we’re shedding light on some additional people at Google. Many people work behind the scenes to improve the user experience at Google, and we should look to highlight even more of those people.
