Dear Zagat A hearty thanks for your 30 years of service. Now go away.

Page 2 of 2

Lecuyer credits the kitchen for Makoto’s long ride atop the charts. The chefs select only the freshest seasonal ingredients, from fish to vegetables, no matter what the price, she says. The cooks also prepare the omakase plates with a nod to authenticity. “We serve the food we want to eat,” Lecuyer says, “the way we want to eat it.…Our customers get it and understand it. That’s why we’ve been highly listed.”

Now, if there’s one question you don’t want to ask the owner or manager of an elegant Japanese restaurant like Makoto, it’s this: Do you beg or cheat for Zagat votes?

“We don’t do that,” Lecuyer says, noting that servers aren’t allowed to talk to customers. “We don’t know who is doing the survey and is not doing the survey. We don’t know.”

Such ignorance may be a blessing to Makoto, but it’s a bane for diners who want to know how legitimate Zagat’s ratings are. Team Zagat in New York won’t answer any questions about its survey methodology, and Tim and Nina Zagat turned down my interview request. It’s more of the same stonewalling that has worked for 30 years for the Zagats and their guides, which now cover subjects ranging from dining to golf in more than 100 countries. The company’s attitude forces you to make a snap decision: You either trust it or you lump it.

A lot of people are opting for the latter. According to a New York Post article from earlier this month, not only are Zagat sales “down dramatically,” but the company moved too slowly online, “allowing Yelp and others to dominate the market.” Zagat, the paper wrote, laid off about 16 people in May. The founders themselves seemed to see the writing on the wall a year earlier: The Zagats tried to sell their company for a reported $200 million last year but couldn’t find any buyers in that price range and pulled Zagat off the auction block.

advertisement

It’s easy to see why people choose Yelp, Chowhound, Urban Spoon, and OpenTable over Zagat, both the guide and the online site. The obsolete-before-you-buy-it Zagat book is $14.95 a shot, and Zagat.com charges nearly $25 a year to access its complete site, including those ratings that can be seriously outdated. Plus, the mere act of creating an account with Zagat.com requires that you provide the kind of personal information—mailing address, telephone, birth year—that other sites have decided to forgo.

By contrast, Yelp, Chowhound, Urban Spoon, and other sites are free, and most already have established communities where members interact with each other on particular topics and restaurants. Even better for diners looking for recommendations on Yelp or OpenTable, they can see exactly how many people have commented on a particular restaurant—and how those reviews have been averaged into an overall rating. Even the minimal transparency on these sites makes Zagat seem like Stalinist Russia.

“Zagat is not a primary source [for information] anymore,” says Dean Gold, the chef and owner of Dino in Cleveland Park, which scores a decent 21 rating for food from Zagat. Adds Gold, a college-trained statistician: “Zagat, of all the major sources, probably has the lowest levels of reliability” because of its self-selected survey base, which provides little to no information on the people who actually cast ballots.

At this point, OpenTable may be the most reliable of the sites that aggregate restaurant ratings. Site administrators send review surveys only to those diners who have honored their OpenTable reservation, and the diners have approximately 30 days to fill out the forms. This process guarantees two things, says Ann Shepherd, vice president of marketing for OpenTable: 1) that every review is actually based on a meal eaten at the restaurant; 2) that the meal was eaten recently, while the memory of it is still fresh.

Those are two promises that you will never hear from Zagat, a guide that looks destined to follow so many other print publications into oblivion.

Zagat’s 2010 Makoto blurb shows why the guides are “refrigerator-magnet poetry,” “dubious,” good only for a “lonely traveler…searching for a good place to sup before masturbating himself to sleep.”

Grade Creepiness: As first pointed out in SmartMoney magazine in 2007, Zagat food grades have spiked dramatically over the years for restaurants in the New York guide. Same goes for the D.C. area: In the 1992 guide, only 13 restaurants earned grades of 25 points or higher (out of a possible 30). In 2010, more than 60 restaurants topped the 25-point mark. Even more startling, 72 percent of D.C. restaurants with actual ratings in the latest guide earned a grade of 20 or higher, which means that nearly three-quarters of our eateries are “very good” or better. No one sucks here anymore.

The Method, Man: Readers have no idea how many votes are required before the results are considered statistically relevant to merit a Zagat grade (Zagat’s Barbalato told me that Makoto edged Inn at Little Washington, 28.9024 to 28.8495, but would not say how many votes were cast). What’s more, they have no idea if the grade represents the votes of the restaurateur’s spouse and 100 close friends or a statistically sound sample of D.C. area diners. All they know is this: The voters are self-selected, which is a pool almost guaranteed to skew results. Readers don’t even know if voters actually ate the restaurants in question. OpenTable, by contrast, posts reviews only from diners who have honored their online reservation.

Cut-and-Paste Prose: Zagat editors take a ransom-note approach to writing descriptions of the restaurants in their guides. Compare that to consumer-oriented sites, such as Yelp or DonRockwell, where amateur critics can relate their entire dining experiences without fear that an editor will place a dis about “uncomfortable seating” right next to some yahoo’s Pollyanna piffle about a “wonderful experience.” Zagat is refrigerator-magnet poetry at a time when people want the Library of Congress at their fingertips.

You’ve Been Duped: For those who purchased the 2010 edition of the Washington D.C./Baltimore Zagat guide, the Makoto entry might seem familiar. For good reason. It’s the exact same entry as last year’s. The exact same awkwardly phrased copy. The exact same dubious grades. And yet you’d be hard-pressed to learn from the introduction that the 2010 guide is merely an update. Here’s the truth: The even-year Zagat guides are based on the surveys from the previous year. Do you know how much a restaurant can change in two years, let alone two months? Would you trust a review from a year ago on Yelp?

Solitary Confinement: Zagat bases your estimated check on a single dinner and drink, plus tip, which is rather symbolic. Social network sites want to create a community around a common interest in food, which explains not only Myspace Local’s recent move into online restaurant reviews but also local restaurateurs’ embrace of Facebook and its ability to connect supporters. Zagat, by contrast, still conjures up images of the lonely traveler, that burgundy guide tucked into his back pocket, searching for a good place to sup before masturbating himself to sleep.

Caught in a Binder: Like newspapers and magazines, Zagat is dependent on print, where presumably the company still earns most of its revenues, despite the fact that its most timely and user-friendly information (menus, maps, and recent reviews) is found online. Zagat withholds survey ratings from its free Web pages in hopes that you’ll plunk down $24.95 a year to find out what the voting public, in whatever numbers, thought of places like Makoto over a year ago. It’s a hopeless online business plan, but it’s probably a better deal than the $14.95 you pay for the actual paperback guide.

Our Readers Say

Nice piece. Agree about Zagat's time being passed, but have to question your upholding Yelp as the future of dining reviews. Have a look at the top rated DC restaurants on Yelp. No 1. is that hot new place all the foodies and critics are buzzing about: Obelisk (all the way down at #24 on Washingtonian's 2009 list), and at No 2 is, you guessed it, good ol' Makoto. As The Minutemen once titled a song, "The Roar of the Masses Could be Farts".
Did anyone else smirk after reading this article in the printed paper, only to find yet another rehashed list of stale restaurant reviews on the following pages? Perhaps City Paper should take its own advice and can this practice, or limit printing reviews to those created in the past 60 days.
Foodie Frank,

I think that's a fair point, but I would say this in defense of those reprinted blurb reviews: They are months old at the moldiest, since a number were drawn from the 2009 Dining Guide, which was completed in June. Also remember this: We eat our own meals and pay for them ourselves. Zagat constantly gets free data supplied by people whom you don't know if they pay for their meals -- or even eat them.
"Zagat constantly gets free data supplied by people whom you don't know if they pay for their meals -- or even eat them." That's making an assumption about Zagat's sample pool. And who says on-line sources aren't vulnerable to "ballot-stuffing"?
Zagat's reviewers are self-selected? Well, isn't signing on to a website to submit a review also self-selection? Self-selection could be weeded out by doing a random exit-polling of customers coming out of the restaurant - but how likely is that.
First of all great article.

We are now coming to a time where the uneducated masses from Yelp are actually influencing food consumers for localities. This is a step backwards, because many yelp types rate average restaurants as a "5". Having no basis for discrimination, and unsophisticated palates trained by McDonalds, they overrate mediocrity, and reward slavish old themes.

We need guides more democratic than Zagat, but having at least written by those having a modicum of culinary knowledge, unlike Yelp.
Please, let Zagat stay. I get visitors from all over the world and without Zagat, I'd be so stressed out trying to make restaurant recommendations to them. So far, none of my visitors have complained about Zagat.

Thank you, Zagat, for keeping me sane!
Does anyone else find it ironic and borderline asinine that an author from a bankrupt local weekly is trying to cheerlead the end of a print publication? The old City Paper, you know before they went BANKRUPT, had much better writing, more writers, deeper coverage, cut fewer corners, etc... but unless you're on a vendetta who has the time to do the deep dive and show the comparisons of WCP's downfall. By even going after Zagat (really? what is it 1995?) in a day when Yelp and Citysearch are the big players and with juicy stories involving payola for advertising restaurants, current ballot-stuffing, strong arming tactics, etc going on with those sites it shows that WCP is a real print dinosaur. This story would have been great 10 years ago.
This article brings back so many memories. Shortly after my husband and I moved into our co-op I was returning from lunch with my friend Slim and right in front of our building was a small person handing out booklets. My doorman Benny was trying to shoo him away but Slim put a halt to that and urged me to accept a copy. Apparently one of her children had been given one the week before. "Oh take one" Slim demanded. "It's very funny. I think the Harvard Lampoon is behind it." Reluctantly I took one-it turned out to be a Zagat Guide (or, as they would have it, "Guide") "Read what they say about where we just had lunch" Slim told me, turning to the page where our restaurant was listed. I first noticed the hieroglyphics next to the name (Slim and I both assumed Masonic origins) and then I read that our restaurant "serves" "good" "food". I thught that very insulting, since the restaurant was indeed very good, at least it was then, and those irony quotes (like "luxury" Disney cruises) put me off. But as I kept reading I saw there "was" "a" "kind" "of" "poetry", albeit crude, behind the efforts of "Tim" and "Nina" "Zagat". (Slim, to her dying day,swore that "Nina Zagat" was a pseudonym of Thomas Pynchon's).

Thank you again for bringing back that lovely afternoon with Slim and the "Zagat" "Guide",

Valda
Zagat's reviews are way more useful than Citypaper's. Actually, Zagat, the Post, Washingtonian, and even AOL Cityguide all have more useful restaurant reviews than Citypaper.

Zagat isn't some wizard of oz mystery...it's a survey. I've participated...I've been quoted in print, and I've even won a random $100 gift certificate for participating. All you need to do is provide ratings from 0-3 for three categories, do this for three restaurants, and you get a free copy of their print guide for that city. What's so fucking erudite about that?

Seriously...Zagat has some uber-foodsnobs, and some others who wouldn't know a steak from a Big Mac. But for the most part it's a very accurate, focused, and comprehensive survey with lots of great restaurant info.

If you want to take them down, Citypaper, maybe have your editors and readers focus on nightlife or something...but simply bitching and whining for Zagat to go away doesn't make your paper any better.
Zagat also promotes illegal votes by offering a free guide for anyone who submits at least 1 vote for that area. In essence someone can submit fraudulent votes and still receive a free guide when it publishes. Also, the editor will remove the restaurants info if they had a bad experience or alter the votes if it is their favorite in the area. The websitte is impossible to cancel once you pay for the monthly service apparently this is intentional to maintain a high subscriber and voting pool which Zagat sells the listing to other companies with your eating habits and personla info.
Why isn't there a link to the SmartMoney story? It reveals in depth, the methodology and problems with the Zagat system. It also reveals that this article hewed very closely in methodology to a story written two years ago. Here's the link: http://www.smartmoney.com/spending/rip-offs/zagat-math-20819/
By the way, to get an average of 28.8495 by averaging integer votes, you need at least 93 votes (if 28.8495 is a correctly-rounded version of the true average) or at least 113 votes (if 28.8495 is a truncated version of the true average).

To get 28.9024, you need at least 41 votes.

Leave a Comment

Note: HTML tags are not allowed in comments.
Comments Shown. Turn Comments Off.
...