Yelp Official Blog

Case Study: An Upset Business Owner

As Lawrence Coburn (of competitive review site RateItAll.com) points out today on Huffington Post owners can really struggle with negative feedback. His post does a good job of shining a light on the recent controversy, as well as the landscape for reviews sites in general.

It's very hard to disassociate yourself from negative reviews about your business and it's easy to take it personally. Below is a case study in just that...

Here we have an advertiser that has a 4 star average rating and 375 reviews on their page, yet suggests we remove positive reviews "replac[ing] them with poor quality reviews." To provide readers some perspective this business currently has: 148 five star reviews, 138 four star reviews and just 9 one star reviews.

The owner also complains of a pushy sales person, but actually works with an account manager whose job is to make clients happy, help them use our business owners site, and demonstrate enough value to their clients to encourage them to stay in our ad program.

The note:

On 01-25-09, I requested that my monthly advertising be terminated by Yelp. There were two reasons for my requested termination. First and foremost, my sales rep Mandy was pushy, aggressive and insensitive to my needs. She regularly talked over me and treated me more like a used car salesperson would. Second, I could not validate, nor verify the benefits of my $300.00 monthly advertising budget with you.
    Since my termination request I have documented copies of my daily yelp reviews. Apparently, you have selectively chosen to pull the 5 star reviews from my site and replace them with poor quality reviews. I must tell you that I do not view this "strong arm" tactic favorably. Although it is well with in your rights to monitor reviews, your recent actions come across more like "payback" for terminating my advertising with you. If I ever were to reconsider advertising with Yelp in the future this would not enhance my feelings to do so.
    This morning I met with my some of my fellow Silicon Valley Restaurateurs and this topic came up for discussion. Apparently, I am not the only person who has experienced your company's wrath.

Sincerely,
YYYY (redacted)

C.c. Gavin Newsome
    City of Saratoga
    Saratoga Chamber of Commerce
    San Jose Mercury News
    Metro News
    Wave Magazine

Our Reply:

Hi YYYY (redacted)

My name is Jed and I run the sales department over here at Yelp. Your recent email to “Ideas to Improve Yelp” was brought to my attention and I wanted to respond personally.

First of all, thanks for your previous business and candid feedback here.

I will personally look into the treatment that you received in terms of talking over you and behaving like a “used car salesperson.” You referred to her as Mandy and I believe your Account Manger was YYYY(redacted). She is not in fact in sales but in Account Management and her job is to help you get the most of your account on Yelp. Either way, it is not acceptable that you felt that way and I will make sure that we address the issue.

In terms of validating your $300 per month – I am sorry that you did not see value in the advertising program. You clearly have a great business as evidenced by your 4 star rating and approximately 375 reviews, and I am sure you will do well on Yelp with or without the additional exposure of our ads. For what it is worth, the ads that you bought clearly did work in increasing the number of people viewing your page on Yelp (from 780 per month before you started to 2096 per month in your last month in the program). Ultimately your opinion is all that matters however.

Now let me address the last paragraph in which you accuse Yelp of manipulating reviews as a “strong-arm” tactic and as “payback” for terminating your advertising package.

We categorically would never add or remove reviews unnaturally as a result of an advertising relationship with Yelp. I cannot be strong enough in saying that. Reviews are treated the same for all businesses whether they are an advertiser or not (you have the ability to designate – clearly marked – your favorite review if you are in the advertising program)

I have copied a section from our corporate blog  - link as well – that is called “9 Myths About Yelp.” In it you will find a very clear explanation about why your review number may have fluctuated as well as clarifying some other realities of Yelp.

I would be happy to have a conversation to address your concerns and would be available to speak to the group of restaurant owners that you reference. We pride ourselves on connecting people with great local businesses and are mostly doing this for free when most other directories are pay to play. Communication and education are often the key in clarifying these situations.

Here is the link that I referenced before:

http://officialblog.yelp.com/2009/02/9-myths-about-yelp.html

I have also copied below.

Again – thanks for the feedback XXXX(redacted) and feel free to reach out if you would like any further clarification.

Sincerely,
Jed

Interview with a Business Owner and Yelp Advertiser...

Joe from Keetsa took the time to sit down with us and share his candid views on Yelp. I hope you find the following clips interesting. I plan on making this a regular feature of our Official Yelp Blog.

9 Myths about Yelp

Here at Yelp we're trying to make the world a better place by helping people to connect with great local businesses.  We've grown very quickly in a few years, and our site is different from what most people were previously used to with yellow pages and other directories.  As a result, people sometimes misunderstand how we do business, have questions about how the site works, or just plain don't like the idea of consumer reviews.  Here are some of the Yelp Myths that we've heard most often:

Myth # 1: To rank highly in Yelp search results, businesses have to buy advertising

Reality: Yelp search result pages often feature one orange-background, clearly-labeled Sponsored Result at the top of the page.  We got the idea for this design from the major search engines. Look at the ad designs compared here and, if you find yourself even a little confused about the Yelp ad, please send over any suggestions about how to make it even more obvious.
Yelp's numbered search results (all of the other listings in white background) have absolutely nothing to do with who is paying us.  For example, check out this search result for the very popular "Restaurant, San Francisco" query.  As of February '09, only 1 of these top 10 resulting restaurants (Masa, at #9) is a Yelp advertiser.

Myth #2: Advertisers get to manipulate their reviews (e.g., remove negative reviews, add positive reviews, re-order their reviews, etc.)

Reality: Advertisers on Yelp pay for... well, ads.  These orange-background, clearly-labeled Sponsored Results appear in various places on the site including in a single spot at the top of search (see Myth #1).  Along with their ads, advertisers also get to add a photo slide show to their business page, and they get to promote one favorite review at the top of their business page under the header "One of Advertiser's Favorite Reviews."  Here's an example of what this looks like. With the exception of this one "Favorite Review," consumer reviews for advertisers are handled exactly the same as consumer reviews for all other non-paying businesses.  So no, advertisers never get special powers to remove a negative review, add a positive one or move reviews around on their page.

Myth #3: Yelp salespeople manipulate reviews for prospective advertisers (for example, offers to remove a negative review if a new client signs up, or a threat to remove positive reviews if the business owner does not choose to advertise with Yelp)

Reality: We have every reason to trust the smart, hard-working and ethical salespeople who work at Yelp.  Beyond this, to avoid even an appearance of impropriety, we've taken several steps to ensure no member of our team is tempted to game the system.  Specifically:
1. Yelp salespeople do not have access to the system that deletes reviews; only a few members of Yelp engineering and user support team have this access, and they literally work on different floors within the office.
2. Every Yelp salesperson signs an agreement that s/he will not write reviews of any business while employed by Yelp.  We trust our teammates in sales to live up to this commitment.  We also have several monitoring systems in place to ensure nobody (accidentally or otherwise) crosses this line.
3. Through our vigilance, we once did find a salesperson who encouraged a friend to write a positive review for a prospective client (that the friend had actually patronized). The salesperson's role at Yelp ended that day.
4. When a new advertiser signs up with Yelp, the relationship is handed off to an Account Manager.  The Account Manager then takes the client through a 30 minute phone training session -- and confirms that reviews have nothing to do with advertising.
5. After the training call, the Account Manager sends a follow up survey that asks each client how much s/he agrees with the following statement: "I understand that Reviews are completely separate from the Yelp Ad Program, and that there is an automated filter that may suppress some of my reviews whether or not I am a client."  Any client who does not click "Completely Agree" in this case gets yet another follow-up call for clarification.

Myth #4: Yelp removes positive reviews from businesses its staff does not like, or from businesses that do not pay for advertising

Reality: A review you may have seen on Yelp previously is no longer there; this happens.  The review in question may have "disappeared" for one of three reasons:
1. The review may have been suppressed by Yelp's automated Review Filter, which is always out there looking for suspicious reviewing activity (like those anonymous rants and raves you see on other sites). 
2. The writer may have removed her own review; she has the right to do that at any time
3. Another user believed the review violated Yelp's Review Guidelines and sent it to our customer service team for review. The customer service team agreed, then manually removed the review.

Both our customer service team and the Review Filter work exactly the same way for advertisers as they do for non-advertisers.

Myth #5: Yelp salespeople harass business owners by calling incessantly, and will never stop

Reality: Our salespeople call local businesses to introduce our targeted advertising programs; we're trying to establish warm new relationships, never to irritate someone. Persistence is part of sales, and if a business manager asks one of our salespeople to "keep in touch" or "call next week," we will!  On the other hand, our salespeople don't like wasting time (yours or theirs), so if you tell your Yelp sales rep to call back in three months, we'll do that.  And if you really want us never to call again, we will add you to our "Do Not Call" list.  If you ever become aware of a Yelp team member violating your specific request along these lines, please let us know.

Myth #6: Reviewers on Yelp are very young, mostly in high school or college

Reality: In January 2009, 94% of Yelp reviewers were over 23 years old. This means about 6% of reviewers fall into that "high school or college age" category.

Myth #7: Yelp reviewers get sued regularly by business owners for writing negative reviews

Reality: Freedom of speech is a well-protected right, whether that speech is online or in any other form. As a result, reviewers are well within their rights to express their opinions (ranging from "I love the ambiance" to "The cashier was rude to me") and relate their true experiences ("I ordered the shrimp scampi” and “The doctor wouldn’t accept my insurance”).  Some of the most successful business owners on Yelp use their free business owner's account to contact positive and negative reviewers alike, even when they feel a review is unfair or incorrect.  Reviewers are usually thrilled to get a well-meaning response.  The handful of business owners who have pursued the “nuclear option” of suing someone over a negative reviewer have met with little success, high legal bills, and a lot more attention focused on the negative review than they originally bargained for.  When that happens, we make it a point to involve ourselves early on to protect the rights of legitimate reviewers.

Myth #8: Yelp was created to allow whining and complaining about businesses

Reality: Just the opposite. Yelp was created to help consumers find great local businesses. And it's working: more than 85% of the 5 million reviews written to date are 3+ stars.  So if anything, that small percentage of negative reviews provide an important consumer service. Just like in the real world, not every local business experience is going to be perfect.

Myth #9: Business owners have no voice on Yelp

Reality: Every local business can setup a FREE Business Owner's Account to publicly post a description of the business, announce special offers, message customers, add photos, track traffic on their Yelp page and more.

East Bay Express Story Starts to Unravel

Inc, NYTimes, SJ Mercury News, Financial Times, SFWeekly, Greg Sterling (local analyst), Huffington Post, Digital Media (CNet), The Social (CNet),  AppScout (PCMag Network) have now followed up on Kathleen Richards inaccurate piece with stories that further confirm what Yelp has been saying. Most notably:

1. While focusing on allegations that we remove reviews for pay (with a ridiculous "Extortion" headline) Kathleen backtracks in the very last paragraph of her five thousand word "expose":

"Owners who were approached by Yelp in recent months said they were told they could choose one positive review that would appear at the top of their page, which would clearly be denoted as a 'sponsored review.'

Plenty of Yelp advertisers still have negative reviews on their pages. 'You pretty much have to fight tooth or nail to get a bad review moved or removed,' said one East Bay restaurant advertiser, who wished to remain anonymous. Peter Snyderman, the owner of Elite Cafe, said his sales rep never mentioned moving negative reviews.”

2. It appears that a key source of confusion is our anti-spam algorithm which makes a small number of reviews come and go from a typical business' page.

We realize we need to do better at communicating the why and how of this counter-intuitive "feature" and we will. The world has seen a similar story before and I think we know how it ends (happily):

An article on Google from 2004:

"Last November, when Google changed the algorithm that governs how sites are ranked, many businesses registered howls of protest as they watched their sales plummet during the Christmas season. Some accused the company of trying to force them to buy advertising instead of relying on a free listing."

And another article from 2003:

"A significant change to Google's ranking algorithm has caused some web sites to lose top positions for some search terms. The outcry from affected site owners has been unprecedented, in my opinion." ...

"Some feel Google has dropped their sites to make them buy ads. In the short term, purchasing ads will be the only way they can be found. For some, it may even be the only long-term solution. In either case, it means more money for Google."

3. We have always had significant protections in place to separate sales from content (the same "potential" conflict exists at every news outlet):

"He said that members of Yelp’s sales team have no power to control the results displayed on a business’s page. Salespeople and engineers are on separate teams, and sit in separate offices. Reviews are displayed according to a proprietary algorithm, similar to the way in which Google results are determined.

Mr Stoppelman also said members of Yelp’s sales team are banned from posting reviews on the site. 'No salespeople can write reviews,' he said. 'That’s been a longstanding policy since Yelp began.' Other Yelp employees can write reviews, and Mr Stoppelman — an active reviewer — has even given one of his advertisers a one-star review."

Conspiracy Theories and Blackbox Algorithms

It's a catch-22. We have a software program that looks for suspicious reviews and takes them off a businesses page and this allows critics of Yelp to suggest that we've tied this system to sales. Is it only Yelp that faces such accusations? Hardly. Any important system with a similar "blackbox" gets the same conspiracy treatment. Yesterday in the New York Times:

"Google uses a proprietary algorithm to assign 'quality scores' to advertisers’ sites, using measures like the apparent usefulness of the sites. Advertisers with low scores have to pay more for their ads, and many advertisers have complained that Google can use the system to manipulate prices."

Sound familiar? I don't believe it. You probably shouldn't either.

A little humor to end the day...

With all the hubbub about the inaccurate East Bay Express story I thought I'd share this nugget from one of our good-humored advertisers. A San Francisco Irish pub manager sent us the following cheeky note after he read the piece, "Trust me, I'd be first in line if you gave out review vetoes."

Think that about sums it up.

Kathleen Richards - East Bay Express

Updates here: East Bay Express Story Unravels

Today the East Bay Express ran a lengthy story that accuses Yelp of manipulating review order for money. As we've said many-a-time we do not do this and you don't have to take my word for it. Let's take a quick look at an advertiser...

As you can see while the first review is positive (and very clearly marked up) the reviews below are neutral and yes even negative. This is quite normal and if you look around the site and click on some ads you'll find plenty of advertisers with reviews that look like... well, everybody else.

We showed this evidence to Kathleen, but it didn't find prominent placement in her story. Beyond this obvious point I have three additional issues with the story I'd like to highlight (thanks for bearing with me!).

1. Heavy reliance on anonymous sources.

Kathleen relied on five anonymous sources and only three non-anonymous sources. Use of anonymous sources is fraught with hazards and is strongly discouraged by most editors, as explained by the American Journalism Review, "Editors around the country...agoniz[ed] over the use of anonymous sources, fearing they were relying on them too heavily, damaging the press' credibility in the process."

Anonymous Sources:
John, an East Bay restaurateur "scam"
Mark, a former yelp "contractor" and business owner "no interest in curbing illegitimate reviews."
Joe, a business owner "sounds like the Mafia"
Mary, a photographer "I'm a little curious why my reviews are disappearing."
Ellen, an Oakland business owner "The prices were cost-prohibitive"

Named Sources:
Robert Gaustad, Bobby G's
Greg Quinn, Anabelle's Bar
Mary Seaton, Sofa Outlet

2. Reliance on at least one source with serious credibility issues.

Sadly at least one of the named sources is affiliated with a business that has spent a considerable amount of time and energy trying to mislead customers like you with fake reviews. Below is a sample of some of the many reviews we've removed from Sofa Outlet's page:

sofaoutlet2@xxxxxxx.com - rating 5/5 stars - I went into the store not knowing what I was looking for. The sales team really helped me to narrow down what I was looking for and at such a good price. I am now relaxing on my new comfortable leather sofa watching the NBA finals! The delivery staff was great as well! I just went in to get a big leather ottoman and had a superbowl party in which everyone loved the furniture.

sofaoutlet2@yyyyyyy.com - rating 5/5 stars - Great place to shop! Lots of choices and lots of options! I am so happy with all of items that I have purchased from this little cute store hidden in San Mateo. My only complaint is that the sofa came too fast and I was not able to donate my old one in time.

3. The accusatory thrust of article is essentially overturned at the very end.

In her words...

Interviews with more than a dozen local business owners suggest that Yelp sales reps may be wording their sales pitches more carefully these days. Owners who were approached by Yelp in recent months said they were told they could choose one positive review that would appear at the top of their page, which would clearly be denoted as a "sponsored review."

And plenty of Yelp advertisers still have negative reviews on their pages. "You pretty much have to fight tooth or nail to get a bad review moved or removed," said one East Bay restaurant advertiser, who wished to remain anonymous. Peter Snyderman, the owner of Elite Cafe, said his sales rep never mentioned moving negative reviews.

Yelp Love in Apple Stores

Signage in the 5th Avenue Apple Store (NYC)

Photo from the Apple Store in Palo Alto, CA

David Lazarus - LATimes

A recent opinion piece in the LA Times asks the legitimate question does Yelp provide enough disclosure when showing advertisements? Rather than biasing you with my own opinion, why don't you take a quick look and see for yourself. Below are screen shots from websites across the industry all performing the same search. You can click on any of the screen shots to see the page up close.

Here is a Google search for "lawyers". They have a tan background and the word "sponsored link" on the right. You can't click on the disclosure for more info, it's just text in light grey.

Below you'll see a YellowPages.com (AT&T) search for "lawyer". I wasn't able to find any disclosures and it looks like all the top results are advertisers.

  

Only one more, bear with me! An AOL search for "lawyer"... they have a very light blue background color behind their ads and they provide a "Sponsored Links" disclosure on the upper right. The disclosure links to an FAQ page.

 

And the big finale... a Yelp search for "lawyers". The single advertisement on top is colored with a strong tan background (similar to Google, but actually darker). We also have a "Sponsored Result" disclosure and if you put your mouse over it you'll get a helpful pop-up that explains this is a paid placement.

 

So what do you think? Let us know.

Overheard on MUNI in San Francisco

One of our engineers today overheard the following on his commute (said by one business owner to another):

"Yelp keeps you on your game"

Any customer that walks into a local business may be a Yelper, as a result owners now have an even stronger incentive to perform their absolute best for every single customer. When this happens owners reap the benefits of the positive word-of-mouth that is then reflected on Yelp and the marginal customer gets a better experience. I call that win-win!