“The experts make it impossible for an enemy to know where to prepare. They release the attack like a lightning bolt from above the nine-layered heavens.” – Sun Tzu, “The Art of War”
Someone inside the Googleplex has been reading the great Chinese strategist and chose to lob a hefty bolt from the third or fourth layer of Mountain View CA into a panel discussion on click fraud and click auditing at this week’s Search Engine Strategies meeting in San Jose.
The Tuesday meeting seemed slated to be fairly cordial, unlike the war of words that erupted at the February get-together in New York between reps from Google and Yahoo! and notables from the search marketing firms. Six months ago, those two groups traded verbal punches over whether the litigation pursued by search marketers against the engines had been helpful to the ad community in the long run or simply blocked the search for a workable fraud solution while the cases dragged through the courts.
But things are different now. With a Google settlement approved in the Lane’s Gifts click fraud class action suit and preliminary approval of a Yahoo! settlement in another click-fraud class action in California, and with last week’s announcement that the big search engines would join a working group assembled by the Interactive Advertising Bureau to define click fraud and set guidelines for combating it, the timing seemed right for a cordial discussion of where the search industry should turn next in its efforts to reduce bogus ad clicks.
That rapprochement lasted for about 20 minutes, until Shuman Ghosemajumder, Google’s business product manager for trust and safety, announced that his company had just completed a 17-page study detailing what he termed flaws in the way some third-party click auditing firms compile data in assembling click-fraud refund requests for their marketer customers.
Links to the report went up on Google’s “Inside AdWords” blog at 9:09 a.m. yesterday (presumably Pacific Daylight time), nine minutes after the start of the “Auditing Paid Listings and Click Fraud Issues” panel. And Google had apparently not shared the findings of the report, entitled “How Fictitious Clicks occur in Third-Party Click Fraud Audit Reports”, with any of the other panelists—three of whom represented SEM firms that do click-fraud auditing. One of those firms was specifically cited in the report for what Google alleged were flawed click-counting methods.
So much for cordiality.
Ghosemajumder said the report, authored by the company’s click quality, outlined “some pervasive and reproducible problems in the methodology used by third-party click fraud auditing firms which have consistently inflated their estimates of how many clicks are actually being marked as fraudulent.”
According to the report, those flawed methods include counting visits to a Web page rather than counting the actual clicks on an ad on that page. As a result, the Google report says, these consultants often wind up counting page reloads, back-button hits and subsequent visits to an advertiser’s site as multiple clicks on that marketer’s Google ad. “A single legitimate click has, through the use of this methodology, resulted in the detection of five fraudulent clicks,” Ghosemajumder said.
The report also points to some third-party click auditors’ reliance on software cookies to track user activity across the network of sites run by their customers. According to Google, this can sometimes combine ad clicks across different ad networks, so that clicks on a Google ad get counted against Yahoo! and vice versa.
Pressed by moderator Jeff Rohr of search marketing firm Optiem LLC to say whether Google thought these errors were accidental or intentional, Ghosemajumder refused to characterize auditors’ intentions. “We’re not making any judgment on what the internal dynamics have been,” he said. But he added that mixing clicks across ad networks have in some cases resulted in fraud claims by advertisers of more than 100%.
“If you looked at this information, it would be strikingly obvious that there was something very, very flawed in the methodology being used to collect these numbers,” he said. “And we think the same. We don’t know why these basic sanity checks have not occurred on the part of the third-party audit firms. We don’t know why these numbers are inflated to such an extent that even the most basic check against an advertiser’s own data would verify that more clicks are being counted by these firms than even occurred for that advertiser.”
The last seven pages of the Google report featuring detailed analysis of the flaws Google has identified in the click-fraud reports sent to them by three click-audit firms: ClickFacts, Click Forensics and AdWatcher. Google said that all the reports cited overstate the fraud problem for their clients by including these fictitious clicks.
Tom Cuthbert, president and CEO of Click Forensics, was part of the panel discussion yesterday. He declined to discuss the Google findings before reading the report, but said it was “pretty cool” that the document included an examination of some click-fraud reports produced by his company and would read the report.
But Lori Weiman, director of KeywordMax, a search marketing firm that provides click-audit services for its clients, voiced the group’s irritation at being ambushed by the Google report. “It would have been really nice if you’d shared that with us in advance of this event,” she told Ghosemajumder. “We could have been more thoughtful in our response.”
Ghosemajumder replied that Google had shared findings of this type with advertisers and auditors in the past and been told that third-party auditors’ reporting methods were simply different. “We want to be able to call attention to this so that advertisers can take action to protect themselves as well,” he said. He said the Google report identified steps auditors could take to eliminate overcounting of clicks from page reloads and added that he was “surprised” these measures hadn’t already been taken.
Jessie Stricchiola, founder of click-audit firm Alchemist Media and an early advocate for click-fraud reform by the search engines, pointed out that legal action from disgruntled advertisers was the driving force behind the search engines’ recent responsiveness to marketers’ fraud concerns, not any self-induced “profound sense of empathy for the plight of the advertiser.”
“It’s taken a really long time to get a little bit of cooperation [from the search engines], and I want you to remember that,” she told Ghosemajumder. “To some extent, it’s the fox guarding the hen house, and I think that’s what the industry is trying to move away from and become more cooperative.”
For another kind of “sanity check”, Stricchiola pointed to a section of the independent evaluation of Google’s click fraud defenses produced by NYU professor Alexander Tuzhilin and made public a few weeks ago. The report found Google’s anti-fraud measures “reasonable” overall. But Tuzhilin also noted that the company was slow to make some basic corrections in its accounting of clicks—as Stricchiola explained to the SES audience.
The problem was users who double-clicked on paid search ads, an old habit of PC functionality that’s no longer necessary. As Tuzhilin noted and Stricchiola quoted, Google counted each of those double clicks separately and charged advertisers for them until a policy change was instituted in March 2005. Tuzhilin’s report theorized that the need for the change was evident long before then, but that the Google business unit dragged its feet on making the improvement because the financial repercussions of not charging for the second click would have been, as Tuzhilin wrote, “non-trivial”.
Cuthbert pointed out that his firm had mounted a Web microsite to encourage search marketers who suspect fraud to join its Click Fraud Network. The site, named www.ReasonableIsNotEnough.com, is an allusion to the finding of a recent independent report that Google’s click-fraud detection efforts were “reasonable” given the limits of the pay-per-click model.
“Let’s agree that you and I are never going to agree on your methodology versus mine,” Cuthbert told Ghosemajumder. “But we should agree on what we do at the end of the month. You take this data from me; as the contractual final arbiters you guys decide which clicks are valid and which are not; you pass that information back, and let’s move on. We built that system and made it available, and we’re eager to move forward.”
John Slade, senior director of global product management for Yahoo! Search Marketing, said that debating differences of methodology was “comparing apples not to apples but to Buicks”.
“Methodologies vary, because it’s a complicated problem and you’ve got different people trying to do the right thing,” he said. He said he placed his greatest hope in the efforts by the IAB working group to come up with standard definitions of click fraud and guidelines for counting bogus clicks.
Asked by an audience member if click fraud would eventually correct itself through market forces—an opinion recently voiced by Google CEO Eric Schmidt—Weiman said that would require that all search marketers competing on a keyword track their SEM campaigns and then acted on that data, reduced their spending on keywords and engines that seemed to have poor conversion rates.
“We know over a third of all advertisers aren’t doing that tracking,” she said. “And not everybody’s bidding on a keyword for a direct-response sale. Some are bidding for branding, and some just want the traffic.” Self-correcting click fraud is “a lovely idea,” she said, but could only happens if every bidder had the same goals and the same depth of knowledge about their terms’ performance.