PPC Analytics is only wishful thinking

[Dave]

I am in a debate with someone who doesn’t believe search is any more accountable, measurable or effective as a marketing channel than traditional media like TV, radio and magazines. It’s a bit like arguing with someone who still believes the earth is flat.

My only problem is that the arguments made aren’t that bad. I don’t agree with them, but the ideas are deceptively simple to understand. I’m having a hard time coming up with equally simple counterpoints to de-bunk them.

Can you help me out?
Here are the main points.

PPC Analytics data can not be verified on a high volume website:

What if we shut PPC & Search off for a month or two and then turned it back on? The resulting drop & spike in visits and revenue should indicate the “Real” value of the ad campaign but given the large fluctuations in our competitive conditions and seasonality the resulting drop then spike in traffic and revenue is not accurately measurable.

PPC traffic can not be properly tested or accurately measured. The PPC ads are either there or not. You can’t show ads to only half the audience. There is no way to set up a control group and measure the difference in performance.

People will find the website anyway:

We do not rank well organically in the SERPS for generic phrases like “Blue Widgets” so we purchase unnecessary PPC advertisements.

The PPC advertisements are unnecessary because people will find the website anyway. If they don’t find us on that particular query, it’s no big deal because they will find us on the next one or the one after that. Or they will find a Widget review website or a Widget aggregator, and then compare prices on our website against the aggregator. Or they will follow a link or whatever. Either way… This is a large purchase decision that usually does not happen with a single query or visit to a single website. People comparison shop and PPC ads are an unnecessary convenience that does not drive the traffic or the sales.

Pretend ROI:

If the lift can’t be accurately measured or verified, then how can we justify attributing ROI to the expense? ROI from PPC/SEM is supported in most situations by anecdotal evidence only just like traditional media. Analytics data is not an accurate representation of reality and we should not pretend that it is.

[rustybrick]

Honestly, this is a very interesting topic.

You are a big brand like "Sony" for example and your goal is for people to learn about your brand, learn about your products and then buy it direct or from a reseller.

If people want to find the Sony Web site, they will eventually get there (assume if they try 10+ times, they may give up, but most sites are not that hard to find).

Sony doesn't necessarily care about ranking well for "TV" because TV is too generic...

I wonder what other people have to say about this? Any practical experience with it?

[Dave]

Thanks Barry,

Yep… We are a big brand in our market but not BIG BRAND like Sony, Visa or Microsoft, nor do we have a dozen brands or subdivisions under a larger brand like Johnson & Johnson.

More than likely… You have probably heard of us if you are looking to buy what we sell.

One SEM goal is branding but that’s a very minor goal. The main SEM goal is direct response marketing to drive sales on the website.

In a similar consumer electronics example:
Lets say the consumer is searching to buy something small that they can easily purchase online directly form the manufacture like an MP3 player and that’s what they are searching for. “MP3 Player”

Lots of people make them so competition is stiff.
Lots of people sell your product but you make more money if they buy directly from your website, even after you factor the cost of SEM.

One strategy is to use SEM as a sales channel or treat Google like a commission based salesman for your MP3 player sales.

If you had a productive commission based salesman, why would you want to cap the max number of sales or commission they can earn or reduce the already low commission % the salesman earns on each sale? Why not just let them drive as many sales as they can as long as it’s cost effective?

The argument against this is that SEM is all just smoke and mirrors. We don’t know for a “Fact” if it’s “Cost Effective” or not for the reasons above. AND… We would have made the sales anyway. So… Why not reduce that expense? Or… Why not spend the money driving the traffic to the website with other, more traditional marketing channels that can’t be accurately measured either?

[mugshot]

What if we shut PPC & Search off for a month or two and then turned it back on? The resulting drop & spike in visits and revenue should indicate the “Real” value of the ad campaign but given the large fluctuations in our competitive conditions and seasonality the resulting drop then spike in traffic and revenue is not accurately measurable.

This is quite confusing, and this is why. For instance, while PPC is on (don't think you can turn off Search), and the revenue is $5,000 and when off, it drops to $2,000 - wouldn't you account the $3,000 to the PPC campaign - plus I'm sure the conversions are being tracked for the PPC campaign.

Saying that seasons play a part is not very relevant - PPC campaigns should be tracked regardless if its Christmas, Valentine's Day, Armageddon or My Birthday :-)

PPC traffic can not be properly tested or accurately measured. The PPC ads are either there or not. You can’t show ads to only half the audience. There is no way to set up a control group and measure the difference in performance.

Why not? It is true that PPC ads are either there or not - just like a TV commercial, radio spot, highway posters, etc...in Marketing, ads are either there or not. Heck, I'd go as far as saying that it's even like the physical store - if it's there, you can measure walk-in traffic sales, if its not - there's nothing to measure except the acidity of the soil :-)

In PPC, control groups are set up in campaigns - you can run several campaigns at once against a 'control group' depending on what you want to measure. For instance, if you're selling GREEN WIDGETS - there's a wide variety of them widgets...so the control group could be the popular widgets and the other groups can be tests of variations of green widgets.

People will find the website anyway:

Now that's like saying, people will find your store anyway - IF they drive around long enough

The whole point of showing your ads on front page (premium space) is so that people will not only find it anyway, but find it fast!

If they don’t find us on that particular query, it’s no big deal because they will find us on the next one or the one after that.

That's very wishful thinking - passive marketing in my opinion. Let's wait for people to find us, even though they are spending money...that's not the way to do it - at least I know it's not.

People comparison shop and PPC ads are an unnecessary convenience that does not drive the traffic or the sales.

Are you delivering the wrong message? Now keep in mind that PPC ads are not just to drive sales, it is also used as branding tools by a lot of major companies. Pepsi does not need to blow 1.5million on superbowl ads but they do it anyways...

Analytics data is not an accurate representation of reality and we should not pretend that it is.

No one is pretending it is - at least they shouldn't. TV, print, and radio advertising has been around for ages - and I don't think demographics provide a real representation of reality either but companies still spend hundreds of millions each year on those channels. How is PPC any different than just another channel of advertisements?

[Dave]

Thanks Mugshot,

I’m no good at using the quotes so please forgive my forum ineptness.

Mugshot Quote:
“The revenue is $5,000 and when off, it drops to $2,000 - wouldn't you account the $3,000 to the PPC campaign.”


In that case, it would be within the margin of error and an obvious drop.

But that is not my situation… I can’t give you any real numbers but let’s say the average revenue was in the area of 10 million and the PPC contribution was about 1 million and seasonal spikes and drops can be as much as 5 million… The margin of error would be grater than the PPC contribution. A drop in 1 Million could be easily attributed to other factors like seasonality or competitive conditions.

Mugshot Quote:
“in Marketing, ads are either there or not.”

Not necessarily…
With banner ads I can serve a real banner ad to half of the audience and a bogus banner ad for a charity or whatever to the other half. Once you have been segmented into one group or the other you will continue to either see our real ads or the bogus ads for as long as we run the test or until you delete your cookies.

So… One group sees our banner ads everywhere they look and the other group sees nothing but banners for a charitable organization.

At the end of the test, we can do a cookie level analysis of everyone who made it to the conformation page from both groups.

Lets say that…
7000 people that Did Not see our banner ads made a purchase anyway.
10000 people in the group who Did see the banners also made a purchase.

Because both groups had an equal number of people, who saw either test or control banners on the same websites during the same time period… We would assume that the real banner ads were responsible for an additional 3000 lift in orders versus the bogus banners.

Banner ads are safe because they CAN be measured this way. It has been incontrovertibly tested against a control group to create a statistically significant and economical lift in sales.

You Can Not measure how many people would have made a purchase anyway with search the way you can with Banner advertising. Search analytics can not measure lift against a control group who does not see search ads during the same period of time. The ads are either there or not. Lift if any in sales could be easily attributed to other factors like seasonality or competitive conditions for the reasons above.

The argument is, if the lift in search can not be accurately measured because of this limitation. You could argue that it might be possible to drive more sales on the website if you do no search marketing at all and save the money to do something else like a TV commercial, print ad or something cool or sexy that can’t be measured either. Or just save the money.

[mugshot]

Well, after I replied, I noticed this was a 2005 topic

But that is not my situation… I can’t give you any real numbers but let’s say the average revenue was in the area of 10 million and the PPC contribution was about 1 million and seasonal spikes and drops can be as much as 5 million… The margin of error would be grater than the PPC contribution. A drop in 1 Million could be easily attributed to other factors like seasonality or competitive conditions.

I guess what I'm trying to say is that if you set up a PPC campaign, you will be able to tell how much revenue comes from the PPC channel during month X. So let's say the PPC campaign made 100 sales in March and 150 sales in April but only 50 in May - wouldn't that be sufficient information for you to figure out if PPC is worth doing or not? The idea is to separate the PPC channel from everything else and figure out numbers for it alone. You can't mix the totals together or else PPC will never be justifiable.

With banner ads I can serve a real banner ad to half of the audience and a bogus banner ad for a charity or whatever to the other half. Once you have been segmented into one group or the other you will continue to either see our real ads or the bogus ads for as long as we run the test or until you delete your cookies.

The group that sees the bogus ads = ad not there But I do see your point. I do agree with banners you can count impressions vs. number of sales, but what if the same person views the banner from home, office, friend's house and then Kinkos? That'd be 4 impressions from 1 person...

Because both groups had an equal number of people, who saw either test or control banners on the same websites during the same time period… We would assume that the real banner ads were responsible for an additional 3000 lift in orders versus the bogus banners.

That's assuming every visitor is unique - which rarely is the case. In an ideal test condition yes, but test conditions are never a reflection of the real campaign when launched right?



I don't think anything can be measured with razorsharp accuracy - there's always a margin of acceptable error that companies have to accept...

[Dave]

Thanks again,

The banner test isn’t perfect. Even if some people belong to one group at work and the other group at home, the results are still dramatic enough to incontrovertibly prove a statistically significant and economical lift in sales even to the harshest critics. I believe that even the average rate of cookie deletion can be measured and factored into the margin of error and final results.

Search on the other hand can not. It can be argued that people will find the site anyway regardless of the PPC ads.

Just like the banner advertising… Many of the conversions that are attributed to search would have happened anyway BUT many would not. The question is “How Many”? Is it a little or a lot? Who knows? (IMO) I believe it’s a lot, but I’m having a hard time proving it in a way that is incontrovertible.

[Dave]

Thanks again Mugshot,
I’m actually getting myself a bit off topic.

The argument that PPC search is not accurately measurable is an argument that I don’t think I can win. The reasoning is too simple.

I might need to adopt a different approach and focus my simple elevator pitch from the benefits of tracking and accountability to what is more easily understood and incontrovertible.

Relevant ads are there when people are searching for them.
Clicks = visits.
An economical % of people who visit convert.
Risk of not participating in PPC is much harder to measure than reward.

[mugshot]

Relevant ads are there when people are searching for them.
Clicks = visits.
An economical % of people who visit convert.
Risk of not participating in PPC is much harder to measure than reward.

That's absotively true! People will click on relevant ads that appeal to their immediate needs.

The major difference is banners attract passive audiences, while PPC attract active audiences - people who are using the internet as a source of information.

If you manage your PPC campaign independantly from other campaigns, I'm sure you will be able to measure it accurately, regardless of seasonal spikes or drops - because then you only have one channel to focus on. It is not a fair comparison to benchmark it against banner ads, rather utilize it as another tool to increase revenue for your company.

And the same can be said for banner ads - the risk of not participating in it is much harder to measure also.

[Dave]

The part that is the most frustrating for me is this debate Can Be Avoided!

The search engines could easily let advertisers pay some sort of a “tracking fee” to place a tracking pixel in the SERPS or publisher network instead of an actual ad for a % of results displayed for a given keyword.

Half the audience always sees the PPC or contextual ad for that keyword and the other half never does until the test is over.

If I could do this… I could measure how many people who did Not see the PPC ad, somehow managed to make it to the conformation page anyway without the help of an advertisement, versus the control group who were shown the ads for that keyword or group of keywords.

A test like this would reveal the lift if any produced by the search engine.
This sort of transparency is a risk for the search engines. I don’t believe they will allow transparency of actual performance because some advertisers would suddenly realize that there is much less ROI than they previously thought.

On the other hand… Some will discover that there is a bonus “Billboard Effect” where the PPC ads helps support the natural listings and helps to increase click-through rates and conversions overall. Other people might also discover that people read the ad impressions and this influences the purchase decision even though the ad wasn’t even clicked on.

Either way… I refuse to believe the folks at Google aren’t smart enough to deploy a better testing environment that Banner ad agencies have been enjoying for years.

I highly suspect that our spending would increase significantly if I could accurately measure the effects of search.

I believe that increasing the transparency of ad effectiveness will bring the medium up to a whole new level of legitimacy and growth.

Until then, I believe I will need to avoid conversations or debates regarding accountability & tracking of our Search ROI.