Pyramid Schemes and Marketing Reporting

I’ve seen a lot of reporting during my time in marketing and much of it has been a dog and pony show. I’ve seen one marketing department hide data that gave credit to functions other than marketing. I worked at a place where every month was better (by some arbitrary metric)  than the month before. Every. Single. Month. It’s like Bernie Madoff is in our marketing department.

If we are good at marketing, we should be seeing consistent growth but we should also see some dips, some places where we missed the mark, some different behavior based on seasonality. If every report is better than the previous, one of two things is happening: the person reporting is cherry picking data or they are playing it very safe. If we are doing everything right, we probably aren’t doing anything great. Failure is a necessary part of growth. We need a month where we tried something new and risky that didn’t pan out. 

When people are reporting on their own efforts, you are going to get bias. No one will want to admit that what tried didn’t work. It is similar to the reason you should never QA or edit your own work. There is also the issue of asking non analysts to do analysis. This means many marketers aren’t even setting up experiments or hypotheses to measure a result against and instead are just using anecdotal evidence. We need a separation between execution and reporting. We need people that will look at the data objectively to find the gaps and issues that are holding us back from being better.

Reporting should be more about learning and finding risks or opportunities than validating our work. Every time we report something to be better than it is, we are borrowing on our future results. Eventually the question will be raised, “if everything is going so well, why aren’t we growing”? It’s just another version of a Ponzi scheme that will eventually come crashing down.