Why better data = better ROI for your business

Before we start talking about Return On Investment (ROI), let us first remember that famous quote by British statistician George Box, “all models are wrong.”

In marketing, it’s easy to trick yourself into believing that the success of any given campaign can be measured through a simple comparison of profit vs. investment. Alternatively, without immediate access to the data, some marketers will instead compare the results generated within the time frame of the campaign against a controlled time frame. This method is also somewhat short-sighted – the customer journey is not quite so simple to track.

Then we have attribution models – a huge leap forward in terms of campaign result analysis. Attribution models help us monitor the journey of each and every customer. However, while these models can help us determine which actions are driving results, they’re not always helpful in understanding the impact of our investments. Most importantly, attribution models are less useful when it comes to the optimisation of our marketing budget. Measuring ROI using algorithmic attribution, on the other hand, could be described as an attribution model on steroids.

Calculating ROI

Calculating ROI is no easy undertaking. Not only do you need an attribution model to make it work, you’ll need a baseline for results and growth, and you’ll need to familiarise yourself with all the external variables you have no control over. You’ll need to understand the maths behind it and, of course, you’ll need access to lots and lots of data. It’s better to build a decent model supported by lots of data, than a great model with only a small amount of data to support it.

Any of these terms confusing you? Let’s explain them in a little more detail:

Baseline

Assuming your product isn’t overpriced, useless or horrible, chances are your product is already selling. You already have some customers loyal to your brand and, hopefully, you’ve seen some day to day growth. Knowing how much you sell without investing time or money into any additional marketing efforts is crucial. Once you know your baseline, you can compare results more accurately post-campaign. But you already knew that. Determining “natural” growth is the tricky part; sometimes growth has nothing to do with marketing, and everything to do with word of mouth and online reviews.

Internal Variables

At this stage, you need to begin the process of comparing initiatives vs. results. Not just in terms of sales; depending on your product or service, initiatives could also include a one time promotion, discount, price adjustment or loyalty programme, while the results could range from website visits to subscriptions. Remember, a single initiative can make an impact in a number of different ways – it’s important that we measure which initiative has had an effect on which result and carefully analyse these behaviours (note the use of the word ‘impact’, correlation does not imply causation).

External Variables

Sometimes you do everything right, but you still don’t get the results you wanted. Conversely, even if you did everything wrong, you could still reap the rewards that come with a successful marketing effort. Welcome to the world of external variables; things we have no control over can still have a huge impact on everything we do. Some things are predictable and creating models around them is relatively easy, e.g. seasons, weather, new trends. Some events are less predictable, sometimes even impossible to plan around – we call these events ‘black swans’.

Attribution Models

Think of an attribution model as a map guiding you along the consumer journey – this map will help you determine the direction they’re going and the roads they’ve already taken. Sometimes you have control over the road they take (paid), sometimes you don’t (organic). This map will help you develop a deeper understanding of your consumers and the individual journeys they’ve taken to arrive at your product or service. The problem starts when you try and optimise your budget based on this information – not every consumer will take the same route, some customers take roads we have no control over, and by focusing on fixing one road we might be leaving another to deteriorate. Attribution models do not take into account external variables – complicating things further. That’s where algorithms come in.

Algorithmic Attribution

Once you’ve taken into account all of the variables, you need to find the model which best suits your campaign. Luckily for you, you can build and test an algorithm to determine which model will work best for your business. Feed the algorithm with all of last year’s variables and compare them with the actual results. If the algorithm works, move forward. If not, readjust and try again. Make sure you’ve used a big enough sample to ensure that your model isn’t a one hit wonder. And more importantly, make sure to continually feed your model more data so that it will adjust to new information.

Next steps

Now you’ve created the perfect algorithm, you can start playing around with it. What would have happened if you had invested more in initiative A and a bit less in initiative B? How should you have distributed your budget between different initiatives? Which combination yielded the best ROI? Knowing how each initiative impacted the results generated will help you to maximise performance and investment.

Side note: the simplicity of model-based analysis can be somewhat misleading (“all models are wrong”). For example, if your AdWords campaign isn’t working it may be because now is not the time to invest in Adwords, but it could also be because your ad doesn’t appeal to your consumers or you’re focusing on the wrong keywords or sites. The truth of the matter is, there is no easy way of determining which marketing initiatives will work and which won’t. You’ve got to make optimisation a priority, before attempting to nail down the perfect marketing initiative.

Keep in mind that predictive analysis is not just about predicting the future (predictive models), it can also be used for classification (descriptive models). When combined, these models are extremely powerful because they enable marketers to predict how a specific marketing initiative will affect a specific type of consumer. However, you have to be careful when making recommendations based on predictive analysis; trends change, prices rise, public interest waines, and new products and services are brought into the market every day. You have to consider every new variable. Nothing is constant forever, but this should not discourage you from modelling, searching for insights, making recommendation or even predictions. Be smart about it, expect to make some errors in judgement. Things may evolve, but it’s better to have an idea of the route you need to take, rather than just follow the road blindly and hope for the best.

Analysing and predicting ROI requires a lot of modelling, a process which is both an art and a science. It’s a difficult task that’s proving more and more necessary in a world where CEOs and CFOs are less interested in brand awareness and brand recall, and more interested in ROI within the marketing department. So if you find yourself thinking back to that opening quote and asking yourself why you should bother, you should probably take a moment to read how George Box went on to complete his quote 11 years after he first said it: “Remember that all models are wrong; the practical question is, how wrong do they have to be to not be useful?”