6 traps to avoid when it comes to taking full advantage of Big Data in your business

We’re a shopper research agency that provides clients with concrete commercial actions they can take to achieve their business outcomes in store
5 Minute Read

6 traps to avoid when it comes to taking full advantage of Big Data in your business26 Oct 2017

The following post was originally written for Inside FMCG Magazine. 

Big Data is sometimes referred to as “Big Mess” by some of my clients, and while Marketers are “talking the talk” on Big Data, they’re often struggling to “walk the walk” when it comes to making the data work effectively for their organisation. In my company’s experience of working with larger data sets, we’ve learned a half-dozen tips and tricks that will help the new user make sense of this information in a more enjoyable and satisfying way.

Loyalty cards, credit cards, digital searches, housing statistics – the data sets available in Australia to help with business decisions are becoming ever more numerous, and ever more impressive and accessible. Data owners are sharing more information than ever before – within Data Protection Act guidelines of course – and innovative warehousing/sharing platforms such as Data Republic are making this information readily accessible to the marketer.

But don’t kid yourself that “access to data” automatically means “success in using data”. Quite the contrary. In many of the companies we speak to, their in-house teams, and/or suppliers, have produced endless visualisations, dashboards, “live” reporting tools, segments, databases… but often with little real insight or “a-ha” moments. Why? Well, here are just six traps clients (and agencies alike) fall in to…


Data is a means to an end, not an end itself.

Many of the “data visualisations” and “dashboards” and “analytics” that I see companies working on lack this one fundamental notion: “What problem are we trying to solve?”

All companies have commercial issues that need to be resolved: they may be losing share to a rival, their category may be stagnating, their profitability may be declining, their customers may be less loyal to their brand, and so on. It is these issues, not data, that keep senior management folk sleepless. You won’t find the C-Suite talking “repertoire sizes”, or “inter-purchase intervals”, or “basket sizes” or “purchase frequencies”; rather you’ll hear them talking “profit”, “sales”, “growth”, “share”. These are the issues that Big Data needs to address.

An analogy here: replacing a single-shot pistol (small data) with a machine-gun (big data) is pointless if you do not have a clear target to aim at. Many companies fail at this first hurdle. They sit down with the data and start to create outputs, whereas the first task should be to sit down with their stakeholders, identify what they need to do their job, and then try to find the answers in Big Data.

Always look for the problem first, then the answer.


In this age of corporate organisation charts, structures, distinct roles & responsibilities, and large teams, it is quite common to observe the “silo effect” within a client business. That is, one part of the business (sales, say) and another part of the business (maybe marketing) work in quite separate ways, to quite separate KPIs and goals, and with “separate data” supporting their decision-making.

Big Data is often handled by one team within the business, perhaps the sales people for scan/transaction data, or the finance/IT team for reporting weekly sales/finances for example. The data can be strong at showing what customers and consumers are buying, or their behaviour patterns.

However, big data seldom tells us the reasons behind hte buying pattern or behaviour. This “insight” often sits within a separate team within the client organisation, as it is considered less about data, and more about understanding customers, and therefore the domain of a “non-technical team”.

The what and the why should both be understood together, side by side. There are several ways to do this. For example, Tesco UK deployed a technique called “trigger surveys”, where a change in behaviour observed in Big Data triggered a drill-down study to seek a reason.

More simply, businesses that encourage the collaboration of their “what” (sales) with their “why” (marketing) teams and align them to the same goals, KPIs and customer story, will get a more holistic understanding of the market.

Always combine the “what” of Big Data with the “why” from other sources.


We’ve all heard of the ‘KISS’ principle: ‘Keep It Simple, Stupid’. This rule applies to anyone handling and presenting information. Numbers are not everyone’s cup of tea, and academics will tell you that fewer than 5% of all students are really good at maths, yet those presenting Big Data insights and dashboards delight in supplying many numbers and many layers of details just because they can.

The Big Data set allows a level of number crunching that has never been seen before, for example we can drill down beyond state and into specific streets/blocks (called statistical area codes), which means that information can be so much more granular and populous. The more granular data needs clever tools and charts to make it easy to understand: tables, pie charts and bar charts were suitable for “small data” but are not suited to handling the many layers of analysis now available.

The invaluable skillset here is a mathematician who can speak in simple terms, who can make the complex appear really obvious. These are rare creatures, so if you find one, keep them central to your organisation. Their ability to explain data in terms a 5-year-old can understand will allow your business to make full use of all the layers of information that can be found in Big Data.


I have attended dozens of client meetings about Big Data where the attendees, and sometimes even the custodians of the data in the organisation are the IT department, or the Chief Information Officer. When I ask why this is the case, the answer is always the same: “They understand how to store, access, and crunch the Big Data.”

I’m not denigrating the value of technical experts. Indeed, they should always form a part of the advisory council. What I am questioning is the value of these teams leading or driving the use of Big Data within your business. Do these people understand your business challenges intimately?Do they communicate with your stakeholders (your retail customers)? Do they address your growth strategy and plans within the business? No? Then they’re probably not ideally qualified to get your Big Data working for you in terms of answering critical questions, driving the business strategy, and influencing your customers.

IT and technical support are what they say on the pack: they are people who understand software and hardware systems, but they should support rather than lead when it comes to Big Data strategy. Strategy needs to be led by people who drive the thinking and decisions within the business.

So, skill up your business people to be technical. Not your technical people to be businessmen.


Having worked in the worlds of management consulting then market research, I am familiar with many of the concerns about business data in general: “We are drowning in data” … “We have data but no insight” … “Our data doesn’t tell a story, or is contradictory” … “We don’t have a united view of the market” … “The data looks at the past and doesn’t tell us what to do in future”.

All these issues still prevail as the world moves towards new sources of data such as loyalty cards, website search statistics and so on. The problems with data don’t suddenly dissipate simply because the source of the numbers changes.

So what can we all do to overcome these challenges? Well, the same old rules still apply to Big Data as apply to the worlds of insight, finance and sales. That is, we still need good story-tellers in our businesses, people who can look across multiple sources and form one holistic story. We still need predictive analytics that take past patterns and project what these might mean in the future, and we still need people who can really get under the skin to understand why people are thinking and acting in ways that don’t seem to make sense.

So, apply the same rules to Big Data as you do to other insights: demand a story, a “so what”, a “what next” from your Big Data colleagues.


Big Data sets all tend to have one thing in common. While they have a large number of observations of behaviour, they tend to collect only a limited amount of information about each transaction.

One credit-card data set we work with, for example, only has eight pieces of useful information about each time a person uses their card. However – and this is the exciting things in the world of Big Data – information can be merged at either the level of household or statistical area (group of households, smaller than a postcode). This is still within the data protection rules with individual households being anonymised, but it lets us build a more rounded picture of what particular types of households are doing.

For example, by merging my household’s credit-card spend with my household’s airline loyalty behaviour with my household’s supermarket behaviour (all the while protecting the anonymity of my house), we can build an understanding of what groceries I buy, where I travel, and what other stores I use, which is a more rounded understanding than one data set, no matter how big, can provide.

So, always think about what you need to know about the customer, and which different data sets could be combined to obtain that holistic understanding.


Finally, Big Data is really not that scary. It is simply new data, that needs to learn some new rules to be successfully applied. But Big Data is really very exciting, when driven by someone who knows how to control it.

No Comments

Post A Comment