Every little bit of data helps… to a point

This comment article is written by Paul Smith

It once seemed that most of us would end up working for Tesco or SERCO as both these companies swallowed up more and more of the economy. 

I have more than a passing interest in Tesco’s current woes as I am responsible for customer insight for my employer and also involved in two national projects looking at how ‘Big Data’ might improve public services.

For those of you not part of this geeky world, ‘Big Data’ involves taking huge sets of information stored on people by private companies and/or government and looking for patterns of behaviour which are strong predictors of future behaviour. 

The classic example which is trotted out in presentations on this is the Tesco Clubcard. This is scanned each time you shop so that Tesco knows exactly who is buying what. They use this information to target offers to you and also to profile what goods need to be stocked in which stores. 

If you go to Bradley Stoke Tesco you will find that it stocks far less ‘world food’ and vegetables than the Eastville store.

Tesco is now suffering major problems and there has been talk of scrapping the Clubcard altogether. But why?

I think it suggests a problem for people like me, and those who rely on us, about the use of data to make decisions about service provision. The problem Tesco has is that the Clubcard only registers what people buy from Tesco (and ESSO garages). Once your customers start deserting you for other providers all you know is what they are buying elsewhere which they used to buy from you. 

So-called retail experts suggest that there are three things going on: 

People are increasingly differentiating their spending buying some items in more upmarket stores like Waitrose and others in cheaper stores like Aldi and Lidl

The overall sale of food is falling (hopefully because people aren’t wasting as much as the figures published a few years ago indicating that a third of food was thrown away). 

The problem for the Clubcard is that it doesn’t know which items are being bought where, what spending is going up-market and what is going to the discounters. 

The problem for any system of analysing data is that you can only investigate the information which you have. The danger of this for those delivering public services is that they often only have the information which is directly relevant to their service from the people who are using it. Often the people who you really want information from are those who aren’t engaging with you. 

For example, a survey of people who use libraries about library services, will be a survey of those who are informed about what is on offer and think the service is worth using. They can tell you how to improve the service they use, but not how to expand the service to those who don’t.

In my work role with a housing provider, I know that issues such as mental health are crucial in terms of designing responsive services but this is not information which we have the expertise to gather, nor do we have the ability under data protection rules to obtain it from those that do have it. 

Unfortunately, we live in a society in which it is culturally acceptable to say “I don’t like maths” and public services are often managed by people who think like this. When presented with data they often accept it without question because they don’t have the skills to challenge them or disregard them because they don’t understand them.

Tesco are not alone in building their understanding on incomplete information but thinking they knew everything. When peoples’ patterns of behaviour change, knowing what they did before loses its value.

My interest in data big and small will continue but it’s crucial to understand its limitations. It’s also useful to remember that often anything held up as an exemplar or which wins awards is probably about to fall apart.

Picture: JuliusKielaitis / Shutterstock.com