A working life in data has made me very sceptical about the stuff and wary about what I conclude from statistics. Even so, all the numbers are a vitally important commodity that should be viewed as part of a country’s infrastructure – our lives would be much worse had we not benefitted from all the analysis that flowed from statistics.
Against that background, the event – “Measuring national well-being” – arranged last week at the LSE in London was a bit of a shocker. The event was part of the consultation being arranged by the ONS to help them get to grips with the request from the government to measure “happiness”. (See the Prime Minister’s speech from last November.) The event was chaired by the BBC’s excellent Mark Easton and had a panel of experts.
The ONS said what you’d expect it to say and Bridget Rosewell was wonderfully crunchy and realistic. But the other three gents on the panel, who were very pro measuring the fluffier stuff, inadvertently put the case very strongly against it. If they had their way the ONS would be asking questions like (to paraphrase each of them): “Are your relationships going well?”, “Do you think there is social justice?”, and “Is there meaning and purpose in your life?”
Response rates to surveys are already in free fall and I fear questions like this will simply help the trend.
Possible issues with such questions are many.
- How do you answer these on a scale of one to ten? Should anyone ever give a perfect 10? Would the interviewer leave behind a card for The Samaritans with anyone who answered 2 or below?
- How influenced should you be by the latest twist in life compared to the trend over decades? (In a great relationship with a good job but I’m cold as the central heating has just broken I’ll give a ……?)
- Is it healthy to think too much about purpose in life, especially during a doorstep interview?
- Is it absolute happiness or relative to those around you - many say that comparisons are the greatest source of unhappiness?
- Are any cultural and regional differences that pop up “real” or just reflecting the way that different people respond to such questions?
- Will these responses change much from year-to-year, let alone month-to-month? Would any change over time be reflecting the latest fashions in the press or some real change – or is the idea to measure the latest fashion?
I could go on.
More importantly, should the ONS – the government – be asking these questions? There are plenty of other organisations doing this and making a decent fist of it have a look at the Young Foundation , OECD and nef. Their results can be cobbled together to give a qualitative picture that might be instructive. But the ONS does quantitative not qualitative – and quantitative is dangerous when it is applied to fluffy topics. What would an average response of 4 against 6 mean? And over time if the figure drifts from 6.1 to 6.2, what would that mean? And how should government respond to a decline? There is no point, presumably, collecting data if it does not lead to an action (or meaningful insight). And what if the average stays the same but people are increasingly polarised along the scale? Distributions, extremes are perhaps more telling than the average.
It looks incredibly like we are about to collect dodgy data for no real purpose. And it is only right to ask if that is the way to spend money when public finances are tight? If the ONS uses this to spring clean their existing data collections to focus on the most useful – and to present and explain data to a wider audience, that’s great. But it should not risk its professional reputation by getting too heavily into dodgy touchy-feely stuff. Yet it is already committed to asking new questions from April – it has said it will and the Prime Minister asked for that. Perhaps surprisingly the questions are not yet chosen – a bit odd given these things are normally trialled and tested months in advance.
The Prime Minister’s speech was well-intentioned. He tried hard to explain away the three main objections as he set them out, namely: that this is a distraction from the major, urgent economic tasks at hand; that improving people’s wellbeing is beyond the realm of government; and, that the whole thing is a bit woolly and impractical. He said; “That’s why anyone who cares about community, about civility, about making this country more family-friendly I think should welcome what the Office for National Statistics is doing.” Umm. It’s a good idea to spark a debate about “what really matters”, but he shouldn’t have got the statisticians to do it.
Let’s be thankful that the ONS has been given only £2 million to fund this work – and that is £2m over four years! That doesn’t buy much change or progress in the world of official statistics. Thank heavens. I’m feeling better already.
Further reading idea: Michael Blastland on the BBC.
To suggest that there is anything new about these data is slightly misleading – DEFRA has been doing something similar for quite some time….
http://sd.defra.gov.uk/2010/07/wellbeing-statistics/
Apparently:
1. People in England rate their satisfaction with life 7.5 (average response when asked in March 2010 to rate satisfaction with life on a scale from 0 to 10);
2. There has been little change in overall life satisfaction in recent years;
3. More people were satisfied with certain aspects of life than in 2007.
Perhaps a dataset to add to your library??