Fans of open data will be delighted with a letter published yesterday by the new head of the government statistics watchdog. Andrew Dilnot, who arrived in post at the start of the month, has announced that government statisticians’ special analyses of data, requested by others, will be made publicly available.
This is a bigger deal than it sounds. We at Timetric try to make as much official data readily available and usable as possible. With well-known headline series we can find the data on the relevant web sites but much more data is available in government than published. Some of this data – typically novel breakdowns of headline series - surfaces when special analyses are requested. They can sometimes offer great insight and frequently make front page news. Yet, the new series are sometimes hard to extract from the statisticians even though they’ve been sent to the body requesting the data. The stories in the media often fail to tell the whole story and very rarely give all the data.
The letter to Full Fact says that: “the Statistics Authority supports the Government’s Open Data initiative and wants to see all statistical material (that is not disclosive of confidential information) readily available in the public domain. The Board has instructed ONS to ensure that responses to ad-hoc requests from the media and other bodies engaged in public debate should be made available on the ONS website at the time the data and analysis is released. The National Statistician is also preparing related guidance for the wider Government Statistical Service.” The fullfact.org take on this is here.
While we’re at it, we can highlight the report of an interview given by Andrew Dilnot to “Civil Service World” the in-house mag for public servants. He makes it pretty clear that he wants to see a major reduction in the practice of pre-release access whereby a variety of – often many – servants, ministers and special advisers get to see official data before it’s published. This early access leaves open the chance to spin the data in the media or accidentally release it too early. Neither of these activities seems to be common but even the small number of instances which are documented must contribute to the public’s diminishing trust in data and government.
Dilnot is quoted as saying: “I just think we should stop it”, because otherwise “there will continue to be accusations that other announcements are made because somebody has seen sight of something that’s going to be embarrassing.”
“In a climate where we want to enhance the sense of public trust in statistics, that retention of control over who gets to see them early is inimical to that sense of public trust,” he added, suggesting that government should not get to decide who sees statistics early and that everyone should see them at the same time.
This would seem to be a strengthening of the stance taken by Dilnot’s predecessor which was set out in a report published a couple of years ago. There’s a bit more background here, on the straightstatistics.org site.