A recurrent theme in these posts – and throughout Computerworld UK – has been the rise of vast, unnecessary and ultimately doomed databases in the UK.
But those stories have been largely sporadic and anecdotal; what has been lacking has been a consolidated, coherent and compelling analysis of what is going on in this area – what is wrong, and how we can fix it.
That analysis has just arrived in the form of the Database State report, commissioned by the Joseph Rowntree Foundation from the Foundation for Information Policy Research (FIPR).
Its authors include Ross Anderson, who chairs the FIPR. He is now Professor of Security Engineering at Cambridge University.
For anyone unfamiliar with his work, I strongly recommend that you acquaint yourself with some of his writings, all freely available on his home page. In particular, his research on the economics of computer security is likely to prove of huge importance in moving forward in this area.
Given his background, and the fact that Anderson is joined by other leading thinkers such as Ian Brown and William Heath, it should come as no surprise that the Database State report is both rigorous and richly detailed.
Indeed, it stands as the definitive description of the database disease that is afflicting the present UK government.
The report's summary findings are shocking:
A quarter of the public-sector databases reviewed are almost certainly illegal under human rights or data protection law; they should be scrapped or substantially redesigned. More than half have significant problems with privacy or effectiveness and could fall foul of a legal challenge.
Fewer than 15% of the public databases assessed in this report are effective, proportionate and necessary, with a proper legal basis for any privacy intrusions. Even so, some of them still have operational problems.
Britain is out of line with other developed countries, where records on sensitive matters like healthcare and social services are held locally. In Britain, data is increasingly centralised, and shared between health and social services, the police, schools, local government and the taxman.
The benefits claimed for data sharing are often illusory. Sharing can harm the vulnerable, not least by leading to discrimination and stigmatisation.
The UK public sector spends over £16 billion a year on IT. Over £100 billion in spending is planned for the next five years, and even the Government cannot provide an accurate figure for cost of its ‘Transformational Government’ programme. Yet only about 30% of government IT projects succeed.
The main part of the report consists of a detailed description and evaluation of 46 government databases. Each is graded red, amber or green according to how illegal or problematic they are. Worryingly, some of the biggest and most important databases are among the red list:
Follow highlights from ComputerworldUK on Twitter