Unleashing the Potential of Open Data

It seems a long while ago now, but June was a pretty hectic month in this neck of the woods, since it saw the final push to get ACTA rejected in the European Parliament. But of course, plenty of other things were happening then, and one in...


It seems a long while ago now, but June was a pretty hectic month in this neck of the woods, since it saw the final push to get ACTA rejected in the European Parliament. But of course, plenty of other things were happening then, and one in particular that I wanted to cover was the release of this UK Open Data White Paper entitled "Unleashing the Potential".

It's a measure of the glimmers of hope that the entire UK open data project emits that the document is available not only as a PDF and in Microsoft Word, but also as an .odt file – kudos to those involved for making this happen.

In a way, that's not so surprising since, as I've noted before, the UK's open data programme really is one of the best things the government has done in the field of technology, and is arguably now leading the world, something the white paper is not shy about pointing out: "In the last two years, the UK has released the largest amount of government data of any country in the world."

The current document reflects that, since it is packed with details of just what has been achieved, and overall it is pretty impressive. I think it's required reading for anyone interested in the areas of openness in general, and open data and open government in particular. As is customary, it's well written, and has embedded in it a series of case histories that leaven its otherwise rather intense narrative. Here are just a few of the highlights I spotted as I read through it.

For example, I hadn't realised that the Freedom of Information Act was being subtly modified to take account of the need to give people the ability to access not just information, but data too:

The Datasets Section of the Protection of Freedoms Act, which received Royal Assent in May 2012, enhanced access to data. This new section pushes public authorities to consider disclosing data that is not already routinely published and sets out what is meant by a dataset – the raw, factual or source information behind a public function – to allow for a better understanding between the person requesting the dataset and the public authority holding the data of what is being sought.

The new section means that, where the FOIA requires release of a dataset, a public authority will now have a responsibility to release the dataset in a form that can be used and re-used by specifying the licence it can be used under and, where reasonably practicable, in a re-usable format. We have created the conditions for accessing raw, unmanipulated datasets and, by working to embed the release of the dataset in a public authority's publication scheme, we have taken an important step to give the public an enhanced right to data.

To ensure a comprehensive approach to the commencement of the Datasets Section in the Protection of Freedoms Act, we have amended Section 45 of the FOIA and will be expanding its accompanying Code of Practice specifically in relation to the new Datasets Section as set out above. The expanded Code of Practice will give guidance to applicants and public authorities on how to deal with requests such as: giving permission for datasets to be re-used; the release of datasets in an electronic form which is capable of re-use; the making of datasets available for re-use in accordance with the terms of a licence; standards applicable to public authorities in connection with the disclosure of datasets; and other administrative issues related to making datasets available for re-use.

This is really rather important, because it broadens the scope of the FOIA considerably, and will help to get the message across to central and local government departments that datasets must also be released, just like the more discrete pieces of information they currently send out.

A reference to open data activities Europe-wide gives us some sense of the scale of things:

Late last year, the European Commission published its proposals to amend the Directive, Communication on Open Data: An engine for innovation, growth and transparent governance, forming part of the EU's 2020 Strategy to promote growth in Europe's economies.

The Commission believes that adopting Open Data principles and removing barriers to re€‘use can generate economic benefits in the order of £33 billion a year. Following from the UK's example, the European Commission believes that opening up public data leads to greater transparency in public administration and promotes social and political engagement. The last key theme of the Commission's proposals centres around its strong belief that increasing the availability of EU-wide public data will lead to better evidence-based policy making across the public sector, resulting in more efficient and effective public services.

I was also pleased to see among the so-called "Information Principles" the following:

Public data will be published using open standards, and following relevant recommendations of the World Wide Web Consortium (W3C)

This principle applies to published Open Data but it is of course wholly consistent with the Government's overall policy approach on the use of open standards in government IT more generally, on which a consultation has recently concluded.

As readers of this column will know, unlike open data, which seems to enjoy almost universal support, open standards is highly contentious. So it's good to see this commitment here.

Interestingly, the currently quite fashionable area of open access also makes an appearance in the report, with the following paragraph specifically about open access for businesses:

For small businesses in particular, relevant research is often difficult to find and expensive to access, limiting the spread of knowledge and innovation. To address this, Research Councils will invest £2 million in the Gateway to Research project which will launch in December 2013. This will provide a single point of access to all aspects of UK publicly funded research. It will serve as a networking tool for use by SMEs, entrepreneurs, intermediaries and other advisors in the field. Research Councils are currently on track to develop a prototype demonstrator portal by November 2012.

Aptly enough, towards the end of the white paper, there is a section dealing with perhaps the new frontier of open data – that held not by government, but by businesses. Obviously, providing access is more problematic, so it's good to see the UK government starting to address the challenges here:

BIS is running a programme, midata, with leading businesses to provide customers with access to personal data held about them in an electronic re-usable format; often referred to as smart disclosure. We are starting to see the fruits of that programme with the energy sector (see Chapter 5). This opens up the exciting prospect of data from both the private and public sectors being brought together with the customer's permission to provide greater insight and the potential development of powerful new services.

The goal of midata is to empower the individual to make better decisions. Transparency is at the heart of the programme, but it also seeks to improve consumer access to and control over their data and ultimately give the power to transfer it to business partners of their choice in exchange for services and other benefits. Naturally, the principles behind the Government's vision for midata have much in common with those that underpin public data.

As this indicates, open data is by no means "done", but remains an extremely active area of research and exploration. The current white paper confirms once again that the UK is right at the forefront here; long may it remain so as its potential is indeed unleashed.

Follow me @glynmoody on Twitter or identi.ca, and on Google+

"Recommended For You"

Open data in the UK: a job half done Openness: An Open Question