Updated 5 May 2016: A leaked draft statutory instruments document has detailed how the government is seeking to compel telecommunications operators to provide real time access to named individuals’ communications within one working day under the recently passed Investigatory Powers Act. This includes encrypted messages.

The government also asks for the capability to "provide and maintain the capability to simultaneously intercept, or obtain secondary data" from 6,500 people at any one time. For more information visit our sister site Techworld.

The Investigatory Powers Bill was agreed upon by both houses of Parliament and was passed into law by Royal Assent on 29 November, making it the Investigatory Powers Act.

Concerns around the balance between privacy and security were apparently sufficiently quelled in Parliament's second chamber by an amendment which stated: "This Act sets out the extent to which certain investigatory powers may be used to interfere with privacy."

Lord Janvrin - who proposed the amendment - said of its approval: “That there is merit in placing a simple statement right at the forefront of the legislation to provide additional clarity that there should be no doubt that privacy protection remains a fundamental priority.”

To save you scrolling here are the main powers the bill brings into law:

Web and phone companies (CSPs) will store records of websites visited by every customer for 12 months for access by police, security services and other public bodies uponissue of a warrant. However, they won't be able to track sites accessed over various VPNs. See: The best VPNs list.

Security services will be legally empowered to bug computers and phones upon approval of a warrant. Companies will be legally obliged to assist these operations and bypass encryption where possible (more on this below).

Security services can acquire and analyse bulk collections of communications data. For example, this could mean a bulk data set such as NHS health records.

Oversight for these operations will come with a new “double-lock”, where any intercept warrants will need ministerial authorisation before being judged by a panel of judges, who will be given power of veto. This panel will be overseen by a single senior judge, the newly created Investigatory Powers Commissioner.

Read on for what the bill includes, the safeguards put in place and the criticism it has faced over the past twelve months.

Update 19 July 2016: Concerns over the government's approach towards encryption have reemerged during the second reading of the Investigatory Powers Bill in the House of Lords last week (you can read the entire debate here).

Earl Howe, minister of state for defence and deputy leader in the House of Lords, said: "It may be entirely sensible for the government to work with [communication service providers] to determine whether it would be reasonably practicable to take steps to develop and maintain a technical capability to remove encryption that has been applied to communications or data."

“Law enforcement and the intelligence agencies must retain the ability to require telecommunications operators to remove encryption in limited circumstances," he said.

Read next: The Snooper's Charter still has an encryption problem

However, Nic Scott, managing director UK and Ireland at enterprise data security specialists Code42, makes the important observation that there are no "half measures" when it comes to encryption. He says: "You either have encryption in place or you don’t. Once you create a backdoor for law enforcement purposes, you are also opening the door to other, potentially malicious, parties."

Update 8 June 2016: The Investigatory Powers Bill was comfortably voted through its third and final reading in the House of Commons, with 444 MPs voting in favour to 69 against. It will now be sent to the House of Lords to be scrutinised by peers.

This comes after the Labour Party said that the bill had met the significant demands made by shadow home secretary Andy Burnham back in March.

Significant Demands

Update 16 March 2016: The shadow home secretary Andy Burnham laid out six areas of concern for the Investigatory Powers Bill back in March: “If the government fails to respond adequately to the concerns that I have raised,” he said, then the Labour party will withdraw its support for the timetabling of this bill. The current timetable as laid out by the government is to pass the bill into law by the end of 2016.

Speaking at the second reading of the bill, Burnham said: “This bill isn’t yet good enough. Simply to block this legislation would in my view be irresponsible, it would leave police and security services in limbo. We must give them the tools to do their job. The public interest lies in getting this right and in not sacrificing quality to meet the deadline.”

The six concerns are as follows:

1. Privacy: “The Home Secretary said [privacy] was hardwired into the bill, but I see them as more cosmetic changes and haven’t directly answered the concerns of the joint committee.” Burnham asked that the bill takes a “presumption of privacy”.

2. Specific powers: “Internet connection records (ICRs) have been described as the modern equivalent of an itemised phone bill, however the joint committee noted that this is not a helpful description.” Burnham went on to explain that there should be a “higher hurdle” for use of this power limited to cases of serious criminal activity rather than any crime. He also asked that the terms “national security” and “economic wellbeing” are defined more explicitly.

3. Internet Connection Records: “Definitions of ICRs (Clause 54) remain vague and I see them becoming more intrusive as technology advances. A stricter definition of what can be included in an ICR should be included. The current confusion is clouding this bill and needs to be clarified.”

4. Bulk Powers: “Routine gathering of large quantities of information from ordinary people does lead to privacy concerns and should be as targeted as possible […] It is for the government still to convince the public that these powers are needed.” Burnham asked specifically for an independent review to conclude in time for report and third reading on this issue.

5. Judicial oversight: “The government has given significant ground on this issue and the bill is stronger as a result, however we believe it could be stronger still […] I have previously shared concern that this leads to a narrower test looking at only the process and reasonableness of the home secretary’s decision, rather than actual merits and substance of the warrant.”

May had earlier reassured the house that judicial commissioners will have access to the same information about a warrant as the home secretary. Burnham recognised this, but continued: “If this is the case why not delete the judicial review clause? To make it absolutely clear that this is not just a double lock, but an equal lock.”

6. Misuse of the powers: “There needs to be safeguards for the collection of data in a lawful manner and we must also agree that there needs to be an overarching law for the obtaining of data and any use that data is subsequently put to. Both should be a criminal offence.”

What next? The bill now enters the committee stage, where the government has the opportunity to make amendments following the issues raised during the second reading. The bill will then return to the House of Commons for the report stage, where MPs have their final opportunity to debate and vote on the bill ahead of the third and final reading and vote.

First reading

The Home Office published the investigatory powers bill on March 1. Following criticism from three joint committees: the science and technology committee on February 1, the intelligence and security committee on February 9 and most importantly, the joint committee for the bill itself on February 11, Home Secretary Theresa May claimed that the revised bill "reflects the majority of the committees’ recommendations.”

The joint committee for the draft investigatory powers bill made 86 recommendations for changes to the bill in its report, concentrating on issues of clarity, judicial oversight and justification of the various powers. For a summary of these suggestions skip ahead one section.

Addressing these specific concerns, May said of the revised bill: “We have strengthened safeguards, enhanced privacy protections and bolstered oversight arrangements.”

Not everyone agrees with this assessment though. Dr. Gus Hosein, executive director of Privacy International said: “It would be shameful to even consider this change cosmetic […] The continued inclusion of powers for bulk interception and bulk equipment interference - hacking by any other name - leaves the right to privacy dangerously undermined and the security of our infrastructure at risk.”

The headlines

When the bill was first proposed Clause 71 led the news agenda. This requires web and phone companies (CSPs) to store records of websites visited by every citizen for 12 months for access by police, security services and other public bodies.

In practice this would take the form of an itemised list of each citizen’s browsing history. This would not be a list of the specific web pages but the main domain (so but not the specific stories you read) so a basic online footprint can be drawn up. One concern here will be around the security of this data, especially in the current climate of TalkTalk customer hacks and data dumps.

The bill seeks to make the power for security services to acquire bulk collections of communications data explicitly legal. For example this could mean a bulk data set such as NHS health records.

Security services will also be legally empowered to bug computers and phones upon approval of a warrant. Companies will be legally obliged to assist these operations and bypass encryption where possible (more on this below).

The science and technology joint committee report tackles the possibility of public concern over the power to hack devices, stating: “The tech industry has legitimate concerns about the reaction of their customers to the possibility that electronic devices could be hacked by the security services,” before stating that the government has a responsibility to inform the public about the extent to which this power may be used.

Oversight for these operations will change, with a new “double-lock” where any intercept warrants will need ministerial authorisation before being judged by a panel of judges, who will be given power of veto. This panel will be overseen by a single senior judge, the newly created Investigatory Powers Commissioner.

For some context, figures from the Home Office, as published by The Guardian, show there were 517,236 authorisations in 2014 of requests for communications data from the police and other public bodies and a further 2,765 interception warrants authorised by ministers.

The joint committee report

The joint committee for the bill issued its report, along with a list of suggested amendments for the bill, on February 11. The suggestions include:

- Clarification over the concept of end-to-end encryption: “The Government still needs to make explicit on the face of the bill that Communications Service Providers (CSPs) offering end-to-end encrypted communication or other un-decryptable communication services will not be expected to provide decrypted copies of those communications if it is not practicable for them to do so.”

Update 4/3/16: On the issue of encryption the government says the revised bill: “Clarifies the government’s position on encryption, making it clear that companies can only be asked to remove encryption that they themselves have applied, and only where it is practicable for them to do so.”

- CSPs being forced to retain internet history data of users should be provided with "whatever technical and financial support is necessary to safeguard the security of the retained data" but the government shouldn’t be responsible for 100 percent of the costs.

Update: 31/3/16: The Don’t Spy On Us coalition, which includes the Open Rights Group and Privacy International, has said: “Proposals to collect the internet connection records (ICRs) of every UK citizen could cost more than £1 billion.” This figure is based on a similar scheme which has since been dropped in Denmark due to the cost. The initial Home Office estimate for the storage of these records was just £174m over ten years.

- "Fuller justification" for bulk surveillance: "We believe that that the lack of a formal case for bulk personal datasets (BPDs) remains a shortcoming when considering the appropriateness of this power."

- There should be no power to ask foreign intelligence agencies to undertake surveillance where the UK authorities cannot, for example in the USA.

Update 4/3/16: The revised bill "explicitly bans our agencies from asking foreign intelligence agencies to undertake activity on their behalf unless they have a warrant approved by a Secretary of State and Judicial Commissioner."

- Hacking should be targeted: "Targeted interception and targeted equipment interference warrants cannot be used as a way to issue thematic warrants concerning a very large number of people."

- An annual report that must contain: "Information about the impact, results and extent of the use of powers in the bill so effective public and parliamentary scrutiny of the results of the powers can take place."

Writing for, Liberal Democrat MP Lord Strasburger, who sat on the joint committee, said: "This bill is a long way from the finished article. It needs more than mere tweaking, it needs to be fundamentally rethought and rebuilt. The Home Office should stop rushing to push it through and take its time to get it right."

The intelligence and security committee criticised the bill for a lack of clarity and transparency around powers and suggested wide ranging amendments to the bill. Mostly though the committee says that the bill could benefit from starting again, saying: "The draft bill adopts a rather piecemeal approach, which lacks clarity and undermines the importance of the safeguards associated with these powers.

"We have therefore recommended that the new legislation contains an entirely new part dedicated to overarching privacy protections, which should form the backbone of the draft legislation around which the exceptional powers are then built. This will ensure that privacy is an integral part of the legislation rather than an add-on."

The science and technology select committee report confronted concerns over the impact the legislation could have on the UK’s technology sector; equipment interference powers and a lack of clarity when it comes to the issue of encryption.

Nicola Blackwood MP, chair of the science and technology committee said: "It is vital we get the balance right between protecting our security and the health of our economy. We need our security services to be able to do their job and prevent terrorism, but as legislators we need to be careful not to inadvertently disadvantage the UK’s rapidly growing tech sector."

The joint committee on the Draft Investigatory Powers Bill was sent 148 sets of evidence raising concerns and views about aspects of the legislation, and May faced questions over these concerns in front of the committee in December.

The politics

There was generally cross-party approval of the bill as first proposed, with Shadow Home Secretary Andy Burnham stating that it was “neither a snooper’s charter nor a plan for mass surveillance.”

Conservative MP David Davis has been one of the more outspoken critics of the proposed legislation. Talking to The Guardian he said: “In every other country in the world, post-Snowden, people are holding their government’s feet to the fire on these issues, but in Britain we idly let this happen […] Because for the past 200 years we haven’t had a Stasi or a Gestapo, we are intellectually lazy about it, so it’s an uphill battle.”

Author and journalist Heather Brooke went one step further. Writing for The Guardian she said: “The spies have gone further than [George Orwell] could have imagined, creating in secret and without democratic authorisation the ultimate panopticon. Now they hope the British public will make it legitimate.”

Edward Snowden tweeted: “By my read, #SnoopersCharter [The Draft Investigatory Powers Bill] legitimises mass surveillance. It is the most intrusive and least accountable surveillance regime in the West.”

Apple submitted a formal submission to the bill committee, specifically around the issue of encryption, on Monday 21 December, expressing: “We believe it would be wrong to weaken security for hundreds of millions of law-abiding customers so that it will also be weaker for the very few who pose a threat. In this rapidly evolving cyber-threat environment, companies should remain free to implement strong encryption to protect customers,” as per The Guardian.

According to YouGov the UK public generally approve of surveillance, with 44 percent of respondents stating it wouldn’t bother them to know that they could be spied upon and they don’t think they are at this time.

Obligations on communications service providers

The use of investigatory powers relies heavily on the cooperation of so-called 'communications service providers' (CSPs) in the UK and overseas. The draft bill clearly outlined a legal duty on British companies to assist in hacking devices (equipment interference warrants).

On the issue of data retention May told the joint committee in January that: “There have been discussions with providers. CSPs have shown me responsiveness on that matter.” However, she avoided giving any detail on how this would work in practice and how much it might cost, saying: “There are no exact figures, I’m happy to provide written evidence of Home Office work in this area.”

The science and technology joint committee report pushed the government for greater clarification when it comes to practical concerns around the retention of this data. The report reads: “There seems still to be confusion about the extent to which ‘internet connection records’ will have to be collected. This in turn is causing concerns about what the new measures will mean for business plans, costs and competitiveness.”

The joint committee has pushed the government to provide: "Whatever technical and financial support is necessary to safeguard the security of the retained data" in its report, but suggests that the government shouldn’t be responsible for 100 percent of the costs.

A spokesperson for UK internet service provider (ISP) BT responded to this obligation by stating: “National security is a critical issue and everyone needs to play their part, including industry. Parliament has long taken the view that the national interest is best served by allowing security and law enforcement authorities access to certain types of data under certain circumstances. We believe there must be a clear legal framework around this regime, one that ensures adequate checks and balances are in place to weigh up any human rights concerns.”

ISP Virgin Media said it “does not monitor or control what customers do online but complies with all lawful requests. It is for Parliament to decide where the balance lies between the needs of law enforcement and citizens’ privacy.’’

ISPs have been cooperating with requests like this since 1984 under obligations outlined in the Telecommunications Act, if requested by the Secretary of State in the interest of national security. This bill looks to write this power into law for the security and intelligence agencies.

The draft bill also outlined a means for ISPs, telecommunications operators and postal operators to receive appropriate contributions to cover the additional costs of these activities.

These providers can appeal requests for data, but only directly to the Secretary of State.


Theresa May has been criticised for her response to the issue of encryption in the bill.

The issue for the security services would be that over the top communications services, like Apple’s iMessage and the popular WhatsApp messaging service, apply end-to-end encryption to all messages, meaning they can’t read them even if they wanted, or were asked to.

In Apple's formal submission to the bill commission the company voiced concern that: "Passages in the bill could give the government the power to demand Apple alters the way its messaging service, iMessage, works" in a way that gives security services the power to eavesdrop on messages, according to The Guardian. Apple CEO Tim Cook has been consistently outspoken in his defense of encryption.

Emails sent using Microsoft Outlook aren’t automatically encrypted in this way and Gmail requires an end-to-end encryption extension for Chrome. Blackberry offers end-to-end encryption between devices through its paid BBM Protected product. The Cisco Spark messaging service has built in end-to-end encryption.


The draft bill specified sensitive professions, namely medical doctors, lawyers, journalists, Members of Parliament and the devolved legislatures, and Ministers of Religion, who will be afforded extra protections under a new code of practice. The safeguards appear to be limited though, with the new judicial authorisation cited, along with the added obligation to ensure that the information being investigated is in the public interest.

In its suggestions for the bill, the joint committee report went further on the issue of privilege, recommending: “That extra protections for privileged and confidential communications should be applied in the same way as is proposed for journalists.”

What next?

The Investigatory Powers Bill has passed the third reading in the House of Lords and will be passed into law by the end of 2016, following final consideration of amendments and Royal Assent.

Find your next job with computerworld UK jobs