The Investigatory Powers Bill - nicknamed the Snoopers' Charter - was agreed upon by both Houses of Parliament and passed into law by Royal Assent on 29 November 2016, making it the Investigatory Powers Act.
However, concerns around the balance between privacy and security have dogged the bill since its inception, and some of its more controversial elements will have to be changed following a successful legal challenge in January 2018.
A UK Court of Appeal ruling found the Data Retention and Investigatory Powers Act (DRIPA) – a previous law covering state surveillance which has been expanded upon with the Investigatory Powers Act of 2016 - was unlawful.
The court ruled that the legislation breached British people's rights by collecting internet activity and phone records and letting public bodies grant themselves access to these personal details with no suspicion of serious crime and no independent sign-off. The latest legislation will have to be "urgently changed" as a result.
The challenge was brought by Labour deputy leader Tom Watson MP, represented by civil liberties charity Liberty.
The High Court then initially agreed with Liberty's crowdfunded legal challenge on April 27. Judges ruled that the provision in the legislation - which allows the government to order private companies to store communications data in bulk, including internet history - breaches the public's right to privacy.
"The court ruled this part of the Act is incompatible with people's fundamental rights because ministers can issue data retention orders without independent review and authorisation – and for reasons which have nothing to do with investigating serious crime," Liberty outlined in a press release.
Martha Spurrier, director of Liberty, said: "Spying on everyone's internet histories and email, text and phone records with no suspicion of serious criminal activity and no basic protections for our rights undermines everything that’s central to our democracy and freedom."
The government now has until 1 November 2018 to amend the legislation and Liberty is challenging the latest Investigatory Powers Act in a separate case in the High Court to be heard later this year.
Spurrier says that this is just a small victory against a law which is "rotten to the core...it still lets the state hack our computers, tablets and phones, hoover up information about who we speak to, where we go, and what we look at online, and collect profiles of individual people even without any suspicion of criminality. Liberty’s challenge to these powers will continue", she said.
The UK's previous use of bulk interception powers was also deemed unlawful by the European Court of Human Rights in September 2018. In its judgement the court deemed the previous regime to have a "lack of oversight of the entire selection process, including the selection of bearers for interception, the selectors and search criteria for filtering intercepted communications, and the selection of material for examination by an analyst."
Fresh amendments to the legislation had been proposed by the government in November 2017, following a European court ruling that said the "general and indiscriminate retention" of personal communications data by police and security services "cannot be considered justified within a democratic society".
The proposed provisions include:
- Requests for personal communications data by the police and other public bodies to be restricted to investigations into crimes that carry a prison sentence of at least six months.
- The introduction of independent authorisation of communications data requests by a new body, known as the Office for Communications Data Authorisations, under the investigatory powers commissioner, Lord Justice Fulford.
- Restricting the use of communications data to investigations into serious crime.
- Additional safeguards which must be taken into account before a data retention notice can be given to a telecommunications or postal operator.
- Clarification of the circumstances in which notification of those whose communications data has been accessed can occur.
- Mandatory guidance on the protection of retained data in line with European data protection standards.
As it stands, the main powers of the law are:
- Web and phone companies (CSPs) will store records of websites visited by every customer for 12 months for access by police, security services and other public bodies upon issue of a warrant.
In practice this would take the form of an itemised list of each citizen's browsing history. This would not be a list of the specific web pages but the main domain (so computerworlduk.com but not the specific stories you read) so a basic online footprint can be drawn up. One concern here will be around the security of this data, especially in the current climate of TalkTalk customer hacks and data dumps.
- Security services will be legally empowered to bug computers and phones upon approval of a warrant. Companies will be legally obliged to assist these operations and bypass encryption where possible (more on this below).
- Security services can acquire and analyse bulk collections of communications data. For example, this could mean a bulk data set such as NHS health records.
- Oversight for these operations will come with a new "double-lock", where any intercept warrants will need ministerial authorisation before being adjudicated by a panel of judges, who will be given power of veto. This panel will be overseen by a single senior judge, the newly created Investigatory Powers Commissioner.
In May 2017, a leaked draft statutory instruments document detailed how the government is seeking to compel telecommunications operators to provide real-time access to named individuals' communications within one working day under the recently passed law. This includes encrypted messages.
The government also asks for the capability to "provide and maintain the capability to simultaneously intercept, or obtain secondary data" from 6,500 people at any one time. For more information visit our sister site Techworld
For some context, figures from the Home Office, as published by The Guardian, show there were 517,236 authorisations in 2014 of requests for communications data from the police and other public bodies and a further 2,765 interception warrants authorised by ministers.
Concerns over the British state's approach towards encryption dogged the government during the bill's passage through Parliament.
The issue for the security services would be that over the top communications providers - like Apple's iMessage and the popular WhatsApp messaging service - apply end-to-end encryption to all messages, meaning they can't read them even if they wanted, or were asked to.
In Apple's formal submission to the Bill Commission, the company voiced concern that: "Passages in the bill could give the government the power to demand Apple alters the way its messaging service, iMessage, works" in a way that gives security services the power to eavesdrop on messages, according to The Guardian. Apple CEO Tim Cook has been consistently outspoken in his defence of encryption.
Emails sent using Microsoft Outlook aren't automatically encrypted in this way and Gmail requires an end-to-end encryption extension for Chrome. Blackberry offers end-to-end encryption between devices through its paid BBM Protected product. The Cisco Spark messaging service has built in end-to-end encryption.
Earl Howe, minister of state for defence and deputy leader in the House of Lords, said following the second reading: "It may be entirely sensible for the government to work with [communication service providers] to determine whether it would be reasonably practicable to take steps to develop and maintain a technical capability to remove encryption that has been applied to communications or data.
"Law enforcement and the intelligence agencies must retain the ability to require telecommunications operators to remove encryption in limited circumstances," he said.
However, Nic Scott, managing director UK and Ireland at enterprise data security specialists Code42, makes the important observation that there are no "half measures" when it comes to encryption. He says: "You either have encryption in place or you don't. Once you create a backdoor for law enforcement purposes, you are also opening the door to other, potentially malicious, parties."
The shadow home secretary Andy Burnham laid out six areas of concern for the Investigatory Powers Bill back in March 2016.
Speaking at the second reading of the bill, Burnham said: "This bill isn't yet good enough. Simply to block this legislation would in my view be irresponsible, it would leave police and security services in limbo. We must give them the tools to do their job. The public interest lies in getting this right and in not sacrificing quality to meet the deadline."
I want to see major changes to #IPBill. Will set out plan later to get them. My judgment is this will achieve more than outright opposition.— Andy Burnham (@andyburnhammp) March 15, 2016
The six concerns were as follows:
1. Privacy: "The Home Secretary said [privacy] was hardwired into the bill, but I see them as more cosmetic changes and haven't directly answered the concerns of the joint committee." Burnham asked that the bill takes a "presumption of privacy".
2. Specific powers: "Internet connection records (ICRs) have been described as the modern equivalent of an itemised phone bill, however the joint committee noted that this is not a helpful description." Burnham went on to explain that there should be a "higher hurdle" for use of this power limited to cases of serious criminal activity rather than any crime. He also asked that the terms "national security" and "economic wellbeing" are defined more explicitly.
3. Internet Connection Records: "Definitions of ICRs (Clause 54) remain vague and I see them becoming more intrusive as technology advances. A stricter definition of what can be included in an ICR should be included. The current confusion is clouding this bill and needs to be clarified."
4. Bulk Powers: "Routine gathering of large quantities of information from ordinary people does lead to privacy concerns and should be as targeted as possible […] It is for the government still to convince the public that these powers are needed." Burnham asked specifically for an independent review to conclude in time for report and third reading on this issue.
5. Judicial oversight: "The government has given significant ground on this issue and the bill is stronger as a result, however we believe it could be stronger still […] I have previously shared concern that this leads to a narrower test looking at only the process and reasonableness of the home secretary's decision, rather than actual merits and substance of the warrant."
May had earlier reassured the house that judicial commissioners will have access to the same information about a warrant as the home secretary. Burnham recognised this, but continued: "If this is the case why not delete the judicial review clause? To make it absolutely clear that this is not just a double lock, but an equal lock."
6. Misuse of the powers: "There needs to be safeguards for the collection of data in a lawful manner and we must also agree that there needs to be an overarching law for the obtaining of data and any use that data is subsequently put to. Both should be a criminal offence."
The Home Office first published its draft investigatory powers bill on November 4, 2015.
Following criticism from three joint committees: the science and technology committee on February 1, the intelligence and security committee on February 9 and most importantly, the joint committee for the bill itself on February 11, Home Secretary Theresa May revised the bill, claiming that it "reflects the majority of the committees' recommendations".
The joint committee for the draft investigatory powers bill made 86 recommendations for changes to the bill in its report, concentrating on issues of clarity, judicial oversight and justification of the various powers.
Addressing these specific concerns, May said of the revised bill: "We have strengthened safeguards, enhanced privacy protections and bolstered oversight arrangements."
Not everyone agreed with this assessment though. Dr. Gus Hosein, executive director of Privacy International said: "It would be shameful to even consider this change cosmetic […] The continued inclusion of powers for bulk interception and bulk equipment interference - hacking by any other name - leaves the right to privacy dangerously undermined and the security of our infrastructure at risk."
The joint committee report
The joint committee for the bill issued its report, along with a list of suggested amendments for the bill, on February 11, 2016. The suggestions include:
- Clarification over the concept of end-to-end encryption: "The Government still needs to make explicit on the face of the bill that Communications Service Providers (CSPs) offering end-to-end encrypted communication or other un-decryptable communication services will not be expected to provide decrypted copies of those communications if it is not practicable for them to do so."
On the issue of encryption the government said the revised bill: "Clarifies the government's position on encryption, making it clear that companies can only be asked to remove encryption that they themselves have applied, and only where it is practicable for them to do so."
- CSPs being forced to retain internet history data of users should be provided with "whatever technical and financial support is necessary to safeguard the security of the retained data" but the government shouldn't be responsible for 100 percent of the costs.
The Don't Spy On Us coalition, which includes the Open Rights Group and Privacy International, responded: "Proposals to collect the internet connection records (ICRs) of every UK citizen could cost more than £1 billion." This figure is based on a similar scheme which has since been dropped in Denmark due to the cost. The initial Home Office estimate for the storage of these records was just £174m over ten years.
- "Fuller justification" for bulk surveillance: "We believe that that the lack of a formal case for bulk personal datasets (BPDs) remains a shortcoming when considering the appropriateness of this power."
- There should be no power to ask foreign intelligence agencies to undertake surveillance where the UK authorities cannot, for example in the USA.
The revised bill "explicitly bans our agencies from asking foreign intelligence agencies to undertake activity on their behalf unless they have a warrant approved by a Secretary of State and Judicial Commissioner."
- Hacking should be targeted: "Targeted interception and targeted equipment interference warrants cannot be used as a way to issue thematic warrants concerning a very large number of people."
- An annual report that must contain: "Information about the impact, results and extent of the use of powers in the bill so effective public and parliamentary scrutiny of the results of the powers can take place."
Writing for Wired.com, Liberal Democrat MP Lord Strasburger, who sat on the joint committee, said: "This bill is a long way from the finished article. It needs more than mere tweaking, it needs to be fundamentally rethought and rebuilt. The Home Office should stop rushing to push it through and take its time to get it right."
The intelligence and security committee criticised the bill for a lack of clarity and transparency around powers and suggested wide ranging amendments to the bill. Mostly though the committee says that the bill could benefit from starting again, saying: "The draft bill adopts a rather piecemeal approach, which lacks clarity and undermines the importance of the safeguards associated with these powers.
"We have therefore recommended that the new legislation contains an entirely new part dedicated to overarching privacy protections, which should form the backbone of the draft legislation around which the exceptional powers are then built. This will ensure that privacy is an integral part of the legislation rather than an add-on."
The science and technology select committee report confronted concerns over the impact the legislation could have on the UK's technology sector, equipment interference powers and a lack of clarity when it comes to the issue of encryption.
Nicola Blackwood MP, chair of the science and technology committee said: "It is vital we get the balance right between protecting our security and the health of our economy. We need our security services to be able to do their job and prevent terrorism, but as legislators we need to be careful not to inadvertently disadvantage the UK's rapidly growing tech sector."
The joint committee on the Draft Investigatory Powers Bill was sent 148 sets of evidence raising concerns and views about aspects of the legislation, and May faced questions over these concerns in front of the committee in December.
There was generally cross-party approval of the bill as first proposed, with Shadow Home Secretary Andy Burnham stating that it was "neither a snooper's charter nor a plan for mass surveillance."
Conservative MP David Davis has been one of the more outspoken critics of the proposed legislation. Talking to The Guardian he said: "In every other country in the world, post-Snowden, people are holding their government's feet to the fire on these issues, but in Britain we idly let this happen […] Because for the past 200 years we haven't had a Stasi or a Gestapo, we are intellectually lazy about it, so it's an uphill battle."
Author and journalist Heather Brooke went one step further. Writing for The Guardian she said: "The spies have gone further than [George Orwell] could have imagined, creating in secret and without democratic authorisation the ultimate panopticon. Now they hope the British public will make it legitimate."
Edward Snowden tweeted: "By my read, #SnoopersCharter [The Draft Investigatory Powers Bill] legitimises mass surveillance. It is the most intrusive and least accountable surveillance regime in the West."
Apple submitted a formal submission to the bill committee, specifically around the issue of encryption, on Monday 21 December, expressing: "We believe it would be wrong to weaken security for hundreds of millions of law-abiding customers so that it will also be weaker for the very few who pose a threat. In this rapidly evolving cyber-threat environment, companies should remain free to implement strong encryption to protect customers," as per The Guardian.
According to YouGov the UK public generally approve of surveillance, with 44 percent of respondents stating it wouldn't bother them to know that they could be spied upon and they don't think they are at this time.
Obligations on communications service providers
The use of investigatory powers relies heavily on the cooperation of so-called 'communications service providers' (CSPs) in the UK and overseas. The draft bill clearly outlined a legal duty on British companies to assist in hacking devices (equipment interference warrants).
On the issue of data retention May told the joint committee in January that: "There have been discussions with providers. CSPs have shown me responsiveness on that matter." However, she avoided giving any detail on how this would work in practice and how much it might cost, saying: "There are no exact figures, I'm happy to provide written evidence of Home Office work in this area."
The science and technology joint committee report pushed the government for greater clarification when it comes to practical concerns around the retention of this data. The report reads: "There seems still to be confusion about the extent to which 'internet connection records' will have to be collected. This in turn is causing concerns about what the new measures will mean for business plans, costs and competitiveness."
The joint committee has pushed the government to provide: "Whatever technical and financial support is necessary to safeguard the security of the retained data" in its report, but suggests that the government shouldn't be responsible for 100 percent of the costs.
A spokesperson for UK internet service provider (ISP) BT responded to this obligation by stating: "National security is a critical issue and everyone needs to play their part, including industry. Parliament has long taken the view that the national interest is best served by allowing security and law enforcement authorities access to certain types of data under certain circumstances. We believe there must be a clear legal framework around this regime, one that ensures adequate checks and balances are in place to weigh up any human rights concerns."
ISP Virgin Media said it "does not monitor or control what customers do online but complies with all lawful requests. It is for Parliament to decide where the balance lies between the needs of law enforcement and citizens' privacy.''
ISPs have been cooperating with requests like this since 1984 under obligations outlined in the Telecommunications Act, if requested by the Secretary of State in the interest of national security. This bill looks to write this power into law for the security and intelligence agencies.
The draft bill also outlined a means for ISPs, telecommunications operators and postal operators to receive appropriate contributions to cover the additional costs of these activities.
These providers can appeal requests for data, but only directly to the Secretary of State.