(The first part of this three-part essay appeared yesterday.)
The gradual but relentless shift from piecemeal, small-scale analogue eavesdropping to constant and total surveillance may also help to explain the public's relative equanimity in the face of these revelations. Once we get beyond the facile idea that if you have nothing to hide, you have nothing to fear – everybody has something to hide, even if it is only the private moments in their lives – there is another common explanation that people offer as to why they are not particularly worried about the activities of the NSA and GCHQ. This is that "nobody would be interested" in what they are up to, and so they are confident that they have not been harmed by the storage and analysis of the Internet data.
This is based on a fundamentally analogue view of what is going on. These people are surely right that no spy is sitting at a keyboard reading their emails or Facebook posts. That's clearly not possible, even if the will were there. But it's not necessary, since the data can be "read" by tireless programs that extract key information at an accelerating pace and diminishing cost thanks to Moore's Law.
People are untroubled by this because most of them can't imagine what today's top computers can do with their data, and think again in analogue terms – the spy sifting slowly through so much information as to be swamped. And that's quite understandable, since even computer experts struggle to keep up with the pace of development, and to appreciate the ramifications.
A post on the Google Search blog from last year may help to provide some sense of just how powerful today's systems are:
When you enter a single query in the Google search box, or just speak it to your phone, you set in motion as much computing as it took to send Neil Armstrong and eleven other astronauts to the moon. Not just the actual flights, but all the computing done throughout the planning and execution of the 11-year, 17 mission Apollo program. That's how much computing has advanced.
Now add in the fact that three billion Google queries are entered each day, and that the NSA's computing capability is probably vastly greater than Google's, and you have some idea of the raw power available for the analysis of the "trivial" data gathered about all of us, and how that might lead to very non-trivial knowledge about our most intimate lives.
In terms of how much information can be held, a former NSA technical director, William Binney, estimates that one NSA data centre currently being built in Utah will be able to handle and process five zettabytes of data – that's five million million gigabytes. If you were to print out that information as paper documents, and store them in traditional filing cabinets, it would require around 42 million million cabinets occupying 17 million square kilometres of floor space.
Neither computing power nor the vast holdings of personal data on their own are a direct threat to our privacy and freedom But putting them together means that the NSA can not only find anything in those 42 million million virtual cabinets more or less instantly, but that it can cross-reference any word on any piece of paper in any cabinet – something that can't even be contemplated as an option for human operators, let alone attempted.
It is this unprecedented ability to consolidate all the data about us, along with the data of our family, friends and acquaintances, and their family, friends and acquaintances (and sometimes even the acquaintances of our acquaintances' acquaintances) that creates the depth of knowledge the NSA has at its disposal whenever it wants it. And while it is unlikely to call up that knowledge for most of us, it only takes a tiny anomalous event somewhere deep in the chain of acquaintance for a suspicion to propagate back through the links to taint all our innocent records, and to cause them to be added to the huge pile of data that will cross-referenced and sifted and analysed in the search for significant patterns so deep that we are unlikely to be aware of them.
Given this understandable, if regrettable, incomprehension on the part of the public about the extraordinary power at the disposal of the NSA, and what it might be able to extract as a result, the key question then becomes: what can we do to bolster our privacy? Until a few weeks ago, most people working in this field would have said "encrypt everything". But the recent revelations that the NSA and GCHQ have succeeded in subverting just about every encryption system that is widely used online seem to destroy even that last hope.
(In tomorrow's instalment: the way forward.)