Haystack - brainchild of Guardian Innovator of the Year Austin Heap - has in less than 24 hours crashed from cause cÃ©lÃ¨bre to epic, life-threatening tragedy. A marketing graduate from a business college, Heap’s positive, naive “Can-Do” attitude and bright-eyed philanthropic spirit would be enough to power a rescue mission. But it takes more than energetic goodwill to solve difficult security problems. A Chaos Computer Club investigator has discovered sadly that Heap and his team’s lack of experience has carried through to the design of Haystack and that this has potentially endangered the lives of Iranian activists.
Haystack was created by Heap and his collaborators in the midst of the 2009 Iranian elections; they worked "72 hours without sleep" to create a tool that would permit ordinary people in Iran to bypass state firewalls and communicate with the outside world, and building on that work Heap sought to formalise their effort into a new tool:
Haystack is a computer program that allows full, uncensored access to the internet even in areas with heavy Internet filtering such as Iran. We use a novel approach to obfuscating traffic that is exceptionally difficult to detect, much less block, but which at the same time allows users to security use normal web browsers and network applications.
He made extraordinarily powerful claims that:
"It's completely secure for the user so the government can't snoop on them. We use many anonymising steps so that identities are masked and it is as safe as possible so people have a safe way to communicate with the world."
And until today the above story, the software’s name and its FAQ provided almost the only details of how Haystack works: a local web-proxy encrypts messages and hides their content amongst other types of web content - a process known as steganography - and sends it to Haystack-owned servers which unpick and decrypt the traffic, re-injecting it into sites such as Twitter and YouTube. The goal of using both encryption and steganography is not only to assure the privacy of the message's content but also to hide the existence of the message at all - for any hint that someone is an activist might be enough to expose them to jail, torture or death.
For all Haystack's talk of complete security its approach to implementation is at odds with with that of the security community at large. The Haystack FAQ says:
Although we sincerely wish we could release Haystack under a free software license, revealing the source code at this time would only aide the authorities in blocking Haystack.
That’s a statement in direct conflict with Kerckhoffs' Principle, a cornerstone of security philosophy. The Principle states that the only security worth doing is that which remains secure even if your enemy knows the totality of how it works. Haystack’s refusal to publish the software is an enormous red-flag to security practitioners, suggesting strongly that some aspect of the security it provides somehow hinges on a parlour trick that - once known - becomes useless or potentially hazardous.
Because US law requires Treasury export licences to be granted to permit the distribution of censorship-circumvention software beyond its borders, Haystack had to apply for the appropriate certificates. Securing them appeared to be a matter of much personal satisfaction for Heap, involving the calling-in of favours in Washington according to an interview at GnomeDex. The reporting of this licence grant seemed almost to give the blessing of the US Government to the project. The project received even more exposure through a BBC television series "The Virtual Revolution" and interviews for The Guardian, culminating in the award of "Innovator of the Year" and immense press coverage.
And then it all turned sour.
Built From Straw
...the ever-modest Heap told Newsweek: "Tomorrow I meet with [Sens. John] McCain, [Bob] Casey, maybe [Carl] Levin, but I don’t know if I will have enough time." (Apparently, the senators have become much more tech-savvy since I left town; perhaps, this comes with age.) And it's not just American media: The Guardian pronounced Heap to be "The Innovator of the Year" -- though personally I would have gone with "The Publicist of the Year,"
Heap responded in kind, and their exchanges continued, Morozov attacking Heap on all fronts - technical, administrative, funding, and public relations. Perhaps this climaxed on September 10th with the curious password-protected post on Heap’s blog, titled “A Conversation With Evgeny” - we don’t know what is in that posting, but it appears coincident with matters coming to a head.
In the late hours of Sunday evening Danny O’Brien of the Committee to Protect Journalists tweeted:
never been angrier than right now. I can't actually describe how broken @haystacknetwork is, because to do so would put people at risk.
O’Brien’s anger was triggered by the fact that, at last, someone with security credentials had been able to examine the code. Jacob Appelbaum of the Chaos Computer Club (himself an individual ) obtained a copy of the client that was circulating “in the wild” (although Morozov claims credit for this). He told us in a phone call that with a small team he was able to break into the code. Rapidly he was able to route traffic through the Haystack network, make connections back to other clients on the Haystack network, and perhaps worst of all he could take the digital fingerprints of the computers used by other people on the Haystack network.
Appelbaum mastered the software so thoroughly that he even got it running under the WINE Windows-emulator for Linux. He wrote:
"Haystack is the worst piece of software I have ever had the displeasure of ripping apart. Charlatans exposed. Media inquiries welcome."
Furthermore, he discovered that even when switched off the software caused behaviour which would identify the user as a potential activist.
Haystack is severely flawed on multiple levels. Probably the worst aspect is that it is easily comprehensively compromised. Its architecture includes a central access point which a moderately competent hacker would be able to quickly locate and master. We saw recently how the Chinese government were apparently able to compromise Google’s secret “Gaia” identity manager and with one strike compromise every account; the exposure in Haystack is just as damaging. A question every sci-fi geek eventually asks is “why would anyone design a self-destruct mechanism into a spaceship - surely that’s where the bad guys will go?”. Above all other flaws, Haystack’s dependence upon central trust is just such a big, red self-destruct button begging to be pushed.
Finally today - the morning of Tuesday September 14th - the project appears to have imploded entirely. After obvious soul-searching surfaced on his Twitter account, Heap’s co-worker Daniel Colascione posted his resignation to the liberationtech mailing list (reposted):
I would like to stress that I am not resigning in shame over the much-maligned test program. It is as bad as Appelbaum makes it out to be. But I maintain that it was a diagnostic tool never intended for dissemination, never mind hype. I did have a solid, reasonable design, and described it in our brief overture of transparency. _That_ is what Haystack would have been. It would have worked!
What I am resigning over is the inability of my organization to operate effectively, maturely, and responsibly. We have been disgraced. I am resigning over dismissing pointed criticism as nonsense. I am resigning over hype trumping security. I am resigning over being misled, and over others being misled in my name.
In the same e-mail Colascione comments that this is the end both of Haystack and the Censorship Research Centre which backed it.
The dust will take weeks, even months to settle. In an interview Appelbaum said that an analysis paper would be published as soon as he could work out how to do it without Iranians being jailed as a result, and cited Colascione as saying that those Iranian activists who have been using the program should “run”.
As Jillian York noted in considerable detail, the lack of analysis from actual experts in the press coverage was breathtakingly irresponsible, culminating in the award from The Guardian. While Heap has fed on the media attention, no-one much has actually asked hard questions like “how do we know this is actually secure?” and “does this enhance risk rather than reduce it?” and the awkward ones like “how much use is an Iranian security manual if you tell the Iranians you have it?”. The media validation has undoubtedly led some trusting souls to actually use this architecturally bankrupt beta-software as if it could actually protect them.
The lesson of all this should be that security is very hard to do properly. It takes more than cryptography, wide-eyed enthusiasm and the confidence that comes from wide praise to actually solve hard security problems and come up with software upon which people can bet their lives. When we find people have opted for media hype instead of proper peer-review or open source we should be concerned.
It’s said that the road to hell is paved with good intentions; let’s hope Heap’s work has not been providing the Iranian government with exactly the snare he intended to disarm.
[This article was written by Alec Muffett of greenlanesecurity.com, with assistance from Simon Phipps]