The White House has settled lawsuits from advocacy groups over millions of missing emails, the legacy of what may have been one of messiest email platform migrations ever.
The two groups, the National Security Archive and Citizens for Responsibility and Ethics in Washington, sued the Executive Office of the President in 2007 claiming that millions of emails were missing, based on information they had received and their own research.
The lawsuit wanted the White House to account for all the missing emails, not only because the law required it, but because of their importance to history.
The settlement (PDF document) with the White House, announced today by the groups that brought the suit, sets the terms for hunting for additional White House emails. The White House will investigate records from an additional 94 days.
More than 22 million emails have been recovered and more are likely waiting to be found, said Meredith Fuchs, the archive's general counsel. But "I don't think we will ever know whether everything was restored," Fuchs said.
The settlement also requires the White House provide a publicly releasable letter "describing in as much detail as possible" the current Executive Office of the President computer systems, including its email, archiving and backup systems. "This document will include a detailed description of the controls in the system that prevents the unauthorised deletion of records," the settlement says.
That letter is due by 15 January.
"We actually want them to explain what their system is," Fuchs said. But she said the White House, because of security concerns, is likely to focus on explaining "certain characteristics" of the systems and not necessarily the technologies underpinning them.
The email problem began in 2002 and 2003 after the White House moved from Lotus Notes to Microsoft Exchange. As it moved to the new platform, the President's IT staff also discontinued use of legacy, circa 1994, electronic management and archiving system, called Automated Records Management Systems (ARMS). Development began on a new archiving system that ran into its own issues and wasn't implemented.
Without an automated archiving system, the White House relied on manual processes to archive emails, and that's when the problems evidently began. Files were mislabelled and commingled on backup tapes containing all types of information.
Among the techniques used by the White House to determine whether emails were missing was modelling of traffic, from which they founds days that had statistically low email volumes.