IT faces a lot of challenges with email archiving, but the real issue comes down to a difference of opinion, said George Goodall, senior research analyst at Info-Tech Research Group. "Striking the middle ground is very difficult and it makes nobody happy. That's the situation IT always finds itself in, between the rock of corporate counsel and the hard place of user demands," he said.

End users want to keep everything forever because they use their inboxes as knowledge management systems and knowledge stores. Meanwhile, legal counsel and senior management will typically advise IT to not keep anything and delete it immediately, he said.

Goodall's first piece of advice for IT managers developing email archiving policies is to learn how to establish documents from records. "There's a real distinction between what a record is and what a document is, in that records are susceptible to retention schedules and documents aren't. When we talk about minimum retention periods, that applies to records, not documents," he said.

Second, focus on the overall retention schedule for all records because ideally, email is going to be a subset of that, said Goodall. Email should be viewed as part of an overall process on retention schedules, he said, because email itself is more like an envelope or a cover letter. "It isn't necessarily a document type or a record type. It's how you recognise those things internal to it," he said.

Third, focus on the three different motivators applicable to any archiving project, regulation and litigation, IT efficiencies, and worker productivity, and make sure you reconcile them, he said. "Those three factors really have to be balanced. Anytime you are working, let's say, towards worker efficiency but not recognising IT efficiency or the legal implications, there is going to be trouble," he said.

Archiving solutions will focus on the benefits related to litigation concerns, such as searching, auditing, discovery and automated retention periods, said Goodall. But that secondary store for email is another important benefit, he said. "It greatly improves the operational efficiency of a traditional email architecture, because it basically takes all messages coming in and breaks them up into a relational format," he said. This detects duplicate data and multiple versions of the same attachment, he said.

If someone, for example, returns from a honeymoon and emails 30 photos to everyone in the company, everyone's email inbox will have those 30 images, he said. "Email archiving give you the ability to parse all of that... it gives you IT efficiency there," he said.

"A lot of IT managers actually make the case for email archiving, not necessarily from the litigation side, but from the IT efficiency side. It enables them to delay buying storage or improve the efficiency of their Exchange server," he said.

Keep or delete?

In terms of the legal implications of email retention, there are two considerations: regulation and litigation. "From the compliance side, we have to keep things for a minimum period of time and then on the litigation side, we have to basically get rid of things as soon as possible," Goodall said.

The result is a real tension between deciding how long to keep things and what to get rid of, he said. Keeping everything indefinitely "is not necessarily a great strategy" because of concerns such as storage, but "if you keep everything forever, you are covered from a regulation standpoint, he said.

"There are real challenges there because if someone files a motion of discovery... you are going to have to discover that, and if the litigation can prove that you have spoiled information or withheld documentation, that is contempt," he said.

In terms of best practices, what IT managers should keep in mind is the length of time they are forced to archive messages, said Dave Pearson, senior research analyst at IDC Canada. "I think it's safer to assume that you should always have these emails," he said. "You should be looking at a solution where you can keep an indefinite paperless trail of your emails and just assume that you should stay ahead of regulations."

Legal advice

The general perception is that volume is the problem and companies need to have document retention policies, records retention policies and eliminate emails after a certain period of time, said Stephen Maddex, lawyer and associate in the commercial litigation group at Lang Michener. But "organisation is the key," he said.

Before email, businesses "maintained massive databases of records of files in paper and it was never a problem to figure out whether you had documents that responded to a specific litigatio ... because it was organised," he said.

It's not so much that there is a lot of email, but that it is maintained in a "massive, disorganised haystack," he said. "What people figured out was that the value of any given record to litigation wasn't what was written on the record, but it was the value of the cost it would take to remove that needle from the haystack," he said.

Businesses stopped maintaining their records in an organised fashion with email and "that's what created the email problem of discovery," he said.

Another point to keep in mind, he added, is that it is a business decision as to whether or not you keep any files at all. "You could have a policy that says no emails are going to be saved on any server or any device or anything more than 30 days," he said.

If you have a general business policy or organisational structure that purges all emails, then there wouldn't be any obligation to go back and produce all the emails, he said. Exceptions include spoliation (the destruction of evidence) and litigation holds (a stop order).

Developing an email archiving policy is not just a tech issue, said Thomas Sutton, a litigation partner at McCarthy Tétrault. "One of the biggest challenges I see in making sure that the various parts of the enterprise actually do coordinate on this," he said.

Depending on the size of the institution, this could include IT, in-house legal counsel, human resources, corporate security and records management, he said.

Speed and accuracy of retrieval

"The most important thing to realize about email archiving is that it is really all about minimising the costs of retrieval," said Roger Matus, executive vice-president of Safecore.

"Litigation support costs swamp the price of any email archiving system," he said. "If you have to use an older technology to find messages and you are paying someone hundreds of dollars an hour to find them, that very, very quickly overcomes the cost of the system."

IT managers may not realise that they can't hire the lowest-paid member of their staff to review messages, he said. First, you don't want that person reading the email of the CEO. Second, that person might not have the background to be able to make that decision about whether a particular message is relevant for a legal case, he said.

"As soon as you realise those two things, you realise you are going to get a trained litigation support person involved and those people are not cheap," said Matus. "What you want to do is minimise the amount of time that it takes those people to find the relevant messages, so at the end of the day, it's all about the speed and accuracy of retrieval."

Many email archiving solutions are focused on reducing storage costs, but storage is getting cheaper, so the savings vendors show today may not be relevant a year from now, he said. Storage costs drop about 50 per cent a year, whereas litigation costs are going up, said Matus.

Inboxer, an email archiving solution for the mid-sized enterprise market, pre-indexes messages so when a search is performed, the retrieval is quick because the messages are already identified, he said. The solution has identified 80 items that are most frequently searched by users, which include senders and recipients as well as social security numbers and drivers licences, he said.

Records management

Organisations need to set the policies of what needs to be kept and what processes need to be followed, and then IT needs to prove that the process has been followed or the documents have been kept in an immutable way, said Patrick Eitenbichler, director of worldwide product marketing, information management, software at HP.

It's important to have policies in place to capture relevant information, but it may not be necessary to capture all documents from all employees, he said. "If a document was rightfully deleted based on the policy they have set, they are actually much better off from a compliance perspective ... because it is clear they set a policy and they made the best effort to capture the relevant information," he said.

Solutions to check out

Symantec Enterprise Vault is "kind of the product to beat" in terms of market share and functionality, said Goodall. But a lot of document management and storage vendors will have solutions, he said.

Nearly every integrated vendor like HP and IBM has its own solution for Microsoft Exchange or Lotus Notes, said Pearson. But there are also smaller companies like Barracuda, which has an interesting hybrid solution for SMBs, he said.

If you are considering a cloud-based email archiving solution, "the most important aspect and maybe the key differentiator between alternative solutions is in the service level agreements," said Paul Wood of Symantec Hosted Services.

The SaaS model works for any size organisation, said Wood. "A lot of organisations now have reached that tipping point and looking at the same solutions that large Fortune 500-type companies will have... but it will be available to them at a scalable cost using a SaaS model so they don't have to invest huge amounts in their infrastructure in order to provide that solution and that capability.