E-Mail Archiving (KeyStore)
Drivers to Archive
The following list from a survey by Osterman Research highlights the reasons companies gave for keeping email:
- Protecting against lawsuits, intellectual property, from wrongful dismissal claims
- Keeping inappropriate email out of the organisation
- Keeping hate literature our of the organisation
- Defense against accusations of criminal acts
- Keeping employees from being harassed via email
Three basic requirements are commonly expressed when email administrators are asked about their needs for archiving. These being:
- To aid the organisation in meeting legal requirements (Compliance)
- To improve system performance (Capacity)
- To manage the retention of corporate information (e-Policy)
Compliance
The need for ‘compliance’ is driven by various governmental and regulatory demands. The high profile acts of today include SEC, Sarbanes-Oxley and Basel II which were primarily driven by experiences of email mismanagement.
The UK and US Freedom of Information Act laws have increased the visibility of email retention and accessibility during 2005.
Legislation commonly calls for retention periods but may demand deletion following expiration of the retention period. The requirement is usually to copy away all emails relating to subjects, departments or individuals before a user has a chance to manipulate or delete the information, providing a fully secure and audited record of email activity.
System performance and selective retention have nothing to do with compliance; a solution to aid compliance is generally working behind the scenes, invisible to the end-user and with the archived copies accessible only by certain permitted Officers.
Regulations are requiring various industries to store electronic information for a period of time. These new standards are pushing the need to archive.
Typical regulations force organisations to:
- Keep copies of all emails (selected by individual or department)
- Keep copies of all email transactions with third parties
- Maintain copies of the electronic calendars of key members of staff
- Save messages in a secure format, able to be retrieved as and when they are needed
Non-compliance with regulations is serious. In December 2002, The Securities and Exchange Commission, the New York Stock Exchange and NASD fined five firms a total of $8.25 million for failure to preserve email communications.
Each of the firms — Deutsche Bank Securities Inc.; Goldman, Sachs & Co.; Morgan Stanley & Co. Incorporated; Salomon Smith Barney Inc.; and U.S. Bancorp Piper Jaffray Inc. — consented (without admitting or denying the allegations) to findings that each failed to preserve for a period of three years, and/or preserve in an accessible place for two years, electronic communications relating to the business of the firm, including interoffice memoranda and communications.
To meet regulatory requirements, the key is to find an archiving solution that maintains email integrity.
DoD 5015.2-STD, for example, requires that any record (including email), when retrieved, can be reproduced, viewed, and manipulated in the same manner as the original. When it comes time for regulatory audits, you won’t want emails challenged for lack of authentication.
This is one of the main reasons why back-up of email isn’t enough to meet regulatory requirements.
The fast indexing and search for retrieval of email is inherent to true archiving solutions. When you need to track down email, you’ll no doubt need to search millions of messages and their contents in a restricted timeframe.
Back-up just doesn’t allow for this to happen – true archiving solutions are built for the writing away and retrieval of high volumes of email, maintaining full data integrity and audit trails which would stand up in a court of law.
Another point to remember is that searching and retrieving messages within a prescribed time-frame is virtually impossibly to do manually; when the requirement is to retrieve an email out of millions within (say) 48 hours, this does not mean “give the request to the IT department and they must present the data within 48 hours”.
This almost certainly means “your company has 48 hours in which to present the data”, so you need to get the data to the lawyer who probably needs to set it out in the context of the case and to present that within 48 hours. Realistically, the IT dept probably needs to find the data within an hour!
This implies the need for a fully flexible, well managed system. When you look at compliance you will need to bear in mind:
- The regulatory reasons for compliance
- Other legal factors pertaining to data retention
- Whether the data is tamper-proof
- Methods of sampling and review
- Log & audit trails of archive searches – this may involve a review hierarchy of IT, Security and / or Compliance Officers
- The abilities of the company to manage this data
- You may need to prove that you have undertaken all of these and more
- You will need to involve all aspects of management to ensure that the compliance project is not just left to IT, it is an organisation wide activity
Capacity / System Performance
Experience tells us that message volumes and message sizes are rising rapidly. Some types of companies whose focus is sending and receiving large reports (notably marketing, finance type organisations) tend to show high growth in terms of message attachment size, others more simply find that an email based conversation is taking over from the ‘phone as the preferred method of business communication.
The resultant increase in traffic and storage volumes can adversely affect email systems and infrastructures that just weren’t built for the increases we’ve all encountered. The impact of mailbox and information store size on system performance and user productivity is high.
Large information stores will impact the backup/ restore times of the system, potentially impacting the business if failures were to occur at key business hours. A reaction is often to reduce mailbox size by introducing quotas, but this is not necessarily a good thing to do in terms of the organisation (see next section).
Therefore administrators, charged with providing high availability servers and giving users access to their stored data, are turning to archiving solutions. There are a variety of solutions on the market, ranging in terms of cost, complexity, ease of use and manageability.
User invisibility is often a priority: busy administrators don’t have time to train users on a new system. Archiving for capacity management quite simply uses policy-driven central rules to keep critical data locally and archive off older data to a secondary store or other storage media. This means that the performance and availability of email which is critical to the business can be maintained within SLAs, while older (less critical) email may be stored externally and may have a longer agreed back-up or restore window.
If the need is to make drastic reductions to the volume or cost of storage used, then it is appropriate to tackle this but keep in mind that the two main factors affecting archiving project’s success must be balanced: Cost of the storage media used and accessibility or retrieval time of the email to the user.
Impact of Mailbox Quotas & PSTs
Users’ demands to increase their mailbox sizes give IT departments challenges as storage demands become expensive resulting in mailbox quotas.
User reaction to hitting mailbox quotas is alarming.
The following list from a survey by Osterman Research shows how users cope:
- I delete email from my inbox and/or folders
- I create one or more personal archives
- I complain to IT
- I expand my mailbox
- I delete all “sent” email with attachments
If users have to delete email, but they need to retain corporate knowledge to do their job, it follows that they will spend longer and longer trying to find emails.
Estimated lost productivity per employer is 30-40 minutes per week,which amounts to over 5 days per annum and thousands of pounds of lost productivity.
The usual solution is to create a personal archive, known as a PST. However the problems of PSTs are now well known, with major issues such as size limitations and invisibility to the Administrator.
If PSTs are unknown, then they cannot be searched without special software and the ability for a company to limit risk is in jeopardy.
A decision needs to be taken as to whether to:
- allow users to delete emails (losing vital information)
- allow them to create PSTs (creating legal exposure to the company)
- to archive the data in the most suitable way
The problem of mailbox quotas can be resolved by introducing systems that archive from the information store onto a range of different storage media.
This storage can range from a secondary Exchange system (easy to implement with an immediate benefit in performance) to off-line or near-line systems where retrieval is likely to be slower, but the storage costs are almost certainly lower.
