Compliance: Driving the storage boom?

There can be little doubt hat compliance is a weighty issue for most organisations. Indeed, industry analysts and vendors are quick to credit the current explosion in the storage marketplace to specific compliance legislation – but how much of an impact is it actually having on the current storage boom?

By Kate Clifton

“I was talking to an end user who had been in a situation where he had lost his organisation’s back-up tapes,” says Steve Duplessie, founder and senior analyst at the Enterprise Strategy Group. “When we got talking about the processes in place at the time, he told me that tapes often sat on the counter for up to three days before being taken off site for storage. Can you believe that? It is one big flaw.”

And it is flaws like this – no matter how irrelevant they seem at the time – that can bring even the most carefully planned storage strategy to its knees. At which point those responsible for such processes face not only the embarrassment of getting it wrong, but the threat of non-compliance with current legislation regarding the retention and management of information, documents and business-critical data.

Both the technology and popular press have recently had a heyday with high-profile organisations who have suffered at the hands of lost back-up tapes or missing legally-binding e-mails. Think: Morgan Stanley; Perot Systems; and Enron. Arguably the most well-publicised ‘storage disaster story’ in recent months has been that of the Bank of America, which found itself in hot water after losing back-up tapes containing the personal information – including account and credit card details – of more than one-million US government employees. Not only did the bank face the possibility of losing its contract with the government, but it also had to cope with the public embarrassment caused by the mistake, along with months of press speculation about what actually caused it. Many reports suggested that the data on the tapes was not encrypted correctly, after the bank made a formal admission of the error under California State Law 1386, which requires corporations to report any computing-system security breach where unencrypted personal information is stored.

Such problems have forced storage managers in many organisations – especially the larger ones – to carefully re-consider their data-storage policies and procedures for maximum information security. Although, says AIIM, the enterprise content-management association, there is still a “…negative lottery mentality among a large number of end users related to organisations that have been ‘caught’ in embarrassing legal proceedings involving the mismanagement of electronic information. There is an awareness of vulnerability, but a feeling that ‘this could never happen to us’.*

The storage boom
Despite this, the market for storage and archiving solutions is very healthy.

According to figures released by analyst group IDC in 2005, storage sales for the second quarter of the year were up by nearly ten per cent to $5.6bn. And the data-storage hardware business is estimated to be worth $23bn.

This ‘explosion’ in storage awareness – and purchasing – was evident in Ark Group’s 2005 ‘Information Storage Survey’, which canvassed the opinions of more than 100 records managers and IT managers, to name but a few. A massive 73 per cent of respondents already had, or were planning a storage project at their organisation (see Graph 1). A majority 53 per cent were at the implementation stage, while a further 31 per cent were building the business case. The remaining 15 per cent were shortlisting (ten per cent) or purchasing (five per cent) storage technology. The majority of those that had not yet shortlisted or begun to implement new technology were aiming to do so in the first half of 2006.

Industry analysts and vendors have been quick to associate adherence to key compliance legislation with the apparent boom in the storage industry, particularly with regards to the management of unstructured data found in e-mails and instant messages, for example.

In addition, and as a direct result of increasing regulatory requirements, the demands of the storage environment are changing. Not only do organisations need to ensure the smooth running of their storage systems and practices, they need to be able to demonstrate that they are doing so and that all access to data is audited.

And this, says Mike Berry, vice president of sales at ‘forensic’ e-mail archiving company Cryoserver, will be a major focus for organisations over the next year. “Recent figures show that 56 per cent of all UK organisations approached believe their storage requirements will increase due to the demands of compliance,” he says. “The impact of regulatory compliance within Europe (specifically in the UK), the Middle-East and Africa, will be one of the major drivers of IT optimisation and the management of data within all organisations during 2006.”

For UK organisations, much of this heightened interest in compliance-related storage initiatives can be linked directly to US legislation. The impact, on US multi-national organisations, of the introduction of Sarbanes Oxley in 2002, has filtered across to UK subsidiaries and offices, according to Steve Mackey, director of product marketing at data-storage systems provider ADIC. “The general effect has been to make all IT departments raise their standards in data protection and retention,” he says. He adds, however, that is the general compliance-orientated environment, rather than any one piece of UK or US legislation, that is spurring the growth in storage policies.

So far, the ‘early adopters’ of compliance-driven storage implementations have been, predominantly, larger organisations. But, says Mackey, as corporate awareness spreads and storage technology continues to mature, compliance will have an increasing impact on the rest of the market, as the trend trickles down into small and medium-sized enterprises.

Storage of unstructured data
The biggest area of interest in compliance-orientated storage, and the one that many organisations are wising up to, is that of managing unstructured information: the data that resides in e-mails, in particular. And it is here that the main furore over compliance is evident. Especially since e-mail has become such a business-critical and legally-binding form of transaction and communication.

In its 2006 ‘Industry Watch’ report on the role of enterprise content management in storage decisions, AIIM published survey findings suggesting that the challenge of storage and retrieval of unstructured information is “increasing in mindshare” for a significant number of end users. It reports that three out of ten end users say that 40 per cent of their total storage spend is allocated to the management of unstructured information.

Technology providers are responding with increasingly comprehensive systems that replace more cumbersome methods of e-mail management. Using a back-up tape-storage system for e-mails, says Berry, is no longer enough. Not only is it time consuming and costly to search for and retrieve messages in this way, but the systems themselves cannot provide the high level of precision of archiving that today’s legislation demands. How can an organisation delete e-mails in compliance with data-protection demands, for example, if they are stored off site on tapes.

“Those that have been on the fence regarding networked storage can no longer remain there,” says Duplessie. “Organisations must be in a networked environment or they will never have scale or be able to alter things in fast-enough timeframes.”

When storing and archiving e-mail, organisations must bear in mind the rules imposed by legislation such as: the Regulation of Investigatory Powers Act; the Data Protection Act, the Freedom of Information Act; Basel I; and the US Sarbanes-Oxley Act, and then act accordingly. So, older and less-sophisticated systems are at a severe disadvantage. “They were never designed to address the new regulatory requirements and, subsequently, do not provide information assurance,” says Berry.

And while there has been an influx of new storage products designed specifically for e-mail archiving, vendors, seemingly, could still do more to address compliance and storage. Some 28 per cent of respondents to Ark Group’s ‘Information Storage Survey’ named a lack of understanding of current market offerings as a chief obstacle to purchasing or updating information-storage technology. While one respondent, an information scientist within the construction industry, went one step further saying, “The products just don’t do exactly what we need them to, yet.”

Already, there has been movement within storage vendor circles to address such problems. For ADIC, this means addressing the key end-user requirements of rapid data growth, product reliability, ease of management and flexibility, says Mackey.

Also, EMC and AXS-One represent one of a growing number of strategic vendor partnerships in the storage and archiving space. Archiving itself looks set to become one of the main featureS of the evolving storage requirements of organisations, according to Duplessie. “Archiving is interesting because it originally started as a way to groom primary data and pull it out of the back-up stream,” he says. “Now, it’s turned into a records-management business based on finding and retrieving information and data once it has been stored. The market is still in its infancy, but we think it’s going to go crazy.”

Other key drivers
Take away issues surrounding the storage of unstructured data, however, and the impact of compliance on the storage boom becomes more debatable. In Ark Group’s survey, when respondents were asked outright what they were looking to achieve with their storage initiatives, the results were surprising, especially considering all of the hype surrounding compliance and information management. Most (37 per cent) stated that they were looking to improve storage-process efficiency and 11 per cent were seeking to lower overall costs. Just 14 per cent were motivated by compliance (see Graph 2)

This, according to Duplessie, could be due in part to people’s perception (or lack of) and reaction to regulations. “Compliance, first and foremost, is the easiest part of compliance,” he says. “Once you know exactly what you have to do, you make sure that you do it. The real issues with compliance are with corporate governance and best practice – and this is where you can get into all kinds of trouble, as people will always have different definitions of how things should be done,” he says. Respondents to Ark Group’s 2004 ‘Compliance Survey’ also cited overlapping and conflicting legislation (52 per cent) as the main obstacle to compliance initiatives, so Duplessie’s opinion does seem to ring true. If people are not fully aware of, or misunderstand, compliance it is not going to feature significantly in their planning for storage, let alone be a key driver.

Duplessie even goes so far as to say that, despite the vendor excitement and high-profile press coverage, compliance is not one of the main drivers in storage implementations.

This, says Duplessie, is because e-mail – unstructured information – represents a smaller portion of the data growth, which is being experienced by many organisations, than reference data – non-transational, fixed content. There is an economic paradigm that we should ‘keep everything we do’ and that ‘if it was worth creating, then it is worth keeping’. Here, storage becomes more of an enterprise content-management issue, rather than one relating directly to compliance. “I would say that around 95 per cent of the growth that we’re seeing is in reference data, not compliance-oriented records management and storage. Compliance gets all the buzz, but actually represents a small fraction of the growth within this industry.”

The sheer volume of data within organisations cannot be ignored when examining storage drivers. “To compound, this organisations are now retaining information for longer periods,” says Mackey. “Very few companies are currently able to classify data effectively and, as a result, discriminate with regard to retention.” So, a revision and update of storage systems is a must to keep on top of the information deluge. And this has an obvious effect on the market for storage solutions.

Mackey also points to an interrelation between data protection and security, which he believes remain at the top-end of storage management wish-lists. Security is obviously important, particularly when those high-profile storage failures are taken into consideration. And while compliance can be interwoven with security to some extent, in the sense that it clarifies processes and workflows, security is a driver in itself.

Cost, says Duplessie, has also plummeted by around 90 per cent in the past five years. “Storage – and physically acquiring it – is no longer an economic issue, it’s a management issue,” he says. And the trick is to be able to match the appropriate costed storage to the priority of the information being held.

Compliance is a huge – if slightly over-hyped – factor in storage management.. Organisations must adhere to regulations and, for many, the easiest option is to approach a vendor and invest in a storage product.

But, cost, security and process efficiency are all valid incentives to improve upon storage programmes, too.

Duplessie goes on further about incentives for storage strategies. While compliance – whether as a key driver or one of many – may be encouraging storage spending, it is madness to think that a storage solution will make all your problems go away, he says. It is the age-old theory that compliance is as much about process as it is about technology. “If people think that they can buy a ‘magic box’, that is just not the case – and this isn’t emphasised nearly enough,” he says. Managers should be advised to sit at whiteboards with no technology whatsoever and understand the workflow of their roles and the requirements of the technology – how it relates to the necessity to retain and retrieve information – before making any decisions, he recommends.

In the meantime, while scientists have commented that we only use around ten per cent of our mental potential, Duplessie predicts that organisations have so far stored only a small fraction of the information that is required, and over the next two years, any work regarding storage will be double that of what organisations have done previously.

References
* AIIM Industry Watch Executive Summary: ‘The Role of ECM in Storage Decisions: The Why, What, and How of Storing Business Critical Information’, 2006.