Surviving the Data Tsunami

SINGAPORE (12/12/2007) – So voluminous is the data surrounding CIOs that it could almost be likened to a tsunami that will engulf business. In fact an IDC study entitled ‘The Expanding Digital Universe’ estimated that the current size of that universe was 161 billion gigabytes. Moreover, it anticipated that the volume would grow six fold by the end of this decade to 988 billion gigabytes. In effect, that represents a CAGR in global data of 57 percent in over four years.

Much of this growth is represented by the increasing digitization of our lives. We increasingly find ourselves surrounded in our homes by a new generation of devices which output digital files. Organizations are moving away from analog telephony in favor of voice over IP (VoiP). Then there is the growth of e-commerce which is spawning even more digital transactions. The IDC White Paper makes the point that by 2010 much of digital universe will be created by individuals but it argues that organizations will be responsible for the security, privacy, reliability, and compliance of at least 85 percent of it. This is because the data will find its way on to corporate IT systems through Internet downloading and e-mail correspondence.

Increasing legal demands

This challenge is compounded by the increasing legal demands on organizations to not only store information but to have that information accessible. The precedents for these legal responsibilities have arisen from the rulings in a number of high profile cases in the United States. These place the obligations of document ‘discovery’ on business not the plaintiff. All these cases have resulted in million dollar judgements against organizations as prominent as UBS Warburg, Prudential Insurance and Philip Morris because they failed to find documents or else they had deleted them. In all cases the courts concluded that these failings reflected a probable desire to conceal the contents.

Furthermore, these growing data volumes are exacerbating privacy concerns. In fact, 26 percent of CEO respondents to a 2005 McKinsey’s survey identified privacy and security as a threat to the shareholder value of their organizations. CEOs saw how such security breaches could lead to a loss of trust towards their organization. Consumers expect their confidential details to be safeguarded against security threats such as fraud, robbery, and damaging misrepresentations. Again compliance places a responsibility on business to protect their clients against these exposures.

Yet, there is a paradox in these issues. Research reveals that IT security spending in 2005 was at record high levels. Yet, at the same time, 2005 also holds the dubious distinction as the worst year on record for publicly disclosed data breaches. In the US alone, according to Privacyrights.org, there were a total of 130 publicly disclosed data breaches that exposed the private data of more than 55 million American citizens.

Outdated security approach?

Why then are we spending so much on IT security and, apparently, achieving so little. Pundits are claiming that too many CIOs are outdated in their approach to the matter of security. They view the challenge like guarding an old medieval castle where the aim is to keep intruders out. Their focus is on firewalls, access rights and protection from the outside. This may well have worked in the mainframe world of the past. However, in the online world of today IT must let outsiders in. In effect, the IT security challenge is now more like that at an airport. You need to be vigilant with security once people are in the organization.

This requires a CIO to view security more from the context of determining how to protect information. Yet this is easier said than done. Confidential information can be on anything from a mobile phone, a PC or a mainframe and it can be anywhere in the business. It can be in any manner of files. Furthermore, potential security threats are not simple to identify. They could manifest themselves in a lost laptop, a disgruntled employee or a flaw in a firewall. Then there is the fact that staff travel and take confidential data with them.

If this was not hard enough CIOs now have to cater for more and more unstructured data. This is information that lies outside databases such as that generated by applications like e-mail and word processing or in files downloaded from the Internet. This has been estimated to comprise about 60-80 percent of the total data in the average enterprise. Yet this information is very difficult to classify. While much of it should be deleted other parts often contain sensitive data about customers, prospects, staff or corporate strategy. So not only does storing this data haphazardly add costs to the business but also the files contain information that might represent significant security challenges for the organization.

CIOs’ responsibilities with data storage are clearly broadening. Once upon a time all the old EDP Manager had to do was to make sure the data on the mainframe was backed up. Now the CIO has to handle a myriad of data that resides on various machines, in a multitude of applications and in many different file formats. McKinsey in their report ‘A smarter approach to data storage’ recommend that, to ensure effective data security, CIOs need to focus much more on how and where they store files.

Storage menu

McKinsey recommends that, as a starting point, the IT department should create a menu of storage offerings to explain to the end user the various storage options available. This includes the trade-off between costs and capabilities such as speed of recovery, online availability and convenience. McKinsey believe that IT should help the business classify the data in terms of its corporate significance and then assign appropriate levels of storage service to each application. In effect, the task for the CIO is to work with the business to match the most critical applications to the most sophisticated storage offerings available.

IT obviously needs to do better with security. The number of publicly disclosed data breaches is growing and threatens the personal security of millions of people. Yet business is spending more on IT security than ever before. Clearly then, it is about more than money. It appears that effective IT security requires a change in approach. It is more a task of protecting information than physical assets. This entails a cultural change in one’s attitude to IT in relation to security. Many CIOs may feel like the Ancient Mariner marooned among mountains of corporate data. However, as McKinsey shows, they should find rescue if they start first with an appraisal of the sensitivity of the information their company generates and where best that should be stored.

Peter Hind, a freelance consultant with years of experience in the IT industry, co-authored “The IT Manager’s Survival Guide” and has been running enterprise IT executive events for more than a decade.