Michael Osborne has been getting a lot of vendor calls lately pitching a new breed of products, typically called electronic data discovery (EDD) tools. These tools promise to investigate historical data to uncover security breaches, compliance failures and plain old errors in transactions across various enterprise systems, from network administration to accounting. Driven by compliance requirements such as Sarbanes-Oxley and the Health Insurance Portability and Accountability Act, these tools focus on user activities, such as who accessed a database or updated a customer account. The goal is to look at both real-time and historic patterns across multiple databases, networks and applications to find suspicious activities that might indicate insider financial fraud, customer identity theft, compliance policy breaches or theft of proprietary data such as customer contacts or product designs. As the senior security manager at Kimberly-Clark, which makes health and hygiene products, Osborne is interested in ways to prevent supplier or insider fraud, such as detecting sham providers used to steal or launder money. In other organizations, electronic data discovery tools might be used to detect identity theft or violations of information-access policies.\n\nOsborne is not alone in getting these pitches, say analysts and consultants, who warn that CIOs should be cautious. "There\u2019s a lot of vaporware out there," says Avivah Litan, a security research director at Gartner. "You\u2019re seeing vendors build an industry around scare tactics over compliance and security."That\u2019s not to say there aren\u2019t useful technologies available. For example, Osborne is evaluating a tool from Oversight Systems that analyzes accounting information from SAP and other financial systems to detect fraud and errors both in current transactions and in past transactions stored in the SAP system. He\u2019s recommended that Kimberly-Clark seriously consider adopting the technology.At online shopping service provider 2Checkout.com, Tom Denman, the director of risk management, has adopted 41st Parameter\u2019s analysis tools to detect fraud in the shopping and financial transactions that his service handles for online stores. 2Checkout used to rely on real-time security event monitoring tools but found they couldn\u2019t do as thorough an analysis in real-time. Denman now batches customer transactions and uses 41st Parameter tools to analyze them against previous transactions and various fraud patterns, to detect stolen credit cards and the like (one fraud pattern might be the use of a credit card number for online purchases the same day in several countries). Suspect transactions get flagged for human review, prioritized by risk level.The use of historical data correlated across multiple systems and a focus on user activity is what distinguishes EDD from real-time security event monitoring (SEM) tools, which typically are used to monitor network activity for intrusions and viruses. EDD provides more context in which to find fraud or uncover breaches. "The tools can serve the understand-and-prevent function," says Keith Schwalm, vice president of Good Harbor Consulting, a security advisory firm. EDD tools can work as an adjunct to SEM tools, or provide both functions, notes Amrit Williams, a security research director at Gartner. The vendor trend is to merge the two functions into a suite, he adds.Beware the Forensics LabelMany salespeople attach the label "forensics" to their security and compliance analysis tools, and that can be very misleading. In law enforcement circles, "forensics" means a well-defined set of discovery and investigative processes that hold up in court for civil or criminal proceedings. An enterprise that relies on these tools\u2019 records or analysis in, for example, a wrongful termination suit, is probably in for an unpleasant surprise. "It may not hold up in court," says Schwalm, a former Secret Service agent. "Very few vendors have an idea of what the requirements [are for proof, from a legal perspective]. They\u2019re really providing just a paper trail. You should challenge what the vendor means by \u2018forensics capability,\u2019" he adds.One gotcha of using EDD tools for legal purposes is proving the inviolability of the data. Tools that keep or aggregate event logs may not provide access control that lets the enterprise prove that the underlying data is unaltered and accurate. This issue is particularly critical because most vendors pitch their EDD tools as a way of detecting internal threats. Yet an insider is in the best position to access and alter data to cover his tracks or deflect blame to someone else, making truly secure access control and data management policies a must to even consider relying on EDD tools in a legal case. To thwart insider manipulations, critical functions such as setting up new vendors or changing payment destinations should require multiple levels of approval. "One person shouldn\u2019t be minding the whole store," says 2Checkout\u2019s Denman.A related concern is being able to go back to the original raw data, since most EDD tools alter the original data to put it into a searchable database and to make formats from different types of monitoring appliances consistent. Such regularization is necessary to analyze the records, but to be legally effective, there must be a defensible way to show that it didn\u2019t distort the original data, says Gartner\u2019s Litan.There are no broad standards for what constitutes acceptable forensics. Different courts and law enforcement agencies have their own standards, so the CIO should make sure his security experts consult with those organizations to find out what evidence they\u2019ll require to pursue a case. 2Checkout\u2019s Denman has done just that, working with the FBI\u2019s cybercrime task force "to know what they look for." For example, investigators prefer to make forensically sound copies of original data or the best available evidence; they never manipulate original data directly. CIOs should be sure they don\u2019t approach EDD solely as an IT issue. "Let your general counsel manage this," advises Matt Curtin, founder of the forensic computing consultancy Interhack. An attorney can best decide what records would be needed for legal proceedings. And he can set guidelines on cleansing transaction histories: "The longer you keep the data, the more you have to be subpoenaed," Curtin says, "so you\u2019ll be hit for more [discovery] requests." That increases the chances that the other party will find your own errors and mistakes, he notes.Focus on InvestigationWhile the "forensics" label may be misleading, EDD tools can help the enterprise investigate possible security and compliance breaches to identify where a true forensics investigation should take place or to understand a previous breach as part of an effort to strengthen enterprise defenses.Curtin advises that enterprises consider EDD tools that provide search and query capabilities that in-house analysts can use to uncover clues about potential problems, not just canned detection rules. Having lots of monitoring systems isn\u2019t that useful if you don\u2019t know where to focus your attentions. EDD tools can help identify the problematic areas, "so you don\u2019t bother with the rest of the data," he says. But systems that offer only canned analyses don\u2019t let forensics experts do the kind of digging they need to do, forcing them to go through logs and databases manually. "Most companies today run the rules that come out of the box," notes John Summers, global director of managed security at the Unisys consultancy, but for EDD tools to be effective, "rules need to be specific to your business and processes." Good EDD analysis tools let you both customize the rules and conduct your own queries and searches, Curtin and Summers say.It\u2019s also key to remember that current real-time analysis tools focus on a specific type of monitoring, such as credit card fraud detection or intrusion detection, rather than provide broad, enterprisewide risk analysis.Monitor at Multiple LevelsVendors are increasingly focused on EDD as a way to get CIOs\u2019 compliance money, says Unisys\u2019s Summers. Early EDD tools just added reporting to the real-time event- monitoring capabilities offered by security event management (SEM) tools and appliances, he notes, but since summer 2005, vendors have been adding more "pragmatic" compliance-oriented services to the tools now relabeled as EDD. For example, tools that used to focus on firewall and intrusion detection logs are now examining database logs to monitor access to specific data, both to help assess compliance with data access policies and to identify data access patterns that may indicate fraud. By noticing a firewall breach that occurs 30 seconds before unusual database access, for instance, such tools can alert administrators of a possible identity theft. That might lead to an immediate shutdown of access to that database as well as a deeper look into past activities to see if the identity theft has been ongoing. Similarly, EDD tools are also now examining server logs for both compliance and security analysis, he says.To do truly useful monitoring and analysis of data access requires understanding who the users are and what permissions they have, Summers says, so he expects EDD tools to begin monitoring policy servers and directory services in the next year. That requires a cohesive strategy for compliance and security, one that requires coordinating IT, business, security and legal needs. To accomplish that strategy, the CIO needs to ensure that monitoring and analysis is deployed holistically, not by just the security team or the network administration staff. Effective fraud and compliance monitoring requires having the right policies in place to manage data and access, as well as analyzing ongoing events in the network, in key applications and in key data stores.The new breed of EDD tools are fairly expensive and difficult to deploy, notes Gartner\u2019s Williams. Costs for a large enterprise start at $300,000 and can rise beyond $1 million to deploy, since storage needs can be multiple terabytes and require an information management system. The actual deployment can take up to six months if it involves custom development, which is often the case. Over time, the tools will become more standardized and thus easier to deploy as vendors see broad patterns from the custom deployments, Williams notes. But today, the high costs have limited the tools\u2019 adoption mainly to regulated enterprises or ones where fraud costs more than its prevention, he says. For more on the different EDD tools that are available, go to www.cio.com\/041506.EDD tools can be part of an overall security and compliance effort, but by themselves, EDD tools are barely Band-Aids\u2014unless, of course, you\u2019re just making a pro forma, "cover-your-ass investment," says Gartner\u2019s Litan. That kind of lip-service monitoring and analysis may help you complete a checklist to impress naive shareholders, but it won\u2019t really help your company, says Good Harbor\u2019s Schwalm. After all, as Summers of Unisys notes, "most companies already do logs, but no one looks at them."