Finding Value in Your Enterprise Search Options

Who wouldn’t like to outgoogle Google? Given the sophistication of today’s consumer search tools, many CIOs have users (and maybe even the CEO) banging on the door asking why valuable corporate data is still locked away in various places and unsearchable. Trying to capitalize on this need, a bevy of vendors are introducing or revamping enterprise search offerings. Most recently, IBM and Yahoo teamed up to create free, downloadable enterprise search software—the OmniFind Yahoo Edition—to compete with Google’s Mini device for enterprise departments and small to midsize businesses. (Mini search software comes in a preconfigured appliance, starting at $1,995.)

All the flavors of search can be overwhelming. But for enterprises hoping to improve worker efficiency and business processes, it’s vital to understand what the current crop of low-cost, middle and high-end search options can and can’t do.

The free OmniFind Yahoo Edition (designed by IBM to get a foothold in the enterprise search space, with the hope of selling product upgrades and services later) certainly has its limits. But Eric Brierly, CTO at Decision Critical, a company that provides online access to medical training and continuing-education programs for hospitals, nurses and doctors, was able to use it for more than just adding public search capabilities to the company’s website (which is how many entry-level search tools are used). Still, it took some tweaking.

Decision Critical hosts training modules for its customers; each customer has access rights to different modules based on what the customer has licensed or provided. That requires Decision Critical to create and maintain separate Web-based course catalogs for each customer. Brierly has long wanted to simplify the maintenance of course pages and their HTML links to course details, and give clients better keyword searching options. When OmniFind Yahoo came along as a free tool, he decided to see if it might solve his problems.

One key limitation of tools such as OmniFind Yahoo is that they index only HTML pages and common document formats such as Microsoft Word, Excel and PowerPoint documents, and Adobe PDF files. They can’t catalog the contents of databases, ERP systems or other corporate information resources. Brierly extended the free version’s capabilities by creating hyperlinks from the HTML course “start” pages to SQL queries that returned the course details as HTML snippets. Thus, Decision Critical tricked OmniFind Yahoo into indexing its database content. One result: “There are no more broken links, since each link is based on what the search engine actually finds,” Brierly says.

But this approach would not work for other corporate information stores, Brierly acknowledges. That’s just one reason why many enterprises end up using a higher-end tool for mission-critical search needs, says Matt Brown, a principal analyst at Forrester Research.

Bad Search Costs You

For several reasons, CIOs should find search investments easier to justify than they would have a couple of years ago. Despite today’s sophisticated IT efforts, many enterprise managers say simply getting to the right information can be difficult. “Enterprises see Google [search on the Web] and say, ‘I want some of that.’ The search box is now a dominant method of getting access to data outside the enterprise,” says Whit Andrews, a research vice president at Gartner.

An Accenture survey released in January 2007 reveals that U.S. and U.K. managers spend up to two hours each day searching for information, and more than half the information they obtain has no value to them. In addition, 45 percent of respondents say it’s a big challenge to gather information about what other parts of the company are doing. Only 31 percent said that competitor information is hard to get.

Another motivator: Compliance requirements are making business and IT executives seek tools to find information quickly for regulatory filings, government investigations or discovery in legal cases, says Forrester’s Brown. That’s resulted in special-purpose search tools for, say, transaction log files and voice-mail analysis.

But enterprises should be careful not to adopt search in a piecemeal way, Brown advises. “The industry competition has made it difficult for buyers to have a cohesive strategy. For example, Google’s offering is designed to be provisioned by a layperson, so companies end up with a lot of Google appliances churning away at indexing,” he says. (Google’s newest version of the Mini, 2.2, can index up to 50,000 documents in its $1,995 entry-level version, or up to 300,000 documents in its top-of-the-line $8,995 version. IBM OmniFind Yahoo Edition can index up to 500,000 documents.)

Besides wasting network resources, naive deployment of entry-level search tools can both expose private information and hide available documents, says Gartner’s Andrews. That’s because such search engines will scan all servers and documents you point them to, and a server may have data that had been secured by obscurity—no one knew it was there, so it was safe—that is now available in the search engine. And these low-cost engines rarely offer accounts-based access that could restrict access to specific documents based on who’s doing the search, he says.

Conversely, a naive setup could miss some servers with documents that you want to be accessible.

Craft a Cohesive Strategy

A cohesive search strategy does not mean that enterprises need to have a single search platform or a single content index. “It’s OK to have several implementations for different purposes and business units,” Brown says—though the enterprise should map out distinct needs, to ensure that information that should be widely accessible isn’t mistakenly compartmentalized.

For example, it makes sense to have an independent search engine for a company’s public-facing website, to ensure that internal data is not included, and it can make sense to have specialty search engines for log analysis. But it typically doesn’t make sense for sales and customer service to have separate search platforms, since that lets customer information fall through the gaps of the two systems.

That realization led Harris, a communications equipment manufacturing company, to implement several search technologies, each handling distinct tasks, says Janice Lindsay, director of supply chain management. For example, Harris uses a Google search engine to give employees access to documents and intranet HTML pages, so employees can do quick searches on specific information. But when the company decided to use search to help ensure that product engineers and manufacturing staff could find which parts had the best prices and highest quality ratings and which came from preferred suppliers, it developed a search platform using Endeca’s high-end search technology.

That’s because the engineering search needs to direct engineers and others to the best or preferred answers to questions such as “What components meet these engineering requirements?” The results returned are filtered and ranked based on as many as 200 criteria, Lindsay says. The search engine taps into ERP, manufacturing, product design and other systems, as well as into some supplier systems and industry databases to have the context needed for the search engine’s rules to make the recommendations.

The need to integrate with other enterprise systems is a sure pointer toward midrange and high-end search offerings, analysts Brown and Andrews say.

Centralize and Conquer

Many organizations have multiple search technologies in place, but they’re not coordinated. At best, this wastes network and IT resources. At worst, it results in a fractured view of information across the enterprise, compromising product and service quality.

With engineers spread throughout the world, engineering consultancy Arup was concerned that a team working in one office might not know about design approaches being used in other offices, creating uneven quality across locales. So seven years ago, the company brought in a basic search engine. “We immediately drowned with information overload, and people questioned the search results’ validity,” says Tony Sheehan, group knowledge manager.

Arup tried again, this time using the high-end Autonomy search platform. The new system can tap into the company’s databases, financial and human resources systems, and free-form content, either directly or via add-on software. This unified search platform made critical business sense, says Sheehan. The result: Engineers now share the same knowledge no matter where they are based, providing a consistent global level of quality, he says.

National Instruments faced a similar problem. “Our search tools grew over time as the company was learning what search could do,” says Jeff Watts, the instrumentation maker’s former search and communities manager. “With multiple systems, there’s no source of complete information, plus you end up with specialized employees whose knowledge is lost as they leave the company or rotate to other departments,” he says. When the company decided to standardize its Web presence across the globe—providing a single platform that could support multiple languages, local product catalogs and online customer self-service—it also took the opportunity to standardize its internal search platform using a high-end Fast Search & Transfer system.

Law firm Morrison & Foerster knew from the beginning that it wanted a centralized search platform to avoid just such fragmentation, says CIO Jo Haraf. So the thousand-member firm took its time to find a tool that met its needs, rather than deploy interim technologies, she says. The firm ultimately selected midrange solution Recommind because it could do what Haraf calls “gray-area search”—that is, it has the ability to pull in results suggested by, but not explicitly within, the search query—which for a law firm provides a real edge in finding unexpectedly related cases.

Notably, Arup, Harris and National Instruments all realized that they needed to impose their own context and organizational structure to search results to better tune them to their business needs—even though the midrange and high-end systems can infer context as part of their indexing. For example, National Instruments imposed structure and context on its information to help searches across multiple systems more easily find similar information. And Arup imposed its own categories on data—such as projects and people—to ensure that search results would be grouped in the mental baskets used in the company. Watts and Sheehan call the effort difficult but worthwhile.

Take Search Further

What would these search users like to do next with search for their enterprises? Sheehan is considering implementing search actively, not waiting for users to ask questions. The idea: The search engine pulls relevant information into a window as the user types—for instance, providing links to other cantilevered bridge designs if a user is writing a proposal for that kind of structure. “For me, this is the future of search,” he says.

Morrison & Foerster’s Haraf looked at such active search capabilities a few years ago, but rejected the then-current offerings because they interrupted users’ work with alerts. “Users’ initial reaction was, ‘That would drive me crazy; please make it go away,’” she recalls. But a noninvasive approach that suggested relevant cases or subject experts as an attorney was writing a brief or memo? “That would be interesting,” she says.

Galen Gruman is a frequent contributor to CIO. You can reach him at ggruman@zangogroup.com.