Market report: Enterprise search

During this year it is estimated by IDC that 0.988 Zettabytes (1,000,000,000,000,000,000,000 bytes) of data will be produced globally. That’s enough bits and bytes to fill two stacks of paperback books piled between the sun and Pluto.

And this content explosion is increasing fast, according to Whit Andrews, vice president and distinguished analyst at Gartner: “Anecdotal evidence implies that enterprises are facing between 50 and 100 per cent growth per year in the amount of content that they are storing. Additionally, format types are multiplying, as users face audio, video and graphical documents as an increasingly important aspect of their work.”

Andrews predicts that content growth will continue to accelerate, and at the same time users will increasingly expect to have “Google-class access” to multiple content types, both inside and outside of the enterprise.

The scale and complexity of this information overload facing companies has helped the enterprise search industry to enjoy “relatively strong growth” during the recent downturn, says Sue Feldman, IDC’s vice president for Search and Discovery Technologies.

She says: “Vendors report that sales are strong in some key areas such as ecommerce, eDiscovery, and sentiment analysis.”

A new report published by the Joint Research Centre of the European Commission’s Institute for Prospective Technological Studies estimates that the enterprise search solution market, was worth 831 million Euros in 2008 and has the potential to grow between 10 per cent and 15 per cent every year until 2013.

This enterprise search market, which includes IBM (after its acquisition of DataMirror in 2007), Autonomy, Microsoft (after its purchase of Fast Search and Transfer in 2008), Google and Endeca, is in a consolidation phase with “significant market restructurings” Gartner warns.

However Gartner’s Andrews told CIO that this consolidation at a corporate level has not yet resulted in major upheavals in the marketplace.

“In the last few years, relatively few search technologies have precipitously disappeared due to acquisition. Autonomy, in fact, is still supporting some Verity K2 installations five years after the purchase. Providers who have disappeared are more likely to cease support than those who have been acquired. Possible defences include using open source or another technology type,” he advises.

This view was echoed by Paul O’Hagan, industry solutions development manager, IBM. He said that open standards such as OASIS Unstructured Information Management Architecture (UIMA), which was approved last year, and open source projects such as Apache Lucene are key technology drivers in the market.

In addition there will be “an increasing emphasis on social search such as leveraging awareness of highly connected individuals and knowledge experts to drive relevancy ranking algorithms,” O’Hagan forecasts.

Dale Vile, managing director of analyst company Freeform Dynamics, agrees that social media- environments are starting to be taken seriously by companies.

He says: “In this Enterprise 2.0 internal social media world firms need technology that can not only find things, but distinguish the value of the data that is delivered.”

But Simon Price, European director of enterprise search company Recommind, warns that search technology must keep up with the potential Web 2.0 security issues.

He says: “Search isn’t just about enabling employees to find the information they require easily and efficiently, it’s also just as important to ensure that data doesn’t fall into the wrong hands, both internally and externally.”

More widespread adoption of cloud computing is also likely to impact the sector.

IBM’s O’Hagan believes that “over time, as security concerns with proprietary corporate information are effectively addressed it’s expected the trend to cloud computing will accelerate for corporate search.”

Looking to the future, Vile predicts that the enterprise search will become increasingly commoditised.

He says: “Unless there is a legal or regulatory driver companies are increasing viewing search not in isolation, but rather as information management. Search is not an application, but a capability embedded in an application. It does not live in isolation.”

Pic:miragelincc2.0