Why The Motley Fool Chose Enterprise Search Technology to Integrate Content

A key reason IT exists is to get information to people when they want it. At The Motley Fool, that means making any of its 25 million articles, images and discussion threads available quickly when employees or customers look for them. But finding the right technology to do that wasn’t easy.

The classic information-management methods offer various benefits. Data warehouses collect transaction data for easy analysis. Master data management organizes the way data flows through a company so it can be easily located. Content- and document-management systems help people share documents, images and video. But the upkeep required by these approaches can overwhelm employees who have to sort and tag data. The Motley Fool found enterprise search to be the most efficient way to deliver all types of information through automatic indexing of corporate content across disparate applications and databases.

Trying to move all The Motley Fool’s material from content-management systems, databases and other locations into one data warehouse “would be quite a beast,” says Chad Wolfsheimer, vice president of architecture and strategy for the company. “Search is a substitute for migration.”

Because users can type in keywords to find information, enterprise search can more easily help them explore topics across multiple content- and document-management systems, which contain unstructured data that can be difficult to integrate in data warehouses, says Susan Feldman, a research vice president of search and discovery at IDC. (IDC is a sister company to CIO’s publisher.) Enterprise search introduces its own challenges, however, including indexing limitations and inconsistent results.

The Right Solution

The Motley Fool recently settled on the Solr/Lucene search engine from Lucid Imagination, an arm of open-source provider Apache. Solr/Lucene replaced the Google Search Appliance, which The Motley Fool had outgrown, Wolfsheimer says. The company had also rejected three other search engines because they were unpredictable—for example, sometimes omitting new content that should have appeared at the top of search results.

The Motley Fool’s license with Google had capacity limits; by early 2009, Wolfsheimer couldn’t index more content without upgrading. For a while, the company removed some older content from its pool of searchable material to include new items. “But it was a shame for us to have to hide content,” he says. Solr/Lucene doesn’t limit how much can be indexed, Wolfsheimer says. Plus, because it’s open source, his team can freely play with the code to refine the results a search produces. For example, when a customer who pays for a premium subscription searches Fool.com, links to exclusive content bubble to the top of her results.

These changes have increased click-throughs from search results to content by 40 percent, Wolfsheimer says, which shows that more people are finding more content that interests them. “We can help them get what they’re paying for.”

Follow Senior Editor Kim S. Nash on Twitter: @knash99.