CIO — There are times in the high-tech world when it's particularly vexing to figure out just who to believe.

Similar survey data (ostensibly from the same general survey base) can provide contradictory results. Warring vendors bend facts and figures to suit their purposes and unleash FUD. And statistics can be contorted to say just about anything.

Such is the case with two pieces of research and survey data regarding SAP's Solution Manager, an application lifecycle management product that is unsexy as they come though critical to many SAP customers. Or so it would appear.

According to SAP, the Solution Manager "facilitates technical support for distributed systems—with functionality that covers all key aspects of solution deployment, operation and continuous improvement. A centralized, robust application management and administration solution, SAP Solution Manager combines tools, content, and direct access to SAP to increase the reliability of solutions and lower total cost of ownership."

The overuse of the word solution notwithstanding, SAP's "SolMan" product and its new functionalities got warm reviews from an IDC whitepaper published in late 2009. (IDC and CIO.com are both owned by the same parent company, IDG.) The document, by IDC's Elaina Stergiades and Eric Hatcher, was based on interviews IDC conducted with seven SAP customers "to capture their experiences since implementing and using SAP Solution Manager," notes the report.

The results that follow (taken directly from the report) were glowing:

• On average, SAP Solution Manager customers did not have to increase their IT staff sizes to support new activity. This results in an average savings of $393,307 because the customers avoided new hires. Additionally, the current IT staff is 11% more productive since the deployment.
• On average, downtime incidents were reduced by 8.4% and hours per incident were cut by 50.8%.
• The three-year discounted benefit is an average total of $4,489,438 or $126,378 per 1,000 users.
• The results show an aggregate ROI of 463%; the payback period is 4.3 months.
• Results show that customers using SAP Solution Manager in both small and larger numbers of scenarios achieve an ROI over 200% and payback in less than one year.

It probably won't shock you to learn that SAP footed the bill for the research. Clearly written at the top of Page 1 is "Sponsored by: SAP."

Now let's look at a very different set of survey data on Solution Manager and its value to SAP customers, this time from a company named Panaya, which is not a market researcher.

Before I tell you what type of company Panaya is, let's look at the results of their online survey of "347 SAP customers and systems integrators worldwide in April 2010 to learn how SAP customers use SAP Solution Manager and what challenges and benefits they associate with it," notes a Panaya press release.

Continue Reading