How IT Managers Can Get The Most Out of Analyst Services Firms

Psst. See that great stack of paper on the corner of your desk? You know the one. It contains all the latest research from your IT analyst firm?the research that costs you hundreds of thousands of dollars per year.

Get up right now, pick up the pile and toss it in the garbage.

Dump it because most of the written material?especially the market projections?is not worth your time.

Why? Because more often than not, it is simply wrong. Remember GartnerGroup’s late ’80s prediction that OS/2?Microsoft and IBM’s jointly developed operating system?would dominate the desktop within five years? Such predictions go wildly awry because typically they are based on little hard data and they try to cover a vast sweep of time. “Anyone who predicts a market will be a certain size five years out, put a muzzle on them. It’s utterly irresponsible,” says Evan Quinn, chief research officer for the Hurwitz Group, an IT analyst firm in Framingham, Mass.

“We used to joke that every market would be $1 billion by the year 2000,” says Scott McCready, who logged nearly 12 years as an analyst at Yankee Group, IDC and Giga before becoming CEO of CIOview, a Boxborough, Mass., software vendor. “Once you come up with some kind of growth rate, all you have to do is compound that over several years and you’re going to come up with a pretty big number. But [analysts] don’t do a good job getting the constraints of the market right. I honestly think [most forecasters] sit around a campfire.”

And some analyst firms are joined around that campfire by vendors. They accept payment from vendors to create forecasts (as well as other types of research). McCready says that during his analyst days, vendor sponsors pressured him to change unfavorable forecasts to their advantage.

All that might seem more benign if the research weren’t so expensive. According to an exclusive CIO and Darwin survey of 200 CIOs, the average company spent $552,600 last year on analyst firms. That’s a helluva lot of money to be spending on information that may turn out to be wrong (to say nothing of the money your company could lose if it relied too heavily on a faulty prediction).

So why spend celestial sums on advice that is less than divine? Because it’s the price of admission to get you in to talk with these people one-on-one. If you’re looking for value from your analysts, you have to pick up the phone. Choose your analyst firm carefully, then cultivate a one-on-one relationship with your designated guru. Get him or her up to speed on your situation, including all maddening intricacies and details, then ask for an opinion. Face-to-face and over the phone, analysts can be enormously valuable.

At a previous job, Ken Kantor (now IS director at musical system manufacturer Peavey Electronics in Meridian, Miss.) saved hundreds of thousands on a computer hardware purchase because a Gartner analyst shared street prices with him over the phone?something he could not have done in a research report. Kantor also got the analyst in on the conference call with the vendor. “The analyst was telling it like it was, right in front of the vendor. They weren’t very pleased with each other by the end of the call, but I was,” Kantor says.

Maybe someday a brilliant analyst firm will stop basing its business model on that useless stack of paper sitting in your office. Instead, your analyst will give you a call every week to share what he or she has heard from vendors and your colleagues out there in the trenches. But until that day comes, here is our list of six essential best practices for dealing with analyst firms?straight from the mouths of CIOs who use them.

1 Manage Your Analyst Company Subscriptions

MOST COMPANIES (ESPECIALLY LARGE ONES) do not have anyone watching their consumption of analyst services. As a result, many different divisions buy the same analyst services, creating redundancy and wasting money. And the analysts aren’t likely to clue you in, says Norma LaRosa, president and cofounder of Corte Madera, Calif.-based Kensington Group, which analyzes the analysts. “CIOs need to put together an enterprisewide contract so they don’t pay double and triple what they actually need and use,” LaRosa says.

But simply knowing what you are buying isn’t enough. Before signing up with any analyst firm, you need to establish exactly who the research consumers in your company are and exactly what they need. “It’s essential to understand the user group that will use the research. You can do this through internal survey work,” says Perry Gartner, director and lead analyst of IT research reports and services for Outsell, an analyze-the-analysts firm in Burlingame, Calif. (His father is Gideon, the progenitor of GartnerGroup and now Giga Information Group.) And one size doesn’t fit all. In one case Perry Gartner knows of, a CIO was so impressed with Forrester Research’s information that he bought subscriptions for his entire IT staff. But the people in the trenches were not the right audience for Forrester’s high-level view, and they soon clamored for more tactical, hands-on advice that wasn’t available from Forrester’s reports.

It’s a good idea to designate someone to manage the analyst subscriptions and disseminate that information throughout the organization. One caveat: This person should be from the group that uses the research (normally IT), not from the corporate library. “If the librarian is the gatekeeper of the information, you’ll get absolutely no value at all,” says David Cearley, who is senior vice president and coresearch director for Meta Group in Stamford, Conn.

2 Find an Analyst Who Knows Your Industry

WHEN CONSIDERING YOUR ANALYST options, there is a huge temptation to buy based on reputation. Resist it. Gartner, Meta, Forrester and the like are considered one-stop shops, but they all have different backgrounds and emphases and may not provide the kind of depth you need in your industry. The antidote to broad generalities: Hire a boutique research firm?there’s bound to be at least one?that covers your industry.

Matt Maynard, CIO at Pathology Associates Medical Laboratories in Spokane, Wash., uses firms, such as York Harbor, Maine-based Nichols Management Group, that concentrate on the medical laboratory industry. “Because we want to get the most value possible from these firms, we want to focus their efforts as much as possible,” says Maynard. Seek names of boutique firms from your peers in your industry.

3 Demand Real-World Experience

SOME ANALYST FIRMS HIRE KIDS right out of college. We’re sure they’re bright as all get out, but wisdom is the child of experience. Don’t be afraid to dig in to the credentials of the analysts who advise you?and ask to be matched with someone who has been a CIO or at least worked in IT. James Lance, senior vice president of IS and CIO at The Elder-Beerman Stores of Dayton, Ohio, appreciates the fact that his Meta Group analyst used to be a CIO in the retail industry. “He has good know-how in dealing with the issues that are foremost in my mind.” When Lance sought advice on an upcoming telecom project, he was thankful his analyst understood the constraints and communications needs of a retail store.

If your analyst should leave his firm and strike out on his own, it is worthwhile to use him for additional consulting. But if you cancel your previous analyst service in favor of a newly independent contractor, watch out: Most analysts sign noncompete agreements that bar them from working for a competitor or “stealing” clients for at least a year.

4 Don’t Be Afraid To Negotiate

BUYING ANALYST SERVICES IS LIKE BUYING an airline ticket?everyone pays a different price, though the analysts don’t like to admit that. Meta’s Cearley says the onus is on the CIO to ask the sales rep (not the analyst) what other pricing packages are available, especially if they are heavy users. “CIOs generally don’t spend enough time [trying to] understand what all the different options are,” says Cearley.

Educate yourself, and don’t be afraid to negotiate, Perry Gartner advises. “It pays to be insistent on what your limits are. It’s very easy to get intimidated. But CIOs need to say, ’My budget is X, my needs are Y, and if you want my business, you have to work with me to configure a service that meets these parameters,” says Perry Gartner. “Analyst firms are experiencing increasing downward pressure on prices for content services.” The CIOs interviewed for this article said they do not attempt to negotiate price with their analyst firms. But it pays to be bold. The client who complains might just be the one who gets the discount.

5 Get Underneath The Reports And Case Studies

“IT’S LESS IMPORTANT WHAT THE ANALYST has written and more important what they will tell you on the phone. The research is the opening salvo. The real relationship starts when you pick up the phone,” McCready says.

Mike Conlon, CIO at the University of Florida Health Science Center in Gainesville, prepares himself for all his analyst phone calls. He does research ahead of time, puts one of his employees on the line with him in case he forgets something and presses analysts hard on technical issues. “They’re supposed to be the experts. Try to smoke them out a little bit,” he advises.

While you’re on the phone, it pays to ask your analyst to dish the dirt on the latest case study he or she wrote. “Get the analyst on the phone and ask for the real scoop,” says Efrem Mallach, CEO and cofounder of the Kensington Group. Analysts gather a lot of dirt that they don’t end up using in the case study. But they can tell you that information on the phone, says Hurwitz Group’s Quinn. You’re looking for the not-for-publication tidbits that could save you hassle and money during your implementation.

6 Don’t Make Your Analyst Stretch Too Far

WHEN CUSTOMERS PRESS ANALYSTS TO REACH beyond their area of expertise, bad things can happen. “There are a few phrases analysts have difficulty speaking: ’I don’t know’ and ’I don’t cover that in depth,’” says Quinn.

McCready recalls once getting into trouble when a large brokerage house asked him to do some custom research that was outside his domain. His recommendation to the brokerage turned out to be wrong, and they were very angry. “Analysts get stretched. They’re being asked to do all these things. If you say, ’I don’t know the answer to that,’ there would be this awful silence. They would be wondering, What else doesn’t he know?” says McCready.

Don’t expect analysts to have flawless insight in every area. Have your B.S. meter on during every exchange with your analyst. Don’t let him gloss over something. Push him for depth, and if you don’t get enough, don’t follow the advice. Seek your information elsewhere.

THE VERY BEST PRACTICE OF ALL is being aggressive in helping your analysts get to know you. Find out how much one-on-one time you’re allowed with your analysts and bump it up if it’s sparse. Get on the phone, listen to their recommendations, then keep your own counsel, using as many different sources of information as possible. Expect all those sources to melt away when accountability time arrives. Bad advice doesn’t qualify you for a refund or exemption from the consequences within your own company. Says Conlon, “I’m the accountable person. They’re advising. I’m deciding. I’m not forced to do what they tell me.”