Psst. See that great stack of paper on the corner of your desk? You know the one. It contains all the latest research from your IT analyst firm?the research that costs you hundreds of thousands of dollars per year.Get up right now, pick up the pile and toss it in the garbage.Dump it because most of the written material?especially the market projections?is not worth your time.Why? Because more often than not, it is simply wrong. Remember GartnerGroup\u2019s late \u201980s prediction that OS\/2?Microsoft and IBM\u2019s jointly developed operating system?would dominate the desktop within five years? Such predictions go wildly awry because typically they are based on little hard data and they try to cover a vast sweep of time. "Anyone who predicts a market will be a certain size five years out, put a muzzle on them. It\u2019s utterly irresponsible," says Evan Quinn, chief research officer for the Hurwitz Group, an IT analyst firm in Framingham, Mass."We used to joke that every market would be $1 billion by the year 2000," says Scott McCready, who logged nearly 12 years as an analyst at Yankee Group, IDC and Giga before becoming CEO of CIOview, a Boxborough, Mass., software vendor. "Once you come up with some kind of growth rate, all you have to do is compound that over several years and you\u2019re going to come up with a pretty big number. But [analysts] don\u2019t do a good job getting the constraints of the market right. I honestly think [most forecasters] sit around a campfire." And some analyst firms are joined around that campfire by vendors. They accept payment from vendors to create forecasts (as well as other types of research). McCready says that during his analyst days, vendor sponsors pressured him to change unfavorable forecasts to their advantage.All that might seem more benign if the research weren\u2019t so expensive. According to an exclusive CIO and Darwin survey of 200 CIOs, the average company spent $552,600 last year on analyst firms. That\u2019s a helluva lot of money to be spending on information that may turn out to be wrong (to say nothing of the money your company could lose if it relied too heavily on a faulty prediction).So why spend celestial sums on advice that is less than divine? Because it\u2019s the price of admission to get you in to talk with these people one-on-one. If you\u2019re looking for value from your analysts, you have to pick up the phone. Choose your analyst firm carefully, then cultivate a one-on-one relationship with your designated guru. Get him or her up to speed on your situation, including all maddening intricacies and details, then ask for an opinion. Face-to-face and over the phone, analysts can be enormously valuable.At a previous job, Ken Kantor (now IS director at musical system manufacturer Peavey Electronics in Meridian, Miss.) saved hundreds of thousands on a computer hardware purchase because a Gartner analyst shared street prices with him over the phone?something he could not have done in a research report. Kantor also got the analyst in on the conference call with the vendor. "The analyst was telling it like it was, right in front of the vendor. They weren\u2019t very pleased with each other by the end of the call, but I was," Kantor says.Maybe someday a brilliant analyst firm will stop basing its business model on that useless stack of paper sitting in your office. Instead, your analyst will give you a call every week to share what he or she has heard from vendors and your colleagues out there in the trenches. But until that day comes, here is our list of six essential best practices for dealing with analyst firms?straight from the mouths of CIOs who use them.1 Manage Your Analyst Company SubscriptionsMOST COMPANIES (ESPECIALLY LARGE ONES) do not have anyone watching their consumption of analyst services. As a result, many different divisions buy the same analyst services, creating redundancy and wasting money. And the analysts aren\u2019t likely to clue you in, says Norma LaRosa, president and cofounder of Corte Madera, Calif.-based Kensington Group, which analyzes the analysts. "CIOs need to put together an enterprisewide contract so they don\u2019t pay double and triple what they actually need and use," LaRosa says.But simply knowing what you are buying isn\u2019t enough. Before signing up with any analyst firm, you need to establish exactly who the research consumers in your company are and exactly what they need. "It\u2019s essential to understand the user group that will use the research. You can do this through internal survey work," says Perry Gartner, director and lead analyst of IT research reports and services for Outsell, an analyze-the-analysts firm in Burlingame, Calif. (His father is Gideon, the progenitor of GartnerGroup and now Giga Information Group.) And one size doesn\u2019t fit all. In one case Perry Gartner knows of, a CIO was so impressed with Forrester Research\u2019s information that he bought subscriptions for his entire IT staff. But the people in the trenches were not the right audience for Forrester\u2019s high-level view, and they soon clamored for more tactical, hands-on advice that wasn\u2019t available from Forrester\u2019s reports.\nIt\u2019s a good idea to designate someone to manage the analyst subscriptions and disseminate that information throughout the organization. One caveat: This person should be from the group that uses the research (normally IT), not from the corporate library. "If the librarian is the gatekeeper of the information, you\u2019ll get absolutely no value at all," says David Cearley, who is senior vice president and coresearch director for Meta Group in Stamford, Conn.\n2 Find an Analyst Who Knows Your Industry\nWHEN CONSIDERING YOUR ANALYST options, there is a huge temptation to buy based on reputation. Resist it. Gartner, Meta, Forrester and the like are considered one-stop shops, but they all have different backgrounds and emphases and may not provide the kind of depth you need in your industry. The antidote to broad generalities: Hire a boutique research firm?there\u2019s bound to be at least one?that covers your industry.\nMatt Maynard, CIO at Pathology Associates Medical Laboratories in Spokane, Wash., uses firms, such as York Harbor, Maine-based Nichols Management Group, that concentrate on the medical laboratory industry. "Because we want to get the most value possible from these firms, we want to focus their efforts as much as possible," says Maynard. Seek names of boutique firms from your peers in your industry.\n3 Demand Real-World Experience\nSOME ANALYST FIRMS HIRE KIDS right out of college. We\u2019re sure they\u2019re bright as all get out, but wisdom is the child of experience. Don\u2019t be afraid to dig in to the credentials of the analysts who advise you?and ask to be matched with someone who has been a CIO or at least worked in IT. James Lance, senior vice president of IS and CIO at The Elder-Beerman Stores of Dayton, Ohio, appreciates the fact that his Meta Group analyst used to be a CIO in the retail industry. "He has good know-how in dealing with the issues that are foremost in my mind." When Lance sought advice on an upcoming telecom project, he was thankful his analyst understood the constraints and communications needs of a retail store. \nIf your analyst should leave his firm and strike out on his own, it is worthwhile to use him for additional consulting. But if you cancel your previous analyst service in favor of a newly independent contractor, watch out: Most analysts sign noncompete agreements that bar them from working for a competitor or "stealing" clients for at least a year.\n4 Don\u2019t Be Afraid To Negotiate\nBUYING ANALYST SERVICES IS LIKE BUYING an airline ticket?everyone pays a different price, though the analysts don\u2019t like to admit that. Meta\u2019s Cearley says the onus is on the CIO to ask the sales rep (not the analyst) what other pricing packages are available, especially if they are heavy users. "CIOs generally don\u2019t spend enough time [trying to] understand what all the different options are," says Cearley. \nEducate yourself, and don\u2019t be afraid to negotiate, Perry Gartner advises. "It pays to be insistent on what your limits are. It\u2019s very easy to get intimidated. But CIOs need to say, \u2019My budget is X, my needs are Y, and if you want my business, you have to work with me to configure a service that meets these parameters," says Perry Gartner. "Analyst firms are experiencing increasing downward pressure on prices for content services." The CIOs interviewed for this article said they do not attempt to negotiate price with their analyst firms. But it pays to be bold. The client who complains might just be the one who gets the discount.\n5 Get Underneath The Reports And Case Studies\n"IT\u2019S LESS IMPORTANT WHAT THE ANALYST has written and more important what they will tell you on the phone. The research is the opening salvo. The real relationship starts when you pick up the phone," McCready says.\nMike Conlon, CIO at the University of Florida Health Science Center in Gainesville, prepares himself for all his analyst phone calls. He does research ahead of time, puts one of his employees on the line with him in case he forgets something and presses analysts hard on technical issues. "They\u2019re supposed to be the experts. Try to smoke them out a little bit," he advises. \nWhile you\u2019re on the phone, it pays to ask your analyst to dish the dirt on the latest case study he or she wrote. "Get the analyst on the phone and ask for the real scoop," says Efrem Mallach, CEO and cofounder of the Kensington Group. Analysts gather a lot of dirt that they don\u2019t end up using in the case study. But they can tell you that information on the phone, says Hurwitz Group\u2019s Quinn. You\u2019re looking for the not-for-publication tidbits that could save you hassle and money during your implementation.\n6 Don\u2019t Make Your Analyst Stretch Too Far \nWHEN CUSTOMERS PRESS ANALYSTS TO REACH beyond their area of expertise, bad things can happen. "There are a few phrases analysts have difficulty speaking: \u2019I don\u2019t know\u2019 and \u2019I don\u2019t cover that in depth,\u2019" says Quinn. \nMcCready recalls once getting into trouble when a large brokerage house asked him to do some custom research that was outside his domain. His recommendation to the brokerage turned out to be wrong, and they were very angry. "Analysts get stretched. They\u2019re being asked to do all these things. If you say, \u2019I don\u2019t know the answer to that,\u2019 there would be this awful silence. They would be wondering, What else doesn\u2019t he know?" says McCready. \nDon\u2019t expect analysts to have flawless insight in every area. Have your B.S. meter on during every exchange with your analyst. Don\u2019t let him gloss over something. Push him for depth, and if you don\u2019t get enough, don\u2019t follow the advice. Seek your information elsewhere.\n\nTHE VERY BEST PRACTICE OF ALL is being aggressive in helping your analysts get to know you. Find out how much one-on-one time you\u2019re allowed with your analysts and bump it up if it\u2019s sparse. Get on the phone, listen to their recommendations, then keep your own counsel, using as many different sources of information as possible. Expect all those sources to melt away when accountability time arrives. Bad advice doesn\u2019t qualify you for a refund or exemption from the consequences within your own company. Says Conlon, "I\u2019m the accountable person. They\u2019re advising. I\u2019m deciding. I\u2019m not forced to do what they tell me."