When reading up on a new technology, the last thing CIOs have time for is drilling down into the methodologies analysts use in preparing their forecasts and surveys. The CIO glances at the paragraph summarizing how many people participated in the survey, or she scans some of the basic forecasting assumptions, and that\u2019s about it. And to some degree, that\u2019s exactly what the analyst firms are counting on. The fact is, most analysts\u2019 methodologies are something less than scientific. Just how much less than scientific is a matter of some mystery. Analysts don\u2019t like to reveal what\u2019s going on behind the curtain. If you are curious, however, don\u2019t be afraid to push them on this."If you\u2019re paying for the research, you have every right to know how those numbers are generated. They usually say, \u2019Well, we have this model that we use.\u2019 You can demand to walk through every single algorithm if you want to," says Scott McCready, a longtime analyst and now president of CIOview, a software vendor in Boxborough, Mass.Mike Conlon, CIO at the University of Florida Health Science Center in Gainesville, just happens to also be a biostatistician. In fact, Conlon has a PhD in the application of statistical principles to medical research. And when he analyzes analyst methodologies, he doesn\u2019t like what he sees.For example, Conlon takes issue with the way his analyst firm (Gartner) uses certain terms that have statistical relevance, such as probability (as in this invented example: "The majority of IT projects are destined for failure: 0.9 probability"). "These probability statements are not probability statements as any statistician would be used to seeing them," says Conlon. "Their use of the term has nothing to do with mathematical models and scientific methodology. It\u2019s jargon. What they mean to show is how confident they are in their prediction."Jamie Popkin, Gartner group vice president and research fellow, explains that the company\u2019s probability statements indicate Gartner\u2019s level of confidence that an event will occur within a certain time frame. So why does Gartner dress them up in a veneer of statistical authority?The probability statements aren\u2019t meant to convey scientific certitude?they\u2019re just an easy shorthand for describing confidence levels, Popkin says.Conlon argues that the statements may be misleading to those not trained in statistics. But to be fair, he says, it would probably be impossible to bring statistical rigor to the world of technology forecasting because the forecast would have to be based on too many assumptions to be valid. Remember that the next time you\u2019re holding a forecast in your hand, no matter how "confident" its author.