3 Things You Need to Know About Gartner Magic Quadrants

The Gartner Magic Quadrant is the passionately discussed tool for product selection in the IT industry. It is criticized for its simplicity, subjectivity and blind spots. Ask any vendor’s marketing professional about the Magic Quadrant and they will fill your ear.

I have experience with the Magic Quadrant, both from the perspective of the author of several and on the vendor side as one who has struggled with its shortcomings. And yet, despite the vitriol, the vendor community and IT leaders continue to support the Magic Quadrant; vendors by promoting Magic Quadrants when they are designated Leaders, and IT management when they use MQs to short-list vendors. The Magic Quadrant is a valuable tool, but it has to be used with full knowledge of the research process and the analysts behind it to get the greatest benefit.

The Magic Quadrant had its inception at research meetings conducted by Gideon Gartner when Gartner Group was still small enough for most of the analysts to fit in one room at 50 Topgallant Road in Stamford, Connecticut.

Gideon was a proponent of “Stalking Horses” thought experiments that would spark innovative thinking. During one meeting an analyst was presenting on a new technology sector. Gideon stepped to the easel and drew a quadrant with two axis bisecting it. He never meant Quadrants to be published and they never were on his watch.

Sometime in the 90s the Quadrant evolved into an official Research Note and today there are hundreds of Magic Quadrants for sectors as defined as Firewalls and as obscure as Cloud Infrastructure as a Service and Web Hosting.

The Magic Quadrant has two axes. The horizontal is “Completeness of Vision” and is a reflection of how many features a product has and the innovative enhancements that are forcing other vendors to react to keep pace. The vertical axis is “Ability to Execute” and is determined by revenue, number and quality of resellers and distributors, number of employees and their distribution between engineering, sales, and support and other business issues. Thus, the upper right quadrant is were the Leaders sit, the lower right is Visionaries, the lower left is Niche, and the upper left is Challengers.

I found myself, in 2000, the author of several security-related Magic Quadrants when the research methodologies were yet to be as formalized as they are today. I used information from Gartner’s DataQuest research to determine which vendors would be on it and I conducted interviews with each vendor. Then, with knowledge gleaned from talking to hundreds of end users, I would determine where the dots would be positioned for each vendor. It was indeed subjective but the process was informed by knowledge of the space.

Today the process is much more rigorous, even onerous, for the analysts. Key criteria are published for inclusion, usually based on some lower limit cut-off for revenue. An analyst has to defend any decision to exclude a vendor if they meet those published criteria.

In addition to frequent briefings from each vendor the analyst tracks the number of inquiries received about each product from the Gartner client base of 11,000 mostly large, enterprises. A questionnaire is put together with as many as 150 questions in spreadsheet form that address everything from feeds and speeds to product revenue by region. When all the results are tabulated, rankings of 1-5 and weights of low-standard-high are applied and the positions of the dots are determined.

Of course, the rankings and weights are subjective but at least a common set of questions and weights are applied and the results retain some objectivity. And, the analyst can indeed use the Magic Quadrant to push an agenda. Analysts are not just researchers and reporters. Their job is to see where the industry is going and help it along a path that addresses the needs of the Gartner client base — large enterprises.

Here are three factors to incorporate in your own product selection methodology that can help you leverage the Gartner magic Quadrant.

First, you only have limited resources to devote to doing the research on the myriad products in each sector. Consider the expertise of the Gartner analysts that compile the MQ. They usually have hands-on experience with at least one of the products. While that may have been years ago they are truly experts in evaluating companies and are fully immersed in determining which vendors are best at addressing your problems.

Your in-house experts may be better technologists than any analyst but they cannot help playing favorites. They resist change and may have a long relationship with a particular vendor. They may be best friends with the incumbent sales person. They often lack objectivity. The Gartner analyst knows more than you do. Leverage that knowledge.

Use the MQ as a starting point. Do not short list the Leaders and ignore the other quadrants. Look at each vendor and apply your own criteria. A vendor in the Niche quadrant may be there because they serve your industry sector exclusively. Or they may serve only your region.

Your own IT infrastructure may qualify a vendor for the short list. If you are an exclusive IBM shop you are going to over-weight the product from IBM. So, take advantage of the leg work the Gartner Analyst has done but apply your own criteria to develop a short list for evaluation and proof of concept. If you are a Gartner client do the industry a favor and keep your analyst in the loop throughout the process. You will be informing future Magic Quadrants.

The single most important thing to keep in mind when using Magic Quadrants in your vendor selection process is that they are written for the Gartner client base. Gartner’s 11,000 clients are the largest organizations in the world and Gartner acknowledges that 80 percent of them are late-adaptors. They are much more likely to buy from HP, IBM, or Oracle than from a start-up with the most cutting-edge solution. If your organization could also be classified as a late adaptor you are probably going to buy a product in the upper half of the MQ. If your IT decisions are more future-forward you will be looking at Visionaries and even Niche vendors.

Treat the Magic Quadrant as a valuable tool but not as a crutch. Mandates like “we only buy from MQ Leaders” will not serve you well. Perhaps Gideon Gartner’s original concept has withstood the test of time. Treat the Magic Quadrant as a Stalking Horse: a starting place, not an end.

Richard Stiennon is chief research analyst at IT-Harvest and the author of the newly published boot “UP and to the RIGHT: Strategy and Tactics of Analyst Influence” available on Amazon.com. Stiennon was a vice president of research for Gartner from 2000-2004. While at Gartner he was given its Thought Leadership Award and was named One of the 50 Most Powerful People in Networking by Network World Magazine. He was also chief marketing officer for Fortinet and has helped dozens of technology vendors with their go-to-market strategies. He is often engaged by investment firms to provide due diligence on acquisitions and investments.