by Kevin Fogarty

Before You Choose a Cloud Computing Vendor: 8 Questions

Apr 27, 2010
Cloud Computing

When you're comparing cloud computing service providers, traditional IT product selection skills may not cut it. Consider these 8 questions for your working checklist.

There are few ways a CIO can look better than by walking in to the CEO’s office to offer a sophisticated technology service that answers a desperate business need without requiring large capital expenses or delays before implementation.

“Flip the switch and there’s the extra capacity. Pay for what you use and shut it off again,” says Steven Peltzman, CIO of the Museum of Modern Art (MoMA) in New York.

Not every cloud provider can do that, of course. Some don’t offer the right kind of service. Infrastructure as a service providers such as Amazon Web Services require too much work up front to suit a relatively small IT project.

Other vendors don’t offer the right kind of security — the kind a CIO gets from knowing that new cloud provider is a partner to be relied on, not one that will disappear or fail after the CIO talked a large portion of the company into relying on it.

Faced with a project list greater than his budget, MoMA’s Peltzman looked into various cloud services as ways to extend MoMA’s capacity in specific ways at specific times. Broad-spectrum IaaS services such as Amazon’s EC3 had plenty of capacity, but the startup took too long, he says. SaaS providers such as’s online ERP were too function-specific, he adds. Instead, he picked a service from Cloudshare that allowed him to create virtual workgroup environments at will, on Cloudshare’s network.

How to Negotiate a Better Cloud Computing Contract

How do you find the right cloud provider? There’s not a consistent checklist either small or large companies can go through to make the selection, according to Bernard Golden, CEO of cloud consulting firm HyperStratus and blogger.

“A manufacturing company isn’t going to have the same checklist as a service company or retailer,” Golden says. “They’re too different. But there is a consistent set of things to look at. Some of them are specific to cloud providers; a lot of them are the same kinds of things you had to look at in outsourcing or any other service provider contract.”

1. How responsive is the cloud company?

“How fast do they call you back?” asks James McKee, president of United Resource Systems, medical-debt collection company based in Lakewood, Colo. “It doesn’t tell you everything, but I like to know how important I am to them and how responsive they are. My clients demand that responsiveness from me; I demand it from my providers.”

Some providers may be more responsive at the beginning of a relationship than later, so checking with other customers on that point is important as well, Golden says.

2. How transparent is the cloud service?

“There’s a lot of mystery in clouds,” according to Chris Wolf, analyst with the Burton Group. There’s no need to understand the underlying infrastructure and the company’s plans to upgrade or reinforce it if you’re just using Google for Gmail. But any company hiring a provider for important business functions deserves to know what kind of technology — and secondary or tertiary service providers — actually makes up the cloud and how reliable it is.

3. How prepared is the cloud provider to answer due-diligence questions?

Some of the most critical questions are the most basic: what does the company do to ensure physical security; what servers and software does it run and what are its arrangements for disaster recovery; are its employees all well trained, background-checked, bonded and secured?

“All the basic stuff is pretty important, but you have to verify that,” Wolf says. “You have to know they’re relatively stable and reliable in hiring and you have to check on things like making sure they have redundant telecom arrangements and high availability/DR options so you don’t go down for three days when they have a power outage.”

4. How much access does the cloud provider offer?

“You should be able to go through your list of criteria with the vendor and get answers to your questions and have them revisit that periodically to demonstrate how they’re living up to your expectations,” Golden says. “If it’s a big contract, you’re going to want to do audits periodically to verify SLA and compliance and security issues.”

5. How much access does the cloud provider deny?

On the other hand, Golden says, no service provider can afford to spend all its time answering questions from customers.

“They still have to provide their service,” Golden says. “Some providers will go through the whole audit with you whenever you want, but how much value is there in having them answer a questionnaire for you when 90 percent of the questions are the same ones that every other customer asks? Some are too far at the other end. Amazon puts up a white paper and says ‘here’s our privacy policy and our security procedures, it’s your responsiblity to go through this and make sure what we do matches with your requirements.’ “

“Amazon is maybe a little too distant, but you don’t want to have a provider verifying their physical security by letting you walk around the data center and look at it. Then you know some other customer is doing the same when you’re not there, and you’d rather no one has physical access to your servers that doesn’t work for that company,” Golden says.

6. What criteria does the cloud provider use for success?

Most IT service contracts define service levels according to bits and bytes and feed-speeds, rather than the effect the customer hopes the service will provide, according to Vince DiMemmo, general manager of cloud and IT services at infrastructure services-provider Equinix, which provides the data-center infrastructure that supports Amazon’s EC3 and other cloud and telecom services.

“Quality of experience is an industry measure of how well an application performs from the point of view of the end user,” DiMemmo says. “We use it for a lot of our end-user contracts and more customers are asking for it.”

7. What cloud services do they use to provide their service?

One of the infrastructure questions to answer is what data-center co-location or data-center service companies provide the infrastructure underneath the service you’re buying. Cloud systems are often built on other clouds which, DiMemmo says, can be an advantage. Hiring service providers who base all or much of their infrastructure in the same set of high-performance databases can make due diligence easier for customers — because you only have one infrastructure to research, rather than a different one for each service, DiMemmo says.

8. What does the cloud provider require of you?

Cloud computing metaphors are designed to make customers feel warm and fuzzy, but those who forget their own responsibilities in a cloud-computing arrangement will never be satisfied with either the division of labor or quality of service they get, Golden says.

“The infrastructure is the provider’s responsibility; the application is yours,” Golden says. “If the application isn’t structured so it will work effectively in a cloud environment, or the interfaces are clunky, or it’s based on a database server that’s on its last legs in your data center, that’s not the cloud provider’s problem. That’s your part of the responsiblity. You have to be sure you’re ready to live up to your end.”

Follow everything from on Twitter @CIOonline.