by John Moore

Ignore Cloud Security Assessment at Your Own Risk

Mar 13, 20139 mins
Cloud ComputingCloud ManagementCloud Security

Companies that deploy software-as-a-service often don't know everything about the security provisions their SaaS providers and partners have in place. Experts say it's because they don't know what to ask, they don't know what to test and they no longer know what's standard for a cloud service provider contract.

As more enterprises embrace software-as-a-service (SaaS), a nagging question has begun to surface: Who’s on the hook for assessing and validating cloud security?

The sometimes-complicated world of cloud computing makes that questions tricky to answer. A SaaS deployment involves the customer, the software provider and, possibly, another party that hosts the cloud software. Some projects may also involve a cloud services broker as an intermediary.

Tips: Is Your Cloud System a Mess? Our Self-Assessment Tool Will Tell You

SaaS apps cover a lot of ground these days, including business-critical functions from email to ERP, yet many cloud customers appear to simply accept whatever a SaaS provider says about its level of security.

Last year, the SANS Institute, an IT security training organization, reported that only 22 percent of the organizations it surveyed rely on extensive testing and validation before putting a outsourced or cloud-based application into production.

Vetting SaaS Providers No Easy Task

Cloud Security

SANS analysts contend it’s not enough to take SaaS providers at their word. At the same time, probing SaaS security can prove difficult for enterprises.

Jim Bird, a SANS analyst and co-author of the study, cites a lack of good guidelines for how to vet a SaaS provider. Tight budgets and limited resources are also considerations. “Most organizations are fighting for resources to secure their own solutions, never mind their suppliers,” Bird says.

Industry executives suggest that SaaS buyers conduct a security assessment of vendors before they buy and annually once they start using the software. Third-party reviews of SaaS vendors, however, may lighten that load somewhat.

Analysis: Government Moves Toward Cloud Computing ‘Perfect Storm’

Auditing standards such as the Statement on Standards for Attestation Engagements No. 16 (SSAE 16) and security frameworks such as ISO 27001 provide buyers with some clues to a cloud provider’s security commitment. In addition, the recently launched Federal Risk and Authorization Management Program (FedRAMP) establishes a cloud security assessment standard for cloud software providers in the government space.

John Keese, CEO of Autonomic Resources, the first FedRAMP-approved cloud service provider, believes this cloud vetting approach may move beyond the government space. “We think this is probably a model that will flow into commercial.”

Cloud Security Assessment Tough But Necessary Job

Paul Hill, a consultant with SystemExperts, a Sudbury, Mass.-based security consulting firm, says customers should step up to the assessment task. “When an enterprise is thinking about using a SaaS vendor or cloud service, it has the responsibility to assess the vendor and determine the risks, liabilities, and responsibilities.”

Hill says an assessment could take the form on an onsite visit and in-depth interviews to review the services. Alternatively, an enterprise may opt to let the vendor perform a self-assessment through a questionnaire. A security review from an independent auditor also contributes to the overall security picture.

Lithium Technologies, a Emeryville, Calif.-based company that focuses on social customer experience solutions, takes a multi-layered approach to assessing SaaS vendor security. The company runs a significant portion of its business in the cloud, tapping Box for collaboration, Okta for identify management and for customer relationship management.

Misha Logvinov, Lithium’s senior vice president and chief customer officer, says the company has established a security evaluation process. One component is finding out how a product is architected and whether it incorporates security in its design, he notes. Typically, Lithium will meet with the cloud vendor’s product management and engineering personnel to discuss the architecture issue.

Tips: SaaS, PaaS, and IaaS: A Security Checklist for Cloud Models

Logvinov says the company also wants to know whether a vendor has a security program in place and whether security is integrated into the software development lifecycle. Lithium also looks for audit and security standards such as SSAE 16 and ISO 27001. “[Since] we essentially delegate more risk to the cloud providers,” he says, “we want to make sure we can fully trust the cloud providers.”

SaaS Testing Misunderstood, Not to Mention Difficult

It’s generally agreed that customers should test. So why aren’t more stepping up?

Lack of clarity is one issue. Deb Radcliff, executive editor of the SANS Analyst Program, says organizations don’t necessarily understand what they need to do in SaaS testing. “There is a lot of confusion about what type of vetting they need to do with a SaaS provider, how to conduct the vetting and then how to maintain the visibility they need when using hosted applications,” she says.

Analysis: IT Service Providers, Customers Battle Over Data Breaches

The nested nature of cloud services further complicates testing. A SaaS provider’s software may run in another company’s hosting facility, for example. Hill says an enterprise shouldn’t stop with an assessment of the cloud service vendor; it must also evaluate the vendor’s third parties, such as colocation facilities and cloud infrastructure services.

“An enterprise should also know about those relationships and what types of assessments the primary cloud service provider has performed when selecting those providers,” he says.

Budget practices and economics also play a role in limiting SaaS testing. Glenn Weinstein, co-founder and CIO at Appirio, a cloud services provider based in San Francisco, says IT organizations may lack a formal budget line item for SaaS testing and instead rely on the vendor to provide security. “It’s still not top of mind in the budgeting process. You don’t see it broken out as a separate line of the security budget.”

There Are No Dumb Cloud Security Questions

Just because an enterprise lacks a formal SaaS testing budget doesn’t mean it isn’t asking security questions, Weinstein notes. He’s seen IT security teams invest significant time with cloud vendors as part of the RFP process.

As a cloud service brokerage, Weinstein says Appirio fields client security questions. The company defers some inquires to the SaaS vendor involved in a particular customer engagement—questions regarding infrastructure, data centers and the layers of security around a given application, for example.

Appirio, meanwhile, directly addresses questions related to its own security process, Weinstein notes. The company, or its business partners, may need to access a SaaS application on the customer’s behalf. This means clients are interested in how Appirio protects data from internal breaches.

Specifically, customers may ask how the company handles data in transit, or in the development environment, or when it is passed among consulting partners, Weinstein notes, adding that customers continue to grapple with what to ask of their cloud providers. “We are in the very early days,” he says, “and the types of questions that customers ask about the cloud…will continue to change.”

If anything, Weinstein would like to see more probing questions from customers. “We still see a lot of questions aimed at considerations that are pretty well shored up at this point.”

How-to: 10 Disaster Preparedness Questions to Ask Your Cloud Services Providers

An RFP might ask cloud vendors about penetration testing or distributed denial of service vulnerability, but Weinstein says the top enterprise providers have those issues well in hand. He’d prefer to see RFPs ask about configuration security, authentication options, and the provider’s ability to control access to data among employees and third parties. He suggests that those questions more closely address the security surrounding cloud applications.

For SaaS vendors, customer questions may focus on security audits. builds cloud apps on’s platform. Jeremy Roche, president and CEO of, says its larger customers in particular are not only interested in the security of the underlying platform, but also FinancialForce’s application layer security.

In the last 12 months, Roche says, they have demanded certification “over and above what we get from the base platform itself.” Customers are especially interested in SSAE 16 as a sign of a sound SaaS provider, he adds, noting that customers have asked for it “on multiple occasions.” To that end, the company recently went through a SSAE 16 audit, which examines a service organizations’ controls.

Industry Standards Offer SaaS Security Baseline

SSAE 16 is becoming a security baseline of sorts for cloud software providers, so much so that SANS’ Bird says the audit standard “should be a requirement for any major SaaS solution.”

Commentary: What You Really Need to Know About Cloud Security

That said, customers may look for evidence of SaaS security beyond SSAE 16. Roche specifically points to the U.S.-European Union Safe Harbor framework as one example. The program becomes relevant for European customers who subscribe to cloud services that host data in the U.S. American companies self-certify that they comply with the safe harbor framework.

Some cloud vendors, however, ask for outside auditors and consultants to assess their security in accordance with industry standards. The ISO 27001 and ISO 27002 information security standards are among those SaaS vendors follow, with “an increasing number of cloud vendors” using a third-party for annual assessments, Hill says.

Hill says such standards are useful as a starting point. Indeed, third-party certification provides a level of assurance and may cut down on the amount of security assessment and validation a customer has to do on its own.

That’s the thinking behind FedRAMP. The government program offers a standard security assessment process for cloud solutions including SaaS. Cloud vendors that successfully complete a FedRAMP review are granted a provisional security authorization, which agencies government-wide may leverage. The idea is to eliminate redundant security checks. The General Services Administration, which administers FedRAMP, contends will program save about $200,000 per authorization.

A third-party assessment organization performs the FedRAMP check, which takes into account 298 security controls. Keese, whose Cary, N.C.-based company obtained a FedRAMP authorization in December, believes cloud vendors can expect the assessment to take 12 to 16 months. They may bemoan such a difficult process, he says, “but it is difficult for a reason. Your computer security practices can’t be wishful thinking.”