5 Security Questions to Ask Your Software Vendor

Security Innovation, a risk assessment consultancy, provides questions you can ask a software vendor about its development processes. The answers you get will tell you just how much effort is put into security. It’s up to you how much risk you want to assume.

1. Do you review security at each phase of the software development lifecycle?

A good answer: Yes, we have integrated reviews into our product development lifecycle, from requirements definition to code development and testing.

Likelihood of getting this answer: Almost zero. Even companies that have created secure development best practices, like Microsoft, have implemented them only on a small portion of their applications.

2. What methodologies do you use for security testing your products?

A good answer: We have adopted methodologies from a respected security consultancy or large software vendor.

Likelihood of getting this answer: Small. Although some methodologies are required reading and have been adopted by companies like Adobe, McAfee and Symantec, a majority of companies have yet to adopt them. Most software development teams don’t consider security testing to be their responsibility.

3. Do third parties conduct security assessments on your products?

A good answer: Yes, we have a pool of application security companies we use to conduct independent assessments on all of our products.

Likelihood of getting this answer: 50 percent. This is up from about 25 percent two years ago. Third-party security assessments are increasingly a mandatory requirement and show up in RFPs and SLAs for packaged and on-demand software.

4. Do you have security squads that attack your products prior to release?

A good answer: Yes, we create an internal red team that acts as malicious users and complements third-party security assessments.

Likelihood of getting this answer: 20 percent. Though red teams are a growing trend, most companies still lack the internal expertise to dedicate staff to testing.

5. Do you use automated tools for security testing or code review?

A good answer: Yes, we use tools from this reputable vendor for code review during development and tools from that reputable vendor for security scanning our Web applications after deployment.

Likelihood of getting this answer: 20 percent. Adoption of automated tools is increasing, but an untrained engineer doesn’t become better because he learns how to use AutoCAD. He finds value in the tool only after he is trained to use it.

6. What training does your development and testing teams receive specific to application security?

A good answer: We have put our entire application development team through security awareness training as well as technical training aimed at each specific role, such as product manager, architect, developer, tester, auditor and others.

Likelihood of getting this answer: Almost zero. You’re lucky if you can find a company that has trained 10 percent of its application development team. Some forward-thinking software vendors, like SAP, have adopted eLearning and are aggressively training their ranks.

7. What percentage of your software development and testing team is focused on security?

A good answer: Five to 10 percent. Given all the different aspects of software quality (functionality, reliability, performance, usability, accessibility) anything in this range means the vendor is strongly committed to security and recognizes it as an important aspect of software quality.

Likelihood of getting this answer: Zero to 25 percent. Progressive software vendors and development teams acknowledge that security is a non-negotiable business requirement. Others, including on-demand or Software as a Service vendors view security as a cost of doing business because customers demand it.

8. Do you have a dedicated team to assess and respond to security vulnerabilities reported in your products?

A good answer: Yes, we have an incident response team whose job is to determine the seriousness of reported vulnerabilities and work with the product development teams to issue a response to our customers in a timely manner.

Likelihood of getting this answer: 75 percent to 90 percent. Because most software vendors have a way to report and respond to bugs, security defects are easily added to this process. But you should probe as to how (or if) reported security defects are treated differently than nonsecurity defects. Some companies view security defects as just another bug and, as a result, do not elevate them to a higher priority fix.

9. What is your patch release strategy and what tools do you offer for patch deployment?

A good answer: We issue regularly scheduled and fully tested patches to our products each Thursday, if needed. To make patch deployment and management easy, we support the popular configuration management systems such as HP’s CM Patch Manager and Attachmate’s WinINSTALL.

Likelihood of getting this answer: 20 percent. Most vendors don’t offer regularly scheduled releases and even fewer offer fully tested patches. For those who offer both, you need to be aware that the days between public disclosure of a vulnerability and issuance of a patch will be longer for vendors who do not fully test patch releases. You need to decide if timeliness of patch or having fully tested patches are more important for your organization. You seldom get both.

10. What methods do you use to inform customers of vulnerabilities?A good answer: Registered customers have vulnerability information disclosed to them immediately, even before a patch is ready. We generally take two weeks between customer disclosure and full public disclosure. Customers can choose their method of disclosure: e-mail, page/text message, or whatever, and all vulnerabilities are also posted to our customer Web portal.

Likelihood of getting this answer: 10 percent. Vulnerability disclosure is a topic of much debate, and some vendors don’t see the value of preemptive customer disclosure. Others feel that no disclosure at all is the best policy, notifying their customers and the public only after a patch is ready. A large community of security “researchers” expose vendors publicly with no notice.

11. What technical guidance do you provide about vulnerabilities, including how they could be exploited, how they are currently being exploited, how to mitigate?

A good answer: For each vulnerability reported, we develop a threat profile that explains how and if the vulnerability can be exploited. We also identify mitigation strategies, if any, prior to developing and releasing a patch.

Likelihood of getting this answer: 25 percent. Software vendors that practice customer or public vulnerability disclosure are generally diligent about explaining mitigation strategies, if any exist.

12. Do you provide severity ratings for vulnerabilities, and how are they determined?

A good answer: Yes, we subscribe to the Common Vulnerability and Exposure standard developed by MITRE for categorization and we have developed our own software security severity rating system, which we publish publicly.

Likelihood of getting this answer: 25 percent. Companies such as Adobe and Microsoft have been good at defining and sharing their severity rating system. Users still need to be aware, however, that severity is a contextual measure, so judge for yourself how severe a vulnerability is in your environment.

13. Does your company monitor the latest attack trends in the underground community and consider how those trends may affect your software?

A good answer: Yes, we have a research team that searches for new attack trends and techniques. We try to stay in front of hackers by dedicating internal resources to malicious use of our applications and search the known communities where hackers communicate and share information.

Likelihood of getting this answer: 25 percent. Companies who take a proactive stance with security disclosure and severity ratings also generally conduct these types of activities.

14. Do you disclose all vulnerabilities that affect your software?

A good answer: We disclose all vulnerabilities we are aware of in our publicly released software.

Likelihood of getting this answer: 80 percent. But beware of the timing of the disclosure. Some companies only disclose after a patch is ready and posted on the same day as the disclosure, even though they knew about the vulnerability long before that.

15. What are the terms and period of your security support agreement?

A good answer: We ensure that all critical security defects will be fixed within one month of discovery. We use our severity rating to define critical, and we stand by our commitment to eradicate all serious vulnerabilities that might put our customers at risk.

Likelihood of getting this answer: Almost zero. There are very few companies willing to take such a public stance and stand by it contractually. However, more customers are asking for such terms, which will force software vendors to be more attentive to security defects.