It's been a month since the Heartbleed Bug set off a stampede to patch software in everything from network gear to security software as it quickly became evident that vulnerable versions of the OpenSSL encryption code had been very widely deployed.
Heartbleed, which would let a savvy attacker capture passwords or digital certificates, for example, came as a shock when the OpenSSL Group disclosed it on April 7 because it impacted an estimated 60% of servers worldwide ... and much more. But has it been the catastrophe that some feared?
So far, the consensus seems to be no, though some think pinning down Heartbleed-based break-ins is not easy. At best, Heartbleed has been a mammoth inconvenience everywhere as passwords and certificates were swapped out in what became a patching marathon around the globe.
"It ended up being not that easy to exploit," says Bruce Schneier, CTO at Co3 Systems. Schneier is an encryption expert who had initially labelled Heartbleed a "catastrophic" event because of OpenSSL's pervasive use. "We saw some hackers and criminals in the wild using it, but not that much."
But the upheaval caused by Heartbleed -- a coding mistake made two years ago apparently by a German software developer who came forward to admit the mistake -- was absolutely huge as a broad swath of the network and security industry discovered after countless round-the-clock hours of investigating their own products that the vulnerable Heartbleed versions of OpenSSL were often embedded in them. But not all OpenSSL versions were vulnerable.
Cisco incident-response teams have determined roughly "359 Cisco products and services use OpenSSL or a related variant," says Cisco spokesman Nigel Glennie. "281 of these have been confirmed as not affected by the vulnerability, and 78 confirmed as affected. Of those 41 have already had patches provided."
As of today, Cisco was still investigating the product Cisco Virtual Security Gateway for VMware. Since April 9, Cisco has made 19 revisions to its Security Advisory on Heartbleed. Cisco is handing out a "Heartbleed Bug: Assessment Guide" to customers, a six-page reference that explains the nature of the OpenSSL problem and how to remediate it.
"Everyone was using OpenSSL," says Gil Friedrich, vice president of technology at ForeScout Technologies, which says it fixed its own Heartbleed product problem in the first 24 hours after it became known. He says ForeScout also helped banking customers by analyzing the products they used in their networks for vulnerable OpenSSL.
Mandiant, now part of FireEye, in mid-April said hackers got into one customer's network because of Heartbleed. There were a few other random reports of Heartbleed troubles, such as when the Canadian Revenue Agency in April temporarily shut down its website in the middle of tax season after a hacker broke into it and stole 900 social insurance numbers.
But many in the security industry say it seems that attacks exploiting the Heartbleed flaw don't appear to have been common.
"You don't see a lot of exploits on this," says Jim Walter, director of advanced threat research at Intel Security (the new name that McAfee is transitioning to after being acquired by Intel). "We had only a handful of reports on data breaches."
Walter says one explanation is that Heartbleed, which lets a savvy attacker grab 56K of data from memory, is not the best way to go out on the Internet and steal things. "You have no control at all about what you get back," he noted. "There are easier ways to get things."
Symantec also saw no evidence of widespread mass attacks but Symantec's security expert Kevin Hayley says this type of attack "requires you to look at a log file" to find evidence of it against your enterprise.
In terms of attacks based on Heartbleed, "we've definitely seen this as a proof of concept," says Hugh Thompson, chief security strategist and senior vice president at Blue Coat, which has an SSL visibility appliance that can break open SSL traffic to inspect it. He says Heartbleed attacks can be hard to identify because they can look like an odd web transaction. "This one's tough," he says.
CrowdStrike's CTO Dmitri Alperovitch says Heartbleed has been "nasty" if only because at a minimum, companies and individuals need to replace all of their private keys and change passwords that may have been exposed on Heartbleed-vulnerable systems. He says there have been targeted attacks in the wild leveraging Heartbleed vulnerability to hijack user VPN sessions to gain access to internal company assets.
"Leveraging Heartbleed is not extremely difficult," says Alperovitch. "While it is true that only up to 64K of data from memory is returned in each Heartbleed request, the requests can be launched numerous times to retrieve more data in an automated fashion. An attack on a web server to retrieve private SSL keys has been shown to be done in under 10 hours."
Is there reason to feel frustration with the open-source code-development process after Heartbleed?
"Open source code isn't evil," says Walter. Open source has "fostered a great deal of development and creativity" in coding, and the majority of products on the market today use it.
Nonetheless, the shock of Heartbleed has prompted several of industry's heavy hitters to band together to start what's called the "Core Infrastructure Initiative" at the Linux Foundation. This multi-million-dollar project, supported by Amazon Web Services, Cisco, Dell, Rackspace, Facebook, Fujitsu, Google, IBM, Intel, Microsoft, NetApp, VMware and the Linux Foundation, is supposed to help fund "key developers and other resources" to assist in improving open-source security, with OpenSSL slotted as the first project.
"It's a great idea," says Schneier, saying he is getting involved with it. "We definitely need more coordination in doing security reviews on open-source software. Closed vs. open source has been characterized as a cathedral versus a bazaar. Turns out, even bazaar can use a little bit of cathedral."
Ellen Messmer is senior editor at Network World, an IDG website, where she covers news and technology trends related to information security. Twitter: MessmerE. E-mail: firstname.lastname@example.org
Read more about wide area network in Network World's Wide Area Network section.
This story, "Heartbleed Was a Headache, But Far From Fatal" was originally published by Network World.