Security software that uses 'code hooking' opens the door to hackers

The flaws allow hackers to easily bypass exploit mitigations in the OS and third-party apps

Code hooking techniques can introduce exploit mitigation bypasses
Credit: IDGNS

Some of the intrusive techniques used by security, performance, virtualization and other types of programs to monitor third-party processes have introduced vulnerabilities that hackers can exploit.

Researchers from data exfiltration prevention company enSilo found six common security issues affecting over 15 products when they studied how software vendors use 'hooking' to inject code into a process in order to intercept, monitor or modify the potentially sensitive system API (application programming interface) calls made by that process.

Most of the flaws enSilo found allow attackers to easily bypass the anti-exploit mitigations available in Windows or third-party applications, allowing attackers to exploit vulnerabilities that they couldn't otherwise or whose exploitation would have been difficult. Other flaws allow attackers to remain undetected on victims' computers or to inject malicious code into any process running on them, the enSilo researchers said in a report published Tuesday.

The hooking method is used extensively in the antivirus world to monitor for potentially malicious behavior, but is also used by anti-exploitation, virtualization, performance monitoring and sandboxing applications. Some malware programs also hook browser processes to launch so-called Man-in-the-Browser attacks.

Antivirus programs accounted for most of the affected products the security company identified, but one vulnerability also exists in a commercial hooking engine developed by Microsoft and used by over 100 other software vendors.

EnSilo identified affected products from AVG, Kaspersky Lab, McAfee/Intel Security, Symantec, Trend Micro, Bitdefender, Citrix, Webroot, Avast, Emsisoft and Vera Security.

Some of these vendors have released fixes for the flaws, but patching is not easy because it often requires recompiling each affected product individually. Microsoft plans to release patches for its Microsoft Detours hooking engine in August, enSilo said. 

Bitdefender said it fixed the problem on Jan. 19, and has pushed the fix out to affected customers. Symantec said that it fixed the issue in March and Vera Security said that changes to mitigate the risk have been in production for some time already. Security firm Webroot said that it patched the flaw one week after being notified about it in late December and customers received the update automatically from the cloud.

The researchers plan to release technical details of the vulnerabilities during the upcoming Black Hat security conference in Las Vegas in early August.

This is not the first time this type of security weakness has been introduced by antivirus products. Last year, the enSilo researchers found that products from Intel Security, Kaspersky Lab and AVG allocated a memory page with read, write and execute permissions to user-mode processes. This allowed attackers to bypass anti-exploitation protections such as address space layout randomization (ASLR) and data execution prevention (DEP) for third-party applications.

In fact, it was that discovery that fuelled this larger research into hooking implementations used by different products.

It's hard to estimate the number of vulnerable systems, but since the affected software includes the Microsoft Detours engine, the enSilo researchers believe that hundreds of thousands of users might be affected. According to them, the Detours vulnerability has existed for at least eight years.

"Companies using affected software should get patches from the vendors, if available," the enSilo researchers said. "Customers using software from the affected vendors should contact their vendors and demand that the software be patched."

Microsoft said: "Windows is the only platform with a customer commitment to investigate reported security issues, and proactively update impacted devices as soon as possible. We recommend customers use Windows 10 and the Microsoft Edge browser for the best protection. Our standard policy is to provide solutions via our current Update Tuesday schedule.”

To comment on this article and other CIO content, visit us on Facebook, LinkedIn or Twitter.
Download the CIO October 2016 Digital Magazine
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.