The unsponsored test of socially engineered malware protection from NSS Labs used just-collected URLs of malicious sites and downloads. The sites used social engineering lures, such as claims that visitors need to download a fake video codecs to watch a movie, to trick potential victims into downloading the malware.

According to NSS Labs president Rick Moy, these results indicate that Trend Micro stopped 91 percent of downloads by either blocking the URL prior to downloading the file, or recognizing the file as malicious after it was downloaded, but before it was executed/double-clicked. Trend recognized as malicious and stopped an additional 5.5 percent of malware after it was executed but before it could install, for a total block rate of 96.4 percent. Kaspersky came in second with an 87.8 percent overall success rate.

Moy's report notes that Trend Micro's high score was significantly boosted by the company's use of an in-the-cloud reputation system that checks URLs and downloads against a server-based list of known malicious sites and files.

It's important to note that because the execution blocking tests only used the malware that had first made it through the first two tests (blocking the URL or recognizing the download prior to execution), the execution results in the chart don't represent an overall test of any given product's ability to stop malware using behavioral analysis or something similar.

Also, NSS Labs' results don't represent a complete test of a product's overall efficacy, as the results don't measure how well AV might block malware that comes in as an e-mail attachment or any other vector aside from a socially engineered download. The tests also don't include sites that use hidden exploits on Web pages to attempt to install malware without your ever knowing. While exploits sites are highly dangerous, Moy says the attack code they use essentially breaks the method NSS Labs uses to automate downloads and testing (for more exploits and NSS Labs' methodology, see my previous post on IE 8 and browser URL blocking).

Instead, these tests' value lies in their ability to simulate real-world protection against a broad category of threats that are out there right now, based on the critical point of "did it keep the malware from running on the PC." NSS labs gathered lists of suspicious URLs and downloads, filtered and verified them as malicious, and then immediately used the lists to test antivirus products. The company used 3,243 verified URLs over the course of its tests, which were run during July and August.