The software would interact with existing personal firewalls and intrusion detection systems to gather data about anomalous behavior, says Senthil Cheetancheri, the lead researcher on the project he undertook as a grad student at UC Davis from 2004 to 2007. He now works for SonicWall. (Learn more about intrusion detection and prevention products.)

The software would share this data with randomly selected peer machines to determine how prevalent the suspicious activity was, he says. If many machines experience the identical traffic, that increases the likelihood that it represents a new attack for which the machines have no signature.

The specific goal would be to detect self-propagating worms that conventional security products have not seen before.

"It depends on the number of events and the number of computers polled, but if there is a sufficient number of such samples, you can say with some degree of certainty that it is a worm," Cheetancheri says. For that decision, the software uses a well-established statistical technique called sequential hypothesis testing, he says

The detection system is decentralized to avoid a single point of failure that an attacker might target, he says.

The task then becomes what to do about it, he says. In some cases, the cost of a computer being infected with a worm might be lower than the cost of shutting it down, in which case it makes sense to leave it running until a convenient time to clean up the worm, he says.

In other cases, the cost to the business of the worm remaining active might exceed the cost of removing the infected machine from the network, he says.

That cost-benefit analysis would be simple to carry out, he says, but network executives would have to determine the monetary costs and enter them into the software configuration so it can do its calculations he says.

End users would not program or modify the core detection engine, he says. "We don't want to have humans in the loop," he says.

He says he and his fellow researchers have set up an experimental detection engine, but it would have to be modified to run on computers in a live network without interfering with other applications and without being intrusive to end users, Cheetancheri says.

So far no one he knows of is working on commercializing the idea.