1. Security tools generate too many false negatives and false positives
The last thing you want is to miss a real attack while manually sifting through piles of false alarms from your security tools. False positives are not only a major resource drain, but IT managers worry that these could distract IT from real threats.
IDSes have long had a bad reputation for the volume of false positives they can produce. Newer generations of these devices aren't as sensitive and labor-intensive with their alarms as previous revs, but IT managers say they still get flooded with unnecessary alarms for legitimate traffic.
"I spend a lot of time tuning them properly, or we get bombarded with false positives," says Robert Mims, vice president of security and engineering for MedAvant Healthcare Solutions, which runs ISS RealSecure IDS sensors. It's a delicate balance of sorting through the real ones and the ones where the boxes are "crying wolf." So the company has put in a rule to ignore certain unnecessary alerts, he says.
If a false positive is a headache, then a false negative is a nightmare. A false negative is when bad guys, bad traffic, or malware slip past your IPS or antivirus application undetected. These alarms are becoming more of a problem today than false negatives, security experts say.
It's the nature of signature-based security tools. IDS/IPS and AV tools go with what they know, and sometimes that means letting unknown malware slip by or sounding an alarm for something that looks suspicious based on their signature patterns. IT managers say they're frustrated with these limitations of such tools. It's either too much extraneous information or not enough critical information, they say.
The main problem, security experts say, is false expectations on the part of users for just what their security tools can do. "Security products on their best day are only 50 percent solutions out of the box," says Michael Rothman, president of Security Incite. "You have to tune them to your environment and when you screw that up, it results in false positives and false negatives."
Even IPS vendors admit there's no way to get around false positives and negatives. "You're always going to have false positives and negatives with almost any security software you use," including IPSes, IDSes, and vulnerability assessment tools, says Marc Maiffret, CTO and chief hacking officer for eEye Digital Security, which makes an IPS. "I'd love be able to say there were no false positives in our product. But that would be like saying your software is never going to have bugs."
"IPS and reactive security never solved an operational security problem for anyone," says Thomas Ptacek, a researcher with Matasano Security. "You have to be more careful about what you allow on the network and what you have exposed," he says.
But IPS vendors say their protection filters undergo through a major vetting process before they get released. "It gets a pure review. We look for false positives and performance degradation, and we can't release it if it shows a negligible percentage of performance degradation or a false positive," says David Endler, director of security research for 3Com/Tipping Point. "In some cases, coming up with the filter is easy, but getting it past the gates is hard."
