Bulletproof security is a practical impossibility. Anyone who claims to have perfected the art of security is either a fool or a liar, since no security product or schema is foolproof or invincible. What security promises is risk mitigation--assuming that security technology works as advertised. And that's the unspoken problem that undermines security effectiveness, says ICSA Labs.

According to ICSA, nearly 80 percent of all security products it's tested over the last two decades have failed to work as intended during the first round of testing. On average, it takes two to four rounds of testing for a product to earn the lab's certification, and even then they have trouble maintaining their status.

ICSA—an independent division of Verizon—performs testing on many of the most common security products and platforms, including network and Web application firewalls, antivirus applications, intrusion prevention systems, and VPNs (IPSec and SSL). It awards certifications based on common criteria developed in conjunction with the vendors that submit their products for testing. Certification is intended to reflect that a product meets the basic functionality and performance expectations of the community.

In celebrating its 20 years of security product testing, ICSA decided to review the testing and product performance trends of the last two decades. The results are startling--more than three out of four security products failed to deliver on their core functionality. Roughly one-half had problems logging activity for inspection and intelligence correlation. And 40 percent were inherently insecure and susceptible to compromise by hackers.

ICSA Managing Director George Japak says that there's little need for hackers to force their way through security layers when guard technologies themselves will crumble with little effort. In many cases, Japak says, the documentation written for some security products is so bad that the lab testers will call the vendor to make sure they didn't send the wrong material.

The good news, ICSA says, is that most security products will pass certification within two to four cycles. The problems and deficiencies found in the lab give the vendors information for refining and improving the performance of their products. And, as Japak says, it's better for the labs to find the problems than to have shortcomings exposed in the wild.

ICSA isn't bashful about its motivation for the report: the need for quality control in security products that's demonstrated by independent certification - such as the service provided by ICSA Labs. Japak points out those vendors that submit to certification product testing have a higher level of quality, imbedded security assurance and performance.

The problem with this report is that it's coming at a time when end users are questioning the value of the products they've spent millions of dollars on. While even bad security products will provide some level of threat protection, the ICSA findings could give end users some reason for pause when considering new purchases. Many security solution providers are complaining that end users—particularly SMBs—are reticent to invest in new security technologies because they don't believe they're at risk or don't have the budget. The ICSA findings could give them a new reason to doubt the need for security investment.