Software vulnerability testing tools are looking for only 45 percent of the 600 most common vulnerabilities identified by a US government backed security project.
Details of the Common Weakness Enumeration (CWE) project and its initial findings were presented to delegates at the Black Hat security conference in Washington this week.
The CWE, which is sponsored by the US government’s Department of Homeland Security and maintained by a team of workers at non-profit organisation Mitre together with other security professionals, is roughly four months away from publishing a final draft of its software vulnerability encyclopaedia, delegates were told.
Commenting on the vulnerability tools currently available, Sean Barnum, director of knowledge management at Cigital, a software quality assurance company said, "We found that less than half of what we already have in CWE is covered by these tools, so this helps prove that there are a lot of known issues out there that aren't being addressed," said.
"We also thought that the tools would look for the same types of things, but they are actually very different, and there's not a lot of overlap; that's something that developers need to be aware of as they choose tools. You want to right set for aggregated coverage."
Launched in December 2005, CWE seeks to establish a unified, measurable set of software flaws to help developers improve the quality of their products and drive out the types of vulnerabilities that have led to the ongoing malware explosion.
By gathering input on common mistakes identified by developers, researchers and security vendors, the group believes it can create a common language and standard procedures for handling the many different types of loopholes that exist in programs' source code today.
Much of the group’s work has, so far, has revolved around the gathering of vulnerability formats and the various methods used to identify and remediate the coding problems. But the project has recently involved a significant amount of testing of security scanning tools to get a better idea of the capabilities and limitations of those products.
By gaining an understanding of the vulnerabilities that popular code scanning engines can find --and those they can't --CWE can help developers understand the types of issues they will need to look for in their own code, said Bob Martin, a CWE leader and the head of Mitre's related CVE (Common Vulnerability Exposures) Compatibility effort.
"We wanted to evaluate what the tools claim to cover and what they are most effective at finding," Martin said. "Right now, best test is to throw tools at a big pile of code and see what tools find the most vulnerabilities, but we're changing that paradigm into test cases where we now look at the answers so we can evaluate what the tools found and what kinds of complexities they can handle."
CWE's research will not list the names and performance results of the products it is testing, Martin said.