A significant chunk of new malware is not spotted by antivirus programs with some threats remaining a mystery for as long as a month, an analysis of large enterprises by firewall vendor Palo Alto Networks has calculated.
Drawing on three months of data from 1,000 of its own customers Palo Alto's found that that its Wildfire malware detection system spotted 68,047 new malware files, 26,363 (40 percent) of which were not blocked by six unnamed "industry-leading" antivirus programs.
Around 90 percent of these undetected samples arrived via the web with programs taking an average of 20 days to add the threats to their detection systems; a small number of threats delivered via social media and FTP went undetected for more than 31 days.
Detection was better for email, with only 2 percent of threats getting past clients and an average five-day wait for protection.
This is a highly charged issue for antivirus vendors so let's be very clear about what Palo Alto's Modern Malware Review analysis might be telling us and what it might not.
Wildfire is basically a firewall-led system in which unknown binaries are fed back to the cloud to see what they and the traffic they generate is trying to do - the latter element is what allows Wildfire to spot threats antivirus clients can't, or so the theory goes.
Parts of this design aren't a long way from antivirus companies that use cloud fingerprinting also do, although in Palo Alto's case the subsequent blocking of any malware discovered is done at the firewall level rather than by the client.
According to Palo Alto, the inherent problem with web-borne malware is its polymorphism, basically the fact that a server can re-encode the payload to make it appear unique - "malware on demand" to coin a phrase. By contrast, email-borne malware is static and sent out in bulk and that makes it more visible.
What the report doesn't document (and we weren't able to confirm) is whether the antivirus programs were also being used with some kind of web fingerprinting system, which if they were might have boosted their detection success.
However, one can infer from the fact that clients weren't able to spot the unknown malware for days or weeks as suggesting otherwise. On the basis of the programs used, antivirus is failing to detect threats on a worrying scale.
As a maker of high-end application-based firewalls, Palo Alto is not then arguing that antivirus is useless so much that detection should also be placed inside the network itself. This approach chimes with its marketing but is not without some logic.
Palo Alto said it had isolated 100 behaviours that identified the 26,000+ unknown malware threats which rendered them suddenly apparent. These included generating unknown TCP/UDP traffic (30 percent), visiting an unregistered domain (24 percent), sending emails (20 percent), plus a variety of other unorthodox behaviours including connecting to a new DNS, downloading files with incorrect extensions, and visiting recently-registered domains.
This isn't so much a conclusion as a battering ram: conventional antivirus clients don't have a hope of spotting such malware because they are designed to look at files not traffic.
In an age of targeted malware, lethality becomes harder to assess. So six antivirus clients didn't detect over 26,000 samples reckoned by Palto Alto to be malware, but how many of these were serious as opposed to merely a risky nuisance?
The firm's view seems to be that if security managers have to devote too much time to spotting and remediating common malware they will be drained of resources for detecting the smaller number of extremely serious threats.
"It's not enough to simply detect malware out there that is evading traditional security. Enterprises should come to expect more comprehensive prevention from their vendors," said Palo Alto''s senior research analyst, Wade Williamson.
"That's what the Modern Malware Review is signaling - analysing undetected malware in real networks has enabled us to arm IT security teams with actionable information for reducing their exposure against threats they might have otherwise missed."