Evading the security police

Network environments are becoming ever more complex, which inevitably complicates the management of IT security. Intrusion Prevention Systems (IPS) therefore take on the role of security police in the network, intercepting detected threats. They...


Network environments are becoming ever more complex, which inevitably complicates the management of IT security. Intrusion Prevention Systems (IPS) therefore take on the role of security police in the network, intercepting detected threats. They are designed to prevent attacks on vulnerable systems that can only be patched in maintenance windows and which are therefore not entirely secure. In order to detect malware in data traffic, IPS architectures inspect data packets for known exploit patterns.

A new type of IT threat, so-called Advanced Evasion Techniques (AETs), take advantage of this approach. AETs combine and modify ways of disguising an attack or malicious code, making it possible to circumvent virtually any network security solution.

The security of a network depends on numerous criteria. Network, server and security administrators need to understand and correctly use a variety of controls to defend their companies against an evolving threat landscape. In addition to these requirements, the network topology itself often hampers the security of a system. Dynamic and poorly planned network services, for example, may prevent administrators from implementing strict segmentation and firewall policies.

In some industrial networks it may not be possible to install all the necessary operating system updates because outdated software and protocols are being used. The multitude of patches and new versions of operating systems and applications along with their compatibility requirements prevent such systems from being kept up to date with the latest security developments on a regular basis.

This is where the IPS comes in. Unlike a firewall with its defined security policy rules that allow or disallow packets depending on their source, destination, protocol and other properties, IPS devices promise to inspect data traffic for malicious code patterns and only allow it to pass if no threat is detected. If malware attempts to penetrate the network, an IPS will automatically drop the data connection. The techniques used by most systems to inspect data traffic include protocol analysis and attack signatures. These detect predetermined attack patterns used by malware in the network traffic to exploit vulnerabilities in a system. Generally, when a new exploit is discovered, detection methods are implemented in the inspection devices within a few days, even hours. Malicious software that is similar to known threats may be able to be detected and combated with already existing analysis functions.

Advanced Evasion Techniques

Advanced Evasion Techniques sidestep these mechanisms, however. Hackers use evasions to disguise their attacks and circumvent network security systems. Until recently only a handful of evasion techniques were known and most security systems were able to deal with them well. But AETs are a new kind of threat. They are always varying the way they disguise an attack and target different levels in the network traffic to deliver malicious payload to a network without detection. In tests, possibilities for an attack with AETs have been found on IP and transport layers (TCP, UDP) as well as on application layer protocols, including SMB and RPC protocols.

To disguise malware, AETs prey on protocol weaknesses and the permissive nature of network-based communication. They make use of the well known method of de-synchronising detection systems in the network. In such cases the IPS has a different understanding of the protocol state of a data packet than the target host. This can happen, for example, if an IPS can't buffer enough data fragments before applying signatures, or can't re-assemble the fragments correctly. Then the IPS no longer has the original context of the data packet and it rewrites the stream before forwarding it to the target system. This enables data packets to be delivered that look normal and safe, but when interpreted by the end host can turn out to be an exploit against it.

The network security experts at Stonesoft discovered AETs at the end of 2010 and reported the threat to the security authorities. So far Stonesoft has come across almost 150 different kinds of AETs. The true number of AETs and their possible combinations, however, are almost limitless, with variants numbering 2 to the power of 180. To protect a network from them, IPS architectures would need to recognise and cover all the possible variations. But sometimes just a minor change to the number of bytes or the segmentation offset is enough to make an AET attack no longer look like any of the attack patterns stored in an IPS - and despite a fingerprint update, malicious code may be taken for normal data traffic and so penetrate a network. Even the latest IPS systems are unable to cope with this. As there is no alert to a possible threat, the hacker can then look around in the system for a possible weakness or an unpatched server without being challenged, or even take control of the targeted system as an administrator.

Internet protocol vulnerabilities

A key role in the use of AETs is played by TCP/IP, the protocol suite used on the Internet and in the vast majority of networks. It specifies that a system should be conservative in its sending behaviour, but liberal in its receiving behaviour. In other words, it should be careful to send well formed data packets, but should accept any data packet it can interpret. This means that although data packets can be formed in a variety of ways, they should all be interpreted identically by the destination system.

This liberal approach is intended to guarantee interoperability between systems. But at the same time it opens up a multitude of ways for AETs to disguise an attack. This is because different operating systems and applications behave in different ways when receiving packets. So the destination system's application may see something entirely different from what was originally in the network traffic. The new Internet Protocol version 6 (IPv6), which offers a larger address space, will bring companies not only more functions but also new vulnerabilities to attacks using AETs.

The new multicast protocols and the changes to the routing of the new IP version give Advanced Evasion Techniques additional opportunities to disguise attacks on the protocol or transport layer. And, to make matters even more difficult for security managers, there is no wide-ranging experience with IPv6 to draw on. To ensure smooth communication, the new Internet Protocol demands further compromises when it comes to defining what a regular data packet must look like. While providing the necessary compatibility with IPv4, destination systems need to interpret incoming packets even more liberally than before. This offers AETs even greater scope for disguising malware.

Identical data interpretation

IPS systems can only protect networks from Advanced Evasion Techniques if they look for more than just the characteristics of known malware patterns. Security systems need to use different methods for checking data traffic, such as data packets not received by the end host, or protocols that can be decoded in multiple ways. Security devices that can perform a proper multi-layer normalisation process will interpret and fully reassemble data packets in the same way as the end host. They will take into account all the relevant protocol layers for each connection, IPv6 included. This reduces the risk of data packets not compliant with the classic IP rules bypassing the network security system undetected. Moreover, IPSs should be software-based so that updates and security patches can be implemented immediately. This will allow them to offer the best protection from the ever growing number of AETs.

Currently there is no full protection against AETs. So IT security leaders in enterprises today, along with security vendors, will have to address the problem more vigorously. With a view to developing a long-term solution, Stonesoft has created an open community platform at antievasion.com where IT security experts and vendors can exchange ideas on the issue and find information about AETs.

Patch management or the expansion of signature databases alone will not be an adequate solution, as they can never keep up with the dynamic threat patterns. With the IPv6, administrators and security mechanisms will face additional challenges. Since each device is assigned its own IP address, an attacker can address any computer directly without making a detour via a cluster address. AETs that take advantage of new vulnerabilities with IPv6 will smooth the way even more for hackers.

Posted by Ash Patel, UK and Ireland country manager, StoneSoft

"Recommended For You"

Nokia adds Snort-based security to appliance DNS: Early Warning System for Cyber Attacks