What's happening on the enterprise network-or more to the point, what's occurring on the network that should not be-is a major concern of security executives. If someone is trying to hack in, or a virus or worm is spreading, or a denial-of-service attack is underway, there might be evidence of these types of activities before they become a major problem.

Network behaviour analysis (NBA) technology helps organisations detect and stop suspicious activity on corporate networks in a timely manner-possibly preventing, or at least limiting, serious damage from attacks. NBA is designed to give security managers a level of network visibility they need in order to make sure security threats are quickly identified and remedied.

The products analyze network traffic through data gathered from devices such as IP traffic flow systems, or via packet analysis. They use a combination of signature and anomaly detection to alert security and network managers of any activity that appears to be out of the norm, providing a view of the network that lets managers analyze activity and respond before there's damage to systems and data.

"A key benefit of NBA systems is the [network] visibility that they provide," says Lawrence Orans, research director at Gartner, who leads the firm's NBA coverage. Orans says this visibility helps in two areas: network operations (for example, troubleshooting and performance) and security (i.e. malware monitoring and detecting unwanted applications).

NBA can be used to detect behaviour that might be missed by other security technologies such as intrusion prevention systems (IPS), firewalls and security information and event management (SIEM) systems, according to Gartner. Those technologies might not identify threats that they are not specifically configured to look for. Gartner says NBA is suitable as a complementary technology to intrusion detection and prevention software, which is effective for addressing network attacks that can be positively identified.

Vendors addressing the network behaviour analysis market include many of the broader, established network and security companies as well as niche players that specialise in the technology. Those that focus specifically on NBA are Arbor Networks, Lancope, Mazu Networks and Q1 Labs. Companies including Cisco Systems, Internet Security Systems (part of IBM), NetFort Technologies, Sourcefire and Securify (to be acquired by Security Computing) also offer products with some type of NBA capabilities.

Among the common functionality and features of behaviour analysis systems are the use of network flow data to identify suspicious behaviour on the network and where it's coming from; mitigation to stop malicious activity and fix network problems; and reports on all network configurations and user behaviour.

Orans says some NBA vendors are enhancing their products by adding identity capabilities. "Specifically, some vendors have added the ability to map a user [identification] to an IP address," he says. "This provides the benefit of quickly identifying a user who is responsible for anomalous or malicious traffic." So, instead of being notified that a particular IP address is exhibiting anomalous behaviour, a manager can know exactly which user in the organisation is conducting the anomalous behaviour.

"This is especially valuable for forensic analysis," Orans says. "If you are using an NBA system to analyse a breach that occurred in the past-maybe three months ago-then it is often difficult to map the IP address, which is assigned dynamically, to a user. It's difficult unless your NBA system can do it for you.

Before deploying NBA, security managers need to figure out which system is a good fit for their network and how best to use the technology. Here are five tips on evaluating, purchasing and implementing NBA offerings.

1. Before putting in NBA, first deploy intrusion prevention technology.

"NBA systems are best for organisations that have already implemented IPS systems" and are looking for more visibility into their network and network traffic, Orans says. "NBA is not something that you do before IPS or instead of IPS. It is done afterward because it provides visibility."

After successfully deploying IPS and firewalls with appropriate processes for tuning, analysis and remediation, consider adding behaviour analysis to identify network events and behaviour that are undetectable using other techniques, Orans says. He notes that the size of an organisation does matter when it comes to NBA.

"NBA is for large enterprises, it's not for SMEs," Orans says. "The expertise and experience level needed to tune an NBA solution and interpret its results is beyond most SMB network and security professionals."

2. Conduct a thorough analysis prior to selecting a vendor's offering.

It might sound obvious, but NBA systems can cause more harm than good if they're not carefully selected based on the needs of the organisation, existing network components, level of in-house expertise, etc.

When evaluating NBA systems, make sure they meet the organisation's requirements for analysis and reporting, and can be integrated with existing networks. Also, consider how easy or difficult the system is to calibrate and use.

"Think of all the devices you need to collect flows from," says John Kindervag, senior analyst, security and risk management, at Forrester Research in Cambridge, Mass. "Will they all support sending flows? Will enabling flows on the device negatively impact its performance?"

Depository Trust & Clearing Corporation (DTCC), a New York-based firm that provides clearing, settlement and information services for a variety of financial instruments including equities, corporate and municipal bonds, and government and mortgage-backed securities, evaluated several NBA vendors and reviewed market research on the technology within its security department, before selecting a product from Mazu Networks, says Neil Wasserman, vice president, Core and Smart Network Services at DTCC.

"We installed a Mazu demo and ran it through a rigorous evaluation," Wasserman says. "The product met our requirements-and the rest is history."

3. Test before broad rollout.

Experts say it's important to thoroughly test an NBA system before moving ahead with a full-scale implementation. That way, security managers can see what kind of actual reporting they will get on network activity.

"The only way to properly evaluate the tools [is] to install them in your live production network," Kindervag says. "Any other evaluation methodology, lab, etc., will not provide true results."

AirTran Airways, Orlando, Fla., a low-fare airline designed for business travellers, had vendor Lancope conduct an onsite proof-of-concept trial of its StealthWatch product before the system was rolled out broadly, says Michelle Stewart, manager of information security at AirTran. The proof-of-concept "had no impact [on] our production environment and demonstrated the effectiveness of the reporting."

During the implementation, AirTran worked closely with a Lancope engineer and deployed the system according to Lancope best practices, Stewart says.

AirTran's security team uses StealthWatch for network monitoring, reporting and forensics. The network team uses the system to troubleshoot behaviour-related network issues, Stewart says. Managers can examine granular data about network behaviour by zone, node and user, and collect historical data to view trends.

4. Tune NBA systems to cut down on false positives.

Experts says it's important to take the time to effectively tune NBA systems to gather relevant network data and help reduce false positives.

If an organisation fails to fine-tune NBA systems adequately, it might have to contend with a lot of false-positive readings that overburden the network and security managers who need to examine all the alerts.

"We did this tuning exercise immediately upon implementation, and it proved extremely valuable," Stewart says. "After segregating our network geographically and logically into zones, we examined the behaviour within our high-risk zones for volume and type of traffic. In several cases, the port/protocol information we were given from our application vendors was found to be incomplete, but by using StealthWatch we were able to properly fingerprint the application behaviour."

After tuning the zone behaviour policies appropriate to the high-risk zones, "our alarm count was much more manageable and useful," Stewart says. "This information also allows us to properly plan WAN bandwidth growth, as we can determine how much legitimate network traffic is required for business."

5. Use NBA data to help determine network usage patterns.

Stewart says it was important that AirTran managers spend as much time as necessary reviewing the behaviour data gathered to appropriately classify zones, zone policies and services.

"We discovered a great deal about ports, protocols and chattiness of third-party applications during this exercise," Stewart says. "Our zones include geographic segregation, allowing both security and networking to quickly review and treat WAN location issues. We defined server zones by behaviour, allowing more granular control over alerting."

Also, in using NBA systems, it's important to create focused views and logical groupings within the tool that make sense, says Wasserman. "Strive for ease of use and an easy understanding of common sense nomenclature and device or host groupings," he says. "Limit the number of flows that need to be queried or viewed" in order to get useful network information on a more timely basis.

That way, NBA can provide not only greater network visibility, but an effective way to deal with trouble when it arises.