The US terrorist ‘watch list’ alone includes the names of more than million people. Monitoring air bound passengers may seem like a daunting task for many, however, this issue of managing growing amounts of information is not exclusive to governments, but is a problem for industries worldwide.

In today’s digital world every individual has a trail of information that they leave in their wake, whether it be from using credit cards, computers or making a phone call. The volume of this kind of data is increasing exponentially, along with the locations the data is found in. The challenge for security organisations worldwide is to identify which information is important and then from there sharing the data across agencies.

With so much data to manage, the only way to do this is by increasing the level of automated data integration, with alerts being escalated to individual teams when further investigation is required. Without this, it becomes increasingly challenging to fight terror, particularly to verify that people are who they say they are.

With identity theft and manipulation costing millions of pounds every year this is not an area that can be left unaddressed, either by governments or businesses. Without being able to connect the dots between the data that departments gather and maintain, we will not be as effective as possible in lessening the threat of terrorism.

Unless an organisation is able to accurately identify the individuals it is transacting with, it potentially exposes itself to steep fines or prosecution if it fails to comply with legislative requirements set up to combat criminal activity.

This is where technology has a key role to play, integrating data from multiple sources so that it is reliable and of use to an organisation is the key to help stop terrorist activities, such as the Christmas Day bomber.

Governments have a similar need for a reliable, accurate and consistent identification process, in order to meet the needs of the government agency’s mission; preserve public safety and uphold public trust. How data is integrated and profiled is key to ensuring that the dots are joined up and there are several different technologies that can be implemented to help do this.

Solving the problem with technology

Data integration addresses the need to combine data from multiple sources and provide a unified and accurate view of these. In the war against terror, it ensures that the various ‘watch lists’ managed across departments, agencies and countries would be made accessible. Even when naturally occurring data degradation, such as data entry errors, threaten to further complicate data management, integration solutions can reduce false positives through a series of algorithms, which score the likelihood of a match, and improves match rate through techniques such as the use of name variations.

Identity resolution enables an organisation to search, find, match and group identity data by allowing organisations to connect disparate data sources in order to identify matches as well as relationships across multiple data sources.

When organisations can pinpoint this they can effectively arm themselves against threats to public safety, fraudulent transactions or claims, and potential financial crimes. An accurate identity resolution process can help companies and governments address the issues that are related to a number of wider concerns, such as governance, risk and compliance; law enforcement; conflicting and disparate data sources; balancing resources, risks and returns, and emulating expert users.

In the case of the recent attempt another technique known as complex event processing could have flagged two seemingly independent events - that the ticket was paid for in cash, and the suspected terrorist did not have any checked bags - and generated an immediate alert for additional screening given that he was on a watch list.

Essentially, complex event processing compares multiple events with the goal of identifying the meaningful ones. Having this kind of technology in place would enable responses to possible threats as they occur by giving the right people access to the right information when they most need it.

Identity resolution and complex event processing can work together to deliver real-time Operational Intelligence (OI) from all an organisation’s data sources, enabling valuable decisions to be made and implemented before a threat is realised.

The benefits of these technologies are not limited to the confines of one security organisation, but can and should be realised on a worldwide scale. If governments work together by sharing their data to ensure technologies like identity resolution and complex event processing are used it would help to prevent and predict the next terrorist threat.

Beyond the war on terror

It’s not just at the sharp end of terror that technology can play a part. In areas such as fraud detection and anti-money laundering (AML), identity resolution technology can assist financial institutions in combating identity fraud and leverage name matching of hidden patterns and correlations to prevent attempts to disguise identity.

A single view of this information plays its part in the fight against terror, giving authorities a greater ability to prevent money laundering, which may lead to the funding of terrorist campaigns.

In a world where intelligence is our most valuable asset, governments and organisations need to look again at how multiple sources of global information can be managed effectively to collectively benefit the organisation and aid governments in their battle against terror.

Technology can enable security agencies to cut down the time required to detect and respond to threats, by automating the analysis of data and delivering instant intelligence to an individual or enterprise.

With the costs of missed identity data matches running so high in terms of public safety and state security, IT decision makers along with world leaders need to re-evaluate how technology could help them improve their effectiveness on a wider scale.

John Poulter is Senior Vice President EMEA of data integration company Informatica.