JP Morgan is now able to run risk analysis and price its global credit portfolio in near real-time after implementing application-led, High Performance Computing (HPC) capabilities developed by Maxeler Technologies.
The investment bank worked with HPC solutions provider Maxeler Technologies to develop an application-led, HPC system based on Field-Programmable Gate Array (FPGA) technology that would allow it to run complex banking algorithms on its credit book faster.
JP Morgan uses mainly C++ for its pure analytical models and Python programming for the facilitation. For the new Maxeler system, it flattened the C++ code down to a Java code. The company also supports Excel and all different versions of Linux.
Back in 2008, it took eight hours to run the end-of-day risk process, a significant part of which was around moving data, manual data clean-up and verification. As part of a sustained investment over the past three years, the trading desk has dramatically reduced this time. If problems occur with the analyses, the process can now be re-run on a pool of several thousand CPUs. Reducing the overall cycle time has meant that speeding up the calculation portion has had significant business impact.
The risk calculation time has now been reduced to about 238 seconds, with an FPGA time of 12 seconds.
“Being able to run the book in 12 seconds end-to-end and get a value on our multi-million dollar book within 12 seconds is a huge commercial advantage for us,” Stephen Weston, global head of the Applied Analytics group in the investment banking division of JP Morgan, said at a recent lecture to Stanford University students.
“If we can compress space, time and energy required to do these calculations then it has hard business values for us. It gives us ultimately a competitive edge, the ability to run our risk more frequently, and extracting more value from our books by understanding them more fully is a real commercial advantage for us.”
The faster processing times means that JP Morgan can now respond to changes in its risk position more rapidly, rather than just looking back at the risk profile of the previous day, which was produced by overnight analyses.
The speed also allows the bank to identify potential problems and try to deal with them in advance. For example, JP Morgan's exposure before and after potential defaults by European central banks on their debt are sensitive to the order in which defaults may happen. Improvement in calculation speed enables the credit trading desk to run massive numbers of potential scenarios to assess such complex exposures.
Weston described this as understanding the "character" of the risk.
“Where we thought the risk in the book was, was more or less where it was, but it had a different character to it. We found that we had a particularly interesting sensitivity to the ordering of our defaults, which we had never been able to explore before.
“It gives us the ability to look at things that we couldn’t look at before and that’s extremely valuable to us,” he said.
As well as being faster, JP Morgan required a system that was more energy efficient. The company has just under a million square foot of raised floor space in data centres, and power consumption was a particular problem.
“Our data centres don’t run out of space, but they do run out of power, power to run the machines, to cool the machines. We needed a solution that was fast, efficient, reliable and less power-hungry.” said Weston.
Instead of using existing standard multi-core machines, JP Morgan adopted FPGA technology to enable it to ‘pipeline’ instructions. This means calculations can be executed very quickly by breaking down calculations into simple components that can then be built into ‘pipelines’.
“[Being super pipelined] we can do a huge volume of calculations more than we can in a traditional CPU environment,” Weston said.
In an initial proof-of-concept project on 450 trades that was representative of the value of the global portfolio, JP Morgan found that it was able to run calculations 30 times faster on a single node. It then built a 10-node box, with two FPGAs in each node.
“We put our whole portfolio (several hundred thousand trades) on that [and] we managed to get the book to run 130 times faster,” said Weston.
“We then built a 40-node computer, which is basically a supercomputer.”
The project took JP Morgan around three years, and the bank is now looking to push it into other areas of the business, such as high frequency trading.
This article has been updated with additional information from JP Morgan since the version that appeared on 11 July 2011.