Following a week which has seen severe disruption to trading, experts claim that banks and exchanges are likely to continue to face IT related outages unless stronger measures are taken to improve the software underpinning trading systems.
As computer based trading has become the norm across exchanges, there have been a number of high profile software related outages. Last year exchange BATS Global cancelled its stock market listing due to IPO software issues, while the reactivation of old code caused Knights Capital systems to make large volumes of erroneous orders on the NYSE securities exchange, costing the trading firm hundreds of millions of dollars. Facebook's high profile IPO also descended into chaos with Nasdaq blaming the “poor design” of software as computer systems used in establishing the opening price were overwhelmed by order cancellations and updates.
In the past week more outages have occurred. Internal software systems believed to be at fault for problems at Goldman Sachs, then, on Thursday, an even more significant IT outage created an ‘unprecedented’ three hour shutdown of trading on Nasdaq’s options market, an event which has since been labelled the Flash Freeze.
One of the commonalities between the major outages occurring in the past 18 months is that, more often than not, they can be traced back to software failures.
Although the initial reports from Nasdaq’s team have suggested that 'connectivity issues' led to the halt in trading, according to Lev Lesokhin at software analysis firm CAST, it is likely that these are symptomatic of problems with the underlying systems.
“In the past we have seen outages blamed on networks that really shouldn’t have happened in the first place – where the infrastructure just really wasn’t designed to cope with badly designed software, and the outage could have been avoided by having better designed software in the first place," he told ComputerworldUK.
“Sometimes the smoking gun is in the network, but the root cause is in the software.”
There have been moves to prevent the recurrence of problems witnessed with the exchanges and banking firms. In the US the Securities and Exchange Commission has begun a consultation as part of plans to improve controls on trading systems in the past 12 months, and, in response to the latest Nasdaq problems, SEC Chair Mary Jo White this week pledged to work to 'push forward' proposals to introduce new standards.
However despite movements by authorities to ensure that procedures are in place for when the unexpected does occur, the regularity with which outages occurs has not been reduced.
According to Chris Skinner, chief exec at financial services professional network Balatro and director of the Financial Services Club, it is almost impossible to completely stop such issues occurring. The problem is that when systems go down, the repercussions can be huge.
“These outages are inevitable as no system is bulletproof," he said. "The banking systems challenge is how to make it just that however: bulletproof."
“Noone is perfect, and the 0.000000001% errors that do occur are normally insignificant enough not to matter but, when they do, they really do. Just look at Knights Capital.”
Furthermore, due to the huge volumes of trades and complexity of markets even highly reslient systems are not immune to failure, so there is always the possibilty they can go down at some point.
"The outage at Nasdaq is not all that surprising given the complexity of the market structure today across equities and equity options," said David Easthope, research director for capital markets at Celent. "Fragmentation combined with the high number of messages relative to trades occasionally overwhelm systems."
He added: "Markets are tightly coupled and interconnected which means problems in data transmission in one market can affect trading across a range of other markets. I won't say this is inevitable - its just common in the market structure we have in the US.
However Lesokhin contends that more can be done to limit major outages, and says that one of the reasons for the recurring problem of faulty software is that not enough investment has been made in ensuring code is up to scratch before applications reach testing phase. This is in part due to the culture within firms, with speed of delivery seen as the top priority.
“The issue here is that the business side of the exchanges and the banks are cowboys, at least up until the last 12 months when the SEC started to put some focus on this. They don’t care about the technology issue, they just throw money at the problem and they push the IT organisation to get things done as quickly as possible.
"They have gotten really cavalier about pushing on the pedal in terms of getting new features and functions in their trading systems without the proper structural software risk oversight. I think that is coming home to roost now.”
While it may be impossible to totally eradicate errors, Lesokhin says that by more closely analysing software before before applications are tested, faults can be identified before systems are launched. This is where regulators need to step in to make such checks mandatory, he said.
“What the industry hasn’t been good at is measuring the software risk. If you ask a CIO on any of the exchanges if they have a measure or scorecard of how risky a software release is which they are about to put into live use, versus the one that they put in three months ago, they don’t have that, and they need to have that.
"I would dare say that is something which should also be regulated – to have some measure of the inherent software risk that resides in their systems."
CAST has responded to the SEC's plans to tighten regulations, and Lesokhin is confident that adequate safeguards will be put in place. However until changes are made, Lesokhin said, the likelihood of more outages such as the one experienced by Nasdaq remains high.
“I would be willing to bet my next vacation that, sometime before the end of Q3, and certainly before the end of the year, we are going to see another several outages like this," he concluded.