Nationwide has revealed that it carried out full site performance testing in a virtual environment, which enabled it accurately model future customer experience of its new online banking website.
The world’s largest building society re-launched its internet banking website six months ago, giving the 10-year-old service a much-needed refresh.
“We were able to prove that the customer experience would be correct at full volume for today and in the projected volumes for two to five years,” said Andy Armstrong, head of technical testing at Nationwide.
“We haven’t had a single performance-related defected since the six months’ launch. That was done against a virtualised service and it’s gone live and worked.”
Nationwide started looking at refreshing its internet banking site two years ago, which it found to be “clunky” and “difficult to maintain”.
“There are about 36 systems we integrate with to provide the [internet] bank to our customers. We had to do performance testing against the environments [but] to do testing in a real environment would have been expensive – it needs a lot of resources,” said Armstrong.
The company ended up using CA’s service virtualisation product LISA in order to carry out the full site performance testing in a virtual environment, though it took some convincing of the business, Armstrong said.
“[One of the challenges was] convincing people this was the way to go – that we could put a new bank live without testing it in a real-time banking environment.
“We started with baby steps, creating web interfaces, and made videos to show how we can scale up.”
Another issue that Nationwide needed to overcome was to find people with ICL mainframe skills, as some of its systems were based on these, to help implement the Java-based LISA tool.
Testing at Nationwide involves a number of stages, some of which involve live testing in a real environment, some in a virtual environment and some combining the two.
The first stage is unit testing, which involves third-party testing of an application to make sure it works before introducing it into the bank. This is followed by systems integration testing, to ensure that the application integrates with the bank’s existing legacy systems. Some of this stage will be done in a real testing environment, to prove that the application will work.
The next stage is non-functional testing, which includes operational testing, such as testing what happens when a direct debit is made overnight, and security testing. User acceptance testing, which is done by professional testers acting as customers, then test the website.
Finally, before a rollout of the new website to customers, Nationwide carried out a pilot for a small group of staff.
Nonetheless, the successful re-launch of the website means that LISA is being used in other projects in the company.
Nationwide has used LISA in about 10 percent of the 120 projects it is currently running at the building society, and is hoping to extend the use of LISA to the application building division, to speed up the overall application development lifecycle.
“We want to enable the [app] building group to improve quality, so that it reduces the testing stage,” Armstrong said.