Virtualisation and the impact on testing and software quality

There are substantial strategic opportunities in leveraging virtual infrastructure to improve the delivery and management of environments for software development and testing.


There seems to be no escaping the “virtual” buzzword at the moment, where everything that was once real now appears to be virtual; software running on virtual machines with virtual memory and virtual storage is used by virtual teams in virtual meeting rooms on virtual LANs. This is largely due to the current large-scale enterprise uptake of virtual infrastructure as a result of its proven benefits, perhaps aided and abetted by the proliferation of articles on the subject.

This trend towards virtual infrastructure has important benefits, implications and opportunities for our IT projects, strategy, quality assurance and ultimately our business competitiveness. The benefits of reduced hardware, power, cooling and administration costs, the associated green credentials and increased flexibility and continuity have been discussed widely.

Disadvantages have also been widely presented including increased server sprawl (exactly what we were trying to reduce), security implications and new performance and management challenges.

In this article we look at the substantial strategic opportunity presented by leveraging virtual infrastructure to improve the delivery and management of environments for software development and testing.

What is virtual infrastructure?

Virtual infrastructure falls into three broad categories:

Server or platform virtualisation: Basically, this enables one physical server to appear as many servers. Ultimately, it has resulted in a paradigm shift from considering servers as a physical commodity to a logical commodity or service. This is the main type of virtualisation that is commonly referred to today. Common implementations include: Microsoft Hyper-V and Virtual Server, VMWare Server, ESX Server, Citrix Xen amongst many others.

Storage virtualisation: This enables the presentation of a single logical abstraction of a pooled set of underlying physical storage. This single view of data enables enterprise wide data de-duplication, backup and restore and management. Examples include: IBM System Storage SAN NetApp, EMC Invista, Veritas Storage Foundation and NetApp Data OnTap.

Network virtualisation: The combination or division of one or more local networks into a virtual network to provide a single logical view of a related set of resources and functionality. This can be done across separate physical resources across a wide corporate network or by grouping together a set of resources within a virtualised server\platform.

The importance of an environment to software quality

At is most abstract, testing comprises defining a set of inputs that will result in a set of expected outputs according to the requirements and specifications of the system under test. In addition to the inputs and expected outputs, testing requires an environment containing the system under test to process the inputs into output. Without this environment testing cannot take place.

The environment comprises a number of elements that define the state of the system under test and enable testing to take place. This state comprises:

Application version(s): This includes versions and patch levels across all application layers from operating system to application frameworks and business applications.

Application(s) configuration: This defines how the applications communicate and operate and can significantly affect the behaviour under test.

Data: The data utilised by the system under test also affects the validity of the testing. If we are considering performance testing do we have a representative volume? Does the data volume have the data variance and distribution we require? Do we have sufficient data to exercise the functionality we wish to test?

Infrastructure: The network, processor, peripherals, storage and their interfaces can all affect the validity of testing. For example, do we require a 64-bit processor or 32-bit processor to ensure our compatibility testing is valid?

It is critical that this environment has a defined state that aligns with the test objectives. This environment definition then becomes another test asset alongside test documentation, test data and test scripts. Without this well-defined environment, testing is unrepeatable and unable to be audited.

Environment management is concerned with the service of delivery of environments in a defined state to support the teams and activities across the software development lifecycle.

Find your next job with computerworld UK jobs

"Recommended For You"

Modernising the data centre Licensing under virtualisation - time for change