Software testing faces growing skills gap

The accelerating pace of business change and poor information management is leading to a skills gap in software development and testing in the UK. In my 20 plus years in software testing, I’ve witnessed the challenges facing software...


The accelerating pace of business change and poor information management is leading to a skills gap in software development and testing in the UK.

In my 20 plus years in software testing, I’ve witnessed the challenges facing software development and testing on the frontline. Now the industry needs to recognise and act on some of these problems. A lack of skills due to insufficient formal training, the pace of business change and poor information management are all contributing to a skills gap in software development and testing.

The skill-set required for developing and testing software projects is expanding so quickly that many organisations are finding it tough to keep up. Today’s requirement for immediacy of information is becoming an increasing challenge for organisations resulting in minimal time for testing staff to attend formal training. This means that organisations are struggling, simply because they don’t have access to the right skills for the job.

Protect knowledge, protect spend

However, businesses, including many in the UK, are not helping themselves. Time and again, I return to client organisations only to find that they are embarking on another software development improvement exercise in which all knowledge of the previous initiatives have been lost.

I often know more about what they’ve done in the past than the organisation itself does. So we re-do what we’ve already done, because the information or knowledge has not been retained and people from the original initiatives have moved on. It’s great for systems integrators and consultancies, but what a waste of money for the businesses themselves.

Organisations place a lot of importance on their value chain information, such as customer and client data. But all too often the same level of value is not placed on the information created, the knowledge acquired and the lessons learnt during a software development project. As a result, UK companies are wasting large sums of money revisiting software development improvement projects every few years instead of applying the learning from each project incrementally.

There appears to be an underlying belief that once a software development process has been defined then developing a software product can be treated like a manufacturing assembly line. Once we have the process clear, the developers and testers can merrily churn out ‘widgets’. This is completely at odds with the actual skill and knowledge required to work on the often complex and increasingly interconnected systems we find in the majority of large enterprise organisations around the world, including those in the UK.

There’s a world of IP sitting in an organisation; the challenge is now how to effectively manage it. With the move towards the adoption of ‘Agile’ or lightweight software development methods, this IP is now increasingly held in the heads of the project staff and less in static, document-centric formats. As people move around, the static project data just gets lost. Organisations that can manage this tacit knowledge and are able to make it explicit on demand will be more efficient and effective in the future as the need for immediacy of information accelerates.

Mind the (skills) gap

Poor information is not the only driver of this growing software development ‘skills’ gap. With business requirements in this country changing at an accelerating pace, it is increasingly common for software development projects to change tack halfway through, leaving organisations - particularly those with poor support information management strategies - poorly positioned to respond.

Just like their peers in other regions, UK software developers and testers are increasingly too busy to attend formal face-to-face based training and instead are using the tools they have readily available - grabbing the information needed to get the job done. ‘Just in time’ knowledge acquisition via Internet search engines, social media applications and blogs are forming the basis of Personal Learning Environments (PLE) for time-poor software development staff. At the same time, the way the younger generation acquires and uses information, due to influences such as social networking platforms, is impacting formal training.

The ROI from attending a formal training course, I believe, is being questioned due to the realisation that knowledge doesn’t necessarily equate to skill. This certainly is an increasing issue for software testing where testing staff has obtained the relevant certifications, but do not know how to apply that knowledge in the real world projects they find themselves working in.

Agile: Not the panacea and not the bad guy

One response to the rapidly changing business environment has been the widespread adoption of Agile or lightweight development methodologies. These methodologies place the emphasis on breaking down software development projects into smaller, bite-sized chunks, which are then tested and refined on an iterative basis. The more traditional, structured approach to development that tries to predict all the project requirements and potential changes upfront, as well as assuming that very little will go wrong continues to struggle, particularly in rapidly changing business environments.

If you break a project down into smaller, more manageable chunks and work incrementally to achieve the same goals, then you can more easily accommodate any unpredictable changes. This logical approach is actually quite sound and is the way we naturally work - adaptively and incrementally.

However, Agile - and the application lifecycle management (ALM) tools used to support this approach - is frequently seen as a panacea to an organisation’s software development challenges. Successful Agile development requires staff with more skill, discipline and knowledge of both the domain and technologies. These environments also generally need more tool support, which further adds to the list of skills required. Agile also demands significantly higher levels of business involvement on the project and I’m still amazed to find a disproportionate number of organisations attempting to implement Agile where their business is “too busy” to be involved.

With the rush to become Agile, what I’m seeing is we are “throwing the baby out with the bathwater”. The projects from which the Agile methods were documented had a specific context and an organisational ecosystem that enabled these methods to be successful. Taking this concept called ‘Agile’ from one project, plonking it into another and expecting it to work just the same is most likely a recipe for failure.

We are ditching good practice and the lessons learnt from a specific context and organisational eco system. Agile will not address the organisational issues, lack of business involvement, overly complex systems and the poor skills of the staff that existed before. It will however enable you to fail faster.

An example of where we are ignoring the good practices established in the rush to be agile is the use of structured testing approaches and techniques. Structured testing still remains relevant. We shouldn’t ignore the concept just because it has the word ‘structured’ in it. The fundamental testing process followed by a tester is essentially the same, regardless of the development methodology.

If anything, the testing skills and expertise need to be a well-embedded set of intrinsic skills that enable the professional tester to rapidly pick and choose which particular approach and technique to apply. People seem to have lost sight of that and instead want to play with the ‘shiny new toy’ called Agile or Lean without allowing for the learning curve associated with a new way of working.

The right tool - at the right time

UK software developers and testers need to use the entire toolkit they have at their disposal and apply each tool as and when appropriate, rather than adopting one methodology over another. The right tool and the right process in the right context!

Tools are supposed to be enablers, but frequently I see them being ‘disablers’ within the organisation. A tool is often rolled out with minimal training, whether ‘on the job’ or formal because the project or organisation doesn’t have time to train the staff or the budget to utilise mentors. Staff fumble along until frustration and time pressures sees them reverting back to the way they worked prior to the tool being deployed. Usually back to spread sheets and word documents. These tools are far too much of a costly investment to have sitting on the shelf.

For me, a software development project needs to use a set of tools that are appropriate to both the context and the organisation’s ecosystem. By understanding these base constraints, you can pick and choose the most suitable processes and tools. I have a range of tools in my toolkit that I can use in appropriate ways and I think this is the way software development will ultimately need to go.

Tools are not the sole answer to software development challenges. Forward-thinking organisations in this country should stop regarding software development and ALM tools as the answer to their skills problem, but their use will certainly highlight the fact that the right skills may not exist.

This is the wrong way round - the tools will only work if the people have sufficient skills to use them and a deeper understanding on how to apply the tools to the processes and situations they find themselves in on a project. If software development were just a case of creating ‘widgets’, then far more projects would be successful.

Posted by Shane Parkinson, director of Testing Services Group for Capgemini Australia

"Recommended For You"

Borland beefs up ALM toolkit The future of software testing