Heathrow’s T5 software testing issues have highlighted the importance of software testing, I hope that lessons have been learnt and that CIO’s across the world will take note in the future.
I believe that in the future will we see changes in 6 key areas of software testing:-
1.Accurate information provision (in the language of the recipient)
Software testing’s role is the provision of data to enable go live decisions to be made. If we get this information wrong, wrong decision will be made. If testing can’t provide the right information it actually adds little value.
Key to this is the understanding of what data is required. So many times I see that Test Managers have just produced what they think is right without asking, or even worse don’t understand the goals of the project so don’t know what data should be collected to indicate whether those goals have been achieved.
Test Management tools can help, but they are only as good as the way they are implemented. In a lot of cases this is straight out of the box, with no configuration or understanding of what is to be reported, with a view that the standard reports will be good enough! Test cases are then loaded and then it’s found that the right data is not available!
I believe the way forward for information provision is actually to learn something from the Agile world and start to work/talk to each other. Everyone’s requirement for information is different and so the closer projects work together and people mix the more we can understand and the closer we will be.
I think a software delivery project is much like an orchestra, if each part works together in harmony it can be beautiful, but if only one element works on its own the result will be disastrous. We need to work together so that the software delivery orchestra works, and delivers beautiful results.
2. Building quality in, rather than testing it out, prevention rather than detection
Traditionally software testing’s objective is seen as finding defects, whereas in reality it is a great way of preventing defects.
It continually amazes me how many clients I see who leave testing until the last possible minute. When will companies understand that the earlier a test team is involved, and of course the better trained they are, the more they will recognise problems early in the lifecycle enabling them to be resolved before they become execution defects?
What do I mean? Well, if you knew your back wall was going to fall down, would you wait until it did or do something to stop it falling. The same can be said for a software tester. If early enough in the lifecycle it is identified that elements of the software may fail would it not be better to stop it failing by some corrective action, rather than to leave it to fail and then checking it has!
What I am referring to hear is a risk based approach to testing, understanding what risks the product has at the earliest possible point and putting in place mitigating actions to lessen or completely remove the risk of failure
3. Ensuring that quality is considered and not seen as a burden, alongside time and cost
Many have said that you can’t balance the three sides of the eternal project triangle, time cost and quality. The view seems to be that you can’t have all three you have to pick two!
In my experience, by adopting a collaborative approach and being able to get involved at the earliest point in a project, it is possible to achieve all three.
Whilst working for a large Insurance Company, I liaised very closely with the Development Manager lending him my testing staff so that he could be sure that when his developers had finished developing the code would work. They asked for a week’s extra time, which I agreed to. When the code was delivered to test it just worked and we therefore reduced the lifecycle timescales by 6 weeks and saved over £400k. The developers were happy, the testers were happy and most of all the business got their software early.
The testing costed roughly the same as the back ended budget, but was spread out from a far earlier point, all of the savings were in development not having to support an extended testing window!
The key to this success was the joint ownership of the quality between all parts of the project team. So by focusing on quality early in the lifecycle you can meet the joint objectives of cost, time and quality.
4. Software Testing Qualifications
The reality is that a good software tester or test manager needs a mixture of skills and knowledge to make them successful. In my view the qualifications account for about 10% to 15% of the knowledge required for the practical application in the workplace.
ISEB have recently revised their single Practitioner Exam, into three, the Intermediate, Test Management and Test Analyst, but only prescribe a 3 day training course followed by an exam. There is minimum entry requirement, but they do not include practical test experience, more a passing of time between taking the Foundation exam.
ISTQB on the other hand have just launched their Advanced level qualifications. Like ISEB there are three of them, Test Manager, Test Analysts and Technical Test Analysts, with a minimum training period of 5 days each followed by an exam, with basically the same entry criteria.
I believe software testing qualifications have a very important part to play in the future education and verification of software testers and test managers. The reality is that the qualification together with real practical experience is what makes a good tester. After all would you allow a surgeon who had just passed their exams but had no practical experience to operate on you? I don’t think so.
Employers need to consider what structures and support need to be in place subsequent to a qualification course to ensure what has been learnt is put into practice in the best possible way.
The few standards currently out there are developed by a wide working international group (can be as many as 1000 contributors) who build and review the content. However, by necessity they are generic and some concessions have to be made to gain global agreement.
It continually amazes me that even knowing that the standards exist, testers prefer to start from a blank sheet of paper when constructing their processes and templates. They may not be perfect but to start from a point above zero has to be better than no help at all?
I believe we need to embrace the standards more and take from them the information that is right for the project.
6. Test Process Measurement Models
There are many test process measurement models available for testers to review what they do against. For some very strange reason, a lot of test consultancies will build their own version, mainly for the creation of a sales opportunity, and not to help the customer. As a result, the models are woefully short of the real detail required to help improve processes.
There are many standard models out there, but in my opinion the most effective model available today is the new TMMi Model. It is a model that has been developed out of research (rather than commercial interest) and therefore provides a very independent and structured view.
It works along the same lines as CMMi, providing 5 levels of maturity to measure against and supporting key process areas.
Again as in the 5th point above, it may not be perfect but like standards it is there and has many hours of experience vested in it, so why do people still start from a blank sheet of paper when looking at their process, when a short cut is available?
Details of this model, which has been developed by the TMMi Foundation, an international organisation whose aim is to establish a non commercial test process assessment model for the industry, can be found here.
Geoff Thompson is consultancy director at software quality consultancy Experimentus.
He serves in several professional IT and testing associations. He is a founder member of the ISEB Software Testing Board, International Software Testing Qualifications Board (ISTQB) and TMMi Foundation, of which he is currently Treasurer. Geoff is also Vice Chairman of the BCS Specialist Group in Software Testing.