To me it means influencing the decision to release software into subsequent environments. Over the last few decades testing organisations have matured from an afterthought in the development cycle to trusted partners consisting of architects, automation technicians, tool administrators, performance, mobile and security experts, analysts and many other infrastructure support positions. This is quite an array of roles through which to assess the ‘readiness’ of software. This article looks at how the traditional role of QA is evolving, in step with the transition towards Automation best practices, by focusing on the FRINGE test practice that will change the face of traditional testing strategies. Fringe tests are those which arewhich are not included in the “Happy path” or “Use Case” but are possibly too complex to automate quickly or are rarely executed; they exist in the ‘Fringe’.
They are the RISK in testing; they are the tests that if time is running out are often omitted from execution and classified as RISK based low priority.
Manual testers that have little to no business knowledge add little to no value to the maintenance of test cases. If the majority of the testing becomes automated in the development phase, their roles, too, become redundant. Many offshore and outsourced solutions are based on the creation of test cases based on static requirements; however, if development is based on short term bursts or sprints of requirement fulfilment there will be little to no benefit to this testing effort. Testers must adapt to their changing environment; testing is expensive and often over flowing with redundancy and process for the sake of process. This article looks at why these changes are occurring and what can be done to align with the change.
Traditionally, when a project is in jeopardy, management look at 3 significant game changers; Scope, Cost and Time. If cost is in danger the cogs that control Time and Scope can be adjusted, likewise if time is in danger the cogs that control Cost and Scope are adjusted and so on. However, the stakeholders may not wish to surrender one or two of the three controlling components to meet the delivery, which means that QUALITY is often the sacrificial lamb used to bring the project in. Sacrificing quality may mean executing fewer test cases, accepting lower performance statistics, and lowering stage gate criteria such as the pass-through of defects or fewer sign offs. Regardless of the path selected RISK is increased and, as such, the end user may not have the software available to them that meets their requirements.
In many organisations, Quality Assurance leadership have built their empires and justify their budget by publishing the number of defects found during development. However, they often overlook those defects found in production; the “Escape Ratio” metrics. This is because QA are traditionally looked at as the gatekeepers of defects, not influencers of Quality. How can this be? QA traditionally track test cases (written descriptions on how to verify coverage of a requirement as defined by the business), however, often stop short of offering value added opinions. This is not true of all QA organisations.
However, with the growing trend for off-shoring the execution of test cases to faceless resources with little insight into the business, and often assigned to multiple projects and corporations, this is becoming the norm!
Development efforts which align to the principles of Agile, and utilise best practices from Scrum and eXtreme Programming, hold Quality as their primary target. Therefore, teams are tasked with working diligently to increase both development and product quality. QA contributors are considered to be an integral part of these teams and have an equal say in the estimation and direction of all tasks.
Other significant advances made within QA, outside of closer integration points with others in the development practice, include the distinct practices of Automation. QA Automation is distinctly different from Development Automation; however the practices can overlap and should be promoted aspromoted as an on-going improvement. QA Automation is predominantly used for Functional regression.
QA Automation is expensive and ROI can be significant, however, teams that plan to develop functional automation on code that is still in development (not in Production) take huge risk. This risk is comparable to coding whilst requirements are unavailable or likely to change, meaning that code would need to be updated and revisited. Functional automation should be created over a stable code set. Therefore, QA Automation fits perfectly into Regression Testing; areas that have been manually tested and approved for advancement into subsequent environments. Otherwise the cost of developing the automation rarely justifies the cost in effort.
Development automation, such as Unit testing, is a method by which individual units of source code tests, containing one or more program modules together with associated control data, usage procedures, and operating procedures, are tested to determine if they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application. Unit Testing aids coders by ensuring the tests which confirm their code meets their understanding of the requirement. Unit Tests can also be run and rerun in a build environment, where the failure of a test will alert the developer that a change may have a serious impact on the unit of code being developed. Test Driven Development (TDD) looks for coders to create the Unit Test first, then create code to meet business requirements and align to the test criteria.
Business Requirement: As a customer with no account, enable the user to access the public log in site where they can register as a new account holder and gain access to the common landing page. Ensure that general security on the log in page is upheld where credentials are validated for an existing account.
TDD coder: The coder will assess the requirements and break them down into technical subsets; “Validate e-mail entered on log in page does not exist as a customer”; creating code units such as “Email validate”, “LogInPage_loaded”, “Valid_Customer”, “InvalidCustomer” and more.
TDD is one of the fastest growing practices in development today, as it increases the Quality of code by: enabling the automation of tests to occur earlier, improving communication between individuals andindividuals and reducing the number of defects possibly found in test phases, as well as the costs and delays associated with defects.
Another significant practice that increases the use of TDD is Behavioural Driven Development (BDD). BDD focuses on obtaining a clear understanding of desired software behaviour through discussion with stakeholders. It extends TDD by writing test cases in a natural language that non-programmers can read. Behaviour-driven developers use their native language in combination with the ubiquitous language of domain drive design to describe the purpose and benefit of their code. This allows the developers to focus on why the code should be created, rather than the technical details, and minimizes translation between the technical language in which the code is written and the domain language spoken by the business, users, stakeholders, project managers, etc.
As suggested earlier, functional automation is expensive and, as such, better suited to executing on a stable set of code later in the development cycle as regression suites. However, with the introduction of TDD and BDD, it becomes apparent that automation testing is not only happening earlier in the development cycle but thatitthat it will potentially remove much of the manually performed testingperformed . testing. If this becomes reality, what happens to the teams of manual testers, their infrastructures both on- and offshore, the functional automation testers and the metrics that tell management the current state of testing?
As our title alludes to, the answer is to ‘Say Hello to QA and Goodbye to Testing’. Quality of code is determined by the end user. The business stakeholders may often determine the WANT functionality for the customer but it is the customer that determines the NEED functionality; a gap that if closed has the potential to save much of development effort. We all hear that when a project is over-running, it is testing that is sacrificed. What if testing could be part of the development effort? What would be sacrificed then?
The answer is SCOPE. As we know stakeholders do not like to reduce scope to bring project in on time or on budget. By influencing more of the traditional testing to be executed during the coding function, TDD and BDD, testers add value by acting as consultants to the coding teams with business knowledge and reviews, leaving them available for “Fringe” testing.
“Fringe” testing is the future of the traditional tester. With the inevitable evolution of development based testing practices, the testers role will change and focus on the following:
- Consultant business case advisor
- BDD use case scripter
- Acceptance Test validator/Stage Gate sign off
- Negative test execution
- Fringe Test manager
- Test Data Engineer
- Defect management
Organisations have built large QA infrastructures to support testingsupport testing, vendor-supplied tools to manage test cases, requirement coverage, execution of manual and automated scripts, defect management and metrics to validate the existence of the infrastructure. There is a “feel good” factor that lots of testing builds better products. This is untrue; testing does not build better products, better development methods and earlier testing build better products.
The role of the tester must transition or disappear. Those with product knowledge are in prime position to support the automation effort and author Fringe and negative test cases. Those with data knowledge can transition into the role of Test Data Engineer; a role that combines product, database and test case scenario expertise.
As a consultant in the world of Agile principles, QA methodologies and test data governance, I see the signs of a changing landscape in QA. I see a diminishing return on separating testing professionals from the development effort as the development practices align to highly effective mechanisms via Collaboration, Communication and Quality-centric metrics. I recommend reviewing the roles of your QA teams and refocusing their principles based on quality initiatives and not testing metrics.
Posted by Ray Scott, GT Agile