White PapersThe Standish Group’s Chaos Report and other studies describe what has been the industry reality for decades: The majority of software projects fail to achieve schedule and budget goals. The poor quality of software is one of the biggest reasons behind these many failures, which often result in massive rework of application requirements, design and code. Such rework extends release cycles and consumes significant additional budgets. To combat the high rate of project failures, it is therefore necessary to better understand the reasons behind the poor quality of the products being developed.
Accumulated industry experience and numerous studies show that the two primary reasons behind poor application quality are defects in requirements specifications and problematic system test coverage. The following sections provide more information about these issues.
ELIMINATE THE TESTING BOTTLENECKMaximize software quality with Requirements Based TestingUntitled DocumentEliminate the testing bottleneck2ContentsExecutive summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3The quality crisis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Errors in requirements specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4Problematic test coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Introducing Requirements Based Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Test early and frequently . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Test with your head, not with your gut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Test with measurement and improvement in mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6The Requirements Based Testing process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Ensuring quality of requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7Designing logical test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8Ensuring quality of test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8Ensuring quality of design and code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8Completing and executing tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9Traceability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9Summary of Requirements Based Testing process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9Implementing Requirements Based Testing with Borland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Borland RBT solution architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Reviewing requirements against business objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11Detecting ambiguities in requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11Applying formality and structure to requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11Reviewing requirements and test cases with various stakeholders . . . . . . . . . . . . . . . . . . . . .11Mapping requirements against use cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12Generation and optimization of test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12Tracing requirements to test cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13Completion and execution of test cases against code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14Borland RBT process consulting support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15Untitled DocumentEliminate the testing bottleneck3Chief among the reasons behind this problem is the fact that, in many software organizations, quality is handled as anafterthought. In such organizations, quality verification efforts typically begin only once code has been completed. At thispoint, clearly the testing team is under the gun to certify the application as quickly as possible, and such certification is oftenperceived as a bottleneck that prevents deployment into production. In this environment, it is difficult to ensure that therequirements are correct; to properly plan tests; to ensure correctness, completeness and solid coverage of requirements; andto gain visibility into the various quality aspects of the tested application. It is no wonder that frequently this proves to be acostly and frustrating exercise to all involved stakeholders.This white paper presents the fundamental principles of Requirements Based Testing (RBT), an approach designed toestablish a radically different testing process that:" Is tightly coupled to application requirements" Is integrated through various activities throughout the software lifecycle" Delivers measurably high, systematic and cost-effective requirements and test coverageBy focusing on the above principles, RBT eliminates the quality bottleneck and ensures that quality is no longer treated asan afterthought.The paper also explains how Borland solutions can help your organization overcome the challenges associated with RBT.The quality crisis The Standish Group s Chaos Report and other studies describe what has been the industry reality for decades: The majorityof software projects fail to achieve schedule and budget goals. The poor quality of software is one of the biggest reasonsbehind these many failures, which often result in massive rework of application requirements, design and code. Such reworkextends release cycles and consumes significant additional budgets. To combat the high rate of project failures, it is thereforenecessary to better understand the reasons behind the poor quality of the products being developed.Executive summaryMost business and IT executives agree that any company that is able to rapidly deliver softwareof high and predictable quality with minimum budgets enjoys a significant advantage over itscompetitors. However, practical experience shows that the challenges associated with softwarequality remain largely unsolved: Despite massive investments in testing automation, only a smallfraction of the IT application portfolio enjoys sufficient test coverage. It is also apparent thatmany applications eventually do not meet their users requirements. These problems lead to pooruser experience, lack of adoption, massive application rework and ultimately loss ofcompetitive advantage.Untitled DocumentEliminate the testing bottleneck4Accumulated industry experience and numerous studies show that the two primary reasons behind poor application qualityare defects in requirements specifications and problematic system test coverage. The following sections provide moreinformation about these issues.Errors in requirements specificationsFor some, it may be surprising that a system can be thoroughly and successfully tested, yet make its users miserable. Thereason is simple: The development team got the requirements wrong.Requirements of complex software applications are often negotiated through two concurrent dialogues, which continuouslyevolve throughout the project lifecycle. These dialogues are focused around two questions: What do we need to build? Whatcan we build? The quality of these dialogues often determines the ultimate quality of the constructed application.Unfortunately, along the way something often goes very wrong. A study by James Martin1shows that the root causes of 56percent of all defects identified in software projects are introduced in the requirements phase. Further, it states that about50 percent of requirements defects are the result of poorly written, ambiguous, unclear and incorrect requirements. Theother 50 percent of requirements defects can be attributed to incompleteness of specification i.e., requirements that weresimply omitted.Figure 1: Distribution of defects in software projectsOther statistics demonstrate similar problems:" 82 percent of application rework is related to errors in requirements2" Problems in requirements represent 44 percent of the reasons behind project cancellations3" Only 54 percent of initial project requirements are actually realized4To summarize, the top two issues with quality of requirements are:" Requirements and specifications are incomplete, due to lack of input from users and other key stakeholders." Requirements are specified poorly, by domain experts who lack skills to define consistent, accurate and testablerequirements, and whose tools don t allow for efficient validation and/or review of the requirements with the users.5 6 %R e u i r e m e n t s D e s i nC o d e O t h e r1 0 %7 %7 %1James Martin, An Information Systems Manifesto 2 James Martin, An Information Systems Manifesto 3 The Standish Group: CHAOS Report4 The Standish Group: CHAOS ReportUntitled DocumentEliminate the testing bottleneck5Problematic test coverageTesting is a tedious and time-consuming activity, requiring careful planning and thorough execution. Achieving good testcoverage is a key goal of testing: the better the test coverage gets, the better the chances to catch defects in the testing phaseand cover all important and corner scenarios. Unfortunately, even if requirements are properly stated (i.e., they arecomplete, accurate and testable), it is still difficult to deliver optimal test coverage.The following reasons make good test coverage exceptionally hard to accomplish:" Tests are often conducted at the end of the development process, as schedules become tighter and tighter. As aresult, test planning and execution are often performed with severe time constraints, which make it more difficultto effectively cover a large percentage of the system s functionality." The complexity of modern applications (and in particular distributed applications) makes it very hard to cover allof the possible scenarios, due to the sheer number of alternative paths through the application. Any attempt tosystematically (or automatically) consider all possible combinations results in a set of test cases that is simply toolarge to manage and takes a very long time to execute (which means further delay in certification efforts)." Application requirements change frequently, but their changes are not properly managed. Since traceabilitybetween requirements and test cases is often not properly maintained, it is difficult to ensure continuous testcoverage of changing requirements.As a result of these issues, existing methods for creation and selection of test cases are mostly informal, and rely heavily onthe experience and skill of testers. It is uncommon that test case design is performed in a systematic and rigorous manner.Many cases are designed based on gut feel and intuition, which is one of the major reasons why software quality andrequired testing effort tend to be unpredictable.Introducing Requirements Based TestingRequirements Based Testing (RBT) is an approach that is implemented through well-defined processes, which target the rootcauses of the quality problems mentioned previously. RBT is based on the following principles:" Test early and frequently." Test with your head, not with your gut." Test with measurement and improvement in mind.Test early and frequentlyRBT promotes a testing process that is integrated throughout the entire software development lifecycle. As soon asrequirements, design and code artifacts are ready, they are reviewed and tested against best practices, business objectives,requirements, use cases and test cases. Testing becomes a parallel activity to the development process, a constant pursuit thatspans all roles and makes all stakeholders aware of quality objectives.As a result, testing is no longer a bottleneck activity, performed after code is delivered under extreme time pressures. Manydefects are detected at earlier development phases, when it is much cheaper to fix them. Additionally, there are significantlyfewer surprises when the code is delivered.Untitled DocumentEliminate the testing bottleneck6Test with your head, not with your gutRBT supports methodical and systematic design of test cases, to ensure predictable test coverage. Following a rigorousprocess, RBT attempts to facilitate test case design that does not rely on the skill or experience of specific testers. Rather, RBTstrives to inject repeatability and method to the test planning process in order to make test coverage more predictable. RBTalso attempts to apply various optimization techniques to produce the minimum number of test cases required for sufficienttest coverage. By doing this, RBT makes the testing cycle more rapid and manageable.Test with measurement and improvement in mindRBT promotes a quality process that can be managed and improved through measurement. Throughout this process,multiple measures help to quantify the status of deliverables and activities. This helps managers and process experts overseequality initiatives across the IT application portfolio.The Requirements Based Testing processRequirements Based Testing was perceived and refined through practical industry experience as well as academic research. Tovarious degrees, it is widely implemented by many organizations to ensure that they define software requirements and designtest cases that provide good test coverage and ensure that quality activities are well integrated throughout the entiredevelopment process.Rather than focusing on the traditional testing activities, which are typically performed after code is delivered, RBT emphasizes the full lifecyclequality activities. These activities are aimed at increasing the quality ofrequirements and test cases, as well as encouraging the utilization ofrequirements and test cases in other phases of the software lifecycle, suchas design and development. This focus ensures that when tests are finallyexecuted, they are truly associated with initial business requirements.Untitled DocumentEliminate the testing bottleneck7The following diagram shows the RBT activities and areas of focus of the RBT process.Figure 2: Requirements Based Testing process flowThe following sections provide additional information about each of the RBT categories of activities.Ensuring quality of requirementsBecause the success of a software project can be directly associated with a good understanding and articulation of itsrequirements, RBT places a great deal of importance on the quality of requirements.To ensure that business needs are eventually met, RBT includes the validation of requirements against businessobjectives. This also optimizes project scope by ensuring that each requirement satisfies at least one business objective. Ifthere is no match between the requirements and business objectives, refinement is necessary.To enable involvement of end users and domain experts, RBT recommends a review of the requirements by thesestakeholders. Feedback of users and domain experts is used to refine the requirements before additional work is done.To validate completeness of the requirements, RBT calls for mapping requirements against a task-oriented orinteraction-oriented view of the system, often captured by use cases. If one or more use case cannot be addressed by therequirements, then the requirements are not complete.To guarantee the consistency and clarity of the requirements, RBT applies language analysis techniques to detectproblematic phrases in the requirements and fix them. A problematic phrase is language that is ambiguous, unclear orinconsistent. Such phrases can be interpreted in multiple ways by different people, which leads to confusion and errors downthe road. Ambiguous terms also result in requirements that are not testable.R e q u i r e m e n t s Q u a l i t yV a l i d a t e a g a i n s tB u s i n e s s O b j e c t i v e sM a p a g a i n s tU s e C a s e sR e v i e w T e s t C a s e sb y R e q s A u t h o r sR e v i e w T e s t C a s e sb y D o m a i n E x p e r t sC o m p l e t e T e s t C a s e sE x e c u t e T e s t sV a l i d a t e dR e q u i r e m e n t sF i xR e q u i r e m e n t sV a l i d a t e dT e s t C a s e sL o g i c a lT e s t C a s e sC o d eR e v i e w T e s t C a s e sb y T e s t E x p e r t sR e v i e w T e s t C a s e sb y D e v e l o p e r sR e v i e w D e s i g n w i t hT e s t C a s e sA m b i g u i t yA n a l y s i sD o m a i n E x p e r tR e v i e w sS t r u c t u r e / F o r m a l i z eR e q u i r e m e n t sD e f i n e / O p t i m i z eT e s t C a s e sT e s t C a s e s Q u a l i t yL o g i c a l T e s t C a s e D e s i g nT e s t E x e c u t i o nD e s i g n & C o d e Q u a l i t yUntitled DocumentEliminate the testing bottleneck8Designing logical test casesTextual requirements are very important, given that almost all business analysts and domain experts use natural language toexpress requirements. However, using textual requirements makes it more difficult to achieve and validate high test coverage.When requirements are expressed in a nonformal fashion using natural language, testers can use only their intuition to claimthat the set of test cases they designed covers 100 percent of the functionality captured in the requirements. In other words,with nonstructured textual requirements, there is no formal way to show complete test coverage, and therefore corner oreven major scenarios often remain untested until defects occur in production.To meet the objective of systematically achieving high test coverage, RBT introduces a process step aimed at creating moreformal and structured representations of the requirements. Once this is done, it is possible to define logical (skeletal) testcases and ensure optimal coverage of requirements, as these logical test cases are eventually evolved into the actual set of teststhat are run against the system.Multiple techniques can be used to provide structure and formality to natural language requirements. The purpose of thesetechniques is to reveal cause-effect relationships embedded within requirements; that is, to express requirements as a set ofconditions (causes) and resulting actions (effects). Cause-effect charting is one of these techniques. Another way to achievesimilar goals is to express requirements as flowcharts, since they naturally depict precedence dependency between actions aswell as conditional branching of activities.Once requirements are expressed in a structured manner, it is much simpler to derive the equivalent test cases for therequirements. A set of logical test cases can be defined (manually or automatically), which is 100 percent equivalent to thefunctionality captured in the requirements. However, this set of test cases may include many redundant cases (that is,overlapping with other test cases). To optimize the number of test cases but still provide full coverage, techniques such asdecision (truth) tables can be applied if cause-effects charts were used to structure the requirements. If flowcharts were usedfor that purpose, then generation of the optimal set of test cases means finding all unique paths on the flowchart, for whichthere are known techniques.Ensuring quality of test casesOnce test cases have been designed, there is a probability that some errors still exist in them. To detect errors in test cases andfix them, RBT incorporates reviews of the test cases by the authors of the requirements and by end users and domainexperts. This phase gives these important stakeholders an opportunity to review the requirements and the test cases in theirstructured form, and catch any defects that were introduced in the transition from natural language to formal form.Ensuring quality of design and codeRBT goes an extra mile to integrate the quality process into the development process, using the structured, validated, high-quality test cases that were generated in the previous phases.To ensure that the design is robust enough to satisfy the requirements, it is reviewed against the logical test cases(as they are simply a different representation of the requirements). If the design cannot meet requirements, then either therequirements are problematic (and additional refinement is necessary) or the design requires rework.To ensure that developers understand what will be tested, and to provide them with a structured view ofrequirements, RBT recommends that test cases be reviewed by the developers (coders).Untitled DocumentEliminate the testing bottleneck9To ensure that the delivered code conforms with the requirements, RBT recommends that individual code modulesbe reviewed against the structured requirements that trace to them. It is much easier to match the algorithmic expressionscaptured in code to the formal view of requirements than to compare it to their nonstructured view.Completing and executing testsUsing the logical test cases, testers can now complete them by defining data inputs and navigation flows, potentially usingtest automation tools. To compare the actual behavior of the system to the expected behavior, the complete set of fullydefined tests can now be executed against the code.TraceabilityWhile not exactly a phase in RBT, traceability has an important role in its success. With RBT, maintaining traceabilityinformation between requirements and test cases and tests is crucial. This information is required for monitoring progressand coverage, as well as properly managing the impact of changes in requirements. Without it, it is more difficult todetermine which test cases or tests should be changed if a specific requirement changes.Summary of Requirements Based Testing processThe table below summarizes the RBT goals, associated objectives and how they are met.GoalsObjectivesMethod UsedEnsuring quality of requirementsEnsure that business needs are eventuallymet, enable involvement of end users anddomain experts, validate completeness ofthe requirements, guarantee the consistencyand clarity of the requirementsReview of requirements against businessobjectives, review of requirements bydomain experts and business users,mapping requirements against use cases,ambiguity analysis of requirementsDesigning logical test casesProvide structure and formality to naturallanguage requirements, derive theequivalent test cases for the requirements,optimize the number of test casesExpress requirements using cause-effectcharts or flowcharts, automated or manualgeneration of test cases, decision (truth)tables or unique path detectionEnsuring quality of test casesTo detect errors in test cases and fix themReview test cases by requirements authors,domain experts and end usersEnsuring quality of design and codeTo ensure that the design is robust enoughto satisfy the requirements, to ensure thatdevelopers understand what will be tested,to ensure that the delivered code conformswith the requirementsReview of test cases by designers anddevelopers, use test cases in code reviewCompleting and executing testsTo compare the actual behavior of thesystem to the expected behaviorFull definition of test cases, execution oftest cases against delivered codeUntitled DocumentEliminate the testing bottleneck10Implementing Requirements Based Testing with BorlandBorland supports the RBT process with a wide array of products, professional services and educational offerings. Borlandtechnology provides a broad basis of support, with products that enable effective gathering and validation of requirements,efficient management of requirements and their traces to other work products, and efficient development of test cases toverify and validate the requirements. Borland training provides skill development in planning quality activities, performing awide variety of work product reviews, and effectively using the suite of Borland technology. Borland consulting offeringsenable organizations to identify the primary gaps in their lifecycle quality management practices and iteratively implementappropriate processes to remove the gaps, melding in the needed skills development and enabling technology along the way.Borland RBT solution architectureBorland requirements management and definition technology supports many of the important concepts of RBT. TheBorland Caliber DefineIT" product can be used to elicit and structure requirements, and to generate and optimize testcases. Once requirements have been properly defined, Borland CaliberRM" is the solution of choice to managerequirement changes and their traceability across the lifecycle. Integrations of Borland s requirements tools with Borland Together and Borland SilkCentral Test Manager products enable designers and testers to efficiently participate in theRBT process (see Figure 3).Figure 3: Borland Requirements Based Testing solution architectureC a l i b e r D e f i n e I TR e q u i r e m e n t s E l i c i t a t i o nR e q u i r e m e n t s S t r u c t u r eF l o w C h a r t sT e s t C a s e G e n e r a t i o nC a l i b e r R MG l o s s a r i e sB a s e l i n e sT o g e t h e rU s e C a s e M o d e l sA c t i v i t y M o d e l sS i l k C e n t r a lT e s t P l a n n i n gT e s t M a n a g e m e n tA p p r o v a l sT r a c e a b i l i t yA u t o m a t i c G e n e r a t i o no f U M L M o d e l sA u t o m a t i c G e n e r a t i o no f T r a c i n g o f T e s t C a s e sT r a c i n g R e q u i r e m e n t st o M o d e l E l e m e n t sT r a c i n g R e q u i r e m e n t st o T e s t C a s e sB i d i r e c t i o n a lS y n c h r o n i z a t i o nUntitled DocumentEliminate the testing bottleneck11The following sections explain how organizations interested in RBT can automate significant parts of the process usingBorland products and integrations.Reviewing requirements against business objectivesBorland Tempo enables business and IT teams to collaborate by capturing and managing business objectives, as well asensuring that a proper process for selecting and kicking off IT projects is in place. Borland s process and training offeringsprovide guidance in how to gather and describe business objectives, business requirements and system requirements. Oncerequirements are articulated, RBT practitioners can use Tempo to evaluate them against agreed-upon business objectives.Detecting ambiguities in requirementsTo help requirements authors detect and resolve ambiguities in requirements, Borland provides skill-building workshops inpeer review techniques for carefully examining key work products (such as requirements, designs, code and test cases) toeliminate defects and rework in the development lifecycle. For some teams, very formal reviews (inspections) areappropriate; for others, simpler walk-throughs or technical reviews may be appropriate. Borland workshops help teams todecide when to plan and execute each type of review.To support these reviews, CaliberRM enables teams to include glossaries of terms that help work product developers andreviewers minimize ambiguity. CaliberRM glossaries enable business analysts to define multiple dictionaries, which includewords that are considered bad terms, and those that should not be used when defining requirements (because they are toogeneral or ambiguous). Glossaries also make it possible to define common terms that are often used in requirements, suchthat all roles have a clear and uniform understanding of their meaning. Once glossaries are in place, CaliberRMautomatically highlights requirements text that matches glossary terms, and provides users with helpful tool-tip data.Applying formality and structure to requirementsBorland Caliber DefineIT helps business analysts structure requirements while capturing them through the definition orelicitation process. DefineIT structures requirements as scenarios, each of which defines a specific high-level interactionwith the application, and consists of several steps. Every step represents a fine-grained action within the scenario.DefineIT shows two structured representations for a scenario: a textual representation of the scenario s steps and agraphical representation of the scenario s steps, which is a flowchart that captures the sequence of steps within a scenario andtheir branching logic. The textual and graphical representations of a scenario are kept synchronized at all times (that is,actions performed on a specific representation are immediately shown in the other one).Since requirements are structured in DefineIT, it is possible to validate their logical consistency. DefineIT provides avalidation pane that shows messages about potential logical problems with a scenario. As an example, if a decision point isadded to a scenario, but all branches lead to the same step, then the decision point does not make sense. DefineIT notifiesthe user about this as well as other problems, and thereby helps to ensure that the scenario is defined in an accurate andlogical manner. The validation pane is updated in real time, as users manipulate the scenario.Reviewing requirements and test cases with various stakeholdersRBT includes the review of requirements and test cases with multiple stakeholders, including domain experts, end users,designers and developers. These review sessions should provide both nontechnical and technical audiences with a clear andconcise understanding of expected system behavior. Their input should be captured and used to iteratively improve andUntitled DocumentEliminate the testing bottleneck12refine the requirements. Borland peer review workshops enable stakeholders and team members alike to have the skills toperform such reviews effectively.The main goal of DefineIT is to close the language gap between business and IT, by facilitating a more structured dialoguebetween the two teams. Business stakeholders are often domain experts, speak the domain language, but lackunderstanding of technical terminology. IT stakeholders are well versed with the technical terminology, but often lackexpertise and understanding of the problem domain. DefineIT synchronizes the two teams by facilitating an accurate andcomplete knowledge transfer from analysts and business stakeholders to the technical team.Caliber DefineIT elicitation capabilities are designed specifically to elicit requirements from various stakeholders through awell-structured review process. Once a scenario is modeled in DefineIT, the storyboarding feature of DefineIT can be used toshow it to the application s end users. This helps to validate that the captured scenario is indeed in sync with whatstakeholders expect.Storyboarding a scenario with DefineIT essentially means walking reviewers through multiple paths defined by the scenariosteps and decision points, while capturing the reviewers feedback and comments, which can be used to improve the scenario.Mapping requirements against use casesThe integrations of CaliberRM and DefineIT with the Borland Together modeling product enable business analysts andarchitects to collaborate and ensure that requirements and use case models are synchronized, such that the software design isbased on validated and accurately specified requirements.This integration of DefineIT and Together enables users to export a DefineIT project to Together as a set of UML-compliantuse case and activity diagrams. This capability essentially transforms the DefineIT scenario representation into equivalentUML models. The resulting diagrams include links to reference materials provided in DefineIT, such as images, and thesecan be directly viewed from within Together.Users can also export DefineIT projects as Business Process Modeling Notation (BPMN) diagrams through OMG s QueryView Transfer (QVT) specifications. DefineIT users can export a project as XMI, and architects can use the QVT capabilitiesof Together to transform it into a BPM diagram. After the BPMN diagram is created, it is also possible to use Together togenerate BPEL for execution.Generation and optimization of test casesCaliber DefineIT can automatically generate a set of test cases for a specific scenario within a project. By doing this, DefineITenables the functional test coverage of the application.DefineIT also helps minimize the number of generated test cases. Each generated test case represents a unique path withinthe scenario. Selecting a specific test case highlights the associated path within the scenario diagram.DefineIT generates a textual description of all the steps that need to be taken within the test case; and for each, specifies theActor, the action taken and the expected result. Generated test cases can be exported to Borland SilkCentral. It is alsopossible to export generated test cases to HTML, which creates an interactive and navigable website where the QA team canview test cases and their associated artifacts.Untitled DocumentEliminate the testing bottleneck13Tracing requirements to test casesBoth CaliberRM and Caliber DefineIT enable testers to associate their test cases with requirements, to ensure that their testsuites ultimately provide full coverage for the complete application scope. This capability is supported via integrations withthe Borland SilkCentral Test Manager.The DefineIT integration with SilkCentral makes it possible to automatically transfer requirements and test cases generatedby DefineIT to SilkCentral. Exported requirements appear in the selected project tree, and for every requirement the testcases and test steps are exported as well. Once export is done, test cases are automatically linked to their respectiverequirements within SilkCentral.CaliberRM provides similar integrations with SilkCentral. Additionally, it provides a robust set of traceability visualizationand analysis tools." The Traceability Matrix is a powerful interactive table that shows all traces between requirements, modelelements, code and tests in a matrix format." The Traceability Diagram is an interactive tool that shows a visual graph of trace dependencies betweenrequirements and other lifecycle elements including test assets.CaliberRM is able to visually mark all traces to or from requirements that were changed as suspect. This capability is key toimpact analysis, as it helps testers quickly find and access all test artifacts that are affected by a change in requirements. Theycan then access impacted test cases in SilkCentral and modify them to cover for the changed requirements.Completion and execution of test cases against codeOnce skeletal test cases are generated by DefineIT and exported into SilkCentral, it is possible to provide detailed testdefinitions and run a series of functional and performance tests using Borland SilkCentral, SilkTest and SilkPerformer. If theRBT process has been followed, at this point the testing team can be assured that test coverage is optimal.Untitled DocumentEliminate the testing bottleneck14SummaryThe table below maps Borland s products and services to the required RBT process components.Borland RBT process consulting support Borland s Lifecycle Quality Management (LQM) solutions are supported by a range of process development and skill-buildingofferings, as well as Borland technology. The specific solution for any customer situation depends on the current state of LQMand the goals for the organization. Borland uses its Accelerate framework to architect and deliver such solutions.Specific elements for LQM include the following:" Executive Workshop for LQM. Several-hour session for organization executives to review industry best practicesfor LQM and set business goals for their organization with respect to improvements in LQM." LQM Gap Analysis Assessment. In-depth analysis of the organization s current LQM processes, skills and technologyRBT Process ComponentSupporting Borland ProductGathering and documenting requirements effectively (all levels)Requirements Gathering, Definition, Management andEngineering workshopsReview of requirements against business objectivesTempo, Peer Review Workshop, CaliberRMReview of requirements by domain experts and business usersPeer Review Workshop, Caliber DefineIT / elicitationMapping requirements against use casesCaliberRM and DefineIT Together integrationsAmbiguity analysis of requirementsPeer Review Workshop, CaliberRM glossariesExpressing requirements formally using cause-effect charts orflowchartsCaliber DefineIT / flowchartsAutomated or manual generation of test casesCaliber DefineIT / auto test-case generation; SilkTest andSilkPerformer trainingOptimization of number of test cases using decision (truth) tablesor unique path detectionCaliber DefineIT / flowcharts; SilkTest and SilkPerformer trainingReview test cases by requirements authors, domain experts andend usersPeer Review Workshop, Caliber DefineIT / elicitationReview of test cases by designers and developersPeer Review Workshop, Caliber DefineIT / elicitation, SilkTest andSilkPerformer trainingCompletion of test cases, execution of test cases against deliveredcodeBorland Silk" product line and trainingTraceability of requirements to test casesCaliberRM and DefineIT integrations with SilkCentral Untitled DocumentEliminate the testing bottleneck15against industry best practices, to identify where the organization might benefit most from improvements; durationdepends on the number of areas examined and size of the organization (generally 3 to 10 days)." Action Planning Workshop. Three-day workshop for the team, which implements the recommendations thatcome from the gap analysis." LQM Consulting. Guidance for the development and deployment of improvements to skills, processes andtechnology, as well as assistance with organizational changes." Peer Review Workshop. Two-day workshop teaching a range of types of work product reviews and how to selectan appropriate style for different purposes; includes hands-on experience doing formal inspections." Silk Testing Suite product and practices training. Full set of training offerings to enable effective use ofSilkCentral Test Manager, SilkTest and SilkPerformer.ConclusionBy integrating quality activities with the overall software development cycle, Requirements Based Testing can help organizationsto systematically deliver predictable levels of software quality. With RBT, quality is not treated as an afterthought, nor are testingefforts perceived as a bottleneck. Borland solutions and process consulting services alone or complementing offerings fromother leading testing vendors can significantly assist with rollouts of Requirements Based Testing.Copyright 2006 Borland Software Corporation. All rights reserved. All Borland brand and product names are service marks, trademarks or registered trademarks of BorlandSoftware Corporation in the United States and other countries. " 24858
