Fundamentals of Software Testing

Bernard Homès, founder of the ISTQB and an IEEE Senior member, has over 40 years of experience in IT and software testing. He headed the design for the Advanced Level syllabus and holds numerous testing certifications. This is his eighth book on software testing, which will once more focus on helping customers to improve the efficiency of their testing processes.

• Preface xiGlossary xviiChapter 1 Fundamentals of Testing 11.1 What is testing? 11.1.1 Software and systems context 11.1.2 Causes of software defects 31.1.3 Role of testing in software development, maintenance and operations 51.1.4 Tests and quality 51.1.5 Terminology 71.2 What is testing? 81.2.1 Origin of defects 81.2.2 Common goals of testing 91.2.3 Examples of objectives for testing 91.2.4 Test and debugging 101.3 Paradoxes and main principles 111.3.1 Testing identifies the presence of defects 111.3.2 Exhaustive testing is impossible 111.3.3 Early testing 121.3.4 Defect clustering 121.3.5 Pesticide paradox 131.3.6 Testing is context dependent 131.3.7 Absence-of-errors fallacy 141.4 Test activities, testware and test roles 141.4.1 Planning 151.4.2 Monitoring and control 161.4.3 Test analysis and design 171.4.4 Test implementation 201.4.5 Test execution 211.4.6 Reporting 231.4.7 Test completion activities 241.4.8 The value of traceability 251.4.9 Impact of context on the test process 251.5 Roles in testing 261.6 Essential skills and "good practices" in testing 261.6.1 Generic skills 261.6.2 Specific skills 271.6.3 Whole team approach 271.6.4 Independence of testing 281.6.5 Levels of independence 291.6.6 Adapt to objectives 301.6.7 Destructive or constructive? 321.6.8 People skills 321.6.9 Change of perspective 331.7 Testers and code of ethics (FL 1.6) 331.7.1 Public 341.7.2 Client and employer 351.7.3 Product 351.7.4 Judgment 371.7.5 Management 371.7.6 Profession 381.7.7 Colleagues 391.7.8 Self 401.8 Sample exam questions 41Chapter 2 Testing Throughout the Software Life Cycle 472.1 Testing through the software development life cycle 472.1.1 Sequential models 482.1.2 Iterative models 502.1.3 Incremental model 522.1.4 RAD 542.1.5 Agile models 552.1.6 Selection of a development model 612.1.7 Positioning tests 622.1.8 Test-first and shift-left approaches 632.2 Test levels and test types 642.2.1 Component-level testing or component tests 652.2.2 Integration-level testing or integration tests 662.2.3 System tests 682.2.4 Acceptance tests 702.2.5 Other levels 722.3 Types of tests 722.3.1 Functional tests 732.3.2 Nonfunctional tests 742.3.3 Tests based on the structure or architecture of the software 762.3.4 Tests associated with changes 772.3.5 Comparisons and examples 792.4 Test and maintenance 802.4.1 Maintenance context 812.4.2 Evolutive maintenance 812.4.3 Corrective maintenance 822.4.4 Retirement and replacement 832.4.5 Regression test policies 832.4.6 SLA validation and acceptance 862.5 Oracles 862.5.1 Problems with oracles 872.5.2 Sources of oracles 872.5.3 Oracle usage 882.6 Process improvements 892.6.1 Objectives 892.6.2 Measurements 892.6.3 Retrospectives and improvements 892.7 Specific cases 902.7.1 Performance tests 902.7.2 Maintainability tests 912.8 Sample exam questions 91Chapter 3 Static Testing 973.1 Static techniques and the test process 973.2 Review process 1003.2.1 Types of reviews 1013.2.2 Roles and responsibilities during reviews 1063.2.3 Phases of reviews 1093.2.4 Success factors for reviews 1223.2.5 Comparison of the types of reviews 1233.3 Static analysis by tools 1253.3.1 Types of static analysis 1253.3.2 Types of defects that can be identified 1303.3.3 Data flow analysis 1313.4 Added value of static activities 1363.5 Sample exam questions 137Chapter 4 Test Design Techniques 1414.1 The test development process 1434.1.1 Terminology 1434.1.2 Traceability 1444.2 Categories of test design techniques 1464.2.1 Black box, white box or gray box 1474.2.2 Experience-based techniques 1484.2.3 Test characteristics 1494.2.4 Limitations and assumptions 1494.3 Black-box techniques 1514.3.1 Equivalence partitioning 1524.3.2 Boundary value analysis 1584.3.3 Decision tables 1614.3.4 Other combinational techniques 1664.3.5 State transition testing 1664.3.6 Use case testing 1734.3.7 Limitations and assumptions 1754.4 Structure-based techniques 1754.4.1 Statement testing and coverage 1784.4.2 Decision testing and coverage 1834.4.3 Other structure-based techniques 1864.4.4 MC/DC coverage 1884.4.5 Limitations and assumptions of structure-based testing 1894.4.6 Coverage level and exit criteria 1904.5 Experience-based technique 1904.5.1 Attacks 1914.5.2 Defect taxonomies 1924.5.3 Error guessing and ad hoc testing 1934.5.4 Exploratory testing 1944.5.5 Limitations and assumptions 1954.6 Collaboration-based test approaches 1964.6.1 Collaborative user stories 1964.6.2 Acceptance criteria 1974.6.3 Acceptance test-driven development 1974.7 Choosing test techniques 1984.8 Sample exam questions 200Chapter 5 Test Management 2095.1 Test organization 2095.1.1 Independence levels 2095.1.2 Roles and responsibilities 2125.1.3 Human and contractual aspects 2145.2 Test planning and estimation 2155.2.1 Planning and evaluation activities 2185.2.2 Test planning activities 2295.2.3 Test documentation 2315.2.4 Entry and exit criteria for test activities 2365.3 Test progress monitoring and control (FL 5.3) 2405.4 Reporting 2435.4.1 What to report, to whom and how? 2435.4.2 Statistics and graphs 2455.5 Transverse processes and activities 2485.5.1 Test data definition 2485.5.2 Configuration management (FL 5.4) 2495.5.3 Change management 2505.6 Risk management (FL 5.2) 2505.6.1 Principles of risk management 2515.6.2 Project risks and product risks 2555.6.3 Introduction to risk management 2565.7 Defect management (FL 5.5) 2595.7.1 Introduction to defect management 2605.7.2 Defect identification 2615.7.3 Actions applied to defects 2665.7.4 Defect disposition 2665.8 Sample exam questions 267Chapter 6 Tools Support for Testing 2776.1 Types of test tools 2776.1.1 Test tool classification 2786.1.2 Tools supporting test management 2786.1.3 Tools supporting requirement management 2796.1.4 Tools supporting static tests 2796.1.5 Modeling tools 2806.1.6 Tools supporting test design and test data creation 2806.1.7 Tools supporting test execution 2816.1.8 Tools supporting test environment management 2816.1.9 Tools supporting test data comparison 2826.1.10 Tools supporting test coverage measurement 2826.1.11 Other test supporting tools 2826.2 Assumptions and limitations of test tools 2836.2.1 Advantages and risks of the tools 2836.2.2 Specific considerations for some tools 2856.3 Selecting and introducing tools in an organization 2896.3.1 Main principles 2896.3.2 Tool selection process 2906.3.3 Test tool implementation 2936.3.4 To build or to buy test tools? 2946.4 Sample exam questions 295Chapter 7 Mock Exam 299Chapter 8 Templates and Models 3138.1 Master test plan 3138.2 Test plan 3158.2.1 Test plan as per IEEE 829-1998 3158.2.2 Test plan as per IEEE 829-2008 3158.3 Test design document 3178.3.1 Test design specifications as per IEEE 829-1998 3178.3.2 Test design document as per IEEE 829-2008 3178.4 Test case 3188.4.1 Test case document as per IEEE 829-1998 3188.4.2 Test case document as per IEEE 829-2008 3188.5 Test procedure 3198.5.1 Test procedure document as per IEEE 829-1998 3198.5.2 Test procedure document as per IEEE 829-2008 3198.6 Test log 3208.6.1 Test log as per IEEE 829-1998 3208.6.2 Test log as per IEEE 829-2008 3208.7 Defect report 3208.7.1 Defect report as per IEEE 829-1998 3208.7.2 Defect report as per IEEE 829-2008 3218.8 Test report 3228.8.1 Test report as per IEEE 829-1998 3228.8.2 Interim test report as per IEEE 829-2008 3228.8.3 Level test report as per IEEE 829-2008 3238.8.4 Master test report as per IEEE 829-2008 323Chapter 9 Answers to the Questions 3259.1 Answers to the end-of-chapter questions 3259.2 Correct answers to the sample paper questions 327References 329Index 333