Improving Product Reliability and Software Quality

MARK A. LEVIN is the Reliability Manager for Product Development at Teradyne, Inc., USA. He has over 36 years of electronics experience working in manufacturing, design and research. TED T. KALAL is a retired Reliability Manager. He has held many positions as a contract engineer and consultant where he focused on design, quality and reliability tasks. JONATHAN RODIN is a Software Engineering Manager at Teradyne, Inc., USA. Jon has 39 years of experience developing software either as a programmer or managing software development projects.

• About the Authors xixList of Figures xxiList of Tables xxvSeries Editor's Foreword xxviiSeries Foreword Second Edition xxixSeries Foreword First Edition xxxiForeword First Edition xxxiiiPreface Second Edition xxxvPreface First Edition xxxviiAcknowledgments xliGlossary xliiiPart I Reliability and Software Quality – It’s a Matter of Survival 11 The Need for a New Paradigm for Hardware Reliability and Software Quality 31.1 Rapidly Shifting Challenges for Hardware Reliability and Software Quality 31.2 Gaining Competitive Advantage 51.3 Competing in the Next Decade –Winners Will Compete on Reliability 51.4 Concurrent Engineering 61.5 Reducing the Number of Engineering Change Orders at Product Release 81.6 Time-to-Market Advantage 91.7 Accelerating Product Development 101.8 Identifying and Managing Risks 111.9 ICM, a Process to Mitigate Risk 111.10 Software Quality Overview 12References 13Further Reading 132 Barriers to Implementing Hardware Reliability and Software Quality 152.1 Lack of Understanding 152.2 Internal Barriers 162.3 Implementing Change and Change Agents 172.4 Building Credibility 192.5 Perceived External Barriers 202.6 Time to Gain Acceptance 212.7 External Barrier 222.8 Barriers to Software Process Improvement 233 Understanding Why Products Fail 253.1 Why Things Fail 253.2 Parts Have Improved, Everyone Can Build Quality Products 283.3 Hardware Reliability and Software Quality –The New Paradigm 283.4 Reliability vs. Quality Escapes 293.5 Why Software Quality Improvement Programs Are Unsuccessful 30Further Reading 314 Alternative Approaches to Implementing Reliability 334.1 Hiring Consultants for HALT Testing 334.2 Outsourcing Reliability Testing 334.3 Using Consultants to Develop and Implement a Reliability Program 344.4 Hiring Reliability Engineers 34Part II Unraveling the Mystery 375 The Product Life Cycle 395.1 Six Phases of the Product Life Cycle 395.2 Risk Mitigation 415.3 The ICM Process for a Small Company 455.4 Design Guidelines 465.5 Warranty 46Further Reading 47Reliability Process 47DFM 486 Reliability Concepts 496.1 The Bathtub Curve 506.2 Mean Time between Failure 516.3 Warranty Costs 536.4 Availability 556.5 Reliability Growth 576.6 Reliability Demonstration Testing 596.7 Maintenance and Availability 626.8 Component Derating 696.9 Component Uprating 70Reference 71Further Reading 72Reliability Growth 72Reliability Demonstration 72Prognostics and Health Management 727 FMEA 737.1 Benefits of FMEA 737.2 Components of FMEA 747.3 Preparing for the FMEA 867.4 Barriers to the FMEA Process 897.5 FMEA Ground Rules 917.6 Using Macros to Improve FMEA Efficiency and Effectiveness 927.7 Software FMEA 947.8 Software Fault Tree Analysis (SFTA) 977.9 Process FMEAs 977.10 FMMEA 998 The Reliability Toolbox 1018.1 The HALT Process 1018.2 Highly Accelerated Stress Screening (HASS) 1218.3 HALT and HASS Test Chambers 1278.4 Accelerated Reliability Growth (ARG) 1288.5 Accelerated Early Life Test (ELT) 1318.6 SPC Tool 1328.7 FIFO Tool 132References 134Further Reading 134FMEA 134HALT 135HASS 136Quality 136Burn-in 136ESS 137Up Rating 1379 Software Quality Goals and Metrics 1399.1 Setting Software Quality Goals 1399.2 Software Metrics 1409.3 Lines of Code (LOC) 1429.4 Defect Density 1429.5 Defect Models 1449.6 Defect Run Chart 1459.7 Escaped Defect Rate 1479.8 Code Coverage 148References 149Further Reading 15010 Software Quality Analysis Techniques 15110.1 Root Cause Analysis 15110.2 The 5 Whys 15110.3 Cause and Effect Diagrams 15210.4 Pareto Charts 15310.5 Defect Prevention, Defect Detection, and Defensive Programming 15410.6 Effort Estimation 157Reference 158Further Reading 15811 Software Life Cycles 15911.1 Waterfall 15911.2 Agile 16111.3 CMMI 16211.4 How to Choose a Software Life Cycle 165Reference 166Further Reading 16612 Software Procedures and Techniques 16712.1 Gathering Requirements 16712.2 Documenting Requirements 16912.3 Documentation 17212.4 Code Comments 17312.5 Reviews and Inspections 17412.6 Traceability 17912.7 Defect Tracking 17912.8 Software and Hardware Integration 180References 182Further Reading 18213 Why Hardware Reliability and Software Quality Improvement Efforts Fail 18313.1 Lack of Commitment to the Reliability Process 18313.2 Inability to Embrace and Mitigate Technologies Risk Issues 18513.3 Choosing the Wrong People for the Job 18613.4 Inadequate Funding 18613.5 Inadequate Resources 19113.6 MIL-HDBK 217 –Why It Is Obsolete 19213.7 Finding But Not Fixing Problems 19513.8 Nondynamic Testing 19613.9 Vibration Testing Too Difficult to Implement 19613.10 The Impact of Late Hardware or Late Software Delivery 19613.11 Supplier Reliability 196Reference 197Further Reading 19714 Supplier Management 19914.1 Purchasing Interface 19914.2 Identifying Your Critical Suppliers 20014.3 Develop a Thorough Supplier Audit Process 20014.4 Develop Rapid Nonconformance Feedback 20114.5 Develop a Materials Review Board (MRB) 20214.6 Counterfeit Parts and Materials 202Part III Steps to Successful Implementation 20515 Establishing a Reliability Lab 20715.1 Staffing for Reliability 20715.2 The Reliability Lab 20815.3 Facility Requirements 21015.4 Liquid Nitrogen Requirements 21015.5 Air Compressor Requirements 21115.6 Selecting a Reliability Lab Location 21215.7 Selecting a Halt Test Chamber 213Reference 22016 Hiring and Staffing the Right People 22116.1 Staffing for Reliability 22116.2 Staffing for Software Engineers 22516.3 Choosing the Wrong People for the Job 22617 Implementing the Reliability Process 22917.1 Reliability Is Everyone’s Job 22917.2 Formalizing the Reliability Process 23017.3 Implementing the Reliability Process 23117.4 Rolling Out the Reliability Process 23117.5 Developing a Reliability Culture 23517.6 Setting Reliability Goals 23617.7 Training 23717.8 Product Life Cycle Defined 23817.9 Proactive and Reactive Reliability Activities 241Further Reading 244Reliability Process 244Part IV Reliability and Quality Process for Product Development 24518 Product Concept Phase 24718.1 Reliability Activities in the Product Concept Phase 24718.2 Establish the Reliability Organization 24818.3 Define the Reliability Process 24918.4 Define the Product Reliability Requirements 24918.5 Capture and Apply Lessons Learned 24918.6 Mitigate Risk 25219 Design Concept Phase 25719.1 Reliability Activities in the Design Concept Phase 25719.2 Set Reliability Requirements and Budgets 25919.3 Define Reliability Design Guidelines 26319.4 Revise Risk Mitigation 26419.5 Schedule Reliability Activities and Capital Budgets 26819.6 Decide Risk Mitigation Sign-off Day 26919.7 Reflect on What Worked Well 27120 Product Design Phase 27320.1 Product Design Phase 27320.2 Reliability Estimates 27420.3 Implementing Risk Mitigation Plans 27620.4 Design for Reliability Guidelines (DFR) 28520.5 Design FMEA 28920.6 Installing a Failure Reporting Analysis and Corrective Action System 29020.7 HALT Planning 29120.8 HALT Test Development 29220.9 Risk Mitigation Meeting 295Further Reading 296FMEA 296HALT 29621 Design Validation Phase 29921.1 Design Validation 29921.2 Using HALT to Precipitate Failures 30121.3 Proof of Screen (POS) 31321.4 Highly Accelerated Stress Screen (HASS) 31521.5 Operate FRACAS 31521.6 Design FMEA 31721.7 Closure of Risk Issues 317Further Reading 318FMEA 318Acceleration Methods 318ESS 318HALT 31922 Software Testing and Debugging 32122.1 Unit Tests 32122.2 Integration Tests 32322.3 System Tests 32422.4 Regression Tests 32422.5 Security Tests 32622.6 Guidelines for Creating Test Cases 32722.7 Test Plans 32822.8 Defect Isolation Techniques 32922.9 Instrumentation and Logging 331Further Reading 33423 Applying Software Quality Procedures 33523.1 Using Defect Model to Create Defect Run Chart 33623.2 Using Defect Run Chart to Know When You Have Achieved the Quality Target 33623.3 Using Root Cause Analysis on Defects to Improve Organizational Quality Delivery 33823.4 Continuous Integration and Test 338Further Reading 33924 Production Phase 34124.1 Accelerating Design Maturity 34124.2 Reliability Growth 34624.3 Design and Process FMEA 351Further Reading 355FMEA 355Quality 356Reliability Growth 356Burn-In 357HASS 35725 End-of-Life Phase 35925.1 Managing Obsolescence 35925.2 Product Termination 36025.3 Project Assessment 360Further Reading 36126 Field Service 36326.1 Design for Ease of Access 36326.2 Identify High Replacement Assemblies (FRUs) 36326.3 Wearout Replacement 36526.4 Preemptive Servicing 36526.5 Servicing Tools 36526.6 Service Loops 36626.7 Availability or Repair Time Turnaround 36726.8 Avoid System Failure Through Redundancy 36726.9 Random versus Wearout Failures 367Further Reading 368Appendix A 369A.1 Reliability Consultants 369A.2 Graduate Reliability Engineering Programs and Reliability Certification Programs 372A.3 Reliability Professional Organizations and Societies 376A.4 Reliability Training Classes 377A.5 Environmental Testing Services 379A.6 HALT Test Chambers 381A.7 Reliability Websites 382A.8 Reliability Software 383A.9 Reliability Seminars and Conferences 384A.10 Reliability Journals 386Appendix B 387B.1 MTBF, FIT, and PPM Conversions 387B.2 Mean Time Between Failure (MTBF) 387B.3 Estimating Field Failures 396B.3.1 Comparing Repairable to Nonrepairable Systems 397Index 399