Trade-off Analytics

Gregory S. Parnell, Ph.D, is a Research Professor in the Department of Industrial Engineering at the University of Arkansas. He is also a senior principal with Innovative Decisions, Inc., a decision and risk analysis firm and has served as Chairman of the Board. Dr. Parnell has published more than 100 papers and book chapters and was lead editor of Decision Making for Systems Engineering and Management, Wiley Series in Systems Engineering (2nd Ed, Wiley 2011) and lead author of the Handbook of Decision Analysis (Wiley 2013). He is a fellow of INFORMS, INCOSE, MORS, and the Society for Decision Professionals.

• List of Contributors xixAbout the Authors xxiForeword xxxiPreface xxxiiiAcknowledgments xliAbout the Companion Website xlv1 Introduction to Trade-off Analysis 1Gregory S. Parnell, Matthew Cilli, Azad M. Madni and Garry Roedler1.1 Introduction 21.2 Trade-off Analyses Throughout the Life Cycle 31.3 Trade-off Analysis to Identify System Value 31.4 Trade-off Analysis to Identify System Uncertainties and Risks 61.5 Trade-off Analyses can Integrate Value and Risk Analysis 61.6 Trade-off Analysis in the Systems Engineering Decision Management Process 81.7 Trade-off Analysis Mistakes of Omission and Commission 91.7.1 Mistakes of Omission 121.7.2 Mistakes of Commission 151.7.3 Impacts of the Trade-Off Analysis Mistakes 181.8 Overview of the Book 201.8.1 Illustrative Examples and Techniques Used in the Book 241.9 Key Terms 241.10 Exercises 25References 262 A Conceptual Framework and Mathematical Foundation for Trade-Off Analysis 29Gregory S. Parnell, Azad M. Madni and Robert F. Bordley2.1 Introduction 292.2 Trade-Off Analysis Terms 302.3 Influence Diagram of the Tradespace 312.3.1 Stakeholder Needs System Functions and Requirements 332.3.2 Objectives 332.3.3 System Alternatives 342.3.4 Uncertainty 362.3.5 Preferences and Evaluation of Alternatives 372.3.6 Resource Analysis 442.3.7 An Integrated Trade-Off Analyses 442.4 Tradespace Exploration 462.5 Summary 462.6 Key Words 472.7 Exercises 48References 483 Quantifying Uncertainty 51Robert F. Bordley3.1 Sources of Uncertainty in Systems Engineering 513.2 The Rules of Probability and Human Intuition 523.3 Probability Distributions 563.3.1 Calculating Probabilities from Experiments 563.3.2 Calculating Complex Probabilities from Simpler Probabilities 583.3.3 Calculating Probabilities Using Parametric Distributions 593.3.4 Applications of Parametric Probability Distributions 623.4 Estimating Probabilities 663.4.1 Using Historical Data 663.4.2 Using Human Judgment 683.4.3 Biases in Judgment 703.5 Modeling Using Probability 723.5.1 Bayes Nets 723.5.2 Monte Carlo Simulation 753.5.3 Monte Carlo Simulation with Dependent Uncertainties 763.5.4 Monte Carlo Simulation with Partial Information on Output Values 773.5.5 Variations on Monte Carlo Simulation 783.5.6 Sensitivity Analysis 783.6 Summary 813.7 Key Terms 813.8 Exercises 83References 864 Analyzing Resources 91Edward A. Pohl, Simon R. Goerger and Kirk Michealson4.1 Introduction 914.2 Resources 924.2.1 People 924.2.2 Facilities 954.2.3 Costs 954.2.4 Resource Space 994.3 Cost Analysis 994.3.1 Cost Estimation 1024.3.2 Cost Estimation Techniques 1084.3.3 Learning Curves 1204.3.4 Net Present Value 1254.3.5 Monte Carlo Simulation 1304.3.6 Sensitivity Analysis 1344.4 Affordability Analysis 1354.4.1 Background 1364.4.2 The Basics of Affordability Analysis Are Not Difficult 1374.4.3 DoD Comparison of Cost Analysis and Affordability Analysis 1384.4.4 Affordability Analysis Definitions 1394.4.5 “Big A” Affordability Analysis Process Guide 1414.5 Key Terms 1474.6 Excercises 149References 1525 Understanding Decision Management 155Matthew Cilli and Gregory S. Parnell5.1 Introduction 1555.2 Decision Process Context 1565.3 Decision Process Activities 1575.3.1 Frame Decision 1595.3.2 Develop Objectives and Measures 1635.3.3 Generate Creative Alternatives 1715.3.4 Assess Alternatives via Deterministic Analysis 1805.3.5 Synthesize Results 1835.3.6 Develop Multidimensional Value Model 1875.3.7 Identify Uncertainty and Conduct Probabilistic Analysis 1905.3.8 Assess Impact of Uncertainty 1925.3.9 Improve Alternatives 1965.3.10 Communicating Trade-Offs 1975.3.11 Present Recommendation and Implementation Plan 1975.4 Summary 1995.5 Key Terms 1995.6 Exercises 200References 2016 Identifying Opportunities 203Donna H. Rhodes and Simon R. Goerger6.1 Introduction 2036.2 Knowledge 2056.2.1 Domain Knowledge 2056.2.2 Technical Knowledge 2056.2.3 Business Knowledge 2056.2.4 Expert Knowledge 2066.2.5 Stakeholder Knowledge 2066.3 Decision Traps 2076.4 Techniques 2106.4.1 Interviews 2106.4.2 Focus Groups 2136.4.3 Surveys 2156.5 Tools 2196.5.1 Concept Map 2196.5.2 System Boundary 2206.5.3 Decision Hierarchy 2206.5.4 Issues List 2216.5.5 Vision Statement 2216.5.6 Influence Diagram 2226.5.7 Selecting Appropriate Tools and Techniques 2236.6 Illustrative Examples 2236.6.1 Commercial 2236.6.2 Defense 2266.7 Key Terms 2286.8 Exercises 230References 2307 Identifying Objectives and Value Measures 233Gregory S. Parnell and William D. Miller7.1 Introduction 2337.2 Value-Focused Thinking 2347.2.1 Four Major VFT Ideas 2357.2.2 Benefits of VFT 2357.3 Shareholder and Stakeholder Value 2367.3.1 Private Company Example 2377.3.2 Government Agency Example 2377.4 Challenges in Identifying Objectives 2387.5 Identifying the Decision Objectives 2397.5.1 Questions to Help Identify Decision Objectives 2397.5.2 How to Get Answers to the Questions 2407.6 The Financial or Cost Objective 2417.6.1 Financial Objectives for Private Companies 2417.6.2 Cost Objective for Public Organizations 2427.7 Developing Value Measures 2437.8 Structuring Multiple Objectives 2437.8.1 Value Hierarchies 2447.8.2 Techniques for Developing Value Hierarchies 2457.8.3 Value Hierarchy Best Practices 2477.8.4 Cautions about Cost and Risk Objectives 2487.9 Illustrative Examples 2487.9.1 Military Illustrative Example 2487.9.2 Homeland Security Illustrative Example 2507.10 Summary 2507.11 Key Terms 2527.12 Exercises 253References 2558 Developing and Evaluating Alternatives 257C. Robert Kenley, Clifford Whitcomb and Gregory S. Parnell8.1 Introduction 2578.2 Overview of Decision-making Creativity and Teams 2588.2.1 Approaches to Decision-Making 2588.2.2 Cognitive Methods for Creating Alternatives 2608.2.3 Key Concepts for Building and Operating Teams 2608.3 Alternative Development Techniques 2638.3.1 Structured Creativity Methods 2638.3.2 Morphological Box 2668.3.3 Pugh Method for Alternative Generation 2708.3.4 TRIZ for Alternative Development 2718.4 Assessment of Alternative Development Techniques 2758.5 Alternative Evaluation Techniques 2768.5.1 Decision-Theory-Based Approaches 2768.5.2 Pugh Method for Alternative Evaluation 2768.5.3 Axiomatic Approach to Design (AAD) 2778.5.4 TRIZ for Alternative Evaluation 2808.5.5 Design of Experiments (DOE) 2808.5.6 Taguchi Approach 2828.5.7 Quality Function Deployment (QFD) 2838.5.8 Analytic Hierarchy Process AHP 2878.6 Assessment of Alternative Evaluation Techniques 2908.7 Key Terms 2908.8 Exercises 290References 2939 An Integrated Model for Trade-Off Analysis 297Alexander D. MacCalman, Gregory S. Parnell and Sam Savage9.1 Introduction 2979.2 Conceptual Design Example 2989.3 Integrated Approach Influence Diagram 3009.3.1 Decision Nodes 3009.3.2 Uncertainty Nodes 3039.3.3 Constant Node 3109.3.4 Value Nodes 3149.4 Other Types of Trade-Off Analysis 3229.5 Simulation Tools 3229.5.1 Monte Carlo Simulation Proprietary Add-Ins 3249.5.2 The Discipline of Probability Management 3249.5.3 SIPmathTM Tool in Native Excel 3249.5.4 Model Building Steps 3259.6 Summary 3299.7 Key Terms 3309.8 Exercises 331References 33510 Exploring Concept Trade-Offs 337Azad M. Madni and Adam M. Ross10.1 Introduction 33710.1.1 Key Concepts Concept Trade-Offs and Concept Exploration 34110.2 Defining the Concept Space and System Concept of Operations 34510.3 Exploring the Concept Space 34610.3.1 Storytelling-Enabled Tradespace Exploration 34610.3.2 Decisions and Outcomes 34710.3.3 Contingent Decision-Making 34710.4 Trade-off Analysis Frameworks 34810.5 Tradespace and System Design Life Cycle 34910.6 From Point Trade-offs to Tradespace Exploration 35110.7 Value-based Multiattribute Tradespace Analysis 35110.7.1 Tradespace Exploration and Sensitivity Analysis 35310.7.2 Tradespace Exploration and Uncertainty 35410.7.3 Tradespace Exploration with Spiral Development 35610.7.4 Tradespace Exploration in Relation to Optimization and Decision Theory 35610.8 Illustrative Example 35910.8.1 Step 1: Determine Key Decision-Makers 35910.8.2 Step 2: Scope and Bound the Mission 36010.8.3 Step 3: Elicit Attributes and Utilities (Preference Capture) 36010.8.4 Step 4: Define Design Vector Elements (Concept Generation) 36210.8.5 Step 5: Develop Model(s) (Evaluation) 36210.8.6 Step 6: Generate the Tradespace (Computation) 36410.8.7 Step 7: Explore the Tradespace (Analysis and Synthesis) 36510.9 Conclusions 36910.10 Key Terms 37110.11 Exercises 372References 37211 Architecture Evaluation Framework 377James N. Martin11.1 Introduction 37711.1.1 Architecture in the Decision Space 37811.1.2 Architecture Evaluation 37911.1.3 Architecture Views and Viewpoints 38011.1.4 Stakeholders 38211.1.5 Stakeholder Concerns 38211.1.6 Architecture versus Design 38311.1.7 On the Uses of Architecture 38411.1.8 Standardizing on an Architecture Evaluation Strategy 38411.2 Key Considerations in Evaluating Architectures 38511.2.1 Plan-Driven Evaluation Effort 38611.2.2 Objectives-Driven Evaluation 38711.2.3 Assessment versus Analysis 38711.3 Architecture Evaluation Elements 38911.3.1 Architecture Evaluation Approach 38911.3.2 Architecture Evaluation Objectives 39011.3.3 Evaluation Approach Examples 39111.3.4 Value Assessment Methods 39111.3.5 Value Assessment Criteria 39311.3.6 Architecture Analysis Methods 39411.4 Steps in an Architecture Evaluation Process 39611.5 Example Evaluation Taxonomy 39811.5.1 Business Impact Factors 39811.5.2 Mission Impact Factors 39811.5.3 Architecture Attributes 39911.6 Summary 40011.7 Key Terms 40011.8 Exercises 402References 40212 Exploring the Design Space 405Clifford Whitcomb and Paul Beery12.1 Introduction 40512.2 Example 1: Liftboat 40612.2.1 Liftboat Fractional Factorial Design of Experiments 40612.2.2 Liftboat Design Trade-Off Space 40912.2.3 Liftboat Uncertainty Analysis 41112.2.4 Liftboat Example Summary 41112.3 Example 2: Cruise Ship Design 41112.3.1 Cruise Ship Taguchi Design of Experiments 41112.3.2 Cruise Ship Design Trade-Off Space 41212.3.3 Cruise Ship Example Summary 41612.4 Example 3: NATO Naval Surface Combatant Ship 41712.4.1 NATO Surface Combatant Ship Stakeholder Need 41812.4.2 NATO Surface Combatant Ship Box–Behnken Design of Experiments 42012.4.3 NATO Surface Combatant Ship Cost-Effectiveness Trade-Off 42112.4.4 NATO Surface Combatant Ship Design Tradespace 42112.4.5 NATO Surface Combatant Ship Design Trade-Off 42212.4.6 NATO Surface Combatant Ship Trade-Off Summary 43012.5 Key Terms 43112.6 Exercises 433References 43513 Sustainment Related Models and Trade Studies 437John E. MacCarthy and Andres Vargas13.1 Introduction 43713.2 Availability Modeling and Trade Studies 43913.2.1 FMDS Background 43913.2.2 FMDS Availability Trade Studies 44913.2.3 Section Synopsis 45313.3 Sustainment Life Cycle Cost Modeling and Trade Studies14 45413.3.1 The Total System Life Cycle Model 45413.3.2 The O&S Cost Model 45613.3.3 Life Cycle Cost Trade Study 45913.4 Optimization in Availability Trade Studies 46413.4.1 Setting Up the Optimization Problem 46413.4.2 Instantiating the Optimization Model 46513.4.3 Discussion of the Optimization Model Results 46813.4.4 Deterministic Sensitivity Analysis 46913.5 Monte Carlo Modeling 47113.5.1 Input Probability Distributions for the Monte Carlo Model 47113.5.2 Monte Carlo Simulation Results 47213.5.3 Stochastic Sensitivity Analysis 47313.6 Chapter Summary 47513.7 Key Terms 47613.8 Exercises 478References 48214 Performing Programmatic Trade-Off Analyses 483Gina Guillaume-Joseph and John E. MacCarthy14.1 Introduction 48314.2 System Acceptance Decisions and Trade Studies 48514.2.1 Acceptance Decision Framework 48614.2.2 Calculating the Confidence That a System Is “Good” 49114.2.3 Acceptance Test Design and Trade Studies 49314.2.4 A “Delay Fix and Test” Cost Model 49914.2.5 The Integrated Decision Model 50414.2.6 Conclusions 51114.3 Product Cancelation Decision Trade Study 51214.3.1 Introduction 51214.3.2 Significance 51314.3.3 Defining Failure 51414.3.4 Developing the Predictive Model 51914.3.5 Research Results 52214.3.6 Model Implementation In Industry 52814.3.7 Predictive Model Deployment in Industry 53014.3.8 When the Decision Has Been Made to Cancel the System 53614.3.9 Conclusion 53714.4 Product Retirement Decision Trade Study 53814.4.1 Introduction 53814.4.2 Legacy HR Systems 53914.4.3 The US NAVY Retirement and Decommission Program for Nuclear-Powered Vessels 54414.4.4 Decision Analysis for Decommissioning Offshore Oil and Gas Platforms in California 55114.4.5 System Retirement and Decommissioning Strategy 55914.4.6 Conclusion 56114.5 Key Terms 56214.6 Exercises 564References 56615 Summary and Future Trends 571Gregory S. Parnell and Simon R. Goerger15.1 Introduction 57115.2 Major Trade-Off Analysis Themes 57215.2.1 Use Standard Systems Engineering Terminology 57215.2.2 Avoid the Mistakes of Omission and Commission 57215.2.3 Use a Decision Management Framework 57215.2.4 Use Decision Analysis as the Mathematical Foundation 57315.2.5 Explicitly Define the Decision Opportunity 57315.2.6 Identify and Structure Decision Objectives and Measures 57415.2.7 Identify Creative Doable Alternatives 57415.2.8 Use the Most Appropriate Modeling and Simulation Technique for the Life Cycle Stage 57515.2.9 Include Resource Analysis in the Trade-Off Analysis 57515.2.10 Explicitly Consider Uncertainty 57515.2.11 Identify the Cost Value Schedule and Risk Drivers 57515.2.12 Provide an Integrated Framework for Cost Value and Risk Analyses 57615.3 Future of Trade-Off Analysis 57615.3.1 Education and Training of Systems Engineers 57715.3.2 Systems Engineering Methodologies and Tools 57715.3.3 Emergent Tradespace Factors 58015.4 Summary 581References 581Index 583