Value of Innovation

Jack J. Phillips, PhD is Chairman of ROI Institute and a world-renowned expert on measurement and evaluation. Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers worldwide. Phillips is also the author or editor of more than a hundred books and hundreds of articles. His work has been featured in the Wall Street Journal, Bloomberg Businessweek, Fortune, and on CNN.Patricia Pulliam Phillips, PhD is an internationally recognized author, consultant, and President and CEO of ROI Institute, Inc. Phillips provides consulting services to organizations worldwide. She helps organizations build capacity in the ROI Methodology by facilitating the ROI certification process and teaching the ROI Methodology through workshops and graduate level courses. She has been an author or editor of more than a hundred books and numerous journals.

• Preface xixAcknowledgements xxiiiAbout the Authors xxv1 The Importance and Challenges of Innovation 1Innovation Hype 2Articles 2Books 2Jobs 3Speeches 3Experience 3The Realities of Innovation 4Innovation is Not New 4Innovation is Necessary for Survival 5Innovation is Equated with Success 5Innovation is Truly Global 6Consumers and Investors Expect Innovation 6Innovation is Often Disruptive 6Innovation is Not a Single Event 7Little Ideas Often Make a Big Difference 7Innovation Comes in Many Types and Forms 8Innovation Spans Many Different Horizons 8Trouble in Paradise: The Misconceptions 9Misconception 1: Small Companies are More Innovative 10Misconception 2: Uncontested Markets are Good for Innovation 10Misconception 3: Spending More on R&D Increases Innovation 10Misconception 4: Companies Need More Radical Innovation 10Misconception 5: Open Innovation Turbocharges R&D 11Misconception 6: R&D Needs to be More Relevant 11Misconception 7: Wall Street Rewards Innovation 11Innovation Challenges 12Innovation is Expensive 12Managing Innovation is Difficult 13An Innovation Culture is Necessary for Success 13Innovation Requires Many Personas 14Innovation Success Rates Need to Improve 16The Value of Innovation is Unclear 16Final Thoughts 172 Status and Concerns about Innovation Measurement 19Innovation: Definition, Models, and Measures 20Sources of Innovation 21Measurement Shifts 23Measurement Shifts are Common 23Value Perception. 24The Search for Money 24Hoping, Knowing, Proving, and Showing Value 25Innovation is Systematic 25Macro View of Measurement 27Industry Level Measures 29Company Level 30Concerns about Company Level Measures 30Micro View of Measurement 32Final Thoughts 343 The Case for a New System 35Innovation: A Cost or an Investment? 36The Value of Innovation: A Summary 38Intangibles and the Fear of not Investing 38Relationship Between Variables 39ROI Studies 41Types of Data 41Inputs 41Reaction and Planned Action 42Learning 43Application and Implementation 43Impact 44Return on Investment 44How Does Your Current System Stack Up? 45Focus of Use 45Standards 47Contents viiTypes of Data 47Dynamic Adjustments 47Connectivity 48Approach 48Conservative Nature 48Simplicity 48Theoretical Foundation 49Acceptance 49Using Design Thinking to Deliver and Measure Results 49Start with Why: Aligning Projects with the Business 50Make it Feasible: Selecting the Right Solution 51Expect Success: Designing for Results 52Make it Matter: Designing for Input, Reaction, and Learning 52Make it Stick: Designing for Application and Impact 53Make it Credible: Measuring Results and Calculating ROI 53Tell the Story. Communicating Results to Key Stakeholders 54Optimize the Results: Using Black Box Thinking to Increase Funding 54Requirements for the Value of Innovation: A Measurement Process 55ROI Measurement Methodology 56Terminology: Projects, Solutions, Participants . . . 57Final Thoughts 574 Introducing the ROI Methodology 59The ROI Methodology 60Types of Data 60The Initial Analysis 63The ROI Process Model 65Planning the Evaluation 66Evaluation Purpose 66Feasibility 67Data Collection Plan 68ROI Analysis Plan 68Project Plan 71Collecting Data 71Isolating the Effects of the Project 72Converting Data to Monetary Values 72Identifying Intangible Benefits 73Tabulating Project Costs 74Calculating the Return on Investment 74Reporting 75Operating Standards and Philosophy 75Implementing and Sustaining the Process 76Benefits of This Approach 76Aligning with Business 77Validating the Value Proposition 77Improving Processes 77Enhancing the Image and Building Respect 78Improving Support 78Justifying or Enhancing Budgets 78Building a Partnership with Key Executives 79Earning a Seat at the Table 79Final Thoughts 795 Aligning Innovation Projects to the Organization 81Creating Business Alignment 83The Purpose of Alignment 83Disciplined Analysis 84Determining the Potential Payoff 86Obvious Versus not-so-obvious Payoff 87The Cost of a Problem 89The Value of an Opportunity 90To Forecast or not to Forecast? 90Determining Business Needs 90The Opportunity 91Hard Data Measures 91Soft Data Measures 92Tangible versus Intangible Benefits: A Better Approach 93Impact Data Sources 94Determining Performance Needs 95Analysis Techniques 95A Sensible Approach 95Determining Learning Needs 96Determining Preference Needs 97Case Study: Southeast Corridor Bank 98Payoff Needs 98Business Needs 99Performance Needs 99The Solution 100Learning Needs 101Preference Needs 102Developing Objectives for Innovation Projects 102Reaction Objectives 102Learning Objectives 103Application and Implementation Objectives 103Impact Objectives 104ROI Objectives 105Final Thoughts 1066 Collecting Data Along Chain of Impact with a Toolbox of Methods 107Questionnaires and Surveys 109Types of Questions and Statements 109Design Issues 110A Detailed Example 111Improving the Response Rate for Questionnaires and Surveys 113Using Interviews 120Types of Interviews 121Interview Guidelines 121Using Focus Groups 122Applications for Evaluation 122Guidelines 123Measuring with Tests 124Measuring with Simulation 124Task Simulation 124Role-Playing/Skill Practice 125Using Observation 125Guidelines for Effective Observation 125Observation Methods 127Using Action Plans 128Using Action Plans Successfully 129Advantages/Disadvantages of Action Plans 132Using Performance Contracts 133Monitoring Business Performance Data 134Existing Measures 134Developing New Measures 135Selecting the Appropriate Method for Each Level 135Type of Data 135Participants’ Time for Data Input 136Manager Time for Data Input 136Cost of Method 137Disruption of Normal Work Activities 137Accuracy of Method 137Utility of an Additional Method 137Cultural Bias for Data Collection Method 138Final Thoughts 1387 Measuring Reaction and Perceived Value 139Why Measure Reaction and Perceived Value? 140Customer Satisfaction 141Immediate Adjustments 141Predictive Capability 141Important but not Exclusive 142Sources of Data 143Participants 143Participant Managers 143Other Team Members 143Internal or External Customers 144Project Leaders and Team Members 144Sponsors and Senior Managers 144Records and Previous Studies 144Areas of Feedback 145Data Collection Timing 146Data Collection Methods 146Questionnaires and Surveys 146Interviews 147Focus Groups 147Using Reaction Data 147Final Thoughts 1488 Measuring Learning 149Why Measure Learning and Confidence? 150The Importance of Intellectual Capital 151The Learning Organization 152The Compliance Issue 152The Use and Development of Competencies 152The Role of Learning in Innovation Projects 153The Challenges and Benefits of Measuring Learning 153Challenges 154The Benefits of Measuring Learning 154Measurement Issues 155Project Objectives 155Typical Measures 155Timing 156Data Collection Methods 157Questionnaires and Surveys 157Performance Tests 157Technology and Task Simulations 158Case Studies 159Role-Playing and Skill Practice 159Informal Assessments 159Administrative Issues 160Reliability and Validity 160Consistency 161Pilot Testing 161Scoring and Reporting 161Using Learning Data 162Final Thoughts 1629 Measuring Application and Implementation 163Why Measure Application and Implementation? 165Information Value 165Project Focus 166Problems and Opportunities 166Reward Effectiveness 167Challenges 167Linking with Learning 168Building Data Collection into the Project 168Ensuring a Sufficient Amount of Data 168Addressing Application Needs at the Outset 169Measurement Issues 169Methods 169Objectives 170Areas of Coverage 170Data Sources 170Timing 170Responsibilities 171Data Collection Methods 171Using Questionnaires to Measure Application and Implementation 172Using Interviews, Focus Groups, and Observation 172Using Action Plans 172Barriers to Application 174Application Data Use 174Final Thoughts 17510 Measuring Impact 177Why Measure Business Impact? 178Higher-Level Data 178A Business Driver for Projects 179“The Money” for Sponsors 179Easy to Measure 180Collecting Effective Impact Measures 180Data Categories 180Metric Fundamentals 181Identifying Specific Measures Linked to Projects 182Business Performance Data Monitoring 183Identify Appropriate Measures 184Convert Current Measures to Usable Ones 184Develop New Measures 184Data Collection Methods 185Using Action Plans to Develop Business Impact Data 185Using Performance Contracts to Measure Business Impact 187Using Questionnaires to Collect Business Impact Measures 189Measuring the Hard to Measure 190Everything Can Be Measured 190Perceptions are Important 191Every Measure Can Be Converted to Money, but not Every Measure Should Be 191Special Emphasis on Intangibles 192Final Thoughts 19211 Isolating the Effects of Innovation 193Why the Concern over this Issue? 196Reality 196Myths 196Preliminary Issues 198Chain of Impact 198Identify other Factors: A First Step 199Isolation Methods 200Control Groups 200Trend Line Analysis 203Mathematical Modeling 205Estimates 206Participants’ Estimate of Impact 206Manager’s Estimate of Impact 209Customer Estimates of Project Impact 209Internal or External Expert Estimates 210Estimate Credibility: The Wisdom of Crowds 210Calculate the Impact of other Factors 212Select the Technique 213Final Thoughts 21412 Converting Data to Money 215Why Convert Data to Monetary Values? 217Value Equals Money 217Impact is More Understandable 217Converting to Monetary Values is Similar to Budgeting 218Monetary Value is Vital to Organizational Operations 218Monetary Values are Necessary to UnderstandProblems and Cost Data 219Key Steps in Converting Data to Money 219Standard Monetary Values 222Converting output Data to Money 222Calculating the Cost of Quality 223Converting Employee Time Using Compensation 227Finding Standard Values 228When Standard Values are not Available 229Using Historical Costs from Records 229Time 229Availability 230Access 230Accuracy 230Using Input from Experts 230Using Values from External Databases 231Linking with other Measures 232Using Estimates from Participants 233Using Estimates from the Management Team 233Using Project Staff Estimates 234Technique Selection and Finalizing Value 234Choose a Technique Appropriate for the Type of Data 235Move from Most Accurate to Least Accurate 235Consider Source Availability 235Use the Source with the Broadest Perspective on the Issue 236Use Multiple Techniques When Feasible 236Apply the Credibility Test 236Consider the Possibility of Management Adjustment 238Consider the Short-Term/Long-Term Issue 238Consider an Adjustment for the Time Value of Money 239Final Thoughts 23913 Addressing Intangibles 241Why Intangibles are Important 244Intangibles are the Invisible Advantage 244We are Entering the Intangible Economy 245More Intangibles are Converted to Tangibles 245Intangibles Drive Innovation Projects 246The Magnitude of the Investment 246Measuring and Analyzing Intangibles 246Measuring the Intangibles 247Converting to Money 249Identifying and Collecting Intangibles 251Analyzing Intangibles 252Final Thoughts 25314 Measuring ROI 255Why Monitor Costs and Measure ROI? 258Fundamental Cost Issues 259Fully Loaded Costs 259Costs Reported without Benefits 260Develop and Use Cost Guidelines 261Sources of Costs 262Prorated versus Direct Costs 262Employee Benefits Factor 263Specific Costs to Include 263Initial Analysis and Assessment 264Development of Project Solutions 264Acquisition Costs 264Implementation Costs 264Maintenance and Monitoring 265Support and Overhead 265Evaluation and Reporting 265The ROI Calculation 265Benefits/Costs Ratio 266ROI Formula 267ROI Objective 269Other ROI Measures 270Payback Period (Breakeven Analysis) 270Discounted Cash Flow 271Internal Rate of Return 271Final Thoughts 27215 Forecasting Value, Including ROI 273Why Forecast ROI? 278Expensive Projects 279High Risks and Uncertainty 279Postproject Comparison 279Compliance 280The Trade-offs of Forecasting 280Preproject ROI Forecasting 282Basic Model 282Basic Steps to Forecast ROI 283Sources of Expert Input 287Securing Input 287Conversion to Money 288Estimate Project Costs 288Case Study 289Forecasting with a Pilot Program 293Forecasting with Reaction Data 293Case Study: Forecasting ROI from Reaction Data 294Use of the Data 295Forecasting Guidelines 296Final Thoughts 29916 Reporting Results 301The Importance of Communicating Results? 303Communication is Necessary to Make Improvements 303Communication is Necessary to Explain the Contribution 303Communication is a Politically Sensitive Issue 304Different Audiences Need Different Information 304Principles of Communicating Results 304Communication Must Be Timely 305Communication Should Be Targeted to Specific Audiences 305Media Should Be Carefully Selected 305Communication Should Be Unbiased and Modest in Tone 305Communication Must Be Consistent 306Make the Message Clear 306Testimonials Must Come from Respected Individuals 306The Audience’s Bias of the Project Will Influence the Communication Strategy 306Storytelling is Essential 307The Process for Communicating Results 307The Need for Communication 308The Communication Plan 309The Audience for Communications 309Basis for Selecting the Audience 311Information Development: The Impact Study 312Media Selection 312Meetings 312Interim and Progress Reports 314Routine Communication Tools 315E-mail and Electronic Media 316Project Brochures and Pamphlets 316Case Studies 316Delivering the Message 316Routine Feedback on Project Progress 317Storytelling 319Presentation of Results to Senior Management 320Reactions to Communication 322Final Thoughts 32217 Implementing and Sustaining ROI 323Why is this Important? 324Resistance is Always Present 326Implementation is the Key to Success 326Consistency is Needed 326Efficiency 326Value is Maximized 326Implementing the Process: Overcoming Resistance 327Review Current Results 328Developing Roles and Responsibilities 328Identifying a Champion 329Developing the ROI Leader 329Establishing a Task Force 329Assigning Responsibilities 330Establishing Goals and Plans 331Setting Evaluation Targets 331Developing a Plan for Implementation 332Revising or Developing Policies and Guidelines 332Preparing the Project Team 334Involving the Project Team 334Using ROI as a Learning and Project Improvement Tool 334Teaching the Team 334Initiating ROI Studies 335Selecting the Initial Project 335Developing the Planning Documents 335Reporting Progress 336Establishing Discussion Groups 336Preparing the Sponsors and Management Team 336Removing Obstacles 337Dispelling Myths 337Delivering Bad News 338Using the Data 338Monitoring Progress 339Final Thoughts 340References 343Index 351