Uncertainty and Information

GEORGE J. KLIR, PhD, is currently Distinguished Professor of Systems Science at Binghamton University, SUNY. Since immigrating to the U.S. in 1966, he has held positions at UCLA, Fairleigh Dickinson University, and Binghamton University. He is a Life Fellow of IEEE, IFSA, and the Netherlands Institute for Advanced Studies. He has served as president of SGSR, IFSR, NAFIPS, and IFSA. He has published over 300 research papers and sixteen books, and has edited ten books. He has also served as Editor in Chief of the International Journal of General Systems since 1974 and of the IFSR International Book Series on Systems Science and Engineering since 1985. He has received numerous professional awards, including five honorary doctoral degrees, Bernard Bolzano's Gold Medal, Arnold Kaufmann's Gold Medal, and the SUNY Chancellor's Award for "Exemplary Contributions to Research and Scholarship." He is listed in Who's Who in America and Who's Who in the World. His current research interests include intelligent systems, soft computing, generalized information theory, systems modeling and design, fuzzy systems, and the theory of generalized measures. He has guided twenty-nine successful doctoral dissertations in these areas. Some of his research has been funded by grants from NSF, ONR, the United States Air Force, NASA, Sandia Labs, NATO, and various industries.

"..will establish a better understanding of the complex concepts…will make significant contributions toward stimulating research in the area of generalized information theory." (Computing Reviews.com, October 17, 2006) "…contains comprehensive and up-to-date coverage…can serve as a graduate-level text and a reference for researchers and practitioners…" (IEEE Computer Magazine, February 2006)

• Preface xiiiAcknowledgments xvii1 Introduction 11.1. Uncertainty and Its Significance 11.2. Uncertainty-Based Information 61.3. Generalized Information Theory 71.4. Relevant Terminology and Notation 101.5. An Outline of the Book 20Notes 22Exercises 232 Classical Possibility-Based Uncertainty Theory 262.1. Possibility and Necessity Functions 262.2. Hartley Measure of Uncertainty for Finite Sets 272.2.1. Simple Derivation of the Hartley Measure 282.2.2. Uniqueness of the Hartley Measure 292.2.3. Basic Properties of the Hartley Measure 312.2.4. Examples 352.3. Hartley-Like Measure of Uncertainty for Infinite Sets 452.3.1. Definition 452.3.2. Required Properties 462.3.3. Examples 52Notes 56Exercises 573 Classical Probability-Based Uncertainty Theory 613.1. Probability Functions 613.1.1. Functions on Finite Sets 623.1.2. Functions on Infinite Sets 643.1.3. Bayes’ Theorem 663.2. Shannon Measure of Uncertainty for Finite Sets 673.2.1. Simple Derivation of the Shannon Entropy 693.2.2. Uniqueness of the Shannon Entropy 713.2.3. Basic Properties of the Shannon Entropy 773.2.4. Examples 833.3. Shannon-Like Measure of Uncertainty for Infinite Sets  91Notes  95Exercises  974 Generalized Measures and Imprecise Probabilities 1014.1. Monotone Measures 1014.2. Choquet Capacities 1064.2.1. Möbius Representation 1074.3. Imprecise Probabilities: General Principles 1104.3.1. Lower and Upper Probabilities 1124.3.2. Alternating Choquet Capacities 1154.3.3. Interaction Representation 1164.3.4. Möbius Representation 1194.3.5. Joint and Marginal Imprecise Probabilities 1214.3.6. Conditional Imprecise Probabilities 1224.3.7. Noninteraction of Imprecise Probabilities 1234.4. Arguments for Imprecise Probabilities 1294.5. Choquet Integral 1334.6. Unifying Features of Imprecise Probabilities 135Notes 137Exercises 1395 Special Theories of Imprecise Probabilities 1435.1. An Overview 1435.2. Graded Possibilities 1445.2.1. Möbius Representation 1495.2.2. Ordering of Possibility Profiles 1515.2.3. Joint and Marginal Possibilities 1535.2.4. Conditional Possibilities 1555.2.5. Possibilities on Infinite Sets 1585.2.6. Some Interpretations of Graded Possibilities 1605.3. Sugeno l-Measures 1605.3.1. Möbius Representation 1655.4. Belief and Plausibility Measures 1665.4.1. Joint and Marginal Bodies of Evidence 1695.4.2. Rules of Combination 1705.4.3. Special Classes of Bodies of Evidence 1745.5. Reachable Interval-Valued Probability Distributions 1785.5.1. Joint and Marginal Interval-Valued Probability Distributions 1835.6. Other Types of Monotone Measures 185Notes 186Exercises 1906 Measures of Uncertainty and Information 1966.1. General Discussion 1966.2. Generalized Hartley Measure for Graded Possibilities 1986.2.1. Joint and Marginal U-Uncertainties 2016.2.2. Conditional U-Uncertainty 2036.2.3. Axiomatic Requirements for the U-Uncertainty 2056.2.4. U-Uncertainty for Infinite Sets 2066.3. Generalized Hartley Measure in Dempster–Shafer Theory 2096.3.1. Joint and Marginal Generalized Hartley Measures 2096.3.2. Monotonicity of the Generalized Hartley Measure 2116.3.3. Conditional Generalized Hartley Measures 2136.4. Generalized Hartley Measure for Convex Sets of Probability Distributions 2146.5. Generalized Shannon Measure in Dempster-Shafer Theory 2166.6. Aggregate Uncertainty in Dempster–Shafer Theory 2266.6.1. General Algorithm for Computing the Aggregate Uncertainty 2306.6.2. Computing the Aggregated Uncertainty in Possibility Theory 2326.7. Aggregate Uncertainty for Convex Sets of Probability Distributions 2346.8. Disaggregated Total Uncertainty 2386.9. Generalized Shannon Entropy 2416.10. Alternative View of Disaggregated Total Uncertainty 2486.11. Unifying Features of Uncertainty Measures 253Notes 253Exercises 2557 Fuzzy Set Theory 2607.1. An Overview 2607.2. Basic Concepts of Standard Fuzzy Sets 2627.3. Operations on Standard Fuzzy Sets 2667.3.1. Complementation Operations 2667.3.2. Intersection and Union Operations 2677.3.3. Combinations of Basic Operations 2687.3.4. Other Operations 2697.4. Fuzzy Numbers and Intervals 2707.4.1. Standard Fuzzy Arithmetic 2737.4.2. Constrained Fuzzy Arithmetic 2747.5. Fuzzy Relations 2807.5.1. Projections and Cylindric Extensions 2817.5.2. Compositions, Joins, and Inverses 2847.6. Fuzzy Logic 2867.6.1. Fuzzy Propositions 2877.6.2. Approximate Reasoning 2937.7. Fuzzy Systems 2947.7.1. Granulation 2957.7.2. Types of Fuzzy Systems 2977.7.3. Defuzzification 2987.8. Nonstandard Fuzzy Sets 2997.9. Constructing Fuzzy Sets and Operations 303Notes 305Exercises 3088 Fuzzification of Uncertainty Theories 3158.1. Aspects of Fuzzification 3158.2. Measures of Fuzziness 3218.3. Fuzzy-Set Interpretation of Possibility Theory 3268.4. Probabilities of Fuzzy Events 3348.5. Fuzzification of Reachable Interval-Valued Probability Distributions 3388.6. Other Fuzzification Efforts 348Notes 350Exercises 3519 Methodological Issues 3559.1. An Overview 3559.2. Principle of Minimum Uncertainty 3579.2.1. Simplification Problems 3589.2.2. Conflict-Resolution Problems 3649.3. Principle of Maximum Uncertainty 3699.3.1. Principle of Maximum Entropy 3699.3.2. Principle of Maximum Nonspecificity 3739.3.3. Principle of Maximum Uncertainty in GIT 3759.4. Principle of Requisite Generalization 3839.5. Principle of Uncertainty Invariance 3879.5.1. Computationally Simple Approximations 3889.5.2. Probability–Possibility Transformations 3909.5.3. Approximations of Belief Functions by Necessity Functions 3999.5.4. Transformations Between l-Measures and Possibility Measures 4029.5.5. Approximations of Graded Possibilities by Crisp Possibilities 403Notes 408Exercises 41110 Conclusions 41510.1. Summary and Assessment of Results in Generalized Information Theory 41510.2. Main Issues of Current Interest 41710.3. Long-Term Research Areas 41810.4. Significance of GIT 419Notes 421Appendix A Uniqueness of the U-Uncertainty 425Appendix B Uniqueness of Generalized Hartley Measure in the Dempster–Shafer Theory 430Appendix C Correctness of Algorithm 6.1 437Appendix D Proper Range of Generalized Shannon Entropy 442Appendix E Maximum of GSa in Section 6.9 447Appendix F Glossary of Key Concepts 449Appendix G Glossary of Symbols 455Bibliography 458Subject Index 487Name Index 494