Grid

Dr Maozhen Li is currently Lecturer in Electronics and Computer Engineering, in the School of Engineering and Design at Brunel University, UK. From January 1999 to January 2002, he was Research Associate in the Department of Computer Science, Cardiff University, UK. Dr Li received his PhD degree in 1997, from the Institute of Software, Chinese Academy of Sciences, Beijing, China. His research interests are in the areas of Grid computing, problem-solving environments for large-scale simulations, software agents for semantic information retrieval, multi-modal user interface design and computer support for cooperative work. Since 1997, Dr Li has published 30 research papers in prestigious international journals and conferences. Dr Mark Baker is a hardworking Reader in Distributed Systems at the University of Portsmouth. He also currently holds visiting chairs at the universities of Reading and Westminster. Mark has resided in the relative safety of academia since leaving the British Merchant, where he was a navigating officer, in the early 1980s. Mark has held posts at various universities, including Cardiff, Edinburgh and Syracuse. He has a number of geek-like interests, which his research group at Portsmouth help him pursue. These include wide-area resource monitoring, messaging systems for parallel and wide-area applications, middleware such as information and security services, as well as performance evaluation and modelling of computer systems.Mark’s non-academic interests include squash (getting too old), DIY (he may one day finish his house off), reading (far too many science fiction books), keeping the garden ship-shape and a beer or two to reduce the pain of the aforementioned activities.

"It could serve as a good textbook and would certainly be a good addition to the reference libraries of technologists, academics, and students." (IEEE Distributed Systems Online, December 2006) "…lots of valuable information." (Computing Reviews.com, May 11, 2006)"…a complete, clear, systematic, and practical understanding of the technologies that enable the Grid." (IEEE Computer Magazine, August 2005)"…a good addition to the reference library…" (IEEE DS Online, January 2007)

• About the Authors xiiiPreface xvAcknowledgements xixList of Abbreviations xxi1 An Introduction to the Grid 11.1 Introduction 11.2 Characterization of the Grid 11.3 Grid-Related Standards Bodies 41.4 The Architecture of the Grid 51.5 References 6Part One System Infrastructure 92 OGSA and WSRF 11Learning Objectives 11Chapter Outline 112.1 Introduction 122.2 Traditional Paradigms for Distributed Computing 132.2.1 Socket programming 142.2.2 RPC 152.2.3 Java RMI 162.2.4 DCOM 182.2.5 CORBA 192.2.6 A summary on Java RMI, DCOM and CORBA 202.3 Web Services 212.3.1 SOAP 232.3.2 WSDL 242.3.3 UDDI 262.3.4 WS-Inspection 272.3.5 WS-Inspection and UDDI 282.3.6 Web services implementations 292.3.7 How Web services benefit the Grid 332.4 OGSA 342.4.1 Service instance semantics 352.4.2 Service data semantics 372.4.3 OGSA portTypes 382.4.4 A further discussion on OGSA 402.5 The Globus Toolkit 3 (GT3) 402.5.1 Host environment 412.5.2 Web services engine 422.5.3 Grid services container 422.5.4 GT3 core services 432.5.5 GT3 base services 442.5.6 The GT3 programming model 502.6 OGSA-DAI 532.6.1 OGSA-DAI portTypes 542.6.2 OGSA-DAI functionality 562.6.3 Services interaction in the OGSA-DAI 582.6.4 OGSA-DAI and DAIS 592.7 WSRF 602.7.1 An introduction to WSRF 602.7.2 WSRF and OGSI/GT3 662.7.3 WSRF and OGSA 692.7.4 A summary of WSRF 702.8 Chapter Summary 702.9 Further Reading and Testing 722.10 Key Points 722.11 References 733 The Semantic Grid and Autonomic Computing 77Learning Outcomes 77Chapter Outline 773.1 Introduction 783.2 Metadata and Ontology in the Semantic Web 793.2.1 RDF 813.2.2 Ontology languages 833.2.3 Ontology editors 873.2.4 A summary of Web ontology languages 883.3 Semantic Web Services 883.3.1 DAML-S 893.3.2 OWL-S 903.4 A Layered Structure of the Semantic Grid 913.5 Semantic Grid Activities 923.5.1 Ontology-based Grid resource matching 933.5.2 Semantic workflow registration and discovery in myGrid 943.5.3 Semantic workflow enactment in Geodise 953.5.4 Semantic service annotation and adaptation in ICENI 983.5.5 PortalLab – A Semantic Grid portal toolkit 993.5.6 Data provenance on the Grid 1063.5.7 A summary on the Semantic Grid 1073.6 Autonomic Computing 1083.6.1 What is autonomic computing? 1083.6.2 Features of autonomic computing systems 1093.6.3 Autonomic computing projects 1103.6.4 A vision of autonomic Grid services 1133.7 Chapter Summary 1143.8 Further Reading and Testing 1153.9 Key Points 1163.10 References 116Part Two Basic Services 1214 Grid Security 1234.1 Introduction 1234.2 A Brief Security Primer 1244.3 Cryptography 1274.3.1 Introduction 1274.3.2 Symmetric cryptosystems 1284.3.3 Asymmetric cryptosystems 1294.3.4 Digital signatures 1304.3.5 Public-key certificate 1304.3.6 Certification Authority (CA) 1324.3.7 Firewalls 1334.4 Grid Security 1344.4.1 The Grid Security Infrastructure (GSI) 1344.4.2 Authorization modes in GSI 1364.5 Putting it all Together 1404.5.1 Getting an e-Science certificate 1404.5.2 Managing credentials in Globus 1464.5.3 Generate a client proxy 1484.5.4 Firewall traversal 1484.6 Possible Vulnerabilities 1494.6.1 Authentication 1494.6.2 Proxies 1494.6.3 Authorization 1504.7 Summary 1514.8 Acknowledgements 1514.9 Further Reading 1514.10 References 1525 Grid Monitoring 1535.1 Introduction 1535.2 Grid Monitoring Architecture (GMA) 1545.2.1 Consumer 1555.2.2 The Directory Service 1565.2.3 Producers 1575.2.4 Monitoring data 1595.3 Review Criteria 1615.3.1 Scalable wide-area monitoring 1615.3.2 Resource monitoring 1615.3.3 Cross-API monitoring 1615.3.4 Homogeneous data presentation 1625.3.5 Information searching 1625.3.6 Run-time extensibility 1625.3.7 Filtering/fusing of data 1635.3.8 Open and standard protocols 1635.3.9 Security 1635.3.10 Software availability and dependencies 1635.3.11 Projects that are active and supported; plus licensing 1635.4 An Overview of Grid Monitoring Systems 1645.4.1 Autopilot 1645.4.2 Control and Observation in Distributed Environments (CODE) 1685.4.3 GridICE 1725.4.4 Grid Portals Information Repository (GPIR) 1765.4.5 GridRM 1805.4.6 Hawkeye 1855.4.7 Java Agents for Monitoring and Management (JAMM) 1895.4.8 MapCenter 1925.4.9 Monitoring and Discovery Service (MDS3) 1965.4.10 Mercury 2015.4.11 Network Weather Service 2055.4.12 The Relational Grid Monitoring Architecture (R-GMA) 2095.4.13 visPerf 2145.5 Other Monitoring Systems 2175.5.1 Ganglia 2175.5.2 GridMon 2195.5.3 GRM/PROVE 2205.5.4 Nagios 2215.5.5 NetLogger 2225.5.6 SCALEA-G 2235.6 Summary 2255.6.1 Resource categories 2255.6.2 Native agents 2255.6.3 Architecture 2265.6.4 Interoperability 2265.6.5 Homogeneous data presentation 2265.6.6 Intrusiveness of monitoring 2275.6.7 Information searching and retrieval 2315.7 Chapter Summary 2335.8 Further Reading and Testing 2365.9 Key Points 2365.10 References 236Part Three Job Management and User Interaction 2416 Grid Scheduling and Resource Management 243Learning Objectives 243Chapter Outline 2436.1 Introduction 2446.2 Scheduling Paradigms 2456.2.1 Centralized scheduling 2456.2.2 Distributed scheduling 2466.2.3 Hierarchical scheduling 2486.3 How Scheduling Works 2486.3.1 Resource discovery 2486.3.2 Resource selection 2516.3.3 Schedule generation 2516.3.4 Job execution 2546.4 A Review of Condor, SGE, PBS and LSF 2546.4.1 Condor 2546.4.2 Sun Grid Engine 2696.4.3 The Portable Batch System (PBS) 2746.4.4 LSF 2796.4.5 A comparison of Condor, SGE, PBS and LSF 2886.5 Grid Scheduling with QoS 2906.5.1 AppLeS 2916.5.2 Scheduling in GrADS 2936.5.3 Nimrod/G 2936.5.4 Rescheduling 2956.5.5 Scheduling with heuristics 2966.6 Chapter Summary 2976.7 Further Reading and Testing 2986.8 Key Points 2986.9 References 2997 Workflow Management for the Grid 301Learning Outcomes 301Chapter Outline 3017.1 Introduction 3027.2 The Workflow Management Coalition 3037.2.1 The workflow enactment service 3057.2.2 The workflow engine 3067.2.3 WfMC interfaces 3087.2.4 Other components in the WfMC reference model 3097.2.5 A summary of WfMC reference model 3107.3 Web Services-Oriented Flow Languages 3107.3.1 XLANG 3117.3.2 Web services flow language 3117.3.3 WSCI 3137.3.4 BPEL4WS 3157.3.5 BPML 3177.3.6 A summary of Web services flow languages 3187.4 Grid Services-Oriented Flow Languages 3187.4.1 GSFL 3187.4.2 SWFL 3217.4.3 GWEL 3217.4.4 GALE 3227.4.5 A summary of Grid services flow languages 3237.5 Workflow Management for the Grid 3237.5.1 Grid workflow management projects 3237.5.2 A summary of Grid workflow management 3297.6 Chapter Summary 3307.7 Further Reading and Testing 3317.8 Key Points 3327.9 References 3328 Grid Portals 335Learning Outcomes 335Chapter Outline 3358.1 Introduction 3368.2 First-Generation Grid Portals 3378.2.1 A three-tiered architecture 3378.2.2 Grid portal services 3388.2.3 First-generation Grid portal implementations 3398.2.4 First-generation Grid portal toolkits 3418.2.5 A summary of the four portal tools 3488.2.6 A summary of first-generation Grid portals 3498.3 Second-Generation Grid Portals 3508.3.1 An introduction to portlets 3508.3.2 Portlet specifications 3558.3.3 Portal frameworks supporting portlets 3578.3.4 A Comparison of Jetspeed, WebSphere Portal and GridSphere 3688.3.5 The development of Grid portals with portlets 3698.3.6 A summary on second-generation Grid portals 3718.4 Chapter Summary 3728.5 Further Reading and Testing 3738.6 Key Points 3738.7 References 374Part Four Applications 3779 Grid Applications – Case Studies 379Learning Objectives 379Chapter Outline 3799.1 Introduction 3809.2 GT3 Use Cases 3809.2.1 GT3 in broadcasting 3819.2.2 GT3 in software reuse 3829.2.3 A GT3 bioinformatics application 3879.3 OGSA-DAI Use Cases 3879.3.1 eDiaMoND 3879.3.2 ODD-Genes 3889.4 Resource Management Case Studies 3889.4.1 The UCL Condor pool 3889.4.2 SGE use cases 3899.5 Grid Portal Use Cases 3909.5.1 Chiron 3909.5.2 GENIUS 3909.6 Workflow Management – Discovery Net Use Cases 3919.6.1 Genome annotation 3919.6.2 SARS virus evolution analysis 3919.6.3 Urban air pollution monitoring 3929.6.4 Geo-hazard modelling 3949.7 Semantic Grid – myGrid Use Case 3949.8 Autonomic Computing – AutoMate Use Case 3959.9 Conclusions 3979.10 References 398Glossary 401Index 419