Hongcai Zhang is currently an Assistant Professor with the State Key Laboratory of Internet of Things for Smart City and the Department of Electrical and Computer Engineering at the University of Macau, China. Prior to this, he was a postdoctoral scholar with the University of California, USA, from 2018-2019. His current research interests include Internet of Things for smart energy, optimal operation and optimization of power and transportation systems, and grid integration of distributed energy resources. He has published over 70 JCR Q1/Q2 journal papers with 3 identified as ESI highly cited papers, and is an Associate Editor for IEEE Transactions on Power Systems and the Journal of Modern Power Systems and Clean Energy. Yonghua Song is currently a Chair Professor and the Director of the State Key Laboratory in the Internet of Things for Smart City, both at the University of Macau, China. He is also the Vice President of Chinese Electrotechnical Society, an international member of Academia Europaea, a fellow of the Royal Academy of Engineering (UK), and an IEEE fellow. He has long been engaged in developing renewable electrical power systems and smart energy research. He has published over 200 scientific journal papers and authored/edited 10 books. He has won the second prize of the State Scientific and Technological Progress Award and the prize for Scientific and Technological Progress from the Ho Leung Ho Lee Foundation, China. Ge Chen is currently a Postdoctoral Research Associate with Purdue University, USA. His research interests include the Internet of Things for smart energy, optimal operation, and data-driven optimization under uncertainty.Peipei Yu is currently a Ph.D. candidate in electrical and computer engineering at the University of Macau, China. Her research interests include learning-based control, ancillary services for demand response, and integrated energy systems. She has published 7 JCR Q1/Q2 journal papers.

• 1. IntroductionPART I: ENERGY SYSTEM OPERATION BASED ON CONSTRAINT LEARNING2. Fundamentals of Constraint Learning and Its Application in Deterministic Energy System Operation Problems3. Extending Constraint Learning to Energy System Operations under Uncertain Environments4. Ensuring Accuracy of Constraint learning in the Face of Imbalanced Operational Datasets5. Overcoming Measurement Limitations by Combining Constraint Learning with Measurement Recovery6. Mathematical Insights and Computationally-efficient Implementations of Constraint LearningPART II: ENERGY SYSTEM CONTROL BASED ON SAFE-REINFORCEMENT LEARNING7. Training-efficient Intrinsic-motived Reinforcement Learning Control for Energy Systems with Soft Operation Constraint8. Physical Layer-based Safe Reinforcement Learning Control for Energy Systems with Accurate Formula of Hard Operation Constraint9. Barrier Function-based Safe Reinforcement Learning Control for Energy Systems with Partially Formulable Hard Operation Constraint10. CVaR-based Safe Reinforcement Learning Control for Energy Systems without Formula of Hard Operation Constraint11. Conclusion

Hongcai Zhang is currently an Assistant Professor with the State Key Laboratory of Internet of Things for Smart City and the Department of Electrical and Computer Engineering at the University of Macau, China. Prior to this, he was a postdoctoral scholar with the University of California, USA, from 2018-2019. His current research interests include Internet of Things for smart energy, optimal operation and optimization of power and transportation systems, and grid integration of distributed energy resources. He has published over 70 JCR Q1/Q2 journal papers with 3 identified as ESI highly cited papers, and is an Associate Editor for IEEE Transactions on Power Systems and the Journal of Modern Power Systems and Clean Energy. Yonghua Song is currently a Chair Professor and the Director of the State Key Laboratory in the Internet of Things for Smart City, both at the University of Macau, China. He is also the Vice President of Chinese Electrotechnical Society, an international member of Academia Europaea, a fellow of the Royal Academy of Engineering (UK), and an IEEE fellow. He has long been engaged in developing renewable electrical power systems and smart energy research. He has published over 200 scientific journal papers and authored/edited 10 books. He has won the second prize of the State Scientific and Technological Progress Award and the prize for Scientific and Technological Progress from the Ho Leung Ho Lee Foundation, China. Ge Chen is currently a Postdoctoral Research Associate with Purdue University, USA. His research interests include the Internet of Things for smart energy, optimal operation, and data-driven optimization under uncertainty.Peipei Yu is currently a Ph.D. candidate in electrical and computer engineering at the University of Macau, China. Her research interests include learning-based control, ancillary services for demand response, and integrated energy systems. She has published 7 JCR Q1/Q2 journal papers.

• 1. IntroductionPART I: ENERGY SYSTEM OPERATION BASED ON CONSTRAINT LEARNING2. Fundamentals of Constraint Learning and Its Application in Deterministic Energy System Operation Problems3. Extending Constraint Learning to Energy System Operations under Uncertain Environments4. Ensuring Accuracy of Constraint learning in the Face of Imbalanced Operational Datasets5. Overcoming Measurement Limitations by Combining Constraint Learning with Measurement Recovery6. Mathematical Insights and Computationally-efficient Implementations of Constraint LearningPART II: ENERGY SYSTEM CONTROL BASED ON SAFE-REINFORCEMENT LEARNING7. Training-efficient Intrinsic-motived Reinforcement Learning Control for Energy Systems with Soft Operation Constraint8. Physical Layer-based Safe Reinforcement Learning Control for Energy Systems with Accurate Formula of Hard Operation Constraint9. Barrier Function-based Safe Reinforcement Learning Control for Energy Systems with Partially Formulable Hard Operation Constraint10. CVaR-based Safe Reinforcement Learning Control for Energy Systems without Formula of Hard Operation Constraint11. Conclusion

Hongcai Zhang is currently an Assistant Professor with the State Key Laboratory of Internet of Things for Smart City and the Department of Electrical and Computer Engineering at the University of Macau, China. Prior to this, he was a postdoctoral scholar with the University of California, USA, from 2018-2019. His current research interests include Internet of Things for smart energy, optimal operation and optimization of power and transportation systems, and grid integration of distributed energy resources. He has published over 70 JCR Q1/Q2 journal papers with 3 identified as ESI highly cited papers, and is an Associate Editor for IEEE Transactions on Power Systems and the Journal of Modern Power Systems and Clean Energy. Yonghua Song is currently a Chair Professor and the Director of the State Key Laboratory in the Internet of Things for Smart City, both at the University of Macau, China. He is also the Vice President of Chinese Electrotechnical Society, an international member of Academia Europaea, a fellow of the Royal Academy of Engineering (UK), and an IEEE fellow. He has long been engaged in developing renewable electrical power systems and smart energy research. He has published over 200 scientific journal papers and authored/edited 10 books. He has won the second prize of the State Scientific and Technological Progress Award and the prize for Scientific and Technological Progress from the Ho Leung Ho Lee Foundation, China. Ge Chen is currently a Postdoctoral Research Associate with Purdue University, USA. His research interests include the Internet of Things for smart energy, optimal operation, and data-driven optimization under uncertainty.Peipei Yu is currently a Ph.D. candidate in electrical and computer engineering at the University of Macau, China. Her research interests include learning-based control, ancillary services for demand response, and integrated energy systems. She has published 7 JCR Q1/Q2 journal papers.

• 1. IntroductionPART I: ENERGY SYSTEM OPERATION BASED ON CONSTRAINT LEARNING2. Fundamentals of Constraint Learning and Its Application in Deterministic Energy System Operation Problems3. Extending Constraint Learning to Energy System Operations under Uncertain Environments4. Ensuring Accuracy of Constraint learning in the Face of Imbalanced Operational Datasets5. Overcoming Measurement Limitations by Combining Constraint Learning with Measurement Recovery6. Mathematical Insights and Computationally-efficient Implementations of Constraint LearningPART II: ENERGY SYSTEM CONTROL BASED ON SAFE-REINFORCEMENT LEARNING7. Training-efficient Intrinsic-motived Reinforcement Learning Control for Energy Systems with Soft Operation Constraint8. Physical Layer-based Safe Reinforcement Learning Control for Energy Systems with Accurate Formula of Hard Operation Constraint9. Barrier Function-based Safe Reinforcement Learning Control for Energy Systems with Partially Formulable Hard Operation Constraint10. CVaR-based Safe Reinforcement Learning Control for Energy Systems without Formula of Hard Operation Constraint11. Conclusion