Advances in Analytics for Learning and Teaching – serie

This book includes a collection of expanded papers from the 2019 Sino-German Symposium on AI-supported educational technologies, which was held in Wuhan, China, March, 2019. The contributors are distinguished researchers from computer science and learning science. The contributions are organized in four sections: (1) Overviews and systematic perspectives , (2) Example Systems,(3) Algorithms, and (4) Insights gained from empirical studies. For example, different data mining and machine learning methods to quantify different profiles of a learner in different learning situations (including interaction patterns, cognitive modes, knowledge skills, interests and emotions etc.) as well as connections to measurements in psychology and learning sciences are discussed in the chapters.

This book includes a collection of expanded papers from the 2019 Sino-German Symposium on AI-supported educational technologies, which was held in Wuhan, China, March, 2019. The contributors are distinguished researchers from computer science and learning science. The contributions are organized in four sections: (1) Overviews and systematic perspectives , (2) Example Systems,(3) Algorithms, and (4) Insights gained from empirical studies. For example, different data mining and machine learning methods to quantify different profiles of a learner in different learning situations (including interaction patterns, cognitive modes, knowledge skills, interests and emotions etc.) as well as connections to measurements in psychology and learning sciences are discussed in the chapters.

The book aims to advance global knowledge and practice in applying data science to transform higher education learning and teaching to improve personalization, access and effectiveness of education for all. Currently, higher education institutions and involved stakeholders can derive multiple benefits from educational data mining and learning analytics by using different data analytics strategies to produce summative, real-time, and predictive or prescriptive insights and recommendations. Educational data mining refers to the process of extracting useful information out of a large collection of complex educational datasets while learning analytics emphasizes insights and responses to real-time learning processes based on educational information from digital learning environments, administrative systems, and social platforms.This volume provides insight into the emerging paradigms, frameworks, methods and processes of managing change to better facilitate organizational transformation toward implementation of educational data mining and learning analytics. It features current research exploring the (a) theoretical foundation and empirical evidence of the adoption of learning analytics, (b) technological infrastructure and staff capabilities required, as well as (c) case studies that describe current practices and experiences in the use of data analytics in higher education.

The book aims to advance global knowledge and practice in applying data science to transform higher education learning and teaching to improve personalization, access and effectiveness of education for all. Currently, higher education institutions and involved stakeholders can derive multiple benefits from educational data mining and learning analytics by using different data analytics strategies to produce summative, real-time, and predictive or prescriptive insights and recommendations. Educational data mining refers to the process of extracting useful information out of a large collection of complex educational datasets while learning analytics emphasizes insights and responses to real-time learning processes based on educational information from digital learning environments, administrative systems, and social platforms.This volume provides insight into the emerging paradigms, frameworks, methods and processes of managing change to better facilitate organizational transformation toward implementation of educational data mining and learning analytics. It features current research exploring the (a) theoretical foundation and empirical evidence of the adoption of learning analytics, (b) technological infrastructure and staff capabilities required, as well as (c) case studies that describe current practices and experiences in the use of data analytics in higher education.

This book A Beginner’s Guide to Learning Analytics is designed to meet modern educational trends’ needs. It is addressed to readers who have no prior knowledge of learning analytics and functions as an introductory text to learning analytics for those who want to do more with evaluation/assessment in their organizations. The book is useful to all who need to evaluate their learning and teaching strategies. It aims to bring greater efficiency and deeper engagement to individual students, learning communities, and educators.Covered here are the key concepts linked to learning analytics for researchers and practitioners interested in learning analytics. This book helps those who want to apply analytics to learning and development programs and helps educational institutions to identify learners who require support and provide a more personalized learning experience. Like chapters show diverse uses of learning analytics to enhance student and faculty performance. It presents a coherent framework for the effective translation of learning analytics research for educational practice to its practical application in different educational domains. This book provides educators and researchers with the tools and frameworks to effectively make sense of and use data and analytics in their everyday practice. This book will be a valuable addition to researchers’ bookshelves.

This book A Beginner’s Guide to Learning Analytics is designed to meet modern educational trends’ needs. It is addressed to readers who have no prior knowledge of learning analytics and functions as an introductory text to learning analytics for those who want to do more with evaluation/assessment in their organizations. The book is useful to all who need to evaluate their learning and teaching strategies. It aims to bring greater efficiency and deeper engagement to individual students, learning communities, and educators.Covered here are the key concepts linked to learning analytics for researchers and practitioners interested in learning analytics. This book helps those who want to apply analytics to learning and development programs and helps educational institutions to identify learners who require support and provide a more personalized learning experience. Like chapters show diverse uses of learning analytics to enhance student and faculty performance. It presents a coherent framework for the effective translation of learning analytics research for educational practice to its practical application in different educational domains. This book provides educators and researchers with the tools and frameworks to effectively make sense of and use data and analytics in their everyday practice. This book will be a valuable addition to researchers’ bookshelves.

This edited volume fills the gaps in existing literature on visualization and dashboard design for learning analytics. To do so, it presents critical tips to stakeholders and acts as guide to efficient implementation. The book covers the following topics: visualization and dashboard design for learning analytics, visualization and dashboard preferences of stakeholders, learners’ patterns on the dashboard, usability of visualization techniques and the dashboard, dashboard and intervention design, learning and instructional design for learning analytics, privacy and security issues about the dashboard, and future directions of visualization and dashboard design.This book will be of interest to researchers with interest in learning analytics and data analytics, teachers and students in higher education institutions and instructional designers, as it includes contributions from a wide variety of educational and psychological researchers, engineers, instructional designers, learning scientists, and computer scientists interested in learning analytics.

This edited volume fills the gaps in existing literature on visualization and dashboard design for learning analytics. To do so, it presents critical tips to stakeholders and acts as guide to efficient implementation. The book covers the following topics: visualization and dashboard design for learning analytics, visualization and dashboard preferences of stakeholders, learners’ patterns on the dashboard, usability of visualization techniques and the dashboard, dashboard and intervention design, learning and instructional design for learning analytics, privacy and security issues about the dashboard, and future directions of visualization and dashboard design.This book will be of interest to researchers with interest in learning analytics and data analytics, teachers and students in higher education institutions and instructional designers, as it includes contributions from a wide variety of educational and psychological researchers, engineers, instructional designers, learning scientists, and computer scientists interested in learning analytics.

In this book, we primarily focus on studies that provide objective, unobtrusive, and innovative measures (e.g., indirect measures, content analysis, or analysis of trace data) of SEL skills (e.g., collaboration, creativity, persistence), relying primarily on learning analytics methods and approaches that would potentially allow for expanding the assessment of SEL skills and competencies at scale. What makes the position of learning analytics pivotal in this endeavor to redefine measurement of SEL skills are constant changes and advancements in learning environments and the quality and quantity of data collected about learners and the process of learning. Contemporary learning environments that utilize virtual and augmented reality to enhance learning opportunities accommodate for designing tasks and activities that allow learners to elicit behaviors (either in face-to-face or online context) not being captured in traditional educational settings. Novel insights provided in the book span across diverse types of learning contexts and learner populations. Specifically, the book addresses relevant and emerging theories and frameworks (in various disciplines such as education, psychology, or workforce) that inform assessments of SEL skills and competencies. In so doing, the book maps the landscape of the novel learning analytics methods and approaches, along with their application in the SEL assessment for K-12 learners as well as adult learners. Critical to the notion of the SEL assessment are data sources. In that sense, the book outlines where and how data related to learners' 21st century skills and competencies can be measured and collected. Linking theory to data, the book further discusses tools and methods that are being used to operationalize SEL and link relevant skills and competencies with cognitive assessment. Finally, the book addresses aspects of generalizability and applicability, showing promising approaches for translating research findings into actionable insights that would inform various stakeholders (e.g., learners, instructors, administrators, policy makers).

In this book, we primarily focus on studies that provide objective, unobtrusive, and innovative measures (e.g., indirect measures, content analysis, or analysis of trace data) of SEL skills (e.g., collaboration, creativity, persistence), relying primarily on learning analytics methods and approaches that would potentially allow for expanding the assessment of SEL skills and competencies at scale. What makes the position of learning analytics pivotal in this endeavor to redefine measurement of SEL skills are constant changes and advancements in learning environments and the quality and quantity of data collected about learners and the process of learning. Contemporary learning environments that utilize virtual and augmented reality to enhance learning opportunities accommodate for designing tasks and activities that allow learners to elicit behaviors (either in face-to-face or online context) not being captured in traditional educational settings. Novel insights provided in the book span across diverse types of learning contexts and learner populations. Specifically, the book addresses relevant and emerging theories and frameworks (in various disciplines such as education, psychology, or workforce) that inform assessments of SEL skills and competencies. In so doing, the book maps the landscape of the novel learning analytics methods and approaches, along with their application in the SEL assessment for K-12 learners as well as adult learners. Critical to the notion of the SEL assessment are data sources. In that sense, the book outlines where and how data related to learners' 21st century skills and competencies can be measured and collected. Linking theory to data, the book further discusses tools and methods that are being used to operationalize SEL and link relevant skills and competencies with cognitive assessment. Finally, the book addresses aspects of generalizability and applicability, showing promising approaches for translating research findings into actionable insights that would inform various stakeholders (e.g., learners, instructors, administrators, policy makers).

Digital Education is recognised as a key transformative innovation for K-12 school and university teaching and learning, as well as, for professional development and vocational training. As a result, blended and online courses are nowadays widely deployed to meet the needs of K-12, higher education and vocational training students, as well as, the needs for professional development of in-service professionals. In this context, important professional roles in digital education and training, such as, the Instructional Designers, who design and develop online and blended courses, and the Trainers or Tutors who support the delivery of these online and blended courses, require new professional competences compared to those assumed at the traditional face to face education and tra ining programs.This is particularly relevant today, in the post Covid-19 era, where educational organisations, leaders and teachers are challenged with reinventing their teaching and learning environments to offer higher quality, more accessible and inclusive teaching, learning and assessment. Educational Data Literacy (EDL) is a core competence for all education professionals, including school teachers, instructional designers and tutors of online and blended learning courses, as well as educational institutions' leaders. Nevertheless, existing professional competence frameworks for educators pay little attention to EDL, missing out the potential of using emerging EDL methods and tools in online and blended teaching and learning - thus there is a need for extending existing professional competence frameworks for educators with new competences to accommodate the emerging field of EDL. To this end, this brief monograph presents a comprehensive proposal of an Educational Data Literacy Competence Profile (EDL-CP) framework for education professionals, as well as, exemplary learning outcomes for the proposed EDL-CP framework, and use-case examples for indicative target groups, namely instructional designers, e-Trainers and K-12 school teachers. The work of this book has been produced within the project "Learn2Analyze — An Academia-Industry Knowledge Alliance for enhancing Online Training Professionals’ (Instructional Designers and e-Trainers) Competences in Educational Data Analytics" which is co-funded by European Commission through the Erasmus+ Program (Cooperation for innovation and the exchange of good practices – Knowledge Alliances).

Digital Education is recognised as a key transformative innovation for K-12 school and university teaching and learning, as well as, for professional development and vocational training. As a result, blended and online courses are nowadays widely deployed to meet the needs of K-12, higher education and vocational training students, as well as, the needs for professional development of in-service professionals. In this context, important professional roles in digital education and training, such as, the Instructional Designers, who design and develop online and blended courses, and the Trainers or Tutors who support the delivery of these online and blended courses, require new professional competences compared to those assumed at the traditional face to face education and tra ining programs.This is particularly relevant today, in the post Covid-19 era, where educational organisations, leaders and teachers are challenged with reinventing their teaching and learning environments to offer higher quality, more accessible and inclusive teaching, learning and assessment. Educational Data Literacy (EDL) is a core competence for all education professionals, including school teachers, instructional designers and tutors of online and blended learning courses, as well as educational institutions' leaders. Nevertheless, existing professional competence frameworks for educators pay little attention to EDL, missing out the potential of using emerging EDL methods and tools in online and blended teaching and learning - thus there is a need for extending existing professional competence frameworks for educators with new competences to accommodate the emerging field of EDL. To this end, this brief monograph presents a comprehensive proposal of an Educational Data Literacy Competence Profile (EDL-CP) framework for education professionals, as well as, exemplary learning outcomes for the proposed EDL-CP framework, and use-case examples for indicative target groups, namely instructional designers, e-Trainers and K-12 school teachers. The work of this book has been produced within the project "Learn2Analyze — An Academia-Industry Knowledge Alliance for enhancing Online Training Professionals’ (Instructional Designers and e-Trainers) Competences in Educational Data Analytics" which is co-funded by European Commission through the Erasmus+ Program (Cooperation for innovation and the exchange of good practices – Knowledge Alliances).

This edited volume remedies existing deficiencies in the literature on artificial intelligence and education in the context of work. The topics addressed by this book are: • Supporting formal and informal learning through AI• Human-machine collaboration for learning at the workplace, including the potential of human-AI interaction in professional and vocational education contexts, design, use, and evaluation of human-AI hybrid systems for learning• Intelligent and Interactive Technologies for Learning, including natural language processing and speech technologies; data mining and machine learning;  knowledge representation and reasoning; semantic web technologies,  chat bot-mediated learning, and conversational learning, • AI-enabled applications for skills management and personalized learning, such as AI-enabled coaching, personalized skill management, and intelligent tutoring systems. • Case studies for the implementation and use of AI-enabled learning and performance solutions, such as personal learning experience platforms, and automated performance feedback.

This edited volume remedies existing deficiencies in the literature on artificial intelligence and education in the context of work. The topics addressed by this book are: • Supporting formal and informal learning through AI• Human-machine collaboration for learning at the workplace, including the potential of human-AI interaction in professional and vocational education contexts, design, use, and evaluation of human-AI hybrid systems for learning• Intelligent and Interactive Technologies for Learning, including natural language processing and speech technologies; data mining and machine learning;  knowledge representation and reasoning; semantic web technologies,  chat bot-mediated learning, and conversational learning, • AI-enabled applications for skills management and personalized learning, such as AI-enabled coaching, personalized skill management, and intelligent tutoring systems. • Case studies for the implementation and use of AI-enabled learning and performance solutions, such as personal learning experience platforms, and automated performance feedback.

Educational Data Analytics (EDA) have been attributed with significant benefits for enhancing on-demand personalized educational support of individual learners as well as reflective course (re)design for achieving more authentic teaching, learning and assessment experiences integrated into real work-oriented tasks.This open access textbook is a tutorial for developing, practicing and self-assessing core competences on educational data analytics for digital teaching and learning. It combines theoretical knowledge on core issues related to collecting, analyzing, interpreting and using educational data, including ethics and privacy concerns. The textbook provides questions and teaching materials/ learning activities as quiz tests of multiple types of questions, added after each section, related to the topic studied or the video(s) referenced. These activities reproduce real-life contexts by using a suitable use case scenario (storytelling), encouraging learners to link theory with practice; self-assessed assignments enabling learners to apply their attained knowledge and acquired competences on EDL. By studying this book, you will know where to locate useful educational data in different sources and understand their limitations; know the basics for managing educational data to make them useful; understand relevant methods; and be able to use relevant tools; know the basics for organising, analysing, interpreting and presenting learner-generated data within their learning context, understand relevant learning analytics methods and be able to use relevant learning analytics tools; know the basics for analysing and interpreting educational data to facilitate educational decision making, including course and curricula design, understand relevant teaching analytics methods and be able to use relevant teaching analytics tools; understand issues related with educational data ethics and privacy.This book is intended for school leadersand teachers engaged in blended (using the flipped classroom model) and online (during COVID-19 crisis and beyond) teaching and learning; e-learning professionals (such as, instructional designers and e-tutors) of online and blended courses; instructional technologists; researchers as well as undergraduate and postgraduate university students studying education, educational technology and relevant fields.

Educational Data Analytics (EDA) have been attributed with significant benefits for enhancing on-demand personalized educational support of individual learners as well as reflective course (re)design for achieving more authentic teaching, learning and assessment experiences integrated into real work-oriented tasks.This open access textbook is a tutorial for developing, practicing and self-assessing core competences on educational data analytics for digital teaching and learning. It combines theoretical knowledge on core issues related to collecting, analyzing, interpreting and using educational data, including ethics and privacy concerns. The textbook provides questions and teaching materials/ learning activities as quiz tests of multiple types of questions, added after each section, related to the topic studied or the video(s) referenced. These activities reproduce real-life contexts by using a suitable use case scenario (storytelling), encouraging learners to link theory with practice; self-assessed assignments enabling learners to apply their attained knowledge and acquired competences on EDL. By studying this book, you will know where to locate useful educational data in different sources and understand their limitations; know the basics for managing educational data to make them useful; understand relevant methods; and be able to use relevant tools; know the basics for organising, analysing, interpreting and presenting learner-generated data within their learning context, understand relevant learning analytics methods and be able to use relevant learning analytics tools; know the basics for analysing and interpreting educational data to facilitate educational decision making, including course and curricula design, understand relevant teaching analytics methods and be able to use relevant teaching analytics tools; understand issues related with educational data ethics and privacy.This book is intended for school leadersand teachers engaged in blended (using the flipped classroom model) and online (during COVID-19 crisis and beyond) teaching and learning; e-learning professionals (such as, instructional designers and e-tutors) of online and blended courses; instructional technologists; researchers as well as undergraduate and postgraduate university students studying education, educational technology and relevant fields.

This book is about practicable learning analytics, that is able to become a successful part of practice, ultimately leading to improved learning and teaching. The aim of the book is to shift our perspective on learning analytics creation and implementation from that of “designing of” technology to that of “designing for” a system of practice. That is, any successful implementation of learning analytics requires a systematic approach, which the book explains through the lens of the Information Systems Artefact, constituting of the three interdependent artefacts: “technical”, “information” and “social”.The contributions of this book go beyond a consideration of particular humans such as teachers and students, and their individual activities to consider the larger systems of activity of which analytics become part of. The chapters included in this book present different cases of learning analytics implementation across countries, and the related opportunities and challenges related to generalizability of the results.The book is written for designers, students and educators of learning analytics who aim to improve learning and teaching through learning analytics.

This book is about practicable learning analytics, that is able to become a successful part of practice, ultimately leading to improved learning and teaching. The aim of the book is to shift our perspective on learning analytics creation and implementation from that of “designing of” technology to that of “designing for” a system of practice. That is, any successful implementation of learning analytics requires a systematic approach, which the book explains through the lens of the Information Systems Artefact, constituting of the three interdependent artefacts: “technical”, “information” and “social”.The contributions of this book go beyond a consideration of particular humans such as teachers and students, and their individual activities to consider the larger systems of activity of which analytics become part of. The chapters included in this book present different cases of learning analytics implementation across countries, and the related opportunities and challenges related to generalizability of the results.The book is written for designers, students and educators of learning analytics who aim to improve learning and teaching through learning analytics.

This book integrates foundational ideas from psychology, immersive digital learning environments supported by theories and methods of the learning sciences, particularly in pursuit of questions of cognition, behavior and emotion factors in digital learning experiences. New and emerging foundations of theory and analysis based on observation of digital traces are enhanced by data science, particularly machine learning, with extensions to deep learning, natural language processing and artificial intelligence brought into service to better understand higher-order thinking capacities such as self-regulation, collaborative problem-solving and social construction of knowledge. As a result, this edited volume presents a collection of indicators or measurements focusing on learning processes and related behavior, (meta-)cognition, emotion and motivation, as well as social processes. In addition, each section of the book includes an invited commentary from a related field, such as educational psychology, cognitive science, learning science, etc.

This book integrates foundational ideas from psychology, immersive digital learning environments supported by theories and methods of the learning sciences, particularly in pursuit of questions of cognition, behavior and emotion factors in digital learning experiences. New and emerging foundations of theory and analysis based on observation of digital traces are enhanced by data science, particularly machine learning, with extensions to deep learning, natural language processing and artificial intelligence brought into service to better understand higher-order thinking capacities such as self-regulation, collaborative problem-solving and social construction of knowledge. As a result, this edited volume presents a collection of indicators or measurements focusing on learning processes and related behavior, (meta-)cognition, emotion and motivation, as well as social processes. In addition, each section of the book includes an invited commentary from a related field, such as educational psychology, cognitive science, learning science, etc.

This book is about the current state of research in online assessment. The growth of this field is set to accelerate exponentially with emerging opportunities for automatic data collection and analysis. Yet, the future of online assessment faces major challenges including, perhaps most importantly, the extent to which assessments, when enabled by technology, can serve simultaneously the needs of learners, teachers and those of the enterprise of education. This book details, specifically, the multiple ways in which online assessment can be utilized, such as:Providing virtual coaching or tutoringOffering appropriate scaffoldingAllowing analysis of student decision-making  Providing the mechanism for students to review and comment on each others digital creationsCreating a space for online discussionProviding expert coaching for modeling and animation workWith the increased availability of vast and highly varied amounts of data from learners, teachers, learning environments, and administrative systems within educational settings, further opportunities arise for advancing pedagogical assessment practice (Ifenthaler et al., 2018). This book fully details these opportunities, as well as privileges and constraints of analytics-enhanced assessment,  harnessing formative as well as summative data from learners and their contexts in order to facilitate learning processes in near real-time and help decisionmakers to improve learning environments.

This book is about the current state of research in online assessment. The growth of this field is set to accelerate exponentially with emerging opportunities for automatic data collection and analysis. Yet, the future of online assessment faces major challenges including, perhaps most importantly, the extent to which assessments, when enabled by technology, can serve simultaneously the needs of learners, teachers and those of the enterprise of education. This book details, specifically, the multiple ways in which online assessment can be utilized, such as:Providing virtual coaching or tutoringOffering appropriate scaffoldingAllowing analysis of student decision-making  Providing the mechanism for students to review and comment on each others digital creationsCreating a space for online discussionProviding expert coaching for modeling and animation workWith the increased availability of vast and highly varied amounts of data from learners, teachers, learning environments, and administrative systems within educational settings, further opportunities arise for advancing pedagogical assessment practice (Ifenthaler et al., 2018). This book fully details these opportunities, as well as privileges and constraints of analytics-enhanced assessment,  harnessing formative as well as summative data from learners and their contexts in order to facilitate learning processes in near real-time and help decisionmakers to improve learning environments.

This book shows how artificial intelligence grounded in learning theories can promote individual learning, team productivity and multidisciplinary knowledge-building. It advances the learning sciences by integrating learning theory with computational biology and complexity, offering an updated mechanism of learning, which integrates previous theories, provides a basis for scaling from individuals to societies, and unifies models of psychology, sociology and cultural studies. The book provides a road map for the development of AI that addresses the central problems of learning theory in the age of artificial intelligence including: optimizing human-machine collaborationpromoting individual learningbalancing personalization with privacydealing with biases and promoting fairnessexplaining decisions and recommendations to build trust and accountabilitycontinuously balancing and adapting to individual, team and organizational goalsgenerating and generalizing knowledge across fields and domainsThe book will be of interest to educational professionals, researchers, and developers of educational technology that utilize artificial intelligence.

This book shows how artificial intelligence grounded in learning theories can promote individual learning, team productivity and multidisciplinary knowledge-building. It advances the learning sciences by integrating learning theory with computational biology and complexity, offering an updated mechanism of learning, which integrates previous theories, provides a basis for scaling from individuals to societies, and unifies models of psychology, sociology and cultural studies. The book provides a road map for the development of AI that addresses the central problems of learning theory in the age of artificial intelligence including: optimizing human-machine collaborationpromoting individual learningbalancing personalization with privacydealing with biases and promoting fairnessexplaining decisions and recommendations to build trust and accountabilitycontinuously balancing and adapting to individual, team and organizational goalsgenerating and generalizing knowledge across fields and domainsThe book will be of interest to educational professionals, researchers, and developers of educational technology that utilize artificial intelligence.

This volume explores the rapidly advancing field of technology-supported knowledge assessment. Across academia, research on learning and instruction, AI-based analysis, psychology, and education, there is a pressing need for a comprehensive collection of foundations and methodologies related to knowledge. While the market offers books on individual and locally developed methods, a holistic overview is currently lacking. It aims to fill that gap, inspiring projects globally and benefiting knowledge-intensive developments in both digital and traditional learning environments. Understanding the state and processes of knowledge often poses a bottleneck in the quality of designs and implementations. This book addresses this challenge by focusing on mostly automated, easy-to-implement strategies, supporting the crucial task of understanding knowledge.

This volume explores the rapidly advancing field of technology-supported knowledge assessment. Across academia, research on learning and instruction, AI-based analysis, psychology, and education, there is a pressing need for a comprehensive collection of foundations and methodologies related to knowledge. While the market offers books on individual and locally developed methods, a holistic overview is currently lacking. It aims to fill that gap, inspiring projects globally and benefiting knowledge-intensive developments in both digital and traditional learning environments. Understanding the state and processes of knowledge often poses a bottleneck in the quality of designs and implementations. This book addresses this challenge by focusing on mostly automated, easy-to-implement strategies, supporting the crucial task of understanding knowledge.