Methodology of Educational Measurement and Assessment – serie

This handbook provides an overview of major developments around diagnostic classification models (DCMs) with regard to modeling, estimation, model checking, scoring, and applications. It brings together not only the current state of the art, but also the theoretical background and models developed for diagnostic classification. The handbook also offers applications and special topics and practical guidelines how to plan and conduct research studies with the help of DCMs. Commonly used models in educational measurement and psychometrics typically assume a single latent trait or at best a small number of latent variables that are aimed at describing individual differences in observed behavior. While this allows simple rankings of test takers along one or a few dimensions, it does not provide a detailed picture of strengths and weaknesses when assessing complex cognitive skills. DCMs, on the other hand, allow the evaluation of test taker performance relative to a potentially large number of skill domains. Most diagnostic models provide a binary mastery/non-mastery classification for each of the assumed test taker attributes representing these skill domains. Attribute profiles can be used for formative decisions as well as for summative purposes, for example in a multiple cut-off procedure that requires mastery on at least a certain subset of skills. The number of DCMs discussed in the literature and applied to a variety of assessment data has been increasing over the past decades, and their appeal to researchers and practitioners alike continues to grow. These models have been used in English language assessment, international large scale assessments, and for feedback for practice exams in preparation of college admission testing, just to name a few. Nowadays, technology-based assessments provide increasingly rich data on a multitude of skills and allow collection of data with respect to multiple types of behaviors. Diagnostic models can be understood as an ideal match for these types of data collections to provide more in-depth information about test taker skills and behavioral tendencies.

This book offers a comprehensive overview of current, innovative approaches to assessing generic and domain-specific learning outcomes in higher education at both national and international levels. It discusses the most significant initiatives over the past decade to develop objective, valid, and reliable assessment tools and presents state-of-the-art procedures to adapt and validate them for use in other countries. The authors highlight key conceptual and methodological challenges connected with intra-national and cross-national assessment of learning outcomes in higher education; introduce novel approaches to improving assessment, evaluation, testing, and measurement practices; and offer exemplary implementation frameworks. Further, they examine the results of and lessons learned from various recent, world-renowned research programs and feasibility studies, and present results from their own studies to provide new insights into how to draw valid conclusions about learning outcomesachieved in various contexts.

This open access book presents a large number of innovations in the world of operational testing. It brings together different but related areas and provides insight in their possibilities, their advantages and drawbacks. The book not only addresses improvements in the quality of educational measurement, innovations in (inter)national large scale assessments, but also several advances in psychometrics and improvements in computerized adaptive testing, and it also offers examples on the impact of new technology in assessment. Due to its nature, the book will appeal to a broad audience within the educational measurement community. It contributes to both theoretical knowledge and also pays attention to practical implementation of innovations in testing technology.

This open access book presents a large number of innovations in the world of operational testing. It brings together different but related areas and provides insight in their possibilities, their advantages and drawbacks. The book not only addresses improvements in the quality of educational measurement, innovations in (inter)national large scale assessments, but also several advances in psychometrics and improvements in computerized adaptive testing, and it also offers examples on the impact of new technology in assessment. Due to its nature, the book will appeal to a broad audience within the educational measurement community. It contributes to both theoretical knowledge and also pays attention to practical implementation of innovations in testing technology.

This open access methodological book summarises existing analysing techniques using data from PIAAC, a study initiated by the OECD that assesses key cognitive and occupational skills of the adult population in more than 40 countries. The approximately 65 PIAAC datasets that has been published worldwide to date has been widely received and used by an interdisciplinary research community. Due to the complex structure of the data, analyses with PIAAC datasets are very challenging. To ensure the quality and significance of these data analyses, it is necessary to instruct users in the correct handling of the data. This methodological book provides a standardised approach to successfully implementing these data analyses. It contains examples of and tools for the analysis of the PIAAC data using different statistical approaches and software, and it offers perspectives from various disciplines. The contributing authors have hands-on experience of using PIAAC data, and/or they have conducteddata analysis workshops with these data.

This open access methodological book summarises existing analysing techniques using data from PIAAC, a study initiated by the OECD that assesses key cognitive and occupational skills of the adult population in more than 40 countries. The approximately 65 PIAAC datasets that has been published worldwide to date has been widely received and used by an interdisciplinary research community. Due to the complex structure of the data, analyses with PIAAC datasets are very challenging. To ensure the quality and significance of these data analyses, it is necessary to instruct users in the correct handling of the data. This methodological book provides a standardised approach to successfully implementing these data analyses. It contains examples of and tools for the analysis of the PIAAC data using different statistical approaches and software, and it offers perspectives from various disciplines. The contributing authors have hands-on experience of using PIAAC data, and/or they have conducteddata analysis workshops with these data.

This book defines and describes a new discipline, named “computational psychometrics,” from the perspective of new methodologies for handling complex data from digital learning and assessment. The editors and the contributing authors discuss how new technology drastically increases the possibilities for the design and administration of learning and assessment systems, and how doing so significantly increases the variety, velocity, and volume of the resulting data. Then they introduce methods and strategies to address the new challenges, ranging from evidence identification and data modeling to the assessment and prediction of learners’ performance in complex settings, as in collaborative tasks, game/simulation-based tasks, and multimodal learning and assessment tasks.Computational psychometrics has thus been defined as a blend of theory-based psychometrics and data-driven approaches from machine learning, artificial intelligence, and data science. All these together provide a better methodological framework for analysing complex data from digital learning and assessments. The term “computational” has been widely adopted by many other areas, as with computational statistics, computational linguistics, and computational economics. In those contexts, “computational” has a meaning similar to the one proposed in this book: a data-driven and algorithm-focused perspective on foundations and theoretical approaches established previously, now extended and, when necessary, reconceived. This interdisciplinarity is already a proven success in many disciplines, from personalized medicine that uses computational statistics to personalized learning that uses, well, computational psychometrics. We expect that this volume will be of interest not just within but beyond the psychometric community.In this volume, experts in psychometrics, machine learning, artificial intelligence, data science and natural language processing illustrate their work, showing how the interdisciplinary expertise of each researcher blends into a coherent methodological framework to deal with complex data from complex virtual interfaces. In the chapters focusing on methodologies, the authors use real data examples to demonstrate how to implement the new methods in practice. The corresponding programming codes in R and Python have been included as snippets in the book and are also available in fuller form in the GitHub code repository that accompanies the book.

This book defines and describes a new discipline, named “computational psychometrics,” from the perspective of new methodologies for handling complex data from digital learning and assessment. The editors and the contributing authors discuss how new technology drastically increases the possibilities for the design and administration of learning and assessment systems, and how doing so significantly increases the variety, velocity, and volume of the resulting data. Then they introduce methods and strategies to address the new challenges, ranging from evidence identification and data modeling to the assessment and prediction of learners’ performance in complex settings, as in collaborative tasks, game/simulation-based tasks, and multimodal learning and assessment tasks.Computational psychometrics has thus been defined as a blend of theory-based psychometrics and data-driven approaches from machine learning, artificial intelligence, and data science. All these together provide a better methodological framework for analysing complex data from digital learning and assessments. The term “computational” has been widely adopted by many other areas, as with computational statistics, computational linguistics, and computational economics. In those contexts, “computational” has a meaning similar to the one proposed in this book: a data-driven and algorithm-focused perspective on foundations and theoretical approaches established previously, now extended and, when necessary, reconceived. This interdisciplinarity is already a proven success in many disciplines, from personalized medicine that uses computational statistics to personalized learning that uses, well, computational psychometrics. We expect that this volume will be of interest not just within but beyond the psychometric community.In this volume, experts in psychometrics, machine learning, artificial intelligence, data science and natural language processing illustrate their work, showing how the interdisciplinary expertise of each researcher blends into a coherent methodological framework to deal with complex data from complex virtual interfaces. In the chapters focusing on methodologies, the authors use real data examples to demonstrate how to implement the new methods in practice. The corresponding programming codes in R and Python have been included as snippets in the book and are also available in fuller form in the GitHub code repository that accompanies the book.

This book 'Essays on Contemporary Psychometrics' provides an overview of contemporary psychometrics, the science devoted to the advancement of quantitative measurement practices in psychology, education and the social sciences. The volume consists of four parts, each having several chapters on cutting-edge work in the field. Part I, General Perspectives on Psychometrics, includes expert views on topics such as psychological models vs. measurement models, using tests in decision making, artificial intelligence, and psychometric network models. Part II, Factor Analysis and Classical Test Theory, the type of psychometrics that is still used most often in the social and behavioral sciences, includes state-of-the-art contributions on test-score reliability, change-score reliability, handling missing data in principal component analysis, test equating, and conditional standard errors of measurement. Part III, Item Response Theory, the leading form ofpsychometrics in modern educational measurement, includes discussions of sampling from many conditional distributions, transparent score reporting, nonparametric item response theory, and targeted testing. Part IV, New Psychometrics, discusses recently developed ideas beyond classical test theory and item response theory, including topics related to computer adaptive testing, response-time modelling, validity indices, diagnostic classification models, and the sparse latent class model for ordinal measurements.Together, these four parts provide an overview of the current state-of-the-art in psychometrics in educational measurement. They are a valuable source of information for graduate students who (intend to) study psychometrics and need an overview of the field, and for researchers interested in the current developments in the field.Chapters [3], [5], [8], [16] and [19] are available open access under a Creative Commons Attribution 4.0 International License via link.springer.com.

This book 'Essays on Contemporary Psychometrics' provides an overview of contemporary psychometrics, the science devoted to the advancement of quantitative measurement practices in psychology, education and the social sciences. The volume consists of four parts, each having several chapters on cutting-edge work in the field. Part I, General Perspectives on Psychometrics, includes expert views on topics such as psychological models vs. measurement models, using tests in decision making, artificial intelligence, and psychometric network models. Part II, Factor Analysis and Classical Test Theory, the type of psychometrics that is still used most often in the social and behavioral sciences, includes state-of-the-art contributions on test-score reliability, change-score reliability, handling missing data in principal component analysis, test equating, and conditional standard errors of measurement. Part III, Item Response Theory, the leading form ofpsychometrics in modern educational measurement, includes discussions of sampling from many conditional distributions, transparent score reporting, nonparametric item response theory, and targeted testing. Part IV, New Psychometrics, discusses recently developed ideas beyond classical test theory and item response theory, including topics related to computer adaptive testing, response-time modelling, validity indices, diagnostic classification models, and the sparse latent class model for ordinal measurements.Together, these four parts provide an overview of the current state-of-the-art in psychometrics in educational measurement. They are a valuable source of information for graduate students who (intend to) study psychometrics and need an overview of the field, and for researchers interested in the current developments in the field.Chapters [3], [5], [8], [16] and [19] are available open access under a Creative Commons Attribution 4.0 International License via link.springer.com.

This Open Access book presents the results of an interdisciplinary research program to utilize data from the multicohort German National Educational Panel Study (NEPS), which included over 100.000 participants in six nationally representative panel studies. Renowned researchers from the fields of sociology, psychology, educational science, economics, and survey methodology have used the (longitudinal) data for their substantive and/or methodological questions and present important results of their research projects. This edited volume contains contributions from the following four topics: (1) Competence Development: Individual Characteristics, Learning Environments, and other Contextual Factors, (2) Educational Transitions and Pathways: Influencing Factors and Outcomes, (3) Vocational Training and Labour Market, and (4) Individuals with Migration Background. It provides essential insights for researchers, postdocs, PhD students, and university students of different scientific disciplines interested in educational sciences as well as for policy makers who have to deal with educational problems in modern societies.

This Open Access book presents the results of an interdisciplinary research program to utilize data from the multicohort German National Educational Panel Study (NEPS), which included over 100.000 participants in six nationally representative panel studies. Renowned researchers from the fields of sociology, psychology, educational science, economics, and survey methodology have used the (longitudinal) data for their substantive and/or methodological questions and present important results of their research projects. This edited volume contains contributions from the following four topics: (1) Competence Development: Individual Characteristics, Learning Environments, and other Contextual Factors, (2) Educational Transitions and Pathways: Influencing Factors and Outcomes, (3) Vocational Training and Labour Market, and (4) Individuals with Migration Background. It provides essential insights for researchers, postdocs, PhD students, and university students of different scientific disciplines interested in educational sciences as well as for policy makers who have to deal with educational problems in modern societies.

This book introduces Q methodology, a method developed nearly a century ago and now recognized as a comprehensive approach for studying subjectivity in the human sciences. The text situates Q methodology historically and delves into its philosophical and technical aspects in relation to educational philosophy, both at collective and individual levels. It showcases Q methodology's relevance and value across numerous educational domains, including counsellor education, cultural foundations, instructional technology, leadership studies, education policy, teaching practices, educational psychology, assessment, measurement, and child studies. It further speculates on Q methodology's future in education research and offers insights for research across various educational fields.With contributions from diverse applications authored by international educational scholars, this edited volume offers a sophisticated introduction to Q methodology in education. It is applicable to the work of researchers across quantitative and qualitative disciplines in education.

This book introduces Q methodology, a method developed nearly a century ago and now recognized as a comprehensive approach for studying subjectivity in the human sciences. The text situates Q methodology historically and delves into its philosophical and technical aspects in relation to educational philosophy, both at collective and individual levels. It showcases Q methodology's relevance and value across numerous educational domains, including counsellor education, cultural foundations, instructional technology, leadership studies, education policy, teaching practices, educational psychology, assessment, measurement, and child studies. It further speculates on Q methodology's future in education research and offers insights for research across various educational fields.With contributions from diverse applications authored by international educational scholars, this edited volume offers a sophisticated introduction to Q methodology in education. It is applicable to the work of researchers across quantitative and qualitative disciplines in education.

This edited volume provides a comprehensive overview of the technology-driven innovations that have transformed National and International Large-Scale Assessments over the past decade, along with the research and considerations that have accompanied these changes. It documents the groundwork for ongoing innovations and explores future directions as these assessments continue to evolve. While the impact of technology on education and assessments is difficult to predict, best practices and emerging principles for transitioning from paper-based to digital formats can be described. The volume offers guidelines and examples from experts who have transitioned assessments like PISA, PIAAC, TIMSS, PIRLS, and NAEP into the digital era. It also highlights research on digital-based assessments, showcasing new insights from log file and process data, as well as innovative domains.

This edited volume provides a platform for experts from various fields to introduce and discuss their different perspectives on the topic of teamwork and collaborative problem solving. It brings together researchers in organizational teaming, educational collaboration, tutoring, simulation, and gaming as well as those involved in statistical and psychometric process modelling. This book seeks to channel this expertise towards advances in the measurement and assessment of cognitive and non-cognitive skills of individuals and teams.

This book addresses problems and challenges that face educational measurement at a time when multipurpose usage of observational data from educational assessments, tests and international surveys has become a growing global trend. While the construction of educational measures and use of educational data offer many opportunities, they also require awareness of the numerous threats to validity and methods of reducing such threats. Written by leading international scholars, the book demonstrates the complexity of educational measurement by addressing three broad and interrelated topics. The first part discusses cognitive abilities, including studies on fluid intelligence, its improvement and its links to aptitude tests for admission to higher education. The second part focuses on the effects of school and teacher-related factors on school outcomes at individual and group levels, and uses international studies to analyze causes. The third part presents analytical techniques and measurement methods to improve reliability, for example factor analysis using Bayesian estimators, bi-factor analysis, model misfit and solutions, and discusses balance issues in reporting test results.The book provides examples of state-of-the-art analytical techniques for pursuing fundamental research problems, and the latest advances in measurement methods, with a focus on validity improvement. Eminent researchers discuss and provide insights into questions such as: Is it possible to train individuals to think at a higher level than normal for their age? What determines prospective preschool teachers’ skill to perceive mathematics-related preschool situations? Can international indicator design and instruments be improved to use trends and national context variables more efficiently? Can indicator data at national, school and class levels be compared easier? Are value-added measures of teacher effectiveness valid when it comes to hiring and promoting teachers? Is money better spenton teacher training than on smaller class-size? How do theory and empirical statistical data intertwine in building structures of understanding? This book is inspired by the career and personal influence of the Swedish scholar Professor Jan-Eric Gustafsson, renowned for his research on individual differences, especially the structure of cognitive abilities, and on the effects of education on knowledge and skills.

This volume brings together educational effectiveness research and international large-scale assessments, demonstrating how the two fields can be applied to inspire and improve each other, and providing readers direct links to instruments that cover a broad range of topics and have been shown to work in more than 70 countries. The book’s initial chapters introduce and summarize recent discussions and developments in the conceptualization, implementation, and evaluation of international large-scale context assessments and provide an outlook on possible future developments. Subsequently, three thematic sections – “Student Background”, “Outcomes of Education Beyond Achievement”, and “Learning in Schools” – each present a series of chapters that provide the conceptual background for a wide range of important topics in education research, policy, and practice. Each chapter defines a conceptual framework that relates recent findings in the educational effectiveness research literature to current issues in education policy and practice. These frameworks were used to develop interesting and relevant indicators that may be used for meaningful reporting from international assessments, other cross-cultural research, or national studies. Using the example of one particular survey (the Programme for International Student Assessment (PISA 2015)), this volume links all theoretical considerations to fully developed questionnaire material that was field trailed and evaluated in questionnaires for students and their parents as well as teachers and principals in their schools. The primary purposes of this book are to inform readers about how education effectiveness research and international large-scale assessments are already interacting to inform research and policymaking; to identify areas where a closer collaboration of both fields or input from other areas could further improve this work; to provide sound theoretical frameworks for future work in both fields; and finally to relate these theoretical debates to currently available and evaluated material for future context assessments.

This book addresses challenges in the theoretically and empirically adequate assessment of competencies in educational settings. It presents the scientific projects of the priority program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes,” which focused on competence assessment across disciplines in Germany.The six-year program coordinated 30 research projects involving experts from the fields of psychology, educational science, and subject-specific didactics. The main reference point for all projects is the concept of “competencies,” which are defined as “context-specific cognitive dispositions that are acquired and needed to successfully cope with certain situations or tasks in specific domains” (Koeppen et al., 2008, p. 62). The projects investigate different aspects of competence assessment: The primary focus lies on the development of cognitive models of competencies, complemented by the construction of psychometric models based on these theoretical models. In turn, the psychometric models constitute the basis for the construction of instruments for effectively measuring competencies.The assessment of competencies plays a key role in optimizing educational processes and improving the effectiveness of educational systems. This book contributes to this challenging endeavor by meeting the need for more integrative, interdisciplinary research on the structure, levels, and development of competencies.

This book summarizes the international evidence on methodological issues in standard setting in education. By critically discussing the standard-setting practices implemented in the Nordic countries and by presenting new methodological approaches, it offers fresh perspectives on the current research. Standard setting targets crucial societal objectives by defining educational benchmarks at different achievement levels, and provides feedback to policy makers, schools and teachers about the strengths and weaknesses of a school system. Given that the consequences of standard setting can be dramatic, the quality of standard setting is a prime concern. If it fails, repercussions can be expected in terms of arbitrary evaluations of educational policy, wrong turns in school or teacher development or misplacement of individual students. Standard setting therefore needs to be accurate, reliable, valid, useful, and defensible.  However, specific evidence on the benefits and limits of different approaches to standard setting is rare and scattered, and there is a particular lack with respect to standard setting in the Nordic countries, where the number of national tests is increasing and there are concerns about the time and effort spent on testing at schools without feedback being provided. Addressing this gap, the book offers a discussion on standard setting by respected experts as well as profound and innovative insights into fundamental aspects of standard setting including conclusions for future methodological and policy-related research.

This book describes how to use test equating methods in practice. The non-commercial software R is used throughout the book to illustrate how to perform different equating methods when scores data are collected under different data collection designs, such as equivalent groups design, single group design, counterbalanced design and non equivalent groups with anchor test design. The R packages equate, kequate and SNSequate, among others, are used to practically illustrate the different methods, while simulated and real data sets illustrate how the methods are conducted with the program R. The book covers traditional equating methods including, mean and linear equating, frequency estimation equating and chain equating, as well as modern equating methods such as kernel equating, local equating and combinations of these. It also offers chapters on observed and true score item response theory equating and discusses recent developments within the equating field. More specifically it covers the issue of including covariates within the equating process, the use of different kernels and ways of selecting bandwidths in kernel equating, and the Bayesian nonparametric estimation of equating functions. It also illustrates how to evaluate equating in practice using simulation and different equating specific measures such as the standard error of equating, percent relative error, different that matters and others.

This book is open access under a CC BY-NC 2.5 license.​​This book describes the extensive contributions made toward the advancement of human assessment by scientists from one of the world’s leading research institutions, Educational Testing Service. The book’s four major sections detail research and development in measurement and statistics, education policy analysis and evaluation, scientific psychology, and validity. Many of the developments presented have become de-facto standards in educational and psychological measurement, including in item response theory (IRT), linking and equating, differential item functioning (DIF), and educational surveys like the National Assessment of Educational Progress (NAEP), the Programme of international Student Assessment (PISA), the Progress of International Reading Literacy Study (PIRLS) and the Trends in Mathematics and Science Study (TIMSS). In addition to its comprehensive coverage of contributions to the theory and methodology of educational and psychological measurement and statistics, the book gives significant attention to ETS work in cognitive, personality, developmental, and social psychology, and to education policy analysis and program evaluation. The chapter authors are long-standing experts who provide broad coverage and thoughtful insights that build upon decades of experience in research and best practices for measurement, evaluation, scientific psychology, and education policy analysis. Opening with a chapter on the genesis of ETS and closing with a synthesis of the enormously diverse set of contributions made over its 70-year history, the book is a useful resource for all interested in the improvement of human assessment.

This book offers a comprehensive overview of current, innovative approaches to assessing generic and domain-specific learning outcomes in higher education at both national and international levels. It discusses the most significant initiatives over the past decade to develop objective, valid, and reliable assessment tools and presents state-of-the-art procedures to adapt and validate them for use in other countries. The authors highlight key conceptual and methodological challenges connected with intra-national and cross-national assessment of learning outcomes in higher education; introduce novel approaches to improving assessment, evaluation, testing, and measurement practices; and offer exemplary implementation frameworks. Further, they examine the results of and lessons learned from various recent, world-renowned research programs and feasibility studies, and present results from their own studies to provide new insights into how to draw valid conclusions about learning outcomesachieved in various contexts.

This edited volume provides a platform for experts from various fields to introduce and discuss their different perspectives on the topic of teamwork and collaborative problem solving. It brings together researchers in organizational teaming, educational collaboration, tutoring, simulation, and gaming as well as those involved in statistical and psychometric process modelling. This book seeks to channel this expertise towards advances in the measurement and assessment of cognitive and non-cognitive skills of individuals and teams.

This book addresses problems and challenges that face educational measurement at a time when multipurpose usage of observational data from educational assessments, tests and international surveys has become a growing global trend. While the construction of educational measures and use of educational data offer many opportunities, they also require awareness of the numerous threats to validity and methods of reducing such threats. Written by leading international scholars, the book demonstrates the complexity of educational measurement by addressing three broad and interrelated topics. The first part discusses cognitive abilities, including studies on fluid intelligence, its improvement and its links to aptitude tests for admission to higher education. The second part focuses on the effects of school and teacher-related factors on school outcomes at individual and group levels, and uses international studies to analyze causes. The third part presents analytical techniques and measurement methods to improve reliability, for example factor analysis using Bayesian estimators, bi-factor analysis, model misfit and solutions, and discusses balance issues in reporting test results.The book provides examples of state-of-the-art analytical techniques for pursuing fundamental research problems, and the latest advances in measurement methods, with a focus on validity improvement. Eminent researchers discuss and provide insights into questions such as: Is it possible to train individuals to think at a higher level than normal for their age? What determines prospective preschool teachers’ skill to perceive mathematics-related preschool situations? Can international indicator design and instruments be improved to use trends and national context variables more efficiently? Can indicator data at national, school and class levels be compared easier? Are value-added measures of teacher effectiveness valid when it comes to hiring and promoting teachers? Is money better spenton teacher training than on smaller class-size? How do theory and empirical statistical data intertwine in building structures of understanding? This book is inspired by the career and personal influence of the Swedish scholar Professor Jan-Eric Gustafsson, renowned for his research on individual differences, especially the structure of cognitive abilities, and on the effects of education on knowledge and skills.

This volume brings together educational effectiveness research and international large-scale assessments, demonstrating how the two fields can be applied to inspire and improve each other, and providing readers direct links to instruments that cover a broad range of topics and have been shown to work in more than 70 countries. The book’s initial chapters introduce and summarize recent discussions and developments in the conceptualization, implementation, and evaluation of international large-scale context assessments and provide an outlook on possible future developments. Subsequently, three thematic sections – “Student Background”, “Outcomes of Education Beyond Achievement”, and “Learning in Schools” – each present a series of chapters that provide the conceptual background for a wide range of important topics in education research, policy, and practice. Each chapter defines a conceptual framework that relates recent findings in the educational effectiveness research literature to current issues in education policy and practice. These frameworks were used to develop interesting and relevant indicators that may be used for meaningful reporting from international assessments, other cross-cultural research, or national studies. Using the example of one particular survey (the Programme for International Student Assessment (PISA 2015)), this volume links all theoretical considerations to fully developed questionnaire material that was field trailed and evaluated in questionnaires for students and their parents as well as teachers and principals in their schools. The primary purposes of this book are to inform readers about how education effectiveness research and international large-scale assessments are already interacting to inform research and policymaking; to identify areas where a closer collaboration of both fields or input from other areas could further improve this work; to provide sound theoretical frameworks for future work in both fields; and finally to relate these theoretical debates to currently available and evaluated material for future context assessments.

This book addresses challenges in the theoretically and empirically adequate assessment of competencies in educational settings. It presents the scientific projects of the priority program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes,” which focused on competence assessment across disciplines in Germany.The six-year program coordinated 30 research projects involving experts from the fields of psychology, educational science, and subject-specific didactics. The main reference point for all projects is the concept of “competencies,” which are defined as “context-specific cognitive dispositions that are acquired and needed to successfully cope with certain situations or tasks in specific domains” (Koeppen et al., 2008, p. 62). The projects investigate different aspects of competence assessment: The primary focus lies on the development of cognitive models of competencies, complemented by the construction of psychometric models based on these theoretical models. In turn, the psychometric models constitute the basis for the construction of instruments for effectively measuring competencies.The assessment of competencies plays a key role in optimizing educational processes and improving the effectiveness of educational systems. This book contributes to this challenging endeavor by meeting the need for more integrative, interdisciplinary research on the structure, levels, and development of competencies.

This book summarizes the international evidence on methodological issues in standard setting in education. By critically discussing the standard-setting practices implemented in the Nordic countries and by presenting new methodological approaches, it offers fresh perspectives on the current research. Standard setting targets crucial societal objectives by defining educational benchmarks at different achievement levels, and provides feedback to policy makers, schools and teachers about the strengths and weaknesses of a school system. Given that the consequences of standard setting can be dramatic, the quality of standard setting is a prime concern. If it fails, repercussions can be expected in terms of arbitrary evaluations of educational policy, wrong turns in school or teacher development or misplacement of individual students. Standard setting therefore needs to be accurate, reliable, valid, useful, and defensible.  However, specific evidence on the benefits and limits of different approaches to standard setting is rare and scattered, and there is a particular lack with respect to standard setting in the Nordic countries, where the number of national tests is increasing and there are concerns about the time and effort spent on testing at schools without feedback being provided. Addressing this gap, the book offers a discussion on standard setting by respected experts as well as profound and innovative insights into fundamental aspects of standard setting including conclusions for future methodological and policy-related research.

This book describes how to use test equating methods in practice. The non-commercial software R is used throughout the book to illustrate how to perform different equating methods when scores data are collected under different data collection designs, such as equivalent groups design, single group design, counterbalanced design and non equivalent groups with anchor test design. The R packages equate, kequate and SNSequate, among others, are used to practically illustrate the different methods, while simulated and real data sets illustrate how the methods are conducted with the program R. The book covers traditional equating methods including, mean and linear equating, frequency estimation equating and chain equating, as well as modern equating methods such as kernel equating, local equating and combinations of these. It also offers chapters on observed and true score item response theory equating and discusses recent developments within the equating field. More specifically it covers the issue of including covariates within the equating process, the use of different kernels and ways of selecting bandwidths in kernel equating, and the Bayesian nonparametric estimation of equating functions. It also illustrates how to evaluate equating in practice using simulation and different equating specific measures such as the standard error of equating, percent relative error, different that matters and others.