Gunther Sigott – författare

This book marks ten years of language test development and language testing research in Austria. Part I describes the development of large-scale tests at all levels of the educational system. The documentation covers German as the language of instruction for the majority of pupils, the so-called modern foreign languages English, French, Italian and Spanish, as well as the classical languages Latin and Ancient Greek. Part II brings together a considerable body of research which has been conducted in the course and the wake of the development and implementation of the new assessment philosophy during the past ten years. The studies are accommodated in a framework of test validation, which could serve as orientation for future research in language testing in Austria.Das Buch blickt auf zehn Jahre professioneller Testentwicklung und Sprachtestforschung in Österreich zurück. Teil I beschreibt die Entwicklung von Testsystemen auf allen Ebenen des Bildungssystems. Die Dokumentation umfasst Deutsch, das für die große Mehrheit Unterrichtssprache ist, die sogenannten modernen Fremdsprachen Englisch, Französisch, Italienisch und Spanisch sowie die klassischen Sprachen Latein und Altgriechisch. Teil II umfasst eine beachtliche Anzahl von Untersuchungen, die während und nach der Entwicklung und Implementierung der neuen Prüfungsphilosophie in den letzten zehn Jahren durchgeführt wurden. Die Untersuchungen werden in ein Validierungskonzept eingeordnet, das als Orientierung für zukünftige Sprachtestforschung in Österreich dienen könnte.

In this book, the author investigates a central issue in language testing research: What are the key features that contribute to item difficulty in a reading test? Results of various statistical analyses of the multiple-choice reading items from the Austrian 2009 baseline reading test show a significant correlation between empirical item difficulty and both cognitive processes and metacognitive strategies. The findings thus provide evidence of the construct validity of the baseline reading test on the one hand, but are equally relevant to teaching practice on the other hand. The teaching of reading at lower secondary level in the Austrian context needs to focus more on cognitive processes of different complexity and the active teaching of metacognitive reading strategies.

The book introduces a framework for examining the validity of tests that aim to assess second-language (L2) proficiency development over time and/or in relation to L2 instruction. It also reports the findings of a longitudinal study that aimed to examine the sensitivity to change of a test of L2 proficiency development. Specifically, the study examined changes over time in Progress scores and the linguistic characteristics of essays written in response to Progress by learners who took the test before, during and after a period of L2 instruction in different countries. The book furthers our understanding of the nature of L2 proficiency as it develops over time and in relation to L2 instruction and provides a framework that can be used in future endeavours to design and validate tests of L2 proficiency development. The book is intended for graduate students, test developers, and researchers doing research in applied linguistics and L2 assessment.

This book explores the construct of language in use, specifically as operationalised through different item types in the Austrian Matura (school-leaving exam). Empirical research on some of these item types is scarce. The author reports on a mixed-methods study. The theoretical frameworks employed are Purpura’s (2004) model of language ability and Weir’s (2005) socio-cognitive framework. The findings suggest that the tasks under investigation assess grammatical form and meaning at the sub-sentential and sentential level. Different item types were also found to target different elements of lexicogrammatical competence. The study contributes to understanding the nature of language in use and sheds light on the application of the socio-cognitive framework to the validation of language in use tasks.

Human ratings are subject to various forms of error and bias. Since the early days of performance assessment, this problem has been sizeable and persistent. For example, expert raters evaluating the quality of an essay, an oral communication, or a work sample, often come up with different ratings for the very same performance. In cases like this, assessment outcomes largely depend upon which raters happen to provide the rating, posing a threat to the validity and fairness of the assessment. This book provides an introduction to a psychometric approach that establishes a coherent framework for drawing reliable, valid, and fair inferences from rater-mediated assessments, thus answering the problem of inevitably fallible human ratings: many-facet Rasch measurement (MFRM). Throughout the book, sample data taken from a writing performance assessment are used to illustrate key concepts, theoretical foundations, and analytic procedures, stimulating the readers to adopt the MFRM approach in their current or future professional context.

This book intends to provide a theoretical overview of examining candidates’ oral abilities in different examination formats. In particular, it explores specific partner effects on discourse outcomes and proficiency ratings in the framework of paired-task performance. Two investigations, both set in the context of a proposed Hungarian school leaving examination in English, were carried out in order to contribute to a better understanding of the assumed impact of the chosen variables. Study One investigates discourse differences between examiner-to-examinee interaction and peer-to-peer interaction. Study Two explores the impact of the peer partner’s proficiency. The results show that partner effects related to this variable cannot be predicted as either harmful or beneficial since no statistically significant difference was found between 30 candidates’ scores in different examination conditions.

This publication was awarded the Jacqueline Ross TOEFL Dissertation Award in 2009. Since its publication in 2001, the Common European Framework of Reference (CEFR) has been the most frequently-cited performance standard in language testing in Europe. To help test providers and users with score interpretation in relation to the CEFR levels, the Council of Europe published the ‘Manual for relating language examinations to the CEFR’. Even though the linking process set out in the Manual is primarily based on judgements by trained participants, judgement-making in this context remains largely unexplored. The research presented in this book addresses this issue by employing quantitative and qualitative methods. Despite the judges’ good understanding of how language ability progresses from lower to higher CEFR levels, it was found that describing test content and examinee performance was not without problems and decision-making was affected by a number of factors that were irrelevant to the judgement task. The results provide a better understanding of judgement-making during the CEFR linking process, which has important implications for examination providers and users of CEFR-aligned test scores.