Gunther Sigott – författare

Although second language learners’ pragmatic competence (their ability to use language in context) is an essential part of their general communicative competence, it has not been a part of second language tests. This book helps fill this gap by describing the development and validation of a web-based test of ESL pragmalinguistics. The instrument assesses learners’ knowledge of routine formulae, speech acts, and implicature in 36 multiple-choice and brief-response items. The test’s quantitative and qualitative validation with 300 learners showed high reliability and provided strong evidence of the instrument’s construct validity. Its web-based format makes it easy to administer and score. The book ends with a discussion of future research directions in assessing second language pragmatics.

This book was the winner of the 2005 Jacqueline A. Ross Dissertation Award for second or foreign language testing.The assessment of speaking skills is an important part of the language learning process, whether carried out by teachers in the classroom or through proficiency tests administered by testing agencies. However, the complex and subjective nature of speaking assessment raises a number of important questions about the validity, reliability and fairness of test procedures. This book, which is based on research into the speaking test of the International English Language Testing System (IELTS), shows how the performance of learners in oral proficiency interviews cannot be taken unequivocally as an indication of their ability, as variation in interviewer behaviour can have an impact both on learner performance and on ratings outcomes.

The book examines how individual differences in candidates’ levels of extraversion interact with facets which have been shown to affect scores obtained on oral performance tests. Three major studies are presented: the first is concerned with the individual candidate in the model; the second study investigates the role of the rater in the oral test process; the final study is concerned with the degree of interactivity required of pairs of candidates, each of whom also acts as an interlocutor in the performance of an oral test task. Results of the studies confirm that when an appropriate instrument is used to assess personality, and when theoretically sound hypotheses derived from the psychological literature are tested, significant differences can be observed in the responses of introverts and extraverts on particular oral test tasks.

This book marks ten years of language test development and language testing research in Austria. Part I describes the development of large-scale tests at all levels of the educational system. The documentation covers German as the language of instruction for the majority of pupils, the so-called modern foreign languages English, French, Italian and Spanish, as well as the classical languages Latin and Ancient Greek. Part II brings together a considerable body of research which has been conducted in the course and the wake of the development and implementation of the new assessment philosophy during the past ten years. The studies are accommodated in a framework of test validation, which could serve as orientation for future research in language testing in Austria.Das Buch blickt auf zehn Jahre professioneller Testentwicklung und Sprachtestforschung in Österreich zurück. Teil I beschreibt die Entwicklung von Testsystemen auf allen Ebenen des Bildungssystems. Die Dokumentation umfasst Deutsch, das für die große Mehrheit Unterrichtssprache ist, die sogenannten modernen Fremdsprachen Englisch, Französisch, Italienisch und Spanisch sowie die klassischen Sprachen Latein und Altgriechisch. Teil II umfasst eine beachtliche Anzahl von Untersuchungen, die während und nach der Entwicklung und Implementierung der neuen Prüfungsphilosophie in den letzten zehn Jahren durchgeführt wurden. Die Untersuchungen werden in ein Validierungskonzept eingeordnet, das als Orientierung für zukünftige Sprachtestforschung in Österreich dienen könnte.

In this book, the author investigates a central issue in language testing research: What are the key features that contribute to item difficulty in a reading test? Results of various statistical analyses of the multiple-choice reading items from the Austrian 2009 baseline reading test show a significant correlation between empirical item difficulty and both cognitive processes and metacognitive strategies. The findings thus provide evidence of the construct validity of the baseline reading test on the one hand, but are equally relevant to teaching practice on the other hand. The teaching of reading at lower secondary level in the Austrian context needs to focus more on cognitive processes of different complexity and the active teaching of metacognitive reading strategies.

The book introduces a framework for examining the validity of tests that aim to assess second-language (L2) proficiency development over time and/or in relation to L2 instruction. It also reports the findings of a longitudinal study that aimed to examine the sensitivity to change of a test of L2 proficiency development. Specifically, the study examined changes over time in Progress scores and the linguistic characteristics of essays written in response to Progress by learners who took the test before, during and after a period of L2 instruction in different countries. The book furthers our understanding of the nature of L2 proficiency as it develops over time and in relation to L2 instruction and provides a framework that can be used in future endeavours to design and validate tests of L2 proficiency development. The book is intended for graduate students, test developers, and researchers doing research in applied linguistics and L2 assessment.

This book explores the construct of language in use, specifically as operationalised through different item types in the Austrian Matura (school-leaving exam). Empirical research on some of these item types is scarce. The author reports on a mixed-methods study. The theoretical frameworks employed are Purpura’s (2004) model of language ability and Weir’s (2005) socio-cognitive framework. The findings suggest that the tasks under investigation assess grammatical form and meaning at the sub-sentential and sentential level. Different item types were also found to target different elements of lexicogrammatical competence. The study contributes to understanding the nature of language in use and sheds light on the application of the socio-cognitive framework to the validation of language in use tasks.

Human ratings are subject to various forms of error and bias. Since the early days of performance assessment, this problem has been sizeable and persistent. For example, expert raters evaluating the quality of an essay, an oral communication, or a work sample, often come up with different ratings for the very same performance. In cases like this, assessment outcomes largely depend upon which raters happen to provide the rating, posing a threat to the validity and fairness of the assessment. This book provides an introduction to a psychometric approach that establishes a coherent framework for drawing reliable, valid, and fair inferences from rater-mediated assessments, thus answering the problem of inevitably fallible human ratings: many-facet Rasch measurement (MFRM). Throughout the book, sample data taken from a writing performance assessment are used to illustrate key concepts, theoretical foundations, and analytic procedures, stimulating the readers to adopt the MFRM approach in their current or future professional context.

This volume, in honor of Allan James, collects a range of articles from different domains of English studies as a token of Allan James''s academic interests and his integrative approach to the field. The contributions in linguistics encompass a spectrum of topics including world Englishes, professional discourse, language acquisition, collocation, translation, and multilingualism. Cultural aspects in language teaching and in literary analysis enrich the reading and hint at Allan James'' Welsh and Celtic roots while also going beyond that.

This book intends to provide a theoretical overview of examining candidates’ oral abilities in different examination formats. In particular, it explores specific partner effects on discourse outcomes and proficiency ratings in the framework of paired-task performance. Two investigations, both set in the context of a proposed Hungarian school leaving examination in English, were carried out in order to contribute to a better understanding of the assumed impact of the chosen variables. Study One investigates discourse differences between examiner-to-examinee interaction and peer-to-peer interaction. Study Two explores the impact of the peer partner’s proficiency. The results show that partner effects related to this variable cannot be predicted as either harmful or beneficial since no statistically significant difference was found between 30 candidates’ scores in different examination conditions.

This publication was awarded the Jacqueline Ross TOEFL Dissertation Award in 2009. Since its publication in 2001, the Common European Framework of Reference (CEFR) has been the most frequently-cited performance standard in language testing in Europe. To help test providers and users with score interpretation in relation to the CEFR levels, the Council of Europe published the ‘Manual for relating language examinations to the CEFR’. Even though the linking process set out in the Manual is primarily based on judgements by trained participants, judgement-making in this context remains largely unexplored. The research presented in this book addresses this issue by employing quantitative and qualitative methods. Despite the judges’ good understanding of how language ability progresses from lower to higher CEFR levels, it was found that describing test content and examinee performance was not without problems and decision-making was affected by a number of factors that were irrelevant to the judgement task. The results provide a better understanding of judgement-making during the CEFR linking process, which has important implications for examination providers and users of CEFR-aligned test scores.