International Journal of Language Testing

The current paper intends to exploit the Many Facet Rasch Model to investigate and compare the impact of situations (items) and raters on test takers' performance on the Written Discourse Completion Test (WDCT) and Discourse Self-Assessment Tests (DSAT). In this study, the participants were 110 English as a Foreign Language (EFL) students at Vali-e-Asr University of Rafsanjan in Iran. The students were asked to complete the WDCT and rate themselves on that test. Four raters scored the WDCT tests. According to the FACETS results, there were significant differences in students' performance between the two methods. The stable fit statistics and differing levels of difficulty measures for each test method indicated that each test had a unique way of differentiating the test taker's pragmatic ability. Based on the results, both DSAT and WDCT are acceptable measures for pragmatic ability; however, there are some fitness problems in DSAT. This shows the unpredictable pattern of ratings in the DSAT. It is recommended to have rater training to obtain more accurate results from the DSAT. Finally, the implications were discussed.

The Elicited Imitation Test (EIT) is widely recognized for its reliability in research settings as a proficiency assessment tool. However, there exists a need to examine its predictive validity in English as a Foreign Language (EFL) classrooms. This study investigates the extent to which the EIT, alongside the Oxford Placement Test (OPT), can predict students' academic achievements in an English for Academic Purposes course, including overall grade point average and scores in listening, speaking, grammar, and vocabulary. The study also examines the relationship between students' perceptions of their listening and speaking skills and their EIT performance. The study involves 41 participants, with data analysis conducted using both regression and correlation methods. Results show that the EIT significantly predicts students' grade point average and language skills. Students' self-perceived speaking and listening abilities reasonably align with their actual performance on the EIT, and it seems that factors related to comprehension weigh heavily in their considerations. These findings have significant implications for EFL research and pedagogy.

The present study aimed to investigate the impact of language assessment literacy enhancement (LALE) on Iranian high school EFL students’ assessment as learning of writing skill. It also aimed to examine if LALE affects Iranian EFL students’ attitudes toward assessment as learning. To this end, 80 intermediate-level high-school EFL learners were selected and randomly categorized into the experimental and control groups. Both groups wrote an essay in the pre-test phase. Then, The ESL Composition Scale was used to teach the students what makes good writing and what criteria and standards they are supposed to learn to be able to write and rate the essays of their own and their peers. The control group received no instructional information on assessment rubrics. The experimental and control groups wrote another essay on a specific topic in the post-test phase. In the qualitative phase of the study, ten high school EFL students from the experimental group were interviewed regarding their attitudes toward the practice of assessment as learning of the writing skill in their English classes. The findings indicated that LALE significantly affected Iranian high school EFL students’ assessment as learning of writing skill. Moreover, students believed peer and self-assessment techniques are rarely implemented in Iranian high school EFL classes. They were also uncertain and felt uncomfortable judging, evaluating, criticizing, and rating their peers. Besides, they felt that they were not knowledgeable and capable enough to play the role of an assessor.

A critical component of cognitive diagnostic models (CDMs) is a Q-matrix that stipulates associations between items of a test and their required attributes. The present study aims to develop and empirically validate a Q-matrix for the listening comprehension section of the International English Language Testing System (IELTS). To this end, a listening comprehension test of the IELTS was administered to 820 Iranian test takers. According to theories, taxonomies, and models of second/foreign language (L2) listening comprehension, previous studies on the utility of CDMs to L2 listening comprehension, detailed content analysis of the test items, and consultation with several content experts, an initial Q-matrix was first developed. Through the technique suggested by de la Torre and Chiu (2016), along with checking heatmap plots and mesa plots using the GDINA package in R, the Q-matrix was then empirically validated. Generally, six attributes were extracted for the listening section, namely, (1) Linguistic knowledge (LKA), (2) understanding prosodic patterns (UPP), (3) ability to understand and make paraphrases (PAR), (4) ability to understand specific factual information such as names, numbers, and so forth (UFI), (5) ability to understand explicit information (UEI), and (6) ability to make inference (INF). Finally, the results of the fit of the GDINA model to the data, at both item and test levels, indicated the adequate model-data fit and the plausibility of the Q-matrix. The implications of the study were also discussed.

Assessing language skills in an integrative form has drawn the attention of assessment experts in recent years. While some research data exists on integrative listening/reading-to-write assessment, there is comparatively little research literature on listening-to-speak integrated assessment. Also, little attention has been devoted to the role of individual attributes within the context of integrated assessment. The objective of the current research was to investigate the relationship between integrated listening/speaking assessment and individual characteristics of ambiguity tolerance (AT), use of cognitive/metacognitive strategies, and foreign language anxiety (FLA). Oxford Quick Placement Test was used to homogenize 60 EFL learners in terms of language proficiency (B2-C1). Additionally, integrated listening/speaking performances were collected using sample TOEFL-iBT tests. The transcribed spoken samples were evaluated by two raters using TOEFL-iBT rubrics in terms of overall description, delivery, language use, and topic development. Additionally, information on individual characteristics was gathered by means of 3 different questionnaires. Data analysis revealed that FLA had a negative relationship while AT and the use of cognitive and metacognitive strategies had a positive correlation with integrated listening/speaking test performance. Individual differences have generally been neglected in the assessment literature, but this study revealed that performance on integrated listening/speaking tests can be affected by language-irrelevant constructs such as individual attributes in addition to test-takers’ language competence.

Although researchers have used practical procedures for developing professional teacher education, such as reflective practice and action research, the role of learning-oriented assessment (LOA) in building an active learner-centered pre-service teacher education has received scarce scholarly attention. Hence, the purpose of this study is to explore how collective and reflective LOA can enhance EFL pre-service teachers’ learner-centered teacher education. For this purpose, 15 Iranian EFL pre-service teachers were selected through a convenience sampling procedure from a teacher-training college in Iran. The study was built on a qualitative intrinsic case study, the data were collected from the participants through observations, self- and peer-assessment practices, portfolios, and semi-structured interviews, and the collected data were inductively analyzed. The results of the inductive thematic analysis indicated that pre-service teachers became active, cooperative, and flexible. Firstly, pre-service teachers concentrated on reflective self-assessment, collaborative peer-assessment, and portfolios. Secondly, pre-service teachers were more self-directed and autonomous by taking more actions, planning their development, making decisions, and learning. Thirdly, pre-service teachers accelerated their positive emotions, such as growth mindset, flexibility, curiosity, and criticality. The findings promise implications for implementing learning–oriented assessment to promote teachers’ agency and autonomy in their teaching careers.

This article delved into the realm of EFL assessment in Iraq by investigating the beliefs of Iraqi EFL teachers about assessment practices they employed in their classroom and determining whether there was any congruency between their beliefs and actual assessment practices. For this purpose, 140 experienced Iraqi EFL teachers were selected by convenience sampling and the data were collected by the Teachers’ Assessment Practices Belief Questionnaire and the Teachers’ Assessment Practices Questionnaire. The findings of Pearson correlation and descriptive analysis revealed that the cognitive level of assessments (e.g., reasoning and application), types of assessments (e.g., portfolios and concept mapping), and evaluation criteria (e.g., improvement and student effort) were highly valued by the teachers. Regarding assessment practices in the classroom, the participants reported obtaining, elucidating, and responding to learning evidence and assisting students to acquire a positive orientation of learning (making learning explicit). Promoting learning autonomy (an expanded opportunity to assume increased autonomy in defining their learning goals and evaluating both their work and that of their peers) was also found in the teachers’ reports. Besides, an interest in assisting students adhere to performance goals stipulated by the curriculum using careful questioning and assessed by scores and grades (performance orientation). Implications and suggestions of the study are discussed in the article.

Progress in International Reading Literacy Study (PIRLS) is an international comparative study of reading comprehension that is administered by the International Association for the Evaluation of Educational Achievement (IEA) to measure the reading competency of fourth grade students in the participating countries. The test is a very comprehensive measure of reading literacy which covers a wide variety of reading subskills and texts. The purpose of the current study is to evaluate the psychometric qualities of PIRLS 2021 using the national data of Kazakhstan. The 2-parameter item response theory was used to evaluate the test. Item difficulty, discrimination, infit, outfit, and the root mean square deviance (RMSD) fit statistics were examined. The multiple-group IRT was used to examine differential item functioning. Findings sowed that PIRLS 2021 is a robust measure of reading comprehension and does not exhibit differential item functioning (DIF). However, based on the 2PL discrimination and infit and outfit values, several items are problematic. The implications of this finding are discussed.

The essential requirement of validity in second language assessment and testing is of paramount importance since it guarantees that tests properly gauge the targeted constructs and produce propitious interpretations. In this light, there seems to be a pressing need to carefully assess outdated conceptions of validity in English language testing to ensure they are in lockstep with current linguistic and sociocultural circumstances. This review examines the book "Ethics and Context in Second Language Testing: Rethinking Validity in Theory and Practice," edited by M. Rafael Salaberry, Albert Weideman, and Wei-Li Hsu (2023), and it unravels new horizons for expanding the comprehension of validity beyond traditional frameworks. The review underscores the primary subjects addressed in the book's content, such as ethical issues, contextual factors affecting test design, and novel approaches for assessing validity in real-world settings. The review also stresses the book's critical analysis of existing paradigms and recommends a more nuanced perspective on validity that incorporates ethical considerations and contextual significance. To this end, the current review constitutes an important reference for researchers and practitioners to encourage them to critically examine and reframe validity in second language testing.

The Pearson Test of English Academic (PTE-A), a widely used high-stakes language proficiency test for university admissions and migration purposes, underwent a notable change from a three-hour to a two-hour version in November 2021. The implementation of the new version has prompted inquiries into the washback effects on various stakeholders. Focusing on a small sample of Chinese test takers (n=10), this paper explores washback effects following the revision of PTE-A and the complexity of test preparation through in-depth semi-structured interviews. The findings suggest a shorter test length is preferred and several different methods are adopted for test preparation which gives evidence to the positive washback. However, participants reported some confusion regarding certain test items, leading to the adoption of construct-irrelevant methods. This, in turn, may affect the face validity of PTE-A. While addressing the literature gap in this field, recommendations for improving the test design to better meet test takers’ needs are provided.

The obligatory prevalence of online education during the COVID-19 pandemic has drawn researchers’ attention to the challenges involved in foreign language pedagogy in such virtual educational contexts. Against this backdrop, this study investigated the impact of online self-regulated learning, use of communication strategies, and test anxiety on Iranian English as a Foreign Language (EFL) learners’ online speaking test performance. For this purpose, 132 EFL learners were given the e-Oxford Quick Placement Test and the speaking part of a sample A2 Key and B1 Preliminary test. Next, translated versions of the given measures were administered to the pre- and intermediate EFL learners and the obtained data were subjected to Structural Equation Modeling analyses that verified strong links between online self-regulated learning and the use of communication strategies, test anxiety and online self-regulated learning, and test anxiety and EFL learners’ use of communication strategies. Furthermore, the direct impacts of online self-regulated learning and use of communication strategies on learners’ online speaking test performance were verified; however, test anxiety was found to indirectly impact the learners’ online speaking test performance through its negative effect on EFL learners’ online self-regulated learning and use of communication strategies. In addition, online self-regulated learning turned out to be the strongest predictor of the learners’ online speaking test performance. As for the implications of the findings, it appeared that the attested model lends support to Bachman and Palmer’s (1996) language use framework illustrating test performance as a vulnerable construct affected by test takers' attributes and features of the test tasks and the impact of construct-irrelevant factors like test takers’ personal characteristics on their test performance.

Writing assessments in the EFL context today have put a great concern on language acquisition and proficiency testing. However, diverse techniques and practices require a comprehensive synthesis to better understand trends, gaps, and best practices. This study reviews and synthesizes prior research on writing assessments in EFL contexts. It examines the dominant assessment approaches, their effectiveness, and their impact on EFL learners' writing skills holistically. The study employs a meta-synthesis approach. Fifty journal articles indexed by Scopus, ERIC, Crossref, Google Scholar, DOAJ, Copernicus, and other indexing databases between 2018 and 2024 were selected as the data to review in this study. The data selection was conducted to ensure the relevance and quality of the literature reviewed. The results include a variety of assessment practices such as teacher feedback, self-assessment, peer-assessment, blended or collaborative assessment, automated writing evaluation (AWE), and artificial intelligence in EFL writing settings. The study also concludes that, despite the diversity and complexity of writing assessments in EFL contexts, there is a strong need for more standardized and contextually valid assessment practices. Effective assessments align with instructional goals and offer meaningful feedback that supports student learning. The study recommends developing comprehensive assessment frameworks tailored to specific EFL contexts, providing more training for educators in assessment literacy, and conducting further research into the long-term impact of various assessment practices on the writing development of EFL learners.