Multiple-choice items

Multiple-choice (MC) item format is commonly used in educational assessments due to its economy and effectiveness across a variety of content domains. However, numerous studies have examined the quality of MC items in high-stakes and higher education assessments and found many flawed items, especially in terms of distractors. These faulty items lead to misleading insights about the performance of students and the final decisions. The analysis of distractors is typically conducted in educational assessments with multiple-choice items to ensure high quality items are used as the basis of inference. Item response theory (IRT) and Rasch models have received little attention for analyzing distractors. For that reason, the purpose of the present study was to apply the Rasch model, to a grammar test to analyze items’ distractors of the test. To achieve this, the current study investigated the quality of 10 instructor-written MC grammar items used in an undergraduate final exam, using the items responses of 310 English as a foreign language (EFL) students who had taken part in an advanced grammar course. The results showed the acceptable fit to the Rasch model and high reliability. Malfunctioning distractors were identified.

The assessment of listening abilities is one of the least understood, least developed and, yet, one of the most important areas of language testing and assessment. It is particularly important because of its potential wash-back effects on classroom practices. Given the fact that listening tests play a great role in assessing the language proficiency of students, they are expected to enjoy a high level of construct validity. The present study was dedicated to investigating the construct validity of three different test formats, namely, multiple-choice, gap filling on summary (also called listening summary cloze), and fill-in-the-blank, used to evaluate the listening comprehension of EFL learners. In order to achieve the purpose of the study, three passages with relatively similar readability levels were used for the construction of 9 listening tests, that is, each appeared in three formats. Following a counter-balanced design, the tests were administered to 91homogeneous EFL learners divided into three groups. The statistical analysis of the results revealed that the multiple-choice test enjoyed the highest level of construct validity. Moreover, a repeated measure one-way ANOVA demonstrated that the fill-in-the-blank task was the most difficult with the MC test as the easiest for the participants.