The Effect of Task Characteristics on Mastery/Non-mastery Decisions

Wojciech Malec

Wojciech Malec Wydział Nauk Humanistycznych, Katolicki Uniwersytet Lubelski Jana Pawła II w Lublinie

Słowa kluczowe: pomiar sprawdzający; efekt metody; aspekty metody testowania; forma zadania; decyzje klasyfikacyjne; testowanie kolokacji

Abstrakt

[Abstrakt tylko w j. polskim / Abstract only in Polish]

Wpływ właściwości zadania testowego na decyzje klasyfikacyjne

Artykuł jest opisem badania, którego celem było udzielenie odpowiedzi na pytanie, czy właściwości zadania testowego mają wpływ na decyzje klasyfikacyjne w kontekście pomiaru sprawdzającego. Zastosowane narzędzia pomiaru, które sprawdzały znajomość kolokacji (łączliwości wyrazów) języka angielskiego, składały się zarówno z zadań zamkniętych, jak i otwartych różnego typu. Wszystkie testy kolokacji charakteryzowały się wysokim poziomem rzetelności decyzji klasyfikacyjnych oraz trafnością teoretyczną. Analiza wariancji z powtarzanym pomiarem wykazała istotność efektu metody testowania, polegającego na tym, że osiągnięcie przez studenta progu zaliczeniowego zależy od zastosowanej formy zadania testowego.

Bibliografia

Alderson, J. C. and A. H. Urquhart. (1985). The effect of students’ academic discipline on their performance on ESP reading tests. Language Testing 2, 192–204.

Arnaud, P. (1989). Vocabulary and grammar: a multitrait-multimethod investigation. AILA Review 6, 56–65.

Arnaud, P. and S. J. Savignon. (1997). Rare words, complex lexical units and the advanced learner. In: J. Coady and T. Huckin (eds.), Second Language Vocabulary Acquisition: A Rationale for Pedagogy (pp. 157-173). Cambridge: Cambridge University Press.

Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press.

Bachman, L. F. (2004). Statistical Analyses for Language Assessment. Cambridge: Cambridge University Press.

Bachman, L. F. and A. S. Palmer (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly 16, 449–465.

Bachman, L. F. and A. S. Palmer (1996). Language Testing in Practice: Designing and Developing Useful Language Tests. Oxford: Oxford University Press.

Bonk, W. (2001). Testing ESL learners’ knowledge of collocations. In: T. Hudson and J. D. Brown (eds.), A Focus on Language Test Development: Expanding the Language Proficiency Construct across a Variety of Tests (pp. 113-142). Honolulu: University of Hawai’i Second Language Teaching and Curriculum Center.

Brown, J. D. (1990). Short-cut estimates of criterion-referenced test consistency. Language Testing 7, 77-97.

Brown, J. D. and T. Hudson (2002). Criterion-referenced Language Testing. Cambridge: Cambridge University Press.

Corrigan, A. and J. A. Upshur (1982). Test method and linguistic factors in foreign language tests. Iral 20, 313–321.

Field, A. (2000). Postgraduate statistics: cluster analysis. [Online at: http://www.sussex.ac.uk/users/ andyf/cluster.pdf]

Field, A. (2005). Discovering Statistics Using SPSS (Second edition). London: Sage Publications.

Francuz, P. and R. Mackiewicz R. (2005). Liczby nie wiedzą, skąd pochodzą: przewodnik po metodologii i statystyce nie tylko dla psychologów. Lublin: Wydawnictwo KUL.

Freedle, R. and I. Kostin (1993). The Prediction of TOEFL Reading Comprehension Item Difficulty for Expository Prose Passages for Three Item Types: Main Idea, Inference, and Supporting Idea Items. Research Report No. 93-13. Princeton, NJ: Educational Testing Service.

Fulcher, G. (1996). Testing tasks: issues in task design and the group oral. Language Testing 13, 23-51.

Haladyna, T. M., S. M. Downing, and M. C. Rodriguez (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education 15, 309-334.

Hamp-Lyons, L. and S. Prochnow (1991). Prompt difficulty, task type, and performance. In: S. Anivan (ed.), Current Developments in Language Testing (pp. 58-76). Singapore: SEAMEO Regional Language Centre.

Hoey, M. (1991). Patterns of Lexis in Text. Oxford: Oxford University Press.

Hurlburt, D. (1954). The relative value of recall and recognition techniques for measuring precise knowledge of word meanings: nouns, verbs, adjectives. Journal of Educational Research 47, 561-576.

Ito, A. (2004). Two types of translation tests: their reliability and validity. System 32, 395-405.

Kennedy, P. and W. B. Walstad (1997). Combining multiple-choice and constructed-response test scores: an economist’s view. Applied Measurement in Education 10, 359-375.

Kobayashi, M. (2002). Method effects on reading comprehension test performance: text organization and response format. Language Testing 19, 193-220.

Kobayashi, M. (2004). Investigation of test method effects: text organization and response format: a response to Chen, 2004. Language Testing 21, 235-244.

Kobayashi, W. (2005). An investigation of method effects on reading comprehension test performance. Paper presented at the 4th Annual JALT Pan-SIG Conference, Tokyo, Japan, 14-15 May 2005.

Malec, W. (2006). The Impact of Item Format on Test Performance in Criterion-referenced Assessment of Collocations. Unpublished PhD thesis, KUL, Lublin.

Malec, W. (2007). Efekt metody w pomiarze sprawdzającym na przykładzie testowania kolokacji języka angielskiego. In: B. Niemierko and M. K. Szmigel (eds.), Uczenie się i egzamin w oczach uczniów (pp. 305-315). Kraków: Grupa Tomami.

Mondria, J. A. and B. Wiersma (2004). Receptive, productive, and receptive + productive L2 vocabulary learning: what difference does it make? In: P. Bogaards and B. Laufer (eds.). Vocabulary in a Second Language: Selection, Acquisition and Testing (pp. 79-100). Amsterdam: John Benjamins Publishing Company.

O’Leary, M. (2001). Item format as a factor affecting the relative standing of countries in the Third International Mathematics and Science Study (TIMSS). Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, WA.

Paribakht, T. S. and M. Wesche (1997). Vocabulary enhancement activities and reading for meaning in second language vocabulary acquisition. In: J. Coady and T. Huckin (eds.), Second Language Vocabulary Acquisition: A Rationale for Pedagogy (pp. 174-200). Cambridge: Cambridge University Press.

Read, J. (2000). Assessing Vocabulary. Cambridge: Cambridge University Press.

Read, J. and C. A. Chapelle (2001). A framework for second language vocabulary assessment. Language Testing 18, 1-32.

Schmitt, N. (1999). The relationship between TOEFL vocabulary items and meaning, association, collocation and word-class knowledge. Language Testing 16, 189-216.

Shohamy, E. (1984). Does the testing method make a difference? The case of reading comprehension. Language Testing 1, 147-170.

Shohamy, E. and O. Inbar (1991). Construct validation of listening comprehension tests: the effect of text and question type. Language Testing 8, 23-40.

Spaan, M. (1993). The effect of the prompt in essay examinations. In: D. Douglas and C. Chapelle (eds.). A New Decade of Language Testing Research (pp. 98-122). Alexandria, VA: TESOL Publications.

Trujillo, J. L. (2005). The Effect of Format and Language on the Observed Scores of Secondary-English Speakers. Unpublished PhD thesis, Florida State University, Tallahassee.

Ying-hui, H. (2006). An investigation into the task features affecting ELF listening comprehension test performance. The Asian EFL Journal Quarterly 8(2), 33-54.