|
Audience/Level: ESL/EFL teachers and teacher trainers of all levels
Assessment is often a fearsome topic for students and teachers alike. In most English as a second language and English as a foreign language contexts, the assessment of students’ second language skills is usually designated to a unit or testing center, and teachers do not have to worry about creating major tests in their daily activities. However, this does not mean that teachers never have to assess students’ language skills and never have to prepare their own assessment materials—they do, and they must adjust their instruction accordingly.
A common practice is to find online quizzes for in-class diagnostic purposes or as instructional activities. With the advent of technology, there is no shortage of assessment materials, and one can easily find many free online quizzes at various lengths in a given topic, but the real challenge is to choose valid and reliable ones in the abundance of such materials.
In this Quick Tip, I present an easy-to-remember framework so that teachers can evaluate the materials they find online by quickly looking for specific qualities. I have named this framework "WARP V," inspired by the well-known TV series Star-Trek. The activity name stands for the five major qualities of second language assessment: washback, authenticity, reliability, practicality, and validity (Brown & Abeywickrama, 2010). WARP V is an escalated speed in which a ship improves its performance to its peak, but not every ship can handle the WARP V speed—the weaker ships fall apart. Similarly, when teachers design or find a test, they can put it across the WARP V analysis and see how the test is doing across these benchmarks. If the test survives the WARP V analysis, it’s an effective test.
|
Benchmark |
Definition |
Criterion |
|
Washback |
Washback is the learning from the assessment, either while preparing for it or remembering certain assessment experiences (e.g., going over the answers after the test). Positive washback refers to expected test effects. Negative washback refers to the unexpected or harmful consequences of a test. |
Are students learning from this test by taking it or preparing for it?
Does the test have any potential detrimental effects? |
|
Authenticity |
Authenticity may be achieved by making the test more natural and meaningful. |
Is the language in the test as natural as possible?
Are items contextualized rather than isolated?
Are the topics meaningful (relevant, interesting) for learners?
Do the tasks approximate real-world tasks? |
|
Reliability |
A reliable test is consistent and dependable. If you give the same test to the same student or matched students on two different occasions, the test would yield similar results. |
Would the students score the same if they took the test again?
Are there any outside factors such as an illness, street noise, technology troubles, or other physical conditions that might impede students’ performance? |
|
Practicality |
An effective test is practical. |
Is the test affordable?
Does it stay within appropriate time constraints?
Is it easy to administer?
Does it have a scoring/evaluation procedure that is specific and efficient? |
|
Validity |
Test validity refers to the extent to which inferences made from assessment results are appropriate, meaningful, and useful in terms of the purpose of the assessment. |
Is the test measuring what it is intended for or something else?
Is the test measuring the intended language skill or construct? |
The WARP V framework provides teachers a method to quickly evaluate assessment instruments before adoption and gives them a common language to talk about and reflect on their observations on a given assessment medium. For an example of the WARP V framework in use, see Appendix A (.pdf), and for a blank WARP V template, see Appendix B (.docx).
Reference
Brown, H. D., & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices. White Plains, NY: Pearson Education.
Baburhan Uzum is an assistant professor in the Department of Language, Literacy, and Special Populations at Sam Houston State University, Huntsville, Texas, USA. His research interests include multicultural education and second and foreign language teacher education.
|
