Summative assessment, also called assessment of learning, usually generates a lot of anxiety on ELLs because it measures what students have learned after a usually long period and does not allow for reteaching and reassessing the accumulated content. Conversely, formative assessment, or assessment for learning (Black & Wiliam, 1998), is aimed at measuring what students are learning bit by bit and allows for revisiting the content and reassessing the learners.

At a large Binational Center in Brasilia, Brazil, with around 3,000 adult students, we aimed to shift a whole assessment system of adult learners from a summative model, with traditional tests, to a formative, outcomes-based model, focused primarily on skills-based and performance-based assessments.

Shifting to the New Assessment System

The Old Way (Summative)

In the past, students were assessed by way of a final written and an oral exam, each worth 100 points, at the end of a 10-week module. This generated learner anxiety because of the accumulation of content and the fear of doing poorly and failing the course. In addition, the tests were purely summative. The accumulation of content covered on the test also jeopardized the alignment of the learning outcomes, instructional strategies, and the assessment, because practicality issues led to the combination of many outcomes in one test section and to the inclusion of many selected-response items. Besides, due to the weight of the written test, teachers spent a large amount of class time on worksheets to prepare students for the test, rather than on oral communication, the primary goal of the course.

The New Way (Formative)

In order to overcome these drawbacks, a new model was implemented. The students take six short assessments, perfectly aligned with the learning outcomes and instructional strategies, without a long interval between the teaching and the assessment, and are allowed to retake any of the assessments at the end of the module if the outcomes are not met. More important, the new assessment instruments focus on the four skills rather than on grammar and vocabulary as the former instruments did.

A Sample Assessment of Speaking

Below is an example of a speaking assessment that demonstrates the formative aspect of the new assessment system.

Outcome: Students will be able to talk about past experiences related to music and expand the conversation by providing details on the experience.

Assessment task: Ask your partner some questions about a musical experience. Begin with a “have you ever” question and then ask some follow-up questions.

Students practice as much as they need until they feel ready. The teacher has pairs record their dialogue using their smartphones or iPads brought to the classroom. They then listen to the recording and decide whether they want to redo it or not, based on the assessment checklist. Different pairs can give feedback on each other’s recording and redo the assignment.

The teacher assesses students by way of “real-time, almost surreptitious recording of student verbal and nonverbal behavior” (Brown, 2004, p. 267), using a short, customized checklist that focuses on what students were able to do regarding content, language use, vocabulary, pronunciation, and fluency.

Student and Teacher Feedback

After the piloting of the project, students were asked to compare the two assessment systems. The total number of students involved in the piloting stage was 170. An overwhelming majority of the students in the pilot project, 94%, reported preferring the new model. The main reasons were that they felt less anxious, they revised the content more frequently, there was a greater focus on oral communication in class, and the assessment was more accurate because it was ongoing.

Eighteen teachers piloted the new assessment system. They also demonstrated preference for the new model because they felt they were able to provide timely, personalized feedback and assess students at their best, rather than under exam-generated anxiety. In fact, only 6% of the teachers who piloted the project responded that they supported it only partially.

Mainstreaming the New System: Dealing With Mental Models

While the piloting of the new assessment system with a small group of students and teachers proved highly successful, mainstreaming the project presented some challenges. The reason for this lies in teachers’ and students’ mental models regarding assessment. Mental models are our ideas about how the world works, and they influence our actions. Mental models that limit people’s ability to change should be examined (Senge et al., 2012). An analysis of the qualitative feedback provided by students and teachers in multiple surveys about the new assessment system led to the identification of four main mental models regarding assessment which clashed with the rationale behind the new assessment system:

1. There are too many assessments and too much time is spent on assessments.

In the past, students spent the last two 75-minute class days taking a written and an oral test. In the current system, they spend from 20–30 minutes doing five short assessments, usually focused on a single skill. The sixth assessment, writing, is done at home, not in class. Hence, the time spent on assessments has not increased, but the teachers’ and students’ perception is that it has. This shows a fixed mindset (Dweck, 2006) regarding how students should be assessed—only at the end of a course, to sum up what has been learned.

2. Assessments are too easy. Students get higher grades than they used to, so they won’t be ready for the next level.

This belief reflects the world view that assessments are meant to be hard and challenging, and that obtaining good results is not for everyone. If the assessment is easy, it is not measuring knowledge effectively. The idea of mastery learning, that is, redoing the task until you master it, is totally alien to students and teachers with this mental model.

3. Sometimes a whole class is spent on the oral assessment.

This occurs because, as the example above shows, students have time to practice, analyze their performance, and redo it if they are not happy. There is a lot of learning going on in this process. In the past, students spent a whole class taking a written exam and this was never a source of criticism. Doing the same with oral practice, the ultimate goal of adult students in the program, seems odd to students and teachers with this mental model.

4. Grammar is not being assessed.

Students are used to being assessed by way of written tests with fill-in-the-blank or multiple-choice items. Conversely, in performance assessment, their use of grammar is assessed in discourse. As the speaking activity exemplified above shows, grammar is explicitly assessed, but by way of performance assessment. However, students do not perceive this because the way of assessing grammar is not in accordance with their mental model.

Managing Change

Managing a paradigm shift in assessment in a large, traditional institution takes time and effort. The key elements to promote change and examples on how they can be dealt with are:

a) Develop a growth mindset in teachers and students.

Take advantage of opportunities to talk about how all students can achieve positive results if they are given the right conditions and resources and if they make the necessary effort. Rather than only praise students’ positive results, focus on the process that led to the result. Also, help teachers and students understand that there aren’t “weak” and “strong” students, but, rather, students who are using effective and ineffective strategies to learn.

b) Challenge mental models.

Collect and share data that challenge old mental models about assessment and student achievement. Encourage teachers to have explicit conversations with students about the benefits of ongoing assessment and about the fact that effective assessment doesn’t have to be difficult and tricky or aim at ranking students.

c) Build a shared vision.

Promote ongoing meetings with teachers and surveys with students to obtain feedback about assessment. Guarantee student and teacher buy-in by valuing their input on assessment practices and implementing their suggestions, thus making them feel that they have a voice. Choose a number of teachers to become “assessment specialists” or “assessment gurus,” who colleagues can reach out to when faced with challenges and doubts regarding assessment.

d) Create a learning community.

Promote courses and workshops and use the school intranet and/or closed teacher groups on social media to share literature about assessment so teachers can advance in their assessment literacy and, thus, make informed judgments and decisions. Encourage teachers to contribute with new assessment instruments and to give talks and workshops in conferences on the topic of assessment.

Managing change can be a daunting task, so rather than attempting to change a whole system overnight, we must focus on changing one student and one teacher at a time.

References

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. doi: 10.1080/0969595980050102

Brown, H. D. (2004). Language assessment – principles and classroom practice. White Plains, NY: Longman.

Dweck, C. (2006). Mindset – How you can fulfill your potential. New York, NY: Random House.

Senge, P. M., Cambron-McCabe, N., Lucas, T., Smith, B., Dutton, J., & Kleiner, A. (2012). Schools that learn – A fifth-discipline fieldwork for educators, parents, and everyone who cares about education. New York, NY: Crown Business.

Isabela Villas Boas is the academic superintendent of Casa Thomas Jefferson, Brasilia, Brazil. She holds an MTESL Degree from Arizona State University and a doctorate degree from Universidade de Brasilia. She is a frequent presenter at TESOL conferences and recently published a book with TESOL Press entitled Perspectives in Teaching English in a Binational Center in Brazil.