From the Principal's Office: Call for Skepticism and Caution When Using Test Scores in Teacher Evaluations

6/19/2013 12:00:00 AM
“We need to be careful that the tests we use in a properly designed teacher-appraisal system do, in fact, contribute to a valid (that is accurate) inference about a teacher’s quality.” W. James Popham, Evaluating America’s Teachers: Mission Possible?
North Carolina took the plunge this year and started using test scores as part of teacher and principal evaluations. The state has even invented a "new" kind of test, called a "Measure of Student Learning" in order to make sure there is plenty of test data to go around. What is particularly telling is how "carefully" the state crafted the term "Measures of Student Learning." It's as if somehow, not calling it a test, makes it not a test. State level educational logic never ceases to amaze me. Of course, the state then started calling these "Measures of Student Learning" something else. They started calling them "Common Exams." Notice again, the careful use of the word "exam" rather than "test." It's almost as if you don't call it a test, it isn't a test, but apparently state level policymakers haven't heard the old saw about a rose still being a rose even if it has another name.

Besides North Carolina's struggle with what to call their newly implemented tests, there's still the question of what the unintended consequences of having thousands of teachers "teaching to the test" is going to do for students in our state. Ultimately, being able brag that your students "Have the best scores in the world" is most likely what politicians and state level education officials are after. That's why they see salvation through test scores as the means to the "Educational Promised Land." Ultimately, there's a flawed logic driving this whole accountability and testing movement: it's the whole idea that learning can be entirely reduced to bubble sheet answer sheets and taken in a single sitting. And, that teachers can't be trusted to tell when a student has demonstrated that they have learned or not.

In my years as an educator, I have been amazed how trusting and accepting educators in North Carolina are when it comes to the latest policy flowing down from on high. It's as if they accept that those at the state level know more than they do, or somehow have access to magical information they do not have. So, when they implement something like the use of test scores in evaluations, many educators accept that the powers that be at the state level know what they are doing, so they trust them. Given the history of reform ideas and educational policy that travels down from on high, this "trust" is highly misplaced. I like to think that state level education officials mean well, but what often has happened during my career, these ideas when implemented locally have sometimes been a disaster and have been sometimes downright bad for kids. Instead of being so trusting, I submit that all educators in the schools and districts need to become skeptics and ask tough questions of our state-level, and federal level policymakers. We should never accept the "trust me, this will work" answer.

It is in this spirit of skepticism, I turn to Popham's book, Evaluating America's Teachers: Mission Possible? and our state's venture into making high stakes testing even more high stakes. In spite of what our state-level policymakers say, I am not fully satisfied that North Carolina's tests are adequate measures of educator effectiveness, and a healthy skepticism is still in order. This whole push to add test scores to teacher and principal evaluations has been a rush from the start. Depending on when you asked questions, how the tests were to be implemented has changed multiple times throughout the last two years. Never mind the fact that not a single teacher in North Carolina even saw the test before they were implemented. In their rush to have "test data" it's as if our state level policymakers think "any old data will do." They have failed to take the time to establish whether any of these tests really tell us anything about teaching quality.

In light of our state's push into "higher stakes testing," I think Popham reminds us of some important key issues and ideas about tests and teacher evaluations that state politicians and policymakers seem to forget.
  • “Tests are not valid or invalid. Instead, it is a test-based inference whose validity is at issue.” In other words, it isn't the test that’s valid or invalid, it is the inferences drawn from those tests that have these qualities. It boils down to whether you can actually make an inference based on the test or not. The question is whether North Carolina's tests, which have been implemented haphazardly and a thrown-together-manner, actually tell us anything at all about the quality of teaching in our classrooms. Can I honestly say Teacher A is a good teacher because she added "this much" value to her students Measures of Student Learning? Seems to me that it puts a great deal of faith in a single test.
  • “Tests allow us to make inferences about a test taker. This inference, depending on the appropriateness of the test as a support for the inference being made, may be valid or invalid.” As Popham points out, the inference we make about the learner may be valid or invalid depending on the “appropriateness of the test” in its role to support the inference being made. As we know, the word validity is the extent to which that inference, or conclusion, is well-founded or corresponds to the real world. This boils down to whether the inference we draw about a student is valid or not. For example, should we infer, based on a student’s test scores that he is not proficient in the subject, we must be satisfied that the test we are using is the “appropriate measure,” and we must also make sure the conclusion we draw considers all real world facts. Ignoring a student’s socio-economic status, or even whether he experienced a death in the family, can make our inference about he student’s proficiency invalid. Then there's the whole issue about making an inference about a teacher or principal's effectiveness using this same test. Has North Carolina sufficiently established the appropriateness of their Measures of Student Learning, End of Grade Tests, End of Course Tests, as instruments that allow for making inferences about teacher and principal quality? I'm not sure they have.
As North Carolina moves forward with a teacher and principal appraisal instrument that uses test scores to determine effectiveness, all educators need to educate themselves and scrupulously ask questions of policymakers.

As Popham suggests, “If heavy importance is being given to students’ performances on state tests for which there is no evidence supporting such an evaluative usage, then teachers (I would add principals too) might wish to engage in further study of this issue so that, armed with pertinent arguments, they can attempt to persuade educational decision makers that more appropriate evidence should be sought.” In other words, all educators, administrators, and teachers need to study how North Carolina or any state is using test scores to determine educator effectiveness.

Administrators owe it to their teachers, and themselves, to understand that some of these tests were never designed to determine educator effectiveness, so that data needs to be viewed with skepticism. Test scores in North Carolina currently are only 1/6th of the teacher evaluation, and effective administrators are going to keep this in mind and not let the allure of numbers numb them to the other 5 standards.

cross posted at the21stcenturyprincipal.blogspot.com

J. Robinson has decades of experience as a K12 Principal, Teacher, and Technology Advocate. Read more at The 21st Century Principal.

comments powered by Disqus
Tweets
Photo GalleriesView All Galleries >
Acer C720-2844 Chromebook

(www.acer.com) The Acer C720-2844 Chromebook model delivers speedy performance, a quick boot time of seven seconds, and a matte anti-glare display tha...

Britannica ImageQuest

(www.britannica.com) Britannica Digital Learning has upgraded ImageQuest, a resource for schools and libraries that provides nearly three million rig...

ClassFlow

(www.classflow.com) Promethean has released ClassFlow, a free, all-in-one, cloud-based teaching tool that lets teachers create and deliver interactive...

Adobe Voice

(www.adobe.com) Adobe has released Adobe Voice, an animated video app for the iPad that lets users create and share video stories. The app incorporate...

DeskBoard

(www.boxlight.com) The BOXLIGHT DeskBoard mobile cart adjusts both height and tilt for the P8 ultra short throw interactive projector on a white surfa...

Core 36M

(www.bretford.com) Bretford has introduced Core 36M, a 36-unit charging cart that is optimized for Chromebooks but which also supports most tablets, l...

Edmentum Sensei

(www.edmentum.com) Edmentum Sensei is a mobile optimized solution that helps administrators visualize and track overall school, teacher, and student p...

HMH Player

(www.hmhco.com) HMH has released HMH Player, a new native app for iOS and Google Chrome that streamlines the learning experience for improved digital ...

Juice Power System

(www.bretford.com) Bretford has unveiled an easy-to-use modular power system with exchangeable power components. The Juice Power System uses unique &#...

LightSail

(www.lightsailed.com) LightSail Education has announced a literacy accelerator that combines books with in-text embedded assessments and real-time dat...

myON

(www.myon.com) myON has expanded its digital library to include a set of literacy tools to further personalize the reading experience for students whi...

Nervanix Clarity

(www.nervanix.com) Nervanix has released Clarity, an application that monitors student attention levels as they study and guides them to revisit conce...

MathBall

(www.robotslab.com) RobotsLAB has introduced MathBall, a smart sensor basketball and tablet system that offers curricula in algebra, pre-calculus, phy...

MobileAsset.EDU

(www.waspbarcode.com) Wasp Barcode’s MobileAsset.EDU solutions include everything administrators need to account for their assets, from software...

OpenEd Assessment Creation Tool

(www.opened.io) OpenEd has announced a free tool that lets teachers easily create assessments with the question types required by Common Core standard...

Panasonic TH-80LFB70U

(www.panasonic.com) Panasonic’s TH-80LFB70U interactive LED display features high-speed, multi-touch, interactive capabilities to promote partic...

penveu interactive display system

(www.penveu.com) The penveu interactive display system is a handheld device that turns projectors and large screen displays into interactive whiteboar...

PresentationPro

(www.califone.com) Califone has updated its PresentationPro speaker. The PA310 readily connects with computers, LCD projectors, mobile devices, intera...

PowerSync+ Mobile Companion App

(www.bretford.com) Bretford Manufacturing, Inc. has announced the availability of the companion app for its PowerSync+ enabled charge and sync produc...

PureCharge Carts and Stations for iPad

(www.bretford.com) Bretford Manufacturing, Inc. has debuted the PureCharge family of iPad and iPad mini charging carts and stations. By offering pre-i...

ProQuest Research Companion

(www.proquest.com) ProQuest’s new information literacy product, Research Companion, offers videos that guide users through the research process,...

Sphere2 & Class Send Student Engagement Software Platform

(www.averusa.com) AVer Information has developed a Student Engagement platform, providing teachers and students with the tools to transmit document ca...

TabChargeCT2

(www.averusa.com) AVer has released the TabChargeCT2 charge cart solution, which can hold up to 40 Chromebooks, iPads, Android or Windows tablets, lap...

VoiceLift with Instant Alert and Emergency Video Monitoring

(www.extron.com) The Instant Alert function of the Extron VoiceLift Microphone, combined with a PoleVault, WallVault, or PlenumVault classroom AV sys...

SMART Board 6065

(www.smarttech.com) The SMART Board 6065 is an ultra HD, 4K interactive flat panel that offers collaborative capabilities while ensuring lessons run s...

Gaggle Unity Partner Program

(www.gaggle.net) The new Gaggle Unity Partner Program provides free data integration services for all educational technology vendors. Through the Gagg...

Waterford Early Learning, Reading, Math & Science

(www.waterford.org) Waterford Early Learning Cloud can be used at home or to supplement lessons in classrooms. It can also be used for individual adap...

NetSupport School

(www.netsupportschool.com) The latest version of NetSupport School allows teachers to monitor and collaborate with any mix of technology. An enhanced ...

Camtasia

(www.techsmith.com/camtasia) TechSmith’s Camtasia gives teachers the tools to record lessons, create videos, and engage their audiences. Educato...

Panasonic 3E

(www.panasonic.com) Intel has teamed up with Panasonic to announce the Panasonic 3E, which uses the Intel Education 2-in-1 reference design. Designed ...