Assessing Formative Assessment

1/15/2006 5:00:00 AM

It's no secret that much of the focus in K-12 education has shifted to assessment. While summative assessments — evaluations conducted at the end of an academic year — are nothing new, formative assessments that reveal student performance throughout the year are increasingly being used by districts to better target curriculum, instruction, and staff development. Many companies have developed technology-based tools to help school districts make these efforts more manageable. But how do you know which ones will work best for you?

1. How do formative assessment systems work?

Formative assessment tools come in two main varieties: Web-based and software-based. The Web-based tools, also known as application service providers, are online versions of traditional paper-based tests. Vendors offering this approach include Tungsten Learning, STI, SchoolNet, Scantron, Renaissance Learning, Pearson School Systems, and Harcourt Assessment (Edusoft plans to introduce online assessment in the coming year).

The software-based tools come as programs to be uploaded to a server or to individual computers in a networked classroom. They include products such as Focus on Standards from ETS and state-specific applications from CTB/McGraw-Hill.

With both of these options, students get on a computer in a lab or in the back of the classroom and take an assessment that the teacher has queued up for them. In some cases, teachers can tinker with program protocols so different students receive different sets of questions. With the right presets, for instance, special needs students in a particular class can receive questions that quiz concepts at a more basic level.

There is also an emerging generation of formative assessment tools that work with handheld computers. These systems take perhaps the least cumbersome approach, combining hardware and software in a way that lets educators administer research-based assessments without the manual score calculations and data entry often required to meet state and federal guidelines. Educators administer these tests verbally in one-on-one situations and record answers and observations directly onto the handhelds. Vendors in this niche include Wireless Generation and Liberty Solutions.


Harcourt Assessment's Stanford Learning First is a custom-built, Web-based summative and formative assessment tool that is configured to state standards.

2. What subject areas do formative assessment tools cover?

Most formative assessment solutions cover math, reading, science, social studies, and English language arts.

In most cases, assessment technology comes equipped with a database of subject-specific standard questions.

In rare cases, certain sets of questions in particular subject areas come as add-on modules, and districts are required to purchase the content after they buy the assessment technology itself.

A few products are designed to assess knowledge in specific areas only. The Benchmark Assessment System from Tungsten Learning and the mCLASS solution from Wireless Generation only assess skills in reading and math, for example. Before you decide which assessment tool to purchase, it's important to make sure you're investing money in a tool that assesses all of the subject areas you need to test.


STIAssessment from Software Technology automatically generates individual and aggregate reports by class, school, and population subgroups.

3. Where do assessment questions come from?

Most vendors hire writers with curriculum development experience to formulate questions for formative assessment tools. One reason traditional textbook publishers like Houghton Mifflin, Harcourt Brace Jovanovich, McGraw-Hill, and others have jumped into the formative assessment market is because they have tens of thousands of these questions on file already. It's important to investigate content fully — some vendors may use questions that were created years ago and therefore fail to adequately evaluate knowledge on modern subjects.

The questions themselves can take many forms, ranging from multiple-choice to short-answer questions or longer performance tasks engaging students in real-world problems. Sound assessment questions meet technical criteria of validity, the degree to which particular uses and interpretations of assessment results can be justified, and reliability, the degree to which scores are free of measurement error. One simple way to determine whether an assessment's questions meet those standards is to ask the vendor who authored the assessment. Also, it's important for districts to do psychometric research to analyze the items in the assessments and make sure they do the job.

4. How do I know if a formative assessment tool is aligned to my state standards?

Just about every formative assessment vendor claims its system is aligned to state standards, but district-level leaders must dig deeper to see just how valuable the system actually is. The best formative assessment tools are those that exist in several iterations that were developed from the very beginning with specific state standards in mind. Unfortunately, many vendors cut corners by conducting one-time development efforts, then aligning those efforts to meet the standards of each individual state. To find out more about your assessment, ask for a blueprint of the assessment itself. Laws in most states require that vendors specify whether their tools fulfill their educational standards, and from this document, educators can get a good sense of how a particular assessment measures up to others on the market.

5. In what forms do formative assessment systems generate data?

All formative assessment systems include methods to communicate results through reports. In most cases, these reports are generated online, giving teachers the option to share results with parents by passing on a Web address and some log in information. Many assessment tools also come with the option of printable reports, which can be added to a student's file or scanned as part of a summative diagnostic. Some formative assessment vendors offer modules that integrate assessment data into student information systems for reporting across a variety of data categories.

6. Once I get the data, what do I do with it?

Assessment data is only as good as the actions that arise from it. Only a handful of formative assessment systems include tutorials to help educators understand what data means. As a result, many formative assessment tools aren't used to their full potential. Compounding this problem is the fact that few professional development providers have enough contextual information about a particular classroom environment to make a difference. Currently, the best way to solve this problem is to turn to a consultant (such as the Assessment Training Institute in Portland, Ore.) or hire an in-house data expert.

7. How much classroom time should formative assessment take up?

No teacher will embrace a new technology that requires a lot of valuable classroom time. Districts must therefore select a formative assessment tool that's quick and easy for teachers to use and then train them how to use it. Finally, districts must instruct their teachers how to incorporate the technology into ordinary lesson plans. Many vendors offer optional support and training services; consultants do the same. However you go about familiarizing teachers with formative assessment, buy-in is essential. If teachers don't support the technology from the beginning, chances are it won't be assessing very much.

Matt Villano is a California-based freelance writer.

Pick Her Brain

Last year, when Lakeland School Corp., in LaGrange, Ind., selected formative assessment tools from Software Technology, MIS Coordinator Gail Nezworski was right at the center of the implementation. "Assessment is wonderful because instead of teaching to specific standards, you can target your teaching approach for each individual student. In our district, we have used assessment tools to empower our teachers to determine how much their students understand by giving them the ability to get real-time results. As teachers finish each lesson, instead of waiting for a high-stakes test at the end of the year, they do the assessment right then and there. After the assessment, if it's clear that a student needs more time to understand a subject, the teacher can spend more time on the subject with that student. If not, the teacher can move on. The system allows for teacher flexibility and ensures that every student is learning what they need to learn in the way they can learn it best."

Who's Who

CTB/McGraw-Hill
www.ctb.com

ETS
www.ets.org

Harcourt Assessment
www.harcourtassessment.com

Liberty Source (Tango)
www.tango-software.com

Pearson NCS
www.pearsonncs.com

Pearson School Systems
www.pearsonschoolsystems.com

Princeton Review
www.princetonreview.com

Renaissance Learning
www.renlearn.com

Scantron
www.scantron.com

SchoolNet
www.schoolnet.com

STI
www.sti-k12.com

Wireless Generation
www.wirelessgeneration.com

Buyer's Checklist
  • Select a medium for your formative assessment: Web-based, software-only, or a combination of software and hardware.
  • Make sure the tool evaluates all necessary subject areas.
  • Check to see that the questions in the tool's database have been written by a professional and are relatively current.
  • Investigate the product blueprint to ensure compliance with your state's standards.
  • Test the tool's data-generation mechanism to be sure the data will be easy to manage and contextually worthwhile.
  • Train teachers how to use the tool, and make sure it is something they will use in their classrooms on a regular basis.

Want to read more stories like this?

Get our Free Newsletter Here!

comments powered by Disqus
T&L Partner Post
Tweets