Redefining Data-Driven Decision Making

from School CIO

Throughout the country, district superintendents expect their chief information officers and IT staff to play an instrumental role in helping educators increase the quality of education. But the processes and steps to improving performance are often not so clear.

In 2003-2004, educators from 12 districts in the St. Louis metropolitan area decided to tackle this complicated issue head on. They participated in a series of workshops in which they identified and analyzed factors that strongly influence student learning. After exploring numerous barriers to achievement, it became clear that any solution would have to be systemic in nature and could not rely solely on information technology. The concept that emerged was referred to as Educational Performance Management (EPM): “Educators using data and decision support tools to continuously improve educational practice for the purpose of increasing student learning.”

EPM: The Vision

The St. Louis group took its analysis a step further by defining six integrated disciplines, which, along with a variety of decision support technologies, would enable educators to do their jobs more efficiently and effectively. Following are snapshots of each discipline.

1. Knowledge Leverage

Knowledge is the essential ingredient to successful educational practice. Therefore, EPM positions professional learning and development as high priority themes for districts and buildings. Additionally, knowledge accrued by educators becomes a major contributing factor during annual performance and developmental reviews. The EPM Professional Accountability & Contribution Evaluation (PACE) method provides a progressive approach to assessing professional growth; identifying developmental needs; interpreting individual and team performance; and calculating fair rates of compensation.

Leveraging knowledge is achieved by sharing facts and insights across educator “communities of interest.” EPM facilitates both the contribution and selective retrieval of facts, insights, experiences, and rules-of-thumb (heuristics) from a broad collection of shared databases.

2. Instructional Differentiation

Educators have little time to implement this effective strategy for accommodating individual student needs and helping students optimize their learning experience. To facilitate instructional differentiation, EPM tools are used to automate the design of multiple lesson plans with varying degrees of content, detail, and difficulty. The design process interprets several inputs including a characterization of the student’s aptitudes/interests, learning grade-level, and curriculum schedule. Time saved by educators via differentiated lesson design is freed for higher-impact instructional or developmental activities.

3. Directional Alignment

The group identified a collection of primary objectives that, if achieved, would eliminate many of the constraints surrounding the goal of increased student learning:

  • Knowledgeable, skilled educators
  • Motivated, engaged students
  • Curriculum objectives that meet state-level expectations, accommodate individual student aptitudes/interests, and position all students for success on summative high-stakes assessments
  • Ongoing translation of educational research into instructional and administrative strategies and techniques, with subsequent infusion in daily practice
  • Differentiation of lesson content and strategy on an individual student basis
  • Explicit specification of learner objectives for every student, educator, and parent
  • Targeted, beneficial, and timely improvement to educational practice
  • Ubiquitous knowledge transfer without boundaries, thereby accelerating collaborative problem solving and individual professional learning

EPM enables the specification of these and other objectives in ways that can be cross-compared using “dimensions” (building, class, subject, potential impact, established priority, completion date, responsible individual or team, etc.). Linking objectives via a supporting/supported rubric allows educators to clearly understand the direction their district/building is taking and the ways in which their contribution to specific objectives may affect performance from the district down to individual student levels.

4. Practice Integration

Each instructional or administrative practice is defined in terms of a sequence of activities, required resources, and information/knowledge that is created or shared during the conduct of the activity. Practice “integration” is accomplished by examining the “end-points” of an activity sequence and identifying what other practices might use the products/services emerging at these junctures.

Specifying and evaluating educational practice often leads to the identification of efficiency and effectiveness improvements. In addition to supporting continuous improvement, practice specifications help educators gain better insight into “how things work” in a district or building. Finally, practice specification forms the basis for designing explicit roles associated with each job/position. Role specifications become the starting point for PACE reviews and enable the clear identification of knowledge/skill requirements associated with each instructional or administrative practice.

5. Situation Analytics

Closely akin to data-driven decision making (D3M), this EPM discipline and related toolsets help surface trends and correlations that often hide within large volumes of data. EPM ensures that users do not require an in-depth understanding of information technology in order to meaningfully interact with available data, analyses, and knowledge-based interpretations.

Elements of student data maintained by EPM extend well beyond disaggregated summative test scores and attendance (a student’s engagement rate, reading comprehension/fluency/retention index, critical thinking skill level, etc.). EPM also requires a broad spectrum of text-based and numeric data, thus involving database structures that are more complex than typical data warehouse implementations (objective and strategy statements, priorities, role specifications, student characterizations, research findings, instructional and administrative practice specifications, standards, and actions projects).

6. Improvement Diligence

Many improvement initiatives fail to generate desired results due to one or more common faults:

  • Objectives not characterized in terms of a measurable factor or not properly prioritized
  • Actions not tracked to completion with regular status reviews
  • Unclear or excused accountability for action completion
  • Poor planning or lack of time/resources or strategies for mitigating risk

EPM integrates the formal discipline of change management into the daily lives of teachers and administrators in order to capitalize on improvement opportunities and ensure long-term maturation of educational practice.

In Closing

The challenges surrounding EPM implementation are daunting, the anticipated benefits are far-reaching, and the amount of change to the status quo represents nothing short of a paradigm shift. If you would like to get more details about EPM and the six disciplines, e-mail me at KeithWaters@charter.net.

Keith Waters is a process consultant based in Missouri.