Many schools, districts, and states have responded to No Child Left Behind by implementing data management systems, including data warehouses, to help them efficiently organize and analyze the enormous amount of student and school performance data they generate each year.
According to the November 2003 Eduventures report, Making Sense of the Data, the market for data management and analysis systems has grown to $145 million and will continue to expand each year. As the industry has grown, so has the number of vendors providing data warehousing tools and services. In fact, there are now more than 20 vendors serving this market, some with more than 10 years of experience and others just signing up their first customers. This expansion benefits school and district leaders in that it provides additional options and creates increased competition. On the downside, it makes selecting the right solution that much more challenging.
Whether you're buying an off-the-shelf data warehouse or intend to build your own, the following questions will guide you through the critical process of selecting a product or finalizing a product design.
1. Does the solution integrate the information systems you want to explore?
A good data warehousing solution will allow you to integrate data from any source, whether it's your district-wide student information system or informal databases housed on spreadsheets. Accordingly, look for providers with specific experience integrating data from student information, assessment management, special education, human resources, and finance systems, as well as data from Microsoft Excel, Microsoft Access, comma-separated values, field-length flat files, and tab-delimited formats. You'll also want a solution that allows for unlimited longitudinal data.
Some data warehousing vendors have established strategic partnerships with student information system vendors. These alliances allow the warehousing vendors to develop better tools for data extraction and cleansing (sometimes called ETL for Extract Transform and Load). When selecting a product, ask your vendors to describe their partnerships and provide references for successful projects.
2. Does the data model fit your needs?
The goal of a data warehouse is to allow users to explore relationships between information bits that were previously disconnected. How does classroom attendance relate to performance on standardized tests? What's the link between professional development and course grades? How you'll be analyzing these connections and others will depend greatly on the type of database technology being used.
Many data warehouses, for example, are built around predefined multidimensional data cubes. In this design, the vendor builds an optimized database using a limited number of commonly used data elements such as demographics and test scores. The result is a highly efficient system with extremely fast response times as users navigate through the optimized dimensions of the cube. (The software used to analyze data cubes is known as online analytical processing tools.)
A less efficient, but far more robust, alternative to cube-based solutions are relational database management systems. Most conventional databases employ a RDBMS structure where each data element is stored only once and is connected to every other data element. This allows users to develop ad hoc queries to analyze any data in the system. Data is not consolidated or optimized to improve search efficiency, but all details of all data elements are available for exploration and analysis.
3. What data domains are supported?
Every piece of information collected by a school that relates to an individual student's performance should be available for analysis. Look for solutions that, at a minimum, include the following data domains:
- Student demographics
- School and course enrollment
- Course and instructor attributes
- Student attendance
- Standardized assessment results
- Course grades
- Special education status, plans, services, and events
- English proficiency
- Discipline referrals
- Extracurricular involvement
- Transportation, food service, and medical
- Staff demographics, attendance, and professional development
- Program tracking and evaluation
- Budgets by account, department, and location.
4. What's the update frequency of the system?
Generally, a data warehouse is used for analytical purposes rather than day-to-day operations, so it's not critical the warehouse include the most recent data such as yesterday's attendance. More likely you'll find vendors offering updates that range from weekly to annually. Before selecting a product, decide what data elements are needed for your analytical purposes, determine the frequency with which you need to refresh the warehouse, and then choose a product that will meet your needs.
Also consider how long the company will take to complete an update once they receive a fresh data set. Depending on the data type and the complexity of the extraction and cleansing process, vendors provide turnaround times ranging from overnight to three weeks or more. If you've just received your high-stakes testing data and want to quickly integrate it into your warehouse so you can prepare reports for various audiences, a three-week wait may be unacceptable.
5. Where is the data warehouse hosted and how do I access it?
Typically schools and districts will benefit from having the warehousing provider handle the entire process, from data extraction and transformation to loading and hosting the resulting database on district servers. School users can then access their warehouse via the Internet. This approach allows the vendor to maintain and update their database efficiently and relieves the district of additional technical burdens. Schools do not lose direct control of their data, because the component systems can still be housed and managed locally.
Be sure to ask detailed questions about how the vendor will ensure the security and integrity of the data you furnish them. First, ask about the physical security of your data both in transit from your location to the vendor's location and on-site at the vendor's premises. Second, ask about the electronic security of your data. Data sent via Web systems should be secured using the Secured Sockets Layer protocol and security certificates. If using the vendor to host your data warehouse, then review their security policy to ensure that it includes adequate firewall, encryption, and password protections.
6. What level of support is available?
Flexible options for face-to-face, telephone, and e-mail support, including convenient help desk hours, should be key components of your warehousing package. Some vendors will host periodic user group meetings and provide user forums on their Web sites. Beyond purely technical support, it can be helpful if a company provides educational consulting services to assist with more complex questions related to designing queries and understanding results.
7. How long will the implementation take?
Find out how much time will be required for the data warehousing provider to implement the product. Depending on the size of the district and the quantity of data integrated into the system, you can expect at least 90 to 120 days for the initial round of data extraction and transformation. Training will require additional time ranging from one or two days for end users to one or two weeks for high-end technical users.
8. What K-12 experience does the vendor have?
While data warehousing in education has been slowly expanding for almost a decade, a very small number of vendors have extensive experience in schools with real data and real problems. Look for companies that understand and speak fluently about the unique difficulties of integrating education data. Ask your prospective vendor for a list of its customers, and contact a number of other districts not on their list to thoroughly investigate their experience with the vendor.
Todd McIntire is the vice president of achievement for Edison Schools.
Here are seven attributes your data warehousing system should offer.
- Integration tools for your key student information systems, including informal databases held in spreadsheets
- Easy-to-use tools for conducting online analytical processing or building ad hoc queries
- Comprehensive data extraction and cleansing tools
- Support for all of the data domains you wish to analyze, including support for longitudinal capture and reporting
- Multilayer security, including firewall, SSL, certificates, and encryption
- Rapid turnaround of data updates
- Ample professional development training services and technical support
The following companies represent some of the many vendors in the fast-growing data management market offering tools and services for K-12.
BearingPoint collaborates with school districts to create custom data warehouses. Their solutions include longitudinal reporting tools for data on test scores, special education services, and certification and demographic data regarding school employees. (866) 276-4768; www.bearingpoint.com
Confluent's ConfluentEDU is a data warehousing and data mining solution that includes a graphical interface for creating customized pie charts, bar graphs and other visual renderings and reports. ConfluentIQ is the company's business intelligence solution. (502) 212-4013; www.confluentasp.com
EDsmart offers EDdiscover, an OLAP cube-based system that provides interactive access to various views of information; and EDexplore, a Web-enabled data reporting, analysis, and warehousing system that employs a RDBMS-based approach to analysis and reporting. (860) 225-3316; www.edsmartinc.com
eScholar's Complete Data Warehouse was the first data warehouse designed specifically to meet the needs of K-12 education and is the most widely used solution, serving nearly 800 school districts. The company also offers eScholar NOW, which lets districts access longitudinal assessment data. (877) 328-2969; www.escholar.com
IBM's Insight at School data warehousing program is built on the IBM DB2 Universal Database and can be installed on a number of standard platforms. Partner products include Business Objects, Cognos, and Brio. (800) 426-9990; www.ibm.com/solutions/education
SAS Education Performance Management includes data warehousing, reporting, and SASinSchool curriculum software. Technical components include SAS Enterprise ETL Server, SAS Intelligence Storage, and SAS Intelligence Platform. (800) 727-0025; www.sas.com
SchoolNet's Unifying Technology Platform includes a data warehouse, but also lets districts integrate technology systems from curriculum and assessment providers under a single user interface. A role and permission system allows for secure single-sign-on to all district data and applications. (866) 328-2374; www.schoolnet.com
TetraData makes data management products for state departments of education, K-12 districts, and private organizations. The company's EASE-e Analysis Suite is a data warehouse, mining, analysis, and reporting system. EASE-e Data Services provide data collection, validation, and warehousing services. (864) 458-8243; www.tetradata.com
Interested in learning more about data collection and management issues? Our Web site houses a range of relevant articles.
How to Perform a Data Makeover by Richard Hoffman
The true value of your district's reservoir of information depends largely upon two things-the quality of the data and the tools you have to analyze it.
Data: Mining with a Mission by Judy Salpeter
Data-driven decision-making is the buzz phrase of choice for the new decade. But once we've got the information, how do we use it to yield results? Here, 20 school administrators share their expertise.
Student Information Systems Demystified by Todd McIntire
The increasing demand for accurate, timely data means schools and districts are relying heavily on SIS technologies. What are key facts you need to know before investing in a new system?
Digging for Data by Todd McIntire
No Child Left Behind's challenge: collect, analyze, and use information to improve student performance. We offer case studies of three districts that are getting it right.
The Administrator's Guide to Data-Driven Decision-Making by Todd McIntire
With the right information at the right time, school leaders can evaluate educational programs and create services to meet student and staff needs.