For most users, Meta's 2024 integration of AI across its social media tools was just another update. But for schools in Puno, Peru—a predominantly rural region where 79% of teachers rely on WhatsApp extensively for instruction—it raised an uncomfortable question. What happens to student data when essential infrastructure evolves without consent?
These teachers have used WhatsApp for years to coordinate schedules, share assignments, and communicate with families because it doesn't use mobile data—a critical advantage in their low-connectivity area. But the platform now gives AI access to educational interactions without school oversight or control over sensitive student data.
This dynamic isn't unique to Peru. Recent legal complaints show vendors launched AI in U.S. classrooms under existing contracts without renewed consent. And when these tools become infrastructure, whether due to cost constraints, connectivity issues, or reluctance to change, schools lose the ability to say no once AI is part of the package.
But platform evolution is only part of the problem. While districts grapple with vendors adding AI features, staff are bypassing institutional governance entirely by using unapproved AI tools on their personal devices. A recent survey shows 78% of education employees know of colleagues using unauthorized AI tools, and the problem is growing.
This article examines how schools are losing control of student data as AI enters classrooms through both vendor platforms and unauthorized staff adoption—and why existing data governance policies are ill-equipped to address either pathway.
3 Ways AI Integration Undermines Student Data Privacy
When the public thinks of AI in schools, they might picture the illicit use of ChatGPT to crank out essays and solve complex math equations. But education leaders know AI's use cases are far more vast and varied, with features embedded in grading software, learning analytics, and other tools schools have already entrusted with student data.
Unfortunately, these platforms can now add AI features without renewed consent. Everything from Google Classroom and Microsoft Teams to learning management systems (LMS) and WhatsApp is now able to analyze student questions, assessment responses, and behavioral patterns thanks to AI integration.
This impacts student data governance in three ways:
1. Student data becomes permanent
Once student data is used to train AI systems, the models retain patterns and information derived from those data points. Current unlearning methods can remove individual records from databases but not the learned behaviors retained by complex neural networks. This means that even if a school requests deletion, AI output may still contain details derived from students’ performance, writing style, or learning profiles.
Analysts have warned that the embedding of student traits raises long-term identity and fairness risks, especially if reused to train commercial AI models. Emerging frameworks still lack effective mechanisms to ensure models “forget” once trained—a critical issue in education, where consent is more often delegated by institutions than by individuals.
2. Behavioral profiling happens without consent
While platforms such as WhatsApp deploy end-to-end encryption for content, metadata (e.g., message timing, class interaction frequency, or question-response length) remains visible to servers and analytics systems.
MIT’s research on metadata protection shows that this secondary data can reveal behavioral and emotional patterns, such as learning difficulties or absenteeism trends, even without message content exposure. When analyzed alongside LMS logs or classroom camera data, metadata can construct a high-resolution behavioral map of students, effectively profiling cognitive and social engagement patterns without direct consent.
3. Vendor lock-in eliminates oversight
As schools embed AI-ready ecosystems from Microsoft, Google, and Meta, they may increasingly depend on proprietary APIs for grading, attendance analysis, or chatbot tutoring. Recent reports note this dependency constrains contract renegotiations: smaller districts lack the capacity to audit algorithms or demand granular data deletion timelines, effectively accepting “as-is” privacy terms dictated by vendors.
Even well-resourced institutions may find it difficult to exit or migrate systems once data pipelines and assessment workflows are tightly integrated. This dependency amplifies vendor power and blurs accountability, making oversight reactive rather than preventive.
The 2024 cases against IXL Learning and other companies demonstrate all three of these risks in practice: AI features were added to widely used platforms under existing contracts, creating permanent data retention, behavioral profiling via metadata, and limited institutional recourse for schools already dependent on these systems.
While districts grapple with the ramifications of platform evolution, another crisis is unfolding simultaneously: educators adopting AI tools that bypass governance entirely.
How Shadow AI Makes Things Worse
The effects of unvetted AI integration compound when staff use unauthorized AI tools. And the 78% who know about unauthorized use are just the beginning: sixty-two percent of education employees have fed work-related content into AI without district approval, and 50% report more unauthorized use than a year ago.
The root cause is a dangerous gap between how confidently employees use AI tools and how little confidence organizations actually have in the tools' security, accuracy, or compliance.
Eighty-four percent believe unauthorized AI protects their info, 52% see little or no risk in unauthorized use, and 42% didn't know approval was needed.
This confidence-compliance gap creates serious institutional risk. When staff enter student data into unauthorized AI tools—for instance, excerpts from assessment data into ChatGPT to generate report card comments—they may inadvertently violate laws such as FERPA and COPPA.
And education IT leaders are struggling to keep up:
- 84% agree employees adopt AI faster than IT can assess.
- 83% say it’s challenging to control unauthorized AI use.
- Only 40% say their schools have clear, enforced AI policies—meaning 60% are operating without effective governance.
In some ways, unauthorized staff AI use feels more egregious than platform integration risks. It's easier to blame a teacher using AI on a personal device to generate quiz questions and answer keys than a district locked into contracts with Microsoft or Google.
But this mindset misses the point. Both problems stem from the same failure: schools lack the frameworks to address AI that enters through existing infrastructure or through staff adoption that outpaces oversight. And both create the same consequences: permanent data embedding, metadata exposure, and loss of institutional control.
The question isn't who's to blame. It's how to build governance that addresses both pathways before the window for intervention closes.
4 Ways To Maintain Control of Data
For most districts, crafting data governance policies is a months- to years-long process that involves assessing current tools, needs, and resources, followed by drafting, stakeholder review, and board approval. But no matter how carefully education leaders develop these policies, the truth is these aren't designed for AI.
Traditional data policies assume platforms remain static, that tool adoption happens only after approval, and that initial consent suffices for ongoing use. These focus on preventing data from leaving approved systems but can't address what happens when those same systems implement AI features or when staff bypass approval entirely.
These policies can't account for data permanence, metadata exposure, or eroding institutional leverage. Instead, effective AI governance requires an approach built on the following four pillars:
1. Set clear guidelines for sensitive data entry
Guidelines should define what student data can or cannot enter AI systems, whether those systems are authorized or not. For example, schools may restrict teachers from entering student names, grades, or essay text into external AI tools such as ChatGPT unless the tool is specifically approved or governed by the district.
Questions for your district:
- What types of student data does our policy explicitly prohibit from entering into AI tools?
- Are there clear, written guidelines available for staff on what data is considered sensitive or protected?
2. Establish platform oversight protocols
Schools must decide how to monitor and respond when vendors add AI features. This might look like the IT department reviewing vendor updates quarterly to identify whether new AI features have been added and notifying staff about any changes affecting data privacy.
Questions for your district:
- How do we find out when vendors add or change AI features on our platforms?
- Who is responsible for monitoring platform updates and communicating risks to staff?
3. Encourage mandatory staff training
It's essential that schools understand data implications, metadata risks, and why approval matters. This might look like all teachers completing an annual online module about risks of sharing student data with unauthorized AI tools and recognizing metadata privacy issues.
Questions for your district:
- Have all staff received baseline training on AI data risks in the past year?
- Does our training cover emerging risks, such as metadata exposure, and use real classroom examples?
4. Provide enforcement and approved alternatives
Schools should provide vetted alternatives rather than blanket bans. For example, the district might provide a vetted AI writing assistant integrated within the learning management system so teachers do not need to use unsupported external tools.
Questions for your district:
- Do staff have easy access to a list of approved AI tools and know how to request new ones?
- How do we ensure enforcement of policies—what are the steps if unauthorized AI use is reported?
Together, these four pillars create a governance framework that addresses both platform evolution and unauthorized adoption. The question is whether schools will implement them proactively or wait until a data breach, regulatory violation, or community crisis forces reactive measures.
Regaining Control
Schools can regain control of student data by establishing clear policies, implementing platform oversight protocols, training staff on AI risks, and providing vetted alternatives. But the window for action is closing. Vendors keep adding AI features, unauthorized use is growing, and every delay builds dependency, as seen in schools from Peru to the U.S.
Schools that take proactive measures now can still determine what AI does with student data and which tools enter classrooms. Those that wait will find these decisions made for them.
