Schools are teaching students AI literacy by exposing chatbot mistakes

The teachers and principals watched as an instructor prompted an AI tool to “create a map of the world.”

The result drew gasps of disbelief and laughter. Projected on a large screen was a map riddled with bizarre errors. Mali was spelled “Mail.” Egypt was identified as “Sopth.” In place of Libya was something simply called “Africa.” Dozens of other countries’ names were misspelled or written in pure gibberish.

“If you ever want kids not to over trust these tools, try the map demo,” said the instructor, Amanda Bickerstaff, founder and CEO of AI For Education, an organization that helps schools draft AI policies and train teachers and students in what she calls safe, ethical and effective use of the technology.

After initially trying to ban AI use, a growing number of U.S. public schools are trying a new strategy: encouraging classroom experimentation, partly so students can see its shortcomings, including generative AI’s notorious tendency to “hallucinate,” or fabricate information.

AI literacy has become a buzzword of this back-to-school season as educators try to strike a balance between equipping students with the skills they need for an AI-driven future while preventing them from outsourcing their thinking to chatbots.

There is no single definition of what AI literacy means, or how it should be taught. Tech giants like OpenAI, Google and Anthropic offer schools training sessions in how to use their AI tools. But for educators, there is a growing consensus that AI literacy is about more than learning how to use generative AI and write good prompts.

“Good AI literacy,” said Rebecca Winthrop, director of the Center for Universal Education at the Brookings Institution, “includes knowing when not to use it.”

Some districts build their own approaches on AI literacy

Thirty-seven states have now published official AI guidance that schools can use as a blueprint. South Carolina is not one of them.

So the state’s second-largest school district set out to do it on its own, said Lucas Clamp, deputy superintendent of the 50,000-student Charleston County School District. “It has been a huge endeavor.”

A year ago, the district contacted AI for Education, which helped draft guidance for Chicago Public Schools, Houston and dozens of other districts.

Phase 1 was laying the foundation. The district developed an AI policy, seeking input from teachers, students and parents.

The district organized student focus groups and learned that student AI use was ubiquitous but unguided. Teachers and students said they were navigating the technology on their own and wanted clear rules and instruction.

Studies show a majority of teenagers and teachers nationwide are using AI for schoolwork, but they’re teaching themselves how to use it; very few say they have a technical understanding or have received formal training on how it works.

Schools and parents are eager to avoid mistakes made with social media, where kids were left to explore the online world without learning about addictive algorithms, comparison culture and other downsides that have been linked to a youth mental health crisis and dwindling attention spans.

As the school year begins in Charleston, Phase 2 kicks in: training. Teachers and middle and high schoolers will get a mix of online and in-person instruction on how AI tools work and how to use them effectively.

The student course walks through familiar scenarios — like asking a chatbot to help with a research paper — to show that generative AI might deliver wildly inaccurate information, including made-up studies, with unflinching authority. “Always verify any factual information you get from GenAI, especially for your classwork,” the course advises.

A section on data privacy cautions against sharing personal information with AI tools because conversations can be stored, used for data training and potentially leaked.

High school teacher Ray Knauer attended the summer training and left with “teachable moments” for his AP Research class. He plans to show examples of AI bias and hallucinations to fuel student discussions.

“We’re all trying to figure this out right now: What’s the right balance so we don’t scare students away from AI but don’t encourage them to run to it,” Knauer said. “We have to teach students: You can’t just get an answer and move on. You have to use analytical skills and dig deeper.”

Some states offer top-down guidance on AI in classrooms

Utah was an early pioneer in taking a statewide approach to AI in education.

In 2024, the board of education named Matt Winters as its AI education specialist, making Utah the first state to create a full-time position overseeing the technology in schools.

Over the past year, Winters led AI training for over 7,000 teachers, almost a third of Utah’s public school instructors. He is helping districts shape AI policies, which they are required by state law to have in place by July 2027.

“There is a clear method from the top in Utah,” said Chris Agnew, director of the Generative AI for Education Hub at Stanford University. “When you’re tackling big change, it’s easier to have coherence if everyone is singing from the same hymn book.”

A handful of other states have since replicated Winters’ post, including Maine, West Virginia and Georgia.

In other states, a patchwork of AI approaches has emerged, with variance from one school district to another. Extensive AI literacy training requires steep investment, raising concerns that kids in under-resourced districts might not get the same attention.

In Utah, training starts with the basics of how AI works to teach an understanding of AI bias, hallucinations, ethical concerns and data privacy needs, Winters said. Teachers can then pursue additional training on specific tools.

“If I had to give advice: Figure out how to make training accessible to everyone,” said Emma Moss, who oversees AI at the Canyons School District and also led an eight-person team that traveled the state doing over 150 AI literacy trainings. The training was funded by a $500,000 AI literacy grant from a local healthcare provider. “We said, ‘We will come to you.’ Whether you are in a rural area or urban, it didn’t matter.”

The state has also played a key role in procuring AI tools, Winters said. He negotiated data privacy agreements along with discounted prices, which has allowed rural and under-resourced districts to get access.

“Everyone needs to understand what this technology is and how it works,” said Winters, who cautions against blanket bans. “AI literacy is much larger than saying: ‘Don’t use AI.’ It’s about how we can prepare kids for the future.”

Changing adult mindsets is also key, said Kristina Yamada, the state Board of Education’s digital technology specialist.

“I think teachers need to change their mentality about students using AI to cheat,” Yamada said. There has always been cheating, she said. It’s up to teachers to speak students’ language and convince them they can’t rely on AI for all the answers.

For younger kids it means starting to talk about AI early, with conversations like, “People can lie. A program can lie. A robot can lie,” Yamada said. “For older students it’s different: ‘Sure, you can use ChatGPT to write computer code. But when the program stops working, you need to be able to fix it.’”

___

Gecker reported from San Francisco.

___

The Associated Press’ education coverage receives financial support from multiple private foundations. AP is solely responsible for all content. Find AP’s standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.