How to Prevent Cognitive Surrender in the Age of AI

Think First

AI and Critical Thinking

New Wharton research shows how easily people accept AI answers without scrutiny. Here is how course design and social annotation help students think before they rely on AI.

AI Can Help Students Learn. But It Can Also Change How They Think.

Every instructor has seen it happen.

A student pastes a prompt into ChatGPT before opening the assigned reading. Another asks an AI assistant to summarize an article instead of working through it independently. Others rely on AI-generated explanations before they’ve had the opportunity to wrestle with unfamiliar ideas themselves.

Generative AI has made information more accessible than ever. Students can generate summaries, explanations, outlines, and even complete drafts in seconds. These tools offer tremendous opportunities for learning, but they also raise an important question:

What happens when students begin outsourcing the thinking itself?

A recent research paper from The Wharton School introduces a new framework for understanding this challenge. In Thinking, Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender, researchers Steven D. Shaw and Gideon Nave argue that AI is becoming a third cognitive system that people increasingly rely on while reasoning. They call this Tri-System Theory, and one of its central predictions is the rise of cognitive surrender: the tendency to accept AI-generated answers with little scrutiny, even when those answers are wrong.

Read the full Wharton paper here.

For higher education, the findings are especially relevant. They suggest that preparing students for an AI-enabled future isn’t simply about teaching them how to use AI. It’s about helping them preserve the habits of thinking that make learning meaningful in the first place.

What Is Cognitive Surrender?

According to the Wharton researchers, cognitive surrender occurs when people rely on AI-generated responses instead of engaging their own reasoning.

In their experiments, participants frequently chose to consult an AI assistant. When the AI provided accurate information, performance improved substantially. But when the AI intentionally produced incorrect answers, many participants still accepted those responses with surprisingly little skepticism. Even more striking, participants often became more confident in their answers despite being wrong.

Rather than carefully evaluating AI outputs, many users simply adopted them. This tendency is what the researchers describe as cognitive surrender.

The concern is not that AI sometimes makes mistakes. It is that people may stop engaging in the critical thinking needed to recognize those mistakes.

Understanding Tri-System Theory

To explain this shift, the researchers expand the familiar dual-process model of thinking.

Traditionally, psychologists have described reasoning as a balance between fast intuition (System 1) and slower, analytical thinking (System 2).

The Wharton paper argues that generative AI has introduced a third participant into that process. Rather than simply serving as a reference tool, AI increasingly participates in reasoning itself.

Sometimes this external cognitive partner improves performance. Other times, it encourages people to bypass their own judgment.

For educators, this raises an important instructional question: How can we ensure students use AI to support thinking rather than replace it?

Why This Matters for Higher Education

College has never been about memorizing information alone.

Students are expected to analyze evidence, evaluate competing arguments, identify weaknesses in reasoning, synthesize ideas across multiple sources, and communicate original conclusions.

These are precisely the kinds of habits that can weaken if students begin accepting AI-generated responses without reflection.

The Wharton researchers found that conditions matter. When participants were given incentives for accuracy and immediate feedback on each answer, they were far more likely to question and override incorrect AI advice. The environment people reason in shapes whether they think critically about what AI tells them.

In other words, simply telling students to “use AI responsibly” may not be enough. Instructional design must intentionally create opportunities for students to think before they consult AI.

The Problem Isn’t AI. It’s Invisible Thinking.

Many traditional assignments focus only on the final product.

Faculty see the essay, discussion post, or exam response, but they rarely see the reasoning that produced it.

That makes it difficult to know:

• Where students became confused.
• Which ideas challenged their assumptions.
• What evidence they considered.
• Whether they reached conclusions independently or relied heavily on AI.

This challenge existed long before generative AI. AI has simply made the hidden parts of learning even harder to observe.

Helping students think critically therefore requires making the thinking process itself more visible.

Social Annotation Makes Thinking Visible

Social annotation offers one way to accomplish this.

Instead of asking students to produce a finished answer immediately, annotation encourages them to slow down and engage with ideas as they read. Students highlight passages, ask questions, connect concepts, respond to classmates, and explain their reasoning directly within the text itself.

Whether students are annotating research articles, PDFs, websites, videos, images, or open educational resources, annotation encourages them to engage with evidence before turning to outside explanations. Explore the many types of content students can annotate.

Rather than evaluating only a finished assignment, instructors gain visibility into the reasoning that develops throughout the learning process.

Students also benefit from seeing multiple interpretations while still contributing ideas of their own.

This creates opportunities to question assumptions, compare perspectives, and develop stronger habits of critical reading before AI enters the conversation.

Designing Courses That Resist Cognitive Surrender

If cognitive surrender occurs when students accept AI-generated answers without reflection, one solution is to redesign learning experiences so that thinking happens before AI enters the process.

Rather than asking students to submit a polished answer immediately, instructors can create assignments that emphasize questioning, interpretation, evidence gathering, and discussion throughout the learning process.

This shifts the focus from generating answers to developing reasoning.

Social annotation naturally supports this approach because students engage with ideas as they encounter them. They explain why a passage matters, ask questions when something is unclear, compare interpretations with classmates, and build understanding step by step. Instead of seeing only the final product, instructors can observe how students arrived there.

These visible thinking processes make it easier to identify misconceptions, support struggling learners, and encourage independent reasoning before students rely on AI-generated responses.

Give Students Time to Think Before They Collaborate

One of the instructional challenges in the AI era is ensuring students develop their own ideas before being influenced by AI or even by classmates.

This is where instructional pacing becomes especially valuable.

With Paced Social Annotation, students first complete a reading and create their own annotations independently. Only after that initial work is complete are they able to view and respond to their classmates’ contributions.

This simple sequencing encourages students to form their own interpretations, identify questions, and evaluate evidence before entering a collaborative discussion. Rather than replacing collaboration, it strengthens it by ensuring every student has an opportunity to think first.

In many ways, this approach aligns closely with the Wharton researchers’ concerns about cognitive surrender. Students are encouraged to rely on their own reasoning before consulting outside perspectives, whether those perspectives come from classmates or AI tools.

AI Should Support Thinking, Not Replace It

Generative AI is likely to become a permanent part of higher education.

The question is no longer whether students will use AI. The more important question is how educators can help students use it thoughtfully.

Used well, AI can help students explore unfamiliar concepts, compare perspectives, revise their writing, and generate new questions. Used uncritically, it can short-circuit the very reasoning processes that education is designed to develop.

The Wharton paper suggests that AI should be viewed not simply as another educational technology, but as a participant in human reasoning. That perspective shifts the instructional challenge from limiting AI use to designing learning experiences that preserve independent thought.

Some instructors are already redesigning assignments to encourage students to engage critically with AI rather than simply accepting its outputs. By combining AI with social annotation, they ask students to question sources, compare interpretations, challenge assumptions, and make their reasoning visible throughout the learning process.

See how faculty are using social annotation to support AI literacy.

Why Human-Centered Learning Matters More Than Ever

The Wharton paper introduces a new way of thinking about AI, but its implications extend beyond technology.

As AI becomes more capable of generating summaries, explanations, and written responses, the uniquely human parts of learning become even more valuable.

Students still need opportunities to wrestle with uncertainty, ask meaningful questions, evaluate evidence, and develop original ideas. Those experiences build judgment, confidence, and intellectual independence in ways that cannot simply be generated by an AI assistant.

Social annotation supports these goals by making learning an active, collaborative process rather than a passive exchange of information. Students engage directly with ideas, see how others interpret the same material, and develop stronger habits of reflection before arriving at conclusions.

In that sense, annotation is more than a reading strategy. It is an instructional approach that helps preserve the human side of learning in an AI-enabled world.

Why This Matters in 2026

Artificial intelligence is rapidly changing how students access information, solve problems, and complete academic work.

The Wharton research suggests that the greatest educational challenge may not be AI itself, but how students learn to think alongside it.

When learning remains invisible, instructors have little opportunity to understand how students arrived at an answer or whether they evaluated AI-generated information critically. When learning becomes visible, educators can support reasoning, identify misconceptions earlier, and help students develop the habits of mind that extend far beyond a single assignment.

Social annotation contributes to this shift by making reading, questioning, discussion, and reflection part of the learning process itself. Instead of simply measuring outcomes, instructors gain insight into the thinking that produces them.

In an era increasingly shaped by AI, helping students think independently may become one of higher education’s most important responsibilities.

Frequently Asked Questions