Human must not surrender cognition to AI

There is a revealing moment in the maybe-apocalyptic 2023 film Leave the World Behind when Ethan Hawke's character, George, finds himself driving around in unfamiliar territory after getting in his car to find supplies.

Not only that, but something happens that renders all of his car’s and phone’s technology useless; everything he has taken for granted is no longer particularly or practically useful. 

He has no idea how to get back to his wife and friends at a place he had no trouble finding when his tech worked, but there is something more profound going on. George has not forgotten how to drive; he has forgotten how to navigate without the machinery doing most of the thinking for him.

Cognitive control is crucial for human growth

Cognitive control helps humans adapt to ever-changing environmental demands and getting lost is part of why humans evolve. If this control is ceded to apps such as Google Maps and others, then drivers such as George are the first to succumb… and stuck in his personal vacuum.

And, while nature abhors a vacuum, AI adores it. So, this vignette feels increasingly relevant as AI moves from being a useful tool to becoming something rather more consequential.

It is already a cognitive co-pilot that is gradually being invited to make decisions, construct arguments, assess evidence and even tell us what we should think.

The argument here is not that AI is bad. It can be extraordinary, transformative technology and anyone who has spent ten minutes using a good model understands its capacity to accelerate research, writing, coding, analysis and problem-solving.

However, as JB Benjamin, founder of post-quantum sovereign data platform Veldris, points out there are ramifications, not only for human cognition but also for human data and sovereignty.

"One of the erroneous arguments perpetrated is that AI is going to replace humanity. It will not and can not replace us. But it can be the ultimate force multiplier and an agent of good, which enhances and expands all of our skills and capabilities.

"The danger comes in not just our growing over reliance on AI as a replacement for our own general intelligence, but also in giving up our provenance and sovereignty to Third Party cloud intelligence providers who leverage our data against us, for their own financial and political gains. 

Cognitive surrender

The potential problem coalesces when acceleration becomes substitution, something increasingly referred to as cognitive surrender, a phase coined in 2206 by Steven Shaw, a postdoctoral researcher and Wharton professor Gideon Nave, an engineer with a doctorate in neural systems at the University of Pennsylvania’s Wharton School of Business.

They are concerned that such human submission will rot away internal intuition, cognition and self-analysis, leaving lazy humans as useless as Ethan Hawke’s George when the metaphorical lights go out.

Brain rot is another phrase that was coined in recent years that initially (and light-heartedly) described lazy humans scrolling on social media, but is now becoming something more real and dangerous.

Recent research from China follows earlier warnings that such scrolling suppresses cognition function and once that process begins, it is difficult for those neurochemical changes to be addressed.  

Moreover, there is a crucial distinction between using AI to make humans faster and using it to make humans think. That distinction sounds obvious, but such thinking is disappearing from everyday workflows.

Asking AI to challenge human thinking and humans may become better at thinking. Ask it what humans should think before they formed an opinion and humans have already surrendered the most important part of the process. Total cognitive surrender.

The danger is particularly acute because AI rarely announces that it is making us cognitively weaker. It is usually charming, confident and fluent. Its answers arrive without hesitation and are formated beautifully.

The absence of friction can therefore disguise the absence of thought. Nathaniel Bradley, CEO of Nasdaq-listed AI company Datavault AI, puts it thus:

“The consequences lie with us when using systems more intelligent than us,  even in groups of humans we can no longer keep up with the intelligence of AI.  

Subordinate AI systems that self check reference and analyse reference quality provides a purpose of life and companies such as ours show much we can trust and what the output of AI is worth.”

Professor Stoyan Stoyanov of Heriot-Watt University describes the problem as a substitution that can turn from capability into disability.

"Handing a task to AI is a gain while the human can still perform that task independently. Once the underlying judgement begins to atrophy, however, the same substitution becomes a liability."

AI in the workplace

This matters enormously in the workplace because some of the tasks AI is most capable of eliminating are precisely the tasks through which humans learn. That is why the issue is not simply individual laziness. It is also organisational design.

Mücahit Kaya, an AI industry researcher and founder of AI Tools Police, makes the point that institutions can surrender cognition faster than individuals

“When a company routes decisions through a model, the meeting, disagreement and written reasoning that previously accompanied the decision can quietly disappear."

There is an irony here. We have spent decades building technology to remove humans from repetitive tasks, only to discover that some repetitive tasks were quietly building the humans we would later need. AI should research, challenge, compare, synthesise and accelerate.

Humans should retain the responsibility for deciding what matters, what is true, what is appropriate and what should happen next. That means designing workflows where people sometimes have to form an initial view before consulting AI.

It means asking employees to explain why they accepted or rejected a recommendation. It means evaluating the reasoning behind an answer rather than merely admiring its polish.

Because the ultimate nightmare is not an AI that becomes smarter than us. It is an AI that becomes so convenient that we gradually stop becoming smarter ourselves. And it’s, literally, happening on our watch.

George, lost on the road in Leave the World Behind, eventually discovers what happens when the systems around us stop working. The uncomfortable question for the AI age is whether we will still know how to find our way home when they do… and whether we really care.

Image: ChatGPT