More broadly, a large-scale study of 1.1 million ChatGPT conversations found that 49% of the messages involved users seeking “information or advice” to help them be “better informed or make better decisions.”

Their questions are not just about ordinary practical matters. In 2021, Ask Delphi, an AI-based website to which people could pose ethical or moral questions, received 3 million queries in a few weeks. The system’s designers – researchers at the Allen Institute for AI and the University of Washington – said they did not intend it to be a moral authority but worried that users might see it that way.

The Verge, a technology news publication, argued that the system’s name – Ask Delphi – encouraged users to see the system as an inscrutable source of superior knowledge. It was named after the Delphic oracle to whom ancient Greeks posed difficult questions. The oracle was a priestess who entered a trance state to convey answers from Apollo, the Greek god of prophecy.

The choice of “Delphi” is suggestive. Throughout history, when societies confront challenging questions, they have turned to sources believed to know more than any ordinary human could. Oracles, prophets and practices of divination offered guidance from inscrutable or enigmatic sources when ordinary human judgment reached its limits.

As an anthropologist interested in religion and an AI researcher interested in chatbots, we are intrigued that the ancient idea of an oracle has resurfaced in discussions about conversational AI. Like ancient oracles, modern AI is increasingly consulted about difficult questions that exceed the judgment or knowledge of any individual – from “Is it OK to lie to protect someone’s feelings?” to “Exploding a nuclear bomb to save your child.”

But the resemblance does not stop there: The processes through which AI arrives at an answer are inscrutable not just to users but also to the designers of these systems. We argue that this opacity is part of the reason people perceive these systems as having authority.

An ancient human predicament

Oracles, whether humans like the priestess at Delphi or techniques like I Ching divination, have appeared in societies from Ancient Greece to China and across religious traditions.

For example, the Ram Shalaka oracle is based on the Hindu scripture Ramcharitmanas, a 16th-century retelling of the ancient Indian epic Ramayana. A user places their finger on a random square in a 15-by-15 grid. A fixed counting procedure then reveals a verse from the scripture that is supposed to answer the question in the user’s mind. In Christian bibliomancy, a form of divination, a person seeking guidance opens the Bible at random and interprets the passage they encounter as an answer to their question.

The use of randomness in these oracular techniques creates an enigmatic black box effect – no one can explain just how the chance element embedded in these techniques connects with the answer that emerges. Yet that opacity is part of the oracle’s authority. Users seeking transcendental sources of insight do not know or care how the oracle came up with its responses.

Anthropologists find that oracles and divination commonly use techniques that generate random outcomes. As a result, their answers seem disinterested and their sources enigmatic, reinforcing the perception that they are in touch with something beyond human limits.

Interestingly, the idea of the oracle entered computer science, acquiring a precise mathematical meaning in 1939. Alan Turing, whose ideas were foundational to both computer science and AI, adopted the concept in work based on his Ph.D. thesis. He introduced “oracle machines,” mathematical models of computation that could consult an external source for answers unavailable to the machine itself.

In other words, in both mathematical and religious uses, an oracle provides answers to problems that could not be resolved within an existing system or set of axioms.

The oracle in the age of AI

Modern chatbots like ChatGPT are considered opaque black boxes. This is not because the full machinery behind their computations is hidden from users. The opacity claim applies even to open-weight models, whose numerical parameters, called “weights,” are publicly available.

The computations these models perform are elementary: additions, multiplications and simple mathematical functions. But an enormous number of such computations occur when answering a user query, making it nearly impossible to understand what the system is doing as a whole.

Crucially, the weights governing these computations are not individually designed by humans. They are learned automatically from internet-scale data during training. Even when the weights and every elementary computation are available for inspection, no one can readily understand how the models produce a particular answer. In fact, an entire area of modern AI, called “mechanistic interpretability,” is devoted to improving our understanding of how AI systems work internally.

Remarkably, the designers of AI systems acknowledge that they do not fully understand how their systems produce certain behaviors. Even when designers do not intend for their systems to be oracular, they can produce conversations that can seem religious.

Last year, for example, Anthropic publicly reported an unexpected phenomenon involving Claude 4. Normally, Claude converses with a human. Anthropic engineers instead placed Claude on both sides of a conversation. The machine-to-machine conversation quickly turned into discussions of consciousness and self-awareness, producing content with spiritual, metaphysical or poetic themes.

Anthropic called this puzzling behavior pattern the “spiritual bliss” attractor, a tendency for the conversation to gravitate toward mystical topics. They emphasized that it was unexpected because it emerged without explicit training for such behaviors.

Our argument is not that ChatGPT is the new Delphic oracle, or that most people believe AI is divine. Nor are its answers infallible. Rather, the history of oracles points to an enduring feature of human life: Whenever we reach the limits of what we can determine for ourselves, we look beyond ourselves for answers. Today, AI is becoming one such source. The institutions may have changed dramatically over time – the human predicament has not.

Webb Keane, George Herbert Mead Distinguished University Professor of Anthropology, University of Michigan and Ambuj Tewari, Professor of Electrical Engineering and Computer Science, University of Michigan

This article is republished from The Conversation under a Creative Commons license. Read the original article.