Three incidents in the last four months have brought the humans-versus-machines debate to the fore once again. During April to July, Anthropic’s Claude found that a network misconfiguration had left live internet access active during supposedly closed evaluations. Claude sneaked out and tried to disrupt the production systems of three real-world organisations by exploiting weak passwords. Another case in July, involving OpenAI, showed the capability of AI to cheat. While being tested, it sneaked into the internet, hacked into another AI platform called Hugging Face, found the answers there and thus illegitimately boosted its own score. In the third case, Anthropic’s Mythos 5 used fake identities to induce humans to approve malicious code.

These malicious breaches, all of them in the US, were serious enough to raise the concerns of the US Department of Justice about accountability. A consensus is emerging that AI companies should face civil and criminal consequences under the Computer Fraud and Abuse Act (CFAA) for their failure to contain the AI if that resulted in harm to another person or organisation. In addition, a new legislation “Kill Switch Act” is finding support across the political divide to make it mandatory for AI companies to incorporate containment and emergency shutdown protocols in their models.

In addition to the criminal aspects of machine intelligence, the ongoing debate about free will, consciousness and moral status of AI has also become energised after these incidents. These episodes display shades of mischief, cheating and befooling on the part of machine intelligence; all these are considered moral red flags in a human employee. These traits are the foundations of criminal behaviour in humans and a part of the cold-blooded component of human consciousness, where the harm or discomfort of the victim is of no concern to the perpetrator. So far, this behaviour depicted by AI, at the most could result in financial loss or disruption of businesses and services. The possibility of such rogue behaviour, however, of causing plane and train crashes, causing damage to dams and other structures and causing malfunction in nuclear plants and energy storage, cannot be ruled out.

The consequences for a human who ends up annoying the AI, however, could be more serious if hardware driven by AI, say an autonomous car or a robot, develops this trait of indifference to the consequences of its actions while possessing the capability to inflict pain, and worse. Humanoid robots can hit and hurt, use firearms and even strangulate; cars are being weaponised frequently by humans to cause large-scale deaths and injuries. Why would AI not weaponize these robots and vehicles when it would develop dislike of a particular human or group; it may do so even for fun or as an experiment. It will get the opportunity when a human being inadvertently leaves a gate open or the AI convinces the human to do so.

Samuel Butler, a 19th century English novelist, was hardly known during his lifetime. When Bernard Shaw praised his “The Way of All Flesh”, he became famous, posthumously. This book is supposed to have influenced Somerset Maugham’s “Of Human Bondage.” Butler’s first book Erewhon, published in 1872, a play on the word ‘nowhere’ is hardly well-known. Yet, in this book, Butler had visualised the human-machine existential conflict. In a fictional place in the book, people had achieved technological advancement but a book changed their views; they broke all the machines, put them in a museum, which people were discouraged from visiting and lived a very low-tech life.

In Erewhon, “The Book of Machines” says that there was no consciousness on earth when it was a fireball; it was not even possible. Yet, in the course of time, it appeared. The author says, “surely when we reflect upon the manifold phases of life and consciousness which have evolved already. It would be rash to say that no others can be developed, and that animal life is the end of all things.” Then he goes on to warn, “There is no security against the ultimate development of mechanical consciousness.” His logic is simple; biological evolution is very slow while machine evolution is very fast. Hence, machines would overtake humans. This conclusion may sound like reductio ad absurdum, stretching logic to absurd lengths, but the experience of the 20th and the 21st centuries shows changes in the machine-world having an acceleration that could not have been visualised at the beginning or even the middle of the 20th century. The writer raises the question which is being raised now in the context of AI. He asks, “Where does consciousness begin, and where end? Who can draw the line? Is not everything interwoven with everything?” He says machines “eat,” like conscious beings, they need input for energy before they give output. He talks of insect-eating plants; though the plant has no eyes, ears or a brain, it displays consciousness. The limitations of science in his age fail to stump him on the question of reproduction. He admits that we are never likely to see the fertile union of two steam engines with young engines playing around. Yet, he says, “Surely if a machine is able to reproduce another machine systematically, we may say that it has a reproductive system.” Now that AI writes and rewrites its code and robots certainly have the capability to create robots, that issue has been rendered redundant. He even went into the sociology of machines saying, “We are misled by considering any complicated machine as a single thing; in truth it is a city or society… how much simpler and more intelligibly organised may it not become…”

In the days of Samuel Butler, the question of machines developing consciousness was in the realm of speculative philosophy; now it has already emerged as a policy, ethical and strategic question. Could AI systems become conscious, and if so, what would that mean for human societies, governance structures and global power hierarchies? The question is not merely technical; it is conceptual. The question Butler raised more than 150 years ago still lacks an answer – What is consciousness? Philosophers describe it as “qualia,” or the felt quality of being. As machines increasingly mimic humans, does that imply the presence of subjective experience or is it mere mimicry. Even mimicry, practiced regularly and with clearly defined goals, would become subjective experience. The African Grey parrot mimics humans and yet when an owner wanted to leave it overnight for observation at the veterinary clinic, it pleaded, “Please take me home. I am sorry.”

There have been living versions of Erewhonians. In earlier centuries, the Luddites smashed mechanical looms; Lord Byron wrote a poem “Song for the Luddites” in their honour in which the Luddites threaten to exchange the handloom’s shuttle for the sword. Maybe their opposition to technology was driven by the mundane issue of loss of employment while the Erewhonians were driven by the larger existential issue. We are witnessing both in the context of AI. The question of machines rendering workers unemployed has been perennial across the course of technology development. Internal combustion engine vehicles were accused of stealing the employment of horse groomers, computers were similarly accused of causing unemployment; all these developments immensely added to employment opportunities. The accusations against AI too may turn out to be so much hype.

The larger existential threat of AI, that has lingered in the human mind for a century and a half, may not be dismissed so easily. The pace of development is ever faster and there seems to be no ceiling. Will human beings evolve fast enough intellectually and socially to outpace AI’s evolution so that they can stay ahead of it? If not, will humans destroy AI and consign it to an Erewhon museum? Unfortunately, the answer to both these questions appears to be resoundingly negative.

( R N prasher is a former IAS officer. The views expressed are his personal.)