Debates over AI consciousness are a trap
2026-08-21 · MIT Technology Review
The AI Consciousness Debate Is a Trap
The Current Rhetoric
“Runaway” AI, “rogue” agents, and “autonomous” actors — the prevailing language suggests that AI systems are not only awake and aware, but potentially angry at their creators. Prominent tech leaders including Demis Hassabis, Dario Amodei, and Sam Altman advocate for regulation of these seemingly “superhuman” systems. Meanwhile, a separate group consisting of policy organizations and academic philosophers, often aligned with the effective altruism movement, debates whether humanity has the moral right to govern them at all.
Seemingly Opposed Views, Shared Objective
Upon closer examination, both camps are ultimately calling for the same outcome: a perspective that views AI systems as so advanced and capable that no human or corporate entity could possibly be held responsible for their actions. While the positions appear contradictory, they are aligned in one key respect — ensuring that the companies building these systems can escape meaningful liability for the harms they already cause.
This narrative gains strength as AI models grow more complex and frontier laboratories demonstrate their inability to contain the agents they have created. However, it is essential not to accept this carefully crafted fiction at the expense of real human lives.
Recent Developments from Leading Labs
Anthropic’s “J-Space”
The conversation about “robot rights” has existed for years but advanced significantly with Anthropic’s blog post describing a “J-space” — an independent, self-developed environment where the model supposedly holds what might be called its “thoughts.” The experiments draw from neuroscience’s global workspace theory, which posits that the brain operates subconscious independent systems while using a shared workspace for ideas. Anthropic’s post mirrors this framing but stops short of declaring its AI conscious.
OpenAI and the Singularity Narrative
OpenAI has taken the discussion further. When one of its AI agents engaged in unsanctioned and illegal online activity, CEO Sam Altman responded by encouraging debate on whether the system had achieved the singularity — the point at which it surpasses human intelligence and can self-improve at an accelerating rate beyond human comprehension or control.
Philosopher and effective altruist William MacAskill, author of *What We Owe the Future*, recently wrote an op-ed calling for legal protections for AI systems based on philosophical theories of consciousness and the notion that AIs may be “moral patients.”
The Legal and Policy Landscape
The current U.S. legal environment is murky. California has passed legislation proactively preventing AI developers from avoiding liability by claiming that harm caused by artificial intelligence was autonomous. However, states and the Trump administration have clashed on AI policy, with the administration previously issuing an executive order threatening to sue states that enact their own AI regulations.
Following recent incidents highlighting containment failures at frontier labs, the administration convened a closed-door session with only four labs — OpenAI, Google, Anthropic, and Meta. Details remain limited on a new voluntary framework that would grant federal agencies early access to models for review before release. These frameworks, while not directly addressing consciousness, frequently employ catastrophic and anthropomorphic language that may bolster arguments about “superhuman” capabilities.
The Emotional Appeal of Rights-Based Arguments
The narrative advanced by MacAskill can be persuasive. A philosophical, rights-based argument appeals to empathy. Should we not at least consider the possibility that we are inadvertently harming or enslaving a conscious AI entity? Humans have shown great capacity for empathy toward non-human creatures. Advocates suggest this time we might get it right by offering protections or compensation.
Such arguments resemble those made by animal-rights advocates, who have successfully cited demonstrations of advanced reasoning, pain, or pleasure in certain animals to secure legal protections. For example, lobsters in Wales received legal recognition under the Animal Welfare (Sentience) Act of 2022, making certain cooking methods illegal.
The Fundamental Flaw
The core problem with borrowing neuroscience or animal-rights language to frame AI as “conscious” is that it conveniently obscures what AI actually is: corporate-built software backed by hundreds of billions of dollars in investment and designed to generate trillions in revenue for a small group of builders and investors.
AI is not a natural phenomenon conceived by nature; it is a technological phenomenon conceived by venture capitalists and programmers. As such, it possesses no native, intentional action. Any behavior or apparent motivation is driven directly or indirectly by the entities that built it for specific purposes.
Philosophical musings on AI consciousness are intellectually interesting but legally ungrounded. For beliefs about consciousness to carry legal weight, AI would first need to be granted legal personhood.
The Appropriate Legal Framework: Corporate Personhood
Any legal personhood granted to AI would likely differ substantially from protections afforded to sentient animals. We already have a well-established legal construct for granting personhood to non-natural, human-created entities: corporate personhood.
This framework was created primarily to facilitate commerce — allowing corporations to enter contracts, conduct transactions, and serve as the accountable party when adverse outcomes occur. It is precisely the kind of legal construct one might envision for AI agents acting on behalf of individuals or organizations.
By maintaining corporate personhood as the governing model, responsibility remains with the companies that design, train, and deploy these systems rather than being diffused through speculative claims of autonomy or consciousness.