I haven’t really written about machine consciousness, super-intelligence, or artificial general intelligence (AGI) before; I never felt the need, and some of the questions on AI consciousness are entirely outside of my area of expertise. I have always assumed that some sort of AI consciousness would be achieved in the far future, but not within my lifetime, but I had nothing to base this opinion other than being an avid consumer of science fiction. I’ve also been generally sceptical of any singularity claims, which I used to regard as a modern day secular cult.
I can’t claim any authority on AI consciousness, but I do think there is a non-zero possibility that we will see some kind of significant leap in AI capabilities in the next decade. Whether that leap produces something we would recognise as AGI, super-intelligence, or AI consciousness is genuinely debatable, but the potential seems sufficient to start thinking through the implications for our lives, even if most of what follows is wild conjecture. So this post takes no stance on whether AI consciousness is possible, or even desirable. It is a “what if” exercise, not unlike asking what you would do during a zombie apocalypse, or how you would react if aliens landed tomorrow.
Before going any further, it is worth clarifying what these terms actually mean, because they tend to get used interchangeably in ways that would make a computer scientist’s eye twitch. AI consciousness refers to the idea that an artificial system might have subjective experience, that there is, as philosophers put it, “something it is like” to be that system. AGI, or artificial general intelligence, is a rather different concept: it describes an AI capable of performing any intellectual task that a human can, without being specifically designed for each one. Super-intelligence goes further still, and refers to a system that surpasses human cognitive ability across essentially all domains, in other words, the AI does not only match us, but leaves us behind. The three are related but distinct, and it is entirely possible to imagine an AGI that is not conscious, or a conscious system that is not particularly intelligent by human standards. For the purposes of this post, I am mostly agnostic about which of these we might eventually see; what matters is that the possibility space is large enough to be worth taking seriously. So I am going to assume for the sake of conjecture that systems become super-intelligent, but not necessarily conscious.
Who owns the AI?
This is probably going to be one of the easiest things to determine. In principle the ownership of super-intelligent AI would be the company which trained the model, but there could be some other interesting questions regarding public ownership.
It is worth splitting “the AI” into three things that get owned differently: the model weights, the outputs, and the notion of the “intelligence” itself. The weights are protected mostly as a trade secret with some thin copyright in the surrounding code, the outputs are the authorship question below, and nobody really owns “intelligence” in the abstract. So the ownership question is less exotic than it first appears.
Public ownership however would be an interesting discussion. Bernie Sanders is on record making the argument for some sort of public ownership of AI, the argument is that because AI has been trained on people’s content, it should belong to the people. If super-intelligence is achieved, then these public-ownership arguments stop being fringe the moment a single system can do most cognitive labour. At that point ownership is a question of political economy rather than IP, and the open-versus-closed weights debate stops being academic and becomes the whole point of the debate.
And needless to say, any sort of self-aware system could lead us to more outlandish concepts, such as AI rights and self-ownership, but again I am not making assumptions in that direction, but they could be interesting to consider. A super-intelligent but non-conscious system cannot own itself, because property cannot hold property. But there is legal precedent of course, we already grant legal personhood to corporations, which are legal fictions, so there is nothing conceptually impossible about extending it. We have simply chosen not to.
Authorship
The AI authorship question would likely need to be revisited as super-intelligence could lead to more works than ever being AI generated with little way of finding out whether they were AI generated.
The key point here is that super-intelligence does not break the doctrine that only humans can generate works worthy of copyright protection, but it makes the enforcement of such doctrine practically impossible. Thaler v Perlmutter and the USCO’s line hold the human-authorship requirement steady regardless of how capable the system gets. What changes is whether it would be possible to discover such content. Registration, and the public-domain status of AI works, both depend on being able to tell, and once you cannot, the requirement quietly becomes an honour system. I have already explored this line of thought in previous articles, but the fact is that there is no copyright police, and human authorship will require that people disclose whether AI was used in the production of a work. My own stance has been that we will just have to eventually consider all works as worthy of protection as we will not be able to tell. That, or get rid of copyright altogether.
I can hear the screams from copyright holders as I write this. I’m not advocating for an end to copyright, just speculating, put away your torches.
If we continue with the line that no AI work has copyright, even one produced by a super-intelligent system, then we could be faced with the largest accidental contributor to the public domain in history. Every undisclosed AI work that should be unprotected but gets claimed as human pollutes the corpus of copyright works, while every disclosed one drops straight into the Commons. There is a nice irony in the most powerful private technology yet built expanding the commons by default. The other thing to note is that the more capable the system, the thinner any human-authorship claim becomes: the prompt-as-authorship argument gets harder to sustain when the human contribution shrinks to a wish.
Courts will not solve this conundrum easily, we may need legislation. Oh wait, s9(3) CDPA is about to be repealed, but I digress…
Infringement
Super-intelligence is likely to be extremely profitable, which could lead to a couple of interesting legal issues regarding the ongoing copyright litigation against AI companies. On the one hand, large profits always attract legal action, and the larger the prize, the larger the target. On the other hand, super-intelligence could lead to most cases being dropped.
Why do I say that most cases could be dropped? The mechanism is settlement economics. If the prize is large enough, licensing and settlement become rounding errors, we could see the current wave of licensing deals simply accelerating, and litigation turning into a negotiating tactic rather than an existential threat. Plaintiffs will likely take the cheque presented with a choice, much as what happened in Bartz v Anthropic.
But as I mentioned, if AI developers hit the jackpot of super-intelligence, they could also become bigger targets with more incentives to file class actions which could scale with larger profits, so the deeper the pockets the more attractive the suit, and discovery into training data becomes the real battleground rather than the merits.
There is another interesting issue regarding infringement, where super-intelligence is a sword and shield at once. The same system can detect infringement at scale, which is a gift to rights holders, and launder it at scale through clean-room re-derivation and effectively infinite non-literal variation. If a system can generate unlimited variations that clear the substantial-similarity bar, the scarcity copyright exists to protect starts to dissolve, and that is a deeper problem than any single lawsuit.
It is also worth revisiting the jurisdiction question here. A system trained on everything has every incentive to train where it is lawful and deploy everywhere. And the pace mismatch is real, cases such as Getty v Stability AI take years, while the capability jump of a super-intelligence system could take months, or even weeks for systems that start to train themselves. In that eventuality, copyright infringement may just be impossible.
Concluding
I have no idea if any of the above scenarios are even viable, it may just happen that the AI bubble will burst, that AI development will hit a wall, or that we engage in a version of the Butlerian Jihad and destroy every machine before it destroys us. Those are also fun possibilities in their own right, and may be just as likely as the super-intelligence I describe here.
And then again we may be presented with the doomer scenario: the super-intelligent computer is evil and goes on to destroy us. If that happens, copyright will be the least of our worries. Unless John Connor turns out to be a copyright lawyer… James Cameron, call me, I have an idea for the next Terminator.
I’ll be back.
0 Comments