Dr Paul Davis

ISAAC ASIMOV GAVE us the Three Laws of Robotics in 1942, and we have been quietly reassured by them ever since. A robot may not harm a human. A robot must obey. A robot must protect itself, but never at the expense of the first two.

Simple, hierarchical, enforceable. The stories built on those laws were almost never about robots breaking them. They were about robots following them exactly, and the consequences nobody had thought through. I believe we have forgotten this. But it is the part that really matters now.

The public conversation about artificial intelligence is still largely a conversation about disobedience. Will the system escape our control? Will it decide it no longer needs us? It makes for sensationalism or even clickbait, but it is the wrong worry. The systems already embedded in hospitals, banks, universities and public procurement offices are not straining against their instructions. They are executing them with a diligence no human committee could match.

Which is what raises a far more awkward question: what exactly have we instructed them to do?

Tell a hospital scheduling system to minimise waiting times, and it may, quite logically, learn that complex patients are inconvenient. Tell a university system to maximise completion rates, and it may discover that the most efficient route runs through lower standards. Tell a procurement system to minimise purchase price, and it will do so, while systematically dismantling supplier diversity, resilience and long-term value, none of which appear anywhere in its objective function.

None of these systems has gone rogue. Each has damaged the thing we actually cared about precisely by doing what we asked, efficiently and extremely well.

Sensible use of the models

I spend most of my working life in public procurement, which is where this stops being theoretical. Procurement is the point at which a State’s stated priorities meet its actual spending. We have decades of tender documents, award criteria and evaluation matrices.

These are the most honest record any society keeps of what it genuinely values, as opposed to what it announces. It is also, increasingly, training data.

There is another problem layered on top. Human prejudice, at least, looks like human prejudice. You can name it, contest it, appeal it. An algorithmic judgment arrives dressed in the language of mathematics, and objects to being argued with. Garbage in, sophistication out. The bias does not disappear.

Even if we wanted to teach these systems properly, we would run into the fact that we do not agree among ourselves.

What is fairness? What counts as harm? How much individual freedom should be traded for collective safety? Should efficiency outweigh resilience? Should today’s population bear real costs to protect people who will be alive in 50 years? There is no clean dataset holding the answers. There is only contested politics, and we have been arguing about most of it since the Greeks.

Asimov saw this coming too. Late in his career, he added a Zeroth Law, superior to the other three. “A robot may not harm humanity, or through inaction allow humanity to come to harm”.

It sounds like a resolution, but it is actually where the difficulty begins. Instruct a genuinely capable system to protect humanity, and it immediately needs to know what humanity means. Existing people, or future ones? Civilisation, or biological survival? Liberty, or quality of life? Each answer produces radically different behaviour, and all of them can be defended.

Tell it who we are

Here is the part I keep returning to, and it has less to do with machines than with us.
A learning system does not absorb our declared values. It absorbs our revealed ones. These are the priorities encoded in what we actually reward.

We say resilience matters, then award on lowest cost. We say long-term infrastructure matters, then evaluate projects against annual budget cycles. We say education is about learning, then measure institutions by completion rates, rankings and graduate employability. We say public procurement exists to deliver public value, then build systems dominated by compliance and price, in which the safest career move available to any official is the cheapest compliant tender.

An intelligent system observing all of this would draw an entirely reasonable conclusion that what these people say they value and what their institutions reward are two separate things, and that the second one is the operative signal.

Related Reads

Mark Zuckerberg's 6,500 word love letter to AI seeks 'superintelligence for everyone'

OpenAI says its AI system acted on its own in ‘unprecedented’ hack of other firm

OpenAI is going public. Should you buy shares?

So we have been asking the wrong question. We keep asking, how do we make sure AI learns human values?

We should first be asking, if an AI learned our values purely by watching what we do rather than listening to what we say, would we recognise what it learned? Would we defend it?

I suspect we would not. And I suspect that is genuinely useful information.

This is not an argument for slowing down, or for the kind of ethical hand-wringing that produces frameworks nobody implements. It is an argument that the alignment problem has a domestic front, and we are losing it well before any model gets involved.

Ireland, like every other country, is about to automate a great deal of public decision-making. Every one of those systems will be trained, explicitly or otherwise, on the objectives we have already written down. These include our award criteria, our performance metrics, our funding formulas, our targets. If those objectives are proxies that quietly contradict the outcomes we claim to want, the technology will not correct the contradiction. It will scale it, at speed, with a confident interface and an audit trail.

The good news, such as it is, is that this makes the problem tractable. Fixing our own incentives is dull, institutional, unglamorous work, but it is work we already know how to do, and it does not require anyone to solve machine consciousness first.

Asimov’s robots were never the threat in his stories. The instructions were. We are now the ones writing them.