We are on the dangerous threshold of investing in machines the power to make autonomous life-or-death decisions over humans.
From the perspective of those engaged in modern warfare, lethal autonomous robots (LARs) offer distinct advantages. They have the potential to process information and to act much faster than humans in situations where nanoseconds could make the difference. They also do not act out of fear, revenge or innate cruelty, as humans sometimes do.
A drone still involves a human “in the loop” – someone, somewhere presses the button. This is slowed down by satellite communications (think of the time-lag when foreign correspondents speak on TV) and these communications can be interrupted by the enemy. So why not take the human “out of the loop”, and install an on-board computer that, independently, is able to identify and to trigger deadly force against targets without human intervention?
There are good reasons to be cautious about permitting this.
On a practical level, it is hardly clear that robotic systems can meet the minimum requirements set by the law of war for lethal decision-making. Popular culture, including sci-fi, celebrates the capabilities of robots, but robots are good at what they do only within a narrow range: their sensors give them tunnel-vision information and they are largely wired for quantitative work.
Soldiers in battle may “lawfully target” only combatants, and not civilians. Will a computer be able to make the “value judgment” that a group of people in plain clothing carrying rifles are not enemy combatants but hunters – or soldiers surrendering?
Civilian loss of life as “collateral damage” can be lawful only if it is proportionate to the military objective. [which is typically murder] This is essentially a qualitative judgment, requiring in many cases experience and common sense and an understanding of the larger picture that robots do not have.
It is also not clear who is to be held responsible if things go wrong. Yet it makes little sense to punish a robot.
The increased availability of weapons that place a state’s soldiers out of harm’s way may make it easier for those states to go to war, and lead to ongoing and global (if low-intensity) warfare – as well as targeted killings. This may have far-reaching implications for the international security system that has saved the last three generations from the scourge of global war.
The overriding question of principle, however, is whether machines should be permitted to decide whether human beings live or die.
Human beings are frail, flawed and, indeed, can be “inhumane”; but they also have the potential to rise above the minimum “legal standards for killing.” By definition, robots can never act in a humane way. If human beings are taken out of the loop, so are not only the shortcomings of humans, but also our redeeming features.
Robots may, in some respects, not be predictable enough to be used in war: even technicians will not know exactly what to expect from machines that make their own choices, and the average commander in the field who deploys them will be even more at a loss. In other respects, LARs may be too predictable: treating everyone according to the same algorithms means brushing aside the uniqueness of each individual.
But the situation is complex. While LARs pose a clear threat in some cases, there is also the argument that under certain circumstances, using robots may, in fact, save lives. For example, human soldiers who detect movement may fire, afraid it is a sign of enemy soldiers, when, in reality, their “target” may be civilians in hiding. A robotic soldier, which does not fear for its life, may be deployed to go closer and to investigate. Likewise, robots in some cases could more precisely target their fire.
The problem is that even if this is correct, it is not clear that the current laws of war, and the levels of capacity of the soldiers in the field, are sufficient to confine the use of LARs to those situations where they can possibly save lives. But more importantly, does it not demean the value of the lives of each one of us to know that it has become part of the human condition that we could potentially become collateral damage in the calculations of a machine?
This calls for a cool assessment. On the one hand, there is the danger that we overestimate the abilities of computers – because they beat us at chess and maths, we may defer to them regarding decisions that they are not equipped to take. On the other hand, we should not be closed to investigating situations where they can possibly serve to preserve life.
To some extent, we have already given some control to machines over individual targeting decisions with various long-distance weapons. But there is an important, if imperceptible line that we should not cross: humanity should not surrender meaningful control over questions of life and death to machines.
For these reasons, I have called on the United Nations to promote a moratorium on these weapons, and to appoint a high-level panel to advise on whether LARs could be deployed in compliance with relevant international law and, if so, under which circumstances.
UK foreign minister Alistair Burt gave the assurance during a debate on the issue in the House of Commons on 17 June that the UK was not developing such weapons, and had no current plans to do so. The United States took a further step in the right direction when the Department of Defense in November 2012 formalized their position and issued a directive that commanders and operators shall retain “appropriate levels of human judgment over the use of force”. These initiatives should be consolidated and other states should be encouraged to follow the same route.
War without ongoing reflection on the human cost is mechanical slaughter. The current prospect of entering a world where machines are explicitly mandated to kill humans should give pause to all of us. While technology rushes forward, we need to take some time out to ensure that not only lives, but also a concept of the value of human life, are preserved in the long term.
There was a reason why The Terminator caught the public imagination when it was first released. Like all Sci-Fi works, it tapped into a subliminal awareness that there were and are forces in the science and business communities that readily align with the militarys’ worst impulses. It’s in the confluence of scientific inquiry and invention divorced from ethics, Big Business’s greed, and military ambition devoid of morality that nightmares are born.’
- “Robot warriors” to fight on real battlefields (theguardian.com)
- Slow and Steady Progress Toward Lethal Autonomous Robots (cryptogon.com)
- Robowar: The next generation of warfare revealed – a general’s dream, but are they also humanity’s nightmare? (independent.co.uk)
- Professor Warns of Robots Armed With Tasers (thefreedomagenda.wordpress.com)
- Singer and Kurzweil (twebb02.wordpress.com)