Should AI Be Allowed to Kill?
For decades, the question of machine-authorised killing has remained purely speculative. Now it does not. A March 2020 UN report on Libya recorded what may be the first autonomous weapon attack on humans. Government of National Accord-aligned forces deployed Turkish-made STM Kargu-2 loitering munitions against retreating Haftar-aligned soldiers. The drones reportedly used onboard machine-learning object classification to identify, pursue, and engage targets without direct operator control.
The report described the system as a “fire, forget, and find” capability requiring no operator-to-munition data link. Analysts still dispute whether the drones caused casualties while operating autonomously. Nevertheless, the incident marked a watershed in modern warfare. Militaries could now, in principle, remove humans from the final decision to kill. This is no longer an academic debate. It is an open question in international law, defense procurement, and battlefield practice, and the technical and legal regimes designed to regulate it are still catching up.
Defining the Threshold
The International Committee of the Red Cross defines an autonomous weapon system as one that has autonomy over its “critical functions”, which means it can search, detect, identify, track, select and attack targets without human intervention. This is unlike remotely piloted systems, such as the MQ-9 Reaper, where a human still selects and authorises each engagement. The idea of “meaningful human control” — effective oversight by humans and the ability to intervene or shut down a system in real time — is now at the heart of UN negotiations precisely because no state can agree on how much latency, discretion, or algorithmic judgement is compatible with meaningful human control.
Case Study: Gaza and the “Rubber Stamp” Problem
The most significant real-world test of human oversight did not involve a fully autonomous drone. Instead, it involved AI decision-support systems that nominally kept a human “in the loop.” According to +972 Magazine and Local Call, Israeli intelligence officers described the IDF’s Lavender system. It used mass surveillance and pattern analysis to identify suspected Hamas and Palestinian Islamic Jihad operatives. The system reportedly generated at least 37,000 target recommendations during the war’s first six weeks. Officers reportedly reviewed each recommendation in as little as 20 seconds before authorising it. Sources said this review often became a formality rather than meaningful human oversight. They claimed officers treated the system’s output as though it were a human decision.
Lavender’s own Army review found 90 per cent of his classifications were correct, which means 1 in 10 people flagged did not have a verified militant association – a known error rate agreed to in advance, in the name of speed in processing. Other accounts claim that commanders were willing to sacrifice 15 or 20 civilians to kill a low-level operative. A companion system, “Where’s Daddy?”, tracked targets to their family homes prior to a strike. It is an important case for policymakers. The human reviewer here is nominal, and thus there is no real control. The review cannot be a rubber-stamp exercise given the institutional pressures involved.
Case Study: Ukraine and the Proliferation Curve
Ukraine is the world’s largest real-life laboratory for AI-enabled targeting at scale. The number of global drone-strike events rose from around 364 in 2018 to over 42,000 in 2025, with the Russia-Ukraine war accounting for about four-fifths of these global events. Ukraine’s arsenals do not include fully autonomous systems, but officials, including Deputy Defense Minister Yuriy Myronenko, have said that autonomy has been “partially implemented” in terminal-guidance modules such as the domestically produced TFL-1 — a roughly $150 add-on that enables lock-on-target navigation and continued strike capability even if Russian jamming cuts the operator’s control link.
Russia’s defense minister said drones killed or wounded more than 240,000 Russian soldiers in 2025 alone, and by March 2026 drones accounted for 96 per cent of Russia’s reported battlefield casualties that month. And this growth is not limited to state militaries alone: According to ACLED data, non-state armed groups deploying drone weaponry increased from 10 in 2010 to 469 in 17 countries in 2025. That means the possibility of autonomous targeting software spreading among actors who are completely outside any accountability framework.
The Regulatory Landscape Is Losing the Race
The U.S. Department of Defense’s Directive 3000.09 was last updated in January 2023 and requires a review by a senior official before autonomous or semi-autonomous weapons can be deployed, although that review can be skipped in cases of “urgent military need”. The directive only affects the DoD, not the CIA or other agencies. The UN is moving toward restraint: 156 countries backed a General Assembly resolution on autonomous weapons in November 2025, and Secretary-General António Guterres and ICRC President Mirjana Spoljaric have called for a binding treaty by 2026 to ban systems that operate without human control.
Some 120 to 127 countries now support such a treaty. The Convention on Certain Conventional Weapons requires consensus, while the United States, Russia, Israel and the United Kingdom continue to favour non-binding guidelines over prohibition. Their position has effectively deadlocked the forum. Meanwhile, defense-market analysts expect the global military AI market to grow from about $9.8–$10 billion in 2024–2025 to between $19 billion and $42 billion by the early to mid-2030s.
The Argument for Machine Judgment
The strongest case for autonomy has been made by roboticist Ronald Arkin, who points out that human soldiers regularly commit atrocities and war crimes under combat stress and argues that a well-engineered “ethical governor” – a software programme encoding rules of engagement and international humanitarian law – could in principle do better than fatigued, frightened or vengeful humans, reducing net civilian harm. Philosophers such as Robert Sparrow and Ryan Tonkens argue that no algorithm yet reliably assesses surrender, proportionality, or context as well as a flawed human can. They also contend that when humans are removed, an accountability vacuum arises: no combatant, commander, or programmer can be straightforwardly held responsible when a machine kills wrongly.
An Unresolved Question
Gaza and Libya show that the question is not whether fully autonomous killing is coming, but whether “human control” means anything in a world where institutional incentives favour speed over scrutiny. This is one of the most important questions of international law today. There is no diplomatic consensus, and a deadline for a UN treaty is looming.
References
- United Nations Security Council — Final Report of the Panel of Experts on Libya, S/2021/229. Documents the reported deployment of Kargu-2 loitering munitions against retreating forces in Libya.
- International Committee of the Red Cross — ICRC Position on Autonomous Weapon Systems. Defines autonomous weapons and recommends legally binding restrictions on unpredictable systems and attacks against humans.
- The Guardian — “The Machine Did It Coldly”: Israel Used AI to Identify 37,000 Hamas Targets. The Lavender allegations, with limited human review, claimed 90% accuracy and civilian-casualty thresholds.
- U.S. Department of Defense — Directive 3000.09: Autonomy in Weapon Systems. Establishes American policy and reviews requirements and responsibilities for autonomous and semi-autonomous weapons.
About the Author
