Lords Committee on AI in Weapons Systems: AI harms, humans vs computers, and unethical Russians

Peter Burt
First evidence session. Click to watch video

A special investigation set up by the House of Lords is now taking evidence on the development, use and regulation of artificial intelligence (AI) in weapon systems.  Chaired by crossbench peer Lord Lisvane, a former Clerk of the House of  Commons, a stand-alone Select Committee is considering the utility and risks arising from military uses of AI.

The committee is seeking written evidence from members of the public and interested parties, and recently conducted the first of its oral evidence sessions.  Three specialists in international law, Noam Lubell of the University of Essex, Georgia Hinds, Legal Advisor at International Committee of the Red Cross (ICRC), and Daragh Murray of Queen Mary University of London, answered a variety of questions about whether autonomous weapon systems might be able to comply with international law and how they could be controlled at the international level.

One of the more interesting issues raised during the discussion was the point that, regardless of military uses, AI has the potential to wreak a broad range of harms across society, and there is a need to address this concern rather than racing on blindly with the development and roll-out of ever more powerful AI systems.  This is a matter which is beginning to attract wider attention.  Last month the Future of Life Institute published an open letter calling for all AI labs to immediately pause the training of AI systems more powerful than GPT-4 for at least six months.  Over 30,000 researchers and tech sector workers have signed the letter to date, including Stuart Russell, Steve Wozniak, Elon Musk, and Yuval Noah Harari.

Leaving aside whether six months could be long enough to resolve issues around AI safety, there is an important question to be answered here.  There are already numerous examples of cases where existing computerised and AI systems have caused harm, regardless of what the future might hold.  Why, then, are we racing forward in this field?  Has the combination of tech multinationals and unrestrained capitalism become such an unstoppable juggernaut that humanity is literally no longer able to control where the forces we have created are taking us?  If not, then why won’t governments intervene to put the brakes on the development and use of AI, and what interests are they actually working to protect?  This is unlikely to be a line of inquiry the Lords Committee will be pursuing.  Read more →

Armed drone proliferation update – March ’23

Chris Cole

We have fully updated our list of countries operating medium altitude, long endurance (MALE) armed drones as typified by the MQ-9 Reaper and Bayraktar TB2.  Just to emphasis, our list does not include states possessing loitering munitions (sometimes dubbed ‘suicide’ or kamikaze’ drones by the media) or other, one-off use smaller systems.

According to our data, thirty-four countries currently possess armed MALE drones although  some of these countries may not be currently using them operationally. Out of the 34 states known to possess armed drones 12 have used them in  cross border strikes and 11 have used them for strikes within their own borders.

Since our last update in May 2022, seven new countries have acquired armed drones:  Burkina Faso, Djibouti, Mali, Niger, Poland, Somalia, Togo and Tunisia.  All have acquired them from Turkey.  While Bayraktar TB2 armed drones are being used in Somalia it is not clear if they are being used by Turkish or Somali forces.

In addition to these 34 countries, we have identified another 14 countries that are likely to acquire armed drones in the near future although this is almost certainly a significant underestimate as  proliferation of these systems, particularly from Turkey, is rapidly increasing.

The full list of countries that possess armed drones, the type of armed drones they operate, which countries are likely to become operators in the near future, and a short narrative report on each country is available on our page: ‘Who Has Armed Drones?’

Turkey’s armed drone exports

Over the past five years (2018 – 2022) Turkey has become the predominant exporter of armed drones.  Out of the 20 countries that first gained armed drone capability in that period, four received drones from China, France armed its US drones and Russia began using its indigenously developed ‘Orion’ (Inokhodets) armed drone. The other 14 all received armed drones from Turkey.  Read more →

Defence Secretary confirms UK drone targeted killing in Syria

Chris Cole

At Defence Questions yesterday (13 March), the Defence Secretary, Ben Wallace, confirmed suspicions that the UK strike in Syria in December 2022 was a British drone targeted killing.  Choosing his words carefully, Wallace told the House of Commons:

“I want to update the House on a strike that took place a few weeks ago, as is our agreement on strikes under Operation Shader. In late December, an RAF Reaper remotely piloted aircraft conducted a strike against a leading Daesh member in al-Bab, northern Syria. The individual’s activity was related to chemical and biological weapons. The Reaper’s crew minimised potential risk to civilians before firing two Hellfire missiles, both of which struck the target accurately. These actions are vital to degrading such terrorist threats, protecting British citizens and supporting our international partners.”

The Ministry of Defence (MoD) had issued a short update two days after the strike:

“On Tuesday 20 December, a Reaper remotely piloted aircraft kept close observation on a building near Al Bab in northern Syria where at least one active Daesh terrorist was known to be present.  Great care was taken to ensure that any potential risks to civilians were understood and minimised before the Reaper’s crew fired a salvo of two Hellfire missiles which both struck the target accurately.”

Local sources reported at the time that two civilians – a woman and child – were injured in the strike and that one individual, a Yemeni national and ISIS commander was either severely injured or killed (see Airwars round-up of media reporting on the strike).  He was named locally by his nom de guerre of Abu Yasser al-Yemeni.

The local civilian civil defence force, know as the White Helmets, shared footage of the aftermath of the strike showing the house completely demolished.

Drone Wars submitted a Freedom of Information (FoI) request to the MoD in January asking if an investigation was underway into the reports of civilian casualties arising from the strike.  The MoD has refused to answer the request citing national security and international relations exemptions.  We have submitted an appeal.  Read more →

Ukraine drones may grab all the headlines, but armed drones are enabling lethal force around the globe

Chris Cole
President Zelensky stand with a ‘suicide drone’ in Kyiv, Oct 2022

As we reach the first anniversary of Russia’s invasion of Ukraine, you could be forgiven for thinking that the on-going conflict in eastern Europe has become the epicentre of the use of drones.

However, while the use of UAVs by Ukrainian and Russian forces has been very significant, it is important to be aware that there are real and important differences between the use of mainly small drone systems by parties in that conflict, and the use of large armed drones by other states such as the US, UK, Israel and Turkey even since the beginning of 2023.

Drones use in the Ukraine war

Over the past year, hardly a report on the war has failed to mention Ukraine’s use of surveillance drones to zero in Ukrainian artillery and rocket attacks on Russian forces or more recently, Russia’s use of Iranian  so called ‘suicide drones’ to attack Ukrainian targets.

Early on in the conflict, Ukraine deployed a number of larger armed Turkish Bayraktar TB2 drones.  Media reports at the time lauded the use of these as a game-changer and some even went so far as to suggest that use of armed drones would be strategically significant in the conflict.  However it quickly became apparent that the Bayraktar drones were very vulnerable to air-to-ground missiles as many were shot-down or crashed (see our crash database) and they quickly disappeared from the battlefield.  Some suggest that a few Bayraktars remain hidden and are being used covertly or kept for future operations but it is impossible to verify such claims.

Russia has at least one type of the larger armed medium altitude, long endurance (MALE) drone in its inventory – the Inokhodets or Orion (not to be confused with the much smaller and ubiquitous surveillance drone, the Orlan).  However, like the Bayraktar, the armed drone seems to have disappeared from the skies after one was shot down in April 2022.

Both sides have also occasionally used very old, soviet-era unmanned aircraft such as the Tupolev Strizh or Reys as missiles.

More recently, Russia has also used systems acquired from Iran. These have mainly been the Shahed 131/136 which are  technically loitering munitions that can only be used once, and have gained the moniker of ‘suicide drones’ in the press.  Alongside Russia is known to have acquired Iranian Mohajer-6 armed drones (one was filmed being fished out of the Black Sea after it was shot down/crashed) and, according to US sources, the Shahed 191 /129 armed UAVS, but these have not been seen in use.

Alongside the use of loitering munitions, both sides have primarily used small, short range drones for reconnaissance and  surveillance as well as targeting of artillery and rocket systems. While the use of drones in this way has been very significant – indeed perhaps the most significant use of drones for this purpose in any conflict until now – it is very different to how some states are using armed drones elsewhere.

Armed drone attacks outside of Ukraine virtually ignored

Even since the beginning of 2023, let alone the start of the Ukraine war, there has been significant use of armed drones by the states including the US, Israel and Turkey to conduct unlawful attacks.  These strikes, however, only get a fraction of the amount of media attention that drone use in Ukraine has, and are virtually ignored by the international community.  Read more →

Latest update shows UK drones spreading across air, land and sea

Chris Cole

We’ve updated our directory of current UK aerial drones and drone development programmes and wanted to highlight that, while drones have been mainly the preserve of the Air Force, they are now increasingly being acquired and used by the British Army and the Royal Navy.  Meanwhile, although the MoD is keen to point to the imminent arrival of its new armed drone, which they have dubbed ‘The Protector’, problems lie ahead.

Protector problems ahead

The replacement for the UK’s Reaper drone – dubbed ‘the Protector’ by the UK but called SkyGuardian by the manufacturer (and everyone else really) –  is supposed to be in service by mid-2024.  While the first aircraft from the production line has been delivered to the RAF it remains in the US for on-going testing and training.  However, two significant problems need to be addressed over the next 18 months before these drones become operational.

Firstly, recruitment and retention of personnel to operate the drones has been an on-going problem as Sir Stephen Lovegrove, then MoD permanent secretary, told the Commons public accounts committee in 2020.  This is likely to be even more so now as crews will be based permanently in Lincoln rather than having the option of being deployed to the sunnier climes of Las Vegas, after the UK shut down its US-based drone operations.

General Atomics promotional graphic visualising Protector flying over London

The RAF partly overcame recruitment issues by drafting in Royal Australian Air Force (RAAF) pilots.  As the RAAF  was set to purchase SkyGuardian drones it made sense to the RAAF to send pilots to operate UK armed drones as they would then get training and experience of using these systems before their drones arrived in Australia.  However in April 2022, Australia abruptly cancelled its planned purchase of SkyGuardian drones due to budget problems following the setting up of AUKUS alliance and the plan to build new nuclear submarines.  Given this, it seems likely the RAAF will not be so keen to provide personnel for the UK’s drone programme for much longer. Read more →

New trials of AI-controlled drones show push towards ‘killer robots’ as Lords announces special inquiry

Chris Cole
General Atomics Avenger controlled by AI in trial

Two recently announced trials of AI-controlled drones dramatically demonstrates the urgent need to develop international controls over the development and use of lethal autonomous weapon systems known as ‘killer robots’.

In early January, the UK Ministry of Defence (MoD) announced that a joint UK-US AI taskforce had undertaken a trial of its ‘AI toolbox’ during an exercise on Salisbury Plain in December 2022.  The trial saw a number of Blue Bear’s Ghost drones controlled by AI which was updated during the drone’s flight. The experiments said the MoD, “demonstrated that UK-US developed algorithms from the AI Toolbox could be deployed onto a swarm of UK UAVs and retrained by the joint AI Taskforce at the ground station and the model updated in flight, a first for the UK.”  The trials were undertaken as part of the on-going US-UK Autonomy and Artificial Intelligence Collaboration (AAIC) Partnership Agreement.  The MoD has refused to give MPs sight of the agreement.

Two weeks later, US drone manufacturer General Atomics announced that it had conducted flight trials on 14 December 2022 where an AI had controlled one of its large Avenger drones from the company’s own flight operations facility in El Mirage, California.

Blue Bear Ghost drones in AI in trail on Salisbury Plain

General Atomics said in its press release that the AI “successfully navigated the live plane while dynamically avoiding threats to accomplish its mission.” Subsequently, AI was used to control both the  drone and a ‘virtual’ drone at the same time in order to “collaboratively chase a target while avoiding threats,” said the company.  In the final trial, the AI “used sensor information to select courses of action based on its understanding of the world state. According to the company, “this demonstrated the AI pilot’s ability to successfully process and act on live real-time information independently of a human operator to make mission-critical decisions at the speed of relevance.”

Drone Wars UK has long warned that despite denials from governments on the development of killer robots, behind the scenes corporations and militaries are pressing ahead with testing, trialling and development of technology to create such systems. As we forecast in our 2018 report ‘Off the Leash’ armed drones are the gateway to the development of lethal autonomous systems.  Whiles these particular trials will not lead directly to the deployment of lethal autonomous systems, byte-by-byte the building blocks are being put in place.

House of Lords Special Committee

Due to continuing developments in this area we were pleased to learn that the House of Lords voted to accept Lord Clement-Jones’ proposal for a year-long inquiry by a special committee to investigate the use of artificial intelligence in weapon systems.  We will monitor the work of the Committee throughout the year but for now here is the accepted proposal in full:  Read more →