Supervisors

Suitable for

Abstract

Autonomous agents deployed in the real world need to be robust against adversarial attacks on sensory inputs. Robustifying agent policies requires anticipating the strongest attacks possible. We recently demonstrated that existing observation-space attacks on reinforcement learning agents have a common weakness: while effective, their lack of information-theoretic detectability constraints makes them detectable using automated means or human inspection. Detectability is undesirable to adversaries as it may trigger security escalations. In response, we introduced illusory attacks, a novel form of adversarial attack on sequential decision-makers that is both effective and of epsilon-bounded statistical detectability. In this project, we will be extending illusory attacks to dynamic patch attacks in autonomous driving settings. To this end, we consider a threat model in which dynamic patches can be projected into the car’s input sensors (be it visual, or using LIDAR spoofing [4]). For example, an adversarial car could project adversarial patches into the car behind it using a digital screen. This could mislead the car behind it into believing that the car in the neighbouring lane is ceding when it is not, thus causing inefficient or dangerous situations.

In this project, we will develop novel methods that ensure that the patch attacks generated are illusory in entity space, i.e. the car’s internal dynamics model of the world will perceive these attacks to be temporally and logically consistent - unlike in conventional adversarial attacks. To this end, we will leverage the latest advances in world model learning and reinforcement learning self-play. Furthermore, we will investigate possible ways of defending against illusory attacks both in entity and input space, building up on our recent work in out-of-distribution dynamics detection [2]. We will work with state-of-the-art autonomous driving simulators [3], and develop a novel RL benchmark environment suite for adversarial attacks in autonomous driving. If successful, we will then demonstrate our novel dynamic patch attacks in an exemplary real-world setting. This project is designed to lead to publication. We are looking for a highly-motivated student.

[1] Tim Franzmeyer, Stephen Marcus McAleer, Joao F. Henriques, Jakob Nicolaus Foerster, Philip Torr, Adel Bibi, Christian Schroeder de Witt, Illusory Attacks: Detectability Matters in Adversarial Attacks on Sequential Decision-Makers, https://openreview.net/forum?id=F5dhGCdyYh (under review at ICLR 2024)

[2] Linas Nasvytis, Kai Sandbrink, Jakob Foerster, Tim Franzmeyer and Christian Schroeder de Witt, Rethinking Out-of-Distribution Detection for Reinforcement Learning: Advancing Methods for Evaluation and Detection, AAMAS 2024 (Oral)

[3] https://github.com/waymo-research/waymax

[4] Yulong Cao et al., Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving, ACM SIGSAC 2019