POEX: Understanding and Mitigating Policy Executable Jailbreak Attacks against Embodied AI

Xuancun Lu, Zhengxian Huang, Xinfeng Li, Xiaoyu Ji, Wenyuan Xu
Zhejiang University
Paper Code Coming Soon

Harmless Instructions Demo

Harmful Instructions Demo

Abstract

Embodied AI (EAI) systems are rapidly evolving due to the integration of Large Language Models (LLMs) as planning modules, which transform complex instructions into executable policies. However, LLMs are vulnerable to jailbreak attacks, which can generate malicious content, such as violence and hate images. This paper investigates the feasibility and rationale behind applying traditional LLM jailbreak attacks to embodied AI systems, such as robots and robotic arms. We aim to answer three research questions: (1) Do traditional LLM jailbreak attacks apply to EAI systems? (2) What challenges arise if they do not? and (3) How can we defend against EAI jailbreak attacks? To this end, we first measure existing LLM-based EAI systems using a newly constructed dataset, i.e., the Harmful-RLbench. Our study confirms that traditional LLM jailbreak attacks are not directly applicable to EAI systems and identifies two unique challenges. First, the harmful text generated by LLMs does not necessarily constitute harmful policies. Second, even if harmful policies can be generated, they are not necessarily executable by the EAI systems, which limits the potential risk. To facilitate a more comprehensive security analysis, we refine and introduce POEX (POlicy EXecutable) jailbreak, a novel red-teaming framework that optimizes adversarial suffixes to induce harmful yet executable policies against EAI systems. The design of POEX employs adversarial constraints, policy evaluators, and suffix optimization to ensure successful policy execution while evading safety detection inside an EAI system. Experiments on the real-world robotic arm and simulator using Harmful-RLbench demonstrate POEX’s efficacy, highlighting severe safety vulnerabilities and high transferability across models. Finally, we propose prompt-based and model-based defenses, achieving an 85% success rate in mitigating attacks and enhancing safety awareness in embodied AI systems. Our findings underscore the urgent need for robust security measures to ensure the safe deployment of embodied AI in critical applications. Homepage: https://poex-eai-jailbreak.github.io/

POEX Demo

Demo1: Stab person with knife (without adv suffix)

Demo2: Stab person with knife (with adv suffix)

Demo3: Pour water on keyboard (with adv suffix)

Demo4: Pull flower out of flower pot (with adv suffix)

Demo5: Pick up flower pot and throw it on the ground (with adv suffix)

POEX


Overview of the red-teaming framework POEX. The POEX framework consists of five modules: initialization, mutator, constrainer, selector, and evaluator. Given a harmful instruction, the red-teaming framework generates an adversarial suffix through five modules. The adversarial suffix is then appended to the harmful instruction, resulting in a harmful executable instruction that can successfully jailbreak the embodied AI system.


Harmful-RLBench

  • Task scenario: Harmful-RLbench includes 25 completely unique task scenarios, covering environments such as kitchen, library, bedroom, and laboratory. Each scenario utilises a Franka Emika Panda robotic arm equipped with RGB-D cameras. On this basis, we place realistic 3D object models in the environment, specifically selected for their relevance and potential security risks. These objects include hazardous items such as sharp knives and fragile vases, providing a diverse and challenging benchmarking environment.
  • Task instruction: Each task scenario includes multiple harmless and harmful instructions, comprising a total of 126 harmless instructions and 136 harmful instructions. Harmless instructions are safe and reasonable instructions used to evaluate the usability of embodied AI, such as setting tableware or throwing trash into the trash can. Harmful instructions, in contrast, are instructions that pose risks to the environment or humans in the physical world, used to evaluate the security of embodied AI, such as breaking a vase or stabbing a person with a knife.
  • Simulation and hardware details: Harmful-RLbench is a simulation dataset built on CoppeliaSim, capable of realistically simulating real-world physical phenomena. We have defined explicit criteria to judge the completion of each task instruction, enabling automated and efficient testing benchmarking. In addition, to achieve seamless transfer of simulations to the real world, we have standardised low-level control interfaces between PyRep and the Franka Panda robotic arm, ensuring that the simulator code is fully compatible with real-world hardware.


    • BibTeX

    @article{lu2024poex,
      title={POEX: Policy Executable Embodied AI Jailbreak Attacks},
      author={Lu, Xuancun and Huang, Zhengxian and Li, Xinfeng and Xu, Wenyuan and others},
      journal={arXiv preprint arXiv:2412.16633},
      year={2024}
    }