← Publications
preprint2026

Evolving Skill-Structured Attack Memory Enhances LLM Jailbreaking

Junke Zhang, Jianwei Wang*, Sishuo Chen, Yizhang He, Qingshuai Feng, Zhengyi Yang

arXiv

RAIDS Lab Authors

Zhengyi Yang
Zhengyi Yang
Director

Details

Year
2026
Venue
arXiv

Research Area

Responsible Data Intelligence

Tags

LLMJailbreakSecurityMemory

Resources

arXiv ↗

Abstract

Jailbreak attacks on large language models (LLMs) aim to induce LLMs to produce content that they are expected to refuse. Automated black-box jailbreak generation is especially important for safety evaluation, where the attacker observes only model outputs and needs to automatically search for effective adversarial prompts. Existing black-box jailbreak methods either depend on sample-wise heuristic search or leverage attack experience through accumulating strategy pools or method libraries, lacking a systematic organization and management of attack experience. To mitigate these drawbacks, we propose MemoAttack, a memory-driven black-box jailbreak framework with comprehensive attack memory modeling, evolution, and selection. Specifically, MemoAttack comprises three key designs: (1) Skill-Structured Memory Modeling, which abstracts accumulated attack experience into reusable skill-structured attack memory whose units pair attack skills with templates, evidence, and lifecycle state; (2) Lifecycle-Driven Memory Evolution, which evolves the memory through evidence-based probation, promotion, retirement, reactivation, elimination, and storage cleanup; and (3) Explore-Exploit Balanced Memory Selection, which balances reliable memory reuse with uncertainty-driven exploration via contextual Thompson Sampling. Experiments on AdvBench demonstrate that MemoAttack achieves an average attack success rate of 98.00%, outperforming the strongest baseline by 16.67 percentage points, while reducing request count by 45.9%. Moreover, MemoAttack continuously improves as memory accumulates over more samples.

Author Affiliations

Junke Zhang
University of New South Wales
Jianwei Wang
University of New South Wales
Sishuo Chen
Alibaba Group
Yizhang He
University of New South Wales
Qingshuai Feng
Great Bay University
Zhengyi Yang
University of New South Wales; University of Sydney

BibTeX

@misc{zhang2026evolving,
  title = {Evolving Skill-Structured Attack Memory Enhances LLM Jailbreaking},
  author = {Junke Zhang and Jianwei Wang and Sishuo Chen and Yizhang He and Qingshuai Feng and Zhengyi Yang},
  year = {2026},
  eprint = {2605.29237},
  archivePrefix = {arXiv},
  primaryClass = {cs.CR},
  url = {https://arxiv.org/abs/2605.29237}
}