OrchANN: A Unified I/O Orchestration Framework for Skewed Out-of-Core Vector Search

RAIDS Lab Authors

Zhengyi Yang

Director

Details

Research Area

Tags

Resources

Abstract

Approximate nearest neighbor search (ANNS) at billion scale is fundamentally an out-of-core problem: vectors and indexes live on SSD, so performance is dominated by I/O rather than compute. Under skewed semantic embeddings, existing out-of-core systems break down: a uniform local index mismatches cluster scales, static routing misguides queries and inflates the number of probed partitions, and pruning is incomplete at the cluster level and lossy at the vector level, triggering "fetch-to-discard" reranking on raw vectors. We present OrchANN, an out-of-core ANNS engine that uses an I/O orchestration model for unified I/O governance along the route-access-verify pipeline. OrchANN selects a heterogeneous local index per cluster via offline auto-profiling, maintains a query-aware in-memory navigation graph that adapts to skewed workloads, and applies multi-level pruning with geometric bounds to filter both clusters and vectors before issuing SSD reads. Across five standard datasets under strict out-of-core constraints, OrchANN outperforms four baselines including DiskANN, Starling, SPANN, and PipeANN in both QPS and latency while reducing SSD accesses. Furthermore, OrchANN delivers up to 17.2x higher QPS and 25.0x lower latency than competing systems without sacrificing accuracy.

Author Affiliations

@misc{huan2025orchann,
  title = {OrchANN: A Unified I/O Orchestration Framework for Skewed Out-of-Core Vector Search},
  author = {Chengying Huan and Lizheng Chen and Zhengyi Yang and Shaonan Ma and Rong Gu and Renjie Yao and Zhibin Wang and Mingxing Zhang and Fang Xi and Jie Tao and Gang Zhang and Guihai Chen and Chen Tian},
  year = {2025},
  eprint = {2512.22838},
  archivePrefix = {arXiv},
  primaryClass = {cs.DB},
  url = {https://arxiv.org/abs/2512.22838}
}