RSI-Inference Bench is a long-horizon benchmark for coding agents that improve the inference stack of large language models. It follows a model through its life in a production engine, vLLM or SGLang: supported on day 0, optimized after launch, and repaired when a regression lands. An agent works on the real engine, on real GPUs, and is graded by running what it built.
Apple's RSI-Inference Bench scores coding agents on 37 real vLLM and SGLang inference tasks via x.com
Discussion