PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
A runtime memory-management system for MoE LLM serving that dynamically quantizes expert weights at runtime to balance GPU memory between model weights and KV cache, using a quality-aware planner with offline sensitivity, online routing statistics, and prompt residuals.