Blog Tags GitHub

load-balancing

5 posts tagged with "load-balancing"

Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP

块级粒度的灵活上下文并行范式，通过任意点对点通信 + bin-packing 调度，在 256 张 NVIDIA GPU 上实现近线性扩展，attention MFU 提升 1.13x–2.21x

parallelism-cp distributed-training long-context efficient-attention load-balancing communication llm-training

Fairness in Serving Large Language Models

提出 Virtual Token Counter (VTC) 公平调度算法，实现 LLM 推理服务的 token 粒度公平共享，证明 2x 紧上界

llm-serving fair-scheduling virtual-token-counter continuous-batching load-balancing system-design

A Survey on Mixture of Experts in Large Language Models

MoE 在大语言模型中的全面综述，涵盖门控函数、专家网络、训练推理方案、系统设计和应用

survey mixture-of-experts llm sparse-activation conditional-computation gating-function load-balancing distributed-training

FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling

FlowKV 提出低成本低延迟的 KV Cache 传输方案和负载感知调度器，实现分离式 LLM 推理框架的 96% 传输延迟降低

llm-serving disaggregated-inference kv-cache load-balancing system-design nccl heterogeneous-gpu

February 25, 2026

Efficient Scaling of LLM Training with Flexible Context Parallelism

FCP：面向异构长上下文数据的自适应上下文并行策略，支持非2幂并行度、毫秒级调度开销，训练吞吐提升1.46倍

distributed-training context-parallelism long-context llm-training mllm-training load-balancing dynamic-parallelism megatron-lm