llm-training

efficient-attention distributed-training large-language-model llm-training parallelism-dp llm-inference

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

使用模型并行训练数十亿参数语言模型

efficient-attention llm-training llm-inference

Scaling Laws for Neural Language Models

神经语言模型的缩放定律：模型大小、数据集大小和计算量与损失之间的幂律关系

kernel-optimization llm-training inference-serving

TMA-Adaptive FP8 Grouped GEMM: Eliminating Padding

消除 Hopper GPU 上低精度训练和推理的填充需求

audio mixture-of-experts diffusion-transformer llm-training audio-generation

Qwen3-Omni Technical Report

阿里巴巴Qwen团队的统一多模态模型，在文本、图像、音频、视频上无性能退化

June 7, 2026

H2SGEMM: Emulating FP32 GEMM on Ascend NPUs using FP16 Units with Precision Recovery

在华为 Ascend NPU 上使用 FP16 单元模拟 FP32 GEMM 的精度恢复方法

kernel-optimization quantization llm-training

June 7, 2026

Microbenchmarking NVIDIA's Blackwell Architecture: An in-depth Architectural Analysis

NVIDIA B200 GPU 微基准测试与架构深度分析

system-optimization benchmark llm-training

June 7, 2026

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations

面向细粒度和稀疏 MoE 的 IO 和 Tile 感知优化加速器

mixture-of-experts kernel-optimization llm-training

image-generation reinforcement-learning diffusion-model llm-training diffusion-transformer

Qwen-Image Technical Report

Qwen 图像生成基础模型技术报告

video-generation attention-mechanism quantization llm-training model-compression

LongLive: Real-time Interactive Long Video Generation

实时交互式长视频生成框架

quantization llm-training architecture survey

Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling

NVFP4 量化精度提升的自适应块缩放方法

reinforcement-learning video-generation llm-training diffusion-model multimodal

AR-CoPO: Align Autoregressive Video Generation with Contrastive Policy Optimization

通过对比策略优化对齐自回归视频生成

model-compression pruning llm-training survey vlm

EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation

通过知识蒸馏和模态自适应剪枝压缩视觉语言模型

reinforcement-learning llm-training survey pruning

The Landscape of Agentic Reinforcement Learning for LLMs: A Survey

LLM 智能体强化学习综述：从被动序列生成器到自主决策智能体

system-optimization reinforcement-learning llm-inference llm-training attention-mechanism

Seer: Online Context Learning for Fast Synchronous LLM Reinforcement Learning

一种通过组感知上下文学习加速同步 LLM 强化学习 Rollout 的系统

weight-pruning model-compression attention-mechanism llm-training architecture

Sparsity Induction for Accurate Post-Training Pruning of Large Language Models

通过稀疏性诱导提升 LLM 后训练剪枝精度

llm-training mixture-of-experts long-context parallelism-dp vlm

Seed1.5-VL Technical Report

通用多模态理解与推理视觉语言基础模型，532M视觉编码器 + 20B活跃参数MoE LLM

llm-training reinforcement-learning benchmark

Seed-Coder: Let the Code Model Curate Data for Itself

模型驱动的代码预训练数据构建方法，最小化人工干预

llm-training system-optimization parallelism-tp

RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training

通过尾部批处理和全栈系统优化解决RL训练中长尾rollout导致的GPU利用率低问题

diffusion-model llm-training diffusion-transformer

Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model

基于扩散的代码大语言模型，通过Block Diffusion持续预训练超越自回归基线

reinforcement-learning llm-training llm-inference distributed-training memory-efficiency

强化学习基础设施(RL Infra)优化必读论文与技术全景

覆盖分布式RL、RLHF系统、环境加速、采样优化、通信优化、内存优化等方向

parallelism communication llm-training distributed-training mixture-of-experts

RL Infra 系统设计：GPU硬件×网络拓扑×模型结构→并行策略×资源配比×流水线

从硬件约束出发，推导RL训练的最优并行策略、资源分配和流水线设计

June 2, 2026

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language

通过平滑激活离群值实现LLM的W8A8量化，支持高达530B参数模型的无损量化

quantization llm-training mixture-of-experts

June 2, 2026

MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling

基于微缩放和自动缩放的高效FP8 LLM训练方法，实现34%吞吐量提升

efficient-attention llm-training inference-serving quantization

June 1, 2026

Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models

面向延迟最优的混合小语言模型，通过深度-宽度优化、算子搜索和权重归一化实现SOTA效率

llm-training long-context efficient-attention

kernel-optimization memory-efficiency llm-training

Minuet: Accelerating 3D Sparse Convolutions on GPUs

面向GPU的3D稀疏卷积加速引擎

quantization survey llm-training

Understanding Stragglers in Large Model Training Using What-if Analysis

使用What-if分析理解大模型训练中的Straggler问题

mixture-of-experts diffusion-model llm-training

Tanbr: Tree-Structured Adaptive Neural Bandit Router for Online MoE Inference

基于树结构自适应神经老虎机路由器的在线MoE推理优化，实现45%延迟降低和25%内存节省

model-compression pruning llm-training

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large

利用软激活稀疏性和低秩分解的LLM压缩方法

mixture-of-experts memory-efficiency llm-training llm-inference

SpecMoE: A Fast and Efficient Mixture-of-Experts Inference via Self-Assisted Speculative Decoding

通过自辅助推测解码实现高效MoE推理，最高4.30×吞吐量提升

robotics llm-training diffusion-model reinforcement-learning vla vlm

Qwen-VLA

统一的具身基础模型，通过DiT动作解码器将视觉-语言建模扩展到连续动作生成

survey llm-training llm-inference

Principles and Methodologies for Serial Performance Optimization

串行性能优化的系统方法论，包含三个原则和八种方法

attention-mechanism multimodal llm-training robotics diffusion-transformer

VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model

A lightweight bridging paradigm that enables SOTA-level VLA performance with only 0.5B parameters, without robotic data pre-training

llm-training reinforcement-learning video-generation vla vlm

Cosmos-Predict2.5: World Simulation with Video Foundation Models for Physical

NVIDIA Cosmos世界基础模型2.5代：面向Physical AI的视频世界仿真

reinforcement-learning system-optimization llm-training vla

RL-VLA³: A Flexible and Asynchronous Reinforcement Learning Framework for VLA Training

面向VLA训练的灵活异步强化学习框架

robotics llm-training inference-serving diffusion-model vla vlm

Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution

小米机器人团队发布的先进VLA模型，通过精心设计的训练方案和部署策略实现高性能实时执行

diffusion-model llm-training diffusion-transformer vla

Fast-WAM: Do World Action Models Need Test-time Future Imagination?

世界动作模型是否需要测试时未来想象？

llm-training memory-efficiency distributed-training

MegaTrain: Full Precision Training of 100B+ Parameter Large Language Models on a Single GPU

MegaTrain：在单GPU上全精度训练100B+参数大语言模型

llm-training inference-serving kv-cache kernel-optimization

DVM: A Bytecode Virtual Machine Approach for Dynamic Tensor Computation

DVM：基于字节码虚拟机的动态张量计算方法

communication llm-training parallelism-sp parallelism-dp

Transforming the Use of Earth Observation Data: Exascale Training of a Generative

利用历史先验的生成式压缩模型实现高达10000倍地球观测数据压缩

benchmark llm-training mixture-of-experts scheduling

The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

MiniMax-M2系列：小型激活释放最大真实世界智能

mixture-of-experts llm-training model-compression

Laguna M1-XS2: A 103B Open-Weights Frontier Coding Agent

Poolside Laguna M1-XS2：103B参数开放权重前沿编码Agent

long-context llm-training reinforcement-learning

LongSkywork: A Training Recipe for Efficiently Extending Context Length in

高效扩展大语言模型上下文长度的训练方案

llm-training reinforcement-learning parallelism-tp

HybridFlow: A Flexible and Efficient RLHF Framework

灵活高效的 RLHF 分布式训练框架

quantization llm-training distributed-training vla vlm

COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training

通过压缩优化器状态和激活实现高效FP8训练

pruning model-compression llm-training

HashAttention: Semantic Sparsity for Faster Inference

HashAttention：语义稀疏性实现更快推理

mixture-of-experts quantization llm-training

DeepSeek-V3 Technical Report

DeepSeek-V3: 671B 参数 MoE 语言模型技术报告

efficient-attention long-context llm-training scheduling

Qwen2.5-1M Technical Report

支持100万token上下文长度的Qwen2.5模型

long-context benchmark llm-training

From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models

从 128K 到 4M：高效训练超长上下文大语言模型

llm-training system-optimization reinforcement-learning

AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training

面向高效 LLM 后训练的异步流式 RL 框架

model-compression llm-inference llm-training

Self-Distillation for Multi-Token Prediction

通过自蒸馏提升多token预测的接受率和推理效率

reinforcement-learning llm-training distributed-training vlm

Relax: An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale

面向全模态后训练的异步强化学习引擎