SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
使用 INT8 量化注意力计算 + K 平滑 + FP16 累加器的即插即用推理加速方法
All tags
model-compression
42 posts tagged with "model-compression"
SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
使用 INT4 线程级量化和 FP8 矩阵乘加速注意力计算，配合 Q+K 平滑与两级累加策略保持精度
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
将稀疏注意力与线性注意力融合的 trainable attention 方法，用于加速 DiT 视频生成模型
VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression
训练高效的自压缩框架，利用预训练 VLM 自身作为内在编码器压缩视觉 token
LingBot-World-Infinity: Infinite Worlds with Versatile Interactions
一个因果视频生成世界模型，实现无限长、实时响应的交互式世界模拟，支持多种角色动作和环境事件，并引入Pilot-Director双智能体协作框架
Robust Representation Consistency Model via Contrastive Denoising
rRCM reformulates generative denoising as discriminative latent-space learning for certified robustness with 85× speedup
Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling
MrFlow achieves 10x+ training-free acceleration for flow-matching diffusion models via a four-stage low-to-high resolution pipeline
Efficient Track Anything
高效视频目标分割与跟踪模型
Efficient Diffusion Models: A Survey
高效扩散模型综述：算法级、系统级和框架视角
ConsistTalk: Intensity Controllable Temporally Consistent Talking Head Generation with Diffusion Noise Search
具有扩散噪声搜索的强度可控时间一致说话头生成
Unveiling Super Experts in Mixture-of-Experts Large Language Models
发现MoE大语言模型中的超级专家：少量关键专家对模型性能的决定性影响
LongLive: Real-time Interactive Long Video Generation
实时交互式长视频生成框架
Rolling Forcing: Autoregressive Long Video Diffusion in Real Time
实时自回归长视频扩散生成技术
MotionStream: Real-Time Video Generation with Interactive Motion Controls
实时交互式运动控制视频生成
StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation
实时直播视频生成的流式系统
Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation
面向高质量实时交互式视频生成的自回归扩散蒸馏正确方法
Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generation
面向快速视频生成的自洽分布匹配与缓存感知训练框架
X-Cache: Cross-Chunk Block Caching for Few-Step Autoregressive World Models Inference
面向少步自回归世界模型推理的跨分块缓存加速
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation
通过知识蒸馏和模态自适应剪枝压缩视觉语言模型
SparseLLM: Towards Global Pruning for Pre-trained Language Models
一种将全局剪枝分解为可管理子问题的LLM压缩框架
FlashVLA: Think Twice, Act Once - Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models
首个免训练、即插即用的VLA推理加速框架，通过动作复用和视觉token剪枝实现55.7% FLOPs降低和36.0%延迟降低
EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models
一种无需训练的结构化推理加速框架，用于加速扩散式视觉-语言-动作模型
Improving MoE Compute Efficiency by Composing Weight and Data Sparsity
通过组合权重稀疏性和数据稀疏性提升 MoE 计算效率
Sparsity Induction for Accurate Post-Training Pruning of Large Language Models
通过稀疏性诱导提升 LLM 后训练剪枝精度
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
通过低秩Key缓存和CPU卸载Value缓存实现高吞吐量长上下文LLM推理
PEEK: Picking Essential frames via Efficient Knowledge distillation
通过知识蒸馏实现高效视频帧选择的查询无关方法，在低帧预算下显著优于现有方法
Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
PostDiff：免训练扩散模型压缩框架，通过混合分辨率去噪和混合模块缓存降低单步推理成本
Towards Efficient Large Language Model Serving: A Survey on System-Aware KV
系统感知的KV缓存优化综述，从时间、空间、结构三个维度系统梳理LLM推理中的KV缓存管理技术
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
全方向校准的LLM量化技术，通过LWC和LET实现PTQ效率与QAT性能的统一
ZEN: Empowering Distributed Training with Sparsity-driven Data Synchronization
稀疏驱动的数据同步系统，实现分布式训练5.09×通信加速
SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large
利用软激活稀疏性和低秩分解的LLM压缩方法
DMEP: Adaptive and Fine-grained Module-wise Expert Pruning for Efficient LoRA-MoE
动态模块级专家剪枝框架，实现LoRA-MoE微调参数减少43%且精度不降
EnViT: Enhancing the Performance of Early-Exit Vision Transformers via Exit-Aware Structured Dropout-Enabled Self-Distillation
通过退出感知结构化dropout增强早期退出Vision Transformer性能
Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
基于最大流的异构GPU集群LLM推理服务系统
Laguna M1-XS2: A 103B Open-Weights Frontier Coding Agent
Poolside Laguna M1-XS2：103B参数开放权重前沿编码Agent
HashAttention: Semantic Sparsity for Faster Inference
HashAttention：语义稀疏性实现更快推理
Self-Distillation for Multi-Token Prediction
通过自蒸馏提升多token预测的接受率和推理效率
PULSE: Understanding and Exploiting Weight Update Sparsity for Communication-Efficient
分布式 RL 后训练大语言模型面临两个通信瓶颈：
SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference
通过解耦视觉稀疏性，在预填充和解码阶段分别进行查询无关剪枝和查询感知检索，实现高效VLM推理
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models
通过剪枝感知预训练实现架构无关的高效边缘语言模型，首次将LLM压缩提升到预训练阶段
Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
通过动态输入剪枝和缓存感知掩码实现高效LLM推理，降低计算和内存开销。
Enabling Unstructured Sparse Acceleration on Structured Sparse Accelerators
在结构化稀疏加速器上实现非结构化稀疏加速，通过TASD方法弥合软硬件稀疏模式不匹配的鸿沟。