Blog Tags GitHub

kv-cache-optimization

5 posts tagged with "kv-cache-optimization"

Sparse Attention推理技术全景: 从KV缓存压缩到硬件感知加速

系统梳理Sparse Attention在LLM推理中的技术演进、核心方法与工程实践

sparse-attention efficient-attention kv-cache-optimization pruning llm-inference

A Survey of Low-bit Large Language Models

低比特大语言模型综述：基础、系统和算法。

quantization survey llm-inference kernel-optimization kv-cache-optimization

Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching

通过异步KV缓存预取技术加速LLM推理吞吐量，减少内存访问延迟。

efficient-attention kv-cache-optimization llm-inference architecture distributed-training

February 20, 2025

LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention

LServe通过统一稀疏注意力机制实现高效长序列LLM服务，优化KV缓存管理和注意力计算。

sparse-attention pruning llm-inference long-context kv-cache-optimization

November 28, 2024

Marconi: Prefix Caching for the Era of Hybrid LLMs

Marconi为混合架构LLM提出高效前缀缓存机制，支持注意力层和非注意力层的统一缓存管理。

transformer-variant llm-inference attention-mechanism long-context kv-cache-optimization