欢迎阅读 FlashInfer 文档！¶
FlashInfer 是一个用于大型语言模型的库和内核生成器，它提供了 FlashAttention、PageAttention 和 LoRA 等 LLM GPU 内核的高性能实现。FlashInfer 专注于 LLM 服务和推理，并在各种场景中提供最先进的性能。
入门
教程
PyTorch API 参考
- FlashInfer 注意力内核
- flashinfer.gemm
- flashinfer.fused_moe
- flashinfer.cascade
- flashinfer.comm
- flashinfer.sparse
- flashinfer.page
- flashinfer.sampling
- flashinfer.sampling.sampling_from_probs
- flashinfer.sampling.top_p_sampling_from_probs
- flashinfer.sampling.top_k_sampling_from_probs
- flashinfer.sampling.min_p_sampling_from_probs
- flashinfer.sampling.top_k_top_p_sampling_from_logits
- flashinfer.sampling.top_k_top_p_sampling_from_probs
- flashinfer.sampling.top_p_renorm_probs
- flashinfer.sampling.top_k_renorm_probs
- flashinfer.sampling.top_k_mask_logits
- flashinfer.sampling.chain_speculative_sampling
- flashinfer.topk
- flashinfer.logits_processor
- flashinfer.norm
- flashinfer.rope
- flashinfer.rope.apply_rope_inplace
- flashinfer.rope.apply_llama31_rope_inplace
- flashinfer.rope.apply_rope
- flashinfer.rope.apply_llama31_rope
- flashinfer.rope.apply_rope_pos_ids
- flashinfer.rope.apply_rope_pos_ids_inplace
- flashinfer.rope.apply_llama31_rope_pos_ids
- flashinfer.rope.apply_llama31_rope_pos_ids_inplace
- flashinfer.rope.apply_rope_with_cos_sin_cache
- flashinfer.rope.apply_rope_with_cos_sin_cache_inplace
- flashinfer.activation
- flashinfer.quantization
- flashinfer.green_ctx
- flashinfer.fp4_quantization
- flashinfer.testing