Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
attention.md 2.35 KB
Newer Older
helloyongyang's avatar
add paper doc  
helloyongyang committed
1
2 
# 注意力机制
helloyongyang's avatar
update attention papers  
helloyongyang committed
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69 
### Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity

[paper](https://arxiv.org/abs/2502.01776) | [code](https://github.com/svg-project/Sparse-VideoGen)

### Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation

[paper](https://arxiv.org/abs/2505.18875)

### Training-free and Adaptive Sparse Attention for Efficient Long Video Generation

[paper](https://arxiv.org/abs/2502.21079)

### DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Training

[paper](https://arxiv.org/abs/2502.07590)

### MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention

[paper](https://github.com/microsoft/MInference)

### FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion

[paper](https://arxiv.org/abs/2506.04648)

### VORTA: Efficient Video Diffusion via Routing Sparse Attention

[paper](https://arxiv.org/abs/2505.18809)

### Training-Free Efficient Video Generation via Dynamic Token Carving

[paper](https://arxiv.org/abs/2505.16864)

### RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancy

[paper](https://arxiv.org/abs/2505.21036)

### Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation

[paper](https://arxiv.org/abs/2506.19852)

### VMoBA: Mixture-of-Block Attention for Video Diffusion Models

[paper](https://arxiv.org/abs/2506.23858)

### SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference

[paper](https://arxiv.org/abs/2502.18137) | [code](https://github.com/thu-ml/SpargeAttn)

### Fast Video Generation with Sliding Tile Attention

[paper](https://arxiv.org/abs/2502.04507) | [code](https://github.com/hao-ai-lab/FastVideo)

### PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models

[paper](https://arxiv.org/abs/2506.16054)

### Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light

[paper](https://arxiv.org/abs/2504.16922)

### Astraea: A GPU-Oriented Token-wise Acceleration Framework for Video Diffusion Transformers

[paper](https://arxiv.org/abs/2506.05096)

### ∇NABLA: Neighborhood Adaptive Block-Level Attention

[paper](https://github.com/gen-ai-team/Wan2.1-NABLA)