Commits · 108fe021657ea1f7a012299f424d673e86512df2 · OpenDAS / ollama

17 Mar, 2025 1 commit
- sample: make mutations in transforms explicit (#9743) · 108fe021
  Parth Sareen authored Mar 17, 2025
```
* updated minP to use early exit making use of sorted tokens
```
  108fe021
13 Mar, 2025 1 commit
- sample: separate softmax and temperature transforms (#9732) · 5c0b6639
  Parth Sareen authored Mar 13, 2025
  
  5c0b6639
12 Mar, 2025 3 commits
- sample: do all sorting in topK · 4aeb67ef
  ParthSareen authored Mar 12, 2025
  
  4aeb67ef
- sample: simplify top_k=0 sorting · 3ba91634
  ParthSareen authored Mar 12, 2025
  
  3ba91634
- sample: use container/heap for top_k · 1b7433b7
  ParthSareen authored Mar 12, 2025
  
  1b7433b7
10 Mar, 2025 2 commits
- sample: add numerical stability to temperature/softmax transform (#9631) · 7e34f4fb
  Parth Sareen authored Mar 10, 2025
  
  7e34f4fb
- sample: temporarily use grammars for constrained generation in new engine (#9586) · e093db92
  Jeffrey Morgan authored Mar 10, 2025
  
  e093db92
07 Mar, 2025 1 commit

sample: improve ollama engine sampler performance (#9374) · 0682dae0

Parth Sareen authored Mar 07, 2025

This change bring in various interface cleanups along with greatly improving the performance of the sampler.

Tested with llama3.2 on local machine.
Improves performance from ~ 70 tokens/s -> 135 tokens/s with topK(40) enabled.
Without topK performance is ~ 110 tokens/s

0682dae0

25 Feb, 2025 1 commit
- sample: add sampling package for new engine (#8410) · 0b7e1676
  Parth Sareen authored Feb 24, 2025
  
  0b7e1676