Skip to content

GitLab

Switch branch/tag
  1. 31 Jan, 2025 1 commit
    • Chen Zhang's avatar
      [v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603) · 89003c40
      Chen Zhang authored 
      

This pr adds extra key to block hash, to generate different hash value
for two blocks with the same token string but different extra_keys in
their parent blocks. For example, it can generate different hash value
for the second block of the following two requests:
```python
request1 = make_request(
        request_id=0,
        prompt_token_ids=[_ for _ in range(6)],
        mm_positions=[{
            "offset": 0,
            "length": 3
        }, {
            "offset": 3,
            "length": 3
        }],
        mm_hashes=["hash1", "hash2"],
    )
    request2 = make_request(
        request_id=1,
        prompt_token_ids=[_ for _ in range(6)],
        mm_positions=[{
            "offset": 0,
            "length": 3
        }, {
            "offset": 3,
            "length": 3
        }],
        mm_hashes=["hash3", "hash2"],
    )
```

---------
Signed-off-by: default avatarChen Zhang <zhangch99@outlook.com>
      89003c40
  3. 30 Jan, 2025 2 commits
  5. 29 Jan, 2025 2 commits
  7. 28 Jan, 2025 4 commits
  9. 27 Jan, 2025 3 commits
  11. 26 Jan, 2025 2 commits
  13. 24 Jan, 2025 2 commits
  15. 23 Jan, 2025 5 commits
  17. 22 Jan, 2025 4 commits
  19. 21 Jan, 2025 2 commits
  21. 20 Jan, 2025 3 commits
  23. 19 Jan, 2025 1 commit
  25. 18 Jan, 2025 1 commit
  27. 17 Jan, 2025 2 commits
  29. 16 Jan, 2025 1 commit
  31. 15 Jan, 2025 5 commits