Skip to content

GitLab

Switch branch/tag
  1. 31 Jan, 2025 1 commit
    • Chen Zhang's avatar
      [v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603) · 89003c40
      Chen Zhang authored 
      

This pr adds extra key to block hash, to generate different hash value
for two blocks with the same token string but different extra_keys in
their parent blocks. For example, it can generate different hash value
for the second block of the following two requests:
```python
request1 = make_request(
        request_id=0,
        prompt_token_ids=[_ for _ in range(6)],
        mm_positions=[{
            "offset": 0,
            "length": 3
        }, {
            "offset": 3,
            "length": 3
        }],
        mm_hashes=["hash1", "hash2"],
    )
    request2 = make_request(
        request_id=1,
        prompt_token_ids=[_ for _ in range(6)],
        mm_positions=[{
            "offset": 0,
            "length": 3
        }, {
            "offset": 3,
            "length": 3
        }],
        mm_hashes=["hash3", "hash2"],
    )
```

---------
Signed-off-by: default avatarChen Zhang <zhangch99@outlook.com>
      89003c40
  3. 27 Jan, 2025 1 commit
  5. 24 Jan, 2025 1 commit
  7. 23 Jan, 2025 1 commit
  9. 22 Jan, 2025 1 commit
  11. 21 Jan, 2025 1 commit
  13. 17 Jan, 2025 1 commit
  15. 15 Jan, 2025 1 commit
  17. 13 Jan, 2025 1 commit
  19. 12 Jan, 2025 1 commit
  21. 10 Jan, 2025 1 commit
  23. 06 Jan, 2025 2 commits
  25. 04 Jan, 2025 2 commits
  27. 03 Jan, 2025 1 commit
  29. 01 Jan, 2025 1 commit
  31. 31 Dec, 2024 2 commits
  33. 28 Dec, 2024 2 commits
  35. 27 Dec, 2024 1 commit
  37. 26 Dec, 2024 1 commit
  39. 18 Dec, 2024 1 commit
  41. 13 Dec, 2024 1 commit
  43. 12 Dec, 2024 1 commit
  45. 03 Dec, 2024 1 commit
  47. 28 Nov, 2024 2 commits
  49. 26 Nov, 2024 1 commit
  51. 25 Nov, 2024 1 commit
  53. 22 Nov, 2024 1 commit
  55. 13 Nov, 2024 2 commits
  57. 11 Nov, 2024 1 commit
  59. 08 Nov, 2024 1 commit