Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "3b0cb7945f5c9dbe55d5f76720ddf4e475c11169"
Switch branch/tag
  1. 31 Jan, 2025 1 commit
    • Chen Zhang's avatar
      [v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603) · 89003c40
      Chen Zhang authored 
      

This pr adds extra key to block hash, to generate different hash value
for two blocks with the same token string but different extra_keys in
their parent blocks. For example, it can generate different hash value
for the second block of the following two requests:
```python
request1 = make_request(
        request_id=0,
        prompt_token_ids=[_ for _ in range(6)],
        mm_positions=[{
            "offset": 0,
            "length": 3
        }, {
            "offset": 3,
            "length": 3
        }],
        mm_hashes=["hash1", "hash2"],
    )
    request2 = make_request(
        request_id=1,
        prompt_token_ids=[_ for _ in range(6)],
        mm_positions=[{
            "offset": 0,
            "length": 3
        }, {
            "offset": 3,
            "length": 3
        }],
        mm_hashes=["hash3", "hash2"],
    )
```

---------
Signed-off-by: default avatarChen Zhang <zhangch99@outlook.com>
      89003c40
  3. 30 Jan, 2025 2 commits
  5. 29 Jan, 2025 1 commit
  7. 28 Jan, 2025 1 commit
  9. 27 Jan, 2025 1 commit
  11. 23 Jan, 2025 1 commit
  13. 22 Jan, 2025 1 commit
  15. 17 Jan, 2025 1 commit
  17. 15 Jan, 2025 3 commits
  19. 12 Jan, 2025 1 commit
  21. 08 Jan, 2025 2 commits
  23. 01 Jan, 2025 1 commit
  25. 31 Dec, 2024 2 commits
  27. 19 Dec, 2024 1 commit
  29. 18 Dec, 2024 2 commits
  31. 17 Dec, 2024 1 commit
  33. 14 Dec, 2024 2 commits
  35. 13 Dec, 2024 2 commits
  37. 10 Dec, 2024 1 commit
  39. 08 Dec, 2024 1 commit
  41. 28 Nov, 2024 1 commit
  43. 22 Nov, 2024 1 commit
  45. 13 Nov, 2024 1 commit
  47. 11 Nov, 2024 1 commit
  49. 08 Nov, 2024 2 commits
  51. 22 Oct, 2024 1 commit