Skip to content

GitLab

Switch branch/tag
  1. 11 Mar, 2025 2 commits
  3. 09 Mar, 2025 2 commits
  5. 08 Mar, 2025 1 commit
  7. 07 Mar, 2025 1 commit
    • Graham King's avatar
      feat: Python bring-your-own-engine with our tokenizer (#47) · 12714d90
      Graham King authored 
      Instead of using `out=pystr:<my.py>` we can now do this:
```
dynemo-run out=pytok:/home/graham/my_python_engine.py --model-path <hf-repo-checkout>
```

That engine will receive and respond with tokens. Here's an example engine file:
```
import asyncio

async def generate(request):
    yield {"token_ids":[791]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[6864]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[315]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[9822]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[374]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[12366]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[13]}
```

Also reduce duplication by making the bindings engine use the llm lib engine.
      12714d90
  9. 06 Mar, 2025 1 commit
  11. 05 Mar, 2025 1 commit
  13. 04 Mar, 2025 2 commits
  15. 28 Feb, 2025 1 commit
  17. 27 Feb, 2025 1 commit
  19. 25 Feb, 2025 4 commits