Skip to content

GitLab

Switch branch/tag
  1. 04 Apr, 2025 1 commit
    • Graham King's avatar
      feat: Python decorator dynamo_worker takes optional `static` parameter without etcd (#494) · 88ad3425
      Graham King authored 
      Adds `@dynamo_worker(static = True)` to create a static worker which has a predictable name and hence does not require discovery or `etcd` to be running. There can only be a single static worker per namespace / component / endpoint trio.

This contrasts with the default dynamic `dynamo_worker` endpoints we have now, which get a unique random name (based on namespace/component/endpoint), and are discovered by ingress components using etcd.

Also change the hello_world example to use `dynamo_worker(static = True)` so that it is exercised and demonstrated somewhere.

For NIM.
      88ad3425
  3. 03 Apr, 2025 1 commit
  5. 02 Apr, 2025 1 commit
  7. 01 Apr, 2025 1 commit
  9. 17 Mar, 2025 1 commit
  11. 11 Mar, 2025 2 commits
  13. 09 Mar, 2025 2 commits
  15. 08 Mar, 2025 1 commit
  17. 07 Mar, 2025 1 commit
    • Graham King's avatar
      feat: Python bring-your-own-engine with our tokenizer (#47) · 12714d90
      Graham King authored 
      Instead of using `out=pystr:<my.py>` we can now do this:
```
dynemo-run out=pytok:/home/graham/my_python_engine.py --model-path <hf-repo-checkout>
```

That engine will receive and respond with tokens. Here's an example engine file:
```
import asyncio

async def generate(request):
    yield {"token_ids":[791]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[6864]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[315]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[9822]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[374]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[12366]}
    await asyncio.sleep(0.1)
    yield {"token_ids":[13]}
```

Also reduce duplication by making the bindings engine use the llm lib engine.
      12714d90
  19. 06 Mar, 2025 1 commit
  21. 05 Mar, 2025 1 commit
  23. 04 Mar, 2025 2 commits
  25. 28 Feb, 2025 1 commit
  27. 27 Feb, 2025 1 commit
  29. 25 Feb, 2025 4 commits