"examples/nyu2/data/prepare_data.py" did not exist on "ecc6e480c4ea9bac1f7c69440ed44697a1aa9d1a"
  1. 12 Mar, 2025 1 commit
    • Graham King's avatar
      feat(pystr): Pass command line arguments (#123) · 995f71cc
      Graham King authored
      Command line arguments are passed to the python engine like this:
      ```
      dynamo-run out=pystr:my_python_engine.py -- -n 42 --custom-arg Orange --yes
      ```
      
      The python engine receives the arguments in `sys.argv`. The argument list will include some standard ones as well as anything after the `--`.
      
      This input:
      ```
      dynamo-run out=pystr:my_engine.py /opt/models/Llama-3.2-3B-Instruct/ --model-name llama_3.2 --tensor-parallel-size 4 -- -n 1
      ```
      
      is read like this:
      ```
      async def generate(request):
          .. as before ..
      
      if __name__ == "__main__":
          print(f"MAIN: {sys.argv}")
      ```
      
      and produces this output:
      ```
      MAIN: ['my_engine.py', '--model-path', '/opt/models/Llama-3.2-3B-Instruct/', '--model-name', 'llama3.2', '--http-port', '8080', '--tensor-parallel-size', '4', '--base-gpu-id', '0', '--num-nodes', '1', '--node-rank', '0', '-n', '1']
      ```
      
      This allows quick iteration on the engine setup. Note how the `-n` `1` is included. Flags `--leader-addr` and `--model-config` will also be added if provided to `dynamo-run`.
      995f71cc
  2. 11 Mar, 2025 2 commits
    • Graham King's avatar
      fix(pystr): Output python errors (#99) · 9c7b1ead
      Graham King authored
      If the python file raises an exception we print it like Python would.
      
      ```
      $ ./target/debug/dynamo-run in=http out=pystr:~/Temp/cn47/1_e.py --model-name test
      
      Traceback (most recent call last):
        File "/home/graham/Temp/cn47/1_e.py", line 17, in generate
          raise MyException("The message")
      1_e.MyException: The message
      ```
      9c7b1ead
    • Graham King's avatar
      feat(dynamo-run): Upgrade mistral.rs (#97) · d99b188d
      Graham King authored
      - Latest from repo, many improvements
      - Support most of the OpenAI request features (temperature, top_p, etc)
      - Download models from Hugging Face if necessary
      d99b188d
  3. 10 Mar, 2025 2 commits
  4. 08 Mar, 2025 1 commit