"megatron/training/arguments.py" did not exist on "5c45db4a79e91f2fb5620120594aa1727d2f7b37"
-
Daniel Hiltgen authored
This should resolve a number of memory leak and stability defects by allowing us to isolate llama.cpp in a separate process and shutdown when idle, and gracefully restart if it has problems. This also serves as a first step to be able to run multiple copies to support multiple models concurrently.
58d95cc9