"...gtest/git@developer.sourcefind.cn:yangql/googletest.git" did not exist on "879916a9393ef4af84ebe8b331220586dd8cafbb"
  • Graham King's avatar
    fix(mistralrs): Disable paged attention (#234) · fd95f37b
    Graham King authored
    Under load it sometimes drops a request. The request gets added to the batch (sequence) and immediately gets a FinishReason Stop. Not sure why. It doesn't happen with the default scheduler (non-paged attention), so switch to that for now.
    fd95f37b
batch.rs 10.2 KB