Benchmark info (#138)

2bfa234f · Casper · GitHub · 0f3214b1 · 2bfa234f
Unverified Commit 2bfa234f authored Nov 02, 2023 by Casper Committed by GitHub Nov 02, 2023
Hide whitespace changes
Inline Side-by-side

Showing with 3 additions and 0 deletions

README.md README.md +3 -0

No files found.
--- a/README.md
+++ b/README.md
@@ -197,6 +197,9 @@ generation_output = model.generate(

 ## Benchmarks

+- GPU: RTX 3090
+- Command: `python examples/benchmark --model_path <hf_model>`
+
 | Model Name    | Version | Batch Size | Prefill Length | Decode Length | Prefill tokens/s | Decode tokens/s | Memory (VRAM)    |
 |---------------|---------|------------|----------------|---------------|------------------|-----------------|------------------|
 | Vicuna 7B     | GEMM    | 1          | 64             | 64            | 2618.88          | 125.428         | 4.57 GB (19.31%) |