[fix] regression testing oss+sharded_ddp only (#281)

- tighter regression detection, based on the best case vs. worst case - still run all configurations, useful for comparisons but not a target

[fix] regression testing oss+sharded_ddp only (#281)
- tighter regression detection, based on the best case vs. worst case - still run all configurations, useful for comparisons but not a target
fc1a40e1 · Benjamin Lefaudeux · GitHub · 3c727ec5 · fc1a40e1
Unverified Commit fc1a40e1 authored Dec 29, 2020 by Benjamin Lefaudeux Committed by GitHub Dec 29, 2020
Show whitespace changes
Inline Side-by-side

Showing with 2 additions and 1 deletion

.circleci/config.yml .circleci/config.yml +2 -1

No files found.
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -135,7 +135,8 @@ run_oss_benchmark: &run_oss_benchmark
  - run:
      name: Run OSS Benchmark
      command: |
-        python benchmarks/oss.py --check_regression --world_size 4 --reference_speed 660 --reference_memory 1120 --reference_loss 0.023
+        python benchmarks/oss.py --world_size 4 --epochs 2
+        python benchmarks/oss.py --check_regression --world_size 4 --optim_type oss_sharded_ddp --reference_speed 660 --reference_memory 930 --reference_loss 0.023

 run_oss_gloo: &run_oss_gloo
  - run: