Unverified Commit 43620c3f authored by Yifan Xiong's avatar Yifan Xiong Committed by GitHub
Browse files

Docs - Update README and version for v0.2.0 release (#111)

Update README and version for v0.2 release.
parent fb7d4a73
# SuperBench
[![MIT licensed](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE)
[![Lint](https://github.com/microsoft/superbenchmark/workflows/Lint/badge.svg)](https://github.com/microsoft/superbenchmark/actions?query=workflow%3ALint)
[![Build Image](https://github.com/microsoft/superbenchmark/workflows/Build%20Image/badge.svg)](https://github.com/microsoft/superbenchmark/actions/workflows/build-image.yml)
[![Codecov](https://codecov.io/gh/microsoft/superbenchmark/branch/main/graph/badge.svg?token=DDiDLW7pSd)](https://codecov.io/gh/microsoft/superbenchmark)
[![Website](https://img.shields.io/website?down_color=lightgrey&url=https%3A%2F%2Faka.ms%2Fsuperbench)](https://aka.ms/superbench)
[![Latest Release](https://img.shields.io/github/release/microsoft/superbenchmark.svg)](https://github.com/microsoft/superbenchmark/releases/latest)
[![Docker Pulls](https://img.shields.io/docker/pulls/superbench/superbench.svg)](https://hub.docker.com/r/superbench/superbench/tags)
[![License](https://img.shields.io/github/license/microsoft/superbenchmark.svg)](LICENSE)
| Azure Pipelines | Build Status |
| :---: | :---: |
| cpu-unit-test | [![Build Status](https://dev.azure.com/msrasrg/SuperBenchmark/_apis/build/status/microsoft.superbenchmark?branchName=main)](https://dev.azure.com/msrasrg/SuperBenchmark/_build/latest?definitionId=77&branchName=main) |
| gpu-unit-test | [![Build Status](https://dev.azure.com/msrasrg/SuperBenchmark/_apis/build/status/cuda-unit-test?branchName=main)](https://dev.azure.com/msrasrg/SuperBenchmark/_build/latest?definitionId=80&branchName=main) |
| cpu-unit-test | [![Build Status](https://dev.azure.com/msrasrg/SuperBenchmark/_apis/build/status/cpu-unit-test?branchName=main)](https://dev.azure.com/msrasrg/SuperBenchmark/_build/latest?definitionId=77&branchName=main) |
| cuda-unit-test | [![Build Status](https://dev.azure.com/msrasrg/SuperBenchmark/_apis/build/status/cuda-unit-test?branchName=main)](https://dev.azure.com/msrasrg/SuperBenchmark/_build/latest?definitionId=80&branchName=main) |
| ansible-integration-test | [![Build Status](https://dev.azure.com/msrasrg/SuperBenchmark/_apis/build/status/ansible-integration-test?branchName=main)](https://dev.azure.com/msrasrg/SuperBenchmark/_build/latest?definitionId=82&branchName=main) |
__SuperBench__ is a validation and profiling tool for AI infrastructure.
_Check SuperBench website for more details._
📢 [v0.2.0](https://github.com/microsoft/superbenchmark/releases/tag/v0.2.0) has been released!
## _Check [aka.ms/superbench](https://aka.ms/superbench) for more details._
## Trademarks
......
<svg width="712" height="603" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><filter id="fx0" x="-10%" y="-10%" width="120%" height="120%" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse"><feComponentTransfer color-interpolation-filters="sRGB"><feFuncR type="discrete" tableValues="0 0"/><feFuncG type="discrete" tableValues="0 0"/><feFuncB type="discrete" tableValues="0 0"/><feFuncA type="linear" slope="0.101961" intercept="0"/></feComponentTransfer><feGaussianBlur stdDeviation="4 4"/></filter><filter id="fx1" x="-10%" y="-10%" width="120%" height="120%" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse"><feComponentTransfer color-interpolation-filters="sRGB"><feFuncR type="discrete" tableValues="0 0"/><feFuncG type="discrete" tableValues="0 0"/><feFuncB type="discrete" tableValues="0 0"/><feFuncA type="linear" slope="0.101961" intercept="0"/></feComponentTransfer><feGaussianBlur stdDeviation="4 4"/></filter><filter id="fx2" x="-10%" y="-10%" width="120%" height="120%" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse"><feComponentTransfer color-interpolation-filters="sRGB"><feFuncR type="discrete" tableValues="0 0"/><feFuncG type="discrete" tableValues="0 0"/><feFuncB type="discrete" tableValues="0 0"/><feFuncA type="linear" slope="0.101961" intercept="0"/></feComponentTransfer><feGaussianBlur stdDeviation="4 4"/></filter><filter id="fx3" x="-10%" y="-10%" width="120%" height="120%" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse"><feComponentTransfer color-interpolation-filters="sRGB"><feFuncR type="discrete" tableValues="0 0"/><feFuncG type="discrete" tableValues="0 0"/><feFuncB type="discrete" tableValues="0 0"/><feFuncA type="linear" slope="0.101961" intercept="0"/></feComponentTransfer><feGaussianBlur stdDeviation="4 4"/></filter><filter id="fx4" x="-10%" y="-10%" width="120%" height="120%" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse"><feComponentTransfer color-interpolation-filters="sRGB"><feFuncR type="discrete" tableValues="0 0"/><feFuncG type="discrete" tableValues="0 0"/><feFuncB type="discrete" tableValues="0 0"/><feFuncA type="linear" slope="0.0705882" intercept="0"/></feComponentTransfer><feGaussianBlur stdDeviation="3.55556 3.55556"/></filter><filter id="fx5" x="-10%" y="-10%" width="120%" height="120%" filterUnits="userSpaceOnUse" primitiveUnits="userSpaceOnUse"><feComponentTransfer color-interpolation-filters="sRGB"><feFuncR type="discrete" tableValues="0 0"/><feFuncG type="discrete" tableValues="0 0"/><feFuncB type="discrete" tableValues="0 0"/><feFuncA type="linear" slope="0.101961" intercept="0"/></feComponentTransfer><feGaussianBlur stdDeviation="4.88889 4.88889"/></filter><clipPath id="clip6"><rect x="284" y="65" width="712" height="603"/></clipPath><clipPath id="clip7"><path d="M571.667 414.5C550.588 414.5 533.5 431.588 533.5 452.667L533.5 605.333C533.5 626.412 550.588 643.5 571.667 643.5L760.333 643.5C781.412 643.5 798.5 626.412 798.5 605.333L798.5 452.667C798.5 431.588 781.412 414.5 760.333 414.5ZM284 65 996 65 996 668 284 668Z" fill-rule="evenodd" clip-rule="evenodd"/></clipPath><clipPath id="clip8"><rect x="-5" y="-5" width="299" height="263"/></clipPath><clipPath id="clip9"><rect x="0" y="0" width="292" height="256"/></clipPath><clipPath id="clip10"><path d="M560.668 407.5C539.588 407.5 522.5 424.588 522.5 445.668L522.5 598.333C522.5 619.412 539.588 636.5 560.668 636.5L753.333 636.5C774.412 636.5 791.5 619.412 791.5 598.333L791.5 445.668C791.5 424.588 774.412 407.5 753.333 407.5ZM284 65 996 65 996 668 284 668Z" fill-rule="evenodd" clip-rule="evenodd"/></clipPath><clipPath id="clip11"><rect x="-5" y="-5" width="305" height="263"/></clipPath><clipPath id="clip12"><rect x="0" y="0" width="296" height="256"/></clipPath><clipPath id="clip13"><path d="M543.834 398.5C522.663 398.5 505.5 415.663 505.5 436.834L505.5 590.166C505.5 611.337 522.663 628.5 543.834 628.5L745.166 628.5C766.337 628.5 783.5 611.337 783.5 590.166L783.5 436.834C783.5 415.663 766.337 398.5 745.166 398.5ZM284 65 996 65 996 668 284 668Z" fill-rule="evenodd" clip-rule="evenodd"/></clipPath><clipPath id="clip14"><rect x="-5" y="-5" width="313" height="266"/></clipPath><clipPath id="clip15"><rect x="0" y="0" width="305" height="257"/></clipPath><clipPath id="clip16"><rect x="-5" y="-5" width="322" height="269"/></clipPath><clipPath id="clip17"><rect x="0" y="0" width="314" height="261"/></clipPath><clipPath id="clip18"><rect x="-5" y="-5" width="321" height="119"/></clipPath><clipPath id="clip19"><rect x="0" y="0" width="313" height="111"/></clipPath><linearGradient x1="496" y1="188.5" x2="785" y2="188.5" gradientUnits="userSpaceOnUse" spreadMethod="reflect" id="fill20"><stop offset="0" stop-color="#DEB3CF"/><stop offset="0.27" stop-color="#DEB3CF"/><stop offset="1" stop-color="#EED8E6"/></linearGradient><clipPath id="clip21"><rect x="-5" y="-5" width="329" height="128"/></clipPath><clipPath id="clip22"><rect x="0" y="0" width="321" height="119"/></clipPath><linearGradient x1="496" y1="308.5" x2="785" y2="308.5" gradientUnits="userSpaceOnUse" spreadMethod="reflect" id="fill23"><stop offset="0" stop-color="#DEB3CF"/><stop offset="0.27" stop-color="#DEB3CF"/><stop offset="1" stop-color="#EED8E6"/></linearGradient></defs><g clip-path="url(#clip6)" transform="translate(-284 -65)"><path d="M459 99.979C459 81.2129 474.213 66 492.979 66L782.021 66C800.787 66 816 81.2129 816 99.979L816 634.021C816 652.787 800.787 668 782.021 668L492.979 668C474.213 668 459 652.787 459 634.021Z" fill="#E5E5E5" fill-rule="evenodd"/><g clip-path="url(#clip7)"><g clip-path="url(#clip8)" filter="url(#fx0)" transform="translate(523 404)"><g clip-path="url(#clip9)"><path d="M13.3285 51.4959C13.3285 30.4166 30.4166 13.3285 51.4959 13.3285L240.161 13.3285C261.24 13.3285 278.328 30.4166 278.328 51.4959L278.328 204.161C278.328 225.24 261.24 242.328 240.161 242.328L51.4959 242.328C30.4166 242.328 13.3285 225.24 13.3285 204.161Z" stroke="#E5E5E5" stroke-miterlimit="8" fill="#BFBFBF" fill-rule="evenodd"/></g></g></g><path d="M533.5 452.667C533.5 431.588 550.588 414.5 571.667 414.5L760.333 414.5C781.412 414.5 798.5 431.588 798.5 452.667L798.5 605.333C798.5 626.412 781.412 643.5 760.333 643.5L571.667 643.5C550.588 643.5 533.5 626.412 533.5 605.333Z" stroke="#E5E5E5" stroke-width="1.33333" stroke-miterlimit="8" fill="#BFBFBF" fill-rule="evenodd"/><g clip-path="url(#clip10)"><g clip-path="url(#clip11)" filter="url(#fx1)" transform="translate(512 397)"><g clip-path="url(#clip12)"><path d="M13.3285 51.496C13.3285 30.4167 30.4166 13.3285 51.496 13.3285L244.161 13.3285C265.24 13.3285 282.328 30.4167 282.328 51.496L282.328 204.161C282.328 225.24 265.24 242.328 244.161 242.328L51.496 242.328C30.4166 242.328 13.3285 225.24 13.3285 204.161Z" stroke="#E5E5E5" stroke-miterlimit="8" fill="#BFBFBF" fill-rule="evenodd"/></g></g></g><path d="M522.5 445.668C522.5 424.588 539.588 407.5 560.668 407.5L753.333 407.5C774.412 407.5 791.5 424.588 791.5 445.668L791.5 598.333C791.5 619.412 774.412 636.5 753.333 636.5L560.668 636.5C539.588 636.5 522.5 619.412 522.5 598.333Z" stroke="#E5E5E5" stroke-width="1.33333" stroke-miterlimit="8" fill="#BFBFBF" fill-rule="evenodd"/><g clip-path="url(#clip13)"><g clip-path="url(#clip14)" filter="url(#fx2)" transform="translate(495 388)"><g clip-path="url(#clip15)"><path d="M13.3285 51.6626C13.3285 30.4913 30.4913 13.3285 51.6626 13.3285L252.994 13.3285C274.166 13.3285 291.328 30.4913 291.328 51.6626L291.328 204.994C291.328 226.166 274.166 243.328 252.994 243.328L51.6626 243.328C30.4913 243.328 13.3285 226.166 13.3285 204.994Z" stroke="#E5E5E5" stroke-miterlimit="8" fill="#BFBFBF" fill-rule="evenodd"/></g></g></g><path d="M505.5 436.834C505.5 415.663 522.663 398.5 543.834 398.5L745.166 398.5C766.337 398.5 783.5 415.663 783.5 436.834L783.5 590.166C783.5 611.337 766.337 628.5 745.166 628.5L543.834 628.5C522.663 628.5 505.5 611.337 505.5 590.166Z" stroke="#E5E5E5" stroke-width="1.33333" stroke-miterlimit="8" fill="#BFBFBF" fill-rule="evenodd"/><g clip-path="url(#clip16)" filter="url(#fx3)" transform="translate(478 376)"><g clip-path="url(#clip17)"><path d="M12.8284 51.996C12.8284 30.3643 30.3643 12.8284 51.996 12.8284L261.661 12.8284C283.293 12.8284 300.828 30.3643 300.828 51.996L300.828 208.661C300.828 230.293 283.293 247.828 261.661 247.828L51.996 247.828C30.3643 247.828 12.8284 230.293 12.8284 208.661Z" fill="#FFFFFF" fill-rule="evenodd"/></g></g><path d="M488 425.168C488 403.536 505.536 386 527.168 386L736.832 386C758.464 386 776 403.536 776 425.168L776 581.832C776 603.464 758.464 621 736.832 621L527.168 621C505.536 621 488 603.464 488 581.832Z" fill="#FFFFFF" fill-rule="evenodd"/><path d="M505.5 429.501C505.5 416.245 516.245 405.5 529.501 405.5L735.5 405.5C748.755 405.5 759.5 416.245 759.5 429.501L759.5 525.5C759.5 538.755 748.755 549.5 735.5 549.5L529.501 549.5C516.245 549.5 505.5 538.755 505.5 525.5Z" stroke="#C377A9" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M641 360 641 376.012 639 376.012 639 360ZM644 374.678 640 380.012 636 374.678Z" fill="#AA4A89"/><path d="M641 238 641 255.456 639 255.456 639 238ZM644 254.122 640 259.456 636 254.122Z" fill="#AA4A89"/><path d="M860 102.5C860 90.0738 870.074 80 882.5 80L972.5 80C984.926 80 995 90.0738 995 102.5L995 354.5C995 366.926 984.926 377 972.5 377L882.5 377C870.074 377 860 366.926 860 354.5Z" fill="#E5E5E5" fill-rule="evenodd"/><text font-family="Segoe UI,Segoe UI_MSFontService,sans-serif" font-weight="400" font-size="21" transform="translate(888.695 237)">Profiling</text><path d="M285 102.167C285 89.9246 294.925 80 307.167 80L395.833 80C408.075 80 418 89.9246 418 102.167L418 354.833C418 367.075 408.075 377 395.833 377L307.167 377C294.925 377 285 367.075 285 354.833Z" fill="#E5E5E5" fill-rule="evenodd"/><text font-family="Segoe UI,Segoe UI_MSFontService,sans-serif" font-weight="400" font-size="21" transform="translate(304.866 237)">Validation</text><path d="M639.5 487.5C639.5 482.53 643.53 478.5 648.5 478.5L741.5 478.5C746.471 478.5 750.5 482.53 750.5 487.5L750.5 523.5C750.5 528.471 746.471 532.5 741.5 532.5L648.5 532.5C643.53 532.5 639.5 528.471 639.5 523.5Z" stroke="#C377A9" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><text fill="#AA4A89" font-family="Microsoft YaHei,Microsoft YaHei_MSFontService,sans-serif" font-weight="400" font-size="16" transform="translate(669.563 500)">Model <tspan font-size="16" x="-18" y="20">Benchmark</tspan></text><path d="M285 416.167C285 403.925 294.925 394 307.167 394L395.833 394C408.075 394 418 403.925 418 416.167L418 631.833C418 644.075 408.075 654 395.833 654L307.167 654C294.925 654 285 644.075 285 631.833Z" fill="#E5E5E5" fill-rule="evenodd"/><text font-family="Segoe UI,Segoe UI_MSFontService,sans-serif" font-weight="400" font-size="21" transform="translate(305.849 532)">Diagnosis</text><path d="M860 416.5C860 404.074 870.074 394 882.5 394L972.5 394C984.926 394 995 404.074 995 416.5L995 631.5C995 643.926 984.926 654 972.5 654L882.5 654C870.074 654 860 643.926 860 631.5Z" fill="#E5E5E5" fill-rule="evenodd"/><text font-family="Segoe UI,Segoe UI_MSFontService,sans-serif" font-weight="400" font-size="21" transform="translate(877.861 532)">Simulation</text><path d="M352 377 352 390.261 350 390.261 350 377ZM355 388.927 351 394.261 347 388.927Z" fill="#AA4A89"/><path d="M346.455 144C346.455 141.24 348.715 139 351.5 139 354.285 139 356.545 141.24 356.545 144 356.545 145.667 356.545 145.667 356.545 145.667 363.273 145.667 363.273 145.667 363.273 145.667 363.273 152.333 363.273 152.333 363.273 152.333 339.727 152.333 339.727 152.333 339.727 152.333 339.727 145.667 339.727 145.667 339.727 145.667 346.455 145.667 346.455 145.667 346.455 145.667L346.455 144ZM339.727 145.667C333 145.667 333 145.667 333 145.667 333 189 333 189 333 189 370 189 370 189 370 189 370 145.667 370 145.667 370 145.667 363.273 145.667 363.273 145.667 363.273 145.667M364.955 162.333C351.5 162.333 351.5 162.333 351.5 162.333M364.955 172.333C351.5 172.333 351.5 172.333 351.5 172.333M364.955 182.333C351.5 182.333 351.5 182.333 351.5 182.333M339.727 159C343.091 162.333 343.091 162.333 343.091 162.333 348.136 157.333 348.136 157.333 348.136 157.333M339.727 169C343.091 172.333 343.091 172.333 343.091 172.333 348.136 167.333 348.136 167.333 348.136 167.333M339.727 179C343.091 182.333 343.091 182.333 343.091 182.333 348.136 177.333 348.136 177.333 348.136 177.333" stroke="#AA4A89" stroke-width="1.66667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M926.616 170.616C925.448 171.929 923.552 171.929 922.384 170.616 921.071 169.448 921.071 167.552 922.384 166.384 923.552 165.217 925.448 165.071 926.616 166.384 927.929 167.552 927.929 169.448 926.616 170.616ZM924.427 148C913.192 148 904 157.192 904 168.573 904 179.808 913.192 189 924.427 189 935.808 189 945 179.808 945 168.573 945 157.192 935.808 148 924.427 148ZM935.224 178.641C937.705 176.014 939.164 172.512 939.164 168.573 939.164 166.676 938.872 164.925 938.288 163.32M929.388 154.566C927.783 154.128 926.178 153.836 924.427 153.836 916.402 153.836 909.836 160.402 909.836 168.573 909.836 172.512 911.295 176.014 913.776 178.641M934.932 158.068C926.907 166.093 926.907 166.093 926.907 166.093" stroke="#AA4A89" stroke-width="1.66667" stroke-linecap="square" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M358.202 489.622C356.719 491.098 354.292 492.037 352.404 492.573 352.404 492.573 352.404 492.573 352.404 497 352.404 497 352.404 497 344.315 497 344.315 497 344.315 497 344.315 492.037 340.944 491.5 338.112 489.622 335.685 487.073 335.685 487.073 335.685 487.073 331.775 489.085 331.775 489.085 331.775 489.085 328 481.707 328 481.707 328 481.707 331.775 479.695 331.371 478.22 330.831 476.744 330.831 474.732 330.831 473.256 331.371 471.244 331.775 469.768 331.775 469.768 331.775 469.768 328 467.89 328 467.89 328 467.89 331.775 460.378 331.775 460.378 331.775 460.378 335.685 462.39 338.112 459.976 340.944 457.963 344.315 457.427 344.315 457.427 344.315 457.427 344.315 453 344.315 453 344.315 453 352.404 453 352.404 453 352.404 453 352.404 457.024 355.775 457.963 359.146 459.976 361.573 462.39 361.573 462.39 361.573 462.39 364.944 460.378 364.944 460.378 364.944 460.378 369.258 467.89 369.258 467.89 369.258 467.89 365.348 469.768 365.888 471.244 366.022 473.122 366.022 474.732 366.022 475.671 366.022 476.878 365.618 478.22M353.079 466.415C351.73 465.476 350.247 464.939 348.225 464.939 342.427 464.939 338.112 469.902 338.112 475.268 338.112 479.159 339.461 482.11 342.427 483.988M356.18 484.927C369.798 493.512 369.798 493.512 369.798 493.512 371.551 494.585 373.843 494.183 374.921 492.439 376 490.829 375.596 488.549 373.843 487.341 359.955 479.159 359.955 479.159 359.955 479.159 359.685 479.024 359.685 479.024 359.685 479.024 359.82 478.354 359.685 477.683 359.551 477.012 358.742 473.256 354.966 470.707 351.191 471.646 349.843 471.915 348.494 472.585 347.551 473.524 354.157 477.549 354.157 477.549 354.157 477.549 352.539 479.695 352.539 479.695 352.539 479.695 346.202 475.671 346.202 475.671 346.202 475.671 345.663 477.012 345.393 478.488 345.798 479.963 346.607 483.72 350.247 486.134 354.157 485.329 354.697 485.195 355.371 485.061 355.91 484.793L356.18 484.927Z" stroke="#AA4A89" stroke-width="1.66667" stroke-linecap="square" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M904.143 474.341C904.143 473.324 904 472.307 904 471.29 904 459.816 913.175 450.52 924.5 450.52 925.503 450.52 926.364 450.665 927.224 450.665M918.766 491.335C920.629 491.771 922.493 492.061 924.5 492.061 935.825 492.061 945 482.765 945 471.29 945 470.419 945 469.547 944.857 468.821M924.5 487.704C918.622 491.19 918.622 491.19 918.622 491.19 922.206 497 922.206 497 922.206 497M921.49 454.296C927.367 450.81 927.367 450.81 927.367 450.81 923.783 445 923.783 445 923.783 445M941.273 456.184C932.815 456.184 932.815 456.184 932.815 456.184 932.815 464.609 932.815 464.609 932.815 464.609 941.273 464.609 941.273 464.609 941.273 464.609L941.273 456.184ZM909.878 488.721C913.175 488.721 915.899 486.106 915.899 482.765 915.899 479.425 913.175 476.81 909.878 476.81 906.724 476.81 904 479.425 904 482.765 904 486.106 906.724 488.721 909.878 488.721Z" stroke="#AA4A89" stroke-width="1.66667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M4.50763e-06-1 19.2909-0.999913 19.2909 1.00009-4.50763e-06 1ZM17.9576-3.99992 23.2909 0.000104987 17.9576 4.00008Z" fill="#AA4A89" transform="matrix(-1 0 0 1 449.291 229)"/><path d="M426 496 445.297 496 445.297 498 426 498ZM443.964 493 449.297 497 443.964 501Z" fill="#AA4A89"/><path d="M824 228 843.04 228 843.04 230 824 230ZM841.707 225 847.04 229 841.707 233Z" fill="#AA4A89"/><path d="M824 488 843.04 488 843.04 490 824 490ZM841.707 485 847.04 489 841.707 493Z" fill="#AA4A89"/><path d="M929 377 929 390.261 927 390.261 927 377ZM932 388.927 928 394.261 924 388.927Z" fill="#AA4A89"/><path d="M527.5 436.5C527.5 422.693 538.469 411.5 552 411.5 565.531 411.5 576.5 422.693 576.5 436.5 576.5 450.307 565.531 461.5 552 461.5 538.469 461.5 527.5 450.307 527.5 436.5Z" stroke="#AA4A89" stroke-width="1.33333" stroke-miterlimit="8" fill="#F8F7F7" fill-rule="evenodd"/><path d="M556.5 443C555.807 443 555.187 443.28 554.733 443.733 554.28 444.187 554 444.807 554 445.5 554 446.193 554.28 446.813 554.733 447.267 555.187 447.72 555.807 448 556.5 448M541.5 448C540.807 448 540.187 447.72 539.733 447.267 539.28 446.813 539 446.193 539 445.5 539 444.807 539.28 444.187 539.733 443.733 540.187 443.28 540.807 443 541.5 443 556.5 443 556.5 443 556.5 443M542.333 442.167C542.333 425.5 542.333 425.5 542.333 425.5M559 428C561.5 428 561.5 428 561.5 428 562.88 428 564 426.88 564 425.5 564 424.12 562.88 423 561.5 423 560.12 423 559 424.12 559 425.5 559 438.833 559 438.833 559 438.833M561.44 423C544.833 423 544.833 423 544.833 423 543.453 423 542.333 424.12 542.333 425.5M559 438.833C559 445.5 559 445.5 559 445.5M541.5 448C556.5 448 556.5 448 556.5 448 557.88 448 559 446.88 559 445.5M549.833 425.5C554.833 430.5 554.833 430.5 554.833 430.5M554.833 425.5C549.833 430.5 549.833 430.5 549.833 430.5M545.667 435.5C545.667 437.167 545.667 437.167 545.667 437.167M547.92 432.413C546.747 433.587 546.747 433.587 546.747 433.587M546.747 439.08C547.92 440.253 547.92 440.253 547.92 440.253" stroke="#AA4A89" stroke-width="1.66667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><text fill="#AA4A89" font-family="Microsoft YaHei,Microsoft YaHei_MSFontService,sans-serif" font-weight="700" font-size="21" transform="translate(616.43 444)">Executor</text><g clip-path="url(#clip18)" filter="url(#fx4)" transform="translate(487 136)"><g clip-path="url(#clip19)"><path d="M11.8284 32.3075C11.8284 20.9972 20.9972 11.8284 32.3075 11.8284L280.349 11.8284C291.66 11.8284 300.828 20.9972 300.828 32.3075L300.828 78.3494C300.828 89.6596 291.66 98.8284 280.349 98.8284L32.3075 98.8284C20.9972 98.8284 11.8284 89.6596 11.8284 78.3494Z" fill="#FF0000" fill-rule="evenodd"/></g></g><path d="M496 165.479C496 154.169 505.169 145 516.479 145L764.521 145C775.831 145 785 154.169 785 165.479L785 211.521C785 222.831 775.831 232 764.521 232L516.479 232C505.169 232 496 222.831 496 211.521Z" fill="url(#fill20)" fill-rule="evenodd"/><path d="M527 188.5C527 174.969 537.969 164 551.5 164 565.031 164 576 174.969 576 188.5 576 202.031 565.031 213 551.5 213 537.969 213 527 202.031 527 188.5Z" fill="#F8F7F7" fill-rule="evenodd"/><path d="M565 198C538 198 538 198 538 198 538 176 538 176 538 176 565 176 565 176 565 176L565 198ZM538 181.5C565 181.5 565 181.5 565 181.5M562.012 178.75C562.012 178.911 562.142 179.043 562.3 179.043 562.458 179.043 562.581 178.911 562.581 178.75 562.581 178.589 562.458 178.464 562.3 178.464 562.142 178.464 562.012 178.589 562.012 178.75ZM558.779 178.75C558.779 178.911 558.909 179.043 559.06 179.043 559.218 179.043 559.348 178.911 559.348 178.75 559.348 178.589 559.218 178.464 559.06 178.464 558.909 178.464 558.779 178.589 558.779 178.75ZM555.546 178.75C555.546 178.911 555.669 179.043 555.827 179.043 555.986 179.043 556.115 178.911 556.115 178.75 556.115 178.589 555.986 178.464 555.827 178.464 555.669 178.464 555.546 178.589 555.546 178.75Z" stroke="#AA4A89" stroke-width="2" stroke-linecap="square" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><text fill="#AA4A89" font-family="Microsoft YaHei,Microsoft YaHei_MSFontService,sans-serif" font-weight="700" font-size="21" transform="translate(580.947 185)">Command<tspan font-size="21" x="110.667" y="0">-</tspan><tspan font-size="21" x="120" y="0">Line </tspan><tspan font-size="21" x="34.02" y="25">Interface</tspan><tspan font-size="21" x="52.7499" y="409">Node</tspan></text><path d="M527 586C527 572.193 537.969 561 551.5 561 565.031 561 576 572.193 576 586 576 599.807 565.031 611 551.5 611 537.969 611 527 599.807 527 586Z" fill="#F8F7F7" fill-rule="evenodd"/><path d="M566 585.323 566 598 553 598 553 574 566 574 566 585.323M555.779 577.871 563.122 577.871M555.779 590.452 563.122 590.452M555.779 594.032 563.122 594.032" stroke="#AA4A89" stroke-width="1.66667" stroke-linecap="square" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M550 585.323 550 598 538 598 538 574 550 574 550 585.323M540.565 577.871 547.344 577.871M540.565 590.452 547.344 590.452M540.565 594.032 547.344 594.032" stroke="#AA4A89" stroke-width="1.66667" stroke-linecap="square" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><text fill="#AA4A89" font-family="Segoe UI,Segoe UI_MSFontService,sans-serif" font-weight="700" font-size="27" transform="translate(565.599 115)">SuperBench</text><g clip-path="url(#clip21)" filter="url(#fx5)" transform="translate(483 252)"><g clip-path="url(#clip22)"><path d="M15.8284 30.3287C15.8284 22.3204 22.3204 15.8284 30.3287 15.8284L290.328 15.8284C298.336 15.8284 304.828 22.3204 304.828 30.3287L304.828 88.3282C304.828 96.3364 298.336 102.828 290.328 102.828L30.3287 102.828C22.3204 102.828 15.8284 96.3364 15.8284 88.3282Z" fill="#FF0000" fill-rule="evenodd"/></g></g><path d="M496 279.5C496 271.492 502.492 265 510.5 265L770.5 265C778.508 265 785 271.492 785 279.5L785 337.5C785 345.508 778.508 352 770.5 352L510.5 352C502.492 352 496 345.508 496 337.5Z" fill="url(#fill23)" fill-rule="evenodd"/><path d="M528 309C528 295.193 538.521 284 551.5 284 564.479 284 575 295.193 575 309 575 322.807 564.479 334 551.5 334 538.521 334 528 322.807 528 309Z" fill="#F8F7F7" fill-rule="evenodd"/><path d="M567 301.784 567 310.033 539 310.033 539 294 567 294 567 301.784M564.6 319.793 563.229 317.701 542.771 317.701 539.8 322 566.2 322 564.6 319.793M548.029 313.983 557.857 313.983M553.057 310.149 553.057 313.867" stroke="#AA4A89" stroke-width="2" stroke-linecap="square" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><text fill="#AA4A89" font-family="Microsoft YaHei,Microsoft YaHei_MSFontService,sans-serif" font-weight="700" font-size="21" transform="translate(624.197 317)">Runner</text><path d="M515.5 487.5C515.5 482.53 519.53 478.5 524.5 478.5L617.5 478.5C622.471 478.5 626.5 482.53 626.5 487.5L626.5 523.5C626.5 528.471 622.471 532.5 617.5 532.5L524.5 532.5C519.53 532.5 515.5 528.471 515.5 523.5Z" stroke="#C377A9" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><text fill="#AA4A89" font-family="Microsoft YaHei,Microsoft YaHei_MSFontService,sans-serif" font-weight="400" font-size="16" transform="translate(548.456 501)">Micro <tspan font-size="16" x="-20.6133" y="20">Benchmark</tspan></text></g></svg>
......@@ -80,14 +80,26 @@ superbench:
parameters:
<<: *common_model_config
batch_size: 128
cnn_models:
resnet_models:
<<: *default_pytorch_mode
models:
- resnet50
- resnet101
- resnet152
parameters:
<<: *common_model_config
batch_size: 128
densenet_models:
<<: *default_pytorch_mode
models:
- densenet169
- densenet201
parameters:
<<: *common_model_config
batch_size: 128
vgg_models:
<<: *default_pytorch_mode
models:
- vgg11
- vgg13
- vgg16
......@@ -99,26 +111,24 @@ superbench:
By default, all benchmarks in default configuration will be run if you don't specify customized configuration.
If you want to have a quick try, you can modify this config a little bit. For example, only run resnet models.
If you want to have a quick try, you can modify this config a little bit. For example, only run resnet101 model.
1. copy the default config to a file named `resnet.yaml` in current path.
```bash
cp superbench/config/default.yaml resnet.yaml
```
2. enable only `cnn_models` in the config and remove other models except resnet under `benchmarks.cnn_models.models`.
```yaml {3,10-13} title="resnet.yaml"
2. enable only `resnet_models` in the config and remove other models except resnet101 under `benchmarks.resnet_models.models`.
```yaml {3,11} title="resnet.yaml"
# SuperBench Config
superbench:
enable: ['cnn_models']
enable: ['resnet_models']
var:
# ...
# omit the middle part
# ...
cnn_models:
resnet_models:
<<: *default_pytorch_mode
models:
- resnet50
- resnet101
- resnet152
parameters:
<<: *common_model_config
batch_size: 128
......
......@@ -54,6 +54,12 @@ deactivate
You can clone the source from GitHub and build it.
:::note Note
You should checkout corresponding tag to use release version, for example,
`git clone -b v0.2.0 https://github.com/microsoft/superbenchmark`
:::
```bash
git clone https://github.com/microsoft/superbenchmark
cd superbenchmark
......
......@@ -24,6 +24,12 @@ or your private key requires a passphase before use, you can do
sb deploy -f remote.ini --host-password [password]
```
:::note Note
You should deploy corresponding Docker image to use release version, for example,
`sb deploy -f local.ini -i superbench/superbench:v0.2.0-cuda11.1.1`
:::
## Run
After deployment, you can start to run the SuperBench benchmarks on all managed nodes using `sb run` command.
......
......@@ -26,4 +26,4 @@ as well as model-benchmark to measure domain-aware end-to-end deep learning work
The following figure shows the capabilities provided by SuperBench core framework and its extension.
![SuperBench Structure](./assets/architecture.png)
![SuperBench Structure](./assets/architecture.svg)
......@@ -6,5 +6,5 @@
Provide hardware and software benchmarks for AI systems.
"""
__version__ = '0.0.0'
__version__ = '0.2.0'
__author__ = 'Microsoft'
# SuperBench Config
version: v0.2
superbench:
enable: null
var:
......
# SuperBench Config
version: v0.2
superbench:
enable: null
var:
......
......@@ -15,11 +15,11 @@ This blog is to introduce [SuperBench](https://github.com/microsoft/superbenchma
### Easy-to-use CLI
In order to provide good user experience, SuperBench provides a command line interface to help users deploy and run benchmarks.
Empowered by SuperBench CLI, user can deploy and run their benchmarks with only one command, which greatly shorten the learning curve of using tools,
In order to provide good user experience, SuperBench provides a command line interface to help users deploy and run benchmarks.
Empowered by SuperBench CLI, user can deploy and run their benchmarks with only one command, which greatly shorten the learning curve of using tools,
to help user easily evaluate the performance of AI workload.
Below is a simple example to show how to deploy and run benchmarks locally. For more information,
Below is a simple example to show how to deploy and run benchmarks locally. For more information,
please view [CLI Document](https://microsoft.github.io/superbenchmark/docs/cli)
1. Deploy
......@@ -48,9 +48,9 @@ For more information, please view [configuration](https://microsoft.github.io/su
1. Executor Framework
In order to facilitate the benchmarking and validation on large-scale clusters, we designed and implemented a modular and extensible framework.
SuperBench framework includes a runner as control node, as well as multiple executors as worker nodes.
A runner received commands from CLI and distribute to all nodes (worker nodes) in the cluster, collect data, and summarize the results.
In order to facilitate the benchmarking and validation on large-scale clusters, we designed and implemented a modular and extensible framework.
SuperBench framework includes a runner as control node, as well as multiple executors as worker nodes.
A runner received commands from CLI and distribute to all nodes (worker nodes) in the cluster, collect data, and summarize the results.
Each worker will run executor to execute the specified benchmark tasks.
![SuperBench Executor Workflow](../../docs/assets/executor_workflow.png)
......@@ -88,7 +88,7 @@ SuperBench supports a set of benchmarks listed as below.
* BERT models
* GPT-2 models
For the details of each benchmark, please view [micro-benchmarks](https://microsoft.github.io/superbenchmark/docs/benchmarks/micro-benchmarks.md)
For the details of each benchmark, please view [micro-benchmarks](https://microsoft.github.io/superbenchmark/docs/benchmarks/micro-benchmarks.md)
and [model-benchmarks](https://microsoft.github.io/superbenchmark/docs/benchmarks/model-benchmarks.md).
......@@ -96,7 +96,7 @@ and [model-benchmarks](https://microsoft.github.io/superbenchmark/docs/benchmark
We want to extend SuperBench capability to distributed validation and auto-diagnosis, to build a benchmarking eco-system.
The following figure shows the whole picture.
![SuperBench Capabilities and Extension](../../docs/assets/architecture.png)
![SuperBench Capabilities and Extension](../../docs/assets/architecture.svg)
With SuperBench and its extensions, we can support:
......@@ -111,4 +111,4 @@ With SuperBench and its extensions, we can support:
## Call for Contributor
This project welcomes contributions and suggestions.
This project welcomes contributions and suggestions.
......@@ -101,6 +101,7 @@ module.exports = {
announcementBar: {
id: 'supportus',
content:
'📢 <a href="https://microsoft.github.io/superbenchmark/blog/release-sb-v0.2">v0.2</a> has been released! ' +
'⭐️ If you like SuperBench, give it a star on <a target="_blank" rel="noopener noreferrer" href="https://github.com/microsoft/superbenchmark">GitHub</a>! ⭐️',
},
prism: {
......
{
"name": "superbench-website",
"version": "0.0.0",
"version": "0.2.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
......
{
"name": "superbench-website",
"version": "0.0.0",
"version": "0.2.0",
"private": true,
"scripts": {
"docusaurus": "docusaurus",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment