Unverified Commit 6df844b8 authored by binmakeswell's avatar binmakeswell Committed by GitHub
Browse files

[release] grok-1 314b inference (#5490)

* [release] grok-1 inference

* [release] grok-1 inference

* [release] grok-1 inference
parent 848a574c
...@@ -25,6 +25,7 @@ ...@@ -25,6 +25,7 @@
</div> </div>
## Latest News ## Latest News
* [2024/03] [Grok-1 of PyTorch + HuggingFace version is now available!](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
* [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0) * [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0)
* [2024/03] [Open-Sora:Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora) * [2024/03] [Open-Sora:Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora)
* [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer) * [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer)
...@@ -72,6 +73,7 @@ ...@@ -72,6 +73,7 @@
<li> <li>
<a href="#Inference">Inference</a> <a href="#Inference">Inference</a>
<ul> <ul>
<li><a href="#Grok-1">Grok-1: 314B model of PyTorch + HuggingFace Inference</a></li>
<li><a href="#SwiftInfer">SwiftInfer:Breaks the Length Limit of LLM for Multi-Round Conversations with 46% Acceleration</a></li> <li><a href="#SwiftInfer">SwiftInfer:Breaks the Length Limit of LLM for Multi-Round Conversations with 46% Acceleration</a></li>
<li><a href="#GPT-3-Inference">GPT-3</a></li> <li><a href="#GPT-3-Inference">GPT-3</a></li>
<li><a href="#OPT-Serving">OPT-175B Online Serving for Text Generation</a></li> <li><a href="#OPT-Serving">OPT-175B Online Serving for Text Generation</a></li>
...@@ -365,6 +367,12 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt ...@@ -365,6 +367,12 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt
## Inference ## Inference
### Grok-1
An easy-to-use Python + PyTorch + HuggingFace version of 314B Grok-1 Inference.
[[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1)
[[blog]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
[[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1)
<p id="SwiftInfer" align="center"> <p id="SwiftInfer" align="center">
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/SwiftInfer.jpg" width=800/> <img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/SwiftInfer.jpg" width=800/>
</p> </p>
......
...@@ -24,6 +24,7 @@ ...@@ -24,6 +24,7 @@
</div> </div>
## 新闻 ## 新闻
* [2024/03] [Grok-1 of PyTorch + HuggingFace version is now available!](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
* [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0) * [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0)
* [2024/03] [Open-Sora:Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora) * [2024/03] [Open-Sora:Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora)
* [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer) * [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer)
...@@ -71,6 +72,7 @@ ...@@ -71,6 +72,7 @@
<li> <li>
<a href="#推理">推理</a> <a href="#推理">推理</a>
<ul> <ul>
<li><a href="#Grok-1">Grok-1: 3140亿参数PyTorch + HuggingFace推理</a></li>
<li><a href="#SwiftInfer">SwiftInfer:打破LLM多轮对话的长度限制,推理加速46%</a></li> <li><a href="#SwiftInfer">SwiftInfer:打破LLM多轮对话的长度限制,推理加速46%</a></li>
<li><a href="#GPT-3-Inference">GPT-3</a></li> <li><a href="#GPT-3-Inference">GPT-3</a></li>
<li><a href="#OPT-Serving">1750亿参数OPT在线推理服务</a></li> <li><a href="#OPT-Serving">1750亿参数OPT在线推理服务</a></li>
...@@ -358,6 +360,12 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的 ...@@ -358,6 +360,12 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的
## 推理 ## 推理
### Grok-1
方便易用的Python + PyTorch + HuggingFace Grok-1 推理
[[代码]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1)
[[博客]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
[[HuggingFace Grok-1 PyTorch模型权重]](https://huggingface.co/hpcai-tech/grok-1)
<p id="SwiftInfer" align="center"> <p id="SwiftInfer" align="center">
<img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/SwiftInfer.jpg" width=800/> <img src="https://raw.githubusercontent.com/hpcaitech/public_assets/main/colossalai/img/SwiftInfer.jpg" width=800/>
</p> </p>
......
# Grok-1 Inference # Grok-1 Inference
An easy-to-use Python + PyTorch + HuggingFace version of 314B Grok-1.
[[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1)
[[blog]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available)
[[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1)
## Install ## Install
```bash ```bash
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment