"vscode:/vscode.git/clone" did not exist on "8d0a01a5f2b53794e4bc6b734d7b63cb8a9b7d7d"
Unverified Commit b942c094 authored by Harry Mellor's avatar Harry Mellor Committed by GitHub
Browse files

Stop using title frontmatter and fix doc that can only be reached by search (#20623)


Signed-off-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent b4bab816
...@@ -55,6 +55,7 @@ nav: ...@@ -55,6 +55,7 @@ nav:
- contributing/model/registration.md - contributing/model/registration.md
- contributing/model/tests.md - contributing/model/tests.md
- contributing/model/multimodal.md - contributing/model/multimodal.md
- CI: contributing/ci
- Design Documents: - Design Documents:
- V0: design - V0: design
- V1: design/v1 - V1: design/v1
......
--- # Contact Us
title: Contact Us
---
--8<-- "README.md:contact-us" --8<-- "README.md:contact-us"
--- # Meetups
title: Meetups
---
We host regular meetups in San Francisco Bay Area every 2 months. We will share the project updates from the vLLM team and have guest speakers from the industry to share their experience and insights. Please find the materials of our previous meetups below: We host regular meetups in San Francisco Bay Area every 2 months. We will share the project updates from the vLLM team and have guest speakers from the industry to share their experience and insights. Please find the materials of our previous meetups below:
......
--- # Engine Arguments
title: Engine Arguments
---
Engine arguments control the behavior of the vLLM engine. Engine arguments control the behavior of the vLLM engine.
......
--- # Server Arguments
title: Server Arguments
---
The `vllm serve` command is used to launch the OpenAI-compatible server. The `vllm serve` command is used to launch the OpenAI-compatible server.
......
--- # Benchmark Suites
title: Benchmark Suites
---
vLLM contains two sets of benchmarks: vLLM contains two sets of benchmarks:
......
--- # Update PyTorch version on vLLM OSS CI/CD
title: Update PyTorch version on vLLM OSS CI/CD
---
vLLM's current policy is to always use the latest PyTorch stable vLLM's current policy is to always use the latest PyTorch stable
release in CI/CD. It is standard practice to submit a PR to update the release in CI/CD. It is standard practice to submit a PR to update the
......
--- # Summary
title: Summary
---
!!! important !!! important
Many decoder language models can now be automatically loaded using the [Transformers backend][transformers-backend] without having to implement them in vLLM. See if `vllm serve <model>` works first! Many decoder language models can now be automatically loaded using the [Transformers backend][transformers-backend] without having to implement them in vLLM. See if `vllm serve <model>` works first!
......
--- # Basic Model
title: Basic Model
---
This guide walks you through the steps to implement a basic vLLM model. This guide walks you through the steps to implement a basic vLLM model.
......
--- # Multi-Modal Support
title: Multi-Modal Support
---
This document walks you through the steps to extend a basic model so that it accepts [multi-modal inputs](../../features/multimodal_inputs.md). This document walks you through the steps to extend a basic model so that it accepts [multi-modal inputs](../../features/multimodal_inputs.md).
......
--- # Registering a Model
title: Registering a Model
---
vLLM relies on a model registry to determine how to run each model. vLLM relies on a model registry to determine how to run each model.
A list of pre-registered architectures can be found [here](../../models/supported_models.md). A list of pre-registered architectures can be found [here](../../models/supported_models.md).
......
--- # Unit Testing
title: Unit Testing
---
This page explains how to write unit tests to verify the implementation of your model. This page explains how to write unit tests to verify the implementation of your model.
......
--- # Using Docker
title: Using Docker
---
[](){ #deployment-docker-pre-built-image } [](){ #deployment-docker-pre-built-image }
......
--- # Anyscale
title: Anyscale
---
[](){ #deployment-anyscale } [](){ #deployment-anyscale }
[Anyscale](https://www.anyscale.com) is a managed, multi-cloud platform developed by the creators of Ray. [Anyscale](https://www.anyscale.com) is a managed, multi-cloud platform developed by the creators of Ray.
......
--- # Anything LLM
title: Anything LLM
---
[Anything LLM](https://github.com/Mintplex-Labs/anything-llm) is a full-stack application that enables you to turn any document, resource, or piece of content into context that any LLM can use as references during chatting. [Anything LLM](https://github.com/Mintplex-Labs/anything-llm) is a full-stack application that enables you to turn any document, resource, or piece of content into context that any LLM can use as references during chatting.
......
--- # AutoGen
title: AutoGen
---
[AutoGen](https://github.com/microsoft/autogen) is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans. [AutoGen](https://github.com/microsoft/autogen) is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans.
......
--- # BentoML
title: BentoML
---
[BentoML](https://github.com/bentoml/BentoML) allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-compliant image and deploy it on Kubernetes. [BentoML](https://github.com/bentoml/BentoML) allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-compliant image and deploy it on Kubernetes.
......
--- # Cerebrium
title: Cerebrium
---
<p align="center"> <p align="center">
<img src="https://i.ibb.co/hHcScTT/Screenshot-2024-06-13-at-10-14-54.png" alt="vLLM_plus_cerebrium"/> <img src="https://i.ibb.co/hHcScTT/Screenshot-2024-06-13-at-10-14-54.png" alt="vLLM_plus_cerebrium"/>
......
--- # Chatbox
title: Chatbox
---
[Chatbox](https://github.com/chatboxai/chatbox) is a desktop client for LLMs, available on Windows, Mac, Linux. [Chatbox](https://github.com/chatboxai/chatbox) is a desktop client for LLMs, available on Windows, Mac, Linux.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment