feat: OpenAI compatible http service (#123)

Signed-off-by: Ryan Olson <ryanolson@users.noreply.github.com> Co-authored-by: Ryan McCormick <rmccormick@nvidia.com> Co-authored-by: Neelay Shah <neelays@nvidia.com>

feat: OpenAI compatible http service (#123)
Signed-off-by: Ryan Olson <ryanolson@users.noreply.github.com> Co-authored-by: Ryan McCormick <rmccormick@nvidia.com> Co-authored-by: Neelay Shah <neelays@nvidia.com>
ffc6dde1 · Ryan Olson · GitHub · 9d6643b7 · ffc6dde1 · ffc6dde1
Commit ffc6dde1 authored Feb 10, 2025 by Ryan Olson Committed by GitHub Feb 10, 2025
13 changed files
--- a/llm/rust/triton-llm/tests/snapshots/openai_completions__valid_samples-8.snap
+++ b/llm/rust/triton-llm/tests/snapshots/openai_completions__valid_samples-8.snap
--- a/llm/rust/triton-llm/tests/snapshots/openai_completions__valid_samples-9.snap
+++ b/llm/rust/triton-llm/tests/snapshots/openai_completions__valid_samples-9.snap
--- a/llm/rust/triton-llm/tests/snapshots/openai_completions__valid_samples.snap
+++ b/llm/rust/triton-llm/tests/snapshots/openai_completions__valid_samples.snap
--- a/runtime/rust/Cargo.lock
+++ b/runtime/rust/Cargo.lock
--- a/runtime/rust/Cargo.toml
+++ b/runtime/rust/Cargo.toml
--- a/runtime/rust/examples/hello_world/Cargo.toml
+++ b/runtime/rust/examples/hello_world/Cargo.toml
--- a/runtime/rust/examples/hello_world/rust-toolchain.toml
+++ b/runtime/rust/examples/hello_world/rust-toolchain.toml
--- a/runtime/rust/python-wheel/Cargo.lock
+++ b/runtime/rust/python-wheel/Cargo.lock
--- a/runtime/rust/python-wheel/Cargo.toml
+++ b/runtime/rust/python-wheel/Cargo.toml
--- a/runtime/rust/python-wheel/pyproject.toml
+++ b/runtime/rust/python-wheel/pyproject.toml
--- a/runtime/rust/python-wheel/rust-toolchain.toml
+++ b/runtime/rust/python-wheel/rust-toolchain.toml
--- a/runtime/rust/rust-toolchain.toml
+++ b/runtime/rust/rust-toolchain.toml
--- a/runtime/rust/src/pipeline/context.rs
+++ b/runtime/rust/src/pipeline/context.rs