Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
675a9bf5
Unverified
Commit
675a9bf5
authored
Apr 18, 2025
by
Graham King
Committed by
GitHub
Apr 18, 2025
Browse files
chore: Remove TRT-LLM C++ engine in favor of Python one (#747)
parent
d797b4ba
Changes
42
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
0 additions
and
62 deletions
+0
-62
lib/engines/trtllm/src/executor/protocols/stats.rs
lib/engines/trtllm/src/executor/protocols/stats.rs
+0
-22
lib/engines/trtllm/src/lib.rs
lib/engines/trtllm/src/lib.rs
+0
-40
No files found.
lib/engines/trtllm/src/executor/protocols/stats.rs
deleted
100644 → 0
View file @
d797b4ba
// SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use
super
::
kv
::
ForwardPassMetrics
;
use
serde
::{
Deserialize
,
Serialize
};
#[derive(Serialize,
Deserialize,
Debug,
Clone)]
pub
struct
IterStats
{
pub
stats
:
Vec
<
ForwardPassMetrics
>
,
}
lib/engines/trtllm/src/lib.rs
deleted
100644 → 0
View file @
d797b4ba
// SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use
std
::
sync
::
Arc
;
use
dynamo_llm
::
backend
::
ExecutionContext
;
use
dynamo_runtime
::
pipeline
::
error
as
pipeline_error
;
pub
mod
executor
;
/// Create a TRT-LLM engine.
pub
fn
make_engine
<
P
:
ToString
>
(
// A full repo with .engine files, config.json,
model_path
:
P
,
// How many GPUs to use
tensor_parallel_size
:
u32
,
)
->
pipeline_error
::
Result
<
ExecutionContext
>
{
let
config
=
executor
::
config
::
ExecutorConfig
::
builder
()
.model_path
(
model_path
.to_string
())
.tensor_parallel_size
(
Some
(
tensor_parallel_size
))
.build
()
?
;
let
engine
=
executor
::
Executor
::
new
(
config
)
?
;
engine
.start_response_processor
();
engine
.start_kv_event_processor
();
engine
.start_iteration_metrics_processor
();
let
engine
:
ExecutionContext
=
Arc
::
new
(
engine
);
Ok
(
engine
)
}
Prev
1
2
3
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment