Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
62a4a339
Unverified
Commit
62a4a339
authored
Nov 22, 2024
by
Xuehai Pan
Committed by
GitHub
Nov 22, 2024
Browse files
docs: fix module docstrings and copyright headers (#2077)
parent
2797bc34
Changes
83
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
273 additions
and
294 deletions
+273
-294
python/sglang/srt/models/mixtral_quant.py
python/sglang/srt/models/mixtral_quant.py
+14
-14
python/sglang/srt/models/olmo.py
python/sglang/srt/models/olmo.py
+13
-14
python/sglang/srt/models/olmoe.py
python/sglang/srt/models/olmoe.py
+14
-14
python/sglang/srt/models/qwen.py
python/sglang/srt/models/qwen.py
+14
-14
python/sglang/srt/models/qwen2.py
python/sglang/srt/models/qwen2.py
+14
-14
python/sglang/srt/models/qwen2_moe.py
python/sglang/srt/models/qwen2_moe.py
+16
-17
python/sglang/srt/models/stablelm.py
python/sglang/srt/models/stablelm.py
+18
-16
python/sglang/srt/models/torch_native_llama.py
python/sglang/srt/models/torch_native_llama.py
+15
-17
python/sglang/srt/models/xverse.py
python/sglang/srt/models/xverse.py
+13
-14
python/sglang/srt/models/xverse_moe.py
python/sglang/srt/models/xverse_moe.py
+14
-15
python/sglang/srt/models/yivl.py
python/sglang/srt/models/yivl.py
+13
-15
python/sglang/srt/openai_api/adapter.py
python/sglang/srt/openai_api/adapter.py
+13
-15
python/sglang/srt/openai_api/protocol.py
python/sglang/srt/openai_api/protocol.py
+13
-15
python/sglang/srt/sampling/sampling_params.py
python/sglang/srt/sampling/sampling_params.py
+13
-15
python/sglang/srt/server.py
python/sglang/srt/server.py
+13
-15
python/sglang/srt/server_args.py
python/sglang/srt/server_args.py
+13
-15
python/sglang/srt/utils.py
python/sglang/srt/utils.py
+13
-15
python/sglang/test/runners.py
python/sglang/test/runners.py
+13
-14
scripts/playground/test_session_id.py
scripts/playground/test_session_id.py
+11
-12
test/srt/models/test_embedding_models.py
test/srt/models/test_embedding_models.py
+13
-14
No files found.
python/sglang/srt/models/mixtral_quant.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from
# Adapted from
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/mixtral_quant.py#L1
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/mixtral_quant.py#L1
"""Inference-only Mixtral model."""
"""Inference-only Mixtral model."""
from
typing
import
Iterable
,
Optional
,
Tuple
from
typing
import
Iterable
,
Optional
,
Tuple
import
numpy
as
np
import
numpy
as
np
...
...
python/sglang/srt/models/olmo.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from
# Adapted from
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/olmo.py#L1
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/olmo.py#L1
...
...
python/sglang/srt/models/olmoe.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from:
# Adapted from:
# https://github.com/vllm-project/vllm/pull/7922
# https://github.com/vllm-project/vllm/pull/7922
"""Inference-only OLMoE model compatible with HuggingFace weights."""
"""Inference-only OLMoE model compatible with HuggingFace weights."""
from
typing
import
Any
,
Dict
,
Iterable
,
List
,
Optional
,
Tuple
from
typing
import
Any
,
Dict
,
Iterable
,
List
,
Optional
,
Tuple
import
torch
import
torch
...
...
python/sglang/srt/models/qwen.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from
# Adapted from
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/qwen.py#L1
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/qwen.py#L1
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
import
torch
import
torch
...
...
python/sglang/srt/models/qwen2.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from llama2.py
# Adapted from llama2.py
# Modify details for the adaptation of Qwen2 model.
# Modify details for the adaptation of Qwen2 model.
"""Inference-only Qwen2 model compatible with HuggingFace weights."""
"""Inference-only Qwen2 model compatible with HuggingFace weights."""
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
import
torch
import
torch
...
...
python/sglang/srt/models/qwen2_moe.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# coding=utf-8
# Adapted from
# Adapted from
# https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/qwen2_moe.py
# https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/qwen2_moe.py
"""Inference-only Qwen2MoE model compatible with HuggingFace weights."""
"""Inference-only Qwen2MoE model compatible with HuggingFace weights."""
from
typing
import
Any
,
Dict
,
Iterable
,
List
,
Optional
,
Tuple
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
import
torch
import
torch
import
torch.nn.functional
as
F
import
torch.nn.functional
as
F
...
...
python/sglang/srt/models/stablelm.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from:
# Adapted from:
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/stablelm.py#L1
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/stablelm.py#L1
"""Inference-only StableLM-2 (https://huggingface.co/stabilityai/stablelm-2-1_6b)
"""
model compatible with HuggingFace weights."""
Inference-only StableLM-2 (https://huggingface.co/stabilityai/stablelm-2-1_6b)
model compatible with HuggingFace weights.
"""
from
typing
import
Iterable
,
Optional
,
Tuple
from
typing
import
Iterable
,
Optional
,
Tuple
import
torch
import
torch
...
...
python/sglang/srt/models/torch_native_llama.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from
# Adapted from
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/llama.py#L1
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/llama.py#L1
"""Inference-only LLaMA model compatible with HuggingFace weights."""
# PyTorch Tensor Parallel Available for This Model
"""
"""
Inference-only LLaMA model compatible with HuggingFace weights.
This model supports tensor parallelism (TP) using the PyTorch tensor parallel package.
This model supports tensor parallelism (TP) using the PyTorch tensor parallel package.
Reference: https://pytorch.org/docs/stable/distributed.tensor.parallel.html
Reference: https://pytorch.org/docs/stable/distributed.tensor.parallel.html
...
...
python/sglang/srt/models/xverse.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# Adapted from
# Adapted from
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/xverse.py#L1
# https://github.com/vllm-project/vllm/blob/c7f2cf2b7f67bce5842fedfdba508440fe257375/vllm/model_executor/models/xverse.py#L1
...
...
python/sglang/srt/models/xverse_moe.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""Inference-only XVERSE MoE model."""
"""Inference-only XVERSE MoE model."""
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
from
typing
import
Any
,
Dict
,
Iterable
,
Optional
,
Tuple
import
torch
import
torch
...
...
python/sglang/srt/models/yivl.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""Inference-only Yi-VL model."""
"""Inference-only Yi-VL model."""
from
typing
import
Iterable
,
Optional
,
Tuple
from
typing
import
Iterable
,
Optional
,
Tuple
...
...
python/sglang/srt/openai_api/adapter.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""Conversion between OpenAI APIs and native SRT APIs"""
"""Conversion between OpenAI APIs and native SRT APIs"""
import
asyncio
import
asyncio
...
...
python/sglang/srt/openai_api/protocol.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""Pydantic models for OpenAI API protocol"""
"""Pydantic models for OpenAI API protocol"""
import
time
import
time
...
...
python/sglang/srt/sampling/sampling_params.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""Sampling parameters for text generation."""
"""Sampling parameters for text generation."""
from
typing
import
List
,
Optional
,
Union
from
typing
import
List
,
Optional
,
Union
...
...
python/sglang/srt/server.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""
"""
The entry point of inference server.
The entry point of inference server.
SRT = SGLang Runtime.
SRT = SGLang Runtime.
...
...
python/sglang/srt/server_args.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""The arguments of the server."""
"""The arguments of the server."""
import
argparse
import
argparse
...
...
python/sglang/srt/utils.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
"""Common utilities."""
"""Common utilities."""
import
base64
import
base64
...
...
python/sglang/test/runners.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
import
json
import
json
import
multiprocessing
as
mp
import
multiprocessing
as
mp
...
...
scripts/playground/test_session_id.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
# FIXME: Make it a CI test
# FIXME: Make it a CI test
...
...
test/srt/models/test_embedding_models.py
View file @
62a4a339
"""
# Copyright 2023-2024 SGLang Team
Copyright 2023-2024 SGLang Team
# Licensed under the Apache License, Version 2.0 (the "License");
Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
See the License for the specific language governing permissions and
# limitations under the License.
limitations under the License.
# ==============================================================================
"""
import
multiprocessing
as
mp
import
multiprocessing
as
mp
import
unittest
import
unittest
...
...
Prev
1
2
3
4
5
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment