Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
c9130f8f
Commit
c9130f8f
authored
Feb 05, 2025
by
J Wyman
Committed by
GitHub
Feb 05, 2025
Browse files
ci: Add Copyright Verification Scripts w/ Automation (#110)
parent
9322edef
Changes
99
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
1180 additions
and
26 deletions
+1180
-26
.clang-format
.clang-format
+2
-2
.github/workflows/copyright-check.ps1
.github/workflows/copyright-check.ps1
+396
-0
.github/workflows/pre-merge-rust.yml
.github/workflows/pre-merge-rust.yml
+3
-1
.github/workflows/pre-merge.yml
.github/workflows/pre-merge.yml
+21
-0
.github/workflows/trigger_ci_pull.yml
.github/workflows/trigger_ci_pull.yml
+10
-8
.github/workflows/trigger_ci_push.yml
.github/workflows/trigger_ci_push.yml
+12
-10
CONTRIBUTING.md
CONTRIBUTING.md
+1
-1
container/deps/vllm/tests/test_patch_install.py
container/deps/vllm/tests/test_patch_install.py
+15
-0
deploy/Kubernetes/_build/common.ps1
deploy/Kubernetes/_build/common.ps1
+531
-0
deploy/Kubernetes/_build/tester.containerfile
deploy/Kubernetes/_build/tester.containerfile
+66
-0
examples/python/hello_world/operators/encoder_decoder.py
examples/python/hello_world/operators/encoder_decoder.py
+15
-0
examples/python/llm/vllm/benchmark/deploy_llama_70b_context_tp2dp4.sh
...hon/llm/vllm/benchmark/deploy_llama_70b_context_tp2dp4.sh
+15
-0
examples/python/llm/vllm/benchmark/deploy_llama_70b_generate_tp8dp1.sh
...on/llm/vllm/benchmark/deploy_llama_70b_generate_tp8dp1.sh
+15
-0
examples/python/llm/vllm/deploy/deploy_llama_8b_baseline.sh
examples/python/llm/vllm/deploy/deploy_llama_8b_baseline.sh
+14
-0
examples/python/llm/vllm/deploy/deploy_llama_8b_disaggregated.sh
...s/python/llm/vllm/deploy/deploy_llama_8b_disaggregated.sh
+14
-0
examples/python/llm/vllm/deploy/deploy_llama_8b_disaggregated_multinode.sh
...lm/vllm/deploy/deploy_llama_8b_disaggregated_multinode.sh
+14
-0
examples/python/llm/vllm/deploy/parser.py
examples/python/llm/vllm/deploy/parser.py
+15
-0
examples/python/llm/vllm/operators/stages.py
examples/python/llm/vllm/operators/stages.py
+4
-2
examples/python/llm/vllm/operators/vllm.py
examples/python/llm/vllm/operators/vllm.py
+15
-0
icp/protos/icp.proto
icp/protos/icp.proto
+2
-2
No files found.
.clang-format
View file @
c9130f8f
...
...
@@ -2,7 +2,7 @@
BasedOnStyle: Google
IndentWidth: 2
ColumnLimit:
8
0
ColumnLimit:
12
0
ContinuationIndentWidth: 4
UseTab: Never
MaxEmptyLinesToKeep: 2
...
...
.github/workflows/copyright-check.ps1
0 → 100644
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2022-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set-strictmode
-version
latest
.
"
$(
&
git
rev-parse
--show-toplevel
)
/deploy/Kubernetes/_build/common.ps1
"
# == begin common.ps1 extensions ==
$date_key
= '%%DATE%%'
$date_regex
= '(?>(?>\d{4})-)?(?<year>\d{4})'
$
global
:
copyright_matchers
= @(
@{
files = @('.containerfile', '.dockerignore', '.pbtxt', '.ps1', '.py', '.sh', '.toml', '.tpl', '.txt', '.yaml', '.yml', 'Dockerfile')
found_missing =
$false
matches = @(
'# SPDX-FileCopyrightText: Copyright (c) ' +
$date_key
+ ' NVIDIA CORPORATION & AFFILIATES. All rights reserved.'
'# SPDX-License-Identifier: Apache-2.0'
'# Licensed under the Apache License, Version 2.0 (the "
License
");'
'# you may not use this file except in compliance with the License.'
'# You may obtain a copy of the License at'
'# http://www.apache.org/licenses/LICENSE-2.0'
'# Unless required by applicable law or agreed to in writing, software'
'# distributed under the License is distributed on an "
AS
IS
" BASIS,'
'# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.'
'# See the License for the specific language governing permissions and'
'# limitations under the License.'
)
name = 'basic'
regex =
$null
vertical_spacer = '#'
}
@{
files = @('.json')
found_missing =
$false
matches = @(
'"
copyright
": ['
' "
SPDX-FileCopyrightText:
Copyright
(
c
)
' + $date_key + '
NVIDIA
CORPORATION
&
AFFILIATES.
All
rights
reserved.
",'
' "
SPDX-License-Identifier:
Apache-2.0
",'
' "
Licensed
under
the
Apache
License
,
Version
2.0
(
the
\
"License\"
);
",'
' "
you
may
not
use
this
file
except
in
compliance
with
the
License.
",'
' "
You
may
obtain
a
copy
of
the
License
at
",'
' "
http://www.apache.org/licenses/LICENSE-2.0
",'
' "
Unless
required
by
applicable
law
or
agreed
to
in
writing
,
software
",'
' "
distributed
under
the
License
is
distributed
on
an
\
"AS IS\"
BASIS
,
",'
' "
WITHOUT
WARRANTIES
OR
CONDITIONS
OF
ANY
KIND
,
either
express
or
implied.
",'
' "
See
the
License
for
the
specific
language
governing
permissions
and
",'
' "
limitations
under
the
License.
"'
'],'
)
name = 'json'
regex =
$null
vertical_spacer =
$null
}
@{
files = @('.md')
found_missing =
$false
matches = @(
'<!--'
'SPDX-FileCopyrightText: Copyright (c) ' +
$date_key
+ ' NVIDIA CORPORATION & AFFILIATES. All rights reserved.'
'SPDX-License-Identifier: Apache-2.0'
'Licensed under the Apache License, Version 2.0 (the "
License
");'
'you may not use this file except in compliance with the License.'
'You may obtain a copy of the License at'
'http://www.apache.org/licenses/LICENSE-2.0'
'Unless required by applicable law or agreed to in writing, software'
'distributed under the License is distributed on an "
AS
IS
" BASIS,'
'WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.'
'See the License for the specific language governing permissions and'
'limitations under the License.'
'-->'
)
name = 'markdown'
regex =
$null
vertical_spacer = ''
}
@{
files = @('.proto', '.rs')
found_missing =
$false
matches = @(
'// SPDX-FileCopyrightText: Copyright (c) ' +
$date_key
+ ' NVIDIA CORPORATION & AFFILIATES. All rights reserved.'
'// SPDX-License-Identifier: Apache-2.0'
'// Licensed under the Apache License, Version 2.0 (the "
License
");'
'// you may not use this file except in compliance with the License.'
'// You may obtain a copy of the License at'
'// http://www.apache.org/licenses/LICENSE-2.0'
'// Unless required by applicable law or agreed to in writing, software'
'// distributed under the License is distributed on an "
AS
IS
" BASIS,'
'// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.'
'// See the License for the specific language governing permissions and'
'// limitations under the License.'
)
name = 'c-like'
regex =
$null
vertical_spacer = '//'
}
)
$
global
:
copyright_results
= @{
failed_date = @()
failed_header = @()
passed = @()
skipped = @()
unsupported = @()
}
# === end common.ps1 extensions ===
$ignored_files
= @('.clang-format', '.gitattributes', '.gitignore', '.gitkeep', '.patch', 'Cargo.lock', 'LICENSE', 'uv.lock')
write-debug "
<
copyright-check
>
ignored_files
=
[
'$($ignored_files -join "'
,
'")'
]
.
"
$ignored_paths
= @('.github', '.mypy_cache', '.pytest_cache')
write-debug "
<
copyright-check
>
ignored_paths
=
[
'$($ignored_paths -join "'
,
'")'
]
.
"
$ignored_types
= @('.bat', '.gif', '.ico', '.ipynb', '.jpg', '.jpeg', '.patch', '.png', '.pyc', '.pyi', '.rst', '.zip')
write-debug "
<
copyright-check
>
ignored_types
=
[
'$($ignored_types -join "'
,
'")'
]
.
"
$ignored_folders
= @('.git', '__pycache__')
function is_ignored([string]
$path
) {
# write-debug "
<
copyright-check/is_ignored
>
path:
\
"
${path}
\"
.
"
if ((
$null
-eq
$path
) -or (
$path
.length -eq 0) -or (
$path
.endswith('/'))) {
write-debug "
<
copyright-check/is_ignored
>
ignored:
true.
"
return
$true
}
foreach (
$ignored_path
in
$ignored_paths
) {
if (
$path
.startswith(
$ignored_path
)) {
write-debug "
<
copyright-check/is_ignored
>
ignored:
true.
"
return
$true
}
}
foreach (
$ignored_extension
in
$ignored_types
) {
if (
$file
.endswith(
$ignored_extension
)) {
write-debug "
<
copyright-check/is_ignored
>
ignore
=
true.
"
return
$true
}
}
foreach (
$ignored_file
in
$ignored_files
) {
if (
$file
.endswith(
$ignored_file
)) {
write-debug "
<
copyright-check/is_ignored
>
ignore
=
true.
"
return
$true
}
}
$normalized_path
= normalize_path
$file
foreach (
$ignored_folder
in
$ignored_folders
) {
if (
$normalized_path
-contains "
/
${ignored_folder}
/
") {
write-debug "
<
copyright-check/is_ignored
>
ignore
=
true.
"
return
$true
}
}
if (-not(test-path "
${normalized_path}
" -pathtype 'Leaf')) {
write-debug "
<
copyright-check/is_ignored
>
ignore
=
true.
"
return
$true
}
write-debug "
<
copyright-check/is_ignored
>
ignore
=
false.
"
return
$false
}
function build_regex([object]
$matcher
) {
write-debug "
<
copyright-check/build_regex
>
matcher.name:
$
(
$matcher
.
name
)
."
$regex
=
''
foreach
(
$match
in
$matcher
.
matches
)
{
$match
=
$match
-replace
'([\(\)\[\]\.\+\*\\])'
,
'\$1'
$match
=
$match
-replace
'\s+'
,
'\s+'
# Given the amount of inconsistency between using http and https, we'll just regex it away.
$match
=
$match
-replace
'https?://'
,
'https?://'
# Replace the date matcher placeholder w/ the actual regex we'll need.
$match
=
$match
-replace
$date_key
,
$date_regex
$regex
=
"
${regex}${match}
[\n\r\s]+"
if
(
$null
-ne
$matcher
.
vertical_spacer
)
{
$regex
=
"
${regex}
(?>
$(
$matcher
.
vertical_spacer
)
[\n\r\s]+)*"
}
}
write-debug
"<copyright-check/build_regex> -> '
${regex}
'."
return
$regex
}
function
check_header
([
string
]
$path
,
[
object
]
$matcher
)
{
write-debug
"<copyright-check/check_header> path: ""
${path}
""."
write-debug
"<copyright-check/check_header> matcher: ""
$(
$matcher
.
name
)
""."
$command
=
"git log -1 --pretty=""%cs"" --
${file}
"
$output
=
invoke-expression
$command
|
out-string
$output
=
$output
.
trim
()
$last_modified
=
$output
.
substring
(
0
,
4
)
-as
[
int
]
write-debug
"<copyright-check/check_header> last_modified:
${last_modified}
."
if
(
$null
-eq
$matcher
.
regex
)
{
$matcher
.
regex
=
$
(
build_regex
$matcher
)
}
$regex
=
$matcher
.
regex
write-debug
"<copyright-check/check_header> regex: ""
${regex}
""."
$contents
=
read_content
$path
if
((
$null
-eq
$contents
)
-or
(
$contents
.
length
-le
0
))
{
$
global
:
copyright_results
.
skipped
+=
$file
write-detailed
" [SKIP]
${file}
"
'DarkGray'
return
}
if
(
$contents
-match
$regex
)
{
$capture_date
=
$Matches
.
year
-as
[
int
]
if
(
$capture_date
-lt
$last_modified
)
{
$
global
:
copyright_results
.
failed_date
+=
$file
write-error
" [FAIL] Incorrect Date in Header:
${path}
(
${capture_date}
)"
}
else
{
$
global
:
copyright_results
.
passed
+=
$file
write-normal
" [PASS]
${file}
"
}
}
else
{
$
global
:
copyright_results
.
failed_header
+=
$file
write-error
" [FAIL] Invalid/Missing Header:
${file}
"
$matcher
.
found_missing
=
$true
}
}
function
check_file
([
string
]
$file
)
{
write-debug
"<copyright-check/check_file> file: ""
${file}
""."
$path
=
normalize_path
$file
if
(
test-path
$path
-pathtype
'Leaf'
)
{
write-debug
"<copyright-check/check_file> path: ""
${path}
""."
$is_checked
=
$false
foreach
(
$matcher
in
$
global
:
copyright_matchers
)
{
foreach
(
$ext
in
$matcher
.
files
)
{
if
(
$path
.
endswith
(
$ext
))
{
check_header
$path
$matcher
$is_checked
=
$true
break
}
}
if
(
$is_checked
)
{
return
}
}
write-warning
" [WARN] Unsupported:
${file}
"
$
global
:
copyright_results
.
unsupported
+=
$file
}
}
$current_year
=
"
$(
get-date
-format
'yyyy'
)
" -as [int]
write-debug "
<
copyright-check
>
current_year
=
${current_year}
.
"
foreach (
$file
in
$(
git
ls-tree
-r
--name-only
HEAD
)
)
{
$file
=
$file
.trim()
write-debug "
<
copyright-check
>
file:
""
${file}
""
.
"
if (is_ignored
$file
) {
write-detailed "
[
SKIP
]
${file}
" 'DarkGray'
$
global
:
copyright_results
.skipped +=
$file
continue
}
check_file
$file
}
function generate_report() {
$reports_path
=
$
env
:
NVBUILD_REPORTS_PATH
if (
$null
-eq
$reports_path
) {
return
}
if (-not (test-path
$reports_path
-pathtype 'Container')) {
if (test-path
$reports_path
-pathtype 'Leaf') {
return
}
new-item
$reports_path
-itemtype 'Directory' | out-null
}
write-debug "
<
copyright-check/generate_report
>
Generating
check
report.
"
$check_results
= "
<
copyright-verification
>
`n
"
if (
$
global
:
copyright_results
.failed_header.count -gt 0) {
$check_results
+= "
<
failed
reason
=
""
Invalid
or
Missing
Header
""
>
`n
"
foreach (
$file
in
$
global
:
copyright_results
.failed_header) {
$check_results
+= "
<
file
>
${file}
<
/file
>
`n
"
}
$check_results
+= "
<
/failed
>
`n
"
}
if (
$
global
:
copyright_results
.failed_date.count -gt 0) {
$check_results
+= "
<
failed
reason
=
""
Incorrect
Date
""
>
`n
"
foreach (
$file
in
$
global
:
copyright_results
.failed_date) {
$check_results
+= "
<
file
>
${file}
<
/file
>
`n
"
}
$check_results
+= "
<
/failed
>
`n
"
}
if (
$
global
:
copyright_results
.unsupported.count -gt 0) {
$check_results
+= "
<
skipped
reason
=
""
Unsupported
File
Type
""
>
`n
"
foreach (
$file
in
$
global
:
copyright_results
.unsupported) {
$check_results
+= "
<
file
>
${file}
<
/file
>
`n
"
}
$check_results
+= "
<
/skipped
>
`n
"
}
if (
$
global
:
copyright_results
.passed.count -gt 0) {
$check_results
+= "
<
passed
>
`n
"
foreach (
$file
in
$
global
:
copyright_results
.passed) {
$check_results
+= "
<
file
>
${file}
<
/file
>
`n
"
}
$check_results
+= "
<
/passed
>
`n
"
}
if (
$
global
:
copyright_results
.skipped.count -gt 0) {
$check_results
+= "
<
skipped
reason
=
""
Ignored
by
Path
""
>
`n
"
foreach (
$file
in
$
global
:
copyright_results
.skipped) {
$check_results
+= "
<
file
>
${file}
<
/file
>
`n
"
}
$check_results
+= "
<
/skipped
>
`n
"
}
$check_results
+= "
<
/copyright-verification
>
`n
"
$output_path
= "
${reports_path}
/copyright-check.xml
"
write_content
$check_results
$output_path
-overwrite
write-minimal ''
write-minimal "
Copyright
check
report
-
>
${output_path}
"
}
write-normal ''
write-high "
Pass:
$
(
$
global
:
copyright_results
.
passed
.
count
)
, Fail:
$(
$
global
:
copyright_results
.
failed_date
.
count
+
$
global
:
copyright_results
.
failed_header
.
count
)
"
-no_newline
if
(
$
global
:
copyright_results
.
skipped
.
count
-gt
0
)
{
write-high
", Skipped:
$(
$
global
:
copyright_results
.
skipped
.
count
)
"
-no_newline
}
if
(
$
global
:
copyright_results
.
unsupported
.
count
-gt
0
)
{
write-high
", Unsupported:
$(
$
global
:
copyright_results
.
unsupported
.
count
)
"
-no_newline
}
write-high
''
if
(
$
global
:
copyright_results
.
failed_header
.
count
-gt
0
)
{
write-low
''
write-low
'Copyright checkers detected missing or invalid copyright headers:'
write-low
''
foreach
(
$matcher
in
$
global
:
copyright_matchers
)
{
if
(
$matcher
.
found_missing
)
{
write-low
" name:
$(
$matcher
.
name
)
"
write-low
" files:
$(
$matcher
.
files
-join
", "
)
"
write-low
" pattern:
`n
$(
$matcher
.
regex
)
"
write-low
''
}
}
}
if
((
$
global
:
copyright_results
.
failed_date
.
count
-gt
0
)
-or
(
$
global
:
copyright_results
.
failed_header
.
count
-gt
0
))
{
write-high
'Files out of compliance:'
# Final, end of output list of errors.
foreach
(
$path
in
$
global
:
copyright_results
.
failed_header
)
{
write-error
" [FAIL] invalid/missing header:
${path}
"
}
foreach
(
$path
in
$
global
:
copyright_results
.
failed_date
)
{
write-error
" [FAIL] incorrect date:
${path}
"
}
exit
(
-1
)
}
.github/workflows/pre-merge-rust.yml
View file @
c9130f8f
...
...
@@ -27,6 +27,8 @@ on:
-
main
paths
:
-
'
runtime/rust/**'
paths-ignore
:
-
deploy/Kubernetes/**
jobs
:
pre-merge-rust
:
...
...
.github/workflows/pre-merge.yml
View file @
c9130f8f
...
...
@@ -51,6 +51,27 @@ jobs:
-
uses
:
pre-commit/action@v3.0.0
timeout-minutes
:
3
copyright-checks
:
runs-on
:
ubuntu-24.04
container
:
image
:
ghcr.io/triton-inference-server/triton_distributed/helm-tester:0.1.1
options
:
--tty
volumes
:
-
${{ github.workspace }}:/workspace
permissions
:
contents
:
read
packages
:
read
steps
:
-
uses
:
actions/checkout@v4
# Allowlist both variants of the mounted source directory.
-
run
:
git config --global --add safe.directory /__w/triton_distributed/triton_distributed
-
run
:
git config --global --add safe.directory /workspace
-
run
:
pwsh /workspace/.github/workflows/copyright-check.ps1
env
:
NVBUILD_VERBOSITY
:
DETAILED
timeout-minutes
:
2
working-directory
:
/workspace
# providers_validation:
# runs-on: ubuntu-latest
# container:
...
...
.github/workflows/trigger_ci_pull.yml
View file @
c9130f8f
...
...
@@ -15,6 +15,8 @@
on
:
pull_request
:
paths-ignore
:
-
deploy/Kubernetes/**
jobs
:
mirror_repo
:
...
...
.github/workflows/trigger_ci_push.yml
View file @
c9130f8f
...
...
@@ -18,6 +18,8 @@ on:
branches
:
-
'
main'
-
'
r*'
paths-ignore
:
-
deploy/Kubernetes/**
jobs
:
mirror_repo
:
environment
:
GITLAB
...
...
CONTRIBUTING.md
View file @
c9130f8f
...
...
@@ -6,7 +6,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http
s
://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
...
...
container/deps/vllm/tests/test_patch_install.py
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
pytest
try
:
...
...
deploy/Kubernetes/_build/common.ps1
0 → 100644
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set-strictmode
-version
latest
$
global
:
_init_path
=
"
${env:PWD}
"
$
global
:
_git_branch
=
$null
$
global
:
_local
=
$null
$
global
:
_local_srcdir
=
$null
$
global
:
_repository_root
=
$null
$
global
:
_verbosity
=
$null
$
global
:
colors
=
@{
error
=
'Red'
high
=
'Cyan'
low
=
'DarkGray'
medium
=
'DarkBlue'
test
=
@{
failed
=
'Red'
passed
=
'Green'
}
title
=
'Blue'
warning
=
'Yellow'
}
function
cleanup_after
{
write-debug
"<cleanup_after>"
$
(
reset_environment
)
}
function
configure_debug
([
bool
]
$enabled
)
{
write-debug
"<configure_debug> enabled = '
${enabled}
'."
if
(
$enabled
)
{
# Notify user if any environment variables which could affect the build outcome are set prior to the build script running.
$overrides
=
@()
foreach
(
$entry
in
$
(
&
get-childitem
env:
))
{
# We're only looking for environment variables which are used directly by the build scripts (starts with 'NVBUILD_`);
# and we're looking at environment variables which would indirectly affect the build scripts (i.e. `PATH`).
if
(
$entry
.
key
.
startswith
(
'NVBUILD_'
))
{
# No reason to display values which are displayed when the build start, often time provided by the user.
if
(
$entry
.
key
.
endswith
(
'_COMMAND'
)
-or
$entry
.
key
.
endswith
(
'_VERBOSITY'
))
{
continue
}
$overrides
+=
$entry
}
}
if
(
$overrides
.
count
-gt
0
)
{
write-low
' Overriding environment variables:'
foreach
(
$entry
in
$overrides
)
{
write-low
"
$(
$entry
.
key
)
=
$(
$entry
.
Value
)
"
}
}
}
}
function
create_directory
([
string
]
$path
,
[
switch
]
$recreate
)
{
write-debug
"<create_directory> path = '
${path}
'."
write-debug
"<create_directory> recreate =
${recreate}
"
$path_local
=
$
(
to_local_path
$path
)
write-debug
"<ensure_directory> path_local = '
${path_local}
'."
if
(
test-path
$path_local
-pathType
Container
)
{
if
(
$recreate
)
{
remove-item
$path_local
-Recurse
|
out-null
new-item
$path_local
-itemtype
Directory
|
out-null
}
}
else
{
new-item
$path_local
-itemtype
Directory
|
out-null
}
}
function
default_git_branch
{
if
(
is_installed
'git'
)
{
$value
=
"
$(
git
branch
--show-current
)
"
}
else {
$value
= 'main'
}
write-debug "
<
default_git_branch
>
-
>
'${value}'
.
"
return
$value
}
function default_local_srcdir {
$value
=
$(
&
git
rev-parse
--show-toplevel
)
write-debug
"<default_local_srcdir> -> '
${value}
'."
return
$value
;
}
function
default_verbosity
{
$value
=
'NORMAL'
write-debug
"<default_verbosity> -> '
${value}
'."
return
$value
}
function
env_get_git_branch
{
$value
=
$
env
:
NVBUILD_GIT_BRANCH
write-debug
"<env_get_git_branch> -> '
${value}
'."
return
$value
}
function
env_get_local_srcdir
{
$value
=
$
env
:
NVBUILD_LOCAL_SRCDIR
write-debug
"<env_get_local_srcdir> -> '
${value}
'."
return
$value
}
function
env_get_verbosity
{
$value
=
$
env
:
NVBUILD_VERBOSITY
write-debug
"<env_get_verbosity> -> '
${value}
'."
return
$value
}
function
env_set_git_branch
([
string
]
$value
)
{
if (
$null
-eq
$
env
:
NVBUILD_NOSET
) {
write-debug "
<
env_set_git_branch
>
value:
'${value}'
.
"
$
env
:
NVBUILD_GIT_BRANCH
=
$value
}
}
function env_set_local_srcdir([string]
$value
) {
if (
$null
-eq
$
env
:
NVBUILD_NOSET
) {
write-debug "
<
env_set_local_srcdir
>
value:
'${value}'
.
"
$
env
:
NVBUILD_LOCAL_SRCDIR
=
$value
}
}
function env_set_verbosity([string]
$value
) {
if (
$null
-eq
$
env
:
NVBUILD_NOSET
) {
write-debug "
<
env_set_verbosity
>
value:
'${value}'
.
"
$
env
:
NVBUILD_VERBOSITY
=
$value
}
}
function fatal_exit([string]
$message
) {
write-error "
fatal:
${message}
"
exit 1
}
function get_git_branch {
if (
$null
-eq
$
global
:
_git_branch
) {
$value
=
$(
env_set_git_branch
)
if
(
$null
-ne
$value
)
{
set_git_branch
$value
}
else {
set_git_branch
$(
default_git_branch
)
}
}
write-debug
"<get_git_branch> -> '
${global:_git_branch}
'."
return
$
global
:
_git_branch
}
function
get_local_srcdir
{
if
(
$null
-eq
$
global
:
_local_srcdir
)
{
$value
=
$
(
env_get_local_srcdir
)
if
(
$null
-ne
$value
)
{
set_local_srcdir
$value
}
else
{
set_local_srcdir
$
(
default_local_srcdir
)
}
}
write-debug
"<get_local_srcdir> -> '
${global:_local_srcdir}
'."
return
$
global
:
_local_srcdir
}
function
get_repository_root
{
if
(
$null
-eq
$
global
:
_repository_root
)
{
$path
=
$
(
&
git
rev-parse
--show-toplevel
)
$
global
:
_repository_root
=
$
(
normalize_path
$path
)
}
write-debug
"<get_repository_root> '
${global:_repository_root}
'."
return
$
global
:
_repository_root
}
function
get_verbosity
{
if
(
$null
-eq
$
global
:
_verbosity
)
{
$value
=
$
(
env_get_verbosity
)
if
(
$null
-ne
$value
)
{
set_verbosity
$value
}
else
{
set_verbosity
$
(
default_verbosity
)
}
}
write-debug
"<get_verbosity> -> '
${global:_verbosity}
'."
return
$
global
:
_verbosity
}
function
is_debug
{
$value
=
$
(
$null
-ne
$
env
:
NVBUILD_DEBUG_TRACE
)
write-debug
"<is_debug> ->
${value}
."
return
$value
}
function
is_empty
([
string
]
$value
)
{
return [System.String]::IsNullOrWhiteSpace(
$value
)
}
function is_git_ignored([string]
$path
) {
$repo_root
=
$(
get_repository_root
)
if
(
$path
.
startswith
(
$repo_root
)
)
{
$path
=
$path
.substring(
$repo_root
.length)
}
if (
$path
.startswith('/')) {
$path
=
$path
.substring(1)
}
$result
=
$(
&
git
check-ignore
$path
)
return
(
0
-eq
$result
)
}
function is_installed([string]
$command
) {
write-debug "
<
is_installed
>
command
=
'${command}'
.
"
$out
=
$null
-ne
$(
get-command
"
${command}
"
-errorAction
SilentlyContinue
)
write-debug
"<is_installed> ->
${out}
."
return
$out
}
function
is_tty
{
return
-not
(([
System.Console
]::
IsOutputRedirected
)
-or
([
System.Console
]::
IsErrorRedirected
))
}
function
is_verbosity_valid
([
string
]
$value
)
{
return (('NORMAL' -eq
$value
) -or ('MINIMAL' -eq
$value
) -or ('DETAILED' -eq
$value
))
}
function normalize_path([string]
$path
) {
write-debug "
<
normalize-path
>
path:
'${path}'
.
"
$out
= "
$
(
resolve-path
"
${path}
"
-erroraction
'Ignore'
)
"
if ((
$null
-eq
$out
) -or (
$out
.length -eq 0)) {
if (
$path
.startswith(
$(
get_repository_root
)
)
) {
$out
=
$path
}
else {
$out
= "
$
(
get_repository_root
)
/
${path}
"
}
}
if (
$IsWindows
) {
if (
$out
-match "
^
[
A-Z
]:
") {
$out
=
$out
.substring(2)
}
$out
=
$out
.replace('\', '/')
}
write-debug "
<
normalize
-path
>
'${path}'
-
>
'${out}'
.
"
return
$out
}
function read_content([string]
$path
, [switch]
$lines
, [switch]
$bytes
) {
if (is_empty
$path
-or (
$lines
-and
$bytes
)) {
write-error 'usage: read_content {path} [(-bytes|-lines)]' -category InvalidArgument
write-error ' {path} file system path of the file to read contents from.'
write-error ' -bytes when provided content is returned as an array of bytes. mutually exclusive with -lines.'
write-error ' -lines when provided content is returned as an array of strings. mutually exclusive with -bytes.'
write-error ' '
usage_exit 'read_content {path} [(-bytes|-lines)]'
}
write-debug "
<
read_content
>
path
:
'${path}'
.
"
write-debug "
<
read_content
>
bytes
:
${bytes}
.
"
write-debug "
<
read_content
>
lines
:
${lines}
.
"
$path
= normalize_path
$path
if (
$bytes
) {
return get-content -path
$path
-asbytestream -raw
}
if (
$lines
) {
return get-content -path
$path
}
return get-content -path
$path
-raw
}
function reset_environment {
write-debug "
<
reset_environment
>
"
$overrides
= @()
foreach (
$entry
in
$(
&
get-childitem
env:
)
)
{
# We're only looking for environment variables which are used directly by the build scripts (starts with 'NVBUILD_
`)
;
# and we're looking at environment variables which would indirectly affect the build scripts (i.e.
`P
ATH
`)
.
if (
$entry
.key.startswith('NVBUILD_')) {
$overrides
+=
$entry
}
}
if (
$overrides
.count -gt 0) {
foreach (
$entry
in
$overrides
) {
$expression
= '
$env
:' + "
$
(
$entry
.
Key
)
" + ' =
$null
'
invoke-expression "
${expression}
"
if ("
$
(
$entry
.
Key
)
" -ne 'NVBUILD_NOSET') {
write-debug "
<
reset_environment
>
removed
'$($entry.Key)'
.
"
}
}
}
}
function run([string]
$command
) {
if (
$null
-eq
$command
) {
write-error 'usage: run {command}' -category InvalidArgument
write-error ' {command} is the command to execute.'
write-error ' '
usage_exit 'run {command}'
}
write-debug "
<
run
>
command
=
'${command}'
.
"
if ('MINIMAL' -ne
$(
get_verbosity
)
)
{
write-high "
${command}
"
}
invoke-expression "
${command}
" | out-default
$exit_code
=
$LASTEXITCODE
write-debug "
<
run
>
exit_code
=
${exit_code}
.
"
if (
$exit_code
-ne 0) {
write-error "
fatal
:
Command
""
${command}
""
failed
,
returned
${exit_code}
.
" -category fromStdErr
exit
$exit_code
}
}
function set_git_branch([string]
$value
) {
write-debug "
<
set_git_branch
>
value
=
'${value}'
.
"
$
global
:
_git_branch
=
$value
env_set_git_branch
$value
}
function set_local_srcdir([string]
$value
) {
write-debug "
<
set_local_srcdir
>
value
:
'${value}'
.
"
$
global
:
_local_srcdir
=
$value
env_set_local_srcdir
$value
}
function set_verbosity([string]
$value
) {
write-debug "
<
set_verbosity
>
'${value}'
.
"
if (-not(is_verbosity_valid
$value
)) {
throw "
Invalid
verbosity
value
'${value}'
.
"
}
$
global
:
_verbosity
=
$value
env_set_verbosity
$value
}
function to_local_path([string]
$path
) {
write-debug "
<
to_local_path
>
path
:
'${path}'
.
"
if (
$null
-eq
$path
) {
return
$(
get_local_srcdir
)
}
$out
=
$path
.
trim
()
$out
=
$out
.
trim
(
'/'
,
'\'
)
$out
=
join-path
$
(
get_local_srcdir
)
$out
$out
=
$
(
normalize_path
$out
)
return
$out
}
function
typeof
(
$object
)
{
if (
$null
-eq
$object
) {
return 'null'
}
return
$object
.gettype().name
}
function usage_exit([string]
$message
) {
write-error "
usage
:
$message
"
exit 254
}
function value_or_default([string]
$value
, [string]
$default
) {
if ((
$null
-eq
$value
) -or (
$value
.Length -eq 0)) {
return
$default
}
return
$value
}
function write_content([string]
$content
, [string]
$path
, [switch]
$overwrite
) {
if ((
$null
-eq
$path
) -or (
$path
.length -eq 0)) {
write-error 'usage: write_content {content} {path}'
write-error ' {content} is the content to be written to a file.'
write-error ' {path} is the path to file into which to write content.'
usage_exit 'write_content {content} {path}'
}
write-debug "
<
write_content
>
content
=
$
(
$content
.
length
)
bytes.
"
write-debug "
<
write_content
>
path
=
'${path}'
.
"
$path_local
= normalize_path
$path
write-debug "
<
write
-content
>
'${path_local}'
.
"
if (
$null
-eq
$content
) {
$content
= ''
}
if (
$overwrite
-and (test-path
$path_local
)) {
remove-item
$path_local
| out-null
}
$content
| out-file
$path_local
}
function __write([string]
$value
, [string]
$color
, [bool]
$no_newline
) {
if (is_tty) {
$opts
= @{
NoNewline =
$no_newline
}
if ((
$null
-ne
$color
) -and (
$color
.length -gt 0)) {
$opts
.ForegroundColor =
$color
}
write-host
$value
@opts
}
else {
if (-not(
$no_newline
)) {
$value
= "
${value}
`
n
"
}
write-output
$value
}
}
function write-detailed {
param([string]
$value
, [string]
$color
=
$null
, [switch]
$no_newline
)
if ('DETAILED' -eq
$(
get_verbosity
)
)
{
__write
$value
$color
$no_newline
}
}
function write-error([string]
$value
) {
$opts
= @{
color =
$
global
:
colors
.error
no_newline =
$false
}
write-minimal
$value
@opts
}
function write-failed([string]
$value
) {
if (is_tty) {
write-normal ' [Failed]'
$
global
:
colors
.test.failed -no_newline
write-normal "
${value}
"
}
else {
write-output "
Test:
[
Failed
]
${value}
"
}
}
function write-high {
param([string]
$value
, [switch]
$no_newline
)
$opts
= @{
color =
$
global
:
colors
.high
no_newline =
$no_newline
}
write-minimal
$value
@opts
}
function write-low {
param([string]
$value
, [switch]
$no_newline
)
$opts
= @{
color =
$
global
:
colors
.low
no_newline =
$no_newline
}
write-detailed
$value
@opts
}
function write-medium {
param([string]
$value
, [switch]
$no_newline
)
$opts
= @{
color =
$
global
:
colors
.medium
no_newline =
$no_newline
}
write-normal
$value
@opts
}
function write-minimal {
param([string]
$value
, [string]
$color
=
$null
, [switch]
$no_newline
)
__write
$value
$color
$no_newline
}
function write-normal {
param([string]
$value
, [string]
$color
=
$null
, [switch]
$no_newline
)
if ('MINIMAL' -ne
$(
get_verbosity
)
)
{
$opts
= @{
color =
$color
no_newline =
$no_newline
}
__write
$value
@opts
}
}
function write-passed([string]
$value
) {
if (is_tty) {
write-detailed ' [Passed]'
$
global
:
colors
.test.passed -no_newline
write-detailed "
${value}
"
}
else {
write-output "
Test:
[
Passed
]
${value}
"
}
}
function write-title([string]
$value
) {
write-minimal
$value
$
global
:
colors
.title
}
function write-warning([string]
$value
) {
write-minimal
$value
$
global
:
colors
.warning
}
deploy/Kubernetes/_build/tester.containerfile
0 → 100644
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ARG BASE_IMAGE="ubuntu:noble"
FROM ${BASE_IMAGE}
# Set useful labels.
LABEL "base"="ubuntu:noble"
LABEL "configuration"="tester"
LABEL "version"="devel"
# Stop APT (Debian package manager) from complaining about interactivity.
ENV DEBIAN_FRONTEND=noninteractive
# Set additional environment values that make usage more pleasant.
ENV TERM=xterm-256color
# Update / upgrade the base image.
RUN apt-get update \
&& apt-get upgrade --yes \
&& rm -rf /var/lib/apt/lists/*
# Install pre-requisites.
RUN apt-get update \
&& apt-get install --no-install-recommends --yes --fix-missing \
apt-transport-https \
software-properties-common \
git \
wget \
&& rm -rf /var/lib/apt/lists/*
# Download and register the Microsoft repository keys.
RUN wget -q https://packages.microsoft.com/config/ubuntu/24.04/packages-microsoft-prod.deb \
&& dpkg -i packages-microsoft-prod.deb \
&& rm packages-microsoft-prod.deb
# Install Powershell runtime and terminal.
RUN apt-get update \
&& apt-get install --yes \
powershell
# Download Helm and move it to /usr/local/bin.
RUN wget -q https://get.helm.sh/helm-v3.17.0-linux-amd64.tar.gz \
&& tar -zxvf helm-v3.17.0-linux-amd64.tar.gz \
&& mv linux-amd64/helm /usr/local/bin/helm
# Create /workspace and set it to the default folder.
RUN mkdir -p /workspace
# Enable Git operations in the /workspace directory.
RUN printf "[safe]\n directory=/workspace\n" > /root/.gitconfig
WORKDIR /workspace
examples/python/hello_world/operators/encoder_decoder.py
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
numpy
from
triton_distributed.runtime
import
Operator
,
RemoteInferenceRequest
,
RemoteOperator
...
...
examples/python/llm/vllm/benchmark/deploy_llama_70b_context_tp2dp4.sh
View file @
c9130f8f
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set
-e
set
-x
export
VLLM_ATTENTION_BACKEND
=
FLASHINFER
...
...
examples/python/llm/vllm/benchmark/deploy_llama_70b_generate_tp8dp1.sh
View file @
c9130f8f
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set
-e
set
-x
export
VLLM_ATTENTION_BACKEND
=
FLASHINFER
...
...
examples/python/llm/vllm/deploy/deploy_llama_8b_baseline.sh
View file @
c9130f8f
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# FIXME: Convert this script to README steps
...
...
examples/python/llm/vllm/deploy/deploy_llama_8b_disaggregated.sh
View file @
c9130f8f
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# FIXME: Convert this script to README steps
...
...
examples/python/llm/vllm/deploy/deploy_llama_8b_disaggregated_multinode.sh
View file @
c9130f8f
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
export
VLLM_ATTENTION_BACKEND
=
FLASHINFER
export
VLLM_WORKER_MULTIPROC_METHOD
=
spawn
...
...
examples/python/llm/vllm/deploy/parser.py
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
argparse
...
...
examples/python/llm/vllm/operators/stages.py
View file @
c9130f8f
# Copyright (c) 2024, NVIDIA CORPORATION. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
...
...
@@ -11,6 +12,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
abc
import
inspect
import
os
...
...
examples/python/llm/vllm/operators/vllm.py
View file @
c9130f8f
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
argparse
import
json
import
logging
...
...
icp/protos/icp.proto
View file @
c9130f8f
// SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION &
//
AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION &
AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
...
...
Prev
1
2
3
4
5
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment