Skip to content

GitLab

Unverified Commit e0c777ff authored by Anant Sharma's avatar Anant Sharma Committed by GitHub
Browse files

ci: add new required check for pre merge tests (#5740) 


Signed-off-by: default avatarAnant Sharma <anants@nvidia.com>
parent 9f3cc6e9
Show whitespace changes
Inline Side-by-side
Showing with 145 additions and 131 deletions
.github/FILTERS.md
View file @ e0c777ff
...@@ -15,6 +15,7 @@ When you open a PR, CI checks which files changed and runs only relevant jobs: ...@@ -15,6 +15,7 @@ When you open a PR, CI checks which files changed and runs only relevant jobs:
| `docs` | Nothing (classification only) | | `docs` | Nothing (classification only) |
| `examples` | Nothing (classification only) | | `examples` | Nothing (classification only) |
| `ignore` | Nothing (classification only) | | `ignore` | Nothing (classification only) |
| `rust` | Rust pre merge checks |
> **Note:** `docs`, `examples`, and `ignore` don't trigger any CI jobs. They exist to satisfy coverage requirements - every file must match at least one filter. > **Note:** `docs`, `examples`, and `ignore` don't trigger any CI jobs. They exist to satisfy coverage requirements - every file must match at least one filter.
......
.github/actions/changed-files/action.yml
View file @ e0c777ff
...@@ -28,6 +28,9 @@ outputs: ...@@ -28,6 +28,9 @@ outputs:
frontend: frontend:
description: 'Whether frontend files changed' description: 'Whether frontend files changed'
value: ${{ steps.filter.outputs.frontend_any_modified }} value: ${{ steps.filter.outputs.frontend_any_modified }}
rust:
description: 'Whether rust files changed'
value: ${{ steps.filter.outputs.rust_any_modified }}
runs: runs:
using: "composite" using: "composite"
...@@ -82,6 +85,7 @@ runs: ...@@ -82,6 +85,7 @@ runs:
echo "sglang: ${{ steps.filter.outputs.sglang_any_modified }}" echo "sglang: ${{ steps.filter.outputs.sglang_any_modified }}"
echo "trtllm: ${{ steps.filter.outputs.trtllm_any_modified }}" echo "trtllm: ${{ steps.filter.outputs.trtllm_any_modified }}"
echo "frontend: ${{ steps.filter.outputs.frontend_any_modified }}" echo "frontend: ${{ steps.filter.outputs.frontend_any_modified }}"
echo "rust: ${{ steps.filter.outputs.rust_any_modified }}"
echo "" echo ""
echo "=== Files Matching Each Filter ===" echo "=== Files Matching Each Filter ==="
echo "docs: ${{ steps.filter.outputs.docs_all_modified_files }}" echo "docs: ${{ steps.filter.outputs.docs_all_modified_files }}"
...@@ -94,12 +98,13 @@ runs: ...@@ -94,12 +98,13 @@ runs:
echo "sglang: ${{ steps.filter.outputs.sglang_all_modified_files }}" echo "sglang: ${{ steps.filter.outputs.sglang_all_modified_files }}"
echo "trtllm: ${{ steps.filter.outputs.trtllm_all_modified_files }}" echo "trtllm: ${{ steps.filter.outputs.trtllm_all_modified_files }}"
echo "frontend: ${{ steps.filter.outputs.frontend_all_modified_files }}" echo "frontend: ${{ steps.filter.outputs.frontend_all_modified_files }}"
echo "rust: ${{ steps.filter.outputs.rust_all_modified_files }}"
- name: Check for uncovered files - name: Check for uncovered files
shell: bash shell: bash
run: | run: |
# Combine all filter-specific files into one list # Combine all filter-specific files into one list
COVERED_FILES=$(echo "${{ steps.filter.outputs.docs_all_modified_files }} ${{ steps.filter.outputs.examples_all_modified_files }} ${{ steps.filter.outputs.ignore_all_modified_files }} ${{ steps.filter.outputs.ci_all_modified_files }} ${{ steps.filter.outputs.core_all_modified_files }} ${{ steps.filter.outputs.operator_all_modified_files }} ${{ steps.filter.outputs.deploy_all_modified_files }} ${{ steps.filter.outputs.planner_all_modified_files }} ${{ steps.filter.outputs.vllm_all_modified_files }} ${{ steps.filter.outputs.sglang_all_modified_files }} ${{ steps.filter.outputs.trtllm_all_modified_files }} ${{ steps.filter.outputs.frontend_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u) COVERED_FILES=$(echo "${{ steps.filter.outputs.docs_all_modified_files }} ${{ steps.filter.outputs.examples_all_modified_files }} ${{ steps.filter.outputs.ignore_all_modified_files }} ${{ steps.filter.outputs.ci_all_modified_files }} ${{ steps.filter.outputs.core_all_modified_files }} ${{ steps.filter.outputs.operator_all_modified_files }} ${{ steps.filter.outputs.deploy_all_modified_files }} ${{ steps.filter.outputs.planner_all_modified_files }} ${{ steps.filter.outputs.vllm_all_modified_files }} ${{ steps.filter.outputs.sglang_all_modified_files }} ${{ steps.filter.outputs.trtllm_all_modified_files }} ${{ steps.filter.outputs.frontend_all_modified_files }} ${{ steps.filter.outputs.rust_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u)
# Get all modified files # Get all modified files
ALL_FILES=$(echo "${{ steps.filter.outputs.all_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u) ALL_FILES=$(echo "${{ steps.filter.outputs.all_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u)
......
.github/filters.yaml
View file @ e0c777ff
...@@ -137,3 +137,10 @@ frontend: ...@@ -137,3 +137,10 @@ frontend:
- 'components/src/dynamo/frontend/**' - 'components/src/dynamo/frontend/**'
- 'components/src/dynamo/common/**' - 'components/src/dynamo/common/**'
- 'deploy/inference-gateway/**' - 'deploy/inference-gateway/**'
rust:
- '.github/workflows/pre-merge.yml'
- '**/*.rs'
- '**/Cargo.toml'
- '**/Cargo.lock'
- 'deny.toml'
.github/workflows/pre-merge-rust.yml deleted 100644 → 0
View file @ 9f3cc6e9
# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name: Rust pre-merge checks
on:
# Always run this workflow when commits are pushed to main.
push:
branches:
- main
# Run this workflow on pull requests targeting main but only on rust changes.
pull_request:
paths:
- .github/workflows/pre-merge-rust.yml
- '**.rs'
- 'Cargo.toml'
- 'Cargo.lock'
# Cancel any previous check runs for the same pull request to avoid redundant workflows.
concurrency:
group: ${{ github.event_name == 'pull_request' && format('{0}-{1}', github.workflow, github.event.pull_request.number) || format('{0}-{1}', github.workflow, github.run_id) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
tests:
runs-on:
group: Fastchecker
strategy:
# removing kvbm from here - it will fail to test with nixl dep enabled
matrix: { dir: ['.', 'lib/bindings/python', 'lib/runtime/examples', 'launch/dynamo-run'] }
permissions:
contents: read
steps:
- uses: actions/checkout@v4
with:
lfs: true
- name: Set up system dependencies
run: |
# Install protoc for Rust build dependencies (NOTE: much faster than apt install)
PB_REL="https://github.com/protocolbuffers/protobuf/releases"
curl -LO $PB_REL/download/v30.2/protoc-30.2-linux-x86_64.zip
unzip protoc-30.2-linux-x86_64.zip -d $HOME/.local
rm protoc-30.2-linux-x86_64.zip
export PATH="$PATH:$HOME/.local/bin"
protoc --version
- name: Cache cargo artifacts
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-
- name: Set up Rust Toolchain Components
run: rustup component add rustfmt
- name: Verify Code Formatting
working-directory: ${{ matrix.dir }}
run: cargo fmt -- --check
- name: Install and Run cargo-deny
working-directory: ${{ matrix.dir }}
run: |
cargo-deny --version || cargo install cargo-deny@0.19.0
cargo-deny -L error --no-default-features check --hide-inclusion-graph licenses bans --config ${{ github.workspace }}/deny.toml
# Have an explicit step to build tests first to separate time spent on build vs execution.
- name: Compile Tests
working-directory: ${{ matrix.dir }}
run: cargo test --locked --no-run
- name: Run Doc Tests
working-directory: ${{ matrix.dir }}
run: cargo doc --no-deps && cargo test --locked --doc
- name: Run Unit Tests
working-directory: ${{ matrix.dir }}
# NOTE: --all-targets doesn't run doc tests
run: cargo test --locked --all-targets
clippy:
runs-on:
group: Fastchecker
strategy:
matrix: { dir: ['.', 'lib/bindings/python', 'lib/runtime/examples', 'launch/dynamo-run'] }
permissions:
contents: read
steps:
- uses: actions/checkout@v4
with:
lfs: true
- name: Set up system dependencies
run: |
# Install protoc for Rust build dependencies (NOTE: much faster than apt install)
PB_REL="https://github.com/protocolbuffers/protobuf/releases"
curl -LO $PB_REL/download/v30.2/protoc-30.2-linux-x86_64.zip
unzip protoc-30.2-linux-x86_64.zip -d $HOME/.local
rm protoc-30.2-linux-x86_64.zip
export PATH="$PATH:$HOME/.local/bin"
protoc --version
- name: Cache cargo artifacts
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-
- name: Set up Rust Toolchain Components
run: rustup component add clippy
- name: Run Clippy Checks
working-directory: ${{ matrix.dir }}
run: cargo clippy --no-deps --all-targets -- -D warnings
.github/workflows/pre-merge.yml
View file @ e0c777ff
...@@ -13,23 +13,148 @@ ...@@ -13,23 +13,148 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
name: pre_merge name: Pre Merge
run-name: Pre Merge Validation of (${{ github.repository }}/${{ github.ref }}) by ${{ github.actor }}.
on: on:
push:
branches:
- main
- release/*.*.*
pull_request: pull_request:
# Cancel any previous check runs for the same pull request to avoid redundant workflows.
concurrency:
group: ${{ github.event_name == 'pull_request' && format('{0}-{1}', github.workflow, github.event.pull_request.number) || format('{0}-{1}', github.workflow, github.run_id) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs: jobs:
changed-files:
runs-on: ubuntu-latest
outputs:
rust: ${{ steps.changes.outputs.rust }}
steps:
- name: Checkout code
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
fetch-depth: 0
- name: Check for changes
id: changes
uses: ./.github/actions/changed-files
with:
gh_token: ${{ github.token }}
pre-merge-status-check:
runs-on: ubuntu-latest
needs: [changed-files, pre-commit, rust-tests, rust-clippy]
if: always()
steps:
- name: "Check all dependent jobs"
run: |
echo '${{ toJson(needs) }}' | jq -e 'to_entries | map(.value.result) | all(. as $result | ["success", "skipped"] | any($result == .))'
pre-commit: pre-commit:
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
- uses: actions/setup-python@v5 - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
- uses: pre-commit/action@v3.0.0 - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
timeout-minutes: 3 timeout-minutes: 3
rust-tests:
needs: changed-files
if: needs.changed-files.outputs.rust == 'true'
runs-on:
group: Fastchecker
strategy:
# removing kvbm from here - it will fail to test with nixl dep enabled
matrix: { dir: ['.', 'lib/bindings/python', 'lib/runtime/examples', 'launch/dynamo-run'] }
permissions:
contents: read
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
lfs: true
- name: Set up system dependencies
run: |
# Install protoc for Rust build dependencies (NOTE: much faster than apt install)
PB_REL="https://github.com/protocolbuffers/protobuf/releases"
PROTOC_VER="30.2"
PROTOC_ZIP="protoc-${PROTOC_VER}-linux-x86_64.zip"
PROTOC_SHA256="327e9397c6fb3ea2a542513a3221334c6f76f7aa524a7d2561142b67b312a01f"
curl -fsSLO "$PB_REL/download/v${PROTOC_VER}/${PROTOC_ZIP}"
echo "${PROTOC_SHA256} ${PROTOC_ZIP}" | sha256sum -c -
unzip "${PROTOC_ZIP}" -d $HOME/.local
rm "${PROTOC_ZIP}"
export PATH="$PATH:$HOME/.local/bin"
protoc --version
- name: Cache cargo artifacts
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cargo/bin/
~/.cargo/registry
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-
- name: Set up Rust Toolchain Components
run: rustup component add rustfmt
- name: Verify Code Formatting
working-directory: ${{ matrix.dir }}
run: cargo fmt -- --check
- name: Install and Run cargo-deny
working-directory: ${{ matrix.dir }}
run: |
cargo-deny --version || cargo install cargo-deny@0.19.0
cargo-deny -L error --no-default-features check --hide-inclusion-graph licenses bans --config ${{ github.workspace }}/deny.toml
# Have an explicit step to build tests first to separate time spent on build vs execution.
- name: Compile Tests
working-directory: ${{ matrix.dir }}
run: cargo test --locked --no-run
- name: Run Doc Tests
working-directory: ${{ matrix.dir }}
run: cargo doc --no-deps && cargo test --locked --doc
- name: Run Unit Tests
working-directory: ${{ matrix.dir }}
# NOTE: --all-targets doesn't run doc tests
run: cargo test --locked --all-targets
rust-clippy:
needs: changed-files
if: needs.changed-files.outputs.rust == 'true'
runs-on:
group: Fastchecker
strategy:
matrix: { dir: ['.', 'lib/bindings/python', 'lib/runtime/examples', 'launch/dynamo-run'] }
permissions:
contents: read
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
lfs: true
- name: Set up system dependencies
run: |
# Install protoc for Rust build dependencies (NOTE: much faster than apt install)
PB_REL="https://github.com/protocolbuffers/protobuf/releases"
PROTOC_VER="30.2"
PROTOC_ZIP="protoc-${PROTOC_VER}-linux-x86_64.zip"
PROTOC_SHA256="327e9397c6fb3ea2a542513a3221334c6f76f7aa524a7d2561142b67b312a01f"
curl -fsSLO "$PB_REL/download/v${PROTOC_VER}/${PROTOC_ZIP}"
echo "${PROTOC_SHA256} ${PROTOC_ZIP}" | sha256sum -c -
unzip "${PROTOC_ZIP}" -d $HOME/.local
rm "${PROTOC_ZIP}"
export PATH="$PATH:$HOME/.local/bin"
protoc --version
- name: Cache cargo artifacts
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.cargo/bin/
~/.cargo/registry
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: ${{ runner.os }}-cargo-
- name: Set up Rust Toolchain Components
run: rustup component add clippy
- name: Run Clippy Checks
working-directory: ${{ matrix.dir }}
run: cargo clippy --no-deps --all-targets -- -D warnings
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment