GitLab

* bugs fixed for distill_model server

* bug fixed for lora merge

* support dynamic cfg for cfg_distill

* reformat files

* fix: correct frequency computation in WanTransformerInfer

* refactor: restructure API server and distributed inference services

- Removed the old api_server_dist.py file and integrated its functionality into a new modular structure.
- Created a new ApiServer class to handle API routes and services.
- Introduced DistributedInferenceService and FileService for better separation of concerns.
- Updated the main entry point to initialize and run the new API server with distributed inference capabilities.
- Added schema definitions for task requests and responses to improve data handling.
- Enhanced error handling and logging throughout the services.

* refactor: enhance API structure and file handling in server

- Introduced APIRouter for modular route management in the ApiServer class.
- Updated task creation and file download endpoints to improve clarity and functionality.
- Implemented a new method for streaming file responses with proper MIME type handling.
- Refactored task request schema to auto-generate task IDs and handle optional video save paths.
- Improved error handling and logging for better debugging and user feedback.

* feat: add configurable parameters for video generation

- Introduced new parameters: infer_steps, target_video_length, and seed to the API and task request schema.
- Updated DefaultRunner and VideoGenerationService to handle the new parameters for enhanced video generation control.
- Improved default values for parameters to ensure consistent behavior.

* refactor: enhance profiling context for async support

* refactor: improve signal handling in API server

* feat: enhance video generation capabilities with audio support

* refactor: improve subprocess call for audio-video merging in wan_audio_runner.py

* refactor: enhance API server argument parsing and improve code readability

* refactor: enhance logging and improve code comments for clarity

* refactor: update response model for task listing endpoint to return a dictionary

* docs: update API endpoints and improve documentation clarity

* refactor: update API endpoints in scripts for task management and remove unused code

* fix: pre-commit

* add readme

* modify readme

* update lora keys

* update lora extractor/merger tools

* rename lora config and script files

* bug fixed for lora tools

Enable 720p model inference on low-spec GPUs/CPUs and accelerate T5/CLIP quantized models with vLLM operators

Co-authored-by: gushiqiao <gushiqiao@sensetime.com>

* add step & cfg distillation wan model

* add step & cfg distillation wan model

* bug fixed

* Implement distributed inference API server with FastAPI

- Added a new file `api_server_dist.py` to handle video generation tasks using distributed inference.
- Introduced endpoints for task submission, status checking, and result retrieval.
- Implemented image downloading and task management with error handling.
- Enhanced `infer.py` to ensure proper initialization of distributed processes.
- Created a shell script `start_api_with_dist_inference.sh` for easy server startup with environment setup.

This commit establishes a robust framework for managing video generation tasks in a distributed manner.

* Enhance file download endpoint with path traversal protection and error handling

* style: format

* refactor: Enhance video generation functionality with task interruption support

* feat: Add image upload and video generation endpoint with unique task handling

- Introduced a new endpoint `/v1/local/video/generate_form` for video generation that accepts image uploads.
- Implemented unique filename generation for uploaded images to prevent conflicts.
- Enhanced directory management for input and output paths.
- Improved file download response with detailed status and size information.
- Added error handling for distributed inference processes and graceful shutdown procedures.

* reconstruct quantization and fix memory leak bug.

* Support lazy load inference.

* reconstruct quantization

* Fix hunyuan bugs

* deleted tmp file

---------
Co-authored-by: root <root@pt-c0b333b3a1834e81a0d4d5f412c6ffa1-worker-0.pt-c0b333b3a1834e81a0d4d5f412c6ffa1.ns-devsft-3460edd0.svc.cluster.local>
Co-authored-by: gushiqiao <gushqiaio@sensetime.com>
Co-authored-by: gushiqiao <gushiqiao@sensetime.com>

* split dit server from default runner

* split dit server from default runner

* update loading functions

* simplify loader functions and runner functions

* simplify code && split dit service

* simplify code && split dit service

* support split server for cogvideox

* clear code.

* support prompt enhancer server

* bugs fixed

* finished prompt enhancer service

* add load_transformer methods for split server

* add service utils

* [feature] support split servers

* [feature] add server for multi-gpus

* [update] update start parameters for multi-gpu servers

* [update] update start parameters for multi-gpu servers

* [update] update documents for multi-gpu services

* function hunyuan_t2v_tea, hunyuan_t2v_taylorseer, modify the fresh_threshold of taylorseer

* hunyuan i2v,t2v + tea,tay; wan i2v,t2v + tea function, add log files

* 删除了TeaCace Scheduler的多余属性

* 删除了多余目录

* 修复了TeaCaching部分的bug,目前t2v, i2v feature caching均可跑通

* Update attn_weight.py

---------
Co-authored-by: Yang Yong(雍洋) <yongyang1030@163.com>

adding a padding strategy for SP

---------
Co-authored-by: “de1star” <“843414674@qq.com”>

* Support load advance ptq model.

* Update run_wan_i2v_advanced_ptq.sh

---------
Co-authored-by: gushiqiao <gushiqiao@sensetime.com>
Co-authored-by: Yang Yong(雍洋) <yongyang1030@163.com>

* [feature]: add Wan Sparge infer

* Update scripts/run_wan_t2v_sparge.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* [minor]: fix typo and use config style

* [minor]: remove breakpoint

* [feature]: add all attn class

* [minor]: remove args

* [minor]: remove shared weights

---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>