GitLab

Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm" (#40438)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: simon-mo <simon@inferact.ai>

Signed-off-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com>
Co-authored-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com>

Signed-off-by: wuyingjun <wuyingjun_yewu@cmss.chinamobile.com>

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>

Signed-off-by: z1ying <tzzying@outlook.com>

Signed-off-by: Nithin Chalapathi <nithin.ch10@gmail.com>
Signed-off-by: Nithin Chalapathi <nithinc@berkeley.edu>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Michele Gazzetti <michele.gazzetti1@ibm.com>

Signed-off-by: mgoin <mgoin64@gmail.com>

Signed-off-by: JaredforReal <w13431838023@gmail.com>

Signed-off-by: daiyu1111 <2356690121@qq.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: Jigang Zhou <zjg0907008@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: juliendenize <julien.denize@mistral.ai>

[BugFix] KeyError on scope["method"] for realtime api websocket in AuthenticationMiddleware (#36934)
Signed-off-by: daniebrill <50454544+daniebrill@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yuwei <yuwei@dev.local>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Signed-off-by: Ibrahim Arshad <38925737+ibrahim1023@users.noreply.github.com>
Signed-off-by: Li <chuali@amd.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Elham Harirpoush <elham.harirpoush@arm.com>
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: jackcfwang <jackcfwang@tencent.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
Signed-off-by: Richard Zou <zou3519@gmail.com>
Signed-off-by: milesial <milesial@users.noreply.github.com>
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: whx-sjtu <2952154980@qq.com>
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Signed-off-by: jatseng-ai <jatseng@amd.com>
Signed-off-by: jatseng-ai <janet.tseng@amd.com>
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Signed-off-by: xaguilar-amd <xaguilar@amd.com>
Signed-off-by: rdondeti <ravitez.dondeti@gmail.com>
Signed-off-by: Ravitez Dondeti <ravitez.dondeti@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Peter Nguyen <petern0408@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: manu <fortin.emmanuel@gmail.com>
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Rahul-Tuli <rtuli@redhat.com>
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: leeyongjun <jqueen.astro@gmail.com>
Signed-off-by: Ziying Tao <tzzying@outlook.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Vibhav Agarwal <vibhavagarwal5@gmail.com>
Signed-off-by: ShubyM <shubymishra20@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: r266-tech <r266.tech@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Animesh Jain <anijain@umich.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Signed-off-by: EricccYang <yangyang4991@gmail.com>
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Signed-off-by: sihao.li <sihao.li@intel.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Santino Ramos <santinor@inferact.ai>
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: George-ao <yuyiao772@gmail.com>
Signed-off-by: Yuyi Ao <yuyiao772@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com>
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: zzaebok <44357534+zzaebok@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: yuwei <yuwei@dev.local>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Ibrahim Arshad <38925737+ibrahim1023@users.noreply.github.com>
Co-authored-by: Chuan (Richard) Li <chuali@amd.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Ganesh R <ganesh.r@amd.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com>
Co-authored-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Co-authored-by: vnadathur <glvikramn@gmail.com>
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Elham <elham.harirpoush@arm.com>
Co-authored-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Chaofan Wang <jackcfwang@tencent.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Injae Ryou <injaeryou@gmail.com>
Co-authored-by: Richard Zou <zou3519@users.noreply.github.com>
Co-authored-by: milesial <milesial@users.noreply.github.com>
Co-authored-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com>
Co-authored-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com>
Co-authored-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: PatchyTIS <58251192+PatchouliTIS@users.noreply.github.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: jatseng-ai <janet.tseng@amd.com>
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: xaguilar-amd <xavier.aguilarfruto@amd.com>
Co-authored-by: Ravitez Dondeti <dondetir@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Peter Nguyen <petern0408@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Jesus Federico <14651+jefp@users.noreply.github.com>
Co-authored-by: Manu <efortin@users.noreply.github.com>
Co-authored-by: zhanqiuhu <49648934+ZhanqiuHu@users.noreply.github.com>
Co-authored-by: yzong-rh <yzong@redhat.com>
Co-authored-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: Rahul-Tuli <rtuli@redhat.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Co-authored-by: Lee Yongjun <35302114+elwhyjay@users.noreply.github.com>
Co-authored-by: z1ying <55220715+z1ying@users.noreply.github.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Co-authored-by: Vibhav Agarwal <vibhavagarwal5@gmail.com>
Co-authored-by: vibhav-agarwal <vibhav.agarwal@glance.com>
Co-authored-by: ShubyM <shubymishra20@gmail.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: EdalatiAli <aliedalati@cohere.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: r266-tech <r2668940489@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Le Yang <562593859@qq.com>
Co-authored-by: Animesh Jain <anijain@umich.edu>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Zhengxu Chen <zhxchen17@fb.com>
Co-authored-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Co-authored-by: maobaolong <baoloongmao@tencent.com>
Co-authored-by: sihao_li <165983188+1643661061leo@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: zofia <110436990+zufangzhu@users.noreply.github.com>
Co-authored-by: Tihomir Elek <tiho.elek@gmail.com>
Co-authored-by: Yi Liu <yi4.liu@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Santino Ramos <51103228+santiramos27@users.noreply.github.com>
Co-authored-by: haosdent <haosdent@gmail.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Yuyi Ao <yuyiao772@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: mukesh-hai <mukesh@hippocraticai.com>
Co-authored-by: Pedram Razavi <pedram@sierra.ai>

Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Signed-off-by: rishitdholakia13 <123388671+rishitdholakia13@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

[Frontend] Offload blocking preprocessing & postprocessing ops to thread pool for pooling entrypoints. (#39763)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: noobhappylife <aratar1991@hotmail.com>

[Refactor][Parser] Migrate chat completion auto-tool/reasoning/plain streaming to parse_delta (#39446)
Signed-off-by: sfeng33 <4florafeng@gmail.com>

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>

Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: noobhappylife <aratar1991@hotmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Andrew Xia <axia@meta.com>

Signed-off-by: sfeng33 <4florafeng@gmail.com>

Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com>

Signed-off-by: Hyeonki Hong <hyeonki.hong@moreh.io>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Sergey Zinchenko <sergey.zinchenko.rnd@gmail.com>

Signed-off-by: Chang Su <chang.s.su@oracle.com>

Co-authored-by: majianhan <majianhan@kylinos.cn>

Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>