Skip to content

GitLab

Unverified Commit f7cdbcb5 authored by LiangLiu's avatar LiangLiu Committed by GitHub
Browse files

multi-person & animate & podcast (#554) 



- 服务化功能新增（前端+后端）：
1、seko-talk 模型支持多人输入
2、支持播客合成与管理
3、支持wan2.2 animate 模型

- 后端接口新增：
1、 基于火山的播客websocket合成接口，支持边合成边听
2、播客的查询管理接口
3、基于 yolo 的多人人脸检测接口
4、音频多人切分接口

- 推理代码侵入式修改
1、将 animate 相关的 输入文件路径（mask/image/pose等）从固定写死的config中移除到可变的input_info中
2、animate的预处理相关代码包装成接口供服务化使用

@xinyiqin

---------
Co-authored-by: default avatarqinxinyi <qxy118045534@163.com>
parent 61dd69ca
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 8222 additions and 771 deletions
lightx2v/deploy/server/frontend/src/components/MediaTemplate.vue
View file @ f7cdbcb5
......@@ -33,6 +33,7 @@ import {
audioHistory,
imageTemplates,
audioTemplates,
mergedTemplates,
mediaModalTab,
getImageHistory,
getAudioHistory,
......@@ -279,11 +280,11 @@ watch(audioTemplates, (newTemplates) => {
<span>{{ t('loading') }}</span>
</div>
</div>
<div v-if="imageTemplates.length > 0" class="columns-2 sm:columns-2 md:columns-3 lg:columns-4 xl:columns-5 gap-4 px-1">
<div v-for="template in imageTemplates" :key="template.filename"
@click="selectImageTemplate(template)"
<div v-if="mergedTemplates.filter(t => t.image).length > 0" class="columns-2 sm:columns-2 md:columns-3 lg:columns-4 xl:columns-5 gap-4 px-1">
<div v-for="template in mergedTemplates.filter(t => t.image)" :key="template.id"
@click="selectImageTemplate(template.image)"
class="break-inside-avoid mb-4 relative group cursor-pointer rounded-2xl overflow-hidden border border-black/8 dark:border-white/8 hover:border-[color:var(--brand-primary)]/50 dark:hover:border-[color:var(--brand-primary-light)]/50 transition-all hover:shadow-[0_4px_12px_rgba(var(--brand-primary-rgb),0.15)] dark:hover:shadow-[0_4px_12px_rgba(var(--brand-primary-light-rgb),0.2)]">
<img :src="getTemplateFileUrl(template.filename,'images')" :alt="template.filename"
<img :src="template.image.url" :alt="template.image.filename"
class="w-full h-auto object-contain" preload="metadata">
<div
class="absolute inset-0 bg-black/50 dark:bg-black/60 opacity-0 group-hover:opacity-100 transition-opacity flex items-center justify-center">
......@@ -412,31 +413,39 @@ watch(audioTemplates, (newTemplates) => {
<span>{{ t('loading') }}</span>
</div>
</div>
<div v-if="audioTemplates.length > 0" class="space-y-3 px-1">
<div v-for="(template, index) in audioTemplates" :key="template.filename"
@click="selectAudioTemplate(template)"
<div v-if="mergedTemplates.length > 0" class="space-y-3 px-1">
<div v-for="template in mergedTemplates" :key="template.id"
@click="selectAudioTemplate(template.audio)"
class="flex items-center gap-4 p-4 rounded-2xl border border-black/8 dark:border-white/8 hover:border-[color:var(--brand-primary)]/50 dark:hover:border-[color:var(--brand-primary-light)]/50 transition-all cursor-pointer bg-white/80 dark:bg-[#2c2c2e]/80 hover:bg-white dark:hover:bg-[#3a3a3c] hover:shadow-[0_4px_12px_rgba(var(--brand-primary-rgb),0.15)] dark:hover:shadow-[0_4px_12px_rgba(var(--brand-primary-light-rgb),0.2)] group">
<div
class="w-12 h-12 rounded-xl overflow-hidden flex-shrink-0 bg-[color:var(--brand-primary)]/10 dark:bg-[color:var(--brand-primary-light)]/15 flex items-center justify-center">
<img v-if="imageTemplates[index]?.url" :src="imageTemplates[index].url" :alt="t('audioTemplates')" class="w-full h-full object-cover" @error="imageTemplates[index].url = null" />
<img v-if="template.image?.url" :src="template.image.url" :alt="t('audioTemplates')" class="w-full h-full object-cover" @error="template.image.url = null" />
<i v-else class="fas fa-music text-[color:var(--brand-primary)] dark:text-[color:var(--brand-primary-light)] text-xl"></i>
</div>
<div class="flex-1 min-w-0">
<div class="text-[#86868b] dark:text-[#98989d] text-sm flex items-center gap-2 tracking-tight">
<span>{{ t('audioTemplates') }}</span>
<span class="text-[color:var(--brand-primary)] dark:text-[color:var(--brand-primary-light)]">•</span>
<span>{{ getDurationDisplay(template, true) }}</span>
<span v-if="template.audio" class="text-[color:var(--brand-primary)] dark:text-[color:var(--brand-primary-light)]">•</span>
<span v-if="template.audio">{{ getDurationDisplay(template.audio, true) }}</span>
</div>
</div>
<button @click.stop="handleAudioPreview(template, true)"
class="px-4 py-2 rounded-lg transition-all cursor-pointer relative z-10 flex items-center gap-2 flex-shrink-0 tracking-tight"
:class="isPlaying(template, true)
? 'text-red-500 dark:text-red-400 hover:text-red-600 dark:hover:text-red-300'
: 'text-[color:var(--brand-primary)] dark:text-[color:var(--brand-primary-light)] hover:text-[color:var(--brand-primary)]/80 dark:hover:text-[color:var(--brand-primary-light)]/80'"
style="pointer-events: auto;">
<i :class="isPlaying(template, true) ? 'fas fa-stop' : 'fas fa-play'"></i>
<span class="text-sm font-medium">{{ isPlaying(template, true) ? t('stop') : t('preview') }}</span>
</button>
<div class="flex items-center gap-2 flex-shrink-0">
<button v-if="template.image" @click.stop="selectImageTemplate(template.image)"
class="px-4 py-2 rounded-lg transition-all cursor-pointer relative z-10 flex items-center gap-2 tracking-tight text-[color:var(--brand-primary)] dark:text-[color:var(--brand-primary-light)] hover:text-[color:var(--brand-primary)]/80 dark:hover:text-[color:var(--brand-primary-light)]/80"
style="pointer-events: auto;">
<i class="fas fa-image"></i>
<span class="text-sm font-medium">{{ t('useImage') }}</span>
</button>
<button v-if="template.audio" @click.stop="handleAudioPreview(template.audio, true)"
class="px-4 py-2 rounded-lg transition-all cursor-pointer relative z-10 flex items-center gap-2 tracking-tight"
:class="isPlaying(template.audio, true)
? 'text-red-500 dark:text-red-400 hover:text-red-600 dark:hover:text-red-300'
: 'text-[color:var(--brand-primary)] dark:text-[color:var(--brand-primary-light)] hover:text-[color:var(--brand-primary)]/80 dark:hover:text-[color:var(--brand-primary-light)]/80'"
style="pointer-events: auto;">
<i :class="isPlaying(template.audio, true) ? 'fas fa-stop' : 'fas fa-play'"></i>
<span class="text-sm font-medium">{{ isPlaying(template.audio, true) ? t('stop') : t('preview') }}</span>
</button>
</div>
</div>
</div>
<div v-else
......
lightx2v/deploy/server/frontend/src/components/TaskCarousel.vue
View file @ f7cdbcb5
......@@ -188,7 +188,7 @@ const handleCancel = async () => {
await cancelTask(currentTask.value.task_id)
} catch (error) {
console.error('取消任务失败:', error)
showAlert('取消任务失败，请重试', 'danger')
showAlert(t('cancelTaskFailedRetry'), 'danger')
}
}
......@@ -201,7 +201,7 @@ const handleShareTask = async () => {
// copyShareLink 函数内部已经显示了带"查看"按钮的 alert，不需要再次调用
} catch (error) {
console.error('分享失败:', error)
showAlert('分享失败，请重试', 'danger')
showAlert(t('shareFailedRetry'), 'danger')
}
}
......@@ -213,7 +213,7 @@ const handleRetry = async () => {
await resumeTask(currentTask.value.task_id)
} catch (error) {
console.error('重试任务失败:', error)
showAlert('重试任务失败，请重试', 'danger')
showAlert(t('retryTaskFailedRetry'), 'danger')
}
}
......
lightx2v/deploy/server/frontend/src/components/TemplateDetails.vue
View file @ f7cdbcb5
......@@ -27,13 +27,17 @@ const showDetails = ref(false)
// 获取图片素材
const getImageMaterials = () => {
if (!selectedTemplate.value?.inputs?.input_image) return []
return [['input_image', getTemplateFileUrl(selectedTemplate.value.inputs.input_image, 'images')]]
const imageUrl = getTemplateFileUrl(selectedTemplate.value.inputs.input_image, 'images')
if (!imageUrl) return []
return [['input_image', imageUrl]]
}
// 获取音频素材
const getAudioMaterials = () => {
if (!selectedTemplate.value?.inputs?.input_audio) return []
return [['input_audio', getTemplateFileUrl(selectedTemplate.value.inputs.input_audio, 'audios')]]
const audioUrl = getTemplateFileUrl(selectedTemplate.value.inputs.input_audio, 'audios')
if (!audioUrl) return []
return [['input_audio', audioUrl]]
}
// 路由关闭功能
......@@ -127,7 +131,7 @@ onUnmounted(() => {
<video
v-if="selectedTemplate?.outputs?.output_video"
:src="getTemplateFileUrl(selectedTemplate.outputs.output_video,'videos')"
:poster="getTemplateFileUrl(selectedTemplate.inputs.input_image,'images')"
:poster="selectedTemplate?.inputs?.input_image ? getTemplateFileUrl(selectedTemplate.inputs.input_image,'images') : undefined"
class="w-full h-full object-contain"
controls
loop
......
lightx2v/deploy/server/frontend/src/components/TemplateDisplay.vue
View file @ f7cdbcb5
......@@ -139,7 +139,7 @@ onMounted(() => {
<!-- 视频预览 -->
<video v-if="item?.outputs?.output_video"
:src="getTemplateFileUrl(item.outputs.output_video,'videos')"
:poster="getTemplateFileUrl(item.inputs.input_image,'images')"
:poster="item?.inputs?.input_image ? getTemplateFileUrl(item.inputs.input_image,'images') : undefined"
class="w-full h-auto object-contain group-hover:scale-[1.02] transition-transform duration-200"
preload="auto" playsinline webkit-playsinline
@mouseenter="playVideo($event)" @mouseleave="pauseVideo($event)"
......@@ -147,11 +147,15 @@ onMounted(() => {
@ended="onVideoEnded($event)"
@error="onVideoError($event)"></video>
<!-- 图片缩略图 -->
<img v-else
<img v-else-if="item?.inputs?.input_image"
:src="getTemplateFileUrl(item.inputs.input_image,'images')"
:alt="item.params?.prompt || '模板图片'"
class="w-full h-auto object-contain group-hover:scale-[1.02] transition-transform duration-200"
@error="handleThumbnailError" />
<!-- 如果没有图片，显示占位符 -->
<div v-else class="w-full h-[200px] flex items-center justify-center bg-[#f5f5f7] dark:bg-[#1c1c1e]">
<i class="fas fa-image text-3xl text-[#86868b]/30 dark:text-[#98989d]/30"></i>
</div>
<!-- 移动端播放按钮 - Apple 风格 -->
<button v-if="item?.outputs?.output_video"
@click.stop="toggleVideoPlay($event)"
......@@ -211,7 +215,7 @@ onMounted(() => {
<!-- 视频预览 -->
<video v-if="item?.outputs?.output_video"
:src="getTemplateFileUrl(item.outputs.output_video,'videos')"
:poster="getTemplateFileUrl(item.inputs.input_image,'images')"
:poster="item?.inputs?.input_image ? getTemplateFileUrl(item.inputs.input_image,'images') : undefined"
class="w-full h-auto object-contain group-hover:scale-[1.02] transition-transform duration-200"
preload="auto" playsinline webkit-playsinline
@mouseenter="playVideo($event)" @mouseleave="pauseVideo($event)"
......@@ -220,7 +224,7 @@ onMounted(() => {
@error="onVideoError($event)"></video>
<!-- 图片缩略图 -->
<img v-else
:src="getTemplateFileUrl(item.inputs.input_image,'images')"
:src="item?.inputs?.input_image ? getTemplateFileUrl(item.inputs.input_image,'images') : undefined"
:alt="item.params?.prompt || '模板图片'"
class="w-full h-auto object-contain group-hover:scale-[1.02] transition-transform duration-200"
@error="handleThumbnailError" />
......
lightx2v/deploy/server/frontend/src/components/TopBar.vue
View file @ f7cdbcb5
......@@ -15,14 +15,9 @@ import {
initTheme,
toggleTheme,
getThemeIcon,
switchToCreateView
} from '../utils/other'
// 导航到主页面
const goToHome = () => {
showTemplateDetailModal.value = false
showTaskDetailModal.value = false
router.push({ name: 'Generate' })
}
// 初始化主题
onMounted(() => {
......@@ -37,7 +32,7 @@ onMounted(() => {
<div class="flex justify-between items-center max-w-full mx-auto px-6 py-3">
<!-- 左侧 Logo -->
<div class="flex items-center">
<button @click="goToHome"
<button @click="switchToCreateView"
class="flex items-center gap-2.5 px-3 py-2 bg-transparent border-0 rounded-[10px] cursor-pointer transition-all duration-200 hover:bg-black/4 dark:hover:bg-white/6 hover:-translate-y-px active:scale-[0.97]"
:title="t('goToHome')">
<img src="../../public/logo.svg" alt="LightX2V" class="w-6 h-6 sm:w-6 sm:h-6 md:w-8 md:h-8 lg:w-8 lg:h-8" loading="lazy" />
......
lightx2v/deploy/server/frontend/src/router/index.js
View file @ f7cdbcb5
......@@ -6,7 +6,9 @@ import Generate from '../components/Generate.vue'
import Projects from '../components/Projects.vue'
import Inspirations from '../components/Inspirations.vue'
import Share from '../views/Share.vue'
import PodcastGenerate from '../views/PodcastGenerate.vue'
import { showAlert } from '../utils/other'
import i18n from '../utils/i18n'
const routes = [
{
......@@ -22,6 +24,12 @@ const routes = [
{
path: '/share/:shareId', name: 'Share', component: Share, meta: { requiresAuth: false }
},
{
path: '/podcast_generate', name: 'PodcastGenerate', component: PodcastGenerate, meta: { requiresAuth: true }
},
{
path: '/podcast_generate/:session_id', name: 'PodcastSession', component: PodcastGenerate, meta: { requiresAuth: true }
},
{
path: '/home',
component: Layout,
......@@ -114,7 +122,7 @@ router.beforeEach((to, from, next) => {
next('/login')
// 延迟显示提示，确保路由跳转完成
setTimeout(() => {
showAlert('请先登录', 'warning')
showAlert(i18n.global.t('pleaseLoginFirst'), 'warning')
}, 100)
return
}
......
lightx2v/deploy/server/frontend/src/views/Layout.vue
View file @ f7cdbcb5
......@@ -38,7 +38,7 @@ const handleTTSComplete = (audioBlob) => {
showVoiceTTSModal.value = false
// 显示成功提示
showAlert('语音合成完成，已自动添加到音频素材', 'success')
showAlert(t('ttsCompleted'), 'success')
}
</script>
......
lightx2v/deploy/server/frontend/src/views/PodcastGenerate.vue 0 → 100644
View file @ f7cdbcb5
This diff is collapsed.
lightx2v/deploy/task_manager/__init__.py
View file @ f7cdbcb5
This diff is collapsed.
lightx2v/deploy/worker/__main__.py
View file @ f7cdbcb5
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment