Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
wangsen
MinerU
Commits
ece7f8d5
Unverified
Commit
ece7f8d5
authored
Oct 15, 2024
by
Kaiwen Liu
Committed by
GitHub
Oct 15, 2024
Browse files
Merge pull request #6 from opendatalab/dev
Dev
parents
98362a6e
702b6ac9
Changes
551
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
966 additions
and
0 deletions
+966
-0
projects/web/src/pages/extract/formula/formula-popover/index.module.scss
...c/pages/extract/formula/formula-popover/index.module.scss
+18
-0
projects/web/src/pages/extract/formula/formula-popover/index.tsx
...s/web/src/pages/extract/formula/formula-popover/index.tsx
+54
-0
projects/web/src/pages/extract/formula/formula-upload/index.module.scss
...rc/pages/extract/formula/formula-upload/index.module.scss
+87
-0
projects/web/src/pages/extract/formula/formula-upload/index.tsx
...ts/web/src/pages/extract/formula/formula-upload/index.tsx
+132
-0
projects/web/src/pages/extract/formula/index.module.scss
projects/web/src/pages/extract/formula/index.module.scss
+0
-0
projects/web/src/pages/extract/formula/index.tsx
projects/web/src/pages/extract/formula/index.tsx
+11
-0
projects/web/src/pages/extract/index.module.scss
projects/web/src/pages/extract/index.module.scss
+0
-0
projects/web/src/pages/extract/index.tsx
projects/web/src/pages/extract/index.tsx
+0
-0
projects/web/src/pages/extract/table/index.tsx
projects/web/src/pages/extract/table/index.tsx
+5
-0
projects/web/src/pages/extract/table/table-detail/index.tsx
projects/web/src/pages/extract/table/table-detail/index.tsx
+5
-0
projects/web/src/pages/home.module.scss
projects/web/src/pages/home.module.scss
+0
-0
projects/web/src/pages/home.tsx
projects/web/src/pages/home.tsx
+75
-0
projects/web/src/routes/index.tsx
projects/web/src/routes/index.tsx
+17
-0
projects/web/src/store/jobProgress.ts
projects/web/src/store/jobProgress.ts
+183
-0
projects/web/src/store/languageStore.ts
projects/web/src/store/languageStore.ts
+47
-0
projects/web/src/store/mdStore.ts
projects/web/src/store/mdStore.ts
+250
-0
projects/web/src/styles/variable.scss
projects/web/src/styles/variable.scss
+1
-0
projects/web/src/types/extract-task-type.ts
projects/web/src/types/extract-task-type.ts
+21
-0
projects/web/src/utils/download.ts
projects/web/src/utils/download.ts
+44
-0
projects/web/src/utils/locale.ts
projects/web/src/utils/locale.ts
+16
-0
No files found.
projects/web/src/pages/extract/formula/formula-popover/index.module.scss
0 → 100644
View file @
ece7f8d5
.formulaPopover
{
:global
{
.ant-popover-content
,
.ant-popover-inner
{
border-radius
:
12px
!
important
;
overflow
:
hidden
;
box-shadow
:
0px
8px
26px
0px
rgba
(
0
,
0
,
0
,
0
.12
);
}
.ant-popover-inner-content
{
padding
:
24px
!
important
;
}
.ant-popover-arrow
{
display
:
none
!
important
;
}
}
}
projects/web/src/pages/extract/formula/formula-popover/index.tsx
0 → 100644
View file @
ece7f8d5
import
React
,
{
ReactNode
}
from
"
react
"
;
import
{
Popover
}
from
"
antd
"
;
import
IconFont
from
"
@/components/icon-font
"
;
import
{
useIntl
}
from
"
react-intl
"
;
import
style
from
"
./index.module.scss
"
;
interface
IFormulaPopoverProps
{
type
:
string
;
text
?:
string
|
ReactNode
;
}
const
FormulaPopover
=
({
type
,
text
}:
IFormulaPopoverProps
)
=>
{
const
{
formatMessage
}
=
useIntl
();
const
content
=
(
<
div
className
=
"flex flex-col w-[20rem] items-center"
>
{
/* 顺序反了 */
}
{
formatMessage
({
id
:
type
===
"
detect
"
?
"
extractor.formula.popover.extract
"
:
"
extractor.formula.popover.detect
"
,
})
}
<
img
className
=
"w-full mt-4"
src
=
{
type
===
"
extract
"
?
"
https://static.openxlab.org.cn/opendatalab/assets/pdf/svg/extract-formula-extract.svg
"
:
"
https://static.openxlab.org.cn/opendatalab/assets/pdf/svg/extract-formula-detect.svg
"
}
alt
=
"formula-popover"
/>
</
div
>
);
return
(
<
span
className
=
{
""
}
>
<
Popover
content
=
{
content
}
placement
=
"right"
showArrow
=
{
false
}
overlayClassName
=
{
style
.
formulaPopover
}
>
<
span
className
=
"group inline-flex items-center"
>
{
text
}
<
IconFont
type
=
"icon-QuestionCircleOutlined"
className
=
"text-[#121316]/[0.6] text-[15px] mt-[2px] leading-[1rem] group-hover:text-[#0D53DE]"
/>
</
span
>
</
Popover
>
</
span
>
);
};
export
default
FormulaPopover
;
projects/web/src/pages/extract/formula/formula-upload/index.module.scss
0 → 100644
View file @
ece7f8d5
.uploadText
{
font-feature-settings
:
'liga'
off
,
'clig'
off
;
font-family
:
"PingFang SC"
;
font-size
:
18px
;
font-style
:
normal
;
font-weight
:
600
;
line-height
:
24px
;
/* 133.333% */
background
:
linear-gradient
(
107deg
,
#38A0FF
-24
.14%
,
#0D53DE
30
.09%
,
#5246FF
86
.61%
);
background-clip
:
text
;
-webkit-background-clip
:
text
;
-webkit-text-fill-color
:
transparent
;
}
.uploadDescText
{
font-size
:
13px
;
line-height
:
20px
;
font-weight
:
400
;
background
:
linear-gradient
(
107deg
,
rgba
(
18
,
19
,
22
,
0
.6
)
-24
.14%
,
rgba
(
18
,
19
,
22
,
0
.6
)
100
.09%
);
background-clip
:
text
;
-webkit-background-clip
:
text
;
-webkit-text-fill-color
:
transparent
;
margin-bottom
:
1rem
;
margin-top
:
0
.5rem
;
}
.linearText
{
font-size
:
13px
;
line-height
:
20px
;
font-weight
:
400
;
background
:
linear-gradient
(
107deg
,
rgba
(
18
,
19
,
22
,
0
.6
)
-24
.14%
,
rgba
(
18
,
19
,
22
,
0
.6
)
100
.09%
);
background-clip
:
text
;
-webkit-background-clip
:
text
;
-webkit-text-fill-color
:
transparent
;
&
-item
{
font-weight
:
400
;
font-size
:
13px
;
line-height
:
20px
;
margin-right
:
1rem
;
background
:
linear-gradient
(
107deg
,
#38A0FF
-24
.14%
,
#0D53DE
30
.09%
,
#5246FF
86
.61%
);
background-clip
:
text
;
-webkit-background-clip
:
text
;
-webkit-text-fill-color
:
transparent
;
&
:hover
{
background
:
#3477EB
;
background
:
linear-gradient
(
107deg
,
#3477EB
-24
.14%
,
#3477EB
100
.09%
);
background-clip
:
text
;
-webkit-background-clip
:
text
;
-webkit-text-fill-color
:
transparent
;
}
}
}
.uploadSection
{
border-radius
:
12px
;
border
:
1px
dashed
var
(
---
Brand1-6
,
#0D53DE
);
background
:
linear-gradient
(
180deg
,
rgba
(
92
,
147
,
255
,
0
.10
)
-130
.23%
,
rgba
(
255
,
255
,
255
,
1
)
83
.57%
);
display
:
flex
;
flex-direction
:
column
;
justify-content
:
center
;
align-items
:
center
;
filter
:
blur
(
0px
);
height
:
280px
!
important
;
width
:
600px
!
important
;
&
:hover
{
background
:
linear-gradient
(
180deg
,
rgb
(
245
,
248
,
255
)
-130
.23%
,
rgb
(
245
,
248
,
255
)
83
.57%
);
}
}
.textBtn
{
background-image
:
none
!
important
;
background-clip
:
text
;
-webkit-background-clip
:
text
;
-webkit-text-fill-color
:
transparent
;
background
:
linear-gradient
(
111deg
,
#0D53DE
-21
.44%
,
#5246FF
102%
)
!
important
;
background-clip
:
text
!
important
;
-webkit-background-clip
:
text
!
important
;
-webkit-text-fill-color
:
transparent
!
important
;
height
:
1
.5rem
!
important
;
font-weight
:
600
;
height
:
280px
!
important
;
width
:
600px
!
important
;
overflow
:
hidden
;
}
projects/web/src/pages/extract/formula/formula-upload/index.tsx
0 → 100644
View file @
ece7f8d5
import
{
useIntl
}
from
"
react-intl
"
;
import
IconFont
from
"
@/components/icon-font
"
;
import
{
useState
}
from
"
react
"
;
import
cls
from
"
classnames
"
;
import
{
ExtractorUploadButton
}
from
"
../../components/pdf-upload-button
"
;
import
UploadBg
from
"
@/assets/imgs/online.experience/file-upload-bg.svg
"
;
import
style
from
"
./index.module.scss
"
;
import
{
SubmitRes
}
from
"
@/api/extract
"
;
import
{
ADD_TASK_LIST
}
from
"
@/constant/event
"
;
import
{
FORMULA_TYPE
}
from
"
@/types/extract-task-type
"
;
import
{
useNavigate
}
from
"
react-router-dom
"
;
const
FORMULA_ITEM_LIST
=
[
{
type
:
FORMULA_TYPE
.
detect
,
[
`zh-CN-name`
]:
"
公式检测
"
,
[
`en-US-name`
]:
"
Formula Detection
"
,
},
{
type
:
FORMULA_TYPE
.
extract
,
[
`zh-CN-name`
]:
"
公式识别
"
,
[
`en-US-name`
]:
"
Formula Recognition
"
,
},
];
const
FormulaUpload
=
()
=>
{
const
navigate
=
useNavigate
();
const
{
formatMessage
,
locale
}
=
useIntl
();
const
[
formulaType
,
setFormulaType
]
=
useState
(
FORMULA_TYPE
.
detect
);
const
afterUploadSuccess
=
(
data
:
SubmitRes
)
=>
{
navigate
(
`/OpenSourceTools/Extractor/formula/
${
data
?.
id
}
`
);
setTimeout
(()
=>
{
document
.
dispatchEvent
(
new
CustomEvent
(
ADD_TASK_LIST
,
{
detail
:
data
,
})
);
},
10
);
};
const
afterAsyncCheck
=
()
=>
{
return
Promise
.
resolve
(
true
);
};
return
(
<
div
className
=
"relative w-full h-full flex flex-col items-center justify-center "
>
<
div
className
=
"absolute top-10 left-8 hover:!text-[#0D53DE] cursor-pointer"
onClick
=
{
()
=>
navigate
(
"
/OpenSourceTools/Extractor
"
)
}
>
<
IconFont
type
=
"icon-fanhui"
className
=
"mr-2"
/>
<
span
>
{
formatMessage
({
id
:
"
extractor.home
"
})
}
</
span
>
</
div
>
<
div
className
=
"translate-y-[-60px] flex flex-col items-center "
>
<
div
className
=
"mb-[2.25rem]"
>
{
FORMULA_ITEM_LIST
.
map
((
i
)
=>
{
return
(
<
span
key
=
{
i
.
type
}
onClick
=
{
()
=>
setFormulaType
(
i
?.
type
)
}
className
=
{
cls
(
"
relative text-[1.5rem] text-[#121316] cursor-pointer mx-[1.5rem]
"
,
formulaType
===
i
?.
type
&&
"
!text-[#0D53DE] font-semibold
"
)
}
>
{
i
?.[
`
${
locale
||
"
zh-CN
"
}
-name`
as
"
en-US-name
"
]
}
{
formulaType
===
i
?.
type
&&
(
<
span
className
=
"absolute bottom-[-0.75rem] right-[50%] translate-x-[50%] w-[3rem] bg-[#0D53DE] rounded-[2px] h-[0.25rem]"
></
span
>
)
}
</
span
>
);
})
}
</
div
>
<
div
className
=
"text-[1.25rem] text-[#121316]/[0.8] mb-[3rem] text-center w-max-[50rem]"
>
{
formatMessage
({
id
:
formulaType
===
"
extract
"
?
"
extractor.formula.title2
"
:
"
extractor.formula.title
"
,
})
}
</
div
>
<
ExtractorUploadButton
accept
=
"image/png, image/jpg, .png ,.jpg"
afterUploadSuccess
=
{
afterUploadSuccess
}
taskType
=
"extract"
afterAsyncCheck
=
{
afterAsyncCheck
}
extractType
=
{
formulaType
===
FORMULA_TYPE
.
extract
?
"
formula-extract
"
:
"
formula-detect
"
}
className
=
{
style
.
textBtn
}
showIcon
=
{
false
}
text
=
{
<
div
className
=
{
cls
(
style
.
uploadSection
,
"
border-[1px] border-dashed border-[#0D53DE] rounded-xl flex flex-col items-center justify-center
"
)
}
>
<
img
src
=
{
UploadBg
}
className
=
"mb-4"
/>
<
span
className
=
{
cls
(
style
.
uploadText
,
"
text-[18px] leading-[20px]
"
)
}
>
{
formatMessage
({
id
:
"
extractor.formula.upload.text
"
})
}
</
span
>
<
span
className
=
{
cls
(
style
.
uploadDescText
)
}
>
{
formatMessage
({
id
:
"
extractor.formula.upload.accept
"
})
}
</
span
>
<
div
>
<
span
className
=
{
cls
(
style
.
linearText
,
"
cursor-pointer
"
)
}
>
{
formatMessage
({
id
:
"
extractor.formula.upload.try
"
,
})
}
</
span
>
</
div
>
</
div
>
}
></
ExtractorUploadButton
>
</
div
>
<
div
className
=
"absolute bottom-[1.5rem] text-[13px] text-[#121316]/[0.35] text-center leading-[20px] max-w-[64rem]"
>
{
formatMessage
({
id
:
"
extractor.law
"
,
})
}
</
div
>
</
div
>
);
};
export
default
FormulaUpload
;
projects/web
_api/web_api/config/__init__.py
→
projects/web
/src/pages/extract/formula/index.module.scss
View file @
ece7f8d5
File moved
projects/web/src/pages/extract/formula/index.tsx
0 → 100644
View file @
ece7f8d5
import
{
Outlet
}
from
"
react-router-dom
"
;
const
Formula
=
()
=>
{
return
(
<
div
className
=
"relative w-full h-full flex flex-col items-center justify-center "
>
<
Outlet
/>
</
div
>
);
};
export
default
Formula
;
projects/web
_api/web_api/static/__init__.py
→
projects/web
/src/pages/extract/index.module.scss
View file @
ece7f8d5
File moved
web_api/requirements.txt
→
projects/web/src/pages/extract/index.tsx
View file @
ece7f8d5
File moved
projects/web/src/pages/extract/table/index.tsx
0 → 100644
View file @
ece7f8d5
const
ExtractorTable
=
()
=>
{
return
<>
ExtractorTable
</>;
};
export
default
ExtractorTable
;
projects/web/src/pages/extract/table/table-detail/index.tsx
0 → 100644
View file @
ece7f8d5
const
TableDetail
=
()
=>
{
return
<>
TableDetail
</>;
};
export
default
TableDetail
;
projects/web/src/pages/home.module.scss
0 → 100644
View file @
ece7f8d5
projects/web/src/pages/home.tsx
0 → 100644
View file @
ece7f8d5
"
use client
"
;
import
ErrorBoundary
from
"
@/components/error-boundary
"
;
import
styles
from
"
./home.module.scss
"
;
import
{
SlotID
,
Path
}
from
"
@/constant/route
"
;
import
{
BrowserRouter
,
Routes
,
Route
,
Outlet
,
Navigate
,
useLocation
,
HashRouter
,
}
from
"
react-router-dom
"
;
import
{
ExtractorSide
}
from
"
./extract-side
"
;
import
{
LanguageProvider
}
from
"
@/context/language-provider
"
;
import
PDFUpload
from
"
@/pages/extract/components/pdf-upload
"
;
import
PDFExtractionJob
from
"
@/pages/extract/components/pdf-extraction
"
;
export
function
WindowContent
()
{
const
location
=
useLocation
();
const
isHome
=
location
.
pathname
===
Path
.
Home
;
return
(
<>
<
ExtractorSide
className
=
{
isHome
?
styles
[
"
sidebar-show
"
]
:
""
}
/>
<
div
className
=
"flex-1"
>
<
Outlet
/>
</
div
>
</>
);
}
function
Screen
()
{
const
renderContent
=
()
=>
{
return
(
<
div
className
=
"w-full h-full flex"
id
=
{
SlotID
.
AppBody
}
>
<
Routes
>
<
Route
path
=
"/"
element
=
{
<
WindowContent
/>
}
>
<
Route
index
element
=
{
<
Navigate
to
=
"/OpenSourceTools/Extractor/PDF"
replace
/>
}
/>
<
Route
path
=
"/OpenSourceTools/Extractor/PDF"
element
=
{
<
PDFUpload
/>
}
/>
<
Route
path
=
"/OpenSourceTools/Extractor/PDF/:jobID"
element
=
{
<
PDFExtractionJob
/>
}
/>
<
Route
path
=
"*"
element
=
{
<
Navigate
to
=
"/OpenSourceTools/Extractor/PDF"
replace
/>
}
/>
</
Route
>
</
Routes
>
</
div
>
);
};
return
<>
{
renderContent
()
}
</>;
}
export
function
Home
()
{
return
(
<
ErrorBoundary
>
<
LanguageProvider
>
<
HashRouter
>
<
Screen
/>
</
HashRouter
>
</
LanguageProvider
>
</
ErrorBoundary
>
);
}
projects/web/src/routes/index.tsx
0 → 100644
View file @
ece7f8d5
import
{
Routes
,
Route
}
from
"
react-router-dom
"
;
import
PDFUpload
from
"
@/pages/extract/components/pdf-upload
"
;
import
PDFExtractionJob
from
"
@/pages/extract/components/pdf-extraction
"
;
function
AppRoutes
()
{
return
(
<>
<
Route
path
=
"/OpenSourceTools/Extractor/PDF"
element
=
{
<
PDFUpload
/>
}
/>
<
Route
path
=
"/OpenSourceTools/Extractor/PDF/:jobID"
element
=
{
<
PDFExtractionJob
/>
}
/>
</>
);
}
export
default
AppRoutes
;
projects/web/src/store/jobProgress.ts
0 → 100644
View file @
ece7f8d5
import
{
getExtractTaskIdProgress
,
getPdfExtractQueue
,
TaskIdResItem
,
}
from
"
@/api/extract
"
;
import
{
create
}
from
"
zustand
"
;
import
{
useCallback
,
useEffect
,
useRef
,
useState
}
from
"
react
"
;
import
{
useParams
}
from
"
react-router-dom
"
;
import
{
UPDATE_TASK_LIST
}
from
"
@/constant/event
"
;
import
{
useQuery
}
from
"
@tanstack/react-query
"
;
interface
ExtractorState
{
taskInfo
:
TaskIdResItem
;
queueLoading
:
boolean
|
null
;
interfaceError
:
boolean
;
setTaskInfo
:
(
taskInfo
:
TaskIdResItem
)
=>
void
;
setQueueLoading
:
(
loading
:
boolean
|
null
)
=>
void
;
setInterfaceError
:
(
error
:
boolean
)
=>
void
;
}
const
defaultTaskInfo
:
TaskIdResItem
=
{
id
:
0
,
rank
:
0
,
state
:
"
pending
"
,
url
:
""
,
type
:
"
unknown
"
,
queues
:
-
1
,
};
const
useExtractorStore
=
create
<
ExtractorState
>
((
set
)
=>
({
taskInfo
:
defaultTaskInfo
,
queueLoading
:
null
,
interfaceError
:
false
,
setTaskInfo
:
(
taskInfo
:
any
)
=>
set
({
taskInfo
}),
setQueueLoading
:
(
loading
)
=>
set
({
queueLoading
:
loading
}),
setInterfaceError
:
(
error
)
=>
set
({
interfaceError
:
error
}),
}));
export
const
useJobExtraction
=
()
=>
{
const
{
jobID
}
=
useParams
<
{
jobID
:
string
}
>
();
const
{
setTaskInfo
,
setQueueLoading
,
queueLoading
,
taskInfo
,
interfaceError
,
setInterfaceError
,
}
=
useExtractorStore
();
const
timeoutRef
=
useRef
<
NodeJS
.
Timeout
|
null
>
(
null
);
const
[
isPolling
,
setIsPolling
]
=
useState
(
true
);
const
stopTaskLoading
=
()
=>
{
setQueueLoading
(
false
);
};
// Query for task progress
const
taskProgressQuery
=
useQuery
({
queryKey
:
[
"
taskProgress
"
,
jobID
],
queryFn
:
()
=>
{
setQueueLoading
(
true
);
setIsPolling
(
true
);
return
getExtractTaskIdProgress
(
jobID
!
)
.
then
((
res
)
=>
{
if
(
res
?.
state
===
"
done
"
||
res
?.
state
===
"
failed
"
)
{
stopTaskLoading
();
document
.
dispatchEvent
(
new
CustomEvent
(
"
UPDATE_TASK_LIST
"
,
{
detail
:
{
state
:
res
.
state
,
id
:
jobID
},
})
);
}
if
(
res
)
{
setTaskInfo
(
res
);
}
return
res
;
})
.
catch
(()
=>
{
stopTaskLoading
();
setTaskInfo
({
state
:
"
failed
"
});
});
},
enabled
:
false
,
});
// Query for queue status
const
queueStatusQuery
=
useQuery
({
queryKey
:
[
"
queueStatus
"
,
jobID
],
queryFn
:
async
()
=>
{
setQueueLoading
(
true
);
const
response
=
await
getPdfExtractQueue
(
jobID
).
then
((
res
)
=>
{
// setTaskInfo({ rand: "failed" });
if
(
res
)
{
const
targetPendingRunningJob
=
res
?.
filter
(
(
i
)
=>
String
(
i
.
id
)
===
jobID
)?.[
0
];
if
(
targetPendingRunningJob
)
{
setTaskInfo
(
targetPendingRunningJob
);
}
else
{
setIsPolling
(
false
);
setQueueLoading
(
false
);
getExtractTaskIdProgress
(
jobID
!
).
then
((
res
)
=>
{
setTaskInfo
(
res
as
any
);
});
}
}
return
res
;
});
return
response
;
},
enabled
:
isPolling
&&
(
taskProgressQuery
?.
data
?.
state
===
"
running
"
||
taskProgressQuery
?.
data
?.
state
===
"
pending
"
),
refetchInterval
:
2000
,
// Poll every 2 seconds
});
useEffect
(()
=>
{
if
(
taskProgressQuery
.
data
?.
state
===
"
done
"
)
{
stopTaskLoading
();
setInterfaceError
(
false
);
setIsPolling
(
false
);
if
(
timeoutRef
.
current
)
{
clearTimeout
(
timeoutRef
.
current
);
}
else
{
timeoutRef
.
current
=
setTimeout
(()
=>
{
document
.
dispatchEvent
(
new
CustomEvent
(
UPDATE_TASK_LIST
,
{
detail
:
{
state
:
"
done
"
,
jobID
},
})
);
},
10
);
}
}
else
if
(
taskProgressQuery
.
data
?.
state
===
"
failed
"
)
{
stopTaskLoading
();
setInterfaceError
(
true
);
setIsPolling
(
false
);
if
(
timeoutRef
.
current
)
{
clearTimeout
(
timeoutRef
.
current
);
}
else
{
timeoutRef
.
current
=
setTimeout
(()
=>
{
document
.
dispatchEvent
(
new
CustomEvent
(
UPDATE_TASK_LIST
,
{
detail
:
{
state
:
"
failed
"
,
jobID
},
})
);
},
10
);
}
}
// TIP这里得用taskInfo
},
[
taskProgressQuery
.
data
]);
const
refreshQueue
=
()
=>
{
// stop last ID polling
setIsPolling
(
false
);
setTaskInfo
(
defaultTaskInfo
);
taskProgressQuery
.
refetch
();
};
useEffect
(()
=>
{
if
(
jobID
)
{
// stop last ID polling d
setTaskInfo
(
defaultTaskInfo
);
taskProgressQuery
.
refetch
();
}
},
[
jobID
]);
return
{
taskInfo
:
taskInfo
,
isLoading
:
queueLoading
,
isError
:
interfaceError
||
taskProgressQuery
.
isError
||
queueStatusQuery
.
isError
,
refreshQueue
,
};
};
projects/web/src/store/languageStore.ts
0 → 100644
View file @
ece7f8d5
import
{
create
}
from
"
zustand
"
;
import
{
Language
}
from
"
@/constant
"
;
import
{
LOCALE_STORAGE_KEY
}
from
"
@/constant/storage
"
;
type
LanguageType
=
(
typeof
Language
)[
keyof
typeof
Language
];
type
LanguageStore
=
{
language
:
LanguageType
;
setLanguage
:
(
language
:
LanguageType
)
=>
void
;
toggleLanguage
:
()
=>
void
;
};
const
getInitialLanguage
=
():
LanguageType
=>
{
// Try to get language setting from localStorage
const
savedLanguage
=
localStorage
.
getItem
(
LOCALE_STORAGE_KEY
)
as
LanguageType
;
if
(
savedLanguage
&&
Object
.
values
(
Language
).
includes
(
savedLanguage
))
{
return
savedLanguage
;
}
// If no valid language setting in localStorage, try to get browser language
const
browserLanguage
=
navigator
.
language
.
toLowerCase
();
if
(
browserLanguage
.
startsWith
(
"
zh
"
))
{
return
Language
.
ZH_CN
;
}
else
if
(
browserLanguage
.
startsWith
(
"
en
"
))
{
return
Language
.
EN_US
;
}
// Default to Chinese
return
Language
.
ZH_CN
;
};
export
const
useLanguageStore
=
create
<
LanguageStore
>
((
set
)
=>
({
language
:
getInitialLanguage
(),
setLanguage
:
(
language
)
=>
{
localStorage
.
setItem
(
LOCALE_STORAGE_KEY
,
language
);
set
({
language
});
},
toggleLanguage
:
()
=>
set
((
state
)
=>
{
const
newLanguage
=
state
.
language
===
Language
.
ZH_CN
?
Language
.
EN_US
:
Language
.
ZH_CN
;
localStorage
.
setItem
(
LOCALE_STORAGE_KEY
,
newLanguage
);
return
{
language
:
newLanguage
};
}),
}));
projects/web/src/store/mdStore.ts
0 → 100644
View file @
ece7f8d5
// mdStore.ts
import
{
create
}
from
"
zustand
"
;
import
axios
from
"
axios
"
;
import
{
updateMarkdownContent
,
UpdateMarkdownRequest
}
from
"
@/api/extract
"
;
// 确保路径正确
interface
MdContent
{
content
:
string
;
isLoading
:
boolean
;
}
type
AnchorType
=
|
"
span
"
|
"
div
"
|
"
comment
"
|
"
data-attribute
"
|
"
hr
"
|
"
mark
"
|
"
p
"
;
interface
AnchorOptions
{
type
:
AnchorType
;
prefix
?:
string
;
style
?:
string
;
className
?:
string
;
customAttributes
?:
Record
<
string
,
string
>
;
}
const
defaultAnchorOptions
:
AnchorOptions
=
{
type
:
"
span
"
,
prefix
:
"
md-anchor-
"
,
style
:
"
display:none;
"
,
className
:
""
,
customAttributes
:
{},
};
interface
MdState
{
mdContents
:
Record
<
string
,
MdContent
>
;
allMdContent
:
string
;
allMdContentWithAnchor
:
string
;
error
:
Error
|
null
;
currentRequestId
:
number
;
setMdUrlArr
:
(
urls
:
string
[])
=>
Promise
<
void
>
;
getAllMdContent
:
(
data
:
string
[])
=>
string
;
setAllMdContent
:
(
val
?:
string
)
=>
void
;
setAllMdContentWithAnchor
:
(
val
?:
string
)
=>
void
;
getContentWithAnchors
:
(
data
:
string
[],
options
?:
Partial
<
AnchorOptions
>
)
=>
string
;
jumpToAnchor
:
(
anchorId
:
string
)
=>
number
;
reset
:
()
=>
void
;
updateMdContent
:
(
fileKey
:
string
,
pageNumber
:
string
|
number
,
newContent
:
string
)
=>
Promise
<
void
>
;
}
const
MAX_CONCURRENT_REQUESTS
=
2
;
const
initialState
=
{
mdContents
:
{},
allMdContent
:
""
,
allMdContentWithAnchor
:
""
,
error
:
null
,
currentRequestId
:
0
,
};
const
useMdStore
=
create
<
MdState
>
((
set
,
get
)
=>
({
...
initialState
,
reset
:
()
=>
{
set
(
initialState
);
},
setAllMdContent
:
(
value
?:
string
)
=>
{
set
(()
=>
({
allMdContent
:
value
,
}));
},
setAllMdContentWithAnchor
:
(
value
?:
string
)
=>
{
set
(()
=>
({
allMdContentWithAnchor
:
value
,
}));
},
setMdUrlArr
:
async
(
urls
:
string
[])
=>
{
const
requestId
=
get
().
currentRequestId
+
1
;
set
((
state
)
=>
({
currentRequestId
:
requestId
,
error
:
null
}));
const
fetchContent
=
async
(
url
:
string
):
Promise
<
[
string
,
string
]
>
=>
{
try
{
const
response
=
await
axios
.
get
<
string
>
(
url
);
return
[
url
,
response
.
data
];
}
catch
(
error
)
{
if
(
get
().
currentRequestId
===
requestId
)
{
set
((
state
)
=>
({
error
:
error
as
Error
}));
}
return
[
url
,
""
];
}
};
const
fetchWithConcurrency
=
async
(
urls
:
string
[]
):
Promise
<
[
string
,
string
][]
>
=>
{
const
queue
=
[...
urls
];
const
results
:
[
string
,
string
][]
=
[];
const
inProgress
=
new
Set
<
Promise
<
[
string
,
string
]
>>
();
while
(
queue
.
length
>
0
||
inProgress
.
size
>
0
)
{
while
(
inProgress
.
size
<
MAX_CONCURRENT_REQUESTS
&&
queue
.
length
>
0
)
{
const
url
=
queue
.
shift
()
!
;
const
promise
=
fetchContent
(
url
);
inProgress
.
add
(
promise
);
promise
.
then
((
result
)
=>
{
results
.
push
(
result
);
inProgress
.
delete
(
promise
);
});
}
if
(
inProgress
.
size
>
0
)
{
await
Promise
.
race
(
inProgress
);
}
}
return
results
;
};
const
results
=
await
fetchWithConcurrency
(
urls
);
if
(
get
().
currentRequestId
===
requestId
)
{
const
newMdContents
:
Record
<
string
,
MdContent
>
=
{};
results
.
forEach
(([
url
,
content
])
=>
{
newMdContents
[
url
]
=
{
content
,
isLoading
:
false
};
});
set
((
state
)
=>
({
mdContents
:
newMdContents
,
allMdContent
:
state
.
getAllMdContent
(
results
.
map
((
i
)
=>
i
[
1
])),
allMdContentWithAnchor
:
state
.
getContentWithAnchors
(
results
.
map
((
i
)
=>
i
[
1
])
),
}));
}
},
getAllMdContent
:
(
data
)
=>
{
return
data
?.
join
(
"
\n\n
"
);
},
getContentWithAnchors
:
(
data
:
string
[],
options
?:
Partial
<
AnchorOptions
>
)
=>
{
const
opts
=
{
...
defaultAnchorOptions
,
...
options
};
const
generateAnchorTag
=
(
index
:
number
)
=>
{
const
id
=
`
${
opts
.
prefix
}${
index
}
`
;
const
attributes
=
Object
.
entries
(
opts
.
customAttributes
||
{})
.
map
(([
key
,
value
])
=>
`
${
key
}
="
${
value
}
"`
)
.
join
(
"
"
);
switch
(
opts
.
type
)
{
case
"
span
"
:
case
"
div
"
:
case
"
mark
"
:
case
"
p
"
:
return
`<
${
opts
.
type
}
id="
${
id
}
" style="
${
opts
.
style
}
" class="
${
opts
.
className
}
"
${
attributes
}
></
${
opts
.
type
}
>`
;
case
"
comment
"
:
return
`<!-- anchor:
${
id
}
-->`
;
case
"
data-attribute
"
:
return
`<span data-anchor="
${
id
}
" style="
${
opts
.
style
}
" class="
${
opts
.
className
}
"
${
attributes
}
></span>`
;
case
"
hr
"
:
return
`<hr id="
${
id
}
" style="
${
opts
.
style
}
" class="
${
opts
.
className
}
"
${
attributes
}
>`
;
default
:
return
`<span id="
${
id
}
" style="
${
opts
.
style
}
" class="
${
opts
.
className
}
"
${
attributes
}
></span>`
;
}
};
return
data
?.
map
((
content
,
index
)
=>
{
const
anchorTag
=
generateAnchorTag
(
index
);
return
`
${
anchorTag
}
\n\n
${
content
}
`
;
})
.
join
(
"
\n\n
"
);
},
jumpToAnchor
:
(
anchorId
:
string
)
=>
{
const
{
mdContents
}
=
get
();
const
contentArray
=
Object
.
values
(
mdContents
).
map
(
(
content
)
=>
content
.
content
);
let
totalLength
=
0
;
for
(
let
i
=
0
;
i
<
contentArray
.
length
;
i
++
)
{
if
(
anchorId
===
`md-anchor-
${
i
}
`
)
{
return
totalLength
;
}
totalLength
+=
contentArray
[
i
].
length
+
2
;
// +2 for "\n\n"
}
return
-
1
;
// Anchor not found
},
updateMdContent
:
async
(
fileKey
:
string
,
pageNumber
:
string
,
newContent
:
string
)
=>
{
try
{
const
params
:
UpdateMarkdownRequest
=
{
file_key
:
fileKey
,
data
:
{
[
pageNumber
]:
newContent
,
},
};
const
result
=
await
updateMarkdownContent
(
params
);
if
(
result
&&
result
.
success
)
{
// 更新本地状态
set
((
state
)
=>
{
const
updatedMdContents
=
{
...
state
.
mdContents
};
if
(
updatedMdContents
[
fileKey
])
{
updatedMdContents
[
fileKey
]
=
{
...
updatedMdContents
[
fileKey
],
content
:
newContent
,
};
}
// 重新计算 allMdContent 和 allMdContentWithAnchor
const
contentArray
=
Object
.
values
(
updatedMdContents
).
map
(
(
content
)
=>
content
.
content
);
const
newAllMdContent
=
state
.
getAllMdContent
(
contentArray
);
const
newAllMdContentWithAnchor
=
state
.
getContentWithAnchors
(
contentArray
);
return
{
mdContents
:
updatedMdContents
,
allMdContent
:
newAllMdContent
,
allMdContentWithAnchor
:
newAllMdContentWithAnchor
,
};
});
}
else
{
throw
new
Error
(
"
Failed to update Markdown content
"
);
}
}
catch
(
error
)
{
set
({
error
:
error
as
Error
});
throw
error
;
}
},
}));
export
default
useMdStore
;
projects/web/src/styles/variable.scss
0 → 100644
View file @
ece7f8d5
$page-min-witch
:
1260px
;
\ No newline at end of file
projects/web/src/types/extract-task-type.ts
0 → 100644
View file @
ece7f8d5
export
type
ExtractTaskType
=
|
"
pdf
"
|
"
formula-detect
"
|
"
formula-extract
"
|
"
table-recogn
"
;
export
const
EXTRACTOR_TYPE_LIST
=
{
table
:
"
table
"
,
formula
:
"
formula
"
,
pdf
:
"
PDF
"
,
};
export
enum
FORMULA_TYPE
{
extract
=
"
extract
"
,
detect
=
"
detect
"
,
}
export
enum
MD_PREVIEW_TYPE
{
preview
=
"
preview
"
,
code
=
"
code
"
,
}
projects/web/src/utils/download.ts
0 → 100644
View file @
ece7f8d5
export
async
function
downloadFileUseAScript
(
url
:
string
,
filename
?:
string
):
Promise
<
void
>
{
try
{
// 发起请求获取文件
const
response
=
await
fetch
(
url
);
if
(
!
response
.
ok
)
{
throw
new
Error
(
`HTTP error! status:
${
response
.
status
}
`
);
}
// 获取文件内容的 Blob
const
blob
=
await
response
.
blob
();
// 创建一个 Blob URL
const
blobUrl
=
window
.
URL
.
createObjectURL
(
blob
);
// 创建一个隐藏的<a>元素
const
link
=
document
.
createElement
(
"
a
"
);
link
.
style
.
display
=
"
none
"
;
link
.
href
=
blobUrl
;
// 设置下载的文件名
const
contentDisposition
=
response
.
headers
.
get
(
"
Content-Disposition
"
);
const
fileName
=
filename
||
(
contentDisposition
?
contentDisposition
.
split
(
"
filename=
"
)[
1
].
replace
(
/
[
'"
]
/g
,
""
)
:
url
.
split
(
"
/
"
).
pop
()
||
"
download
"
);
link
.
download
=
fileName
;
// 将链接添加到文档中并触发点击
document
.
body
.
appendChild
(
link
);
link
.
click
();
// 清理
document
.
body
.
removeChild
(
link
);
window
.
URL
.
revokeObjectURL
(
blobUrl
);
}
catch
(
error
)
{
console
.
error
(
"
Download failed:
"
,
error
);
}
}
projects/web/src/utils/locale.ts
0 → 100644
View file @
ece7f8d5
export
enum
ELocal
{
"
zh-CN
"
=
"
zh-CN
"
,
"
en-US
"
=
"
en-US
"
,
}
export
const
locale
:
{
[
key
:
string
]:
string
}
=
{
[
ELocal
[
"
zh-CN
"
]]:
"
中文
"
,
[
ELocal
[
"
en-US
"
]]:
"
En
"
,
};
export
const
localeName
:
{
[
key
:
string
]:
string
}
=
{
[
ELocal
[
"
zh-CN
"
]]:
"
nameZh
"
,
[
ELocal
[
"
en-US
"
]]:
"
name
"
,
};
export
const
getLocale
=
()
=>
{
return
localStorage
.
getItem
(
"
umi_locale
"
)
||
ELocal
[
"
zh-CN
"
];
};
Prev
1
…
20
21
22
23
24
25
26
27
28
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment