Skip to content

GitLab

Switch branch/tag
  1. 02 Sep, 2024 2 commits
  3. 30 Aug, 2024 1 commit
    • Xiaomeng Zhao's avatar
      feat(cli&analyze&pipeline): add start_page and end_page args for pagination (#507) · 0f91fcf6
      Xiaomeng Zhao authored 
      * feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.

* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.

* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.
      0f91fcf6
  5. 20 Aug, 2024 2 commits
    • Xiaomeng Zhao's avatar
      fix(pdf-extract): adjust box threshold for OCR detection (#447) · 041b9465
      Xiaomeng Zhao authored 
      Tuned the detection box threshold parameter in the OCR model initialization to improve the
accuracy of text extraction from images. The threshold was modified from 0.6 to
0.3 to filter out smaller detection boxes, which is expected to enhance the quality of the extracted
text by reducing noise and false positives in the OCR process.
      041b9465
    • Xiaomeng Zhao's avatar
      fix(self_modify): merge detection boxes for optimized text region detection (#448) · 3da5c411
      Xiaomeng Zhao authored 
      Merge adjacent and overlapping detection boxes to optimize text region detection in
the document. Post processing of text boxes is enhanced by consolidating them into
larger text lines, taking into account their vertical and horizontal alignment. This
improvement reduces fragmentation and improves the readability of detected text blocks.
      3da5c411
  7. 09 Aug, 2024 3 commits
    • myhloli's avatar
      fix(doc-analyze): adjust image scaling limit to 9000 pixels · 445a397f
      myhloli authored 
      Previously, images were not enlarged if their width or height exceeded 3000 pixels.
This threshold has been increased to 9000 pixels to better handle high-resolutionscans and improve the analysis of documents with larger dimensions.
      445a397f
    • myhloli's avatar
      fix(pdf-extract-kit): ensure table extraction success with additional ending... · 334ccac2
      myhloli authored 
      fix(pdf-extract-kit): ensure table extraction success with additional ending conditionAdd an additional condition to determine the success of table extraction by checking
if the latex_code ends with 'end{table}'. This extends the validation to cover table
environments that may not strictly end with 'end{tabular}', thus improving the robustnessof table recognition processing.
      334ccac2
    • myhloli's avatar
      refactor(pdf_extract_kit): optimize image processing and table recognition... · 29e590a7
      myhloli authored 
      refactor(pdf_extract_kit): optimize image processing and table recognition logicRefactor the image processing logic for OCR and table recognition to ensure
consistency and improve performance. Remove redundant initialization of PIL images,
unify image cropping logic, and streamline the handling of formula detection results.
Also, adjust the table recognition process to improve integration with the updated image
processing logic and enhance overall efficiency.
      29e590a7
  9. 07 Aug, 2024 2 commits
  11. 05 Aug, 2024 1 commit
  13. 04 Aug, 2024 1 commit
    • myhloli's avatar
      fix(pdf-extract): ensure table recognition config defaults to disabled · 52156eae
      myhloli authored 
      If 'table-config' is not present in the configuration file, the table recognition
feature will default to being disabled to ensure consistent behavior. This change
adds a warning log and sets a default configuration for table recognition when the
expected config is missing.
      52156eae
  15. 02 Aug, 2024 1 commit
    • Kaiwen Liu's avatar
      feat(model inference): add table recognition and conversion to LaTeX (#284) · 37925f36
      Kaiwen Liu authored 
      * # add table recognition using struct-eqtable
## Changelog
31/07/20204
- Support table recognition. Table images will be converted into html.

### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json

### caution:
it takes 200s to 500s to convert a single table image using cpu

* # add table recognition using struct-eqtable
## Changelog
31/07/20204
- Support table recognition. Table images will be converted into LaTex.

### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json

### caution:
it takes 200s to 500s to convert a single table image using cpu

* # feat(model inference): add table recognition and convertion to LaTeX

# What's Changed

### New Features

- Add table content recognition, we use weights of [StructEqTable](https://github.com/UniModal4Reasoning/StructEqTable-Deploy) to convert table image to LaTex.

### Instruction

- pip install pypandoc struct-eqtable==0.1.0
- Download [StructEqTable weights](https://huggingface.co/wanderkid/PDF-Extract-Kit/tree/main/models/TabRec

) and put it under models/ directory.
- Edit 'table-mode' value to turn on table recognition function which is turned off by default.
- If you did not download any models before, refer to [how to download models](docs/how_to_download_models_zh_cn.md)。

* add table recognition and convertion to LaTeX

* add table recognition and conversion to LaTeX

* add table recognition and conversion to LaTeX

* add table recognition and conversion to LaTeX

---------
Co-authored-by: default avatarliukaiwen <liukaiwen@pjlab.org.cn>
      37925f36
  17. 01 Aug, 2024 4 commits
  19. 31 Jul, 2024 2 commits
    • liukaiwen's avatar
      # add table recognition using struct-eqtable · d6c58ecc
      liukaiwen authored 
      ## Changelog
31/07/20204
- Support table recognition. Table images will be converted into LaTex.

### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json

### caution:
it takes 200s to 500s to convert a single table image using cpu
      d6c58ecc
    • liukaiwen's avatar
      # add table recognition using struct-eqtable · b29badc1
      liukaiwen authored 
      ## Changelog
31/07/20204
- Support table recognition. Table images will be converted into html.

### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json

### caution:
it takes 200s to 500s to convert a single table image using cpu
      b29badc1
  21. 30 Jul, 2024 1 commit
  23. 29 Jul, 2024 1 commit
  25. 28 Jul, 2024 1 commit
  27. 25 Jul, 2024 1 commit
  29. 24 Jul, 2024 3 commits
  31. 23 Jul, 2024 3 commits
  33. 22 Jul, 2024 2 commits
  35. 18 Jul, 2024 1 commit
  37. 17 Jul, 2024 2 commits
  39. 15 Jul, 2024 1 commit
  41. 14 Jul, 2024 1 commit
    • myhloli's avatar
      refactor(magic_pdf): implement model singleton pattern for custom models · 054abe33
      myhloli authored 
      Introduce a Singleton pattern to manage custom models in the magic_pdf module.
This change improves the efficiency by ensuring that a single instance of the
custom model is created and reused, thereby reducing the overhead of multiple
instantiate calls for the same model configuration.
      054abe33
  43. 12 Jul, 2024 4 commits