Showing
11 changed files
with
458 additions
and
0 deletions
INSTALL_FIX.md
0 → 100644
| 1 | +# 解决 numpy>=1.21.6 安装错误 | ||
| 2 | + | ||
| 3 | +## 问题描述 | ||
| 4 | +在安装 MinerU 项目依赖时遇到错误: | ||
| 5 | +``` | ||
| 6 | +ERROR: Could not find a version that satisfies the requirement numpy>=1.21.6 | ||
| 7 | +``` | ||
| 8 | + | ||
| 9 | +## 解决方案 | ||
| 10 | + | ||
| 11 | +### 方案 1:使用 --user 选项安装(推荐) | ||
| 12 | + | ||
| 13 | +由于 Windows 权限问题,推荐使用 `--user` 选项安装到用户目录: | ||
| 14 | + | ||
| 15 | +```bash | ||
| 16 | +python -m pip install --user --upgrade pip | ||
| 17 | +python -m pip install --user numpy>=1.21.6 | ||
| 18 | +``` | ||
| 19 | + | ||
| 20 | +### 方案 2:使用国内镜像源(推荐中国用户) | ||
| 21 | + | ||
| 22 | +如果网络较慢,可以使用国内镜像源加速安装: | ||
| 23 | + | ||
| 24 | +#### 临时使用镜像源: | ||
| 25 | +```bash | ||
| 26 | +# 使用清华大学镜像源 | ||
| 27 | +python -m pip install --user -i https://pypi.tuna.tsinghua.edu.cn/simple numpy>=1.21.6 | ||
| 28 | + | ||
| 29 | +# 或者使用阿里云镜像源 | ||
| 30 | +python -m pip install --user -i https://mirrors.aliyun.com/pypi/simple/ numpy>=1.21.6 | ||
| 31 | + | ||
| 32 | +# 或者使用中科大镜像源 | ||
| 33 | +python -m pip install --user -i https://pypi.mirrors.ustc.edu.cn/simple/ numpy>=1.21.6 | ||
| 34 | +``` | ||
| 35 | + | ||
| 36 | +#### 永久配置镜像源: | ||
| 37 | +创建或编辑 pip 配置文件: | ||
| 38 | +- Windows: `%APPDATA%\pip\pip.ini` | ||
| 39 | +- 或在用户目录下创建 `pip.ini` | ||
| 40 | + | ||
| 41 | +添加以下内容: | ||
| 42 | +```ini | ||
| 43 | +[global] | ||
| 44 | +index-url = https://pypi.tuna.tsinghua.edu.cn/simple | ||
| 45 | +[install] | ||
| 46 | +trusted-host = pypi.tuna.tsinghua.edu.cn | ||
| 47 | +``` | ||
| 48 | + | ||
| 49 | +### 方案 3:安装完整项目依赖 | ||
| 50 | + | ||
| 51 | +如果需要安装整个 MinerU 项目,可以: | ||
| 52 | + | ||
| 53 | +```bash | ||
| 54 | +# 使用 --user 选项 | ||
| 55 | +python -m pip install --user -e . | ||
| 56 | + | ||
| 57 | +# 或使用国内镜像源 | ||
| 58 | +python -m pip install --user -i https://pypi.tuna.tsinghua.edu.cn/simple -e . | ||
| 59 | +``` | ||
| 60 | + | ||
| 61 | +### 方案 4:修复 pip 警告(可选) | ||
| 62 | + | ||
| 63 | +如果看到如下警告: | ||
| 64 | +``` | ||
| 65 | +WARNING: Ignoring invalid distribution ~ip (C:\Python312\Lib\site-packages) | ||
| 66 | +``` | ||
| 67 | + | ||
| 68 | +可以尝试修复: | ||
| 69 | + | ||
| 70 | +```bash | ||
| 71 | +# 找到损坏的包目录(通常是 ~ip 或 ~ 开头) | ||
| 72 | +# 然后手动删除它们,或运行: | ||
| 73 | +python -m pip cache purge | ||
| 74 | +``` | ||
| 75 | + | ||
| 76 | +### 方案 5:以管理员权限运行(如果上述方案都不行) | ||
| 77 | + | ||
| 78 | +1. 右键点击命令提示符或 PowerShell | ||
| 79 | +2. 选择"以管理员身份运行" | ||
| 80 | +3. 然后执行安装命令 | ||
| 81 | + | ||
| 82 | +## 验证安装 | ||
| 83 | + | ||
| 84 | +安装完成后,可以验证: | ||
| 85 | + | ||
| 86 | +```bash | ||
| 87 | +python -c "import numpy; print(numpy.__version__)" | ||
| 88 | +``` | ||
| 89 | + | ||
| 90 | +应该输出类似:`2.3.4` 或更高版本 | ||
| 91 | + | ||
| 92 | +## 注意事项 | ||
| 93 | + | ||
| 94 | +1. **Python 版本要求**:MinerU 要求 Python >=3.10,<3.14,当前版本 3.12.1 符合要求 | ||
| 95 | +2. **权限问题**:Windows 上安装全局包可能需要管理员权限,推荐使用 `--user` 选项 | ||
| 96 | +3. **网络问题**:如果下载速度慢,强烈建议使用国内镜像源 | ||
| 97 | +4. **PATH 配置**:升级 pip 后,如果提示 PATH 问题,可以手动添加到环境变量,或使用 `python -m pip` 代替 `pip` | ||
| 98 | + | ||
| 99 | +## 常用镜像源 | ||
| 100 | + | ||
| 101 | +- 清华大学:https://pypi.tuna.tsinghua.edu.cn/simple | ||
| 102 | +- 阿里云:https://mirrors.aliyun.com/pypi/simple/ | ||
| 103 | +- 中科大:https://pypi.mirrors.ustc.edu.cn/simple | ||
| 104 | +- 豆瓣:https://pypi.douban.com/simple/ | ||
| 105 | + | ||
| 106 | + |
mineru.egg-info/PKG-INFO
0 → 100644
This diff is collapsed. Click to expand it.
mineru.egg-info/SOURCES.txt
0 → 100644
| 1 | +LICENSE.md | ||
| 2 | +README.md | ||
| 3 | +pyproject.toml | ||
| 4 | +mineru/__init__.py | ||
| 5 | +mineru/version.py | ||
| 6 | +mineru.egg-info/PKG-INFO | ||
| 7 | +mineru.egg-info/SOURCES.txt | ||
| 8 | +mineru.egg-info/dependency_links.txt | ||
| 9 | +mineru.egg-info/entry_points.txt | ||
| 10 | +mineru.egg-info/not-zip-safe | ||
| 11 | +mineru.egg-info/requires.txt | ||
| 12 | +mineru.egg-info/top_level.txt | ||
| 13 | +mineru/backend/__init__.py | ||
| 14 | +mineru/backend/utils.py | ||
| 15 | +mineru/backend/customs/__init__.py | ||
| 16 | +mineru/backend/customs/exporter.py | ||
| 17 | +mineru/backend/pipeline/__init__.py | ||
| 18 | +mineru/backend/pipeline/batch_analyze.py | ||
| 19 | +mineru/backend/pipeline/model_init.py | ||
| 20 | +mineru/backend/pipeline/model_json_to_middle_json.py | ||
| 21 | +mineru/backend/pipeline/model_list.py | ||
| 22 | +mineru/backend/pipeline/para_split.py | ||
| 23 | +mineru/backend/pipeline/pipeline_analyze.py | ||
| 24 | +mineru/backend/pipeline/pipeline_magic_model.py | ||
| 25 | +mineru/backend/pipeline/pipeline_middle_json_mkcontent.py | ||
| 26 | +mineru/backend/vlm/__init__.py | ||
| 27 | +mineru/backend/vlm/model_output_to_middle_json.py | ||
| 28 | +mineru/backend/vlm/utils.py | ||
| 29 | +mineru/backend/vlm/vlm_analyze.py | ||
| 30 | +mineru/backend/vlm/vlm_magic_model.py | ||
| 31 | +mineru/backend/vlm/vlm_middle_json_mkcontent.py | ||
| 32 | +mineru/cli/__init__.py | ||
| 33 | +mineru/cli/client.py | ||
| 34 | +mineru/cli/common.py | ||
| 35 | +mineru/cli/fast_api.py | ||
| 36 | +mineru/cli/gradio_app.py | ||
| 37 | +mineru/cli/models_download.py | ||
| 38 | +mineru/cli/vlm_vllm_server.py | ||
| 39 | +mineru/data/__init__.py | ||
| 40 | +mineru/data/data_reader_writer/__init__.py | ||
| 41 | +mineru/data/data_reader_writer/base.py | ||
| 42 | +mineru/data/data_reader_writer/dummy.py | ||
| 43 | +mineru/data/data_reader_writer/filebase.py | ||
| 44 | +mineru/data/data_reader_writer/multi_bucket_s3.py | ||
| 45 | +mineru/data/data_reader_writer/s3.py | ||
| 46 | +mineru/data/io/__init__.py | ||
| 47 | +mineru/data/io/base.py | ||
| 48 | +mineru/data/io/http.py | ||
| 49 | +mineru/data/io/s3.py | ||
| 50 | +mineru/data/utils/__init__.py | ||
| 51 | +mineru/data/utils/exceptions.py | ||
| 52 | +mineru/data/utils/path_utils.py | ||
| 53 | +mineru/data/utils/schemas.py | ||
| 54 | +mineru/model/__init__.py | ||
| 55 | +mineru/model/layout/__init__.py | ||
| 56 | +mineru/model/layout/doclayoutyolo.py | ||
| 57 | +mineru/model/mfd/__init__.py | ||
| 58 | +mineru/model/mfd/yolo_v8.py | ||
| 59 | +mineru/model/mfr/__init__.py | ||
| 60 | +mineru/model/mfr/utils.py | ||
| 61 | +mineru/model/mfr/pp_formulanet_plus_m/__init__.py | ||
| 62 | +mineru/model/mfr/pp_formulanet_plus_m/predict_formula.py | ||
| 63 | +mineru/model/mfr/pp_formulanet_plus_m/processors.py | ||
| 64 | +mineru/model/mfr/unimernet/Unimernet.py | ||
| 65 | +mineru/model/mfr/unimernet/__init__.py | ||
| 66 | +mineru/model/mfr/unimernet/unimernet_hf/__init__.py | ||
| 67 | +mineru/model/mfr/unimernet/unimernet_hf/modeling_unimernet.py | ||
| 68 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_mbart/__init__.py | ||
| 69 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_mbart/configuration_unimer_mbart.py | ||
| 70 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_mbart/modeling_unimer_mbart.py | ||
| 71 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_mbart/tokenization_unimer_mbart.py | ||
| 72 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_swin/__init__.py | ||
| 73 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_swin/configuration_unimer_swin.py | ||
| 74 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_swin/image_processing_unimer_swin.py | ||
| 75 | +mineru/model/mfr/unimernet/unimernet_hf/unimer_swin/modeling_unimer_swin.py | ||
| 76 | +mineru/model/ocr/__init__.py | ||
| 77 | +mineru/model/ocr/paddleocr2pytorch/__init__.py | ||
| 78 | +mineru/model/ocr/paddleocr2pytorch/pytorch_paddle.py | ||
| 79 | +mineru/model/ori_cls/__init__.py | ||
| 80 | +mineru/model/ori_cls/paddle_ori_cls.py | ||
| 81 | +mineru/model/reading_order/__init__.py | ||
| 82 | +mineru/model/reading_order/layout_reader.py | ||
| 83 | +mineru/model/reading_order/xycut.py | ||
| 84 | +mineru/model/table/__init__.py | ||
| 85 | +mineru/model/table/cls/__init__.py | ||
| 86 | +mineru/model/table/cls/paddle_table_cls.py | ||
| 87 | +mineru/model/table/rec/RapidTable.py | ||
| 88 | +mineru/model/table/rec/__init__.py | ||
| 89 | +mineru/model/table/rec/slanet_plus/__init__.py | ||
| 90 | +mineru/model/table/rec/slanet_plus/main.py | ||
| 91 | +mineru/model/table/rec/slanet_plus/matcher.py | ||
| 92 | +mineru/model/table/rec/slanet_plus/matcher_utils.py | ||
| 93 | +mineru/model/table/rec/slanet_plus/table_structure.py | ||
| 94 | +mineru/model/table/rec/slanet_plus/table_structure_utils.py | ||
| 95 | +mineru/model/table/rec/unet_table/__init__.py | ||
| 96 | +mineru/model/table/rec/unet_table/main.py | ||
| 97 | +mineru/model/table/rec/unet_table/table_recover.py | ||
| 98 | +mineru/model/table/rec/unet_table/table_structure_unet.py | ||
| 99 | +mineru/model/table/rec/unet_table/utils.py | ||
| 100 | +mineru/model/table/rec/unet_table/utils_table_line_rec.py | ||
| 101 | +mineru/model/table/rec/unet_table/utils_table_recover.py | ||
| 102 | +mineru/model/utils/__init__.py | ||
| 103 | +mineru/model/utils/pytorchocr/__init__.py | ||
| 104 | +mineru/model/utils/pytorchocr/base_ocr_v20.py | ||
| 105 | +mineru/model/utils/pytorchocr/data/__init__.py | ||
| 106 | +mineru/model/utils/pytorchocr/data/imaug/__init__.py | ||
| 107 | +mineru/model/utils/pytorchocr/data/imaug/operators.py | ||
| 108 | +mineru/model/utils/pytorchocr/modeling/__init__.py | ||
| 109 | +mineru/model/utils/pytorchocr/modeling/common.py | ||
| 110 | +mineru/model/utils/pytorchocr/modeling/architectures/__init__.py | ||
| 111 | +mineru/model/utils/pytorchocr/modeling/architectures/base_model.py | ||
| 112 | +mineru/model/utils/pytorchocr/modeling/backbones/__init__.py | ||
| 113 | +mineru/model/utils/pytorchocr/modeling/backbones/det_mobilenet_v3.py | ||
| 114 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_donut_swin.py | ||
| 115 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_hgnet.py | ||
| 116 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_lcnetv3.py | ||
| 117 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_mobilenet_v3.py | ||
| 118 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_mv1_enhance.py | ||
| 119 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_pphgnetv2.py | ||
| 120 | +mineru/model/utils/pytorchocr/modeling/backbones/rec_svtrnet.py | ||
| 121 | +mineru/model/utils/pytorchocr/modeling/heads/__init__.py | ||
| 122 | +mineru/model/utils/pytorchocr/modeling/heads/cls_head.py | ||
| 123 | +mineru/model/utils/pytorchocr/modeling/heads/det_db_head.py | ||
| 124 | +mineru/model/utils/pytorchocr/modeling/heads/rec_ctc_head.py | ||
| 125 | +mineru/model/utils/pytorchocr/modeling/heads/rec_multi_head.py | ||
| 126 | +mineru/model/utils/pytorchocr/modeling/heads/rec_ppformulanet_head.py | ||
| 127 | +mineru/model/utils/pytorchocr/modeling/heads/rec_unimernet_head.py | ||
| 128 | +mineru/model/utils/pytorchocr/modeling/necks/__init__.py | ||
| 129 | +mineru/model/utils/pytorchocr/modeling/necks/db_fpn.py | ||
| 130 | +mineru/model/utils/pytorchocr/modeling/necks/intracl.py | ||
| 131 | +mineru/model/utils/pytorchocr/modeling/necks/rnn.py | ||
| 132 | +mineru/model/utils/pytorchocr/postprocess/__init__.py | ||
| 133 | +mineru/model/utils/pytorchocr/postprocess/cls_postprocess.py | ||
| 134 | +mineru/model/utils/pytorchocr/postprocess/db_postprocess.py | ||
| 135 | +mineru/model/utils/pytorchocr/postprocess/rec_postprocess.py | ||
| 136 | +mineru/model/utils/pytorchocr/utils/__init__.py | ||
| 137 | +mineru/model/utils/pytorchocr/utils/resources/arch_config.yaml | ||
| 138 | +mineru/model/utils/pytorchocr/utils/resources/models_config.yml | ||
| 139 | +mineru/model/utils/pytorchocr/utils/resources/pp_formulanet_arch_config.yaml | ||
| 140 | +mineru/model/utils/pytorchocr/utils/resources/dict/ka_dict.txt | ||
| 141 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv4_doc_dict.txt | ||
| 142 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_arabic_dict.txt | ||
| 143 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_cyrillic_dict.txt | ||
| 144 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_devanagari_dict.txt | ||
| 145 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_dict.txt | ||
| 146 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_el_dict.txt | ||
| 147 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_en_dict.txt | ||
| 148 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_eslav_dict.txt | ||
| 149 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_korean_dict.txt | ||
| 150 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_latin_dict.txt | ||
| 151 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_ta_dict.txt | ||
| 152 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_te_dict.txt | ||
| 153 | +mineru/model/utils/pytorchocr/utils/resources/dict/ppocrv5_th_dict.txt | ||
| 154 | +mineru/model/utils/tools/__init__.py | ||
| 155 | +mineru/model/utils/tools/infer/__init__.py | ||
| 156 | +mineru/model/utils/tools/infer/predict_cls.py | ||
| 157 | +mineru/model/utils/tools/infer/predict_det.py | ||
| 158 | +mineru/model/utils/tools/infer/predict_rec.py | ||
| 159 | +mineru/model/utils/tools/infer/predict_system.py | ||
| 160 | +mineru/model/utils/tools/infer/pytorchocr_utility.py | ||
| 161 | +mineru/model/vlm_vllm_model/__init__.py | ||
| 162 | +mineru/model/vlm_vllm_model/server.py | ||
| 163 | +mineru/resources/header.html | ||
| 164 | +mineru/resources/fasttext-langdetect/lid.176.ftz | ||
| 165 | +mineru/utils/__init__.py | ||
| 166 | +mineru/utils/block_pre_proc.py | ||
| 167 | +mineru/utils/block_sort.py | ||
| 168 | +mineru/utils/boxbase.py | ||
| 169 | +mineru/utils/cli_parser.py | ||
| 170 | +mineru/utils/config_reader.py | ||
| 171 | +mineru/utils/cut_image.py | ||
| 172 | +mineru/utils/draw_bbox.py | ||
| 173 | +mineru/utils/enum_class.py | ||
| 174 | +mineru/utils/format_utils.py | ||
| 175 | +mineru/utils/guess_suffix_or_lang.py | ||
| 176 | +mineru/utils/hash_utils.py | ||
| 177 | +mineru/utils/language.py | ||
| 178 | +mineru/utils/llm_aided.py | ||
| 179 | +mineru/utils/magic_model_utils.py | ||
| 180 | +mineru/utils/model_utils.py | ||
| 181 | +mineru/utils/models_download_utils.py | ||
| 182 | +mineru/utils/ocr_utils.py | ||
| 183 | +mineru/utils/pdf_classify.py | ||
| 184 | +mineru/utils/pdf_image_tools.py | ||
| 185 | +mineru/utils/pdf_reader.py | ||
| 186 | +mineru/utils/pdf_text_tool.py | ||
| 187 | +mineru/utils/run_async.py | ||
| 188 | +mineru/utils/span_block_fix.py | ||
| 189 | +mineru/utils/span_pre_proc.py | ||
| 190 | +mineru/utils/table_merge.py | ||
| ... | \ No newline at end of file | ... | \ No newline at end of file |
mineru.egg-info/dependency_links.txt
0 → 100644
| 1 | + |
mineru.egg-info/entry_points.txt
0 → 100644
mineru.egg-info/not-zip-safe
0 → 100644
| 1 | + |
mineru.egg-info/requires.txt
0 → 100644
| 1 | +boto3>=1.28.43 | ||
| 2 | +click>=8.1.7 | ||
| 3 | +loguru>=0.7.2 | ||
| 4 | +numpy>=1.21.6 | ||
| 5 | +pdfminer.six==20250506 | ||
| 6 | +tqdm>=4.67.1 | ||
| 7 | +requests | ||
| 8 | +httpx | ||
| 9 | +pillow>=11.0.0 | ||
| 10 | +pypdfium2>=4.30.0 | ||
| 11 | +pypdf>=5.6.0 | ||
| 12 | +reportlab | ||
| 13 | +pdftext>=0.6.2 | ||
| 14 | +modelscope>=1.26.0 | ||
| 15 | +huggingface-hub>=0.32.4 | ||
| 16 | +json-repair>=0.46.2 | ||
| 17 | +opencv-python>=4.11.0.86 | ||
| 18 | +fast-langdetect<0.3.0,>=0.2.3 | ||
| 19 | +scikit-image<1.0.0,>=0.25.0 | ||
| 20 | +openai<3,>=1.70.0 | ||
| 21 | +beautifulsoup4<5,>=4.13.5 | ||
| 22 | +magika<0.7.0,>=0.6.2 | ||
| 23 | +mineru-vl-utils<1,>=0.1.14 | ||
| 24 | + | ||
| 25 | +[all] | ||
| 26 | +mineru[core] | ||
| 27 | +mineru[vllm] | ||
| 28 | + | ||
| 29 | +[api] | ||
| 30 | +fastapi | ||
| 31 | +python-multipart | ||
| 32 | +uvicorn | ||
| 33 | + | ||
| 34 | +[core] | ||
| 35 | +mineru[vlm] | ||
| 36 | +mineru[pipeline] | ||
| 37 | +mineru[api] | ||
| 38 | +mineru[gradio] | ||
| 39 | + | ||
| 40 | +[gradio] | ||
| 41 | +gradio<6,>=5.34 | ||
| 42 | +gradio-pdf>=0.0.22 | ||
| 43 | + | ||
| 44 | +[pipeline] | ||
| 45 | +matplotlib<4,>=3.10 | ||
| 46 | +ultralytics<9,>=8.3.48 | ||
| 47 | +doclayout_yolo==0.0.4 | ||
| 48 | +dill<1,>=0.3.8 | ||
| 49 | +PyYAML<7,>=6.0.2 | ||
| 50 | +ftfy<7,>=6.3.1 | ||
| 51 | +shapely<3,>=2.0.7 | ||
| 52 | +pyclipper<2,>=1.3.0 | ||
| 53 | +omegaconf<3,>=2.3.0 | ||
| 54 | +torch<3,>=2.6.0 | ||
| 55 | +torchvision | ||
| 56 | +transformers!=4.51.0,<5.0.0,>=4.49.0 | ||
| 57 | +onnxruntime>1.17.0 | ||
| 58 | + | ||
| 59 | +[test] | ||
| 60 | +mineru[core] | ||
| 61 | +pytest | ||
| 62 | +pytest-cov | ||
| 63 | +coverage | ||
| 64 | +fuzzywuzzy | ||
| 65 | + | ||
| 66 | +[vllm] | ||
| 67 | +vllm<0.12,>=0.10.1.1 | ||
| 68 | + | ||
| 69 | +[vlm] | ||
| 70 | +torch<3,>=2.6.0 | ||
| 71 | +transformers<5.0.0,>=4.51.1 | ||
| 72 | +accelerate>=1.5.1 |
mineru.egg-info/top_level.txt
0 → 100644
| 1 | +mineru |
| ... | @@ -13,6 +13,7 @@ from ...utils.model_utils import crop_img, get_res_list_from_layout_res, clean_v | ... | @@ -13,6 +13,7 @@ from ...utils.model_utils import crop_img, get_res_list_from_layout_res, clean_v |
| 13 | from ...utils.ocr_utils import merge_det_boxes, update_det_boxes, sorted_boxes | 13 | from ...utils.ocr_utils import merge_det_boxes, update_det_boxes, sorted_boxes |
| 14 | from ...utils.ocr_utils import get_adjusted_mfdetrec_res, get_ocr_result_list, OcrConfidence, get_rotate_crop_image | 14 | from ...utils.ocr_utils import get_adjusted_mfdetrec_res, get_ocr_result_list, OcrConfidence, get_rotate_crop_image |
| 15 | from ...utils.pdf_image_tools import get_crop_np_img | 15 | from ...utils.pdf_image_tools import get_crop_np_img |
| 16 | +from ...utils.table_cell_bbox import compute_table_cells | ||
| 16 | 17 | ||
| 17 | YOLO_LAYOUT_BASE_BATCH_SIZE = 1 | 18 | YOLO_LAYOUT_BASE_BATCH_SIZE = 1 |
| 18 | MFD_BASE_BATCH_SIZE = 1 | 19 | MFD_BASE_BATCH_SIZE = 1 |
| ... | @@ -106,6 +107,7 @@ class BatchAnalyze: | ... | @@ -106,6 +107,7 @@ class BatchAnalyze: |
| 106 | 'lang':_lang, | 107 | 'lang':_lang, |
| 107 | 'table_img':wireless_table_img, | 108 | 'table_img':wireless_table_img, |
| 108 | 'wired_table_img':wired_table_img, | 109 | 'wired_table_img':wired_table_img, |
| 110 | + 'page_index': index, # 记录页码索引,用于后续关联 | ||
| 109 | }) | 111 | }) |
| 110 | 112 | ||
| 111 | # 表格识别 table recognition | 113 | # 表格识别 table recognition |
| ... | @@ -234,6 +236,81 @@ class BatchAnalyze: | ... | @@ -234,6 +236,81 @@ class BatchAnalyze: |
| 234 | start_index = html_code.find("<table>") | 236 | start_index = html_code.find("<table>") |
| 235 | end_index = html_code.rfind("</table>") + len("</table>") | 237 | end_index = html_code.rfind("</table>") + len("</table>") |
| 236 | table_res_dict["table_res"]["html"] = html_code[start_index:end_index] | 238 | table_res_dict["table_res"]["html"] = html_code[start_index:end_index] |
| 239 | + | ||
| 240 | + # 计算表格单元格的bbox和score | ||
| 241 | + try: | ||
| 242 | + html_final = table_res_dict["table_res"].get("html", "") | ||
| 243 | + if html_final: | ||
| 244 | + # 获取表格bbox(从poly转换为[x1, y1, x2, y2]) | ||
| 245 | + table_res = table_res_dict.get("table_res", {}) | ||
| 246 | + poly = table_res.get("poly", []) | ||
| 247 | + x_coords = None | ||
| 248 | + y_coords = None | ||
| 249 | + | ||
| 250 | + # 安全地检查poly长度和处理numpy数组 | ||
| 251 | + try: | ||
| 252 | + poly_len = len(poly) if hasattr(poly, '__len__') else 0 | ||
| 253 | + if poly_len >= 8: | ||
| 254 | + # 转换为列表并提取坐标,确保处理numpy数组 | ||
| 255 | + poly_list = list(poly) if not isinstance(poly, list) else poly | ||
| 256 | + x_coords = [float(poly_list[0]), float(poly_list[2]), float(poly_list[4]), float(poly_list[6])] | ||
| 257 | + y_coords = [float(poly_list[1]), float(poly_list[3]), float(poly_list[5]), float(poly_list[7])] | ||
| 258 | + table_bbox = [min(x_coords), min(y_coords), max(x_coords), max(y_coords)] | ||
| 259 | + else: | ||
| 260 | + # 如果没有poly,尝试从bbox字段获取 | ||
| 261 | + bbox_list = table_res.get("bbox", []) | ||
| 262 | + if hasattr(bbox_list, '__len__') and len(bbox_list) >= 4: | ||
| 263 | + table_bbox = [float(bbox_list[0]), float(bbox_list[1]), float(bbox_list[2]), float(bbox_list[3])] | ||
| 264 | + else: | ||
| 265 | + table_bbox = [0, 0, 0, 0] | ||
| 266 | + except (ValueError, TypeError, IndexError) as e: | ||
| 267 | + logger.warning(f"Error processing table bbox from poly: {e}") | ||
| 268 | + # 回退到bbox字段 | ||
| 269 | + bbox_list = table_res.get("bbox", []) | ||
| 270 | + if hasattr(bbox_list, '__len__') and len(bbox_list) >= 4: | ||
| 271 | + table_bbox = [float(bbox_list[0]), float(bbox_list[1]), float(bbox_list[2]), float(bbox_list[3])] | ||
| 272 | + else: | ||
| 273 | + table_bbox = [0, 0, 0, 0] | ||
| 274 | + | ||
| 275 | + # 获取score | ||
| 276 | + # 如果score未定义或为0,使用合理的默认值 | ||
| 277 | + raw_score = table_res.get("score", 0.8) | ||
| 278 | + span_score = float(raw_score) | ||
| 279 | + | ||
| 280 | + # 如果score为0或未定义,使用默认值 | ||
| 281 | + if span_score <= 0: | ||
| 282 | + logger.warning(f"Invalid table score: {span_score}, using default 0.8") | ||
| 283 | + span_score = 0.8 | ||
| 284 | + | ||
| 285 | + # 获取OCR结果 | ||
| 286 | + ocr_result = table_res_dict.get("ocr_result", None) | ||
| 287 | + | ||
| 288 | + # 计算裁剪偏移量(OCR结果坐标是相对于裁剪后的表格图像的) | ||
| 289 | + crop_info = None | ||
| 290 | + if x_coords is not None and y_coords is not None: | ||
| 291 | + try: | ||
| 292 | + crop_info = { | ||
| 293 | + 'crop_xmin': min(x_coords), | ||
| 294 | + 'crop_ymin': min(y_coords), | ||
| 295 | + } | ||
| 296 | + except Exception as e: | ||
| 297 | + logger.warning(f"Error creating crop_info: {e}") | ||
| 298 | + crop_info = None | ||
| 299 | + | ||
| 300 | + # 计算cells | ||
| 301 | + cells = compute_table_cells( | ||
| 302 | + html_final, | ||
| 303 | + table_bbox, | ||
| 304 | + span_score, | ||
| 305 | + ocr_result, | ||
| 306 | + crop_info=crop_info | ||
| 307 | + ) | ||
| 308 | + | ||
| 309 | + # 将cells添加到table_res中(table_res是layout_res中table项的引用,会自动同步) | ||
| 310 | + table_res_dict["table_res"]["cells"] = cells | ||
| 311 | + except Exception as e: | ||
| 312 | + logger.warning(f"Failed to compute table cells: {e}") | ||
| 313 | + table_res_dict["table_res"]["cells"] = [] | ||
| 237 | 314 | ||
| 238 | # OCR det | 315 | # OCR det |
| 239 | if self.enable_ocr_det_batch: | 316 | if self.enable_ocr_det_batch: | ... | ... |
| ... | @@ -338,6 +338,10 @@ class MagicModel: | ... | @@ -338,6 +338,10 @@ class MagicModel: |
| 338 | span['latex'] = latex | 338 | span['latex'] = latex |
| 339 | elif html: | 339 | elif html: |
| 340 | span['html'] = html | 340 | span['html'] = html |
| 341 | + # 获取单元格级信息(bbox和score) | ||
| 342 | + cells = layout_det.get('cells', None) | ||
| 343 | + if cells: | ||
| 344 | + span['cells'] = cells | ||
| 341 | span['type'] = ContentType.TABLE | 345 | span['type'] = ContentType.TABLE |
| 342 | elif category_id == CategoryId.InlineEquation: | 346 | elif category_id == CategoryId.InlineEquation: |
| 343 | span['content'] = layout_det['latex'] | 347 | span['content'] = layout_det['latex'] | ... | ... |
mineru/utils/table_cell_bbox.py
0 → 100644
This diff is collapsed. Click to expand it.
-
Please register or login to post a comment