Skip to content

GitLab

Commit f55a786e authored by luopl's avatar luopl
Browse files

Initial commit

parents
Pipeline #1081 canceled with stages
Hide whitespace changes
Inline Side-by-side
Showing with 842 additions and 0 deletions
.idea/.gitignore 0 → 100644
View file @ f55a786e
# Default ignored files
/shelf/
/workspace.xml
.idea/.name 0 → 100644
View file @ f55a786e
README.md
\ No newline at end of file
.idea/inspectionProfiles/profiles_settings.xml 0 → 100644
View file @ f55a786e
<component name="InspectionProjectProfileManager">
<settings>
<option name="USE_PROJECT_PROFILE" value="false" />
<version value="1.0" />
</settings>
</component>
\ No newline at end of file
.idea/misc.xml 0 → 100644
View file @ f55a786e
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="Black">
<option name="sdkName" value="Python 3.8 (sed_pytorch)" />
</component>
<component name="ProjectRootManager" version="2" project-jdk-name="Python 3.8 (sed_pytorch)" project-jdk-type="Python SDK" />
</project>
\ No newline at end of file
.idea/modules.xml 0 → 100644
View file @ f55a786e
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/sed_pytorch.iml" filepath="$PROJECT_DIR$/.idea/sed_pytorch.iml" />
</modules>
</component>
</project>
\ No newline at end of file
.idea/sed_pytorch.iml 0 → 100644
View file @ f55a786e
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$">
<excludeFolder url="file://$MODULE_DIR$/venv" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
<component name="PyDocumentationSettings">
<option name="format" value="PLAIN" />
<option name="myDocStringFormat" value="Plain" />
</component>
<component name="TestRunnerService">
<option name="PROJECT_TEST_RUNNER" value="py.test" />
</component>
</module>
\ No newline at end of file
Dockerfile 0 → 100644
View file @ f55a786e
FROM image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk23.10-py38
\ No newline at end of file
INSTALL.md 0 → 100644
View file @ f55a786e
## Installation
### Requirements
- Linux or macOS with Python ≥ 3.8
- PyTorch ≥ 1.13 is recommended and [torchvision](https://github.com/pytorch/vision/) that matches the PyTorch installation.
Install them together at [pytorch.org](https://pytorch.org) to make sure of this. Note, please check
PyTorch version matches that is required by Detectron2.
- Detectron2: follow [Detectron2 installation instructions](https://detectron2.readthedocs.io/tutorials/install.html).
- OpenCV is optional but needed by demo and visualization
- `pip install -r requirements.txt`
An example of installation is shown below:
```
git clone https://github.com/xb534/SED.git
cd SED
conda create -n sed python=3.8
conda activate sed
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia
pip install -r requirements.txt
cd open_clip/
make install
```
LICENSE 0 → 100644
View file @ f55a786e
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright (c) 2023 Megvii Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
\ No newline at end of file
README.md 0 → 100644
View file @ f55a786e
# SED
## 论文
`SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation`
- https://arxiv.org/abs/2311.15537
## 模型结构
SED的总体架构，首先使用分层编码器（可学习的）和文本编码器（冻结的）来生成像素级的图像-文本成本图。然后，我们引入了一种渐进融合解码器，将不同的层次编码器特征映射和成本图相结合。渐进融合解码器堆栈具有聚合模块（FAM）和跳跃层融合模块（SFM）。此外，我们在解码器中设计了一个类别早期屏蔽（CER），以在不牺牲性能的情况下加速推理速度。
<div align=center>
<img src="./docs/main_arch.png"/>
</div>
## 算法原理
渐进融合解码器的结构如下图所示。渐进融合解码器（GFD）首先在空间和类层次上进行特征聚合(a)，然后采用跳层融合(b)将之前解码层和层次编码器的特征映射结合起来。
<div align=center>
<img src="./docs/Structure of gradual fusion decoder.png"/>
</div>
类别早期屏蔽（CER）结构如下图。在训练(a)期间，在每个解码器层之后附加一个辅助卷积来预测由标签监督的分割图。在推理(b)过程中，采用top-k策略来预测现有的类别，并拒绝下一个解码器层中不存在的类别。
<div align=center>
<img src="./docs/Structure of category early rejection.png"/>
</div>
## 环境配置
### Docker（方法一）
此处提供[光源](https://www.sourcefind.cn/#/service-details)拉取docker镜像的地址与使用步骤，以及[光合](https://developer.hpccube.com/tool/)开发者社区深度学习库下载地址
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-centos7.6-dtk23.10-py38
docker run -it --shm-size=128G -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name sed <your IMAGE ID> bash # <your IMAGE ID>为以上拉取的docker的镜像ID替换，本镜像为：ffa1f63239fc
cd /path/your_code_data/sed_pytorch
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
#从光合社区下载detectron2-0.6
pip install detectron2-0.6-cp38-cp38-linux_x86_64.whl
cd open_clip/
make install
```
### Dockerfile（方法二）
此处提供dockerfile的使用方法
```
docker build --no-cache -t sed:latest .
docker run -it --shm-size=128G -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name sed_pytorch sed bash
cd /path/your_code_data/sed_pytorch
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
#从光合社区下载detectron2-0.6
pip install detectron2-0.6-cp38-cp38-linux_x86_64.whl
cd open_clip/
make install
```
### Anaconda（方法三）
此处提供本地配置、编译的详细步骤，例如：
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
```
#DTK驱动：dtk23.10
# python：python3.8
# torch: 2.1.0
# torchvision: 0.16.0
# detectron2：0.6
conda create -n sed python=3.8
conda activate sed
pip install torch-2.1.0a0+git793d2b5.abi0.dtk2310-cp38-cp38-manylinux2014_x86_64.whl
pip install torchvision-0.16.0+git267eff6.abi0.dtk2310.torch2.1.0-cp38-cp38-manylinux2014_x86_64.whl
pip install detectron2-0.6-cp38-cp38-linux_x86_64.whl
```
`Tips：以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应`
其它依赖环境安装如下：
```
cd /path/your_code_data/sed
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
cd open_clip/
make install
```
## 数据集
SED 内置了对一些数据集的支持。假定数据集存在于环境变量 DETECTRON2_DATASETS 指定的目录中。在此目录下，Detectron2 将查找具有下述结构的数据集。
可以通过导出 DETECTRON2_DATASETS=/path/to/datasets 设置内置数据集的位置。如果未设置，默认值是相对于当前工作目录的 ./datasets，默认数据结构如下:
```
── dataset
│ ├── coco-stuff
│ ├── ADEChallengeData2016
│ ├── ADE20K_2021_17_01
│ ├── VOCdevkit
│ │ ├── VOC2010
│ │ └── VOC2012
```
数据准备详情查看dataset/readme.md。
## 训练
首先下载模型文件CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82K和laionCLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup，放于weights目录下:
[CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82K](https://huggingface.co/laion/CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82K)
[laionCLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup](https://huggingface.co/laion/CLIP-convnext_large_d_320.laion2B-s29B-b131K-ft-soup)
注：使用本地模型文件时修改/open_clip/src/open_clip/pretrained.py文件中download_pretrained_from_hf函数下的cached_file地址为对应的权重本地文件夹地址,例如：
```
#ConvNeXt-B
cached_file = './weights/CLIP-convnext_base_w_320-laion_aesthetic-s13B-b82K/open_clip_pytorch_model.bin'
```
### 单机多卡
```
sh run.sh [CONFIG] [NUM_GPUS] [OUTPUT_DIR] [OPTS]
# For ConvNeXt-B variant
sh run.sh configs/convnextB_768.yaml 4 output/
# For ConvNeXt-L variant
sh run.sh configs/convnextL_768.yaml 4 output/
```
## 推理
模型权重文件下载表格如下，放到weights文件夹下：
| Name | CLIP | Download |
|:------------:|:----------:|--------------------------------------------------------------------------------|
| SED (B) | ConvNeXt-B | [ckpt](https://drive.google.com/file/d/1qx6zGZgSPkF6TObregRz4uzQqSRHrgUw/view) |
| SED-fast (B) | ConvNeXt-B | [ckpt](https://drive.google.com/file/d/1qx6zGZgSPkF6TObregRz4uzQqSRHrgUw/view) |
| SED (L) | ConvNeXt-L | [ckpt](https://drive.google.com/file/d/1zAXE0QXy47n0cVn7j_2cSR85eqxdDGg8/view) |
| SED-fast (L) | ConvNeXt-L | [ckpt](https://drive.google.com/file/d/1zAXE0QXy47n0cVn7j_2cSR85eqxdDGg8/view) |
注意：模型配置文件、clip文件与权重文件应一一对应
### 单卡推理
```
#sh eval.sh [CONFIG] [NUM_GPUS] [OUTPUT_DIR] [OPTS]
#sh eval.sh configs/convnextB_768.yaml 1 output/ MODEL.WEIGHTS path/to/weights.pth
sh eval.sh configs/convnextB_768.yaml 1 output/ MODEL.WEIGHTS ./weights/sed_model_base.pth
# Fast version.
#sh eval.sh configs/convnextB_768.yaml 1 output/ MODEL.WEIGHTS path/to/weights.pth TEST.FAST_INFERENCE True TEST.TOPK 8
sh eval.sh configs/convnextB_768.yaml 1 output/ MODEL.WEIGHTS ./weights/sed_model_base.pth TEST.FAST_INFERENCE True TEST.TOPK 8
```
Demo_for_vis：
```
# python demo/demo_for_vis.py --input image_path --output result_path --opts MODEL.WEIGHTS path/to/your/pretrained_model.pth
# image_path: A list of space separated input images; or a single glob pattern such as 'directory/*.jpg'
python demo/demo_for_vis.py --input ./docs/ADE_val_00000001_Original.jpg --output demo_results --opts MODEL.WEIGHTS ./weights/sed_model_base.pth
```
### 多卡推理
```
#sh run.sh [CONFIG] [NUM_GPUS] [OUTPUT_DIR] [OPTS]
$sh eval.sh configs/convnextB_768.yaml 4 output/ MODEL.WEIGHTS path/to/weights.pth
sh eval.sh configs/convnextB_768.yaml 4 output/ MODEL.WEIGHTS ./weights/sed_model_base.pth
# Fast version.
#sh eval.sh configs/convnextB_768.yaml 4 output/ MODEL.WEIGHTS path/to/weights.pth TEST.FAST_INFERENCE True TEST.TOPK 8
sh eval.sh configs/convnextB_768.yaml 4 output/ MODEL.WEIGHTS ./weights/sed_model_base.pth TEST.FAST_INFERENCE True TEST.TOPK 8
```
## result
原图：
<div align=center>
<img src="./docs/ADE_val_00000001_Original.jpg"/>
</div>
Demo_for_vis可视化结果如下：
<div align=center>
<img src="./docs/ADE_val_00000001.jpg"/>
</div>
### 精度
使用四张DCU-K100卡推理
| Name | CLIP | A-847 | PC-459 | A-150 | PC-59 | PAS-20 |
|:------------:|:-------------------------:|------|------|-------|-------|--------|
| SED (B) | ConvNeXt-B | 11.2 | 18.6 | 31.7 | 57.7 | 94.4 |
| SED-fast (B) | ConvNeXt-B | 11.3 | 18.6 | 31.6 | 57.3 | 94.4 |
| SED (L) | ConvNeXt-L | 13.7 | 22.1 | 35.3 | 60.9 | 96.1 |
| SED-fast (L) | ConvNeXt-L | 13.9 | 22.6 | 35.2 | 60.6 | 96.1 |
## 应用场景
### 算法类别
`图像分割`
### 热点应用行业
`科研,制造,医疗,家居,教育`
## 源码仓库及问题反馈
- https://developer.hpccube.com/codes/modelzoo/sed_pytorch
## 参考资料
- https://github.com/xb534/SED
configs/config.yaml 0 → 100644
View file @ f55a786e
MODEL:
META_ARCHITECTURE: "SED"
BACKBONE:
FREEZE_AT: 0
NAME: "D2SwinTransformer"
SWIN:
EMBED_DIM: 192
DEPTHS: [2, 2, 18, 2]
NUM_HEADS: [4, 8, 16, 32]
WINDOW_SIZE: 12
APE: False
DROP_PATH_RATE: 0.3
PATCH_NORM: True
PRETRAIN_IMG_SIZE: 384
OUT_FEATURES: ["res2", "res3", "res4"]
WEIGHTS: "swin_large_patch4_window12_384_22k.pkl"
PIXEL_MEAN: [123.675, 116.280, 103.530]
PIXEL_STD: [58.395, 57.120, 57.375]
SEM_SEG_HEAD:
NAME: "OpenVocabHead"
IN_FEATURES: ["res2", "res3", "res4"]
IGNORE_VALUE: 255
NUM_CLASSES: 171
TRAIN_CLASS_JSON: "datasets/coco.json"
TEST_CLASS_JSON: "datasets/coco.json"
CLIP_PRETRAINED: "ViT-L/14@336px"
PROMPT_DEPTH: 0
PROMPT_LENGTH: 0
TEXT_GUIDANCE_DIM: 768
TEXT_GUIDANCE_PROJ_DIM: 128
APPEARANCE_GUIDANCE_DIM: 768
APPEARANCE_GUIDANCE_PROJ_DIM: 128
DECODER_DIMS: [64, 32]
DECODER_GUIDANCE_DIMS: [256, 128]
DECODER_GUIDANCE_PROJ_DIMS: [32, 16]
NUM_LAYERS: 4
NUM_HEADS: 4
HIDDEN_DIMS: 128
POOLING_SIZES: [6, 6]
FEATURE_RESOLUTION: [24, 24]
WINDOW_SIZES: 12
ATTENTION_TYPE: "linear"
CLIP_FINETUNE: "attention"
PROMPT_ENSEMBLE_TYPE: "imagenet"
DATASETS:
TRAIN: ("coco_2017_train_stuff_all_sem_seg",)
TEST: ("coco_2017_test_stuff_all_sem_seg",)
SOLVER:
IMS_PER_BATCH: 4
BASE_LR: 0.0002
MAX_ITER: 80000
WARMUP_FACTOR: 1.0
WARMUP_ITERS: 0
WEIGHT_DECAY: 0.0001
OPTIMIZER: "ADAMW"
LR_SCHEDULER_NAME: "WarmupCosineLR"
BACKBONE_MULTIPLIER: 0.01
CLIP_MULTIPLIER: 0.01
CLIP_GRADIENTS:
ENABLED: True
CLIP_TYPE: "full_model"
CLIP_VALUE: 0.01
NORM_TYPE: 2.0
INPUT:
MIN_SIZE_TRAIN: (384, )
MIN_SIZE_TRAIN_SAMPLING: "choice"
MIN_SIZE_TEST: 640
MAX_SIZE_TEST: 2560
CROP:
ENABLED: True
TYPE: "absolute"
SIZE: (384, 384)
SINGLE_CATEGORY_MAX_AREA: 1.0
COLOR_AUG_SSD: True
SIZE_DIVISIBILITY: 384
FORMAT: "RGB"
DATASET_MAPPER_NAME: "mask_former_semantic"
TEST:
EVAL_PERIOD: 5000
SLIDING_WINDOW: False
DATALOADER:
FILTER_EMPTY_ANNOTATIONS: True
NUM_WORKERS: 8
VERSION: 2
CUDNN_BENCHMARK: True
\ No newline at end of file
configs/convnextB_768.yaml 0 → 100644
View file @ f55a786e
_BASE_: config.yaml
MODEL:
META_ARCHITECTURE: "SED"
BACKBONE:
NAME: "CLIP"
WEIGHTS: ""
ENC:
CLIP_MODEL_NAME: "convnext_base_w_320"
CLIP_PRETRAINED_WEIGHTS: "laion_aesthetic_s13b_b82k"
EMBED_DIM: 768
PIXEL_MEAN: [123.675, 116.280, 103.530]
PIXEL_STD: [58.395, 57.120, 57.375]
SEM_SEG_HEAD:
NAME: "SEDHead"
IN_FEATURES: ["res2", "res3", "res4", "res5"]
IGNORE_VALUE: 255
NUM_CLASSES: 171
TRAIN_CLASS_JSON: "datasets/coco.json"
TEST_CLASS_JSON: "datasets/coco.json"
CLIP_PRETRAINED: "Convnext-B"
PROMPT_DEPTH: 0
PROMPT_LENGTH: 0
TEXT_GUIDANCE_DIM: 0
TEXT_GUIDANCE_PROJ_DIM: 0
APPEARANCE_GUIDANCE_DIM: 1024
APPEARANCE_GUIDANCE_PROJ_DIM: 128
DECODER_DIMS: [64, 32, 16]
DECODER_GUIDANCE_DIMS: [512, 256, 128]
DECODER_GUIDANCE_PROJ_DIMS: [32, 16, 8]
NUM_LAYERS: 2
NUM_HEADS: 4
HIDDEN_DIMS: 128
POOLING_SIZES: [2, 2]
FEATURE_RESOLUTION: [24, 24]
WINDOW_SIZES: 12
ATTENTION_TYPE: "linear"
CNEXT_TYPE: "V1"
KERNEL_SIZE: [9, 9, 9]
CLIP_FINETUNE: "full"
PROMPT_ENSEMBLE_TYPE: "imagenet"
INPUT:
MIN_SIZE_TRAIN: (768, )
MAX_SIZE_TRAIN: 2666
MIN_SIZE_TRAIN_SAMPLING: "choice"
MIN_SIZE_TEST: 640
CROP:
ENABLED: True
TYPE: "absolute"
SIZE: (768, 768)
SIZE_DIVISIBILITY: 768
FORMAT: "RGB"
DATASET_MAPPER_NAME: "mask_former_semantic"
SOLVER:
IMS_PER_BATCH: 4
LR_SCHEDULER_NAME: WarmupCosineLR
BASE_LR: 0.0002
MAX_ITER: 80000
BACKBONE_MULTIPLIER: 0.0
CLIP_MULTIPLIER: 0.01
TEST:
EVAL_PERIOD: 10000
FAST_INFERENCE: False
TOPK: 1
\ No newline at end of file
configs/convnextL_768.yaml 0 → 100644
View file @ f55a786e
_BASE_: config.yaml
MODEL:
META_ARCHITECTURE: "SED"
BACKBONE:
NAME: "CLIP"
WEIGHTS: ""
ENC:
CLIP_MODEL_NAME: "convnext_large_d_320"
CLIP_PRETRAINED_WEIGHTS: "laion2b_s29b_b131k_ft_soup"
EMBED_DIM: 768
PIXEL_MEAN: [123.675, 116.280, 103.530]
PIXEL_STD: [58.395, 57.120, 57.375]
SEM_SEG_HEAD:
NAME: "SEDHead"
IN_FEATURES: ["res2", "res3", "res4", "res5"]
IGNORE_VALUE: 255
NUM_CLASSES: 171
TRAIN_CLASS_JSON: "datasets/coco.json"
TEST_CLASS_JSON: "datasets/coco.json"
CLIP_PRETRAINED: "Convnext-L"
PROMPT_DEPTH: 0
PROMPT_LENGTH: 0
TEXT_GUIDANCE_DIM: 0
TEXT_GUIDANCE_PROJ_DIM: 0
APPEARANCE_GUIDANCE_DIM: 1536
APPEARANCE_GUIDANCE_PROJ_DIM: 128
DECODER_DIMS: [64, 32, 16]
DECODER_GUIDANCE_DIMS: [768, 384, 192]
DECODER_GUIDANCE_PROJ_DIMS: [32, 16, 8]
NUM_LAYERS: 2
NUM_HEADS: 4
HIDDEN_DIMS: 128
POOLING_SIZES: [2, 2]
FEATURE_RESOLUTION: [24, 24]
WINDOW_SIZES: 12
ATTENTION_TYPE: "linear"
CNEXT_TYPE: "V1"
KERNEL_SIZE: [9, 9, 9]
CLIP_FINETUNE: "full"
PROMPT_ENSEMBLE_TYPE: "imagenet"
INPUT:
MIN_SIZE_TRAIN: (768, )
MAX_SIZE_TRAIN: 2666
MIN_SIZE_TRAIN_SAMPLING: "choice"
MIN_SIZE_TEST: 640
CROP:
ENABLED: True
TYPE: "absolute"
SIZE: (768, 768)
SIZE_DIVISIBILITY: 768
FORMAT: "RGB"
DATASET_MAPPER_NAME: "mask_former_semantic"
SOLVER:
IMS_PER_BATCH: 4
LR_SCHEDULER_NAME: WarmupCosineLR
BASE_LR: 0.0002
MAX_ITER: 80000
BACKBONE_MULTIPLIER: 0.0
CLIP_MULTIPLIER: 0.01
TEST:
EVAL_PERIOD: 10000
FAST_INFERENCE: False
TOPK: 1
\ No newline at end of file
datasets/README.md 0 → 100644
View file @ f55a786e
# Prepare Datasets for SED
A dataset can be used by accessing [DatasetCatalog](https://detectron2.readthedocs.io/modules/data.html#detectron2.data.DatasetCatalog)
for its data, or [MetadataCatalog](https://detectron2.readthedocs.io/modules/data.html#detectron2.data.MetadataCatalog) for its metadata (class names, etc).
This document explains how to setup the builtin datasets so they can be used by the above APIs.
[Use Custom Datasets](https://detectron2.readthedocs.io/tutorials/datasets.html) gives a deeper dive on how to use `DatasetCatalog` and `MetadataCatalog`,
and how to add new datasets to them.
SED has builtin support for a few datasets.
The datasets are assumed to exist in a directory specified by the environment variable
`DETECTRON2_DATASETS`.
Under this directory, detectron2 will look for datasets in the structure described below, if needed.
```
$DETECTRON2_DATASETS/
coco-stuff/ # COCO-Stuff
ADEChallengeData2016/ # ADE20K-150
ADE20K_2021_17_01/ # ADE20K-847
VOCdevkit/
VOC2010/ # PASCAL Context
VOC2012/ # PASCAL VOC
```
You can set the location for builtin datasets by `export DETECTRON2_DATASETS=/path/to/datasets`.
If left unset, the default is `./datasets` relative to your current working directory.
## Prepare data for [COCO-Stuff](https://github.com/nightrome/cocostuff):
### Expected data structure
```
coco-stuff/
annotations/
train2017/
val2017/
images/
train2017/
val2017/
# below are generated by prepare_coco_stuff.py
annotations_detectron2/
train2017/
val2017/
```
Download the COCO (2017) images from https://cocodataset.org/
```bash
wget http://images.cocodataset.org/zips/train2017.zip
wget http://images.cocodataset.org/zips/val2017.zip
```
Download the COCO-Stuff annotation from https://github.com/nightrome/cocostuff.
```bash
wget http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/stuffthingmaps_trainval2017.zip
```
Unzip `train2017.zip`, `val2017.zip`, and `stuffthingmaps_trainval2017.zip`. Then put them to the correct location listed above. and generate the labels for training and testing.
```
python datasets/prepare_coco_stuff.py
```
## Prepare data for [ADE20K-150](http://sceneparsing.csail.mit.edu):
### Expected data structure
```
ADEChallengeData2016/
annotations/
validation/
images/
validation/
# below are generated by prepare_ade20k_150.py
annotations_detectron2/
validation/
```
Download the data of ADE20K-150 from http://sceneparsing.csail.mit.edu.
```
wget http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
```
Unzip `ADEChallengeData2016.zip` and generate the labels for testing.
```
python datasets/prepare_ade20k_150.py
```
## Prepare data for [ADE20k-847](https://groups.csail.mit.edu/vision/datasets/ADE20K/):
### Expected data structure
```
ADE20K_2021_17_01/
images/
ADE/
validation/
index_ade20k.mat
index_ade20k.pkl
# below are generated by prepare_ade20k_847.py
annotations_detectron2/
validation/
```
Download the data of ADE20k-Full from https://groups.csail.mit.edu/vision/datasets/ADE20K/request_data/
Unzip the dataset and generate the labels for testing.
```
python datasets/prepare_ade20k_847.py
```
## Prepare data for [PASCAL VOC 2012](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit):
### Expected data structure
```
VOCdevkit/
VOC2012/
Annotations/
ImageSets/
JPEGImages/
SegmentationClass/
SegmentationClassAug/
SegmentationObject/
# below are generated by prepare_voc.py
annotations_detectron2
annotations_detectron2_bg
```
Download the data of PASCAL VOC from http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit.
We use SBD augmentated training data as SegmentationClassAug following [Deeplab](https://github.com/kazuto1011/deeplab-pytorch/blob/master/data/datasets/voc12/README.md).
```
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
wget https://www.dropbox.com/s/oeu149j8qtbs1x0/SegmentationClassAug.zip
```
Unzip `VOCtrainval_11-May-2012.tar` and `SegmentationClassAug.zip`. Then put them to the correct location listed above and generate the labels for testing.
```
python datasets/prepare_voc.py
```
## Prepare data for [PASCAL Context](https://www.cs.stanford.edu/~roozbeh/pascal-context/):
### Expected data structure
```
VOCdevkit/
VOC2010/
Annotations/
ImageSets/
JPEGImages/
SegmentationClass/
SegmentationObject/
trainval/
labels.txt
pascalcontext_val.txt
trainval_merged.json
# below are generated by prepare_pascal_context_59.py and prepare_pascal_context_459.py
annotations_detectron2/
pc459_val
pc59_val
```
Download the data of PASCAL VOC 2010 from https://www.cs.stanford.edu/~roozbeh/pascal-context/.
```
wget http://host.robots.ox.ac.uk/pascal/VOC/voc2010/VOCtrainval_03-May-2010.tar
```
Download the annotation for [59](https://codalabuser.blob.core.windows.net/public/trainval_merged.json) and [459](https://www.cs.stanford.edu/~roozbeh/pascal-context/trainval.tar.gz) classes.
```
wget https://codalabuser.blob.core.windows.net/public/trainval_merged.json
wget https://www.cs.stanford.edu/~roozbeh/pascal-context/trainval.tar.gz
```
Unzip `VOCtrainval_03-May-2010.tar` and `trainval.tar.gz`. Then put them to the correct location listed above and generate the labels for testing.
```
python datasets/prepare_pascal_context_59.py
python datasets/prepare_pascal_context_459.py
```
\ No newline at end of file
datasets/ade150.json 0 → 100644
View file @ f55a786e
["wall", "building", "sky", "floor", "tree", "ceiling", "road", "bed ", "windowpane", "grass", "cabinet", "sidewalk", "person", "earth", "door", "table", "mountain", "plant", "curtain", "chair", "car", "water", "painting", "sofa", "shelf", "house", "sea", "mirror", "rug", "field", "armchair", "seat", "fence", "desk", "rock", "wardrobe", "lamp", "bathtub", "railing", "cushion", "base", "box", "column", "signboard", "chest of drawers", "counter", "sand", "sink", "skyscraper", "fireplace", "refrigerator", "grandstand", "path", "stairs", "runway", "case", "pool table", "pillow", "screen door", "stairway", "river", "bridge", "bookcase", "blind", "coffee table", "toilet", "flower", "book", "hill", "bench", "countertop", "stove", "palm", "kitchen island", "computer", "swivel chair", "boat", "bar", "arcade machine", "hovel", "bus", "towel", "light", "truck", "tower", "chandelier", "awning", "streetlight", "booth", "television receiver", "airplane", "dirt track", "apparel", "pole", "land", "bannister", "escalator", "ottoman", "bottle", "buffet", "poster", "stage", "van", "ship", "fountain", "conveyer belt", "canopy", "washer", "plaything", "swimming pool", "stool", "barrel", "basket", "waterfall", "tent", "bag", "minibike", "cradle", "oven", "ball", "food", "step", "tank", "trade name", "microwave", "pot", "animal", "bicycle", "lake", "dishwasher", "screen", "blanket", "sculpture", "hood", "sconce", "vase", "traffic light", "tray", "ashcan", "fan", "pier", "crt screen", "plate", "monitor", "bulletin board", "shower", "radiator", "glass", "clock", "flag"]
datasets/ade847.json 0 → 100644
View file @ f55a786e
["wall", "building, edifice", "sky", "tree", "road, route", "floor, flooring", "ceiling", "bed", "sidewalk, pavement", "earth, ground", "cabinet", "person, individual, someone, somebody, mortal, soul", "grass", "windowpane, window", "car, auto, automobile, machine, motorcar", "mountain, mount", "plant, flora, plant life", "table", "chair", "curtain, drape, drapery, mantle, pall", "door", "sofa, couch, lounge", "sea", "painting, picture", "water", "mirror", "house", "rug, carpet, carpeting", "shelf", "armchair", "fence, fencing", "field", "lamp", "rock, stone", "seat", "river", "desk", "bathtub, bathing tub, bath, tub", "railing, rail", "signboard, sign", "cushion", "path", "work surface", "stairs, steps", "column, pillar", "sink", "wardrobe, closet, press", "snow", "refrigerator, icebox", "base, pedestal, stand", "bridge, span", "blind, screen", "runway", "cliff, drop, drop-off", "sand", "fireplace, hearth, open fireplace", "pillow", "screen door, screen", "toilet, can, commode, crapper, pot, potty, stool, throne", "skyscraper", "grandstand, covered stand", "box", "pool table, billiard table, snooker table", "palm, palm tree", "double door", "coffee table, cocktail table", "counter", "countertop", "chest of drawers, chest, bureau, dresser", "kitchen island", "boat", "waterfall, falls", "stove, kitchen stove, range, kitchen range, cooking stove", "flower", "bookcase", "controls", "book", "stairway, staircase", "streetlight, street lamp", "computer, computing machine, computing device, data processor, electronic computer, information processing system", "bus, autobus, coach, charabanc, double-decker, jitney, motorbus, motorcoach, omnibus, passenger vehicle", "swivel chair", "light, light source", "bench", "case, display case, showcase, vitrine", "towel", "fountain", "embankment", "television receiver, television, television set, tv, tv set, idiot box, boob tube, telly, goggle box", "van", "hill", "awning, sunshade, sunblind", "poster, posting, placard, notice, bill, card", "truck, motortruck", "airplane, aeroplane, plane", "pole", "tower", "court", "ball", "aircraft carrier, carrier, flattop, attack aircraft carrier", "buffet, counter, sideboard", "hovel, hut, hutch, shack, shanty", "apparel, wearing apparel, dress, clothes", "minibike, motorbike", "animal, animate being, beast, brute, creature, fauna", "chandelier, pendant, pendent", "step, stair", "booth, cubicle, stall, kiosk", "bicycle, bike, wheel, cycle", "doorframe, doorcase", "sconce", "pond", "trade name, brand name, brand, marque", "bannister, banister, balustrade, balusters, handrail", "bag", "traffic light, traffic signal, stoplight", "gazebo", "escalator, moving staircase, moving stairway", "land, ground, soil", "board, plank", "arcade machine", "eiderdown, duvet, continental quilt", "bar", "stall, stand, sales booth", "playground", "ship", "ottoman, pouf, pouffe, puff, hassock", "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin", "bottle", "cradle", "pot, flowerpot", "conveyer belt, conveyor belt, conveyer, conveyor, transporter", "train, railroad train", "stool", "lake", "tank, storage tank", "ice, water ice", "basket, handbasket", "manhole", "tent, collapsible shelter", "canopy", "microwave, microwave oven", "barrel, cask", "dirt track", "beam", "dishwasher, dish washer, dishwashing machine", "plate", "screen, crt screen", "ruins", "washer, automatic washer, washing machine", "blanket, cover", "plaything, toy", "food, solid food", "screen, silver screen, projection screen", "oven", "stage", "beacon, lighthouse, beacon light, pharos", "umbrella", "sculpture", "aqueduct", "container", "scaffolding, staging", "hood, exhaust hood", "curb, curbing, kerb", "roller coaster", "horse, equus caballus", "catwalk", "glass, drinking glass", "vase", "central reservation", "carousel", "radiator", "closet", "machine", "pier, wharf, wharfage, dock", "fan", "inflatable bounce game", "pitch", "paper", "arcade, colonnade", "hot tub", "helicopter", "tray", "partition, divider", "vineyard", "bowl", "bullring", "flag", "pot", "footbridge, overcrossing, pedestrian bridge", "shower", "bag, traveling bag, travelling bag, grip, suitcase", "bulletin board, notice board", "confessional booth", "trunk, tree trunk, bole", "forest", "elevator door", "laptop, laptop computer", "instrument panel", "bucket, pail", "tapestry, tapis", "platform", "jacket", "gate", "monitor, monitoring device", "telephone booth, phone booth, call box, telephone box, telephone kiosk", "spotlight, spot", "ring", "control panel", "blackboard, chalkboard", "air conditioner, air conditioning", "chest", "clock", "sand dune", "pipe, pipage, piping", "vault", "table football", "cannon", "swimming pool, swimming bath, natatorium", "fluorescent, fluorescent fixture", "statue", "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system", "exhibitor", "ladder", "carport", "dam", "pulpit", "skylight, fanlight", "water tower", "grill, grille, grillwork", "display board", "pane, pane of glass, window glass", "rubbish, trash, scrap", "ice rink", "fruit", "patio", "vending machine", "telephone, phone, telephone set", "net", "backpack, back pack, knapsack, packsack, rucksack, haversack", "jar", "track", "magazine", "shutter", "roof", "banner, streamer", "landfill", "post", "altarpiece, reredos", "hat, chapeau, lid", "arch, archway", "table game", "bag, handbag, pocketbook, purse", "document, written document, papers", "dome", "pier", "shanties", "forecourt", "crane", "dog, domestic dog, canis familiaris", "piano, pianoforte, forte-piano", "drawing", "cabin", "ad, advertisement, advertizement, advertising, advertizing, advert", "amphitheater, amphitheatre, coliseum", "monument", "henhouse", "cockpit", "heater, warmer", "windmill, aerogenerator, wind generator", "pool", "elevator, lift", "decoration, ornament, ornamentation", "labyrinth", "text, textual matter", "printer", "mezzanine, first balcony", "mattress", "straw", "stalls", "patio, terrace", "billboard, hoarding", "bus stop", "trouser, pant", "console table, console", "rack", "notebook", "shrine", "pantry", "cart", "steam shovel", "porch", "postbox, mailbox, letter box", "figurine, statuette", "recycling bin", "folding screen", "telescope", "deck chair, beach chair", "kennel", "coffee maker", "altar, communion table, lord's table", "fish", "easel", "artificial golf green", "iceberg", "candlestick, candle holder", "shower stall, shower bath", "television stand", "wall socket, wall plug, electric outlet, electrical outlet, outlet, electric receptacle", "skeleton", "grand piano, grand", "candy, confect", "grille door", "pedestal, plinth, footstall", "jersey, t-shirt, tee shirt", "shoe", "gravestone, headstone, tombstone", "shanty", "structure", "rocking chair, rocker", "bird", "place mat", "tomb", "big top", "gas pump, gasoline pump, petrol pump, island dispenser", "lockers", "cage", "finger", "bleachers", "ferris wheel", "hairdresser chair", "mat", "stands", "aquarium, fish tank, marine museum", "streetcar, tram, tramcar, trolley, trolley car", "napkin, table napkin, serviette", "dummy", "booklet, brochure, folder, leaflet, pamphlet", "sand trap", "shop, store", "table cloth", "service station", "coffin", "drawer", "cages", "slot machine, coin machine", "balcony", "volleyball court", "table tennis", "control table", "shirt", "merchandise, ware, product", "railway", "parterre", "chimney", "can, tin, tin can", "tanks", "fabric, cloth, material, textile", "alga, algae", "system", "map", "greenhouse", "mug", "barbecue", "trailer", "toilet tissue, toilet paper, bathroom tissue", "organ", "dishrag, dishcloth", "island", "keyboard", "trench", "basket, basketball hoop, hoop", "steering wheel, wheel", "pitcher, ewer", "goal", "bread, breadstuff, staff of life", "beds", "wood", "file cabinet", "newspaper, paper", "motorboat", "rope", "guitar", "rubble", "scarf", "barrels", "cap", "leaves", "control tower", "dashboard", "bandstand", "lectern", "switch, electric switch, electrical switch", "baseboard, mopboard, skirting board", "shower room", "smoke", "faucet, spigot", "bulldozer", "saucepan", "shops", "meter", "crevasse", "gear", "candelabrum, candelabra", "sofa bed", "tunnel", "pallet", "wire, conducting wire", "kettle, boiler", "bidet", "baby buggy, baby carriage, carriage, perambulator, pram, stroller, go-cart, pushchair, pusher", "music stand", "pipe, tube", "cup", "parking meter", "ice hockey rink", "shelter", "weeds", "temple", "patty, cake", "ski slope", "panel", "wallet", "wheel", "towel rack, towel horse", "roundabout", "canister, cannister, tin", "rod", "soap dispenser", "bell", "canvas", "box office, ticket office, ticket booth", "teacup", "trellis", "workbench", "valley, vale", "toaster", "knife", "podium", "ramp", "tumble dryer", "fireplug, fire hydrant, plug", "gym shoe, sneaker, tennis shoe", "lab bench", "equipment", "rocky formation", "plastic", "calendar", "caravan", "check-in-desk", "ticket counter", "brush", "mill", "covered bridge", "bowling alley", "hanger", "excavator", "trestle", "revolving door", "blast furnace", "scale, weighing machine", "projector", "soap", "locker", "tractor", "stretcher", "frame", "grating", "alembic", "candle, taper, wax light", "barrier", "cardboard", "cave", "puddle", "tarp", "price tag", "watchtower", "meters", "light bulb, lightbulb, bulb, incandescent lamp, electric light, electric-light bulb", "tracks", "hair dryer", "skirt", "viaduct", "paper towel", "coat", "sheet", "fire extinguisher, extinguisher, asphyxiator", "water wheel", "pottery, clayware", "magazine rack", "teapot", "microphone, mike", "support", "forklift", "canyon", "cash register, register", "leaf, leafage, foliage", "remote control, remote", "soap dish", "windshield, windscreen", "cat", "cue, cue stick, pool cue, pool stick", "vent, venthole, vent-hole, blowhole", "videos", "shovel", "eaves", "antenna, aerial, transmitting aerial", "shipyard", "hen, biddy", "traffic cone", "washing machines", "truck crane", "cds", "niche", "scoreboard", "briefcase", "boot", "sweater, jumper", "hay", "pack", "bottle rack", "glacier", "pergola", "building materials", "television camera", "first floor", "rifle", "tennis table", "stadium", "safety belt", "cover", "dish rack", "synthesizer", "pumpkin", "gutter", "fruit stand", "ice floe, floe", "handle, grip, handgrip, hold", "wheelchair", "mousepad, mouse mat", "diploma", "fairground ride", "radio", "hotplate", "junk", "wheelbarrow", "stream", "toll plaza", "punching bag", "trough", "throne", "chair desk", "weighbridge", "extractor fan", "hanging clothes", "dish, dish aerial, dish antenna, saucer", "alarm clock, alarm", "ski lift", "chain", "garage", "mechanical shovel", "wine rack", "tramway", "treadmill", "menu", "block", "well", "witness stand", "branch", "duck", "casserole", "frying pan", "desk organizer", "mast", "spectacles, specs, eyeglasses, glasses", "service elevator", "dollhouse", "hammock", "clothes hanging", "photocopier", "notepad", "golf cart", "footpath", "cross", "baptismal font", "boiler", "skip", "rotisserie", "tables", "water mill", "helmet", "cover curtain", "brick", "table runner", "ashtray", "street box", "stick", "hangers", "cells", "urinal", "centerpiece", "portable fridge", "dvds", "golf club", "skirting board", "water cooler", "clipboard", "camera, photographic camera", "pigeonhole", "chips", "food processor", "post box", "lid", "drum", "blender", "cave entrance", "dental chair", "obelisk", "canoe", "mobile", "monitors", "pool ball", "cue rack", "baggage carts", "shore", "fork", "paper filer", "bicycle rack", "coat rack", "garland", "sports bag", "fish tank", "towel dispenser", "carriage", "brochure", "plaque", "stringer", "iron", "spoon", "flag pole", "toilet brush", "book stand", "water faucet, water tap, tap, hydrant", "ticket office", "broom", "dvd", "ice bucket", "carapace, shell, cuticle, shield", "tureen", "folders", "chess", "root", "sewing machine", "model", "pen", "violin", "sweatshirt", "recycling materials", "mitten", "chopping board, cutting board", "mask", "log", "mouse, computer mouse", "grill", "hole", "target", "trash bag", "chalk", "sticks", "balloon", "score", "hair spray", "roll", "runner", "engine", "inflatable glove", "games", "pallets", "baskets", "coop", "dvd player", "rocking horse", "buckets", "bread rolls", "shawl", "watering can", "spotlights", "post-it", "bowls", "security camera", "runner cloth", "lock", "alarm, warning device, alarm system", "side", "roulette", "bone", "cutlery", "pool balls", "wheels", "spice rack", "plant pots", "towel ring", "bread box", "video", "funfair", "breads", "tripod", "ironing board", "skimmer", "hollow", "scratching post", "tricycle", "file box", "mountain pass", "tombstones", "cooker", "card game, cards", "golf bag", "towel paper", "chaise lounge", "sun", "toilet paper holder", "rake", "key", "umbrella stand", "dartboard", "transformer", "fireplace utensils", "sweatshirts", "cellular telephone, cellular phone, cellphone, cell, mobile phone", "tallboy", "stapler", "sauna", "test tube", "palette", "shopping carts", "tools", "push button, push, button", "star", "roof rack", "barbed wire", "spray", "ear", "sponge", "racket", "tins", "eyeglasses", "file", "scarfs", "sugar bowl", "flip flop", "headstones", "laptop bag", "leash", "climbing frame", "suit hanger", "floor spotlight", "plate rack", "sewer", "hard drive", "sprinkler", "tools box", "necklace", "bulbs", "steel industry", "club", "jack", "door bars", "control panel, instrument panel, control board, board, panel", "hairbrush", "napkin holder", "office", "smoke detector", "utensils", "apron", "scissors", "terminal", "grinder", "entry phone", "newspaper stand", "pepper shaker", "onions", "central processing unit, cpu, c p u , central processor, processor, mainframe", "tape", "bat", "coaster", "calculator", "potatoes", "luggage rack", "salt", "street number", "viewpoint", "sword", "cd", "rowing machine", "plug", "andiron, firedog, dog, dog-iron", "pepper", "tongs", "bonfire", "dog dish", "belt", "dumbbells", "videocassette recorder, vcr", "hook", "envelopes", "shower faucet", "watch", "padlock", "swimming pool ladder", "spanners", "gravy boat", "notice board", "trash bags", "fire alarm", "ladle", "stethoscope", "rocket", "funnel", "bowling pins", "valve", "thermometer", "cups", "spice jar", "night light", "soaps", "games table", "slotted spoon", "reel", "scourer", "sleeping robe", "desk mat", "dumbbell", "hammer", "tie", "typewriter", "shaker", "cheese dish", "sea star", "racquet", "butane gas cylinder", "paper weight", "shaving brush", "sunglasses", "gear shift", "towel rail", "adding machine, totalizer, totaliser"]
\ No newline at end of file
datasets/ade847_fcclip.json 0 → 100644
View file @ f55a786e
["wall, walls, interior wall, brick wall, stone wall", "building, buildings, edifice, edifices", "sky, clouds", "tree, trees", "road, route, street, roads, streets, routes", "floor, flooring", "ceiling", "bed, beds", "sidewalk, pavement", "earth, ground", "cabinet, cabinets, wall mounted cabine", "person, child, girl, boy, woman, man, people, children, girls, boys, women, men", "grass, grass field", "windowpane, window, windows", "car, automobile, cars", "mountain, mount, mountains", "plant, flora, plant life, plants, bushes", "table, tables, tablecloth", "chair, chairs", "curtain, drape, drapery, mantle, pall", "door, double door, doors", "sofa, couch, sofas, couches", "sea, ocean", "painting, picture, paintings, pictures, wallart, framed canvas", "water", "mirror, mirrors", "house exterior", "rug, carpet, carpeting", "shelf, shelves", "armchair, armchairs", "fence, fencing", "field", "lamp, lamps", "rock, stone, rocks, stones", "seat, seats", "river", "desk, desks", "bathtub, bathing tub, bath, tub", "railing, rail", "signboard, sign, signboards, signs", "cushion, cushions", "path", "work surface", "stairs, steps", "column, pillar", "sink", "wardrobe, closet, press, wardrobes, closets", "snow", "refrigerator, icebox", "pedestal", "bridge, span", "blind", "runway", "cliff, drop, drop-off", "sand", "fireplace, hearth, open fireplace", "pillow, pillows", "screen door, shower door", "toilet, commode, crapper, potty", "skyscraper, skyscrapers", "grandstand, covered stand", "box, boxes", "pool table, billiard table, snooker table", "palm tree, palm trees", "double door", "coffee table, cocktail table", "counter", "countertop, counter top, worktop", "chest of drawers, chest, bureau, dresser", "kitchen island", "boat", "waterfall, falls", "stove, kitchen stove, kitchen range, kitchen range, cooking stove", "flower, flowers", "bookcase", "controls", "book, books", "stairway, staircase", "streetlight, street lamp", "computer, computing machine, computing device, data processor, electronic computer, information processing system", "bus, autobus, double-decker, jitney, motorbus, motorcoach, omnibus, passenger vehicle", "swivel chair", "light, light source", "bench, benches", "case, display case, showcase, vitrine", "towel", "fountain", "embankment", "television receiver, television, television set, tv, tv set", "van", "hill", "awning, sunshade, sunblind", "poster, posting, placard, notice, bill, card", "truck, motortruck", "airplane, aeroplane, airplanes, aeroplanes", "pole", "tower, towers", "court", "ball, balls", "aircraft carrier, carrier, flattop, attack aircraft carrier", "buffet, sideboard", "hovel, hut, hutch, shack, shanty", "apparel, wearing apparel, dress, clothes", "minibike, motorbike", "animal, animate being, dog, cat, horse, cow, sheep, zebra, giraffe, bird", "chandelier, pendant, pendent", "step, stair", "booth, cubicle, stall, kiosk", "bicycle, bike", "doorframe, doorcase", "sconce, sconce lamp, sconce light", "pond", "trade name, brand name", "bannister, banister, balustrade, balusters, handrail", "bag, bags, gift bag, paper bag", "traffic light, traffic signal, traffic lights", "gazebo", "escalator, moving staircase, moving stairway", "land, soil", "board, plank", "arcade machine, arcade machines", "eiderdown, duvet, continental quilt", "bar", "stall, stand, sales booth", "playground", "ship", "ottoman, pouf, pouffe, puff, hassock", "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin", "bottle, bottles, water bottle", "cradle", "pot, flowerpot", "conveyer belt, conveyor belt, conveyer, conveyor, transporter", "train, railroad train", "stool, stools", "lake", "tank, storage tank", "ice, water ice", "basket, handbasket", "manhole", "tent, collapsible shelter", "canopy", "microwave, microwave oven", "barrel, cask, barrels, casks", "dirt track", "beam", "dishwasher, dish washer, dishwashing machine", "plate, plates", "crt screen", "ruins", "washer, automatic washer, washing machine", "blanket, cover", "plaything, toy, toys", "food, solid food", "projection screen", "oven", "stage", "beacon, lighthouse, beacon light, pharos", "umbrella", "sculpture, sculptures", "aqueduct", "container", "scaffolding, staging", "exhaust hood", "curb, curbing, kerb", "roller coaster", "horse, equus caballus", "catwalk", "glass, drinking glass", "vase, vases", "central reservation", "carousel", "radiator", "closet", "machine", "pier, wharf, wharfage, dock", "ceiling fan, floor fan", "inflatable bounce game", "pitch", "paper", "arcade, colonnade", "hot tub", "helicopter", "tray, trays", "partition, divider", "vineyard", "bowl", "bullring", "flag, flags", "pot", "footbridge, overcrossing, pedestrian bridge", "shower", "bag, traveling bag, travelling bag, grip, suitcase", "bulletin board, notice board", "confessional booth", "trunk, tree trunk, bole", "forest", "elevator door", "laptop, laptop computer", "instrument panel", "bucket, pail", "tapestry, tapis", "platform", "jacket", "gate", "monitor, monitoring device, monitors", "telephone booth, phone booth, call box, telephone box, telephone kiosk", "spotlight, spot", "ring", "control panel", "blackboard, chalkboard", "air conditioner, air conditioning", "chest", "clock", "sand dune", "pipe, pipage, piping", "vault", "table football", "cannon", "swimming pool, swimming bath", "fluorescent, fluorescent fixture", "statue", "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system", "exhibitor", "ladder", "carport", "dam", "pulpit", "skylight, fanlight", "water tower", "grill, grille, grillwork", "display board", "pane, pane of glass, window glass", "rubbish, trash, scrap", "ice rink", "fruit", "patio", "vending machine", "telephone, phone, telephone set", "net", "backpack, back pack, knapsack, packsack, rucksack, haversack", "jar", "track", "magazine", "shutter", "roof", "banner, streamer", "landfill", "post", "altarpiece, reredos", "hat, chapeau, lid", "arch, archway", "table game", "bag, handbag, pocketbook, purse", "document, written document, papers", "dome", "pier", "shanties", "forecourt", "crane", "dog, domestic dog, canis familiaris", "piano, pianoforte, forte-piano", "drawing", "cabin", "ad, advertisement, advertizement, advertising, advertizing, advert", "amphitheater, amphitheatre, coliseum", "monument", "henhouse", "cockpit", "heater, warmer", "windmill, aerogenerator, wind generator", "pool", "elevator, lift", "decoration, ornament, ornamentation", "labyrinth", "text, textual matter", "printer", "mezzanine, first balcony", "mattress", "straw", "stalls", "patio, terrace", "billboard, hoarding", "bus stop", "trouser, pant", "console table, console", "rack", "notebook", "shrine", "pantry", "cart", "steam shovel", "porch", "postbox, mailbox, letter box", "figurine, statuette", "recycling bin", "folding screen", "telescope", "deck chair, beach chair", "kennel", "coffee maker", "altar, communion table, lord's table", "fish", "easel", "artificial golf green", "iceberg", "candlestick, candle holder", "shower stall, shower bath", "television stand", "wall socket, wall plug, electric outlet, electrical outlet, outlet, electric receptacle", "skeleton", "grand piano, grand", "candy, confect", "grille door", "pedestal, plinth, footstall", "jersey, t-shirt, tee shirt", "shoe", "gravestone, headstone, tombstone", "shanty", "structure", "rocking chair, rocker", "bird", "place mat", "tomb", "big top", "gas pump, gasoline pump, petrol pump, island dispenser", "lockers", "cage", "finger", "bleachers", "ferris wheel", "hairdresser chair", "mat", "stands", "aquarium, fish tank, marine museum", "streetcar, tram, tramcar, trolley, trolley car", "napkin, table napkin, serviette", "dummy", "booklet, brochure, folder, leaflet, pamphlet", "sand trap", "shop, store", "table cloth", "service station", "coffin", "drawer", "cages", "slot machine, coin machine", "balcony", "volleyball court", "table tennis", "control table", "shirt", "merchandise, ware, product", "railway", "parterre", "chimney", "can, tin, tin can", "tanks", "fabric, cloth, material, textile", "alga, algae", "system", "map", "greenhouse", "mug", "barbecue", "trailer", "toilet tissue, toilet paper, bathroom tissue", "organ", "dishrag, dishcloth", "island", "keyboard", "trench", "basket, basketball hoop, hoop", "steering wheel, wheel", "pitcher, ewer", "goal", "bread, breadstuff, staff of life", "beds", "wood", "file cabinet", "newspaper, paper", "motorboat", "rope", "guitar", "rubble", "scarf", "barrels", "cap", "leaves", "control tower", "dashboard", "bandstand", "lectern", "switch, electric switch, electrical switch", "baseboard, mopboard, skirting board", "shower room", "smoke", "faucet, spigot", "bulldozer", "saucepan", "shops", "meter", "crevasse", "gear", "candelabrum, candelabra", "sofa bed", "tunnel", "pallet", "wire, conducting wire", "kettle, boiler", "bidet", "baby buggy, baby carriage, carriage, perambulator, pram, stroller, go-cart, pushchair, pusher", "music stand", "pipe, tube", "cup, cups, drinking glass, drinking glasses", "parking meter", "ice hockey rink", "shelter", "weeds", "temple", "patty, cake", "ski slope", "panel", "wallet", "wheel", "towel rack, towel horse", "roundabout", "canister, cannister, tin", "rod", "soap dispenser", "bell", "canvas", "box office, ticket office, ticket booth", "teacup", "trellis", "workbench", "valley, vale", "toaster", "knife", "podium", "ramp", "tumble dryer", "fireplug, fire hydrant, plug", "gym shoe, sneaker, tennis shoe", "lab bench", "equipment", "rocky formation", "plastic", "calendar", "caravan", "check-in-desk", "ticket counter", "brush", "mill", "covered bridge", "bowling alley", "hanger", "excavator", "trestle", "revolving door", "blast furnace", "scale, weighing machine", "projector", "soap", "locker", "tractor", "stretcher", "frame", "grating", "alembic", "candle, taper, wax light", "barrier", "cardboard", "cave", "puddle", "tarp", "price tag", "watchtower", "meters", "light bulb, bulb, bulbs", "tracks", "hair dryer", "skirt", "viaduct", "paper towel", "coat", "sheet", "fire extinguisher, extinguisher, asphyxiator", "water wheel", "pottery, clayware", "magazine rack", "teapot", "microphone, mike", "support", "forklift", "canyon", "cash register, register", "leaf, leafage, foliage", "remote control, remote", "soap dish", "windshield, windscreen", "cat", "cue, cue stick, pool cue, pool stick", "vent, venthole, vent-hole, blowhole", "videos", "shovel", "eaves", "antenna, aerial, transmitting aerial", "shipyard", "hen, biddy", "traffic cone", "washing machines", "truck crane", "cds", "niche", "scoreboard", "briefcase", "boot", "sweater, jumper", "hay", "pack", "bottle rack", "glacier", "pergola", "building materials", "television camera", "first floor", "rifle", "tennis table", "stadium", "safety belt", "cover", "dish rack", "synthesizer", "pumpkin", "gutter", "fruit stand", "ice floe, floe", "handle, grip, handgrip, hold", "wheelchair", "mousepad, mouse mat", "diploma", "fairground ride", "radio", "hotplate", "junk", "wheelbarrow", "stream", "toll plaza", "punching bag", "trough", "throne", "chair desk", "weighbridge", "extractor fan", "hanging clothes", "dish, dish aerial, dish antenna, saucer", "alarm clock, alarm", "ski lift", "chain", "garage", "mechanical shovel", "wine rack", "tramway", "treadmill", "menu", "block", "well", "witness stand", "branch", "duck", "casserole", "frying pan", "desk organizer", "mast", "spectacles, specs, eyeglasses, glasses", "service elevator", "dollhouse", "hammock", "clothes hanging", "photocopier", "notepad", "golf cart", "footpath", "cross", "baptismal font", "boiler", "skip", "rotisserie", "tables", "water mill", "helmet", "cover curtain", "brick", "table runner", "ashtray", "street box", "stick", "hangers", "cells", "urinal", "centerpiece", "portable fridge", "dvds", "golf club", "skirting board", "water cooler", "clipboard", "camera, photographic camera", "pigeonhole", "chips", "food processor", "post box", "lid", "drum", "blender", "cave entrance", "dental chair", "obelisk", "canoe", "mobile", "monitors", "pool ball", "cue rack", "baggage carts", "shore", "fork", "paper filer", "bicycle rack", "coat rack", "garland", "sports bag", "fish tank", "towel dispenser", "carriage", "brochure", "plaque", "stringer", "iron", "spoon", "flag pole", "toilet brush", "book stand", "water faucet, water tap, tap, hydrant", "ticket office", "broom", "dvd", "ice bucket", "carapace, shell, cuticle, shield", "tureen", "folders", "chess", "root", "sewing machine", "model", "pen", "violin", "sweatshirt", "recycling materials", "mitten", "chopping board, cutting board", "mask", "log", "mouse, computer mouse", "grill", "hole", "target", "trash bag", "chalk", "sticks", "balloon", "score", "hair spray", "roll", "runner", "engine", "inflatable glove", "games", "pallets", "baskets", "coop", "dvd player", "rocking horse", "buckets", "bread rolls", "shawl", "watering can", "spotlights", "post-it", "bowls", "security camera", "runner cloth", "lock", "alarm, warning device, alarm system", "side", "roulette", "bone", "cutlery", "pool balls", "wheels", "spice rack", "plant pots, plant pot, flower pot, flowerpot, planter", "towel ring", "bread box", "video", "funfair", "breads", "tripod", "ironing board", "skimmer", "hollow", "scratching post", "tricycle", "file box", "mountain pass", "tombstones", "cooker", "card game, cards", "golf bag", "towel paper", "chaise lounge", "sun", "toilet paper holder", "rake", "key", "umbrella stand", "dartboard", "transformer", "fireplace utensils", "sweatshirts", "cellular telephone, cellular phone, cellphone, cell, mobile phone", "tallboy", "stapler", "sauna", "test tube", "palette", "shopping carts", "tools", "push button, push, button", "star", "roof rack", "barbed wire", "spray", "ear", "sponge", "racket", "tins", "eyeglasses", "file", "scarfs", "sugar bowl", "flip flop", "headstones", "laptop bag", "leash", "climbing frame", "suit hanger", "floor spotlight", "plate rack", "sewer", "hard drive", "sprinkler", "tools box", "necklace", "bulbs", "steel industry", "club", "jack", "door bars", "control panel, instrument panel, control board, board, panel", "hairbrush", "napkin holder", "office", "smoke detector", "utensils", "apron", "scissors", "terminal", "grinder", "entry phone", "newspaper stand", "pepper shaker", "onions", "central processing unit, cpu, central processor, processor, mainframe", "tape", "bat", "coaster", "calculator", "potatoes", "luggage rack", "salt", "street number", "viewpoint", "sword", "cd", "rowing machine", "plug", "andiron, firedog, dog, dog-iron", "pepper", "tongs", "bonfire", "dog dish", "belt", "dumbbells", "videocassette recorder, vcr", "hook", "envelopes", "shower faucet", "watch", "padlock", "swimming pool ladder", "spanners", "gravy boat", "notice board", "trash bags", "fire alarm", "ladle", "stethoscope", "rocket", "funnel", "bowling pins", "valve", "thermometer", "cups", "spice jar", "night light", "soaps", "games table", "slotted spoon", "reel", "scourer", "sleeping robe", "desk mat", "dumbbell", "hammer", "tie", "typewriter", "shaker", "cheese dish", "sea star", "racquet", "butane gas cylinder", "paper weight", "shaving brush", "sunglasses", "gear shift", "towel rail", "adding machine, totalizer, totaliser"]
\ No newline at end of file
datasets/coco.json 0 → 100644
View file @ f55a786e
["person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush", "banner", "blanket", "branch", "bridge", "building-other", "bush", "cabinet", "cage", "cardboard", "carpet", "ceiling-other", "ceiling-tile", "cloth", "clothes", "clouds", "counter", "cupboard", "curtain", "desk-stuff", "dirt", "door-stuff", "fence", "floor-marble", "floor-other", "floor-stone", "floor-tile", "floor-wood", "flower", "fog", "food-other", "fruit", "furniture-other", "grass", "gravel", "ground-other", "hill", "house", "leaves", "light", "mat", "metal", "mirror-stuff", "moss", "mountain", "mud", "napkin", "net", "paper", "pavement", "pillow", "plant-other", "plastic", "platform", "playingfield", "railing", "railroad", "river", "road", "rock", "roof", "rug", "salad", "sand", "sea", "shelf", "sky-other", "skyscraper", "snow", "solid-other", "stairs", "stone", "straw", "structural-other", "table", "tent", "textile-other", "towel", "tree", "vegetable", "wall-brick", "wall-concrete", "wall-other", "wall-panel", "wall-stone", "wall-tile", "wall-wood", "water-other", "waterdrops", "window-blind", "window-other", "wood"]
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment