Kits Guide

README.md

@@ -3,7 +3,7 @@
 <img src=docs/source/nnDetection.svg width="600px">
 
 ![Version](https://img.shields.io/badge/nnDetection-v1.0-blue)
-![Python](https://img.shields.io/badge/python-3.8-orange)
+![Python](https://img.shields.io/badge/python-3.8+-orange)
 ![CUDA](https://img.shields.io/badge/CUDA-10.1%2F10.2%2F11.0-green)
 ![license](https://img.shields.io/badge/License-Apache%202.0-red.svg)
 
@@ -125,7 +125,7 @@ It can be imported from `nndet.ptmodule` and example can be found in `nndet.ptmo
 # Experiments & Data
 The data sets used for our experiments are not hosted or maintained by us, please give credit to the authors of the data sets.
 Some of the labels were corrected in data sets which we converted and can be downloaded.
-The `Reproducing Experiments` section has an overview of multiple guides which explain the preparation of the data sets.
+The `Experiments` section has an overview of multiple guides which explain the preparation of the data sets.
 
 ## Toy Data set
 Running `nndet_example` will automatically generate an example data set with 3D squares and sqaures with holes which can be used to test the installation or experiment with prototype code (it is still necessary to run the other nndet commands to process/train/predict the data set).
@@ -141,10 +141,25 @@ nndet_example --full [--num_processes]
 The full problem is very easy and the final results should be near perfect.
 After running the generation script follow the `Planning`, `Training` and `Inference` instructions below to construct the whole nnDetection pipeline.
 
-## Reproducing Experiments
+## Experiments
+Besides the self-configuring method, nnDetection acts as a standard interface for many datasets.
+We provide guides to prepare all datasets from our evaluation to the correct and make it easy to reproduce our resutls.
+Furthermore, we provide pretrained models which can be used without investing large amounts of compute to rerun our experiments (see Section `Pretrained Models`).
 
 <div align="center">
 
+### Results
+
+| <!-- --> | <!-- --> | <!-- --> |
+|:--------:|:--------:|:--------:|
+| | [nnDetection V0.1](/docs/results/nnDetectionV001.md) | |
+
+</div>
+
+<div align="center">
+
+### Guides
+
 | <!-- --> | <!-- --> | <!-- --> |
 |:----------------------------------------------------------------:|:-------------------------------------------------:|:---------------------------------------:|
 | [Task 003 Liver](/projects/Task001_Decathlon/README.md)          | [Task 011 Kits](/projects/Task011_Kits/README.md) | [Task 020 RibFrac](/projects/Task020_RibFrac/README.md)      |

projects/Task011_Kits/README.md

+# Kits
+**Disclaimer**: We are not the host of the data.
+Please make sure to read the requirements and usage policies of the data and **give credit to the authors of the dataset**!
+
+Please read the information from the homepage carefully and follow the rules and instructions provided by the original authors when using the data.
+- Homepage: https://kits19.grand-challenge.org/data/
+
+## Setup
+0. Follow the installation instructions of nnDetection and create a data directory name `Task011_Kits`.
+1. Follow the instructions and usage policies to download the data and place all the folders which contain the data and labels for each case into `Task011_Kits / raw`
+2. Run `python prepare.py` in `projects / Task011_Kits / scripts` of the nnDetection repository.
+3. Run `nndet_seg2det 011` to convert the semantic segmentation labels to instance segmentations.
+4. Run ... to download and replace the manually corrected labels.
+
+The data is now converted to the correct format and the instructions from the nnDetection README can be used to train the networks.

projects/Task011_Kits/scripts/prepare.py

+import shutil
+import os
+import sys
+from pathlib import Path
+
+from loguru import logger
+
+from nndet.io import save_json
+from nndet.io.prepare import create_test_split
+from nndet.utils.check import env_guard
+from nndet.utils.info import maybe_verbose_iterable
+
+
+@env_guard
+def main():
+    det_data_dir = Path(os.getenv('det_data'))
+    task_data_dir = det_data_dir / "Task011_Kits"
+    source_data_dir = task_data_dir / "raw"
+
+    if not source_data_dir.is_dir():
+        raise RuntimeError(f"{source_data_dir} should contain the raw data but does not exist.")
+
+    splitted_dir = task_data_dir / "raw_splitted"
+    target_data_dir = task_data_dir / "raw_splitted" / "imagesTr"
+    target_data_dir.mkdir(exist_ok=True, parents=True)
+    target_label_dir = task_data_dir / "raw_splitted" / "labelsTr"
+    target_label_dir.mkdir(exist_ok=True, parents=True)
+
+    logger.remove()
+    logger.add(sys.stdout, level="INFO")
+    logger.add(task_data_dir / "prepare.log", level="DEBUG")
+
+    # save meta info
+    dataset_info = {
+        "name": "Kits",
+        "task": "Task011_Kits",
+        "target_class": None,
+        "test_labels": True,
+
+        "seg2det_stuff": [1,], # define stuff classes: kidney
+        "seg2det_things": [2,], # define things classes: tumor
+        "min_size": 3.,
+
+        "labels": {"0": "lesion"},
+        "labels_stuff": {"1": "kidney"},
+        "modalities": {"0": "CT"},
+        "dim": 3,
+    }
+    save_json(dataset_info, task_data_dir / "dataset.json")
+
+    # prepare cases
+    cases = [str(c.name) for c in source_data_dir.iterdir() if c.is_dir()]
+    for c in maybe_verbose_iterable(cases):
+        logger.info(f"Copy case {c}")
+        case_id = int(c.split("_")[-1])
+        if case_id < 210:
+            shutil.copy(source_data_dir / c / "imaging.nii.gz", target_data_dir / f"{c}_0000.nii.gz")
+            shutil.copy(source_data_dir / c / "segmentation.nii.gz", target_label_dir / f"{c}.nii.gz")
+
+    # create an artificial test split
+    create_test_split(splitted_dir=splitted_dir,
+                      num_modalities=1,
+                      test_size=0.3,
+                      random_state=0,
+                      shuffle=True,
+                      )
+
+
+if __name__ == '__main__':
+    main()

projects/Task017_CADA/scripts/prepare.py

@@ -3,10 +3,10 @@ import shutil
 from pathlib import Path
 
 import SimpleITK as sitk
-from tqdm import tqdm
 
 from nndet.io import save_json
 from nndet.utils.check import env_guard
+from nndet.utils.info import maybe_verbose_iterable
 
 
 def run_prep(source_data: Path, source_label: Path,
@@ -56,7 +56,7 @@ def main():
     # prepare data & label
     case_ids = [(p.stem).rsplit('_', 1)[0] for p in source_data_dir.glob("*.nii.gz")]
     print(f"Found {len(case_ids)} case ids")
-    for cid in tqdm(case_ids):
+    for cid in maybe_verbose_iterable(case_ids):
         run_prep(
             source_data=source_data_dir / f"{cid}_orig.nii.gz",
             source_label=source_label_dir / f"{cid}_labeledMasks.nii.gz",

projects/Task019_ADAM/scripts/prepare.py

@@ -2,11 +2,10 @@ import os
 import shutil
 from pathlib import Path
 
-from tqdm import tqdm
-
 from nndet.io import save_json
 from nndet.io.prepare import instances_from_segmentation
 from nndet.utils.check import env_guard
+from nndet.utils.info import maybe_verbose_iterable
 
 
 def run_prep_fg_v_bg(
@@ -61,7 +60,7 @@ def main():
     # prepare data
     case_ids = [p.stem for p in source_data_dir.iterdir() if p.is_dir()]
     print(f"Found {len(case_ids)} case ids")
-    for cid in tqdm(case_ids):
+    for cid in maybe_verbose_iterable(case_ids):
         run_prep_fg_v_bg(
             case_id=cid,
             source_data=source_data_dir,

projects/Task020_RibFrac/scripts/prepare.py

@@ -3,10 +3,10 @@ import shutil
 from pathlib import Path
 
 import pandas as pd
-from tqdm import tqdm
 
 from nndet.io import save_json
 from nndet.utils.check import env_guard
+from nndet.utils.info import maybe_verbose_iterable
 
 
 def create(
@@ -92,7 +92,7 @@ def main():
     }
     save_json(meta, task_data_dir / "dataset.json")
 
-    for ip, lp in tqdm(list(zip(image_paths, label_paths))):
+    for ip, lp in maybe_verbose_iterable(list(zip(image_paths, label_paths))):
         create(image_source=ip,
                label_source=lp,
                image_target_dir=target_data_dir,