[Fix] Updata `bbox_3d` info in `cam_instances` for `mono3d-det` task (#2046)

* add mmengine assertion

* add docstring

* fix mminstall

* fix semantic segmentation configs

* fix semantic seg

* fix lint

* remove unused imports

* fix

* update pointnet2-s3dis config

* update data_list according to scene_idxs

* remove useless function

* fix bug lack `eval_ann_info` during evaluation

* fix bug

* update doc

* fix lint

* fix dataset converter

* update mmengine version

* delete whitespace

* [Fix] fix point cloud loop visualization error (#1914)

* fix point cloud loop visualization error

* fix browse_dataset

* fix browse_dataset

* support saving lidar_det
Co-authored-by: JingweiZhang12 <zjw18@mails.tsinghua.edu.cn>

* [Enhance] Show instance statistics before and after through pipeline (#1863)

* add instance statistics before and after through pipeline

* add docstring

* support showing cat-wise instance statistics

* show all statistics of the dataset

* small fix

* polish code

* show table

* small fix

* rename some varibles

* update task name and config for fcos3d++ on nus (#1927)

* update docs

* update docs

* Update det3d_dataset.py

* remove unnecessary optional docs

* update docs

* update docstrings

* update docs

* fix lint

* update docs

* [Refactor]: Refactor SASSD (#1901)

* refactor sassd

* rename some varibles

* add ut for sassd

* add some comments

* add some comments

* [Refactor] Use `mmeval.MeanIoU` for `SegMetric` (#1929)

* Use mmeval.MeanIoU

* fix comments and add self.reset

* solve some comments

* fix docstring
Co-authored-by: ChaimZhu <zhuchenming@pjlab.org.cn>

* fix

* fix

* fix

* fix lint

* add args and kwargs

* [Docs] Add mmyolo link in README (#1935)

* [Refactor] Use `mmeval.MeanIoU` for `SegMetric` (#1929)

* Use mmeval.MeanIoU

* fix comments and add self.reset

* add mmyolo
Co-authored-by: ChaimZhu <zhuchenming@pjlab.org.cn>

* [Fix] fix seg metric issues and circle ci (#1962)

* fix circle ci

* update

* update

* [Enhance] Support evaluation for waymo dataset on s3 filesystem when tensorflow>=2.6.0 (#1958)

* support tensorflow>2.6.0

* add filter conditions

* [Fix]: change --pkl to --pkl-path in the data upgrade scripts (#1965)

* [Refactor] update `metainfo` in pkl file and add `categories` into `metainfo` (#1934)

* update metainfo in pkl file

* update

* add version for waymo

* update kitti metric

* [Refactor] Refactor FCAF3D (#1945)

* add fcaf3d config

* support inference of fcaf3d on scannet-3d

* minor changes

* fix config of scannet

* align test precision of facaf3d && support training

* add ut

* fix bugs of dataset conversion and indoor_metric

* support fcaf3d on sunrgbd datasets and add rorate_iou_loss ut

* small fix

* add docstring and typehint

* add typehint for SparseTensor

* remove pdb

* fix fcaf3d ut

* [Docs] Update Chinese documentation (#1891)

* [Enhance] Add an example of pcd to bin format. (#1973)

* [Enhance] Add an example of pcd to bin format.

* Fix installation and typo.

* [Fix]: fix image conversion of Waymo to avoid information loss (#1979)

* add mmeval

* [Refactor] rename `CLASSES` and `PALETTE` to `classes` and `palette` in dataset metainfo (#1932)

* rame CLASS and PALETTE to class and palette

* change mmcv-full to mmcv

* fix comments

* [CI]: fix CI (#2007)

* Update docs && support loading old checkpoints in FCAF3D  (#1974)

* add loading converted keys and README, metafile

* update readme

* reorganize readme

* add version in FCAF3d detector

* fix converter error

* [Fix] fix kitti evaluation bugs on test dataset (#2005)

* fix kitti test evaluation bugs

* update

* [Fix] update the key name of modules from mmdet in cfgs (#2011)

* [Fix] fix instance statistics when only detecting a single class (#2003)

* Update iou_neg_piecewise_sampler.py

* fix lint

* [Enhance] Remove useless json file converter (#1971)

* [Enhance] Refactor the import of `mmdet` (#1947)

* fix config

* refactor import of `mmdet`

* fix lint

* fix lint

* [Enhance] Speed up evaluation on waymo (#2008)

* support fast eval on waymo

* support waymo evaluatioin more flexible and faster

* support waymo evaluatioin more flexible and faster

* renames

* add docs

* add guides for multi-thread evaluation toolkit

* fix docstring

* add download link for idx2metainfo

* add docstring

* set convert_kitti_format=False in Lidar-based methods

* fix docs

* add docstring

* [Features] Support PV_RCNN modules (#1957)

* add pvrcnn module code

* add voxelsa

* fix

* fix comments

* fix comments

* fix comments

* add stack sa

* fix

* fix comments

* fix comments

* fix

* add ut

* fix comments

* fix kitti mono_det bug

* [Fix]Fix a bug in StackQueryAndGroup (#2043)

* fix a bug

* fix a batch inference bug

* fix docs

* [Community] Add 'Projects/' folder, and the first example project (#2042)

* add project in mmdet3d

* fix

* fix

* [Fix] Fix waymo converter (#2040)

* fix waymo converter

* [Fix] Update waymo converter and fix lint

* Update waymo_converter.py

* Update kitti_converter.py

* Update update_infos_to_v2.py

* revert

* fix paths in `update_infos_to_v2` on Windows by using `pathlib` (#2031)

* fix paths on Windows by using `pathlib`

* refactor Path import

* simplify the warning and error when using open3d and MinkowskiEngine (#2027)

* [FIX] replace DefaultFormatBundle/3D with Pack(3D)DetInputs (#1987)

* replace defaultformatbundle3d with pack3ddetinputs

* remove normalize, pad, imagetotensor from configs

* rm unused key 'img_norm_cfg'

* fix lint errors

* fix lint errors

* fix lint error

* xx

* [Docs] Refine the documentation (#1994)

* refine doc

* refine docs

* replace `CLASSES` with `classes`

* update doc

* Minor fix
Co-authored-by: Tai-Wang <tab_wang@outlook.com>

* Update converter

* [Fix] Fix some loading bugs and support fov_image_based mode in Waymo dataset.  (#1942)

* modify sample_id to sample_id and support fov_image_based on waymo dataset

* Update waymo_metric.py

* Minor fix

* Minor fix

* Minor fix

* Minor fix

* Minor fix

* Minor fix

* Minor fixes

* Minor fixes

* Remove optional

* fix dataset instances converting bugs

* Add a blank line to fix the doc compilation format

* Fix the bin file name in waymo_fov config

* Resolve conflicts

* fix ci and other things
Co-authored-by: Tai-Wang <tab_wang@outlook.com>
Co-authored-by: lianqing11 <lianqing11@foxmail.com>
Co-authored-by: ChaimZhu <zhuchenming@pjlab.org.cn>

* [Fix] Add num_features in Lyft dataset info file (#1948)
Co-authored-by: Tai-Wang <tab_wang@outlook.com>

* [Feature] Support FCAF3D on S3DIS dataset in `dev-1.x`  branch (#1984)

* support fcaf3d for s3dis dataset

* Update convert_utils.py

* Update seg3d_dataset.py

* Delete compose.py

* fix import error

* use `mmengine.Compose`

* Update s3dis-3d.py

* Update fcaf3d_2xb8_s3dis-3d-5class.py

* Update s3dis_dataset.py

* update unittest for s3dis

* update docs

* use `mmcv.Compose` instead of `mmengine.Compose`

* update docstring

* fix s3dis preprocessing bug

* Add typehint

* Update config and fix s3dis dataset

* update typehit

* Update convert_utils.py

* Update README and metafile
Co-authored-by: Tai-Wang <tab_wang@outlook.com>

* Update kitti_dataset.py

* replace `List` wiith `Tuple` format to represent `origin`

* Update convert_utils.py

* add comments

* update

* resolve
Co-authored-by: ChaimZhu <zhuchenming@pjlab.org.cn>
Co-authored-by: JingweiZhang12 <zjw18@mails.tsinghua.edu.cn>
Co-authored-by: Qing Lian <lianqing1997@gmail.com>
Co-authored-by: Shaun <cq.xiang@foxmail.com>
Co-authored-by: Wencheng Wu <41542251+274869388@users.noreply.github.com>
Co-authored-by: Erli Ouyang <ouyangerli@gmail.com>
Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com>
Co-authored-by: vavanade <vavanade@users.noreply.github.com>
Co-authored-by: Tai-Wang <tab_wang@outlook.com>
Co-authored-by: lianqing11 <lianqing11@foxmail.com>

mmdet3d/datasets/convert_utils.py

@@ -253,16 +253,17 @@ def get_kitti_style_2d_boxes(info: dict,
         # transform the center from [0.5, 1.0, 0.5] to [0.5, 0.5, 0.5]
         dst = np.array([0.5, 0.5, 0.5])
         src = np.array([0.5, 1.0, 0.5])
-        loc = loc + dim * (dst - src)
-        loc_3d = np.copy(loc)
-        gt_bbox_3d = np.concatenate([loc, dim, rot], axis=1).astype(np.float32)
+        # gravity center
+        loc_center = loc + dim * (dst - src)
+        gt_bbox_3d = np.concatenate([loc_center, dim, rot],
+                                    axis=1).astype(np.float32)
 
         # Filter out the corners that are not in front of the calibrated
         # sensor.
         corners_3d = box_np_ops.center_to_corner_box3d(
             gt_bbox_3d[:, :3],
             gt_bbox_3d[:, 3:6],
-            gt_bbox_3d[:, 6], [0.5, 0.5, 0.5],
+            gt_bbox_3d[:, 6], (0.5, 0.5, 0.5),
             axis=1)
         corners_3d = corners_3d[0].T  # (1, 8, 3) -> (3, 8)
         in_front = np.argwhere(corners_3d[2, :] > 0).flatten()
@@ -291,12 +292,12 @@ def get_kitti_style_2d_boxes(info: dict,
 
         # If mono3d=True, add 3D annotations in camera coordinates
         if mono3d and (repro_rec is not None):
+            # use bottom center to represent the bbox_3d
             repro_rec['bbox_3d'] = np.concatenate(
-                [loc_3d, dim, rot],
-                axis=1).astype(np.float32).squeeze().tolist()
+                [loc, dim, rot], axis=1).astype(np.float32).squeeze().tolist()
             repro_rec['velocity'] = -1  # no velocity in KITTI
 
-            center_3d = np.array(loc).reshape([1, 3])
+            center_3d = np.array(loc_center).reshape([1, 3])
             center_2d_with_depth = points_cam2img(
                 center_3d, camera_intrinsic, with_depth=True)
             center_2d_with_depth = center_2d_with_depth.squeeze().tolist()

mmdet3d/datasets/kitti_dataset.py

@@ -24,6 +24,15 @@ class KittiDataset(Det3DDataset):
             Defaults to dict(use_lidar=True).
         default_cam_key (str): The default camera name adopted.
             Defaults to 'CAM2'.
+        load_type (str): Type of loading mode. Defaults to 'frame_based'.
+
+            - 'frame_based': Load all of the instances in the frame.
+            - 'mv_image_based': Load all of the instances in the frame and need
+              to convert to the FOV-based data type to support image-based
+              detector.
+            - 'fov_image_based': Only load the instances inside the default
+              cam, and need to convert to the FOV-based data type to support
+              image-based detector.
         box_type_3d (str): Type of 3D box of this dataset.
             Based on the `box_type_3d`, the dataset will encapsulate the box
             to its original format then converted them to `box_type_3d`.
@@ -32,15 +41,6 @@ class KittiDataset(Det3DDataset):
             - 'LiDAR': Box in LiDAR coordinates.
             - 'Depth': Box in depth coordinates, usually for indoor dataset.
             - 'Camera': Box in camera coordinates.
-        load_type (str): Type of loading mode. Defaults to 'frame_based'.
-
-            - 'frame_based': Load all of the instances in the frame.
-            - 'mv_image_based': Load all of the instances in the frame and need
-                to convert to the FOV-based data type to support image-based
-                detector.
-            - 'fov_image_based': Only load the instances inside the default
-                cam, and need to convert to the FOV-based data type to support
-                image-based detector.
         filter_empty_gt (bool): Whether to filter the data with empty GT.
             If it's set to be True, the example with empty annotations after
             data pipeline will be dropped and a random example will be chosen

mmdet3d/datasets/waymo_dataset.py

@@ -156,20 +156,12 @@ class WaymoDataset(KittiDataset):
             centers_2d = np.zeros((0, 2), dtype=np.float32)
             depths = np.zeros((0), dtype=np.float32)
 
-        if self.load_type in ['fov_image_based', 'mv_image_based']:
-            gt_bboxes_3d = CameraInstance3DBoxes(
-                ann_info['gt_bboxes_3d'],
-                box_dim=ann_info['gt_bboxes_3d'].shape[-1],
-                origin=(0.5, 0.5, 0.5))
-
-        else:
-            # in waymo, lidar2cam = R0_rect @ Tr_velo_to_cam
-            # convert gt_bboxes_3d to velodyne coordinates with `lidar2cam`
-            lidar2cam = np.array(
-                info['images'][self.default_cam_key]['lidar2cam'])
-            gt_bboxes_3d = CameraInstance3DBoxes(
-                ann_info['gt_bboxes_3d']).convert_to(self.box_mode_3d,
-                                                     np.linalg.inv(lidar2cam))
+        # in waymo, lidar2cam = R0_rect @ Tr_velo_to_cam
+        # convert gt_bboxes_3d to velodyne coordinates with `lidar2cam`
+        lidar2cam = np.array(info['images'][self.default_cam_key]['lidar2cam'])
+        gt_bboxes_3d = CameraInstance3DBoxes(
+            ann_info['gt_bboxes_3d']).convert_to(self.box_mode_3d,
+                                                 np.linalg.inv(lidar2cam))
         ann_info['gt_bboxes_3d'] = gt_bboxes_3d
 
         anns_results = dict(