- 26 Jun, 2020 1 commit
-
-
vivek rathod authored
318497061 by rathodv: 1. Replace strategy.run() with strategy.experimental_run_v2() and replace tensor.ref() with tensor.experimental_ref() to be compatible with TF2.1 runtime on cloud. 2. Fix expected string in failing PY3 tests. -- 318493408 by aom: Implements "Bidirectional Feature Pyramid Network Generators" for BiFPN-based feature extractors (e.g. EfficientDet). -- PiperOrigin-RevId: 318497061
-
- 26 May, 2020 1 commit
-
-
pkulzc authored
* Merged commit includes the following changes: 311933687 by Sergio Guadarrama: Removes spurios use of tf.compat.v2, which results in spurious tf.compat.v1.compat.v2. Adds basic test to nasnet_utils. Replaces all remaining import tensorflow as tf with import tensorflow.compat.v1 as tf -- 311766063 by Sergio Guadarrama: Removes explicit tf.compat.v1 in all call sites (we already import tf.compat.v1, so this code was doing tf.compat.v1.compat.v1). The existing code worked in latest version of tensorflow, 2.2, (and 1.15) but not in 1.14 or in 2.0.0a, this CL fixes it. -- 311624958 by Sergio Guadarrama: Updates README that doesn't render properly in github documentation -- 310980959 by Sergio Guadarrama: Moves research_models/slim off tf.contrib.slim/layers/framework to tf_slim -- 310263156 by Sergio Guadarrama: Adds model breakdown for MobilenetV3 -- 308640...
-
- 12 May, 2020 1 commit
-
-
pkulzc authored
310447280 by lzc: Internal change 310420845 by Zhichao Lu: Open source the internal Context RCNN code. -- 310362339 by Zhichao Lu: Internal change 310259448 by lzc: Update required TF version for OD API. -- 310252159 by Zhichao Lu: Port patch_ops_test to TF1/TF2 as TPUs. -- 310247180 by Zhichao Lu: Ignore keypoint heatmap loss in the regions/bounding boxes with target keypoint class but no valid keypoint annotations. -- 310178294 by Zhichao Lu: Opensource MnasFPN https://arxiv.org/abs/1912.01106 -- 310094222 by lzc: Internal changes. -- 310085250 by lzc: Internal Change. -- 310016447 by huizhongc: Remove unrecognized classes from labeled_classes. -- 310009470 by rathodv: Mark batcher.py as TF1 only. -- 310001984 by rathodv: Update core/preprocessor.py to be compatible with TF1/TF2.. -- 309455035 by Zhichao Lu: Makes the freezable_batch_norm_test run w/ v2 behavior. The main change is in v2 updates will happen right away when running batchnorm in training mode. So, we need to restore the weights between batchnorm calls to make sure the numerical checks all start from the same place. -- 309425881 by Zhichao Lu: Make TF1/TF2 optimizer builder tests explicit. -- 309408646 by Zhichao Lu: Make dataset builder tests TF1 and TF2 compatible. -- 309246305 by Zhichao Lu: Added the functionality of combining the person keypoints and object detection annotations in the binary that converts the COCO raw data to TfRecord. -- 309125076 by Zhichao Lu: Convert target_assigner_utils to TF1/TF2. -- 308966359 by huizhongc: Support SSD training with partially labeled groundtruth. -- 308937159 by rathodv: Update core/target_assigner.py to be compatible with TF1/TF2. -- 308774302 by Zhichao Lu: Internal -- 308732860 by rathodv: Make core/prefetcher.py compatible with TF1 only. -- 308726984 by rathodv: Update core/multiclass_nms_test.py to be TF1/TF2 compatible. -- 308714718 by rathodv: Update core/region_similarity_calculator_test.py to be TF1/TF2 compatible. -- 308707960 by rathodv: Update core/minibatch_sampler_test.py to be TF1/TF2 compatible. -- 308700595 by rathodv: Update core/losses_test.py to be TF1/TF2 compatible and remove losses_test_v2.py -- 308361472 by rathodv: Update core/matcher_test.py to be TF1/TF2 compatible. -- 308335846 by Zhichao Lu: Updated the COCO evaluation logics and populated the groundturth area information through. This change matches the groundtruth format expected by the COCO keypoint evaluation. -- 308256924 by rathodv: Update core/keypoints_ops_test.py to be TF1/TF2 compatible. -- 308256826 by rathodv: Update class_agnostic_nms_test.py to be TF1/TF2 compatible. -- 308256112 by rathodv: Update box_list_ops_test.py to be TF1/TF2 compatible. -- 308159360 by Zhichao Lu: Internal change 308145008 by Zhichao Lu: Added 'image/class/confidence' field in the TFExample decoder. -- 307651875 by rathodv: Refactor core/box_list.py to support TF1/TF2. -- 307651798 by rathodv: Modify box_coder.py base class to work with with TF1/TF2 -- 307651652 by rathodv: Refactor core/balanced_positive_negative_sampler.py to support TF1/TF2. -- 307651571 by rathodv: Modify BoxCoders tests to use test_case:execute method to allow testing with TF1.X and TF2.X -- 307651480 by rathodv: Modify Matcher tests to use test_case:execute method to allow testing with TF1.X and TF2.X -- 307651409 by rathodv: Modify AnchorGenerator tests to use test_case:execute method to allow testing with TF1.X and TF2.X -- 307651314 by rathodv: Refactor model_builder to support TF1 or TF2 models based on TensorFlow version. -- 307092053 by Zhichao Lu: Use manager to save checkpoint. -- 307071352 by ronnyvotel: Fixing keypoint visibilities. Now by default, the visibility is marked True if the keypoint is labeled (regardless of whether it is visible or not). Also, if visibilities are not present in the dataset, they will be created based on whether the keypoint coordinates are finite (vis = True) or NaN (vis = False). -- 307069557 by Zhichao Lu: Internal change to add few fields related to postprocessing parameters in center_net.proto and populate those parameters to the keypoint postprocessing functions. -- 307012091 by Zhichao Lu: Make Adam Optimizer's epsilon proto configurable. Potential issue: tf.compat.v1's AdamOptimizer has a default epsilon on 1e-08 ([doc-link](https://www.tensorflow.org/api_docs/python/tf/compat/v1/train/AdamOptimizer)) whereas tf.keras's AdamOptimizer has default epsilon 1e-07 ([doc-link](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam)) -- 306858598 by Zhichao Lu: Internal changes to update the CenterNet model: 1) Modified eval job loss computation to avoid averaging over batches with zero loss. 2) Updated CenterNet keypoint heatmap target assigner to apply box size to heatmap Guassian standard deviation. 3) Updated the CenterNet meta arch keypoint losses computation to apply weights outside of loss function. -- 306731223 by jonathanhuang: Internal change. -- 306549183 by rathodv: Internal Update. -- 306542930 by rathodv: Internal Update -- 306322697 by rathodv: Internal. -- 305345036 by Zhichao Lu: Adding COCO Camera Traps Json to tf.Example beam code -- 304104869 by lzc: Internal changes. -- 304068971 by jonathanhuang: Internal change. -- 304050469 by Zhichao Lu: Internal change. -- 303880642 by huizhongc: Support parsing partially labeled groundtruth. -- 303841743 by Zhichao Lu: Deprecate nms_on_host in SSDMetaArch. -- 303803204 by rathodv: Internal change. -- 303793895 by jonathanhuang: Internal change. -- 303467631 by rathodv: Py3 update for detection inference test. -- 303444542 by rathodv: Py3 update to metrics module -- 303421960 by rathodv: Update json_utils to python3. -- 302787583 by ronnyvotel: Coco results generator for submission to the coco test server. -- 302719091 by Zhichao Lu: Internal change to add the ResNet50 image feature extractor for CenterNet model. -- 302116230 by Zhichao Lu: Added the functions to overlay the heatmaps with images in visualization util library. -- 301888316 by Zhichao Lu: Fix checkpoint_filepath not defined error. -- 301840312 by ronnyvotel: Adding keypoint_scores to visualizations. -- 301683475 by ronnyvotel: Introducing the ability to preprocess `keypoint_visibilities`. Some data augmentation ops such as random crop can filter instances and keypoints. It's important to also filter keypoint visibilities, so that the groundtruth tensors are always in alignment. -- 301532344 by Zhichao Lu: Don't use tf.divide since "Quantization not yet supported for op: DIV" -- 301480348 by ronnyvotel: Introducing keypoint evaluation into model lib v2. Also, making some fixes to coco keypoint evaluation. -- 301454018 by Zhichao Lu: Added the image summary to visualize the train/eval input images and eval's prediction/groundtruth side-by-side image. -- 301317527 by Zhichao Lu: Updated the random_absolute_pad_image function in the preprocessor library to support the keypoints argument. -- 301300324 by Zhichao Lu: Apply name change(experimental_run_v2 -> run) for all callers in Tensorflow. -- 301297115 by ronnyvotel: Utility function for setting keypoint visibilities based on keypoint coordinates. -- 301248885 by Zhichao Lu: Allow MultiworkerMirroredStrategy(MWMS) use by adding checkpoint handling with temporary directories in model_lib_v2. Added missing WeakKeyDictionary cfer_fn_cache field in CollectiveAllReduceStrategyExtended. -- 301224559 by Zhichao Lu: ...1) Fixes model_lib to also use keypoints while preparing model groundtruth. ...2) Tests model_lib with newly added keypoint metrics config. -- 300836556 by Zhichao Lu: Internal changes to add keypoint estimation parameters in CenterNet proto. -- 300795208 by Zhichao Lu: Updated the eval_util library to populate the keypoint groundtruth to eval_dict. -- 299474766 by Zhichao Lu: ...Modifies eval_util to create Keypoint Evaluator objects when configured in eval config. -- 299453920 by Zhichao Lu: Add swish activation as a hyperperams option. -- 299240093 by ronnyvotel: Keypoint postprocessing for CenterNetMetaArch. -- 299176395 by Zhichao Lu: Internal change. -- 299135608 by Zhichao Lu: Internal changes to refactor the CenterNet model in preparation for keypoint estimation tasks. -- 298915482 by Zhichao Lu: Make dataset_builder aware of input_context for distributed training. -- 298713595 by Zhichao Lu: Handling data with negative size boxes. -- 298695964 by Zhichao Lu: Expose change_coordinate_frame as a config parameter; fix multiclass_scores optional field. -- 298492150 by Zhichao Lu: Rename optimizer_builder_test_v2.py -> optimizer_builder_v2_test.py -- 298476471 by Zhichao Lu: Internal changes to support CenterNet keypoint estimation. -- 298365851 by ronnyvotel: Fixing a bug where groundtruth_keypoint_weights were being padded with a dynamic dimension. -- 297843700 by Zhichao Lu: Internal change. -- 297706988 by lzc: Internal change. -- 297705287 by ronnyvotel: Creating the "snapping" behavior in CenterNet, where regressed keypoints are refined with updated candidate keypoints from a heatmap. -- 297700447 by Zhichao Lu: Improve checkpoint checking logic with TF2 loop. -- 297686094 by Zhichao Lu: Convert "import tensorflow as tf" to "import tensorflow.compat.v1". -- 297670468 by lzc: Internal change. -- 297241327 by Zhichao Lu: Convert "import tensorflow as tf" to "import tensorflow.compat.v1". -- 297205959 by Zhichao Lu: Internal changes to support refactored the centernet object detection target assigner into a separate library. -- 297143806 by Zhichao Lu: Convert "import tensorflow as tf" to "import tensorflow.compat.v1". -- 297129625 by Zhichao Lu: Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration -- 297117070 by Zhichao Lu: Explicitly replace "import tensorflow" with "tensorflow.compat.v1" for TF2.x migration -- 297030190 by Zhichao Lu: Add configuration options for visualizing keypoint edges -- 296359649 by Zhichao Lu: Support DepthwiseConv2dNative (of separable conv) in weight equalization loss. -- 296290582 by Zhichao Lu: Internal change. -- 296093857 by Zhichao Lu: Internal changes to add general target assigner utilities. -- 295975116 by Zhichao Lu: Fix visualize_boxes_and_labels_on_image_array to show max_boxes_to_draw correctly. -- 295819711 by Zhichao Lu: Adds a flag to visualize_boxes_and_labels_on_image_array to skip the drawing of axis aligned bounding boxes. -- 295811929 by Zhichao Lu: Keypoint support in random_square_crop_by_scale. -- 295788458 by rathodv: Remove unused checkpoint to reduce repo size on github -- 295787184 by Zhichao Lu: Enable visualization of edges between keypoints -- 295763508 by Zhichao Lu: [Context RCNN] Add an option to enable / disable cropping feature in the post process step in the meta archtecture. -- 295605344 by Zhichao Lu: internal change. -- 294926050 by ronnyvotel: Adding per-keypoint groundtruth weights. These weights are intended to be used as multipliers in a keypoint loss function. Groundtruth keypoint weights are constructed as follows: - Initialize the weight for each keypoint type based on user-specified weights in the input_reader proto - Mask out (i.e. make zero) all keypoint weights that are not visible. -- 294829061 by lzc: Internal change. -- 294566503 by Zhichao Lu: Changed internal CenterNet Model configuration. -- 294346662 by ronnyvotel: Using NaN values in keypoint coordinates that are not visible. -- 294333339 by Zhichao Lu: Change experimetna_distribute_dataset -> experimental_distribute_dataset_from_function -- 293928752 by Zhichao Lu: Internal change -- 293909384 by Zhichao Lu: Add capabilities to train 1024x1024 CenterNet models. -- 293637554 by ronnyvotel: Adding keypoint visibilities to TfExampleDecoder. -- 293501558 by lzc: Internal change. -- 293252851 by Zhichao Lu: Change tf.gfile.GFile to tf.io.gfile.GFile. -- 292730217 by Zhichao Lu: Internal change. -- 292456563 by lzc: Internal changes. -- 292355612 by Zhichao Lu: Use tf.gather and tf.scatter_nd instead of matrix ops. -- 292245265 by rathodv: Internal -- 291989323 by richardmunoz: Refactor out building a DataDecoder from building a tf.data.Dataset. -- 291950147 by Zhichao Lu: Flip bounding boxes in arbitrary shaped tensors. -- 291401052 by huizhongc: Fix multiscale grid anchor generator to allow fully convolutional inference. When exporting model with identity_resizer as image_resizer, there is an incorrect box offset on the detection results. We add the anchor offset to address this problem. -- 291298871 by Zhichao Lu: Py3 compatibility changes. -- 290957957 by Zhichao Lu: Hourglass feature extractor for CenterNet. -- 290564372 by Zhichao Lu: Internal change. -- 290155278 by rathodv: Remove Dataset Explorer. -- 290155153 by Zhichao Lu: Internal change -- 290122054 by Zhichao Lu: Unify the format in the faster_rcnn.proto -- 290116084 by Zhichao Lu: Deprecate tensorflow.contrib. -- 290100672 by Zhichao Lu: Update MobilenetV3 SSD candidates -- 289926392 by Zhichao Lu: Internal change -- 289553440 by Zhichao Lu: [Object Detection API] Fix the comments about the dimension of the rpn_box_encodings from 4-D to 3-D. -- 288994128 by lzc: Internal changes. -- 288942194 by lzc: Internal change. -- 288746124 by Zhichao Lu: Configurable channel mean/std. dev in CenterNet feature extractors. -- 288552509 by rathodv: Internal. -- 288541285 by rathodv: Internal update. -- 288396396 by Zhichao Lu: Make object detection import contrib explicitly -- 288255791 by rathodv: Internal -- 288078600 by Zhichao Lu: Fix model_lib_v2 test -- 287952244 by rathodv: Internal -- 287921774 by Zhichao Lu: internal change -- 287906173 by Zhichao Lu: internal change -- 287889407 by jonathanhuang: PY3 compatibility -- 287889042 by rathodv: Internal -- 287876178 by Zhichao Lu: Internal change. -- 287770490 by Zhichao Lu: Add CenterNet proto and builder -- 287694213 by Zhichao Lu: Support for running multiple steps per tf.function call. -- 287377183 by jonathanhuang: PY3 compatibility -- 287371344 by rathodv: Support loading keypoint labels and ids. -- 287368213 by rathodv: Add protos supporting keypoint evaluation. -- 286673200 by rathodv: dataset_tools PY3 migration -- 286635106 by Zhichao Lu: Update code for upcoming tf.contrib removal -- 286479439 by Zhichao Lu: Internal change -- 286311711 by Zhichao Lu: Skeleton of context model within TFODAPI -- 286005546 by Zhichao Lu: Fix Faster-RCNN training when using keep_aspect_ratio_resizer with pad_to_max_dimension -- 285906400 by derekjchow: Internal change -- 285822795 by Zhichao Lu: Add CenterNet meta arch target assigners. -- 285447238 by Zhichao Lu: Internal changes. -- 285016927 by Zhichao Lu: Make _dummy_computation a tf.function. This fixes breakage caused by cl/284256438 -- 284827274 by Zhichao Lu: Convert to python 3. -- 284645593 by rathodv: Internal change -- 284639893 by rathodv: Add missing documentation for keypoints in eval_util.py. -- 284323712 by Zhichao Lu: Internal changes. -- 284295290 by Zhichao Lu: Updating input config proto and dataset builder to include context fields Updating standard_fields and tf_example_decoder to include context features -- 284226821 by derekjchow: Update exporter. -- 284211030 by Zhichao Lu: API changes in CenterNet informed by the experiments with hourlgass network. -- 284190451 by Zhichao Lu: Add support for CenterNet losses in protos and builders. -- 284093961 by lzc: Internal changes. -- 284028174 by Zhichao Lu: Internal change -- 284014719 by derekjchow: Do not pad top_down feature maps unnecessarily. -- 284005765 by Zhichao Lu: Add new pad_to_multiple_resizer -- 283858233 by Zhichao Lu: Make target assigner work when under tf.function. -- 283836611 by Zhichao Lu: Make config getters more general. -- 283808990 by Zhichao Lu: Internal change -- 283754588 by Zhichao Lu: Internal changes. -- 282460301 by Zhichao Lu: Add ability to restore v2 style checkpoints. -- 281605842 by lzc: Add option to disable loss computation in OD API eval job. -- 280298212 by Zhichao Lu: Add backwards compatible change -- 280237857 by Zhichao Lu: internal change -- PiperOrigin-RevId: 310447280
-
- 15 Jul, 2019 1 commit
-
-
pkulzc authored
257914648 by lzc: Internal changes -- 257525973 by Zhichao Lu: Fixes bug that silently prevents checkpoints from loading when training w/ eager + functions. Also sets up scripts to run training. -- 257296614 by Zhichao Lu: Adding detection_features to model outputs -- 257234565 by Zhichao Lu: Fix wrong order of `classes_with_max_scores` in class-agnostic NMS caused by sorting in partitioned-NMS. -- 257232002 by ronnyvotel: Supporting `filter_nonoverlapping` option in np_box_list_ops.clip_to_window(). -- 257198282 by Zhichao Lu: Adding the focal loss and l1 loss from the Objects as Points paper. -- 257089535 by Zhichao Lu: Create Keras based ssd + resnetv1 + fpn. -- 257087407 by Zhichao Lu: Make object_detection/data_decoders Python3-compatible. -- 257004582 by Zhichao Lu: Updates _decode_raw_data_into_masks_and_boxes to the latest binary masks-to-string encoding fo...
-
- 22 May, 2019 1 commit
-
-
Zhuoran Liu authored
247226201 by ronnyvotel: Updating the visualization tools to accept unique_ids for color coding. -- 247067830 by Zhichao Lu: Add box_encodings_clip_range options for the convolutional box predictor (for TPU compatibility). -- 246888475 by Zhichao Lu: Remove unused _update_eval_steps function. -- 246163259 by lzc: Add a gather op that can handle ignore indices (which are "-1"s in this case). -- 246084944 by Zhichao Lu: Keras based implementation for SSD + MobilenetV2 + FPN. -- 245544227 by rathodv: Add batch_get_targets method to target assigner module to gather any groundtruth tensors based on the results of target assigner. -- 245540854 by rathodv: Update target assigner to return match tensor instead of a match object. -- 245434441 by Zhichao Lu: Add README for tpu_exporters package. -- 245381834 by lzc: Internal change. -- 245298983 by Zhichao Lu: Add conditional_shape_resizer to config_util -- 245134666 by Zhichao Lu: Adds ConditionalShapeResizer to the ImageResizer proto which enables resizing only if input image height or width is is greater or smaller than a certain size. Also enables specification of resize method in resize_to_{max, min}_dimension methods. -- 245093975 by Zhichao Lu: Exporting SavedModel for Object Detection TPU inference. (faster-rcnn) -- 245072421 by Zhichao Lu: Adds a new image resizing method "resize_to_max_dimension" which resizes images only if a dimension is greater than the maximum desired value while maintaining aspect ratio. -- 244946998 by lzc: Internal Changes. -- 244943693 by Zhichao Lu: Add a custom config to mobilenet v2 that makes it more detection friendly. -- 244754158 by derekjchow: Internal change. -- 244699875 by Zhichao Lu: Add check_range=False to box_list_ops.to_normalized_coordinates when training for instance segmentation. This is consistent with other calls when training for object detection. There could be wrongly annotated boxes in the dataset. -- 244507425 by rathodv: Support bfloat16 for ssd models. -- 244399982 by Zhichao Lu: Exporting SavedModel for Object Detection TPU inference. (ssd) -- 244209387 by Zhichao Lu: Internal change. -- 243922296 by rathodv: Change `raw_detection_scores` to contain softmax/sigmoid scores (not logits) for `raw_ detection_boxes`. -- 243883978 by Zhichao Lu: Add a sample fully conv config. -- 243369455 by Zhichao Lu: Fix regularization loss gap in Keras and Slim. -- 243292002 by lzc: Internal changes. -- 243097958 by Zhichao Lu: Exporting SavedModel for Object Detection TPU inference. (ssd model) -- 243007177 by Zhichao Lu: Exporting SavedModel for Object Detection TPU inference. (ssd model) -- 242776550 by Zhichao Lu: Make object detection pre-processing run on GPU. tf.map_fn() uses TensorArrayV3 ops, which have no int32 GPU implementation. Cast to int64, then cast back to int32. -- 242723128 by Zhichao Lu: Using sorted dictionaries for additional heads in non_max_suppression to ensure tensor order -- 242495311 by Zhichao Lu: Update documentation to reflect new TFLite examples repo location -- 242230527 by Zhichao Lu: Fix Dropout bugs for WeightSharedConvolutionalBoxPred. -- 242226573 by Zhichao Lu: Create Keras-based WeightSharedConvolutionalBoxPredictor. -- 241806074 by Zhichao Lu: Add inference in unit tests of TFX OD template. -- 241641498 by lzc: Internal change. -- 241637481 by Zhichao Lu: matmul_crop_and_resize(): Switch to dynamic shaping, so that not all dimensions are required to be known. -- 241429980 by Zhichao Lu: Internal change -- 241167237 by Zhichao Lu: Adds a faster_rcnn_inception_resnet_v2 Keras feature extractor, and updates the model builder to construct it. -- 241088616 by Zhichao Lu: Make it compatible with different dtype, e.g. float32, bfloat16, etc. -- 240897364 by lzc: Use image_np_expanded in object_detection_tutorial notebook. -- 240890393 by Zhichao Lu: Disable multicore inference for OD template as its not yet compatible. -- 240352168 by Zhichao Lu: Make SSDResnetV1FpnFeatureExtractor not protected to allow inheritance. -- 240351470 by lzc: Internal change. -- 239878928 by Zhichao Lu: Defines Keras box predictors for Faster RCNN and RFCN -- 239872103 by Zhichao Lu: Delete duplicated inputs in test. -- 239714273 by Zhichao Lu: Adding scope variable to all class heads -- 239698643 by Zhichao Lu: Create FPN feature extractor for object detection. -- 239696657 by Zhichao Lu: Internal Change. -- 239299404 by Zhichao Lu: Allows the faster rcnn meta-architecture to support Keras subcomponents -- 238502595 by Zhichao Lu: Lay the groundwork for symmetric quantization. -- 238496885 by Zhichao Lu: Add flexible_grid_anchor_generator -- 238138727 by lzc: Remove dead code. _USE_C_SHAPES has been forced True in TensorFlow releases since TensorFlow 1.9 (https://github.com/tensorflow/tensorflow/commit/1d74a69443f741e69f9f52cb6bc2940b4d4ae3b7) -- 238123936 by rathodv: Add num_matched_groundtruth summary to target assigner in SSD. -- 238103345 by ronnyvotel: Raising error if input file pattern does not match any files. Also printing the number of evaluation images for coco metrics. -- 238044081 by Zhichao Lu: Fix docstring to state the correct dimensionality of `class_predictions_with_background`. -- 237920279 by Zhichao Lu: [XLA] Rework debug flags for dumping HLO. The following flags (usually passed via the XLA_FLAGS envvar) are removed: xla_dump_computations_to xla_dump_executions_to xla_dump_ir_to xla_dump_optimized_hlo_proto_to xla_dump_per_pass_hlo_proto_to xla_dump_unoptimized_hlo_proto_to xla_generate_hlo_graph xla_generate_hlo_text_to xla_hlo_dump_as_html xla_hlo_graph_path xla_log_hlo_text The following new flags are added: xla_dump_to xla_dump_hlo_module_re xla_dump_hlo_pass_re xla_dump_hlo_as_text xla_dump_hlo_as_proto xla_dump_hlo_as_dot xla_dump_hlo_as_url xla_dump_hlo_as_html xla_dump_ir xla_dump_hlo_snapshots The default is not to dump anything at all, but as soon as some dumping flag is specified, we enable the following defaults (most of which can be overridden). * dump to stdout (overridden by --xla_dump_to) * dump HLO modules at the very beginning and end of the optimization pipeline * don't dump between any HLO passes (overridden by --xla_dump_hlo_pass_re) * dump all HLO modules (overridden by --xla_dump_hlo_module_re) * dump in textual format (overridden by --xla_dump_hlo_as_{text,proto,dot,url,html}). For example, to dump optimized and unoptimized HLO text and protos to /tmp/foo, pass --xla_dump_to=/tmp/foo --xla_dump_hlo_as_text --xla_dump_hlo_as_proto For details on these flags' meanings, see xla.proto. The intent of this change is to make dumping both simpler to use and more powerful. For example: * Previously there was no way to dump the HLO module during the pass pipeline in HLO text format; the only option was --dump_per_pass_hlo_proto_to, which dumped in proto format. Now this is --xla_dump_pass_re=.* --xla_dump_hlo_as_text. (In fact, the second flag is not necessary in this case, as dumping as text is the default.) * Previously there was no way to dump HLO as a graph before and after compilation; the only option was --xla_generate_hlo_graph, which would dump before/after every pass. Now this is --xla_dump_hlo_as_{dot,url,html} (depending on what format you want the graph in). * Previously, there was no coordination between the filenames written by the various flags, so info about one module might be dumped with various filename prefixes. Now the filenames are consistent and all dumps from a particular module are next to each other. If you only specify some of these flags, we try to figure out what you wanted. For example: * --xla_dump_to implies --xla_dump_hlo_as_text unless you specify some other --xla_dump_as_* flag. * --xla_dump_hlo_as_text or --xla_dump_ir implies dumping to stdout unless you specify a different --xla_dump_to directory. You can explicitly dump to stdout with --xla_dump_to=-. As part of this change, I simplified the debugging code in the HLO passes for dumping HLO modules. Previously, many tests explicitly VLOG'ed the HLO module before, after, and sometimes during the pass. I removed these VLOGs. If you want dumps before/during/after an HLO pass, use --xla_dump_pass_re=<pass_name>. -- 237510043 by lzc: Internal Change. -- 237469515 by Zhichao Lu: Parameterize model_builder.build in inputs.py. -- 237293511 by rathodv: Remove multiclass_scores from tensor_dict in transform_data_fn always. -- 237260333 by ronnyvotel: Updating faster_rcnn_meta_arch to define prediction dictionary fields that are batched. -- PiperOrigin-RevId: 247226201
-