- 08 Apr, 2021 1 commit
-
-
Nicolas Hug authored
* WIP * clang * docs * extracted out common utils * Use better quantization function and pass tensors as parameters * proper dequantization * Some tests * Dequantization optimization, seems to gain a few ms * clang-format * again * more correct test. Had to remove optimization although it almost works * Also test aligned=True * remove useless part * more docs and comments * Put back optimization with more robust test * Added check for index upper bound * avoid possible overflow * Move common function into common.h * oops * scale=1,zero_point=0 makes more sense * Force batch size of 1 to prevent any indexingbug * format * format again * updated docstring * put back description comment for pre_calc_bilinear_interpolate * revert most changes to docstring as it's taken care of in another PR
-
- 30 Mar, 2021 1 commit
-
-
Nicolas Hug authored
* Add quantized version of nms * Added tests * Compute areas only once * remove calls to dequantize_val * fix return type for empty tensor * flake8 * remove use of scale as it gets cancelled out * simpler int convertion in tests * explicitly set ovr to double * add tests for more values of scale and zero_point * comment about underflow * remove unnecessary accessor * properly convert to float for division * Add comments about underflow * explicitely cast coordinates to float to allow vectorization * clang * clang again * hopefully OK now
-