GitLab

* torchscriptable functions for video io (#1653)

Summary:
Pull Request resolved: https://github.com/pytorch/vision/pull/1653



created new torchscriptable video io functions as part of the api: read_video_meta_data_from_memory and read_video_from_memory.

Updated the implementation of some of the internal functions to be torchscriptable.

Reviewed By: stephenyan1231

Differential Revision: D18720474

fbshipit-source-id: 4ee646b66afecd2dc338a71fd8f249f25a3263bc

* BugFix
Co-authored-by: Jon Guerin <54725679+jguerin-fb@users.noreply.github.com>

* Unify video metadata in VideoClips

* Bugfix

* Make tests a bit more robust

* Unify video backend interfaces

* Remove reference cycle

* Make functions private and enable tests on OSX

* Disable test if video_reader backend not available

* Lint

* Fix import after refactoring

* Fix lint

* Handle corrupted video headers in io

* Catch exceptions while decoding partly-corrupted files

* Add more tests

* extend video reader to support fast video probing

* fix c++ lint

* small fix

* allow to accept input video of type torch.Tensor

* add metadata to video dataset classes. bug fix. more robustness

* query video backend within VideoClips class

* Fix tests

* Fix lint

modified code of io.read_video and io.read_video_timestamps to intepret pts values in seconds (#1331)

* modified code of io.read_video and io.read_video_timestamps to interpret pts values in seconds

* changed default value for pts_unit to pts, corrected formatting

* hanndliing both fractions and floats for start_pts and end_pts, added test cases for pts_unit sec

* moved unit conversion logic to _read_from_stream method

* add _backend argument to __init__() of class VideoClips

* minor fix

* minor fix

* Make backend private in VideoClips

* Fix lint

* [video reader] inception commit

* add method save_metadata to class VideoClips in video_utils.py

* add load_metadata() method to VideoClips class

* add Exception to not catch unexpected events such as memory erros, interrupt

* fix bugs in video_plus.py

* [video reader]remove logging. update setup.py

* remove time measurement in test_video_reader.py

* Remove glog and try making ffmpeg finding more robust

* Add ffmpeg to conda build

* Add ffmpeg to conda build [again]

* Make library path finding more robust

* Missing import

* One more missing fix for import

* Py2 compatibility and change package to av to avoid version conflict with ffmpeg

* Fix for python2

* [video reader] support to decode one stream only (e.g. video/audio stream)

* remove argument _precomputed_metadata_filepath

* remove save_metadata method

* add get_metadata method

* expose _precomputed_metadata and frame_rate arguments in video dataset __init__ method

* remove ssize_t

* remove size_t to pass CI check on Windows

* add PyInit__video_reader function to pass CI check on Windows

* minor fix to define PyInit_video_reader symbol

* Make c++ video reader optional

* Temporarily revert changes to test_io

* Revert changes to python files

* Rename files to make it private

* Fix python lint

* Fix C++ lint

* add a functor object EnumClassHash to make Enum class instances usable as key type of std::unordered_map

* fix cpp format check

Following a similar line of inquiry to pytorch/audio#217

* Expose docs for io and ops package

Had do modify the docstrings to use Napoleon NumPy style, because Napoleon Google Style doesn't support multiple return arguments

* Add video section

* Copy classification scripts for video classification

* Initial version of video classification

* add version

* Training of r2plus1d_18 on kinetics work

Gives even slightly better results than expected, with 57.336 top1 clip accuracy. But we count some clips twice in this evaluation

* Cleanups on training script

* Lint

* Minor improvements

* Remove some hacks

* Lint

* Optimize read_video_timestamps for some formats

* Add some tests

* Miscellaneous fixes and improvements

* Guard against videos without video stream

* Fix lint

* Add test for packed b-frames videos

* Fix missing import

Also extend video saving to support different codecs and options. Notably, we can now save with lossless compression

* Properly order videos with B-frames

* seek doesn't seek to pts, but dts

Find a way of overcoming this problem.

Also make logging less verbose

* Add VideoClips and Kinetics dataset

* Lint + add back missing line

* Adds ClipSampler following Bruno comment

* Change name following Bruno's suggestion

* Enable specifying a target framerate

* Fix test_io for new interface

* Add comment mentioning drop_last behavior

* Make compute_clips more robust

* Flake8

* Fix for Python2

* WIP

* WIP

* Add some documentation

* Improve tests and add GC collection

* [WIP] add timestamp getter

* Bugfixes

* Improvements and travis

* Add audio fine-grained alignment

* More doc

* Remove unecessary file

* Remove comment

* Lazy import av

* Remove hard-coded constants for the test

* Return info stats from read

* Fix for Python-2