GitLab

* [WIP] Add video GPU decoder

* Expose use_dev_frame to python class and handle it internally

* Fixed invalid argument CUDA error

* Fixed empty and missing frames

* Free remaining frames in the queue

* Added nv12 to yuv420 conversion support for host frames

* Added unit test and cleaned up code

* Use CUDA_HOME inside if

* Undo commented out code

* Add Readme

* Remove output_format and use_device_frame optional arguments from the VideoReader API

* Cleaned up init()

* Fix warnings

* Fix python linter errors

* Fix linter issues in setup.py

* clang-format

* Make reformat private

* Member function naming

* Add comments

* Variable renaming

* Code cleanup

* Make return type of decode() void

* Replace printing errors with throwing runtime_error

* Replaced runtime_error with TORCH_CHECK in demuxer.h

* Use CUDAGuard instead of cudaSetDevice

* Remove printf

* Use Tensor instead of uint8* and remove cuMemAlloc/cuMemFree

* Use TORCH_CHECK instead of runtime_error

* Use TORCHVISION_INCLUDE and TORCHVISION_LIBRARY to pass video codec location

* Include ffmpeg_include_dir

* Remove space

* Removed use of runtime_error

* Update Readme

* Check for bsf.h

* Change struct initialisation style

* Clean-up get_operating_point

* Make variable naming convention uniform

* Move checking for bsf.h around

* Fix linter error
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Upgrade to py37

* Update

* add api usage log for io

* cover VideoReader

* cover c++ APIs

* add api usage log for io

* cover VideoReader

* cover c++ APIs

* add _cpp suffix to c++ APIs

* use new API and change cpp format

* remove _cpp suffix

* adopt new API

* lint

* [Windows] Workaround for loading bundled DLLs

Python-3.8+ adds `add_dll_directory` call, see https://docs.python.org/3/whatsnew/3.8.html#ctypes
Simulate this behaviour on older versions of Python runtime by calling
`LoadLibraryExW` with the appropriate flags

Fixes https://github.com/pytorch/vision/issues/4787

* style: Added typing annotations

* style: Fixed lint

* style: Fixed typing

* chore: Updated mypy.ini

* style: Fixed typing

* chore: Updated mypy.ini

* style: Fixed typing compatibility with jit

* style: Fixed typing

* style: Fixed typing

* style: Fixed missing import

* style: Fixed typing of __iter__

* style: Fixed typing

* style: Fixed lint

* style: Finished typing

* style: ufmt the file

* style: Removed unnecessary typing

* style: Fixed typing of iterator
Co-authored-by: Philip Meier <github.pmeier@posteo.de>
Co-authored-by: Prabhat Roy <prabhatroy@fb.com>

This reverts commit 85b78580.

Description:
- Applied `contiguous` on decoded output tensor in decode_jpeg and decode_png ops
- Updated tests and docs

Related to #4880

* modify processPacket to support fast seek

* add fastSeek to ProcessPacket decoder definition

* add fastseek flag to DecoderParametersStruct

* add fastseek flag to the process packet call

* no default params in C++ implementation

* enable flag in C++ implementation

* make order of parameters more normal

* register new seek with python api

* [somewhat broken] test suite for keyframes using pyav

* revert " changes

* add type annotations to init

* Adding tests

* linter

* Flake doesn't show up :|

* Change from unitest to pytest syntax

* add return type
Co-authored-by: Prabhat Roy <prabhatroy@fb.com>

Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* WIP

* cleaner code

* Add tests

* Add docs

* Assert dtype

* put back check

* Address comments
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* initial commit

* initial commit

* flake8

* last nits
Co-authored-by: Prabhat Roy <prabhatroy@fb.com>

Co-authored-by: Prabhat Roy <prabhatroy@fb.com>

* add ufmt as code formatter

* cleanup

* quote ufmt requirement

* split imports into more groups

* regenerate circleci config

* fix CI

* clarify local testing utils section

* use ufmt pre-commit hook

* split relative imports into local category

* Revert "split relative imports into local category"

This reverts commit f2e224cde2008c56c9347c1f69746d39065cdd51.

* pin black and usort dependencies

* fix local test utils detection

* fix ufmt rev

* add reference utils to local category

* fix usort config

* remove custom categories sorting

* Run pre-commit without fixing flake8

* got a double import in merge
Co-authored-by: Nicolas Hug <nicolashug@fb.com>

* style: Added typing annotations

* Specified types for iter and seek.
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Vasilis Vryniotis <vvryniotis@fb.com>

* Fixed missing audio with video_reader backend

* Added unit test

Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Fixed audio-video synchronisation problem in read_video() when using  as unit

* Addressed review comments

* Added unit test

* Fix io imports and expose methods

* fix fmt
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

Co-authored-by: James Thewlis <james@unitary.ai>

Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

This reverts commit 3bfdb427.

* multithreading allowed in stream codec context

* numThreads is passed as a decoder parameter. At this stage code should be unchanged

* enabling multithreading in videoReader API

* moving defaults to header files

* replace long with int64_t because torchscript

* docstring for Num threads

* Enable codec-related heuristics as defaults

* Update torchvision/csrc/io/decoder/stream.cpp
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Fixing build errors

* minor docs

* Linting

* updating defaults for the C++ function calls to be single threaded

* adding special case for single threaded stuff
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

Summary:
We accept python bytes for video in existing API, such as `_read_video_timestamps_from_memory`.
We should do so to `_read_video_timestamps_from_memory` as well.

Reviewed By: fmassa

Differential Revision: D26216777

fbshipit-source-id: 5d7ecf15c5a65dc5c57f17b343e7c63b4933189e
Co-authored-by: Zhicheng Yan <zyan3@fb.com>

Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Simplify code and remove used vars.

* Simplify expressions and remove used parenthesis.

* Jit fixes.

* Making check more readable.

* fixing styles

Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Update ImageReadMode error messages, add newline at the end of image_read_mode.h, replace define with const in image_read_mode.h, add documentation to ImageReadMode enum

* Update readpng_cpu and readjpeg_cpu error messages

* Update image.py documentation

* Add option to write audio to video file

Summary:
I was trying to use torchvision's `write_video` function and realized there was no option to add in the audio.

Thus, this diff contains the changes necessary such that this is possible. This is my first time trying to contribute to this project, so be as harsh as you need!

Reviewed By: fmassa

Differential Revision: D21480083

fbshipit-source-id: 2e11f2c8728d42f86c94068f75b843793d5a94aa

* Fix typo

* Try fix Windows

* Disable test on Windows
Co-authored-by: Joanna Bitton <jbitton@fb.com>

* Fix spelling mistake: orignal -> original

* Spelling fix: orignal -> original
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Add support of mode and remove channels.

* Replacing integer mode with define constants.