GitLab

Co-authored-by: Nikita Shulga <2453524+malfet@users.noreply.github.com>

Co-authored-by: Philip Meier <github.pmeier@posteo.de>

Co-authored-by: Philip Meier <github.pmeier@posteo.de>

Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Revert "[FBcode->GH] remove unused requests functionality (#5014)"

This reverts commit 33123bee.

* Revert "replace requests with urllib (#4973)"

This reverts commit 8d25de7b.

* add requests as hard dependency

* install library stubs in CI

* fix syntax

* add requests to conda dependencies

* fix mypy CI

Co-authored-by: Philip Meier <github.pmeier@posteo.de>

* Move TorchVision conda package to use pytorch-mutex

This is follow up after https://github.com/pytorch/builder/pull/823

 that gets rids of `feature` and migrate it to `run_constrained`

* Update packaging/torchvision/meta.yaml
Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com>

* Update packaging/torchvision/meta.yaml
Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com>
Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com>

`import torchvision` fails if numpy is not installed, so it should have
an explicit dependency on it (which is already the case for wheel
package)

* Unpinned the libjpeg version and fixed jpeg_mem_dest's size type Windows BC issue

* Temporarily get back pinned jpeg lib in order to run tests on CircleCI

* Revert "Temporarily get back pinned jpeg lib in order to run tests on CircleCI"

This reverts commit ab18a354150f750256b61911e54aa48621ef24cd.

* Used using instead of typedef and Fixed comment typo
Co-authored-by: Zhongkai Zhu <zhongkai@fb.com>
Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com>

* Fix inconsistent version pinning across yaml files.

* Pin pillow to >=5.3.0

* Pin pillow>=5.3.0 everywhere

* packaging: Remove pin for jpeg, numpy

These may no longer be necessary due to the default anaconda channel
having the necessary packages now.
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Update packaging/torchvision/meta.yaml
Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* trying stuff

* Put flags back?

* remove second one just to see

* It worked but it's because it's not built by CI. Trying another one

* Try using TORCH_CUDA_ARCH_LIST instead

* oops

* Add new env variable to build/script_env

* set TORCH_CUDA_ARCH_LIST for the rest of the CUDA versions

* don't pass NVCC_FLAGS venv to conda, let's see if it works
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

(cherry picked from commit 2f40a483d73018ae6e1488a484c5927f2b309969)
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Add PyTorch FFmpeg to wheel and conda distributions

* Try to install wget from conda

* Add yq flag on Mac

* Correct copy instructions

* Use cURL on Windows

* Call bzip2 directly due to msys2/MSYS2-packages#1548

* Copy ffmpeg binaries to system-wide directories

* Try to use std:c++17 on Windows

* Try to define ssize_t on Windows

* Use C++14

* Declare AVRational structs explicitly

* Initialize AVRational explicitly

* Replace macro to prevent errors on Windows

* Replace AV_TIME_BASE_Q

* Add library paths for video extension

* Force ffmpeg from pytorch channels?

* Fix clang style warnings

* Update CONDA_CHANNEL_FLAGS

* Fix clang style issues

* Update unittest

* Use FFmpeg 4.2

* Install correct version on Mac

* Pin av version to 8.0.0

* Fix string formatting issue

* Fix pip pinning

* Try with 8.0.1

* Use av 8.0.2

* Remove trailling whitespaces

* Disable test_io_opt.py

* Disable test_datasets_video_utils
Co-authored-by: Francisco Massa <fvsmassa@gmail.com>

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04

* Enable JPEG support using libjpeg directly

* Install libjpeg-turbo8 on Travis

* Fix styling issues

* Do not append to paths if library found in standard library locations

* Add macro flag

* Use custom error handler

* Use 3?

* Use short-lived buffer

* Return TRUE instead of true

* Assert RuntimeError

* Use .jpg extension

* Remove conda-forge

* Use data_ptr instead of accessor

* Use assertTrue for jpeg verification

* Remove unnecessary memcpy

* Debug test on Windows

* Remove PIL from jpeg verification
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04

* Remove JPEG-related code

* Remove installation references to turbojpeg

* Remove further references to turbojpeg

* Fix c++ style issues

* Fix c++ style issues

* Fix libpng-config include flag parsing

* Remove conda-forge

* Remove include dirs from main extension

* Do not pass extra include and library paths to main torchvision extension

* Add libpng to environment.yml

* Remove inexistent imports

* Add instructions regarding environment variables to README

* Fix image extension tests

* Add libpng to environment + test fixes

* Minor improvements

* Remove unused Py2 code

* Add stub comments to prevent deletion while merging

* Reintroduce files in order to prevent deletion during merge

* Remove unwanted merge sections

* Restore libpng conda installation on wheel distributions

* Restore comment

* Fix libpng discovery on Windows

* Fix PEP8 style issues

* Add linking flag on Windows

* Remove parenthesis

* Restore libpng during runtime
Co-authored-by: Edgar Andrés Margffoy Tuay <andfoy@gmail.com>
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

This reverts commit c1a99b7b.

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04

* Remove JPEG-related code

* Remove installation references to turbojpeg

* Remove further references to turbojpeg

* Fix c++ style issues

* Fix c++ style issues

* Fix libpng-config include flag parsing

* Remove conda-forge

* Remove include dirs from main extension

* Do not pass extra include and library paths to main torchvision extension

* Add libpng to environment.yml

* Remove inexistent imports

* Add instructions regarding environment variables to README
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

This reverts commit 766721b1.

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Add PNG reading capabilities

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Add PNG reading capabilities

* Add libpng requirement into conda recipe

* Try to install libjpeg-turbo

* Remove newline

* Add image extension to compilation instructions

* Include png functions as part of the main library

* Update CMakeLists

* Detect if building on conda-build

* Debug

* More debug messages

* Print globbed libreries

* Print globbed libreries

* Point to correct PNG path

* Remove libJPEG preventively

* Debug extension loading

* Link libpng explicitly

* Link with PNG

* Install libpng on conda-based wheel distributions

* Add -y flag

* Add -y flag to yum

* Locate LibPNG on windows conda

* Remove empty else

* Copy libpng16.so

* Copy dylib on Mac

* Improve check on Windows

* Try to install ninja using conda on windows

* Use libpng on Windows

* Package lib on windows wheel

* Point library to the correct place

* Include binaries as part of wheel

* Copy libpng.so on linux

* Look for png.h on Windows when using conda-build

* Do not skip png tests on Mac/Win

* Restore libjpeg-turbo

* Install jpeg-turbo on wheel distributions

* Install libjpeg-turbo from conda-forge on wheel distributions

* Do not pull av on conda-build

* Add pillow disclaimer

* Vendors libjpeg-turbo 2.0.4

* Merge JPEG work

* Remove submodules

* Regenerate circle config

* Fix style issues

* Fix C++ style issues

* More style corrections

* Add JPEG-turbo to linking libraries

* More style corrections

* More style corrections

* More style corrections

* Install libjpeg-turbo-devel

* Install libturbo-jpeg on typing pipeline

* Update Circle template

* Windows and Unix turbojpeg have the same linking name

* Install turbojpeg-devel instead of libjpeg-turbo

* Copy TurboJPEG binaries to wheel

* Move test image

* Move back test image

* Update JPEG test path

* Remove dot from extension

* Move image functions to extension

* Use stdout arg in subprocess

* Disable image extension if libpng or turbojpeg are not found

* Append libpng stdout

* Prevent list appending on lists

* Minor path correction

* Minor error correction

* Add linking flags

* Style issues correction

* Address minor review corrections

* Refactor library search

* Restore access index

* Fix JPEG tests

* Update libpng version in Travis

* Add -y flag

* Remove dot

* Update libpng using apt

* Check libpng version

* Change libturbojpeg binary

* Update import

* Change call

* Restore av in conda recipe

* Minor error correction

* Remove unused comment in travis.yml

* Update README

* Fix missing links

* Remove fixes for 16.04
Co-authored-by: Ryad ZENINE <r.zenine@gmail.com>

* remove six from python code

* remove six from setup.py

* remove six from tests

* remove six from references

* remove six from packaging

* revert str to torch._six._string_classes

* revert str to torch._six._string_classes

* ci: Add verbosity to pytest results, store in ci

Makes the pytest runs for building conda packages more verbose and
stores the results for viewing inside of CircleCI
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* test: Skip inception v3 in test_quantized_models

Was causing timeouts on circleci due to long run time, re-enable when
tests can be brought to a reasonable time again.
Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Temporary fix for CI

* Missing install

* [video reader] inception commit

* add method save_metadata to class VideoClips in video_utils.py

* add load_metadata() method to VideoClips class

* add Exception to not catch unexpected events such as memory erros, interrupt

* fix bugs in video_plus.py

* [video reader]remove logging. update setup.py

* remove time measurement in test_video_reader.py

* Remove glog and try making ffmpeg finding more robust

* Add ffmpeg to conda build

* Add ffmpeg to conda build [again]

* Make library path finding more robust

* Missing import

* One more missing fix for import

* Py2 compatibility and change package to av to avoid version conflict with ffmpeg

* Fix for python2

* [video reader] support to decode one stream only (e.g. video/audio stream)

* remove argument _precomputed_metadata_filepath

* remove save_metadata method

* add get_metadata method

* expose _precomputed_metadata and frame_rate arguments in video dataset __init__ method

* remove ssize_t

* remove size_t to pass CI check on Windows

* add PyInit__video_reader function to pass CI check on Windows

* minor fix to define PyInit_video_reader symbol

* Make c++ video reader optional

* Temporarily revert changes to test_io

* Revert changes to python files

* Rename files to make it private

* Fix python lint

* Fix C++ lint

* add a functor object EnumClassHash to make Enum class instances usable as key type of std::unordered_map

* fix cpp format check

* make densenet scriptable

* make py2 compat

* use torch List polyfill

* fix unpacking for checkpointing

* fewer changes to _Denseblock

* improve error message

* print traceback

* add typing dependency

* add typing dependency to travis too

* Make loading old checkpoints work

* Windows build scripts

Add testing yaml

* Change urls

* Test MS agent

* Switch base image

* Use fb links

* Actually upload wheels (please port to master)
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Propagate more environment variables.
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

* Put macos binaries in the right place
Signed-off-by: Edward Z. Yang <ezyang@fb.com>

Following a similar line of inquiry to pytorch/audio#217