GitLab

* chore: get PyPy 3.7 wheels using NumPy 1.20

* Add Valgrind suppressions after NumPy update
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>

* Demonstrate issue with weakref constructor overloads

* Fix weakref constructor to convert on being passed a non-weakref object

* Improve on nonlocal-scoped variable in test_weakref

* Keep backwards-compatibility by introducing PYBIND11_OBJECT_CVT_DEFAULT macro

* Simplify test_weakref

* Changing pybind11::str to exclusively hold PyUnicodeObject

* Always call PyNumber_Index when casting from Python to a C++ integral type, also pre-3.8

* Fixed on PyPy

* Simplify use of PyNumber_Index, following @rwgk's idea, and ignore warnings in >=3.8

* Reproduce mismatch between pre-3.8 and post-3.8 behavior on __index__ throwing TypeError

* Fix tests on 3.6 <= Python < 3.8

* No, I don't have an uninitialized variable

* Fix use of __index__ on Python 2

* Make types in test_int_convert more ~boring~ descriptive

* Force the builtin module key to be the correct type.

Previously it was always going to be a std::string which converted into
unicode. Python 2 appears to want module keys to be normal str types, so
this was breaking code that expected plain string types in the
builtins.keys() data structure

* Add a simple unit test to ensure all built-in keys are str

* Update the unit test so it will also run on pypy

* Run pre-commit.
Co-authored-by: Jesse Clemens <jesse.clemens@sony.com>

* style: avoid using unintialized variables

Tested with cmake --warn-unintialized -S . -B build

* refactor: use function for possibly uninit vars

* CI: Intel icc/icpc via oneAPI

Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.

CI: Skip Interpreter Tests for Intel

Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  tests/test_embed/CMakeLists.txt:17 (find_package)
```

CI: libc6-dev from GCC for ICC

CI: Run bare metal for oneAPI

CI: Ubuntu 18.04 for oneAPI

CI: Intel +Catch -Eigen

CI: CMake from Apt (ICC tests)

CI: Replace Intel Py with GCC Py

CI: Intel w/o GCC's Eigen

CI: ICC with verbose make

[Debug] Find core dump

tests: use arg{} instead of arg() for Intel

tests: adding a few more missing {}

fix: sync with @tobiasleibner's branch

fix: try ubuntu 20-04

fix: drop exit 1

docs: Apply suggestions from code review
Co-authored-by: Tobias Leibner <tobias.leibner@googlemail.com>

Workaround for ICC enable_if issues

Another workaround for ICC's enable_if issues

fix error in previous commit

Disable one test for the Intel compiler in C++17 mode

Add back one instance of py::arg().noconvert()

Add NOLINT to fix clang-tidy check

Work around for ICC internal error in PYBIND11_EXPAND_SIDE_EFFECTS in C++17 mode

CI: Intel ICC with C++17

docs: pybind11/numpy.h does not require numpy at build time. (#2720)

This is nice enough to be mentioned explicitly in the docs.

docs: Update warning about Python 3.9.0 UB, now that 3.9.1 has been released (#2719)

Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)

* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.

Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.

After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.

* Add tests/examples for std::reference_wrapper<const T>

* Add tests which use mutable/immutable variants

This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.

In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>

Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.

* Add/finish tests that distinguish const& from &

Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.

* Add passing a const to non-const method.

* Demonstrate non-const conversion of reference_wrapper in tests.

Apply formatting presubmit check.

* Fix build errors from presubmit checks.

* Try and fix a few more CI errors

* More CI fixes.

* More CI fixups.

* Try and get PyPy to work.

* Additional minor fixups. Getting close to CI green.

* More ci fixes?

* fix clang-tidy warnings from presubmit

* fix more clang-tidy warnings

* minor comment and consistency cleanups

* PyDECREF -> Py_DECREF

* copy/move constructors

* Resolve codereview comments

* more review comment fixes

* review comments: remove spurious &

* Make the test fail even when the static_assert is commented out.

This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths.  In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.

* apply presubmit formatting

* Revert inclusion of test_freezable_type_caster

There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.

* Add a test that validates const references propagation.

This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.

* mend

* Review comments based changes.

1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.

* formatted files again.

* Move const_ref_caster test to builtin_casters

* Review comments: use cast_op and adjust some comments.

* Simplify ConstRefCasted test

I like this version better as it moves the assertion that matters
back into python.

ci: drop pypy2 linux, PGI 20.7, add Python 10 dev (#2724)

* ci: drop pypy2 linux, add Python 10 dev

* ci: fix mistake

* ci: commented-out PGI 20.11, drop 20.7

fix: regression with installed pybind11 overriding local one (#2716)

* fix: regression with installed pybind11 overriding discovered one

Closes #2709

* docs: wording incorrect

style: remove redundant instance->owned = true (#2723)

which was just before set to True in instance->allocate_layout()

fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)

Make args_are_all_* ICC workarounds unconditional

Disable test_aligned on Intel ICC

Fix test_aligned on Intel ICC

Skip test_python_alreadyset_in_destructor on Intel ICC

Fix test_aligned again

ICC CI: Downgrade pytest

pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.

* refactor: simpler Intel workaround, suggested by @laramiel

* fix: try version with impl to see if it is easier to compile

* docs: update README for ICC
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

fix: only allow integer type_caster to call __int__ method when conversion is allowed; always call __index__ (#2698)

* Only allow integer type_caster to call __int__ or __index__ method when conversion is allowed

* Remove tests for __index__ as this seems to only be used to convert to int in 3.8+

* Take both `int` and `long` types into account for Python 2

* Add test_numpy_int_convert to assert tests currently fail, even though np.intc has an __index__ method

* Also consider __index__ as noconvert to a C++ integer

* New-style classes for Python 2.7; sigh

* Add some tests on types with custom __index__ method

* Ignore some tests in Python <3.8

* Update comment about conversion from np.float32 to C++ int

* Workaround difference between CPython and PyPy's different PyIndex_Check (unnoticed because we currently don't have PyPy >= 3.8)

* Avoid ICC segfault with py::arg()

* CI: Intel icc/icpc via oneAPI

Add testing for Intel icc/icpc via the oneAPI images.
Intel oneAPI is in a late beta stage, currently shipping
oneAPI beta09 with ICC 20.2.

* CI: Skip Interpreter Tests for Intel

Cannot find how to add this, neiter the package `libc6-dev` nor
`intel-oneapi-mkl-devel` help when installed to solve this:
```
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - not found
CMake Error at /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /__t/cmake/3.18.4/x64/cmake-3.18.4-Linux-x86_64/share/cmake-3.18/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  tests/test_embed/CMakeLists.txt:17 (find_package)
```

* CI: libc6-dev from GCC for ICC

* CI: Run bare metal for oneAPI

* CI: Ubuntu 18.04 for oneAPI

* CI: Intel +Catch -Eigen

* CI: CMake from Apt (ICC tests)

* CI: Replace Intel Py with GCC Py

* CI: Intel w/o GCC's Eigen

* CI: ICC with verbose make

* [Debug] Find core dump

* tests: use arg{} instead of arg() for Intel

* tests: adding a few more missing {}

* fix: sync with @tobiasleibner's branch

* fix: try ubuntu 20-04

* fix: drop exit 1

* style: clang tidy fix

* style: fix missing NOLINT

* ICC: Update Compiler Name

Changed upstream with the last oneAPI release.

* ICC CI: Downgrade pytest

pytest 6 does not capture the `discard_as_unraisable` stderr and
just writes a warning with its content instead.

* Use new test pinning requirements.txt

* tests: add notes about intel, cleanup
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

* Adding a valgrind build on debug Python 3.9
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>

* Add Valgrind suppression files

- Introduce suppression file, populate it with a first suppression taken from CPython, and fix one leak in the tests
- Suppress leak in NumPy
- More clean tests!
- Tests with names a-e passing (except for test_buffer)
- Suppress multiprocessing errors
- Merge multiprocessing suppressions into other suppression files
- Numpy seems to be spelled with a big P
- Append single entry from valgrind-misc.supp to valgrind-python.supp, and make clear valgrind-python.supp is only CPython
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>

* Enable test_virtual_functions with a workaround

* Add a memcheck cmake target

- Add a memcheck cmake target
- Reformat cmake
- Appease the formatting overlords - they are angry
- Format CMake valgrind target decently

* Update CI config to new action versions

* fix: separate memcheck from pytest

* ci: cleanup

* Merge Valgrind and other deadsnakes builds
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

Plug leaking function_records in cpp_function initialization in case of exceptions (found by Valgrind in #2746) (#2756)

* Plug leaking function_record objects when exceptions are thrown

* Plug leak of strdup'ed strings in function_record

* Some extra comments about the function_record ownership dance

* Clean up the function_record better, in case of exceptions

* Demonstrate some extra function_record leaks

* Change  DeleteStrings template argument to free_strings runtime argument in destruct(function_record *)

* Zero-state unique_function_record deleter object

* Clarify rvalue reference to unique_ptr parameter in initialize_generic

* Use push_back with const char * instead of emplace_back

Ignore deprecation warnings about old-style __init__/__setstate__ constructors in the tests (originally done in #2746) (#2759)

* Ignore old-style __init__ warnings

* Simplify ignoreOldStyleInitWarnings with py::exec

* Only wrap single class_::defs to ignore DeprecationWarnings about old-style __init__

* Fix leak in the test_copy_move::test_move_fallback

* Fix leaking PyMethodDef in test_class::test_implicit_conversion_life_support

* Plumb leak in test_buffer, occuring when a mutable buffer is requested for a read-only object, and enable test_buffer.py

* Fix weird return_value_policy::reference in test_stl_binders, and enable those tests

* Cleanup nodelete holder objects in test_smart_ptr, and enable those tests

* enum: Add Enum.value property

* simplify

* address review

* Do not set docstring for function when it's empty

* No need to check pointer for `free`

* Use ternary operator to conditionally set `ml_doc`

* Update pytest to 6.2.1 in tests/requirements.txt

* Pin pytest to last supported version for 3.5

* Suppress PytestUnraisableExceptionWarning and use sys.__unraisablehook__ instead of sys.unraisablehook

* Fix filterwarnings mark on old pytest and old Python versions

* Cleanup ignore_pytest_unraisable_warning decorator

* minor cleanup: fixing or silencing flake8 errors

* ci: lock CMake to non-Universal version

* Update .github/workflows/ci.yml
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

fix: also throw in the move-constructor added by the PYBIND11_OBJECT macro, after the argument has been moved-out (if necessary) (#2701)

Adjusting `type_caster<std::reference_wrapper<T>>` to support const/non-const propagation in `cast_op`. (#2705)


* Allow type_caster of std::reference_wrapper<T> to be the same as a native reference.

Before, both std::reference_wrapper<T> and std::reference_wrapper<const T> would
invoke cast_op<type>. This doesn't allow the type_caster<> specialization for T
to distinguish reference_wrapper types from value types.

After, the type_caster<> specialization invokes cast_op<type&>, which allows
reference_wrapper to behave in the same way as a native reference type.

* Add tests/examples for std::reference_wrapper<const T>

* Add tests which use mutable/immutable variants

This test is a chimera; it blends the pybind11 casters with a custom
pytype implementation that supports immutable and mutable calls.

In order to detect the immutable/mutable state, the cast_op needs
to propagate it, even through e.g. std::reference<const T>

Note: This is still a work in progress; some things are crashing,
which likely means that I have a refcounting bug or something else
missing.

* Add/finish tests that distinguish const& from &

Fixes the bugs in my custom python type implementation,
demonstrate test that requires const& and reference_wrapper<const T>
being treated differently from Non-const.

* Add passing a const to non-const method.

* Demonstrate non-const conversion of reference_wrapper in tests.

Apply formatting presubmit check.

* Fix build errors from presubmit checks.

* Try and fix a few more CI errors

* More CI fixes.

* More CI fixups.

* Try and get PyPy to work.

* Additional minor fixups. Getting close to CI green.

* More ci fixes?

* fix clang-tidy warnings from presubmit

* fix more clang-tidy warnings

* minor comment and consistency cleanups

* PyDECREF -> Py_DECREF

* copy/move constructors

* Resolve codereview comments

* more review comment fixes

* review comments: remove spurious &

* Make the test fail even when the static_assert is commented out.

This expands the test_freezable_type_caster a bit by:
1/ adding accessors .is_immutable and .addr to compare identity
from python.
2/ Changing the default cast_op of the type_caster<> specialization
to return a non-const value. In normal codepaths this is a reasonable
default.
3/ adding roundtrip variants to exercise the by reference, by pointer
and by reference_wrapper in all call paths.  In conjunction with 2/, this
demonstrates the failure case of the existing std::reference_wrpper conversion,
which now loses const in a similar way that happens when using the default cast_op_type<>.

* apply presubmit formatting

* Revert inclusion of test_freezable_type_caster

There's some concern that this test is a bit unwieldly because of the use
of the raw <Python.h> functions. Removing for now.

* Add a test that validates const references propagation.

This test verifies that cast_op may be used to correctly detect
const reference types when used with std::reference_wrapper.

* mend

* Review comments based changes.

1. std::add_lvalue_reference<type> -> type&
2. Simplify the test a little more; we're never returning the ConstRefCaster
type so the class_ definition can be removed.

* formatted files again.

* Move const_ref_caster test to builtin_casters

* Review comments: use cast_op and adjust some comments.

* Simplify ConstRefCasted test

I like this version better as it moves the assertion that matters
back into python.

* Added test_thread testing for ostream_redirect segfault recreation

* fix: scoped_ostream_redirect str created outside gil

* Moved threading tests into test_iostream. Cleaned up some formatting. Deleted test_thread.{cpp,py}

* CI: few formatting fixes

* CI: yet another formatting fix

* CI: more formatting fixes. Removed unecessary comment

* Ignore 'warning C4702: unreachable code' in MSVC 2015
Co-authored-by: Nick Bridge <nick.bridge.chess@gmail.com>
Co-authored-by: Nick Bridge <nbridge@jumptrading.com>
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>

* Add argument names to enum_ methods

* Add test_enum::test_docstring_signatures

* style: clang-tidy: modernize-use-using

* style: more clang-tidy checking
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

* Adding missing virtual destructors, to silence clang -Wnon-virtual-dtor warnings.

Tested with clang version 9.0.1-12 under an Ubuntu-like OS.
Originally discovered in the Google-internal environment.

* adding -Wnon-virtual-dtor for GNU|Intel|Clang

* Add __builtins__ to globals argument of `py::exec` and `py::eval` if not present

* Refactor into inline ensure_builtins_in_globals function

* demo kwarg with none(false)

* Reorder and extend tests for arg::none(false) in test_methods_and_attributes.py::test_accepts_none

* Fix arg::none() for keyword arguments

* Add changelog note

* Fix names of no_none_kw test functions
Co-authored-by: Yannick Jadoul <yannick.jadoul@belgacom.net>

* tests/CMakeLists: Robustify against add'l versions of Boost

* refactor: simpler impl
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

* Demonstrate test_factory_constructors.py failure without functional changes from #2335

* Revert "Demonstrate test_factory_constructors.py failure without functional changes from #2335"

This reverts commit ca33a8021fc2a3617c3356b188796528f4594419.

* Fix test crash where registered Python type gets garbage collected

* Clean up some more internal structures when class objects go out of scope

* Reduce length of std::erase_if-in-C++20 comment

* Clean up code for cleaning up type internals

* Move cleaning up of type info in internals to tp_dealloc on pybind11_metaclass

* style: use Black everywhere

* style: minor touchup from review

* feat: basic typing support

* docs: mention syncing as suggested by @rwgk

* docs: update changelog

* docs: copy of warning in limitations

* Throw error_already_set in py::len on failing PyObject_Length

* Fix tests to mach error message on PyPy

* fix: Intel 18+ fully supported

* fix: Intel compiler workaround no longer needed

Followup on #94 now that Intel 18+ is required.

* Deprecated public constructors of module

* Turn documentation comment of module_::add_object into valid doxygen documentation

* Move definition of PYBIND11_DETAIL_MODULE_STATIC_DEF and PYBIND11_DETAIL_MODULE_CREATE macros up

* Move detail::create_top_level_module to module_::create_extension_module, and unify Python 2 and 3 signature again

* Throw error_already_set if module creation fails in module_::create_extension_module

* Mention module_::create_extension_module in deprecation warning message of module_::module_

* Check scope's __dict__ instead of using hasattr when registering classes and exceptions, to allow registering the same name in a derived class scope

* Extend test_base_and_derived_nested_scope test

* Add tests on error being thrown registering duplicate classes

* Circumvent bug with combination of test_class.py::test_register_duplicate_class and test_factory_constructors.py::test_init_factory_alias