GitLab

* Eliminate duplicate TLS keys for loader_life_support stack

This revises the existing fix for
https://github.com/pybind/pybind11/issues/2765 in
https://github.com/pybind/pybind11/pull/3237 to reduce the amount of
TLS storage used.

The shared TLS key is stored in two different ways, depending on
`PYBIND11_INTERNALS_VERSION`.  If `PYBIND11_INTERNALS_VERSION ==
4` (as is currently set), the TLS key is stored in the
`internal::shared_data` map to avoid breaking ABI compatibility.  If
`PYBIND11_INTERNALS_VERSION > 4`, the TLS key is stored directly in
the `internals` struct.

* Fix test_pytypes.py::test_issue2361 failure on PyPy3.7

* Add github actions tests for unstable ABI

updates:
- [github.com/asottile/pyupgrade: v2.25.0 → v2.26.0](https://github.com/asottile/pyupgrade/compare/v2.25.0...v2.26.0

)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Restoring `const` removed from pytypes.h in PR #3254, adding tests reflective of user code that breaks when those `const` are removed.

* clang-tidy NOLINTs (and one collateral fix).

* Inserting PYBIND11_CONST_FOR_STRICT_PLATFORMS

* Trying `defined(__APPLE__)`

* Trying again: `auto it` for strict platforms.

* Adding NOLINTNEXTLINE(bugprone-macro-parentheses), expanding comments.

* Labeling all changes with `PR #3263`, for easy reference, and to make it easy to undo these changes if we decide to do so in the future.

Thanks Aaron for jumping in fixing this!

* Fix thread safety for pybind11 loader_life_support

Fixes issue: https://github.com/pybind/pybind11/issues/2765

This converts the vector of PyObjects to either a single void* or
a per-thread void* depending on the WITH_THREAD define.

The new field is used by each thread to construct a stack
of loader_life_support frames that can extend the life of python
objects.

The pointer is updated when the loader_life_support object is allocated
(which happens before a call) as well as on release.

Each loader_life_support maintains a set of PyObject references
that need to be lifetime extended; this is done by storing them
in a c++ std::unordered_set and clearing the references when the
method completes.

* Also update the internals version as the internal struct is no longer compatible

* Add test demonstrating threading works correctly.

It may be appropriate to run this under msan/tsan/etc.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update test to use lifetime-extended references rather than
std::string_view, as that's a C++ 17 feature.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make loader_life_support members private

* Update version to dev2

* Update test to use python threading rather than concurrent.futures

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unnecessary env in test

* Remove unnecessary pytest in test

* Use native C++ thread_local in place of python per-thread data structures to retain compatability

* clang-format test_thread.cpp

* Add a note about debugging the py::cast() error

* thread_test.py now propagates exceptions on join() calls.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* remove unused sys / merge

* Update include order in test_thread.cpp

* Remove spurious whitespace

* Update comment / whitespace.

* Address review comments

* lint cleanup

* Fix test IntStruct constructor.

* Add explicit to constructor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* Removing NOLINT pointed out by Aaron.

* Removing another NOLINT.

* Enable clang-tidy readability-const-return

* PyTest functional

* Fix regression

* Fix actual regression

* Remove one more NOLINT

* Update comment

Fixes #3252

* Set __file__ constant when using eval_file

* Use const ref

* Use a move instead

* Revert

* Improve test

* Guard test with Python version

* Fix tests

* Dont support Python2 API

* Drop Python2 eval __file__ support

* Hack

* Semisupport Python2

* Take2

* Remove Python2 support

* Adding google-explicit-constructor to .clang-tidy

* clang-tidy explicit attr.h (all automatic)

* clang-tidy explicit cast.h (all automatic)

* clang-tidy detail/init.h (1 NOLINT)

* clang-tidy detail/type_caster_base.h (2 NOLINT)

* clang-tidy pybind11.h (7 NOLINT)

* clang-tidy detail/common.h (3 NOLINT)

* clang-tidy detail/descr.h (2 NOLINT)

* clang-tidy pytypes.h (23 NOLINT, only 1 explicit)

* clang-tidy eigen.h (7 NOLINT, 0 explicit)

* Adding 2 explicit in functional.h

* Adding 4 explicit in iostream.h

* clang-tidy numpy.h (1 NOLINT, 1 explicit)

* clang-tidy embed.h (0 NOLINT, 1 explicit)

* clang-tidy tests/local_bindings.h (0 NOLINT, 4 explicit)

* clang-tidy tests/pybind11_cross_module_tests.cpp (0 NOLINT, 1 explicit)

* clang-tidy tests/pybind11_tests.h (0 NOLINT, 2 explicit)

* clang-tidy tests/test_buffers.cpp (0 NOLINT, 2 explicit)

* clang-tidy tests/test_builtin_casters.cpp (0 NOLINT, 4 explicit)

* clang-tidy tests/test_class.cpp (0 NOLINT, 6 explicit)

* clang-tidy tests/test_copy_move.cpp (0 NOLINT, 7 explicit)

* clang-tidy tests/test_embed/external_module.cpp (0 NOLINT, 1 explicit)

* clang-tidy tests/test_embed/test_interpreter.cpp (0 NOLINT, 1 explicit)

* clang-tidy tests/object.h (0 NOLINT, 2 explicit)

* clang-tidy batch of fully automatic fixes.

* Workaround for MSVC 19.16.27045.0 C++17 Python 2 C++ syntax error.

* Add a missing noexcept to pytypes constructor.

Adds a few missing noexcept to PyType constructors for perf reasons.

* Revert exception ctor

Minor follow-on to PR #1334 (Fix enum value's __int__ returning non-int when underlying type is bool or of char type) (#3232)

* Minor tweaks.

* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.

* More-to-the-point test for Python 3.

* Minor tweaks.

* Restoring tests/pybind11_tests.h version from master, removing just the comment and empty line that was added in PR #3087; those were made obsolete by the pragma cleanup that concluded with PR #3186.

* [ci skip] Restoring tests/test_enum.py from master.

* fix: memory leak in cpp_function (#3228)

* add a test case to check objects are deconstructed in cpp_function

* update the test case about cpp_function

* fix the test case about cpp_function: remove "noexcept"

* Actually calling func. CHECK(stat.alive() == 2); Manually verified that the new tests fails without the change in pybind11.h

* Moving new test to test_callbacks.cpp,py, with small enhancements.

* Removing new test from test_interpreter.cpp (after it was moved to test_callbacks.cpp,py). This restores test_interpreter.cpp to the current state on master.

* Using py::detail::silence_unused_warnings(py_func); to make the intent clear.
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>

updates:
- [github.com/asottile/pyupgrade: v2.24.0 → v2.25.0](https://github.com/asottile/pyupgrade/compare/v2.24.0...v2.25.0)
- [github.com/psf/black: 21.7b0 → 21.8b0](https://github.com/psf/black/compare/21.7b0...21.8b0

)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Adding `ssize_t_cast` to support passing `size_t` or `ssize_t` values where `ssize_t` is needed. (#3219)

* Trivial change to avoid (ssize_t) cast.

* Demo for safe_ssize_t idea.

* Removing safe_ssize_t.cpp (proof-of-concept code) to not upset the GHA Format workflow.

* Completing changes in pytypes.h

* New ssize_t_cast (better replacement for safe_ssize_t).

* clang-format-diff (no manual changes).

* bytes_ssize_t -Wnarrowing reproducer (see PR #2692).

* Backing out tuple(), list() ssize_t support, for compatibility with older compilers (to resolve link failures).

* Bug fix: missing `py::` for `py::ssize_t`

* Restoring tuple(), list() ssize_t support, but passing `size` by value, for compatibility with older compilers (to resolve link failures).

* Full test coverage of all functions with modified signatures.

* Use equivalent_integer for enum's Scalar decision

* Add test for char underlying enum

* Support translating bool type in enum's Scalar

* Add test for bool underlying enum

* Fix comment in test

* Switch from `PYBIND11_CPP20` macro to `PYBIND11_HAS_U8STRING`

* Refine tests
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>

* Call PySys_SetArgv when initializing interpreter.

* Document argc/argv parameters in initialize_interpreter.

* Remove manual memory management from set_interpreter_argv in favor of smart pointers.

* Use size_t for indexers in set_interpreter_argv.

* Minimize macros for flow control in set_interpreter_argv.

* Fix 'unused variable' warning on Py2

* whitespace

* Define wide_char_arg_deleter outside set_interpreter_argv.

* Do sys.path workaround in C++ rather than eval.

* Factor out wchar conversion to a separate function.

* Restore widened_argv variable declaration.

* Fix undeclared widened_arg variable on some paths.

* Use delete[] to match new wchar_t[].

* Fix compiler errors

* Use PY_VERSION_HEX for a cleaner CVE-2008-5983 mode check.

* Fix typo

* Use explicit type for deleter so delete[] works cross-compiler.

* Always use PySys_SetArgvEx because pybind11 doesn't support pythons that don't include it.

* Remove pointless ternary operator.

* Use unique_ptr.reset instead of a second initialization.

* Rename add_program_dir_to_path parameter to clarify intent.

* Add defined() check before evaluating HAVE_BROKEN_MBSTOWCS.

* Apply clang-tidy fixes

* Pre-commit

* refactor: use const for set_interpreter_argv

* Try to fix const issue and allocate vector properly

* fix: copy strings on Python 2

* Applying clang-format-diff relative to master.

The only manual change is an added empty line between pybind11 and system `#include`s.

```
git diff -U0 --no-color master | python3 $HOME/clone/llvm-project/clang/tools/clang-format/clang-format-diff.py -p1 -style=file -i
```
Co-authored-by: Boris Staletic <boris.staletic@gmail.com>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>

* reshape

* more tests

* Update numpy.h

* Update test_numpy_array.py

* array view

* test

* Update test_numpy_array.cpp

* Update numpy.h

* Update numpy.h

* Update test_numpy_array.cpp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* Fix merge bug

* Make clang-tidy happy

* Add xfail for PyPy

* Fix casting issue

* Fix formatting

* Apply clang-tidy

* Address reviews on additional tests

* Fix ordering

* Do a little more reordering

* Fix typo

* Try improving tests

* Fix error in reshape

* Add one more reshape test

* Fix bugs and add test

* Relax test

* streamlining new tests; removing a few stray msg

* Fix style revert

* Fix clang-tidy

* Misc tweaks:
* Comment: matching style in file (///), responsibility sentence, consistent punctuation.
* Replacing `unsigned char` with `uint8_t` for max consistency.
* Removing `1` from `array_view1` because there is only one.

* Partial clang-format-diff.
Co-authored-by: ncullen93 <ncullen.th@dartmouth.edu>
Co-authored-by: NC Cullen <nicholas.c.cullen.th@dartmouth.edu>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf Grosse-Kunstleve <rwgk@google.com>

* reshape

* more tests

* Update numpy.h

* Update test_numpy_array.py

* Update numpy.h

* Update numpy.h

* Update test_numpy_array.cpp

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* Fix merge bug

* Make clang-tidy happy

* Add xfail for PyPy

* Fix casting issue

* Address reviews on additional tests

* Fix ordering

* Do a little more reordering

* Fix typo

* Try improving tests

* Fix error in reshape

* Add one more reshape test

* streamlining new tests; removing a few stray msg
Co-authored-by: ncullen93 <ncullen.th@dartmouth.edu>
Co-authored-by: NC Cullen <nicholas.c.cullen.th@dartmouth.edu>
Co-authored-by: Aaron Gokaslan <skylion.aaron@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf Grosse-Kunstleve <rwgk@google.com>

Co-authored-by: Jouke Witteveen <j.witteveen@cosine.nl>

* Add py::raise_from to enable chaining exceptions on Python 3.3+

* Use 'raise from' in initialization

* Documenting the exact base version of _PyErr_FormatVFromCause, adding back `assert`s.
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* maint(clang-tidy): add a clang-tidy slicing check

* Add self + touch up readme

* Fix typo

updates:
- [github.com/asottile/pyupgrade: v2.23.3 → v2.24.0](https://github.com/asottile/pyupgrade/compare/v2.23.3...v2.24.0

)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* extend msys2 CI

- add 32-bit job
- add c++11/17 c++/interface tests copied from standard ci
- add numpy/scipy

* account for padding of PartialStruct in numpy dtypes test with mingw32

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* msys2 ci: add c++14 tests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Cleanup of file-scoped and globally-scoped warning suppression pragmas across pybind11 header files. (#3201)

* Removing all MSVC C4127 warning suppression pragmas.

* Removing MSVC /WX (WERROR). To get a full list of all warnings.

* Inserting PYBIND11_SILENCE_MSVC_C4127. Changing one runtime if to #if.

* Changing PYBIND11_SILENCE_MSVC_C4127 macro to use absolute namespace (for use outside pybind11 include directory).

* Restoring MSVC /WX (WERROR).

* Removing globally-scoped suppression for clang -Wunsequenced. Based on an experiment under PR #3202 it is obsolete and can simply be removed.

* Removing all pragma from eigen.h

* Removing -Werror or equivalent from tests/CMakeLists.txt

* Restoring tests/CMakeLists.txt from master.

* Adding 4 PYBIND11_SILENCE_MSVC_C4127.

* Compatibility with -Wconversion, -Wdeprecated

* Introducing PYBIND11_COMPATIBILITY_WDEPRECATED_COPY

* Systematically using --verbose for compilations where possible (cmake 3.14 or newer).

Also changing all `cmake -t` to `--target`, `-v` to `--verbose`, `check` to `pytest`, for consistency (to make it easier to pin-point all commands of a certain type).

Also removing one `-j 2` for `pytest` in hopes of reducing flakes due to races in test_iostream and in prints from destructors.

* Commenting out pragmas as an experiment to reproduce previous observation.

* Removing all (newly added, but already commented-out) pragma code, adding HINT use -isystem (as cmake does).

* Restoring ci.yml from master. Those changes are better handled separately. BTW: in the last CI run there was still a test_iostream flake, even without the -j 2 for running the tests (verfied by inspecting the log).

* Fix errant const methods

* Remove NOLINT since clang-tidy is pretty conservative

* Missed one

* Fix a few more errors

* Add reviewer suggested comments

* Run clang-format

* Removing last remnants of pragma block at the top of pybind11.h, defaulting CUDA, GCC7, GCC8 to PYBIND11_NOINLINE_DISABLED, with the option to define PYBIND11_NOINLINE_FORCED.

* Unique SOSIZE prefix to make it easier to extract the sosizes from the GitHub logs.

* Commenting out PYBIND11_WERROR block, for noinline testing.

* Undoing accidental change.

* `#define PYBIND11_NOINLINE_FORCED`

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci



* `#define PYBIND11_NOINLINE_DISABLED`

* Going back to default (removing `#define PYBIND11_NOINLINE_DISABLED`).

* `#define PYBIND11_NOINLINE_FORCED`

* Undoing all changes releated to measuring sosizes.

* Rollback of PR #3030 (Working around Centos 8 failure).

* Disabling -Werror for GNU (experiment).

* Commenting out the entire `if(PYBIND11_WERROR)` again (although that is not expected to make a difference, but who knows what I am overlooking).

* Adding `-DCMAKE_BUILD_TYPE=Release`

* Undoing change to tests/CMakeLists.txt (uncommenting `if(PYBIND11_WERROR)` block).

* post `git rebase master -X theirs` fixups.

* Adding measurements to comment for `PYBIND11_NOINLINE_FORCED`.
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Apply isort

* Tweak isort config

* Add env.py as a known_first_party

* Add one missing known first party

* Make config compat with older isort versions

* Add another comment

* Revert pyproject setting

Bumps [jwlawson/actions-setup-cmake](https://github.com/jwlawson/actions-setup-cmake) from 1.9 to 1.10.
- [Release notes](https://github.com/jwlawson/actions-setup-cmake/releases)
- [Commits](https://github.com/jwlawson/actions-setup-cmake/compare/v1.9...v1.10

)

---
updated-dependencies:
- dependency-name: jwlawson/actions-setup-cmake
  dependency-type: direct:production
  update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Rollback of PR #3030 (Working around Centos 8 failure).

* Adding `-DCMAKE_BUILD_TYPE=Release`

* Improving existing workaround (undoing the originally intended rollback).

* Fixing minor documentation bug.

updates:
- [github.com/asottile/pyupgrade: v2.23.1 → v2.23.3](https://github.com/asottile/pyupgrade/compare/v2.23.1...v2.23.3

)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Removing pragma for GCC -Wattributes, fixing forward declarations.

* Introducing PYBIND11_NOINLINE_FWD to deal with CUDA, GCC7, GCC8.

* Updating PYBIND11_NOINLINE_DCL in Doxyfile.

* Trying noinline, noinline for {CUDA, GCC7, GCC8}

* Trying noinline, inline for {CUDA, GCC7, GCC8}

* Adding GCC -Wattributes `pragma` in 3 header files.

* Introducing PYBIND11_NOINLINE_GCC_PRAGMA_ATTRIBUTES_NEEDED, used in 9 header files.

* Removing ICC pragma 2196, to see if it is still needed.

* Trying noinline, noinline for ICC

* Trying noinline, inline for ICC

* Restoring ICC pragma 2196, introducing PYBIND11_NOINLINE_FORCED, defined for testing.

* Removing code accidentally left in (was for experimentation only).

* Removing one-time-test define.

* Removing PYBIND11_NOINLINE_FWD macro (after learning that it makes no sense).

* Testing with PYBIND11_NOINLINE_DISABLED. Minor non-functional enhancements.

* Removing #define PYBIND11_NOINLINE_DISABLED (test was successful).

* Removing PYBIND11_NOINLINE_FORCED and enhancing comments for PYBIND11_NOINLINE.

* WIP stripping back

* Making -Wattributes pragma in pybind11 specific to GCC7, GCC8, CUDA.

* maint(perf): Optimize Numpy Constructor with additional std::move

* Add more moves