GitLab

Fix a segfault when creating a 0-dimension, c-strides array.

Found via `codespell`

* Add basic support for tag-based static polymorphism

Sometimes it is possible to look at a C++ object and know what its dynamic type is,
even if it doesn't use C++ polymorphism, because instances of the object and its
subclasses conform to some other mechanism for being self-describing; for example,
perhaps there's an enumerated "tag" or "kind" member in the base class that's always
set to an indication of the correct type. This might be done for performance reasons,
or to permit most-derived types to be trivially copyable. One of the most widely-known
examples is in LLVM: https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html

This PR permits pybind11 to be informed of such conventions via a new specializable
detail::polymorphic_type_hook<> template, which generalizes the previous logic for
determining the runtime type of an object based on C++ RTTI. Implementors provide
a way to map from a base class object to a const std::type_info* for the dynamic
type; pybind11 then uses this to ensure that casting a Base* to Python creates a
Python object that knows it's wrapping the appropriate sort of Derived.

There are a number of restrictions with this tag-based static polymorphism support
compared to pybind11's existing support for built-in C++ polymorphism:

- there is no support for this-pointer adjustment, so only single inheritance is permitted
- there is no way to make C++ code call new Python-provided subclasses
- when binding C++ classes that redefine a method in a subclass, the .def() must be
  repeated in the binding for Python to know about the update

But these are not much of an issue in practice in many cases, the impact on the
complexity of pybind11's innards is minimal and localized, and the support for
automatic downcasting improves usability a great deal.

The property returns the enum_ value as a string.
For example:

>>> import module
>>> module.enum.VALUE
enum.VALUE
>>> str(module.enum.VALUE)
'enum.VALUE'
>>> module.enum.VALUE.name
'VALUE'

This is actually the equivalent of Boost.Python "name" property.

- PYBIND11_MAKE_OPAQUE now takes ... rather than a single argument and
  expands it with __VA_ARGS__; this lets templated, comma-containing
  types get through correctly.
- Adds a new macro PYBIND11_TYPE() that lets you pass the type into a
  macro as a single argument, such as:

      PYBIND11_OVERLOAD(PYBIND11_TYPE(R<1,2>), PYBIND11_TYPE(C<3,4>), func)

  Unfortunately this only works for one macro call: to forward the
  argument on to the next macro call (without the processor breaking it
  up again) requires also adding the PYBIND11_TYPE(...) to type macro
  arguments in the PYBIND11_OVERLOAD_... macro chain.
- updated the documentation with these two changes, and use them at a couple
  places in the test suite to test that they work.

This fixes the test code on big-endian architectures: the array support
(PR #832) had hard-coded the little-endian '<' but we need to use '>' on
big-endian architectures.

This updates the `py::init` constructors to only use brace
initialization for aggregate initiailization if there is no constructor
with the given arguments.

This, in particular, fixes the regression in #1247 where the presence of
a `std::initializer_list<T>` constructor started being invoked for
constructor invocations in 2.2 even when there was a specific
constructor of the desired type.

The added test case demonstrates: without this change, it fails to
compile because the `.def(py::init<std::vector<int>>())` constructor
tries to invoke the `T(std::initializer_list<std::vector<int>>)`
constructor rather than the `T(std::vector<int>)` constructor.

By only using `new T{...}`-style construction when a `T(...)`
constructor doesn't exist, we should bypass this by while still allowing
`py::init<...>` to be used for aggregate type initialization (since such
types, by definition, don't have a user-declared constructor).

* Fix segfault when reloading interpreter with external modules

When embedding the interpreter and loading external modules in that
embedded interpreter, the external module correctly shares its
internals_ptr with the one in the embedded interpreter.  When the
interpreter is shut down, however, only the `internals_ptr` local to
the embedded code is actually reset to nullptr: the external module
remains set.

The result is that loading an external pybind11 module, letting the
interpreter go through a finalize/initialize, then attempting to use
something in the external module fails because this external module is
still trying to use the old (destroyed) internals.  This causes
undefined behaviour (typically a segfault).

This commit fixes it by adding a level of indirection in the internals
path, converting the local internals variable to `internals **` instead
of `internals *`.  With this change, we can detect a stale internals
pointer and reload the internals pointer (either from a capsule or by
creating a new internals instance).

(No issue number: this was reported on gitter by @henryiii and @aoloe).

Fix return from `std::map` bindings to `__delitem__`: we should be returning `void`, not an iterator.

Also adds a test for map item deletion.

Found via `codespell`

- UPDATEIFCOPY is deprecated, replaced with similar (but not identical)
  WRITEBACKIFCOPY; trying to access the flag causes a deprecation
  warning under numpy 1.14, so just check the new flag there.
- Numpy `repr` formatting of floats changed in 1.14.0 to `[1., 2., 3.]`
  instead of the pre-1.14 `[ 1.,  2.,  3.]`.  Updated the tests to
  check for equality with the `repr(...)` value rather than the
  hard-coded (and now version-dependent) string representation.

PEP8 indicates (correctly, IMO) that when an annotation is present, the
signature should include spaces around the equal sign, i.e.

    def f(x: int = 1): ...

instead of

    def f(x: int=1): ...

(in the latter case the equal appears to bind to the type, not to the
argument).

pybind11 signatures always includes a type annotation so we can always
add the spaces.

* Make register_dtype() accept any field containers

* Add a test for programmatic dtype registration

In the latest MSVC in C++17 mode including Eigen causes warnings:

    warning C4996: 'std::unary_negate<_Fn>': warning STL4008: std::not1(),
    std::not2(), std::unary_negate, and std::binary_negate are deprecated in
    C++17. They are superseded by std::not_fn(). You can define
    _SILENCE_CXX17_NEGATORS_DEPRECATION_WARNING or
    _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS to acknowledge that you have
    received this warning.

This disables 4996 for the Eigen includes.

Catch generates a similar warning for std::uncaught_exception, so
disable the warning there, too.

In both cases this is temporary; we can (and should) remove the warnings
disabling once new upstream versions of Eigen and Catch are available
that address the warning. (The Catch one, in particular, looks to be
fixed in upstream master, so will probably be fixed in the next (2.0.2)
release).

Pybind11's default conversion to int always produces a long on Python 2 (`int`s and `long`s were unified in Python 3). This patch fixes `int` handling to match Python 2 on Python 2; for short types (`size_t` or smaller), the number will be returned as an `int` if possible, otherwise `long`. Requires Python 2.5+.

This is needed for things like `sys.exit`, which refuse to accept a `long`.

This commit turns on `-Wdeprecated` in the test suite and fixes several
associated deprecation warnings that show up as a result:

- in C++17 `static constexpr` members are implicitly inline; our
  redeclaration (needed for C++11/14) is deprecated in C++17.

- various test suite classes have destructors and rely on implicit copy
  constructors, but implicit copy constructor definitions when a
  user-declared destructor is present was deprecated in C++11.

- Eigen also has various implicit copy constructors, so just disable
  `-Wdeprecated` in `eigen.h`.

py::class_<T>'s `def_property` and `def_property_static` can now take a
`nullptr` as the getter to allow a write-only property to be established
(mirroring Python's `property()` built-in when `None` is given for the
getter).

This also updates properties to use the new nullptr constructor internally.

A few fixes related to how we set `__qualname__` and how we show the
type name in function signatures:

- `__qualname__` isn't supposed to have the module name at the
beginning, but we've been putting it there.  This removes it, while
keeping the `Nested.Class` name chaining.

- print `__module__.__qualname__` rather than `type->tp_name`; the
latter doesn't work properly for nested classes, so we would get
`module.B` rather than `module.A.B` for a class `B` with parent `A`.
This also unifies the Python 3 and PyPy code.  Fixes #1166.

- This now sets a `__qualname__` attribute on the type (as would happen
in Python 3.3+) for Python <3.3, including PyPy.  While not particularly
important to have in earlier Python versions, it's useful for us to be
able to extracted the nested name, which is why `__qualname__` was
invented in the first place.

- Added tests for the above.

Non-user facing. 
Found using `codespell -q 3`

The just-updated flake8 package hits a bunch of:

    E741 ambiguous variable name 'l'

warnings.  This commit renames them all from `l` to `lst` (they are all
list values) to avoid the error.

- For the debian/buster docker build (GCC 7/C++17) install and use the
  system `catch` package; this also renames "COMPILER_PACKAGES" to
  "EXTRA_PACKAGES" since it now contains a non-compiler package.

- Add a status message indicating the catch version being used for
  compiling the embedded tests

- Simplify some bash code by using VAR+=" foo" to append (rather than
  VAR="${VAR} foo"

- Fix CMAKE_INCLUDE_PATH appending: it was prepending the ':' but not
  the existing $CMAKE_INCLUDE_PATH value and so would end up with
  ":/eigen-path" if CMAKE_INCLUDE_PATH was already set.  (This wasn't
  bug that was actually noticed since currently nothing else sets it).

This fixes a bug introduced in b68959e8
when passing in a two-dimensional, but conformable, array as the value
for a compile-time Eigen vector (such as VectorXd or RowVectorXd).  The
commit switched to using numpy to copy into the eigen data, but this
broke the described case because numpy refuses to broadcast a (N,1)
into a (N).

This commit fixes it by squeezing the input array whenever the output
array is 1-dimensional, which will let the problematic case through.
(This shouldn't squeeze inappropriately as dimension compatibility is
already checked for conformability before getting to the copy code).

This changes the caster to return a reference to a (new) local `CharT`
type caster member so that binding lvalue-reference char arguments
works (currently it results in a compilation failure).

Fixes #1116

Fixes #1117

This also matches the Eigen example for the row-major case.

This also enhances one of the tests to trigger a failure (and fixes it in the PR).  (This isn't really a flaw in pybind itself, but rather fixes wrong code in the test code and docs).

The entire test file is already marked as requiring numpy; it isn't
needed on the individual test.

`PyArray_DescrConverter_` doesn't steal a reference to the argument,
and so the passed arguments shouldn't be `.release()`d.

The current C++14 constexpr signatures don't require relaxed constexpr,
but only `auto` return type deduction. To get around this in C++11,
the type caster's `name()` static member functions are turned into
`static constexpr auto` variables.

E.g. trying to convert a `list` to a `std::vector<int>` without
including <pybind11/stl.h> will now raise an error with a note that
suggests checking the headers.

The note is only appended if `std::` is found in the function
signature. This should only be the case when a header is missing.
E.g. when stl.h is included, the signature would contain `List[int]`
instead of `std::vector<int>` while using stl_bind.h would produce
something like `MyVector`. Similarly for `std::map`/`Dict`, `complex`,
`std::function`/`Callable`, etc.

There's a possibility for false positives, but it's pretty low.

Fixes #1069.

To avoid an ODR violation in the test suite while testing
both `stl.h` and `std_bind.h` with `std::vector<bool>`,
the `py::bind_vector<std::vector<bool>>` test is moved to
the secondary module (which does not include `stl.h`).

Fixes #1061.

`T` and `const T &` are compatible types.

Fixes #1046.

There are two separate additions:

1. `py::hash(obj)` is equivalent to the Python `hash(obj)`.
2. `.def(hash(py::self))` registers the hash function defined by
   `std::hash<T>` as the Python hash function.