GitLab

Currently pybind11 doesn't check when you define a new object (e.g. a
class, function, or exception) that overwrites an existing one.  If the
thing being overwritten is a class, this leads to a segfault (because
pybind still thinks the type is defined, even though Python no longer
has the type).  In other cases this is harmless (e.g. replacing a
function with an exception), but even in that case it's most likely a
bug.

This code doesn't prevent you from actively doing something harmful,
like deliberately overwriting a previous definition, but detects
overwriting with a run-time error if it occurs in the standard
class/function/exception/def registration interfaces.

All of the additions are in non-template code; the result is actually a
tiny decrease in .so size compared to master without the new test code
(977304 to 977272 bytes), and about 4K higher with the new tests.

This reduces direct access to internals.registered_types_cpp to
just a few places.

This avoid a hashmap lookup since the pointer to the list of
direct converters is now cached in the typeinfo.

Making `cppfunction` explicit broke `def_property` and friends.
The added tests would not compile without an implicit `cppfunction`.

We have various classes that have non-explicit constructors that accept
a single argument, which is implicitly making them implicitly
convertible from the argument.  In a few cases, this is desirable (e.g.
implicit conversion of std::string to py::str, or conversion of double
to py::float_); in many others, however, it is unintended (e.g. implicit
conversion of size_t to some pre-declared py::array_t<T> type).

This disables most of the unwanted implicit conversions by marking them
`explicit`, and comments the ones that are deliberately left implicit.

`auto var = l[0]` has a strange quirk: `var` is actually an accessor and
not an object, so any later assignment of `var = ...` would modify l[0]
instead of `var`. This is surprising compared to the non-auto assignment
`py::object var = l[0]; var = ...`.

By overloading `operator=` on lvalue/rvalue, the expected behavior is
restored even for `auto` variables.

This also adds the `hasattr` and `getattr` functions which are needed
with the new attribute behavior. The new functions behave exactly like
their Python counterparts.

Similarly `object` gets a `contains` method which calls `__contains__`,
i.e. it's the same as the `in` keyword in Python.

The custom exception handling added in PR #273 is robust, but is overly
complex for declaring the most common simple C++ -> Python exception
mapping that needs only to copy `what()`.  This add a simpler
`py::register_exception<CppExp>(module, "PyExp");` function that greatly
simplifies the common basic case of translation of a simple CppException
into a simple PythonException, while not removing the more advanced
capabilities of defining custom exception handlers.

This adds a static local variable (in dead code unless actually needed)
in the overload code that is used for storage if the overload is for
some convert-by-value type (such as numeric values or std::string).

This has limitations (as written up in the advanced doc), but is better
than simply not being able to overload reference or pointer methods.

This clears the Python error at the error_already_set throw site, thus
allowing Python calls to be made in destructors which are triggered by
the exception. This is preferable to the alternative, which would be
guarding every Python API call with an error_scope.

This effectively flips the behavior of error_already_set. Previously,
it was assumed that the error stays in Python, so handling the exception
in C++ would require explicitly calling PyErr_Clear(), but nothing was
needed to propagate the error to Python. With this change, handling the
error in C++ does not require a PyErr_Clear() call, but propagating the
error to Python requires an explicit error_already_set::restore().

The change does not break old code which explicitly calls PyErr_Clear()
for cleanup, which should be the majority of user code. The need for an
explicit restore() call does break old code, but this should be mostly
confined to the library and not user code.

This commit adds support for forcing alias type initialization by
defining constructors with `py::init_alias<arg1, arg2>()` instead of
`py::init<arg1, arg2>()`.  Currently py::init<> only results in Alias
initialization if the type is extended in python, or the given
arguments can't be used to construct the base type, but can be used to
construct the alias.  py::init_alias<>, in contrast, always invokes the
constructor of the alias type.

It looks like this was already the intention of
`py::detail::init_alias`, which was forward-declared in
86d825f3, but was apparently never
finished: despite the existance of a .def method accepting it, the
`detail::init_alias` class isn't actually defined anywhere.

This commit completes the feature (or possibly repurposes it), allowing
declaration of classes that will always initialize the trampoline which
is (as I argued in #397) sometimes useful.

Type alias for alias classes with members didn't work properly: space
was only allocated for sizeof(type), but if we want to be able to put a
type_alias instance there, we need sizeof(type_alias), but
sizeof(type_alias) > sizeof(type) whenever type_alias has members.

This allows a slightly cleaner base type specification of:

    py::class_<Type, Base>("Type")

as an alternative to

    py::class_<Type>("Type", py::base<Base>())

As with the other template parameters, the order relative to the holder
or trampoline types doesn't matter.

This also includes a compile-time assertion failure if attempting to
specify more than one base class (but is easily extendible to support
multiple inheritance, someday, by updating the class_selector::set_bases
function to set multiple bases).

The current pybind11::class_<Type, Holder, Trampoline> fixed template
ordering results in a requirement to repeat the Holder with its default
value (std::unique_ptr<Type>) argument, which is a little bit annoying:
it needs to be specified not because we want to override the default,
but rather because we need to specify the third argument.

This commit removes this limitation by making the class_ template take
the type name plus a parameter pack of options.  It then extracts the
first valid holder type and the first subclass type for holder_type and
trampoline type_alias, respectively.  (If unfound, both fall back to
their current defaults, `std::unique_ptr<type>` and `type`,
respectively).  If any unmatched template arguments are provided, a
static assertion fails.

What this means is that you can specify or omit the arguments in any
order:

    py::class_<A, PyA> c1(m, "A");
    py::class_<B, PyB, std::shared_ptr<B>> c2(m, "B");
    py::class_<C, std::shared_ptr<C>, PyB> c3(m, "C");

It also allows future class attributes (such as base types in the next
commit) to be passed as class template types rather than needing to use
a py::base<> wrapper.

Replicates Python API including keyword arguments.

Problem
=======

The template trampoline pattern documented in PR #322 has a problem with
virtual method overloads in intermediate classes in the inheritance
chain between the trampoline class and the base class.

For example, consider the following inheritance structure, where `B` is
the actual class, `PyB<B>` is the trampoline class, and `PyA<B>` is an
intermediate class adding A's methods into the trampoline:

    PyB<B> -> PyA<B> -> B -> A

Suppose PyA<B> has a method `some_method()` with a PYBIND11_OVERLOAD in
it to overload the virtual `A::some_method()`.  If a Python class `C` is
defined that inherits from the pybind11-registered `B` and tries to
provide an overriding `some_method()`, the PYBIND11_OVERLOADs declared
in PyA<B> fails to find this overloaded method, and thus never invoke it
(or, if pure virtual and not overridden in PyB<B>, raises an exception).

This happens because the base (internal) `PYBIND11_OVERLOAD_INT` macro
simply calls `get_overload(this, name)`; `get_overload()` then uses the
inferred type of `this` to do a type lookup in `registered_types_cpp`.
This is where it fails: `this` will be a `PyA<B> *`, but `PyA<B>` is
neither the base type (`B`) nor the trampoline type (`PyB<B>`).  As a
result, the overload fails and we get a failed overload lookup.

The fix
=======

The fix is relatively simple: we can cast `this` passed to
`get_overload()` to a `const B *`, which lets get_overload look up the
correct class.  Since trampoline classes should be derived from `B`
classes anyway, this cast should be perfectly safe.

This does require adding the class name as an argument to the
PYBIND11_OVERLOAD_INT macro, but leaves the public macro signatures
unchanged.

- ICPC can't handle the NCVirt trampoline which returns a non-copyable
  type, which is likely due to a constexpr/SFINAE issue. This disables
  the test on that compiler so that at least the rest can be tested.

For example keep_alive<0,1>() should work where the return value may sometimes be None. At present a "Could not allocate weak reference!" exception is thrown.
Update documentation to clarify behaviour of keep_alive when nurse is None or does not support weak references.

This allows exposing a dict-like interface to python code, allowing
iteration over keys via:

    for k in custommapping:
        ...

while still allowing iteration over pairs, so that you can also
implement 'dict.items()' functionality which returns a pair iterator,
allowing:

    for k, v in custommapping.items():
        ...

example-sequences-and-iterators is updated with a custom class providing
both types of iteration.

reference_internal requires an `instance` field to track the returned
reference's parent, but that's just a duplication of what
keep_alive<0,1> does, so use a keep alive to do this to eliminate the
duplication.

The pointer to the first member of a class instance is the same as the
pointer to instance itself; pybind11 has some workarounds for this to
not track registered instances that have a registered parent with the
same address.  This doesn't work everywhere, however: issue #328 is a
failure of this for a mutator operator which resolves its argument to
the parent rather than the child, as is needed in #328.

This commit resolves the issue (and restores tracking of same-address
instances) by changing registered_instances from an unordered_map to an
unordered_multimap that allows duplicate instances for the same pointer
to be recorded, then resolves these differences by checking the type of
each matched instance when looking up an instance.  (A
unordered_multimap seems cleaner for this than a unordered_map<list> or
similar because, the vast majority of the time, the instance will be
unique).