GitLab

This commit adds `error_already_set::matches()` convenience method to
check if the exception trapped by `error_already_set` matches a given
Python exception type. This will address #700 by providing a less
verbose way to check exceptions.

* Support raw string literals as input for py::eval
* Dedent only when needed

* nicer py::capsule destructor mechanism
* added destructor-only version of capsule & tests
* added documentation for module destructors (fixes #733)

The extends the previous unchecked support with the ability to
determine the dimensions at runtime.  This incurs a small performance
hit when used (versus the compile-time fixed alternative), but is still considerably
faster than the full checks on every call that happen with
`.at()`/`.mutable_at()`.

This adds bounds-unchecked access to arrays through a `a.unchecked<Type,
Dimensions>()` method.  (For `array_t<T>`, the `Type` template parameter
is omitted).  The mutable version (which requires the array have the
`writeable` flag) is available as `a.mutable_unchecked<...>()`.

Specifying the Dimensions as a template parameter allows storage of an
std::array; having the strides and sizes stored that way (as opposed to
storing a copy of the array's strides/shape pointers) allows the
compiler to make significant optimizations of the shape() method that it
can't make with a pointer; testing with nested loops of the form:

    for (size_t i0 = 0; i0 < r.shape(0); i0++)
        for (size_t i1 = 0; i1 < r.shape(1); i1++)
            ...
                r(i0, i1, ...) += 1;

over a 10 million element array gives around a 25% speedup (versus using
a pointer) for the 1D case, 33% for 2D, and runs more than twice as fast
with a 5D array.

Fixes #754.

This extends the trivial handling to support trivial handling for
Fortran-order arrays (i.e. column major): if inputs aren't all
C-contiguous, but *are* all F-contiguous, the resulting array will be
F-contiguous and we can do trivial processing.

For anything else (e.g. C-contiguous, or inputs requiring non-trivial
processing), the result is in (numpy-default) C-contiguous layout.

The only part of the vectorize code that actually needs c-contiguous is
the "trivial" broadcast; for non-trivial arguments, the code already
uses strides properly (and so handles C-style, F-style, neither, slices,
etc.)

This commit rewrites `broadcast` to additionally check for C-contiguous
storage, then takes off the `c_style` flag for the arguments, which
will keep the functionality more or less the same, except for no longer
requiring an array copy for non-c-contiguous input arrays.

Additionally, if we're given a singleton slice (e.g. a[0::4, 0::4] for a
4x4 or smaller array), we no longer fail triviality because the trivial
code path never actually uses the strides on a singleton.

Instead of a segfault. Fixes #751.

This covers the case of loading a custom holder from a default-holder
instance. Attempting to load one custom holder from a different custom
holder (i.e. not `std::unique_ptr`) yields undefined behavior, just as
#588 established for inheritance.

We can't support this for classes from imported modules (which is the
primary purpose of a ctor argument base class) because we *have* to
have both parent and derived to properly extract a multiple-inheritance
base class pointer from a derived class pointer.

We could support this for actual `class_<Base, ...> instances, but since
in that case the `Base` is already present in the code, it seems more
consistent to simply always require MI to go via template options.

Fixes #738

The current check for conformability fails when given a 2D, 1xN or Nx1
input to a row-major or column-major, respectively, Eigen::Ref, leading
to a copy-required state in the type_caster, but this later failed
because the copy was also non-conformable because it had the same shape
and strides (because a 1xN or Nx1 is both F and C contiguous).

In such cases we can safely ignore the stride on the "1" dimension since
it'll never be used: only the "N" dimension stride needs to match the
Eigen::Ref stride, which both fixes the non-conformable copy problem,
but also avoids a copy entirely as long as the "N" dimension has a
compatible stride.

Fixes #731.

Generally, this applies to any caster made with PYBIND11_TYPE_CASTER().

Fixes #728.

Allows equivalent integral types and numpy dtypes

Allows use of vectors as python buffers, so for example they can be adopted without a copy by numpy.asarray
Allows faster conversion of buffers to vectors by copying instead of individually casting the elements

* Add value_type member alias to py::array_t (resolve #632)

* Use numpy scalar name in py::array_t function signatures (e.g. float32/64 instead of just float)

The duration calculation was using %, but that's only supported on
duration objects when the arithmetic type supports %, and hence fails
for floats.  Fixed by subtracting off the calculated values instead.

* Add `pytest.ini` config file and set default options there instead of
  in `CMakeLists.txt` (command line arguments).

* Change all output capture from `capfd` (filedescriptors) to `capsys`
  (Python's `sys.stdout` and `sys.stderr`). This avoids capturing
  low-level C errors, e.g. from the debug build of Python.

* Set pytest minimum version to 3.0 to make it easier to use new
  features. Removed conditional use of `excinfo.match()`.

* Clean up some leftover function-level `@pytest.requires_numpy`.

When using pybind::options to disable function signatures, user-defined
docstrings only get appended if they exist, but newlines were getting
appended unconditionally, so the docstring could end up with blank lines
(depending on which overloads, in particular, provided docstrings).

This commit suppresses the empty lines by only adding newlines for
overloads when needed.

This makes array_t respect overload resolution and noconvert by failing
to load when `convert = false` if the src isn't already an array of the
correct type.

Slightly reduces binary size (range for loops over tuple/list benefit
a lot). The iterators are compatible with std algorithms.

The added type aliases are required by `std::iterator_traits`.
Python iterators satisfy the `InputIterator` concept in C++.

Before this, `py::iterator` didn't do any error handling, so code like:
```c++
for (auto item : py::int_(1)) {
    // ...
}
```
would just silently skip the loop. The above now throws `TypeError` as
expected. This is a breaking behavior change, but any code which relied
on the silent skip was probably broken anyway.

Also, errors returned by `PyIter_Next()` are now properly handled.

Fixes test failure on Fedora 25.

Fixes some numpy tests failures on ppc64 in big-endian mode due to
little-endian assumptions.

Fixes #694.

test_eigen.py and test_numpy_*.py have the same
@pytest.requires_eigen_and_numpy or @pytest.requires_numpy on every
single test; this changes them to use pytest's global `pytestmark = ...`
instead to disable the entire module when numpy and/or eigen aren't
available.

This commit largely rewrites the Eigen dense matrix support to avoid
copying in many cases: Eigen arguments can now reference numpy data, and
numpy objects can now reference Eigen data (given compatible types).

Eigen::Ref<...> arguments now also make use of the new `convert`
argument use (added in PR #634) to avoid conversion, allowing
`py::arg().noconvert()` to be used when binding a function to prohibit
copying when invoking the function.  Respecting `convert` also means
Eigen overloads that avoid copying will be preferred during overload
resolution to ones that require copying.

This commit also rewrites the Eigen documentation and test suite to
explain and test the new capabilities.

Numpy raises ValueError when attempting to modify an array, while
py::array is raising a RuntimeError.  This changes the exception to a
std::domain_error, which gets mapped to the expected ValueError in
python.

numpy arrays aren't currently properly setting base: by setting `->base`
directly, the base doesn't follow what numpy expects and documents (that
is, following chained array bases to the root array).

This fixes the behaviour by using numpy's PyArray_SetBaseObject to set
the base instead, and then updates the tests to reflect the fixed
behaviour.

Currently when we do a conversion between a numpy array and an Eigen
Vector, we allow the conversion only if the Eigen type is a
compile-time vector (i.e. at least one dimension is fixed at 1 at
compile time), or if the type is dynamic on *both* dimensions.

This means we can run into cases where MatrixXd allow things that
conforming, compile-time sizes does not: for example,
`Matrix<double,4,Dynamic>` is currently not allowed, even when assigning
from a 4-element vector, but it *is* allowed for a
`Matrix<double,Dynamic,Dynamic>`.

This commit also reverts the current behaviour of using the matrix's
storage order to determine the structure when the Matrix is fully
dynamic (i.e. in both dimensions).  Currently we assign to an eigen row
if the storage order is row-major, and column otherwise: this seems
wrong (the storage order has nothing to do with the shape!).  While
numpy doesn't distinguish between a row/column vector, Eigen does, but
it makes more sense to consistently choose one than to produce
something with a different shape based on the intended storage layout.

With the previous commit, output can be very confusing because you only
see positional arguments in the "invoked with" line, but you can have a
failure from kwargs as well (in particular, when a value is invalidly
specified via both via positional and kwargs).  This commits adds
kwargs to the output, and updates the associated tests to match.

* Make tests buildable independently

This makes "tests" buildable as a separate project that uses
find_package(pybind11 CONFIG) when invoked independently.

This also moves the WERROR option into tests/CMakeLists.txt, as that's
the only place it is used.

* Use Eigen 3.3.1's cmake target, if available

This changes the eigen finding code to attempt to use Eigen's
system-installed Eigen3Config first.  In Eigen 3.3.1, it exports a cmake
Eigen3::Eigen target to get dependencies from (rather than setting the
include path directly).

If it fails, we fall back to the trying to load allowing modules (i.e.
allowing our tools/FindEigen3.cmake).  If we either fallback, or the
eigen version is older than 3.3.1 (or , we still set the include
directory manually; otherwise, for CONFIG + new Eigen, we get it via
the target.

This is also needed to allow 'tests' to be built independently, when
the find_package(Eigen3) is going to find via the system-installed
Eigen3Config.cmake.

* Add a install-then-build test, using clang on linux

This tests that `make install` to the actual system, followed by a build
of the tests (without the main pybind11 repository available) works as
expected.

To also expand the testing variety a bit, it also builds using
clang-3.9 instead of gcc.

* Don't try loading Eigen3Config in cmake < 3.0

It could FATAL_ERROR as the newer cmake includes a cmake 3.0 required
line.

If doing an independent, out-of-tree "tests" build, the regular
find_package(Eigen3) is likely to fail with the same error, but I think
we can just let that be: if you want a recent Eigen with proper cmake
loading support *and* want to do an independent tests build, you'll
need at least cmake 3.0.

* Make string conversion stricter

The string conversion logic added in PR #624 for all std::basic_strings
was derived from the old std::wstring logic, but that was underused and
turns out to have had a bug in accepting almost anything convertible to
unicode, while the previous std::string logic was much stricter.  This
restores the previous std::string logic by only allowing actual unicode
or string types.

Fixes #685.

* Added missing 'requires numpy' decorator

(I forgot that the change to a global decorator here is in the
not-yet-merged Eigen PR)

Now that only one shared metaclass is ever allocated, it's extremely
cheap to enable it for all pybind11 types.

* Deprecate the default py::metaclass() since it's not needed anymore.
* Allow users to specify a custom metaclass via py::metaclass(handle).

In order to fully satisfy Python's inheritance type layout requirements,
all types should have a common 'solid' base. A solid base is one which
has the same instance size as the derived type (not counting the space
required for the optional `dict_ptr` and `weakrefs_ptr`). Thus, `object`
does not qualify as a solid base for pybind11 types and this can lead to
issues with multiple inheritance.

To get around this, new base types are created: one per unique instance
size. There is going to be very few of these bases. They ensure Python's
MRO checks will pass when multiple bases are involved.