GitLab

Adding or removing tests is a little bit cumbersome currently: the test
needs to be added to CMakeLists.txt, the init function needs to be
predeclared in pybind11_tests.cpp, then called in the plugin
initialization.  While this isn't a big deal for tests that are being
committed, it's more of a hassle when working on some new feature or
test code for which I temporarily only care about building and linking
the test being worked on rather than the entire test suite.

This commit changes tests to self-register their initialization by
having each test initialize a local object (which stores the
initialization function in a static variable).  This makes changing the
set of tests being build easy: one only needs to add or comment out
test names in tests/CMakeLists.txt.

A couple other minor changes that go along with this:

- test_eigen.cpp is now included in the test list, then removed if eigen
  isn't available.  This lets you disable the eigen tests by commenting
  it out, just like all the other tests, but keeps the build working
  without eigen eigen isn't available.  (Also, if it's commented out, we
  don't even bother looking for and reporting the building with/without
  eigen status message).

- pytest is now invoked with all the built test names (with .cpp changed
  to .py) so that it doesn't try to run tests that weren't built.

- ICPC can't handle the NCVirt trampoline which returns a non-copyable
  type, which is likely due to a constexpr/SFINAE issue. This disables
  the test on that compiler so that at least the rest can be tested.

There are more enum tests than 'constants and functions'.

Use simple asserts and pytest's powerful introspection to make testing
simpler. This merges the old .py/.ref file pairs into simple .py files
where the expected values are right next to the code being tested.

This commit does not touch the C++ part of the code and replicates the
Python tests exactly like the old .ref-file-based approach.

This commit rewrites the examples that look for constructor/destructor
calls to do so via static variable tracking rather than output parsing.

The added ConstructorStats class provides methods to keep track of
constructors and destructors, number of default/copy/move constructors,
and number of copy/move assignments.  It also provides a mechanism for
storing values (e.g. for value construction), and then allows all of
this to be checked at the end of a test by getting the statistics for a
C++ (or python mapping) class.

By not relying on the precise pattern of constructions/destructions,
but rather simply ensuring that every construction is matched with a
destruction on the same object, we ensure that everything that gets
created also gets destroyed as expected.

This replaces all of the various "std::cout << whatever" code in
constructors/destructors with
`print_created(this)`/`print_destroyed(this)`/etc. functions which
provide similar output, but now has a unified format across the
different examples, including a new ### prefix that makes mixed example
output and lifecycle events easier to distinguish.

With this change, relaxed mode is no longer needed, which enables
testing for proper destruction under MSVC, and under any other compiler
that generates code calling extra constructors, or optimizes away any
constructors.  GCC/clang are used as the baseline for move
constructors; the tests are adapted to allow more move constructors to
be evoked (but other types are constructors much have matching counts).

This commit also disables output buffering of tests, as the buffering
sometimes results in C++ output ending up in the middle of python
output (or vice versa), depending on the OS/python version.

This renames example files from `exampleN` to `example-description`.

Specifically, the following renaming is applied:

example1 -> example-methods-and-attributes
example2 -> example-python-types
example3 -> example-operator-overloading
example4 -> example-constants-and-functions
example5 -> example-callbacks (*)
example6 -> example-sequence-and-iterators
example7 -> example-buffers
example8 -> example-custom-ref-counting
example9 -> example-modules
example10 -> example-numpy-vectorize
example11 -> example-arg-keywords-and-defaults
example12 -> example-virtual-functions
example13 -> example-keep-alive
example14 -> example-opaque-types
example15 -> example-pickling
example16 -> example-inheritance
example17 -> example-stl-binders
example18 -> example-eval
example19 -> example-custom-exceptions

* the inheritance parts of example5 are moved into example-inheritance
(previously example16), and the remainder is left as example-callbacks.

This commit also renames the internal variables ("Example1",
"Example2", "Example4", etc.) into non-numeric names ("ExampleMandA",
"ExamplePythonTypes", "ExampleWithEnum", etc.) to correspond to the
file renaming.

The order of tests is preserved, but this can easily be changed if
there is some more natural ordering by updating the list in
examples/CMakeLists.txt.

1. Renamed PYTHON_* to PYBIND_* everywhere

2. Renamed pybind::array_dtype<> to pybind::array_t<>