Merge branch 'devel'

* devel:
  Add bzip2 archive
  Work round loss of bzip.org website
  Quienten zip output to get output_directory
  Updated comment [ci skip]
  Updated libpng to 1.6.35
  Updated arb to 2.14.0
  Added build stamps
  Add smoke tests for install_python script
  Add script to install new Python on Appveyor
  Add Python 3.7 for Travis CI
  numpy -> NumPy
  git -> Git
  docker -> Docker
  Travis-CI/TravisCI/travis-ci -> Travis CI
  OSX -> macOS

.appveyor.yml

@@ -53,7 +53,6 @@ install:
     # this repo includes a simple package to test appveyor
     - git clone git://github.com/ogrisel/python-appveyor-demo.git
 
-
 build_script:
     # Install build requirements
     - conda install --yes %BUILD_DEPENDS%
@@ -75,3 +74,9 @@ test_script:
     # run tests from install wheel
     - cd ..
     - python -m pyappveyordemo.tests.test_extension
+
+    # Smoke test of install_python script
+    - set PYTHON=C:\Python37
+    - ps: .\install_python.ps1
+    - set PYTHON=C:\Python37-x64
+    - ps: .\install_python.ps1

.travis.yml

@@ -116,7 +116,7 @@ matrix:
       env:
         - PYTHON_VERSION=pypy-6.0
         - VENV=venv
-    # Default OSX (xcode image) is 10.12 (xcode 8.3.3) as of 2018-03-29
+    # Default OSX (xcode image) is 10.13 (xcode 9.4.1) as of 2018-08-03
     # See: https://docs.travis-ci.com/user/reference/osx/
     - os: osx
       osx_image: xcode9.4

README.rst

@@ -2,17 +2,17 @@
 Utilities for building on Travis CI and AppVeyor
 ################################################
 
-A set of scripts to automate builds of OSX and Manylinux1 wheels on the
-`Travis-CI <https://travis-ci.org/>`_ infrastructure, and also Windows
+A set of scripts to automate builds of macOS and Manylinux1 wheels on the
+`Travis CI <https://travis-ci.org/>`_ infrastructure, and also Windows
 wheels on the `AppVeyor <https://ci.appveyor.com/>`_ infrastructure.
 
-The TravisCI scripts are designed to build *and test*:
+The Travis CI scripts are designed to build *and test*:
 
-* Dual architecture OSX wheels;
+* Dual architecture macOS wheels;
 * 64-bit ``manylinux1_x86_64`` wheels, both narrow and wide Unicode builds;
 * 32-bit ``manylinux1_i686`` wheels, both narrow and wide Unicode builds.
 
-You can currently build and test against Pythons 2.7, 3.3, 3.4, 3.5, 3.6.
+You can currently build and test against Pythons 2.7, 3.3, 3.4, 3.5, 3.6, 3.7.
 
 The small innovation here is that you can test against 32-bit builds, and both
 wide and narrow Unicode Python 2 builds, which was not easy on the default
@@ -38,8 +38,8 @@ function.
 The bash scripts are layered, in the sense that they loaded in the following
 sequence:
 
-OSX
-===
+macOS
+=====
 
 See ``multibuild/travis_osx_steps.sh``.
 
@@ -56,7 +56,7 @@ functions and variables in earlier scripts:
 
 See ``travis_osx_steps.sh`` to review source order.
 
-The OSX build / test and phase are on the OSX VM started by travis-ci.
+The macOS build / test and phase are on the macOS VM started by Travis CI.
 Therefore any environment variable defined in the ``.travis.yml`` or bash
 shell scripts listed above are available for your build and test.
 
@@ -67,14 +67,14 @@ default definitions of these functions.  See below for more details.
 Manylinux
 =========
 
-The build phase is in a Manylinux1 docker container, but the test phase is in
+The build phase is in a Manylinux1 Docker container, but the test phase is in
 a clean Ubuntu 14.04 container.
 
 Build phase
 -----------
 
 ``multibuild/travis_linux_steps.sh`` defines the ``build_wheel`` function,
-which starts up the Manylinux1 docker container to run a wrapper script
+which starts up the Manylinux1 Docker container to run a wrapper script
 ``multibuild/docker_build_wrap.sh``, that (within the container) sources the
 following bash scripts:
 
@@ -98,9 +98,9 @@ real ``build_wheel`` function, which now comes (by default) from
 Test phase
 ----------
 
-Testing is in an Ubuntu 14.04 docker container - see
+Testing is in an Ubuntu 14.04 Docker container - see
 ``multibuild/docker_test_wrap.sh``.  ``multibuild/travis_linux_steps.sh``
-defines the ``install_run`` function, which starts up the testing docker
+defines the ``install_run`` function, which starts up the testing Docker
 container with a wrapper script ``multibuild/docker_test_wrap.sh``.  The
 wrapper script sources the following bash scripts:
 
@@ -120,7 +120,7 @@ Standard build and test functions
 *********************************
 
 The standard build command is ``build_wheel``.  This is a bash function.  By
-default the function that is run on OSX, and in the Manylinux container for
+default the function that is run on macOS, and in the Manylinux container for
 the build phase, is defined in ``multibuild/common_utils.sh``.  You can
 override the default function in the project ``config.sh`` file (see below).
 
@@ -136,7 +136,7 @@ defined, before building the wheel, so ``pre_build`` is a good place to build
 any required libraries.
 
 The standard test command is the bash function ``install_run``.  The version
-run on OSX and in the Linux testing container is also defined in
+run on macOS and in the Linux testing container is also defined in
 ``multibuild/common_utils.sh``.  Typically, you do not override this function,
 but you in that case you will need to define a ``run_tests`` function, to run
 your tests, returning a non-zero error code for failure.  The default
@@ -186,7 +186,7 @@ To use these scripts
                 "MNKyBWOzu7JAUmC0Y+JhPKfytXxY/ADRmUIMEWZV977FLZPgYctqd+lqel2QIFgdHDO1CIdTSymOOFZckM9ICUXg9Ta+8oBjSvAVWO1ahDcToRM2DLq66fKg+NKimd2OfK7x597h/QmUSl4k8XyvyyXgl5jOiLg/EJxNE2r83IA="
 
     language: python
-    # The travis Python version is unrelated to the version we build and test
+    # The Travis Python version is unrelated to the version we build and test
     # with.  This is set with the MB_PYTHON_VERSION variable.
     python: 3.5
     sudo: required
@@ -286,7 +286,7 @@ To use these scripts
             --no-update-index
             wheels
 
-  The example above is for a project building from a git submodule.  If you
+  The example above is for a project building from a Git submodule.  If you
   aren't building from a submodule, but want to use ``pip`` to build from a
   source archive on https://pypi.org or similar, replace the first few lines
   of the ``.travis.yml`` file with something like::
@@ -306,7 +306,7 @@ To use these scripts
   You also need this file to specify how to run your tests::
 
     # Define custom utilities
-    # Test for OSX with [ -n "$IS_OSX" ]
+    # Test for macOS with [ -n "$IS_OSX" ]
 
     function pre_build {
         # Any stuff that you need to do before you start building the wheels
@@ -354,12 +354,12 @@ To use these scripts
 * Make sure your project is set up to build on AppVeyor, and you should now
   be ready (for what could be another round of slow debugging).
 
-If your project depends on numpy, you will want to build against the earliest
-numpy that your project supports - see `forward, backward numpy compatibility
+If your project depends on NumPy, you will want to build against the earliest
+NumPy that your project supports - see `forward, backward NumPy compatibility
 <https://stackoverflow.com/questions/17709641/valueerror-numpy-dtype-has-the-wrong-size-try-recompiling/18369312#18369312>`_.
-See the `astropy-wheels travis file
+See the `astropy-wheels Travis file
 <https://github.com/MacPython/astropy-wheels/blob/master/.travis.yml>`_ for an
-example specifying numpy build and test dependencies.
+example specifying NumPy build and test dependencies.
 
 Here are some simple example projects:
 
@@ -369,7 +369,7 @@ Here are some simple example projects:
 * https://github.com/MacPython/dipy-wheels
 
 Less simple projects where there are some serious build dependencies, and / or
-OSX / Linux differences:
+macOS / Linux differences:
 
 * https://github.com/MacPython/matplotlib-wheels
 * https://github.com/python-pillow/Pillow-wheels

common_utils.sh

@@ -139,7 +139,7 @@ function untar {
         tar) tar -xf $in_fname ;;
         gz|tgz) tar -zxf $in_fname ;;
         bz2) tar -jxf $in_fname ;;
-        zip) unzip $in_fname ;;
+        zip) unzip -qq $in_fname ;;
         xz) unxz -c $in_fname | tar -xf ;;
         *) echo Did not recognize extension $extension; exit 1 ;;
     esac

install_python.ps1

+# Install specified Python version.
+# Install only if:
+#    Our current matrix entry uses this Python version AND
+#    Python version is not already available.
+
+$py_exe = "${env:PYTHON}\Python.exe"
+if ( [System.IO.File]::Exists($py_exe) ) {
+    exit 0
+}
+$req_nodot = $env:PYTHON -replace '\D+Python(\d+)(?:-x64)?','$1'
+$req_ver = $req_nodot -replace '(\d)(\d+)','$1.$2.0'
+
+if ($env:PYTHON -eq "C:\Python${req_nodot}-x64") {
+    $exe_suffix="-amd64"
+} elseif ($env:PYTHON -eq "C:\Python${req_nodot}") {
+    $exe_suffix=""
+} else {
+    exit 0
+}
+
+$py_url = "https://www.python.org/ftp/python"
+Write-Host "Installing Python ${req_ver}$exe_suffix..." -ForegroundColor Cyan
+$exePath = "$env:TEMP\python-${req_ver}${exe_suffix}.exe"
+$downloadFile = "$py_url/${req_ver}/python-${req_ver}${exe_suffix}.exe"
+Write-Host "Downloading $downloadFile..."
+(New-Object Net.WebClient).DownloadFile($downloadFile, $exePath)
+Write-Host "Installing..."
+cmd /c start /wait $exePath /quiet TargetDir="$env:PYTHON" Shortcuts=0 Include_launcher=0 InstallLauncherAllUsers=0
+Write-Host "Python ${req_ver} installed to $env:PYTHON"
+
+echo "$(& $py_exe --version 2> $null)"

library_builders.sh

@@ -14,6 +14,8 @@ OPENBLAS_VERSION="${OPENBLAS_VERSION:-0.2.18}"
 ZLIB_VERSION="${ZLIB_VERSION:-1.2.10}"
 LIBPNG_VERSION="${LIBPNG_VERSION:-1.6.21}"
 BZIP2_VERSION="${BZIP2_VERSION:-1.0.6}"
+# BZIP website went down as of August 8 2018
+BZIP_URL=https://web.archive.org/web/20180624184806/http://bzip.org
 FREETYPE_VERSION="${FREETYPE_VERSION:-2.6.3}"
 TIFF_VERSION="${TIFF_VERSION:-4.0.6}"
 JPEG_VERSION="${JPEG_VERSION:-9b}"
@@ -137,7 +139,7 @@ function build_libpng {
 function build_bzip2 {
     if [ -n "$IS_OSX" ]; then return; fi  # OSX has bzip2 libs already
     if [ -e bzip2-stamp ]; then return; fi
-    fetch_unpack http://bzip.org/${BZIP2_VERSION}/bzip2-${BZIP2_VERSION}.tar.gz
+    fetch_unpack $BZIP_URL/${BZIP2_VERSION}/bzip2-${BZIP2_VERSION}.tar.gz
     (cd bzip2-${BZIP2_VERSION} \
         && make -f Makefile-libbz2_so \
         && make install PREFIX=$BUILD_PREFIX)
@@ -343,20 +345,24 @@ function build_pcre {
 }
 
 function build_swig {
+    if [ -e swig-stamp ]; then return; fi
     if [ -n "$IS_OSX" ]; then
         brew install swig > /dev/null
     else
         build_pcre
         build_simple swig $SWIG_VERSION https://prdownloads.sourceforge.net/swig
     fi
+    touch swig-stamp
 }
 
 function build_suitesparse {
+    if [ -e suitesparse-stamp ]; then return; fi
     if [ -n "$IS_OSX" ]; then
         brew install suite-sparse > /dev/null
     else
         yum install -y suitesparse-devel > /dev/null
     fi
+    touch suitesparse-stamp
 }
 
 function build_libtool {

tests/test_library_builders.sh

@@ -17,8 +17,8 @@ suppress build_swig
 # E.g. arb (below) requires a couple of other libraries.
 # Run here just for the output, even though they fail.
 (set +e ;
-    build_github fredrik-johansson/arb 2.13.0 ;
-    build_github glennrp/libpng v1.6.34 ;
+    build_github fredrik-johansson/arb 2.14.0 ;
+    build_github glennrp/libpng v1.6.35 ;
     build_github wbhart/mpir mpir-3.0.0
     )
 suppress build_flex