Unverified Commit 04e209bc authored by Matti Picus's avatar Matti Picus Committed by GitHub
Browse files

Merge pull request #396 from mattip/manylinux_2_24

Support manylinux_2_24
parents 6b3fe1d0 7079e101
......@@ -20,13 +20,19 @@ matrix:
dist: xenial
env:
- TEST_BUILDS=1
# 64-bit builds, manylinux2010
# 64-bit builds
- os: linux
dist: xenial
env:
- TEST_BUILDS=1
- MB_ML_VER=2010
- DOCKER_TEST_IMAGE=multibuild/xenial_{PLAT}
- os: linux
dist: xenial
env:
- TEST_BUILDS=1
- MB_ML_VER=_2_24
- DOCKER_TEST_IMAGE=multibuild/xenial_{PLAT}
# 32-bit builds
- os: linux
dist: xenial
......
......@@ -41,7 +41,8 @@ The Travis CI scripts are designed to build *and test*:
* 64-bit macOS wheels built for macOS 10.9+
* 64/32-bit macOS wheels built for macOS 10.6+
* 64-bit ``manylinuxX_x86_64`` wheels, both narrow and wide Unicode builds, where `X` is any valid Manylinux version, such as `1`, or `2010`
* 64-bit ``manylinuxX_x86_64`` wheels, both narrow and wide Unicode builds,
where `X` is any valid Manylinux version: `1`, `2010`, `2014` or `_2_24`.
* 32-bit ``manylinuxX_i686`` wheels, both narrow and wide Unicode builds
You can currently build and test against Pythons 2.7, 3.5, 3.6, 3.7, 3.8 and 3.9
......@@ -55,7 +56,7 @@ The AppVeyor setup is designed to build *and test*:
* 64-bit Windows ``win_amd64`` wheels
* 32-bit Windows ``win32`` wheels
You can currently build and test against Pythons 2.7, 3.5, 3.6, 3.7, 3.8
You can currently build and test against Pythons 2.7, 3.5, 3.6, 3.7, 3.8, 3.9
*****************
How does it work?
......@@ -91,19 +92,35 @@ shell scripts listed above are available for your build and test.
Build options are controlled mainly by the following environment
variables:
* ``MB_PYTHON_VER`` sets the Python version targetted: ``major.minor.patch`` for CPython, or ``pypy-major.minor`` for PyPy.
* ``MB_PYTHON_OSX_VER`` sets the minimum macOS SDK version for any C extensions. For CPython targets it may be set to 10.6 or 10.9, provided a corresponding Python build is available at `python.org <https://www.python.org/downloads/mac-osx/>`_. It defaults to the highest version available. It's ignored for PyPy targets.
* ``PLAT`` sets the architectures built for any C extensions: ``x86_64`` or ``intel`` for 64-bit or 64/32-bit respectively. It defaults to the same arches as the target Python version: 64-bit for CPython macOS 10.9 or PyPy, and 64/32-bit for CPython 10.6.
In most cases it's best to rely on the defaults for ``MB_PYTHON_OSX_VER`` and ``PLAT``, rather than setting them explicitly. Examples of exceptions to this guideline include:
* setting ``MB_PYTHON_OSX_VER=10.6`` to build a 10.6 64/32-bit CPython wheel for Python 2.7 (default for 2.7 is 10.9 64-bit)
* setting ``MB_PYTHON_OSX_VER=10.6 and PLAT=x86_64`` to build a 10.6 64-bit only wheel (10.6 would normally be 64/32-bit). Such a wheel would still have a platform tag of ``macosx_10_6_intel`` , advertising support for both 64 and 32-bit, but wouldnt work in 32-bit mode. This may be OK given how unlikely it is that there is still anyone actually running Python on macOS in 32-bit mode.
* ``MB_PYTHON_VER`` sets the Python version targetted: ``major.minor.patch``
for CPython, or ``pypy-major.minor`` for PyPy.
* ``MB_PYTHON_OSX_VER`` sets the minimum macOS SDK version for any C
extensions. For CPython targets it may be set to 10.6 or 10.9, provided a
corresponding Python build is available at `python.org
<https://www.python.org/downloads/mac-osx/>`_. It defaults to the highest
version available. It's ignored for PyPy targets.
* ``PLAT`` sets the architectures built for any C extensions: ``x86_64`` or
``intel`` for 64-bit or 64/32-bit respectively. It defaults to the same
arches as the target Python version: 64-bit for CPython macOS 10.9 or PyPy,
and 64/32-bit for CPython 10.6.
In most cases it's best to rely on the defaults for ``MB_PYTHON_OSX_VER`` and
``PLAT``, rather than setting them explicitly. Examples of exceptions to this
guideline include:
* setting ``MB_PYTHON_OSX_VER=10.6`` to build a 10.6 64/32-bit CPython wheel
for Python 2.7 (default for 2.7 is 10.9 64-bit)
* setting ``MB_PYTHON_OSX_VER=10.6 and PLAT=x86_64`` to build a 10.6 64-bit
only wheel (10.6 would normally be 64/32-bit). Such a wheel would still have
a platform tag of ``macosx_10_6_intel`` , advertising support for both 64 and
32-bit, but wouldn't work in 32-bit mode. This may be OK given how unlikely it
is that there is still anyone actually running Python on macOS in 32-bit
mode.
The ``build_wheel`` function builds the wheel, and ``install_run``
function installs and tests it. Look in ``multibuild/common_utils.sh`` for
default definitions of these functions. See below for more details, many of which are common
to macOS and Linux.
default definitions of these functions. See below for more details, many of
which are common to macOS and Linux.
Manylinux
=========
......@@ -122,6 +139,7 @@ variable. The default version is ``1``. Versions that are currently valid are:
* ``2010`` corresponding to manylinux2010 (see `PEP 571 <https://www.python.org/dev/peps/pep-0571>`_).
* ``2014`` corresponding to manylinux2014 and adds more architectures to ``PLAT``
(see `PEP 599 <https://www.python.org/dev/peps/pep-0599>`_).
* ``_2_24`` corresponding to manylinux_2_24 (see `PEP 600 <https://www.python.org/dev/peps/pep-0600>`_).
The environment variable specified which Manylinux docker container you are building in.
......
......@@ -24,7 +24,14 @@ GET_PIP_URL=https://bootstrap.pypa.io/get-pip.py
# with, so it is passed in when calling "docker run" for tests.
UNICODE_WIDTH=${UNICODE_WIDTH:-32}
if [ $(uname) == "Darwin" ]; then IS_MACOS=1; IS_OSX=1; fi
if [ $(uname) == "Darwin" ]; then
IS_MACOS=1; IS_OSX=1;
else
# In the manylinux_2_24 image, based on Debian9, "python" is not installed
# so link in something for the various system calls before PYTHON_EXE is set
which python || export PATH=/opt/python/cp39-cp39/bin:$PATH
fi
# Work round bug in travis xcode image described at
# https://github.com/direnv/direnv/issues/210
......@@ -199,7 +206,15 @@ function untar {
}
function install_rsync {
if [ -z "$IS_MACOS" ]; then
# install rsync via package manager
if [ -n "$IS_MACOS" ]; then
# macOS. The colon in the next line is the null command
:
elif [[ $MB_ML_VER == "_2_24" ]]; then
# debian:9 based distro
[[ $(type -P rsync) ]] || apt-get install -y rsync
else
# centos based distro
[[ $(type -P rsync) ]] || yum_install rsync
fi
}
......
......@@ -47,13 +47,19 @@ else
export CFLAGS="${CFLAGS:-$STRIP_FLAGS}"
export CXXFLAGS="${CXXFLAGS:-$STRIP_FLAGS}"
export FFLAGS="${FFLAGS:-$STRIP_FLAGS}"
if [[ $MB_ML_VER == "1" ]]; then
if [[ $MB_ML_VER == "_2_24" ]]; then
# This is the first opportunity to distinguish between manylinuxes
apt update
if [ "${MB_PYTHON_VERSION:0:4}" == "pypy" ]; then
yum install -y libtool wget
else
yum install -y libtool
# debian:9 based distro
apt install -y wget
fi
elif [[ $MB_ML_VER == "1" ]]; then
# Need libtool, and for pypy need wget
# centos based distro
yum install -y libtool wget
elif [ "${MB_PYTHON_VERSION:0:4}" == "pypy" ]; then
# centos based distro
yum install -y wget
fi
fi
......
......@@ -14,7 +14,7 @@ OPENBLAS_VERSION="${OPENBLAS_VERSION:-0.3.10}"
# We use system zlib by default - see build_new_zlib
ZLIB_VERSION="${ZLIB_VERSION:-1.2.10}"
LIBPNG_VERSION="${LIBPNG_VERSION:-1.6.21}"
BZIP2_VERSION="${BZIP2_VERSION:-1.0.6}"
BZIP2_VERSION="${BZIP2_VERSION:-1.0.7}"
FREETYPE_VERSION="${FREETYPE_VERSION:-2.6.3}"
TIFF_VERSION="${TIFF_VERSION:-4.1.0}"
JPEG_VERSION="${JPEG_VERSION:-9b}"
......@@ -133,7 +133,13 @@ function build_zlib {
# Gives an old but safe version
if [ -n "$IS_MACOS" ]; then return; fi # OSX has zlib already
if [ -e zlib-stamp ]; then return; fi
if [[ $MB_ML_VER == "_2_24" ]]; then
# debian:9 based distro
apt-get install -y zlib1g-dev
else
#centos based distro
yum_install zlib-devel
fi
touch zlib-stamp
}
......@@ -180,10 +186,14 @@ function get_modern_cmake {
local cmake=cmake
if [ -n "$IS_MACOS" ]; then
brew install cmake > /dev/null
elif [[ $MB_ML_VER == "_2_24" ]]; then
# debian:9 based distro
apt-get install -y cmake
else
if [ "`yum search cmake | grep ^cmake28\.`" ]; then
cmake=cmake28
fi
# centos based distro
yum_install $cmake > /dev/null
fi
echo $cmake
......@@ -396,7 +406,11 @@ function build_suitesparse {
if [ -e suitesparse-stamp ]; then return; fi
if [ -n "$IS_MACOS" ]; then
brew install suite-sparse > /dev/null
elif [[ $MB_ML_VER == "_2_24" ]]; then
# debian:9 based distro
apt-get install -y libsuitesparse-dev > /dev/null
else
# centos based distro
yum_install suitesparse-devel > /dev/null
fi
touch suitesparse-stamp
......
#!/bin/bash
# Useful utilities common across manylinux1 builds
# Useful utilities common across manylinux builds
MULTIBUILD_DIR=$(dirname "${BASH_SOURCE[0]}")
source $MULTIBUILD_DIR/common_utils.sh
......@@ -44,7 +44,13 @@ function activate_ccache {
ln -s /parent-home/.ccache $HOME/.ccache
# Now install ccache
if [[ $MB_ML_VER == "_2_24" ]]; then
# debian:9 based distro
suppress apt-get install -y ccache
else
# centos based distro
suppress yum_install ccache
fi
# Create fake compilers and prepend them to the PATH
# Note that yum is supposed to create these for us,
......@@ -60,6 +66,7 @@ function activate_ccache {
# Prove to the developer that ccache is activated
echo "Using C compiler: $(which gcc)"
}
function yum_install {
# CentOS 5 yum doesn't fail in some cases, e.g. if package is not found
# https://serverfault.com/questions/694942/yum-should-error-when-a-package-is-not-available
......
......@@ -3,8 +3,9 @@
source manylinux_utils.sh
source tests/utils.sh
suppress yum_install rsync && suppress yum erase -y rsync \
if [[ $MB_ML_VER != "_2_24" ]]; then
suppress yum_install rsync && suppress yum erase -y rsync \
|| ingest "yum_install valid"
suppress bash -c '! yum_install nonexistent' || ingest "yum_install nonexistent"
barf
\ No newline at end of file
suppress bash -c '! yum_install nonexistent' || ingest "yum_install nonexistent"
barf
fi
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment