• Hemil Desai's avatar
    Fix `build_tools` missing from sdist causing `uv` cached installs to fail (#2684) · fa68781c
    Hemil Desai authored
    
    
    - Include `build_tools/` in the source distribution via `MANIFEST.in` so that cached builds from `uv` (and `pip`) can resolve `setup.py`'s top-level imports
    
    `setup.py` imports from `build_tools` at the top level:
    
    ```python
    from build_tools.build_ext import CMakeExtension, get_build_ext
    from build_tools.te_version import te_version
    from build_tools.utils import cuda_archs, cuda_version, ...
    ```
    
    The `__legacy__` build backend in `pyproject.toml` adds the source root to `sys.path`, so these imports work when building directly from the source tree. However, `build_tools/` is not included in the sdist because:
    
    1. `MANIFEST.in` did not list it
    2. `build_tools/` is not discovered by `find_packages()` (it's a standalone directory at the repo root, not under `transformer_engine/`)
    
    When `uv` caches the sdist and later builds a wheel from it, the sdist is extracted to a temporary directory where `build_tools/` is absent, causing a `ModuleNotFoundError`. Passing `--no-cache` to `uv` works around this by forcing a fresh build from the full source tree.
    
    Added `build_tools` to `MANIFEST.in`:
    
    ```diff
     recursive-include transformer_engine/common/include *.*
    +recursive-include build_tools *.py *.txt
    ```
    
    - [x] `python setup.py sdist` produces a tarball that contains `build_tools/`
    
    ```
    $ tar tzf dist/transformer_engine-*.tar.gz | grep build_tools
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/VERSION.txt
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/__init__.py
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/build_ext.py
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/jax.py
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/pytorch.py
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/te_version.py
    transformer_engine-2.13.0.dev0+82f7ebeb/build_tools/utils.py
    ```
    Signed-off-by: default avatarHemil Desai <hemild@nvidia.com>
    Co-authored-by: default avatarClaude Opus 4.6 <noreply@anthropic.com>
    fa68781c
MANIFEST.in 97 Bytes