maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/mlir/cmake/modules/MLIRDetectPythonEnv.cmake

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

71 lines
3.3 KiB
CMake
Raw Normal View History

[mlir][Python] Support finding pybind11 from the python environment. * Makes `pip install pybind11` do the right thing with no further config. * Since we now require a version of pybind11 greater than many LTS OS installs (>=2.6), a more convenient way to get a recent version is preferable. * Also adds the version spec to find_package so it will skip older versions that may be lying around. * Tested the full matrix of old system install, no system install, pip install and no pip install. Differential Revision: https://reviews.llvm.org/D91903
2020-11-21 09:57:46 +08:00
# Macros and functions related to detecting details of the Python environment.
[mlir][python] Add a warning for cmake version < 3.15. As discussed on discord, we have never actually been able to build with the project-wide published min version of 3.14.3. The buildbot that tests the Python configuration is currently pinned to 3.19.1, and there are a number of non-version/policy controlled features that Python building relies on that makes it unreliable with older versions. Some of the issues are pretty fundamental and I don't know how to do them on the older version. I think that, as an optional feature, at least advertising the PSA as in this patch is a good middle ground until the next project-wide CMake version bump. Also moves setup logic to a macro so that everyone can use it.
2021-10-13 10:32:48 +08:00
# Finds and configures python packages needed to build MLIR Python bindings.
macro(mlir_configure_python_dev_packages)
if(CMAKE_VERSION VERSION_LESS "3.15.0")
message(SEND_ERROR
"Building MLIR Python bindings is known to rely on CMake features "
"that require at least version 3.15. Recommend upgrading to 3.18+ "
"for full support. Detected current version: ${CMAKE_VERSION}")
endif()
# After CMake 3.18, we are able to limit the scope of the search to just
# Development.Module. Searching for Development will fail in situations where
# the Python libraries are not available. When possible, limit to just
# Development.Module.
# See https://pybind11.readthedocs.io/en/stable/compiling.html#findpython-mode
if(CMAKE_VERSION VERSION_LESS "3.18.0")
message(WARNING
"This version of CMake is not compatible with statically built Python "
"installations. If Python fails to detect below this may apply to you. "
"Recommend upgrading to at least CMake 3.18. "
"Detected current version: ${CMAKE_VERSION}"
)
set(_python_development_component Development)
else()
[MLIR] Workaround for python detection problems. Although cmake should be platform-independent, we've observed that some aspects of Python detection don't work on all platforms, even with recent versions of cmake. This appears to be due to bugs in the python detection logic, especially when the NumPy component is required and not located within the python installation. As a workaround, this patch first searches for "Development" before searching for "Development.Module", which seems to workaround the issue. Differential Revision: https://reviews.llvm.org/D118148
2022-01-25 23:53:48 +08:00
# Prime the search for python to see if there is a full
# development package. This seems to work around cmake bugs
# searching only for Development.Module in some environments.
find_package(Python3 ${LLVM_MINIMUM_PYTHON_VERSION}
COMPONENTS Development)
[mlir][python] Add a warning for cmake version < 3.15. As discussed on discord, we have never actually been able to build with the project-wide published min version of 3.14.3. The buildbot that tests the Python configuration is currently pinned to 3.19.1, and there are a number of non-version/policy controlled features that Python building relies on that makes it unreliable with older versions. Some of the issues are pretty fundamental and I don't know how to do them on the older version. I think that, as an optional feature, at least advertising the PSA as in this patch is a good middle ground until the next project-wide CMake version bump. Also moves setup logic to a macro so that everyone can use it.
2021-10-13 10:32:48 +08:00
set(_python_development_component Development.Module)
endif()
find_package(Python3 ${LLVM_MINIMUM_PYTHON_VERSION}
COMPONENTS Interpreter ${_python_development_component} NumPy REQUIRED)
unset(_python_development_component)
message(STATUS "Found python include dirs: ${Python3_INCLUDE_DIRS}")
message(STATUS "Found python libraries: ${Python3_LIBRARIES}")
message(STATUS "Found numpy v${Python3_NumPy_VERSION}: ${Python3_NumPy_INCLUDE_DIRS}")
mlir_detect_pybind11_install()
[mlir] Rework subclass construction in PybindAdaptors.h The constructor function was being defined without indicating its "__init__" name, which made it interpret it as a regular fuction rather than a constructor. When overload resolution failed, Pybind would attempt to print the arguments actually passed to the function, including "self", which is not initialized since the constructor couldn't be called. This would result in "__repr__" being called with "self" referencing an uninitialized MLIR C API object, which in turn would cause undefined behavior when attempting to print in C++. Even if the correct name is provided, the mechanism used by PybindAdaptors.h to bind constructors directly as "__init__" functions taking "self" is deprecated by Pybind. The new mechanism does not seem to have access to a fully-constructed "self" object (i.e., the constructor in C++ takes a `pybind11::detail::value_and_holder` that cannot be forwarded back to Python). Instead, redefine "__new__" to perform the required checks (there are no additional initialization needed for attributes and types as they are all wrappers around a C++ pointer). "__new__" can call its equivalent on a superclass without needing "self". Bump pybind11 dependency to 3.8.0, which is the first version that allows one to redefine "__new__". Reviewed By: stellaraccident Differential Revision: https://reviews.llvm.org/D117646
2022-01-19 19:21:42 +08:00
find_package(pybind11 2.8 CONFIG REQUIRED)
[mlir][python] Add a warning for cmake version < 3.15. As discussed on discord, we have never actually been able to build with the project-wide published min version of 3.14.3. The buildbot that tests the Python configuration is currently pinned to 3.19.1, and there are a number of non-version/policy controlled features that Python building relies on that makes it unreliable with older versions. Some of the issues are pretty fundamental and I don't know how to do them on the older version. I think that, as an optional feature, at least advertising the PSA as in this patch is a good middle ground until the next project-wide CMake version bump. Also moves setup logic to a macro so that everyone can use it.
2021-10-13 10:32:48 +08:00
message(STATUS "Found pybind11 v${pybind11_VERSION}: ${pybind11_INCLUDE_DIR}")
message(STATUS "Python prefix = '${PYTHON_MODULE_PREFIX}', "
"suffix = '${PYTHON_MODULE_SUFFIX}', "
"extension = '${PYTHON_MODULE_EXTENSION}")
endmacro()
[mlir][Python] Support finding pybind11 from the python environment. * Makes `pip install pybind11` do the right thing with no further config. * Since we now require a version of pybind11 greater than many LTS OS installs (>=2.6), a more convenient way to get a recent version is preferable. * Also adds the version spec to find_package so it will skip older versions that may be lying around. * Tested the full matrix of old system install, no system install, pip install and no pip install. Differential Revision: https://reviews.llvm.org/D91903
2020-11-21 09:57:46 +08:00
# Detects a pybind11 package installed in the current python environment
# and sets variables to allow it to be found. This allows pybind11 to be
# installed via pip, which typically yields a much more recent version than
# the OS install, which will be available otherwise.
function(mlir_detect_pybind11_install)
if(pybind11_DIR)
message(STATUS "Using explicit pybind11 cmake directory: ${pybind11_DIR} (-Dpybind11_DIR to change)")
else()
[mlir] use STATUS instead of CHECK_* in MLIRDetectPythonEnv.cmake CHECK_* directives for message() where added in Cmake 3.17, LLVM requires 3.14 as minimum so they may not be intepreted correctly and just print "CHECK_*" into the message stream. Replace them with STATUS. Reviewed By: stellaraccident Differential Revision: https://reviews.llvm.org/D91959
2020-11-23 20:58:03 +08:00
message(STATUS "Checking for pybind11 in python path...")
[mlir][Python] Support finding pybind11 from the python environment. * Makes `pip install pybind11` do the right thing with no further config. * Since we now require a version of pybind11 greater than many LTS OS installs (>=2.6), a more convenient way to get a recent version is preferable. * Also adds the version spec to find_package so it will skip older versions that may be lying around. * Tested the full matrix of old system install, no system install, pip install and no pip install. Differential Revision: https://reviews.llvm.org/D91903
2020-11-21 09:57:46 +08:00
execute_process(
[mlir][Python] Fix the last remaining instance of PYTHON_EXECUTABLE. * Was causing auto-detect of pybind11 to fail on clean configure. Differential Revision: https://reviews.llvm.org/D92043
2020-11-25 01:50:18 +08:00
COMMAND "${Python3_EXECUTABLE}"
[mlir][Python] Support finding pybind11 from the python environment. * Makes `pip install pybind11` do the right thing with no further config. * Since we now require a version of pybind11 greater than many LTS OS installs (>=2.6), a more convenient way to get a recent version is preferable. * Also adds the version spec to find_package so it will skip older versions that may be lying around. * Tested the full matrix of old system install, no system install, pip install and no pip install. Differential Revision: https://reviews.llvm.org/D91903
2020-11-21 09:57:46 +08:00
-c "import pybind11;print(pybind11.get_cmake_dir(), end='')"
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
RESULT_VARIABLE STATUS
OUTPUT_VARIABLE PACKAGE_DIR
ERROR_QUIET)
if(NOT STATUS EQUAL "0")
[mlir] use STATUS instead of CHECK_* in MLIRDetectPythonEnv.cmake CHECK_* directives for message() where added in Cmake 3.17, LLVM requires 3.14 as minimum so they may not be intepreted correctly and just print "CHECK_*" into the message stream. Replace them with STATUS. Reviewed By: stellaraccident Differential Revision: https://reviews.llvm.org/D91959
2020-11-23 20:58:03 +08:00
message(STATUS "not found (install via 'pip install pybind11' or set pybind11_DIR)")
[mlir][Python] Support finding pybind11 from the python environment. * Makes `pip install pybind11` do the right thing with no further config. * Since we now require a version of pybind11 greater than many LTS OS installs (>=2.6), a more convenient way to get a recent version is preferable. * Also adds the version spec to find_package so it will skip older versions that may be lying around. * Tested the full matrix of old system install, no system install, pip install and no pip install. Differential Revision: https://reviews.llvm.org/D91903
2020-11-21 09:57:46 +08:00
return()
endif()
[mlir] use STATUS instead of CHECK_* in MLIRDetectPythonEnv.cmake CHECK_* directives for message() where added in Cmake 3.17, LLVM requires 3.14 as minimum so they may not be intepreted correctly and just print "CHECK_*" into the message stream. Replace them with STATUS. Reviewed By: stellaraccident Differential Revision: https://reviews.llvm.org/D91959
2020-11-23 20:58:03 +08:00
message(STATUS "found (${PACKAGE_DIR})")
[mlir][Python] Support finding pybind11 from the python environment. * Makes `pip install pybind11` do the right thing with no further config. * Since we now require a version of pybind11 greater than many LTS OS installs (>=2.6), a more convenient way to get a recent version is preferable. * Also adds the version spec to find_package so it will skip older versions that may be lying around. * Tested the full matrix of old system install, no system install, pip install and no pip install. Differential Revision: https://reviews.llvm.org/D91903
2020-11-21 09:57:46 +08:00
set(pybind11_DIR "${PACKAGE_DIR}" PARENT_SCOPE)
endif()
endfunction()