mirror of https://github.com/vllm-project/vllm
[build] enable existing pytorch (for GH200, aarch64, nightly) (#8713)
This commit is contained in:
parent
c6bd70d772
commit
d4a2ac8302
|
@ -72,6 +72,29 @@ You can also build and install vLLM from source:
|
|||
$ cd vllm
|
||||
$ pip install -e . # This may take 5-10 minutes.
|
||||
|
||||
.. note::
|
||||
|
||||
This will uninstall existing PyTorch, and install the version required by vLLM. If you want to use an existing PyTorch installation, there need to be some changes:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ git clone https://github.com/vllm-project/vllm.git
|
||||
$ cd vllm
|
||||
$ python use_existing_torch.py
|
||||
$ pip install -r requirements-build.txt
|
||||
$ pip install -e . --no-build-isolation
|
||||
|
||||
The differences are:
|
||||
|
||||
- ``python use_existing_torch.py``: This script will remove all the PyTorch versions in the requirements files, so that the existing PyTorch installation will be used.
|
||||
- ``pip install -r requirements-build.txt``: You need to manually install the requirements for building vLLM.
|
||||
- ``pip install -e . --no-build-isolation``: You need to disable build isolation, so that the build system can use the existing PyTorch installation.
|
||||
|
||||
This is especially useful when the PyTorch dependency cannot be easily installed via pip, e.g.:
|
||||
|
||||
- build vLLM with PyTorch nightly or a custom PyTorch build.
|
||||
- build vLLM with aarch64 and cuda (GH200), where the PyTorch wheels are not available on PyPI. Currently, only PyTorch nightly has wheels for aarch64 with CUDA. You can run ``pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu124`` to install PyTorch nightly, and then build vLLM on top of it.
|
||||
|
||||
.. note::
|
||||
|
||||
vLLM can fully run only on Linux, but you can still build it on other systems (for example, macOS). This build is only for development purposes, allowing for imports and a more convenient dev environment. The binaries will not be compiled and not work on non-Linux systems. You can create such a build with the following commands:
|
||||
|
|
|
@ -18,7 +18,7 @@ prometheus_client >= 0.18.0
|
|||
prometheus-fastapi-instrumentator >= 7.0.0
|
||||
tiktoken >= 0.6.0 # Required for DBRX tokenizer
|
||||
lm-format-enforcer == 0.10.6
|
||||
outlines >= 0.0.43, < 0.1 # Requires torch >= 2.1.0
|
||||
outlines >= 0.0.43, < 0.1
|
||||
typing_extensions >= 4.10
|
||||
filelock >= 3.10.4 # filelock starts to support `mode` argument from 3.10.4
|
||||
partial-json-parser # used for parsing partial JSON outputs
|
||||
|
|
|
@ -0,0 +1,18 @@
|
|||
import glob
|
||||
|
||||
requires_files = glob.glob('requirements*.txt')
|
||||
requires_files += ["pyproject.toml"]
|
||||
for file in requires_files:
|
||||
print(f">>> cleaning {file}")
|
||||
with open(file, 'r') as f:
|
||||
lines = f.readlines()
|
||||
if "torch" in "".join(lines).lower():
|
||||
print("removed:")
|
||||
with open(file, 'w') as f:
|
||||
for line in lines:
|
||||
if 'torch' not in line.lower():
|
||||
f.write(line)
|
||||
else:
|
||||
print(line.strip())
|
||||
print(f"<<< done cleaning {file}")
|
||||
print()
|
Loading…
Reference in New Issue