dnrops
/
candle
mirror of https://github.com/huggingface/candle.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Recommend using maturin. (#717)

This commit is contained in:
Laurent Mazare 2023-09-02 17:19:35 +02:00 committed by GitHub
parent ad796eb4be
commit 21109e1983
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 5 additions and 48 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
Makefile
View File

@ -1,14 +1,4 @@
pyo3-test:
cargo build --profile=release-with-debug --package candle-pyo3
cp -f ./target/release-with-debug/libcandle.so candle.so
PYTHONPATH=.:$PYTHONPATH python3 candle-pyo3/test.py
pyo3-test-macos:
cargo build --profile=release-with-debug --package candle-pyo3
cp -f ./target/release-with-debug/libcandle.dylib candle.so
PYTHONPATH=.:$PYTHONPATH python3 candle-pyo3/test.py
.PHONY: pyo3-test pyo3-test-macos clean-ptx clean test
.PHONY: clean-ptx clean test
clean-ptx:
find target -name "*.ptx" -type f -delete

12
candle-pyo3/README.md
View File

@ -1,11 +1,7 @@
From the top level directory run the following for linux.
```
cargo build --profile=release-with-debug --package candle-pyo3 && cp -f ./target/release-with-debug/libcandle.so candle.so
PYTHONPATH=. python3 candle-pyo3/test.py
```bash
From the `candle-pyo3` directory, enable a virtual env where you will want the
candle package to be installed then run.
Or for macOS users:
```bash
cargo build --profile=release-with-debug --package candle-pyo3 && cp -f ./target/release-with-debug/libcandle.dylib candle.so
PYTHONPATH=. python3 candle-pyo3/test.py
maturin develop
python test.py
```

14
candle-pyo3/quant-llama.py
View File

@ -1,19 +1,5 @@
# This example shows how the candle Python api can be used to replicate llama.cpp.
import os
import sys
# The "import candle" statement below works if there is a "candle.so" file in sys.path.
# Here we check for shared libraries that can be used in the build directory.
BUILD_DIR = "./target/release-with-debug"
so_file = BUILD_DIR + "/candle.so"
if os.path.islink(so_file): os.remove(so_file)
for lib_file in ["libcandle.dylib", "libcandle.so"]:
lib_file_ = BUILD_DIR + "/" + lib_file
if os.path.isfile(lib_file_):
os.symlink(lib_file, so_file)
sys.path.insert(0, BUILD_DIR)
break
import candle
MAX_SEQ_LEN = 4096

15
candle-pyo3/test.py
View File

@ -1,18 +1,3 @@
import os
import sys
# The "import candle" statement below works if there is a "candle.so" file in sys.path.
# Here we check for shared libraries that can be used in the build directory.
BUILD_DIR = "./target/release-with-debug"
so_file = BUILD_DIR + "/candle.so"
if os.path.islink(so_file): os.remove(so_file)
for lib_file in ["libcandle.dylib", "libcandle.so"]:
lib_file_ = BUILD_DIR + "/" + lib_file
if os.path.isfile(lib_file_):
os.symlink(lib_file, so_file)
sys.path.insert(0, BUILD_DIR)
break
import candle
t = candle.Tensor(42.0)