dnrops
/
candle
mirror of https://github.com/huggingface/candle.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
candle/candle-nn
History
Laurent Mazare d01207dbf3
Add a RotatingKVCache. (#2493) 
* Add a RotatingKVCache.

* Add some KvCache tests.

* Test the reset too.

* More kv-cache testing.

* More tests for the rotating kv-cache.

* Improve the api for the rotating cache so that the whole src tensor gets returned when it's overlarge.

* Handle contiguity + bugfix + use in mimi.

* Add a way to test the mimi streaming mode.

* Mimi streaming fixes.

* More rotating kv-cache.

* Fix the attn mask generation.

* Handle the abs case.

* Add some tests for the generated mask.
 		2024-09-23 13:14:32 +02:00
..
benches add where_cond f32 for metal (#2236) 2024-06-02 14:30:06 +02:00
examples Optimize the cat operation on contiguous tensors (#1855) 2024-03-17 10:49:13 +01:00
src Add a RotatingKVCache. (#2493) 2024-09-23 13:14:32 +02:00
tests Add a RotatingKVCache. (#2493) 2024-09-23 13:14:32 +02:00
Cargo.toml Add benchmarks for the candle-nn package (#1995) 2024-04-03 07:03:54 +02:00
README.md Add some missing readme files. (#304) 2023-08-02 10:57:12 +01:00

README.md

candle-nn