dnrops
/
llama.swift
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
86 Commits 2 Branches 3 Tags 2.5 MiB
Commit Graph

86 Commits

Author SHA1 Message Date
Alex Rozanski 713b65df3e add back Makefile and quantize.cpp in correct locations 2023-03-16 02:30:13 +01:00
Alex Rozanski e0d648c817 remove quantize.cpp 2023-03-16 01:30:35 +01:00
Alex Rozanski 058377cb3e add SPM support 2023-03-16 01:17:44 +01:00
Alex Rozanski 5f4905a256 separate Swift and C++/Objective-C++ code into separate libraries 2023-03-16 00:24:12 +01:00
Alex Rozanski b44026b8db remove Makefile and CMakeLists.txt 
will pull in any changes from llama.cpp and adjust the Xcode configuration
to match this.
 2023-03-15 10:49:00 +01:00
Alex Rozanski 56c19351db Merge remote-tracking branch 'llama.cpp/master' 2023-03-15 10:36:15 +01:00
Ronsor 47857e564c
Don't use vdotq_s32 if it's not available (#139) 
* Don't use vdotq_s32 if it's not available

`dotprod` extensions aren't available on some ARM CPUs (e.g. Raspberry Pi 4), so check for them and only use them if they're available.

Reintroduces the code removed in 84d9015 if `__ARM_FEATURE_DOTPROD` isn't defined.

* Update ggml.c

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-14 21:34:37 +02:00
Radoslav Gerganov 60f819a2b1
Add section to README on how to run the project on Android (#130) 2023-03-14 15:30:08 +02:00
Alex Rozanski 8ce62d4135 improve README 2023-03-14 12:03:12 +01:00
Alex Rozanski 27afd915b8 update license file 2023-03-14 11:53:49 +01:00
Alex Rozanski f5779567ed update readme 2023-03-14 11:52:34 +01:00
Alex Rozanski e69e116b82 improve output 2023-03-14 11:45:59 +01:00
Alex Rozanski e99e6f1fbf make sample app interactive 2023-03-14 11:45:59 +01:00
Alex Rozanski 81e97c8585 remove prints to stderr and return NSErrors 2023-03-14 11:45:59 +01:00
Alex Rozanski 2b27f14035 remove debug printf() statements 2023-03-14 11:45:59 +01:00
Alex Rozanski 60458cc580 improve swift API and test app 2023-03-14 11:45:59 +01:00
Alex Rozanski e3df7bb91f add basic error handling 2023-03-14 11:45:59 +01:00
Alex Rozanski e04b99370c pass events from C++ through to Swift 2023-03-14 11:45:59 +01:00
Alex Rozanski b3404f6d27 execute prediction in NSOperation 2023-03-14 11:45:59 +01:00
Alex Rozanski 95722fb0e9 support reverse prompt in config 2023-03-14 11:45:59 +01:00
Alex Rozanski 9e54dc6d56 wire up LlamaRunner to cpp code 2023-03-14 11:45:59 +01:00
Alex Rozanski 406470dd3b move cpp files to separate directory 2023-03-14 11:45:59 +01:00
Alex Rozanski f33e7dbed3 fix file check in llamaTest 2023-03-14 11:45:59 +01:00
Alex Rozanski 2717aee3c1 add basic Objective-C runner and pass model path 2023-03-14 11:45:59 +01:00
Alex Rozanski 7e27bbc02e add basic Xcode project and include cpp files 2023-03-14 11:45:59 +01:00
Georgi Gerganov 97ab2b2578
Add Misc section + update hot topics + minor fixes 2023-03-14 09:43:52 +02:00
Sebastián A 2f700a2738
Add windows to the CI (#98) 2023-03-13 22:29:10 +02:00
Georgi Gerganov c09a9cfb06
CMake build in Release by default (#75) 2023-03-13 21:22:15 +02:00
Georgi Gerganov 7ec903d3c1
Update contribution section, hot topics, limitations, etc. 2023-03-13 19:21:51 +02:00
Georgi Gerganov 4497ad819c
Print system information 2023-03-13 19:15:08 +02:00
Sebastián A ed6849cc07
Initial support for CMake (#75) 2023-03-13 19:12:33 +02:00
Thomas Klausner 41be0a3b3d
Add NetBSD support. (#90) 2023-03-13 18:40:54 +02:00
Pavol Rusnak 671d5cac15
Use fprintf for diagnostic output (#48) 
keep printf only for printing model output

one can now use ./main ... 2>dev/null to suppress any diagnostic output
 2023-03-13 18:39:56 +02:00
Georgi Gerganov 84d9015c4a
Use vdotq_s32 to improve performance (#67) 
* 10% performance boost on ARM

* Back to original change
 2023-03-13 18:36:44 +02:00
uint256_t 63fd76fbb0
Reduce model loading time (#43) 
* Use buffering

* Use vector

* Minor

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-13 18:33:43 +02:00
Val Kharitonov 2a20f48efa
Fix UTF-8 handling (including colors) (#79) 2023-03-13 18:24:18 +02:00
Pavol Rusnak d1f224712d
Add quantize script for batch quantization (#92) 
* Add quantize script for batch quantization

* Indentation

* README for new quantize.sh

* Fix script name

* Fix file list on Mac OS

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-13 18:15:20 +02:00
Georgi Gerganov 1808ee0500
Add initial contribution guidelines 2023-03-13 09:42:26 +02:00
Matvey Soloviev a169bb889c Gate signal support on being on a unixoid system. (#74) 2023-03-13 04:08:01 +01:00
Matvey Soloviev 460c482540 Fix token count accounting 2023-03-13 01:04:41 +01:00
Georgi Gerganov c80e2a8f2a
Revert "10% performance boost on ARM" 
This reverts commit 113a9e83eb.

There are some reports for illegal instruction.
Moved this stuff to vdotq_s32 branch until resolve
 2023-03-13 01:28:08 +02:00
Georgi Gerganov 54a0e66ea0
Check for vdotq_s32 availability 2023-03-13 01:21:03 +02:00
Georgi Gerganov 543c57e991
Ammend to previous commit - forgot to update non-QRDMX branch 2023-03-13 01:05:24 +02:00
Georgi Gerganov 113a9e83eb
10% performance boost on ARM 2023-03-13 00:56:10 +02:00
Matvey Soloviev 404fac0d62
Fix color getting reset before prompt output done (#65) 
(cherry picked from commit 7eb2987619feee04c40eff69b604017d09919cb6)
 2023-03-13 00:07:34 +02:00
Georgi Gerganov 1a0a74300f
Update README.md 2023-03-12 23:39:01 +02:00
Matvey Soloviev 96ea727f47
Add interactive mode (#61) 
* Initial work on interactive mode.

* Improve interactive mode. Make rev. prompt optional.

* Update README to explain interactive mode.

* Fix OS X build
 2023-03-12 23:13:28 +02:00
Marc Köhlbrugge 9661954835
Fix typo in README (#45) 2023-03-12 22:30:08 +02:00
Ben Garney f385f8dee8
Allow using prompt files (#59) 2023-03-12 22:28:36 +02:00
beiller 02f0c6fe7f
Add back top_k (#56) 
* Add back top_k

* Update utils.cpp

* Update utils.h

---------

Co-authored-by: Bill Hamilton <bill.hamilton@shopify.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-12 22:23:15 +02:00