Commit Graph

1016 Commits

Author SHA1 Message Date
Dilshod Tadjibaev 0368409eb3
Add support for loading PyTorch `.pt` (weights/states) files directly to model's record (#1085) 2024-01-25 10:20:09 -05:00
unrenormalizable 4ca3e31601
Fixing jupyter nbs + ONNX IR (#1170) 2024-01-24 13:32:01 -05:00
Guillaume Lagrange e1d873abe2
Add Rust concept notes to Burn Book (#1169) 2024-01-24 12:58:14 -05:00
Joshua Ferguson 3b7d9feede
Elementwise pow op (#1133) 2024-01-24 09:46:57 -05:00
Sylvain Benner 707b61051c
Fix doc build on docs.rs (#1168) 2024-01-23 14:13:57 -05:00
Nathaniel Simard eaa4dc3207
Feat/recorder/custom device (#1165) 2024-01-23 13:05:41 -05:00
Guillaume Lagrange e9d1656687
Add model saving and loading examples (#1164) 2024-01-23 12:00:04 -05:00
Nathaniel Simard c9cc1321b6
Update wgpu version (#1166) 2024-01-23 09:50:51 -05:00
Nathaniel Simard 49e49c9bac
Support more elemwise kernel fusion (#1159) 2024-01-22 16:19:20 -05:00
Nathaniel Simard 635cc0a3e7
[burn-fusion] overlapping optimization builders (#1162) 2024-01-22 16:18:59 -05:00
Guillaume Lagrange 191877f14a
New default recorder (#1161)
* Change DefaultFileRecorder to NamedMpkFileRecorder (no compression)

* Actually, safetensors does not have any checksum for data validation

* Update checksum explainer/recommandation
2024-01-22 12:50:42 -05:00
Guillaume Lagrange 21772a2662
Fix FullPrecisionSettings type for integers (#1163) 2024-01-22 12:08:56 -05:00
James MacGlashan a752cc172c
Allow raw tensors to be serialized/deserialized directly with serde (#1041) 2024-01-20 11:25:49 -05:00
Sylvain Benner 9063701b53
Remove annoying cargo warning (#1108)
Package level profiles are ignored when workspace is enabled so
these lines are useless.

This commit removes them completely as panic, lto and rpath cannot be overridden from the workspace root.

Source: https://doc.rust-lang.org/cargo/reference/profiles.html#overrides

```
The precedence for which value is used is done in the following order (first match wins):

[profile.dev.package.name] — A named package.
[profile.dev.package."*"] — For any non-workspace member.
[profile.dev.build-override] — Only for build scripts, proc macros, and their dependencies.
[profile.dev] — Settings in Cargo.toml.
Default values built-in to Cargo.
Overrides cannot specify the panic, lto, or rpath settings.
```
2024-01-20 09:09:20 -05:00
Aasheesh Singh e3355524a2
Example/regression (#1150) 2024-01-19 21:28:38 -05:00
Nathaniel Simard ebb2306a6b
[burn-fusion] support multiple concurrent optimization streams (#1149) 2024-01-18 13:15:43 -05:00
Nathaniel Simard f9b07878c2
Support generic module (#1147) 2024-01-18 12:02:48 -05:00
wcshds a5bdf38c92
fix the problem of sigmoid gradient generating NaN (#1140)
* use sigmoid derivative formulas

* add test

* fix test error

* move sigmoid to tensor/ops/activation.rs

* use full precision in the default implementation

* rename the param of `sigmoid_backward`
2024-01-16 16:20:18 -05:00
Nathaniel Simard b99726f804
[burn-fusion] save all execution plans for any trigger (#1143) 2024-01-16 14:02:42 -05:00
Guillaume Lagrange 6079f98950
Fix missing ticks in Burn book and remove unused example dependency (#1144)
* Add missing ticks for triu and tril ops

* Removed example unused dependency
2024-01-16 09:03:17 -05:00
Nathaniel Simard 0a6aa5ff64
Fix: Records no longer require Debug and Clone (#1137) 2024-01-15 10:02:28 -05:00
Nathaniel Simard 0a85744d9b
Refactor/fusion stream (#1135) 2024-01-13 14:17:43 -05:00
Guillaume Lagrange 76c935861c
Add fan_out to Conv2dConfig::init (#1138) 2024-01-12 16:02:05 -05:00
Sylvain Benner 9bd2d7b7d4
Refactor serialization of backend-comparison benchmarks (#1131)
* Refactor serialization of benchmarks

* flatten benchmarks data to make it easier to save documents to a database and
query them
* split some information into their own fields like backend and device
* add new seralized info:
  - computed values (mean, median, variance, min, max)
  - number of samples
  - operation name
  - tensor shapes if any
* serialize to separate files, one file per benchmark run
* simplify persistence module to only a save method

* Update bench save file format to use name and uuid

* Compute serialized fields count automatically via a macro

* Rework naming of benchmarks, shapes and add options field

Remove operations field
Correctly create one file per ran benchmark

* Serialize benchmark num_repeats

* Fix expect message to follow the 'should' convention

* Cargo fmt :-)

* Make Clippy happy

* Save files in the burn subdirectory

* Change name of custom_gelu bench to just gelu

* Remove num_repeats from backend-comparison benchmarks

* Fix wrong variable name to compute the median

* Remove false positive possibility in test_mean_duration
2024-01-12 11:15:00 -05:00
Sylvain Benner 6983cede31
Update CONTRIBUTING.md with code guidelines (#1134) 2024-01-12 09:29:57 -05:00
Shahar Papini 76c1632b8e
Move transformer prenorm on the residual path (#1054) 2024-01-12 08:39:17 -05:00
Guillaume Lagrange 535458e7b9
Add example for custom CSV dataset (#1129) 2024-01-11 09:24:25 -05:00
Nathaniel Simard f43b686366
Feat/fusion inplace (#1128) 2024-01-10 12:37:17 -05:00
Guillaume Lagrange 15c2f3b7a1
Fix small typos and dead links in README and burn-book (#1127) 2024-01-10 08:57:28 -05:00
Nathaniel Simard de5f93220f
Wgpu fusion auto-vectorized operations (#1123) 2024-01-08 16:58:39 -05:00
Louis Fortier-Dubois ab67b6b036
slice assign in candle (#1095) 2024-01-08 16:41:34 -05:00
Mykolas Peteraitis e132e21816
Small fix: documentation of Tensor::stack (#1105) 2024-01-08 08:38:27 -05:00
Sylvain Benner 3c056a41e3
[CI] Disable unused windows steps since wgpu tests are skipped (#1107) 2024-01-06 15:19:03 -05:00
Nathaniel Simard f0e10f1499
Refactor/fusion optimization (#1104) 2024-01-06 14:22:50 -05:00
Sylvain Benner 220ce251e0
Fix burn-compute cargo build command line in no-std (#1109)
This allows to copy/paste the output of the cargo build command
line.
2024-01-06 13:37:08 -05:00
Kirill Mavreshko 97297538b1
Remove _devauto fuctions (#518) (#1110) 2024-01-06 13:36:34 -05:00
Sylvain Benner fab344c565
[burn-compute] Add persistent cache for autotune in std environment (#1087)
* Add new persistant cache to tune cache

* Serialize autotune presistent cache using vectors

* Properly load and save the persistent cachegf

* Print an error when autotune cache cannot be loaded

* Add tests for persistent cache

Use the same logic as the already implemented tests

* Cargo fmt

* Silence clippy check about implementing default for CliMetricsRenderer

* Add burn-compute feature flag autotune-persistent-cache

This allow burn-compute to remain no-std compliant

* debug

* Git ignore .dir-locals.el files

* Update documentation for compute_checksum implementation

* Expect messages should be an expectation not an  error message

* Replace silent eprintln! by log:warn! macro

* Remove clippy allow attribute

* Fix typos in documentation

* Move creation of additional client into the test fn that requires it

* Create compute clients in test function to test different checksum

* Revert tui as a default feature in burn-train cargo file

* Use structs for autotune cache entries

* Unpack InMemoryCacheEntry for even better readibility

* Remove uneeded checksum_checked field in no-std env

* Make sure that autotune cache directoy exists

* Add test for autotune cache file path creation

* Add prefix device info to autotune cache file

* Use new compute in autotune cache integration tests

This prevents race condition by always reloading the cache fir
each test.

* Move burn-compute rand depdencencie in dev-dependencies

* Avoid creation of formatted msg except in case of actual error

* Fix burn-compute unused code warning in no-std env
2024-01-05 14:19:33 -05:00
Sylvain Benner b17ac2dfba
[CI] Pin version of CI dependencies from wgpu repo (#1120)
* Pin version of CI dependencies from wgpu repo

Fixes seg fault in wgpu instance creation.

* Fix new_devauto dead code warning

* Add a notice for wgpu
2024-01-05 12:47:31 -05:00
Sylvain Benner a4de93a39f
Fix import and type redefinitions in mnist example crate (#1100)
* Remove the double import for WgpuDevice
* Prioritize wgpu backend over the default ndarray when wgpu feature is set

This fixes `cargo bench --festures wgpu` as `--no-default-features` cannot be
used.
2024-01-02 12:47:44 -05:00
unrenormalizable 40ec289a92
Making Embedding.weight public (#1094) 2023-12-22 18:11:38 -05:00
Nathaniel Simard fceb036c6f
Fix: serde dependency (#1091)
* Re-export serde

* Fix

* USe another strategy

* Fix

* Fix

* Update de book
2023-12-22 16:53:34 -05:00
Nathaniel Simard b0a2b30ed1
Fix/wgpu/fusion (#1093)
* Fix fusion

* Real fix

* Fix
2023-12-22 16:53:19 -05:00
Louis Fortier-Dubois b070706310
Fix/wgpu/tanh (#1090) 2023-12-21 14:12:49 -05:00
Nathaniel Simard d82e6b157b
Fix tests (#1089)
* Fix tests

* Fix fmt

* Fix CI
2023-12-21 13:06:19 -05:00
Kirill Mavreshko 1fd07fcb4a
Explicit device tensors (#1081) 2023-12-20 17:49:59 -05:00
Kelvin Wu 7c6f017c98
Implement chunk for different backends (#1032) 2023-12-20 13:35:59 -05:00
pacowong c1cb77ac2e
Add mish and softplus activation function (#1071) 2023-12-20 12:41:23 -05:00
Louis Fortier-Dubois 4fb5b74087
fix multithread progress aggregation in dataloader (#1083) 2023-12-19 16:28:57 -05:00
Nathaniel Simard 75062c51e0
Refactor/wgpu/binary (#1078)
* Refactor binary

* Fix

* Oups

* Preparation for cmp

* Refactor comparison

* Remove templates

* Cleanup

* Cleanup

* Fix typo

* Code review
2023-12-19 14:44:55 -05:00
Nathaniel Simard b5c49c5bf7
Fusion wgpu compilation cache (#1069)
* Refactor fusion in the wgpu backend

* WIP

* Refactor

* WIP

* Fix inplace ops

* Works ish

* Cleanup Output

* Refactoring

* Refactor Clamp

* Cleanup

* Cleanup

* Updates

* Fix CI

* Code review
2023-12-18 12:16:08 -05:00