Commit Graph

163 Commits

Author SHA1 Message Date
Gadersd 1235b06e25
Improve grammar (#1619) 2024-04-16 09:32:42 -04:00
Sylvain Benner e303e31c8b
Bump next version of Burn to 0.14.0 (#1618) 2024-04-12 17:14:45 -04:00
Guillaume Lagrange 63947d20a2
Fix missing clone derive for MnistBatcher in the book (#1608) 2024-04-12 12:16:28 -04:00
Aasheesh Singh fb1da53a38
support for rotary positional encoding to transformer modules. (#1604)
* add rotary positional encoding to transformer modules.

* fix f64 error

* use num_traits

* add panic condition
2024-04-12 11:45:49 -04:00
Guillaume Lagrange 06ce2b02d6
Fix missing device in custom training loop book example (#1606) 2024-04-12 10:34:04 -04:00
Guillaume Lagrange 0cbe9a927d
Add learner training report summary (#1591)
* Add training report summary

* Fix LossMetric batch size state

* Add NumericEntry de/serialize

* Fix clippy suggestion

* Compact recorder does not use compression (anymore)

* Add learner summary expected results tests

* Add summary to learner builder and automatically display in fit

- Add LearnerSummaryConfig
- Keep track of summary metrics names
- Add model field when displaying from learner.fit()
2024-04-11 12:32:25 -04:00
M S Hrishikesh 80a41b810e
Fixes to code examples in section 5.2 (#1594)
* Fixes to code examples in section 5.2

* A more generic way to get a device for code examples in Burn book section 5.2

* Change run-checks instruction + fix comment spacing

---------

Co-authored-by: hrishim <hrishim@gail.com>
Co-authored-by: Guillaume Lagrange <lagrange.guillaume.1@gmail.com>
2024-04-09 11:12:37 -04:00
Louis Fortier-Dubois f5159b6d22
Refactor: split JitKernel and SourceKernel (#1569)
* refactor execute_dynamic into Execution

* minor change

* extension cfg

* jitkernel and sourcekernel

* add todo statement

* cleanup and docs

* update book

* fix server dependancy on compiler

* refactor into shader information

* refactor to compile shader once

* clippy

* clippy

* clippy

* fix doc

* fix doc

* fmt

* rename feature flag

* refactor

* All broked

* compile at the right time

* todo done

* all dynamic

* all dynamic in template too

* fmt

* fix ci

---------

Co-authored-by: nathaniel <nathaniel.simard.42@gmail.com>
2024-04-05 12:58:10 -04:00
Dilshod Tadjibaev beff9a8c57
Update pytorch-model.md (#1570) 2024-04-02 14:17:51 -05:00
Nathaniel Simard b0c5986d16
Feat/lazy init (#1539) 2024-04-02 10:13:35 -04:00
Dilshod Tadjibaev edc683bc4b
Update module.md (#1557) 2024-03-29 13:14:01 -04:00
Karsten Becker c21d5a3207
Add LeakyReLu implementation (#1208)
* Implement LeakyReLu

* Cargo fmt

* Apply suggestions

* cargo fmt

* Use float_mul_scalar

* Should be grad

* Add to books module

* Move test files

* Update leaky relu to use activation function

* Update tensor.md

* Fix failing test due to approx

* Add back the function comment

* Fix comment per PR feedback

---------

Co-authored-by: Dilshod Tadjibaev <939125+antimora@users.noreply.github.com>
2024-03-27 13:57:51 -05:00
jcmullwh 626457e1c6
Provide Tensor Padding Helpers #960 (#1097)
* Initial padding approach

Create padding implementation for the last two dimensions of Float and Int Tensors.

Create PadMode Enum, allowing Constant padding.

Create Padding Struct with Uniform, Asymmetric, height, and width implementations.

Create tests for the padding implementation.

* Update padding.rs

remove unneeded import

* Update from Merge

Use crate Element

Swap from old from_data() to new from_data_devauto()

* Formatting Changes

Formatting changes from cargo fmt --all

* Additional Format Change

One more format change that cargo fmt didn't get the first time.

* Changes to Example

Modify Example to ensure it works.

* modify naming

better names for impl / input variables.

* Modify API

- Change Padding to PadSize.
- integrate padding value into PadMode.
- update tests and examples.

* Comments and print

Improve comments+naming and remove println

* Pad Fixes

Moved pad to numeric

Simplified PadMode Element

updated tensor creations

fixed doc example

* Fix test location

* Simplified pad API

* Fix for failed unit tests

* Remove bool_full

* Rename `pads` to `padding`

---------

Co-authored-by: Dilshod Tadjibaev <939125+antimora@users.noreply.github.com>
2024-03-27 12:46:55 -05:00
Aasheesh Singh a77979e0b6
add rms norm layer (#1527) 2024-03-25 18:59:11 -04:00
Aasheesh Singh 613e698007
Feat/swiglu (#1507) 2024-03-25 15:55:27 -04:00
Dilshod Tadjibaev 6feda90a8c
Tensor expand operator (#1508)
* Improve CI cache - remove burn-tch artifacts

* PyTorch config deserializer from .pt file

* Update pytorch-model.md

* WIP

* Rename broadcast_to to expand

* Rename broadcast_to expand file

* Implemented fusion backend and fix bugs

* Remove old files

* Remove unused state

* Rename to the correct op name

* Add missing comment

* Fix expand check function doc

* Rename the leftover names

* Rename leftover names
2024-03-22 16:33:53 -05:00
Guillaume Lagrange dc45cf1700
Add `topk` tensor operation (#1497)
* Add topk and topk_with_indices

* Change topk_with_indices test to guarantee order (previously equal elements)
2024-03-22 10:57:20 -04:00
Guillaume Lagrange 47a84cc980
Add tensor sorting operations (#1488)
* Add sort, sort_with_indices and argsort

* Fix wasm build

* Add sort ops autodiff

* Fix TODO parallel comment

* Fix sort_with_indices 1d and add descending options

* Fix clippy

* Fix ascending comment (configurable)
2024-03-20 14:51:04 -04:00
carrotflakes 8911093b88
Add `flip` tensor operator (#1468) 2024-03-18 20:33:39 -05:00
Dilshod Tadjibaev 8a8300c1fb
Add tril_mask, triu_mask and diag_mask ops (#1479) 2024-03-18 10:15:40 -05:00
Guillaume Lagrange cfc0a4d9fb
Fix book MNIST reference (no more huggingface) (#1471) 2024-03-15 15:30:08 -04:00
Arjun31415 4de1272344
Feat: Add Leaky Relu Model (#1467) 2024-03-14 10:53:40 -05:00
WorldSEnder 53eb3ecfa9
Implement Huber loss (#1444)
* Implement Huber loss

Instead of using a sign or abs function, uses clamping to compute
it outside the bounds. This is better for the autodiff backend.

* mention Huber loss in the book

* unify naming of residuals in comments
2024-03-13 12:55:46 -05:00
Dilshod Tadjibaev 7a98b2f663
Add prod and prod_dim tensor ops (#1460) 2024-03-12 14:00:02 -05:00
Dilshod Tadjibaev 0138e16af6
Add Enum module support in PyTorchFileRecorder (#1436)
* Add Enum module support in PyTorchFileRecorder

Fixes #1431

* Fix wording/typos per PR feedback
2024-03-11 11:21:01 -05:00
Dilshod Tadjibaev 9d4fbc5a35
Rename `diagonal` to `eye` tensor op and add missing entry for diagonal to Book tensor section (#1449)
* Update tensor.md

* Rename diagonal to eye

* Remove extra space per PR feedback
2024-03-11 11:00:36 -05:00
Dilshod Tadjibaev 3f7e6bd5bc
Add `sign` tensor operator (#1446) 2024-03-11 10:39:30 -05:00
Dilshod Tadjibaev c7d4c23f97
Support for non-contiguous indexes in PyTorchFileRecorder keys (#1432)
* Fix non-contiguous indexes

* Update pytorch-model.md

* Simplify multiple forwards
2024-03-07 13:40:57 -06:00
Dilshod Tadjibaev 545444c02a
PyTorchFileRecord print debug option (#1425)
* Add debug print option to PyTorchFileRecorder

* Updated documentation and improved print output

* Improve print wording

* Updated per PR feedback
2024-03-06 16:11:37 -06:00
Dilshod Tadjibaev c7834e4658
Tensor `permute` operator (#1410) 2024-03-05 12:29:13 -05:00
Dilshod Tadjibaev 4ed90a988e
Add `bool()` op for numerical tensor (#1402)
Fixes #1395
2024-03-04 12:39:17 -06:00
Dilshod Tadjibaev ed37026630
Update tensor.md (#1401) 2024-03-04 09:05:14 -05:00
Guillaume Lagrange 16d7666611
Add `argwhere` and `nonzero` boolean tensor ops (#1394)
* Add argwhere and nonzero bool tensor ops

* Fix wasm build

* Add missing vec

* Fix wasm cfg placement

* Fix comment
2024-03-04 08:33:59 -05:00
Dilshod Tadjibaev d43a0b3f90
Add is_close and all_close tensor operators (#1389)
* Add is_close and all_close tensor operators

* Fix broken build issues

* Fix the table

* Add tests to candle
2024-03-01 15:37:14 -06:00
Matt Thompson 201e7f87c9
Element wise min/max between a pair of tensors (#1385)
* Added min_pair() and max_pair() methods to numeric Tensors

* Update book with added max_pair() and min_pair() methods.

* Fix spelling typo in comments

* Update comments per change requests
2024-03-01 15:56:48 -05:00
Guillaume Lagrange 3d93e6dae9
Add `not_equal` and `not_equal_elem` tensor ops (#1374)
* Fix tensor.equal_elem usage in book

* Add not_equal and not_equal_elem tensor ops

* Fix "element-wise" usage for correctness and uniformity

* Add bool_not_equal test
2024-03-01 12:13:25 -06:00
Dilshod Tadjibaev 688958ee74
Enhance PyTorchRecorder to pass top-level key to extract state_dict (#1300)
* Enhance PyTorchRecorder to pass top level key to extract state_dict

This is needed for Whisper weight pt files.

* Fix missing hyphens

* Move top-level-key test under crates

* Add sub-crates as members of workspace

* Update Cargo.lock

* Add accidentally omitted line during merge
2024-02-29 12:57:27 -06:00
Guillaume Lagrange 4efc683df4
Upgrade to candle 0.4.1 (#1382)
* Fix python main entrypoint in book example

* Remove candle windows safeguards (#1178)

* Bump candle-core from 0.3.3 to 0.4.1

* Remove windows current known issue
2024-02-29 11:29:11 -06:00
Yu Sun 330552afb4
docs(book-&-examples): modify book and examples with new `prelude` module (#1372) 2024-02-28 13:25:25 -05:00
Louis Fortier-Dubois 576bb44bc8
Feat/autodiff/checkpoint ops (#1358) 2024-02-26 17:19:09 -05:00
Mathias Insley bb5e6faff2
Feat/autotune int ops (#1136)
* Add int_random to int tensor ops

* Int random for tch backend

* Int random for burn-fusion

* int random for autodiff

* Int random for candle backend

* Int random for ndarray backend

* Int random for wgpu backend

* Merge imports

* Typo

* Shader file for int uniform distribution

* Create AutotuneOperationSet and public int_sum_dim_autotune

* Adjust bounds to 0..10

* Create uniform_int_kernel, unit tests, use new kernel

* Reduction kernels for regular and shared memory sum_dim int operations

* Macro that accomadates wgpu IntElement

* Add autotuning to int_mean_dim

* Use correct macro for Int autotuning

* Add int_mean_dim_shared_memory

* Add int_mean_dim and unit test

* Create autotunables for mean_dim

* Run fmt

* Remove comment

* Finish resolving merge conflict, fix doc

* Make the element trait bound a parameter to reduce_tune_ops macro

* Update book

* Fix requested change

* Change range to [0, 255] and update test accordingly

* Forgot to include candle in last commit

* Fix comment

* Use correct int autotune for mean dim

* Fix typo- not sure how this passed earlier

* Resolve syntax issues from merge

* Fix cast_float

* Saving here

* Continue fixing merge conflicts, all tests pass locally

* Run fmt

* Change cast_float to cast_u32_to_float

* Make uniform_int_inner_loop safer

* Be even more explicit about u32 casts

* Skip an intermediate step and cast directly to u32

* Replace JitElement + Element with IntElement

* Run fmt

* This should fix the CI

* This time for sure
2024-02-26 14:53:21 -05:00
Arjun31415 8e23057c6b
Feature Addition: PRelu Module (#1328) 2024-02-24 10:24:22 -05:00
Tushushu 27f2095bcd
Implement Instance Normalization (#1321)
* config

* rename as instances, otherwise won't work

* refactor

* InstanceNormConfig

* remove unused var

* forward

* rename

* based on gn

* unit tests

* fix tests

* update doc

* update onnx doc

* renaming method

* add comment

---------

Co-authored-by: VungleTienan <tienan.liu@vungle.com>
2024-02-23 23:31:43 -06:00
Aasheesh Singh c86db83fa9
Add support for Any, All operations to Tensor (#1342)
* add any, all op implementation for all tensor types

* add op to burn-book

* fix formatting

* refactor tensor operations from numeric to BaseOps.

* fix book doc

* comments fix and add more tests
2024-02-23 10:06:31 -05:00
Dilshod Tadjibaev d6e859330f
Pytorch message updates (#1344)
* Update pytorch-model.md

* Update error.rs
2024-02-22 12:12:50 -06:00
Dilshod Tadjibaev e9bb273700
Pytorch config extraction (#1323)
* PyTorch config deserializer from .pt file

* Update pytorch-model.md

* Format the book section

* Update Cargo.lock

* Recommend to resave config as json

* Fix comment wording
2024-02-20 09:52:36 -06:00
Dilshod Tadjibaev 44266d5fd4
Make all struct CamelCase (#1316) 2024-02-15 13:00:37 -06:00
Dilshod Tadjibaev dfb739c89a
Rename GELU to Gelu (#1311)
* Rename GELU to Gelu

This is to follow Rust naming convention
2024-02-15 10:39:06 -06:00
Jakub 3592f3799a
fix(book): add missing second parameter to CrosEntropyLoss constructor (#1301)
* fix(book): add missing second parameter to CrosEntropyLoss constructor

CrossEntropyLoss::new() expects two parameters, the pad_index and the device

* fix: fix missing closing parenthese
2024-02-15 09:46:41 -05:00
Jakub a68b494531
fix(book): add missing device parameter to mode.init() (#1302) 2024-02-13 09:34:03 -05:00
Nathaniel Simard a9b6dbc177
Enable burn fusion by default (#1223) 2024-02-08 11:28:02 -05:00
Guillaume Lagrange 059c241511
Add vision feature for MNIST in book guide (#1267) 2024-02-07 09:21:35 -05:00
Sylvain Benner 4aa13d6b25
Bump Burn version to 0.13 (#1211) 2024-01-31 16:01:20 -05:00
Dilshod Tadjibaev 14aad24c19
Regex fix pytorch (#1196) 2024-01-31 09:53:52 -05:00
Sylvain Benner 02259ea11c
[xtask] New subcommand: Books (#1192) 2024-01-31 09:09:22 -05:00
Aasheesh Singh 8b4038d004
Merge pull request #1183 from tracel-ai/docs/guide
Docs/guide

Improve guide book: Tensor, Dataset and Example Section.
2024-01-30 10:25:29 -05:00
Aasheesh Singh 4ebab1a538 minor fixes 2024-01-30 10:23:33 -05:00
Dilshod Tadjibaev 67ffa1e54b
Update documentation and book sections on PyTorch import (#1180) 2024-01-29 17:55:20 -05:00
Aasheesh Singh 61dd2c17b9 1. Address comments :)
2. Fix formatting
3. remove dead link
2024-01-29 14:39:24 -05:00
Aasheesh Singh 3e3382c580 fix typo 2024-01-26 06:10:59 -05:00
Aasheesh Singh 9ef9d27012 5. Update dataset.md to explain Dataset transform usages with examples. 2024-01-26 06:05:01 -05:00
Aasheesh Singh 52b11d9a87 4. Add section on Tensor introduction 2024-01-26 04:16:23 -05:00
Aasheesh Singh a5ae89ebee 1. fix broken link in getting-started.md
2. Make guide documentation and file structure consistent with examples/guide.
3. Add helpful comments in guide.
2024-01-25 19:17:28 -05:00
Guillaume Lagrange b9bd42959b
Add vision/mnist dataset (#1176) 2024-01-25 16:16:39 -05:00
Guillaume Lagrange e1d873abe2
Add Rust concept notes to Burn Book (#1169) 2024-01-24 12:58:14 -05:00
Joshua Ferguson 3b7d9feede
Elementwise pow op (#1133) 2024-01-24 09:46:57 -05:00
Nathaniel Simard eaa4dc3207
Feat/recorder/custom device (#1165) 2024-01-23 13:05:41 -05:00
Guillaume Lagrange e9d1656687
Add model saving and loading examples (#1164) 2024-01-23 12:00:04 -05:00
Guillaume Lagrange 191877f14a
New default recorder (#1161)
* Change DefaultFileRecorder to NamedMpkFileRecorder (no compression)

* Actually, safetensors does not have any checksum for data validation

* Update checksum explainer/recommandation
2024-01-22 12:50:42 -05:00
Guillaume Lagrange 6079f98950
Fix missing ticks in Burn book and remove unused example dependency (#1144)
* Add missing ticks for triu and tril ops

* Removed example unused dependency
2024-01-16 09:03:17 -05:00
Guillaume Lagrange 15c2f3b7a1
Fix small typos and dead links in README and burn-book (#1127) 2024-01-10 08:57:28 -05:00
Kirill Mavreshko 97297538b1
Remove _devauto fuctions (#518) (#1110) 2024-01-06 13:36:34 -05:00
Nathaniel Simard fceb036c6f
Fix: serde dependency (#1091)
* Re-export serde

* Fix

* USe another strategy

* Fix

* Fix

* Update de book
2023-12-22 16:53:34 -05:00
Kirill Mavreshko 1fd07fcb4a
Explicit device tensors (#1081) 2023-12-20 17:49:59 -05:00
pacowong c1cb77ac2e
Add mish and softplus activation function (#1071) 2023-12-20 12:41:23 -05:00
Louis Fortier-Dubois 8fc52113bc
Chore/bump v12 (#1048) 2023-12-04 10:47:54 -05:00
Louis Fortier-Dubois 3088c466a5
patch 0.11.1 (#1047) 2023-12-04 10:18:30 -05:00
Timothy 52811f9938
Fix some issues in Burn book (#1042) 2023-12-04 09:23:47 -05:00
Nathaniel Simard ab1b5890f5
Chore/release (#1031) 2023-12-01 14:33:28 -05:00
Maksym Hazevych 87393b2070
Updated explanation to match code (#1012)
Updated the explanation at the beginning to match the actual changes to 'Cargo.toml'. Moved down the 'edition' property to match the style of the initial 'Cargo.toml' file. Removed unnecessary comment for a self-explanatory piece of code. Some rewording.
2023-11-29 09:41:07 -05:00
David Chavez fda9e5d388
feature(tensor): Add triu and tril ops (#1002) 2023-11-29 09:39:42 -05:00
David Chavez 929b1786bb
feature(tensor): Add chunk op (#998) 2023-11-27 09:58:43 -05:00
David Chavez 2fdf9a3591
feature(tensor): Add Tensor::stack (#1004) 2023-11-27 09:57:31 -05:00
Granville Schmidt d0cb7205fa
fix typo in basic-workflow training (#1000) 2023-11-27 09:37:29 -05:00
David Chavez f09baadfc5
feature(tensor): Add narrow op (#996)
* Add narrow methods

* Revert "Add narrow methods"

This reverts commit 9371d87c79.

* Implement a shared version of narrow

* Correct test case

* Update book

* Improve tests
2023-11-24 08:24:06 -05:00
Dilshod Tadjibaev be5bb33788
Update getting-started.md (#987)
Fixes #828
2023-11-21 17:40:40 -05:00
Sylvain Benner 2f079e991b
Fix burn version in the Burn book snippet (#985)
The book assumes the latest version which is 0.10.0 at the time of this commit.
2023-11-21 15:59:25 -05:00
David Chavez 88b442015d
feature(norm): Add GroupNorm (#963)
* Add GroupNorm

* Fix implemenation and add tests

* Address PR comments

* Fix formatting

* Update burn book
2023-11-21 09:22:56 -05:00
David Chavez 49e16b6834
feature(tensor): Add unsqueeze_dim helper (#966) 2023-11-20 10:29:40 -05:00
Rodrigo Pandini 20e9066b57
fix with_file_checkpointer use in training example (#973) 2023-11-20 09:17:29 -05:00
Zsombor 4fc0c27e31
Implement tensor.recip() function to calculate elementwise reciprocals (#953) 2023-11-15 09:17:32 -05:00
dengelt e255472826
Fix typos (#949) 2023-11-12 15:08:39 -05:00
Nathaniel Simard 96524d40a1
[Breaking] Refactor Backend Names (#904) 2023-10-29 18:27:49 -04:00
Nathaniel Simard e581912c8a
Update book (#858) 2023-10-11 09:11:28 -04:00
Nathaniel Simard 620b86de98
Feat training events (#857) 2023-10-10 13:27:03 -04:00
Peter Szilvasi 6c32e23f15
Fix the device initialization in backend.md (#808) 2023-09-16 09:44:34 -04:00
Louis Fortier-Dubois d249d24c07
simplify getting started (#782) 2023-09-07 14:57:32 -04:00
Dilshod Tadjibaev 27952b1f47
Fix broken link and other minor fixes (#780) 2023-09-07 08:44:57 -04:00
Nathaniel Simard 8b3d10c4d3
Cleanup of the book + feature flags (#773)
---------

Co-authored-by: louisfd <louisfd94@gmail.com>
2023-09-06 09:16:36 -04:00
Nathaniel Simard ab4d2f8d63
Book: Add Config, Dataset and Record building blocks (#770) 2023-09-05 17:25:05 -04:00