Karsten Becker
c21d5a3207
Add LeakyReLu implementation ( #1208 )
...
* Implement LeakyReLu
* Cargo fmt
* Apply suggestions
* cargo fmt
* Use float_mul_scalar
* Should be grad
* Add to books module
* Move test files
* Update leaky relu to use activation function
* Update tensor.md
* Fix failing test due to approx
* Add back the function comment
* Fix comment per PR feedback
---------
Co-authored-by: Dilshod Tadjibaev <939125+antimora@users.noreply.github.com>
2024-03-27 13:57:51 -05:00
jcmullwh
626457e1c6
Provide Tensor Padding Helpers #960 ( #1097 )
...
* Initial padding approach
Create padding implementation for the last two dimensions of Float and Int Tensors.
Create PadMode Enum, allowing Constant padding.
Create Padding Struct with Uniform, Asymmetric, height, and width implementations.
Create tests for the padding implementation.
* Update padding.rs
remove unneeded import
* Update from Merge
Use crate Element
Swap from old from_data() to new from_data_devauto()
* Formatting Changes
Formatting changes from cargo fmt --all
* Additional Format Change
One more format change that cargo fmt didn't get the first time.
* Changes to Example
Modify Example to ensure it works.
* modify naming
better names for impl / input variables.
* Modify API
- Change Padding to PadSize.
- integrate padding value into PadMode.
- update tests and examples.
* Comments and print
Improve comments+naming and remove println
* Pad Fixes
Moved pad to numeric
Simplified PadMode Element
updated tensor creations
fixed doc example
* Fix test location
* Simplified pad API
* Fix for failed unit tests
* Remove bool_full
* Rename `pads` to `padding`
---------
Co-authored-by: Dilshod Tadjibaev <939125+antimora@users.noreply.github.com>
2024-03-27 12:46:55 -05:00
Nathaniel Simard
40a26bd2ea
Feat/backend bridge ( #1529 )
2024-03-26 19:24:45 -04:00
Louis Fortier-Dubois
5bac300688
Migrate/jit/interpolate ( #1528 )
...
* separate forward backward
* refactor with pool strategy
* refactor further
* pooling refactored
* refactoring for adaptive wip
* wip adaptive
* adaptive
* delete some wgsl
* avg pool backward
* clippy
* refactor interpolate files
* nearest shader
* nearest
* some boilerplate
* wip
* bilinear
* nearest backward
* cubic
* cleanup
* minor refactor
* add some white space
2024-03-26 08:57:26 -04:00
Louis Fortier-Dubois
37b61ea646
Migrate/jit/adaptive avg pool backward ( #1530 )
...
* separate forward backward
* refactor with pool strategy
* refactor further
* pooling refactored
* refactoring for adaptive wip
* wip adaptive
* adaptive
* delete some wgsl
* avg pool backward
* clippy
* minor refactor
* works
* delete wgsl
2024-03-26 08:38:06 -04:00
Aasheesh Singh
a77979e0b6
add rms norm layer ( #1527 )
2024-03-25 18:59:11 -04:00
Louis Fortier-Dubois
da5b0438ec
Migrate/jit/pooling ( #1509 )
...
* separate forward backward
* refactor with pool strategy
* refactor further
* pooling refactored
* refactoring for adaptive wip
* wip adaptive
* adaptive
* delete some wgsl
* avg pool backward
* clippy
* minor refactor
2024-03-25 16:04:58 -04:00
Aasheesh Singh
613e698007
Feat/swiglu ( #1507 )
2024-03-25 15:55:27 -04:00
Louis Fortier-Dubois
4542ceddca
Migrate/jit/conv2d ( #1493 )
...
* conv2d but bug
* convolution done
* minor clean
* delete wgsl
2024-03-25 10:45:40 -04:00
Sylvain Benner
0adda72316
[backend-comparison] Add system information to benchmark results ( #1495 )
...
* Bump sysinfo crate to 0.30.7
* [backend-comparison] Add CPUs and GPUs system info to results
* [backend-comparison] Add integrated GPUs to gathered system info
* [backend-comparison] Use AutoGraphicsApi wgpu backend selection
2024-03-22 23:24:49 -04:00
Dilshod Tadjibaev
6feda90a8c
Tensor expand operator ( #1508 )
...
* Improve CI cache - remove burn-tch artifacts
* PyTorch config deserializer from .pt file
* Update pytorch-model.md
* WIP
* Rename broadcast_to to expand
* Rename broadcast_to expand file
* Implemented fusion backend and fix bugs
* Remove old files
* Remove unused state
* Rename to the correct op name
* Add missing comment
* Fix expand check function doc
* Rename the leftover names
* Rename leftover names
2024-03-22 16:33:53 -05:00
Guillaume Lagrange
dc45cf1700
Add `topk` tensor operation ( #1497 )
...
* Add topk and topk_with_indices
* Change topk_with_indices test to guarantee order (previously equal elements)
2024-03-22 10:57:20 -04:00
Louis Fortier-Dubois
dd699a90a2
Migrate/jit/matmul tiling 2d ( #1472 )
...
* refactor matmul files
* wip refactor matmul
* everything is memco
* support local arrays
* advancing tiling2d
* advancing tiling2d
* advancing tiling2d
* tiling2d finished but buggy
* configurable unrolling
* not bugged
* fails on unroll
* stupid break
* tiling2d no assumption works
* clippy
* bounds check as bool
* lhs rhs as enum
* tiling 2d major refactor
* remove assign vec4
* variable declarations above loops
* fmt
* clippy
* Fix autotune + unroll
* move val
* clippy
* fmt
---------
Co-authored-by: nathaniel <nathaniel.simard.42@gmail.com>
2024-03-22 08:26:32 -04:00
Sylvain Benner
0a8a3cc9e9
[xtask] Add support for cargo metadata new workspace member format ( #1500 )
2024-03-21 16:04:52 -04:00
Guillaume Lagrange
3e4af41694
Fix sort descending for 1d case ( #1494 )
2024-03-21 07:45:37 -04:00
Guillaume Lagrange
47a84cc980
Add tensor sorting operations ( #1488 )
...
* Add sort, sort_with_indices and argsort
* Fix wasm build
* Add sort ops autodiff
* Fix TODO parallel comment
* Fix sort_with_indices 1d and add descending options
* Fix clippy
* Fix ascending comment (configurable)
2024-03-20 14:51:04 -04:00
Guillaume Lagrange
430f642394
Change assert_approx_eq precision from 3 to 2 ( #1491 )
2024-03-19 12:26:21 -04:00
Rubén J.R
69f1877754
New learning rate schedulers ( #1481 )
2024-03-19 08:28:42 -05:00
carrotflakes
8911093b88
Add `flip` tensor operator ( #1468 )
2024-03-18 20:33:39 -05:00
Dilshod Tadjibaev
8a8300c1fb
Add tril_mask, triu_mask and diag_mask ops ( #1479 )
2024-03-18 10:15:40 -05:00
Louis Fortier-Dubois
c729401fb2
Remove unroll in shared reduce ( #1480 )
2024-03-17 13:56:54 -04:00
Arjun31415
d3af29c5b4
Missing `Debug` derive for Group Norm Config ( #1482 )
2024-03-17 13:12:50 -04:00
Louis Fortier-Dubois
cf3c1ca80a
Migrate/jit/cat ( #1457 )
2024-03-17 11:37:36 -04:00
Louis Fortier-Dubois
41d01b8e19
Migrate/jit/prod ( #1474 )
2024-03-15 18:29:30 -04:00
Arjun31415
4de1272344
Feat: Add Leaky Relu Model ( #1467 )
2024-03-14 10:53:40 -05:00
WorldSEnder
53eb3ecfa9
Implement Huber loss ( #1444 )
...
* Implement Huber loss
Instead of using a sign or abs function, uses clamping to compute
it outside the bounds. This is better for the autodiff backend.
* mention Huber loss in the book
* unify naming of residuals in comments
2024-03-13 12:55:46 -05:00
Dilshod Tadjibaev
7a98b2f663
Add prod and prod_dim tensor ops ( #1460 )
2024-03-12 14:00:02 -05:00
carrotflakes
80aac1dde4
Add Rank0 variant to AdaptorRecordV1 and AdaptorRecordItemV1 ( #1442 )
2024-03-12 13:08:20 -04:00
Kyle Chen
c52c49785d
Add linear learning rate scheduler ( #1443 )
2024-03-12 13:04:12 -04:00
Louis Fortier-Dubois
278fcb3dad
Migrate/jit/mask ( #1456 )
2024-03-12 12:43:05 -04:00
Louis Fortier-Dubois
02d37011ab
Fix/main/print ( #1459 )
2024-03-11 18:52:36 -04:00
Dilshod Tadjibaev
0138e16af6
Add Enum module support in PyTorchFileRecorder ( #1436 )
...
* Add Enum module support in PyTorchFileRecorder
Fixes #1431
* Fix wording/typos per PR feedback
2024-03-11 11:21:01 -05:00
Dilshod Tadjibaev
9d4fbc5a35
Rename `diagonal` to `eye` tensor op and add missing entry for diagonal to Book tensor section ( #1449 )
...
* Update tensor.md
* Rename diagonal to eye
* Remove extra space per PR feedback
2024-03-11 11:00:36 -05:00
Louis Fortier-Dubois
093cbd397d
JIT Migration: PRNG ( #1433 )
...
* wip bernoulli
* wip
* bernoulli works
* uniform works
* done
* remove old
* refactor prng traits
* forgot to save file
* allow
* clippy
* clippy
* scalar commutativity
* array instead of vec
2024-03-11 11:40:27 -04:00
Dilshod Tadjibaev
3f7e6bd5bc
Add `sign` tensor operator ( #1446 )
2024-03-11 10:39:30 -05:00
github-actions[bot]
56f460295a
Combined PRs ( #1439 )
...
* Bump web-time from 1.0.0 to 1.1.0
Bumps [web-time](https://github.com/daxpedda/web-time ) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/daxpedda/web-time/releases )
- [Changelog](https://github.com/daxpedda/web-time/blob/main/CHANGELOG.md )
- [Commits](https://github.com/daxpedda/web-time/compare/v1.0.0...v1.1.0 )
---
updated-dependencies:
- dependency-name: web-time
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump rayon from 1.8.1 to 1.9.0
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.8.1 to 1.9.0.
- [Changelog](https://github.com/rayon-rs/rayon/blob/main/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.8.1...rayon-core-v1.9.0 )
---
updated-dependencies:
- dependency-name: rayon
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump tempfile from 3.10.0 to 3.10.1
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.10.0 to 3.10.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.10.0...v3.10.1 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump the cargo group group with 1 update
Bumps the cargo group group with 1 update: [mio](https://github.com/tokio-rs/mio ).
Updates `mio` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/tokio-rs/mio/releases )
- [Changelog](https://github.com/tokio-rs/mio/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/mio/compare/v0.8.10...v0.8.11 )
---
updated-dependencies:
- dependency-name: mio
dependency-type: indirect
dependency-group: cargo-security-group
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump log from 0.4.20 to 0.4.21
Bumps [log](https://github.com/rust-lang/log ) from 0.4.20 to 0.4.21.
- [Release notes](https://github.com/rust-lang/log/releases )
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/log/compare/0.4.20...0.4.21 )
---
updated-dependencies:
- dependency-name: log
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-03-08 19:27:48 -05:00
Nathaniel Simard
2de270fe0e
Fix tch view data corruption ( #1434 )
2024-03-08 09:55:47 -05:00
Louis Fortier-Dubois
61c0474172
JIT migration: contiguous kernel ( #1424 )
...
* JIT migration: contiguous kernel
* delete wgsl
2024-03-08 08:01:29 -05:00
Louis Fortier-Dubois
040cd55f85
JIT migration: cast kernel ( #1423 )
2024-03-07 17:49:09 -05:00
Louis Fortier-Dubois
9eecc713a4
JIT: Fix min & max values ( #1429 )
...
* real min and max values
* fix
* fmt
2024-03-07 15:10:30 -05:00
Dilshod Tadjibaev
c7d4c23f97
Support for non-contiguous indexes in PyTorchFileRecorder keys ( #1432 )
...
* Fix non-contiguous indexes
* Update pytorch-model.md
* Simplify multiple forwards
2024-03-07 13:40:57 -06:00
Dilshod Tadjibaev
b12646de0a
Truncate debug display for NestedValue ( #1428 )
...
* Truncate debug display for NestedValue
* Fix failing tests
2024-03-07 08:06:31 -05:00
Dilshod Tadjibaev
545444c02a
PyTorchFileRecord print debug option ( #1425 )
...
* Add debug print option to PyTorchFileRecorder
* Updated documentation and improved print output
* Improve print wording
* Updated per PR feedback
2024-03-06 16:11:37 -06:00
Nathaniel Simard
b429cc39c1
Splitted the JIT stuff from the Wgpu stuff ( #1417 )
2024-03-06 11:23:53 -05:00
jackdarlison
3ff6e7170e
Switched arguments in `reshape_args_usize` check ( #1409 )
2024-03-06 08:45:12 -05:00
Aasheesh Singh
0c92c8c8eb
Autodiff/training support for Nearest Interpolation ( #1414 )
...
Add training support for nearest interpolation
---------
Co-authored-by: yurzhang <yurzhang.oi@gmail.com>
Co-authored-by: Dilshod Tadjibaev <939125+antimora@users.noreply.github.com>
2024-03-06 00:12:05 -05:00
Dilshod Tadjibaev
0601dc778b
Update operation.rs ( #1418 )
2024-03-05 12:43:02 -06:00
Dilshod Tadjibaev
c7834e4658
Tensor `permute` operator ( #1410 )
2024-03-05 12:29:13 -05:00
Dilshod Tadjibaev
4ed90a988e
Add `bool()` op for numerical tensor ( #1402 )
...
Fixes #1395
2024-03-04 12:39:17 -06:00
Nathaniel Simard
efbe818465
Refactor wgpu max pooling ( #1398 )
2024-03-04 13:23:11 -05:00