The Burn Book (#625)

This commit is contained in:
Wouter Doppenberg 2023-08-10 20:17:23 +02:00 committed by GitHub
parent 2709e4c36e
commit f819898097
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
32 changed files with 221 additions and 1 deletions

View File

@ -120,7 +120,7 @@ use burn::tensor::{Tensor, Int};
fn function<B: Backend>(tensor_float: Tensor<B, 2>) {
let _tensor_bool = tensor_float.clone().equal_elem(2.0); // Tensor<B, 2, Bool>
let _tensor_int = tensor_float.argmax(1) // Tensor<B, 2, Int>
let _tensor_int = tensor_float.argmax(1); // Tensor<B, 2, Int>
}
```

18
burn-book/.gitignore vendored Normal file
View File

@ -0,0 +1,18 @@
target
# MacOS temp file
.DS_Store
book-test
guide/book
.vscode
tests/burn-book/book/
book/
# Ignore Jetbrains specific files.
.idea/
# Ignore Vim temporary and swap files.
*.sw?
*~

6
burn-book/book.toml Normal file
View File

@ -0,0 +1,6 @@
[book]
authors = ["Wouter Doppenberg"]
language = "en"
multilingual = false
src = "src"
title = "The Burn Book"

33
burn-book/src/SUMMARY.md Normal file
View File

@ -0,0 +1,33 @@
[Overview](../../README.md)
[Motivation](./motivation.md)
- [Usage](./usage/README.md)
- [Installation](./usage/installation.md)
- [Quick Start](./usage/quick-start.md)
- [Concepts](./concepts/README.md)
- [Tensors](./concepts/tensors.md)
- [Modules](./concepts/modules.md)
- [Backend](./concepts/backend.md)
- [Configs](./concepts/configs.md)
- [Learner](./concepts/learner.md)
- [Datasets](./concepts/datasets.md)
- [Metrics](./concepts/metrics.md)
- [Losses](./concepts/losses.md)
- [Optimizers](./concepts/optimizers.md)
- [`no_std`](./concepts/no-std.md)
- [Advanced](./concepts/advanced/README.md)
- [Records](./concepts/advanced/records.md)
- [Macros](./concepts/advanced/macros.md)
- [Examples](./examples/README.md)
- [MNIST](./examples/mnist.md)
- [MNIST Inference on Web](./examples/mnist-inference-on-web.md)
- [Named Tensors](./examples/named-tensors.md)
- [ONNX Inference](./examples/onnx-inference.md)
- [Text Classification](./examples/text-classification.md)
- [Text Generation](./examples/text-generation.md)
[Help](./help.md)
[Contributing](./contributing.md)

View File

@ -0,0 +1 @@
# Concepts

View File

@ -0,0 +1 @@
# Advanced

View File

@ -0,0 +1 @@
# Macros

View File

@ -0,0 +1 @@
# Records

View File

@ -0,0 +1,9 @@
# Backend
Nearly everything in `burn` is based on the `Backend` trait, which enables you to run tensor
operations using different implementations without having to modify your code. While a backend may
not necessarily have autodiff capabilities, the `ADBackend` trait specifies when autodiff is needed.
This trait not only abstracts operations but also tensor, device and element types, providing each
backend the flexibility they need. It's worth noting that the trait assumes eager mode since `burn`
fully supports dynamic graphs. However, we may create another API to assist with integrating
graph-based backends, without requiring any changes to the user's code.

View File

@ -0,0 +1,29 @@
# Configs
The `Config` derive lets you define serializable and deserializable configurations or
hyper-parameters for your [modules](#module) or any components.
```rust
use burn::config::Config;
#[derive(Config)]
pub struct PositionWiseFeedForwardConfig {
pub d_model: usize,
pub d_ff: usize,
#[config(default = 0.1)]
pub dropout: f64,
}
```
The `Derive` macro also adds useful methods to your config, such as a builder pattern.
```rust
fn main() {
let config = PositionWiseFeedForwardConfig::new(512, 2048);
println!("{}", config.d_model); // 512
println!("{}", config.d_ff); // 2048
println!("{}", config.dropout); // 0.1
let config = config.with_dropout(0.2);
println!("{}", config.dropout); // 0.2
}
```

View File

@ -0,0 +1 @@
# Datasets

View File

@ -0,0 +1,31 @@
# Learner
The `Learner` is the main `struct` that let you train a neural network with support for `logging`,
`metric`, `checkpointing` and more. In order to create a learner, you must use the `LearnerBuilder`.
```rust,ignore
use burn::train::LearnerBuilder;
use burn::train::metric::{AccuracyMetric, LossMetric};
use burn::record::DefaultRecordSettings;
fn main() {
let dataloader_train = ...;
let dataloader_valid = ...;
let model = ...;
let optim = ...;
let learner = LearnerBuilder::new("/tmp/artifact_dir")
.metric_train_plot(AccuracyMetric::new())
.metric_valid_plot(AccuracyMetric::new())
.metric_train(LossMetric::new())
.metric_valid(LossMetric::new())
.with_file_checkpointer::<DefaultRecordSettings>(2)
.num_epochs(10)
.build(model, optim);
let _model_trained = learner.fit(dataloader_train, dataloader_valid);
}
```
See this [example](https://github.com/burn-rs/burn/tree/main/examples/mnist) for a real usage.

View File

@ -0,0 +1 @@
# Losses

View File

@ -0,0 +1 @@
# Metrics

View File

@ -0,0 +1 @@
# Models

View File

@ -0,0 +1,32 @@
# Modules
The `Module` derive allows you to create your own neural network modules, similar to PyTorch. The
derive function only generates the necessary methods to essentially act as a parameter container for
your type, it makes no assumptions about how the forward pass is declared.
```rust
use burn::nn;
use burn::module::Module;
use burn::tensor::backend::Backend;
#[derive(Module, Debug)]
pub struct PositionWiseFeedForward<B: Backend> {
linear_inner: Linear<B>,
linear_outer: Linear<B>,
dropout: Dropout,
gelu: GELU,
}
impl<B: Backend> PositionWiseFeedForward<B> {
pub fn forward<const D: usize>(&self, input: Tensor<B, D>) -> Tensor<B, D> {
let x = self.linear_inner.forward(input);
let x = self.gelu.forward(x);
let x = self.dropout.forward(x);
self.linear_outer.forward(x)
}
}
```
Note that all fields declared in the struct must also implement the `Module` trait. The `Tensor`
struct doesn't implement `Module`, but `Param<Tensor<B, D>>` does.

View File

@ -0,0 +1,15 @@
## Support for `no_std`
Burn, including its `burn-ndarray` backend, can work in a `no_std` environment, provided `alloc` is
available for the inference mode. To accomplish this, simply turn off the default features in `burn`
and `burn-ndarray` (which is the minimum requirement for running the inference mode). You can find a
reference example in
[burn-no-std-tests](https://github.com/burn-rs/burn/tree/main/burn-no-std-tests).
The `burn-core` and `burn-tensor` crates also support `no_std` with `alloc`. These crates can be
directly added as dependencies if necessary, as they are reexported by the `burn` crate.
Please be aware that when using the `no_std` mode, a random seed will be generated at build time if
one hasn't been set using the `Backend::seed` method. Also, the
[spin::mutex::Mutex](https://docs.rs/spin/latest/spin/mutex/struct.Mutex.html) is used instead of
[std::sync::Mutex](https://doc.rust-lang.org/std/sync/struct.Mutex.html) in this mode.

View File

@ -0,0 +1 @@
# Optimizers

View File

@ -0,0 +1,25 @@
# Tensors
At the core of burn lies the `Tensor` struct, which encompasses multiple types of tensors, including
`Float`, `Int`, and `Bool`. The element types of these tensors are specified by the backend and are
usually designated as a generic argument (e.g., `NdArrayBackend<f32>`). Although the same struct is
used for all tensors, the available methods differ depending on the tensor kind. You can specify the
desired tensor kind by setting the third generic argument, which defaults to `Float`. The first
generic argument specifies the backend, while the second specifies the number of dimensions.
```rust
use burn::tensor::backend::Backend;
use burn::tensor::{Tensor, Int};
fn function<B: Backend>(tensor_float: Tensor<B, 2>) {
let _tensor_bool = tensor_float.clone().equal_elem(2.0); // Tensor<B, 2, Bool>
let _tensor_int = tensor_float.argmax(1); // Tensor<B, 2, Int>
}
```
As demonstrated in the previous example, nearly all operations require owned tensors as parameters,
which means that calling `Clone` explicitly is necessary when reusing the same tensor multiple
times. However, there's no need to worry since the tensor's data won't be copied, it will be flagged
as readonly when multiple tensors use the same allocated memory. This enables backends to reuse
tensor data when possible, similar to a copy-on-write pattern, while remaining completely
transparent to the user.

View File

@ -0,0 +1 @@
# Contributing

View File

@ -0,0 +1 @@
# Examples

View File

@ -0,0 +1 @@
# MNIST Inference on Web

View File

@ -0,0 +1 @@
# MNIST

View File

@ -0,0 +1 @@
# Named Tensors

View File

@ -0,0 +1 @@
# ONNX Inference

View File

@ -0,0 +1 @@
# Text Classification

View File

@ -0,0 +1 @@
# Text Generation

1
burn-book/src/help.md Normal file
View File

@ -0,0 +1 @@
# Help

View File

@ -0,0 +1 @@
# Motivation

View File

@ -0,0 +1 @@
# Usage

View File

@ -0,0 +1 @@
# Installation

View File

@ -0,0 +1 @@
# Quick Start