Doc: initial architecture documentation (#308)

This commit is contained in:
Nathaniel Simard 2023-04-24 13:07:30 -04:00 committed by GitHub
parent 02abc373d3
commit c9fda41846
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 292 additions and 0 deletions

158
ARCHITECTURE.md Normal file
View File

@ -0,0 +1,158 @@
# Architecture
This file documents most major architectural decisions with the reasoning behind them.
__Sections__
* [Module](#module)
* [Optimization](#optimization)
* [Serialization](#serialization)
## Module
Modules are a way of creating neural network structures that can be easily optimized, saved, and loaded with little to no boilerplate.
Unlike other frameworks, a module does not force the declaration of the forward pass, leaving it up to the implementer to decide how it should be defined.
Additionally, most modules are created using a (de)serializable configuration, which defines the structure of the module and its hyper-parameters.
Parameters and hyper-parameters are not serialized into the same file and both are normaly necessary to load a module for inference.
### Optimization
Optimization is normally done with gradient descent (or ascent for reinforcement learning), and it is important to provide an easy API for optimizing modules.
#### Constraints
1. __Users should be able to control what is optimized.__
Modules can contain anything for maximum flexibility, but not everything needs to be optimized.
2. __Optimizers should have a serializable state that is updated during training.__
Many optimizers keep track of previous gradients to implement some form of momentum.
However, the state can be anything, not just tensors, allowing for easy implementation of any kind of optimizer.
3. __The learning rate can be updated during training.__
Learning rate schedulers are often used during training and should be considered as a key aspect.
#### Solution
The solution to this problem comprises multiple parts.
Firstly, the `Optimizer` trait is quite similar to the `Module` trait in terms of saving and loading the state.
Please refer to the [serialization](#serialization) section for more details.
Secondly, two traits were created.
The `Optimizer` trait is general and relatively unopinionated, with a simple `step` method that takes a learning rate, a module, and the gradients.
The other trait, `SimpleOptimizer`, aims to provide an easier API for implementing new optimizers.
The goal is to allow implementations to avoid handling missing gradients, loading and exporting records, navigating the module parameter structure, handling tracked and untracked tensors, and other such tasks.
Thirdly, each tensor that will be optimized needs to be wrapped into a `Param` struct, which gives them an ID used for (de)serialization and to associate the state of the optimizer to each parameter.
The `Module` trait has two ways to navigate over parameters.
The first one is the `map` function, which returns `Self` and makes it easy to implement any transformation and mutate all parameters.
The second one is the `visit` function, which has a similar signature but does not mutate the parameter tensors.
__SimpleOptimizer__
The `SimpleOptimizer` has two major assumptions:
1. The state of the optimizer is linked to each parameter.
In other words, each parameter has its own optimizer state, decoupled from the other parameters.
2. The state of the optimizer implements `Record`, `Clone`, and has a `'static` lifetime.
The benefits of those assumptions materialize in simplicity with little loss in flexibility.
The state associative type is also generic over the dimension, making it extremely easy to include tensors in the state that share the same dimensionality as its parameter.
To wrap a simple optimizer into the more general `Optimizer` trait, the `OptimizerAdaptor` struct is used.
__OptimizerAdaptor__
The `OptimizerAdaptor` is a simple struct composed of a `SimpleOptimizer` and a hashmap with all records associated with each parameter ID.
When performing an optimization step, the adaptor handles the following:
1. Updates each parameter tensor in the given module using the `Module::map` function.
2. Checks if a gradient for the current tensor exists.
3. Makes sure that the gradient, the tensor, and the optimizer state associated with the current parameter are on the same device.
The device can be different if the state is loaded from disk to restart training.
4. Performs the simple optimizer step using the inner tensor since the operations done by the optimizer should not be tracked in the autodiff graph.
5. Updates the state for the current parameter and returns the updated tensor, making sure it's properly registered into the autodiff graph if gradients are maked as required.
Note that a parameter can still be updated by another process, as is the case with running metrics used in batch norm.
These tensors are still wrapped using the `Param` struct so that they are included in the module's state and given a proper parameter ID, but they are not registered in the autodiff graph.
### Serialization
An important aspect of a deep learning framework is the ability to save and load models from disk.
Despite appearing as a simple feature, it involves numerous constraints that require a proper solution.
#### Constraints
1. __Users should be able to declare the precision of the model to be saved, independent of the backend in use.__
The modules should not be duplicated in RAM in another precision to support this.
Conversion should be done lazily during (de)serialization.
2. __Users should be able to add any field to a module, even fields that are not serializable.__
This can include constants, database connections, other module references, or any other information.
Only parameters should be serialized since the structure of the module itself should be encapsulated with module configurations (hyper-parameters).
3. __Users should be able to declare the format in which the module should be saved.__
This can involve saving to a compressed JSON file or directly to bytes in memory for `no-std` environments.
4. __Users should be able to create a module with its saved parameters without having to initialize the module first.__
This will avoid unnecessary module initialization and tensor loading, resulting in reduced cold start when dealing with inference.
In addition to all of these constraints, the solution should be easy to use.
#### Solution
In order to be able to add any field to a module without requiring it to be (de)serializable, we decouple the module type from its state.
We create a new type for each module that only contains the parameters that need to be saved.
To generate that type automatically, the user must either declare which field is a parameter or a constant, or we assume that each field implements the module trait.
The second solution was chosen as it simplifies the code generation and reduces the size of the user API.
This means that the `Module` trait should be implemented by [primitives types](./burn-core/src/module/param/primitive.rs).
The following diagrams highlight the main types and traits used in the solution.
<div align="center">
<h4>Module Serialization Types</h4>
<img src="./assets/ModuleSerialization.png" width="700px"/>
<div align="left">
The way the types interact with each other is pretty straightforward.
First, a module can be converted into a record using `into_record()`.
Note that tensors can be cloned, but it won't actually copy any data; it will create another reference to the same data.
Then, you can call `record()`, which takes a `RecordSettings` as a parameter.
The function is automatically implemented for each record.
It calls `into_item::<RecordSettings>()` on the record, which creates a serializable item following the given settings.
Note that tensors implement record, and their item is just a wrapper struct that contains information about the precision in which the tensor should be saved or loaded.
No actual copy of the tensor is made until this point.
The tensor is converted to the `Data` struct and then converted into the specified precision only when `serialize()` or `deserialize()` are called, which makes the whole process lazy.
To recapitulate, the `Module` trait has an associated type that implements `Record`, which only contains the parameters of the model.
The `Record` trait has a generic associated type (GAT) that specifies a family of types that can be (de)serialized given any `RecordSettings`.
Records are therefore decoupled from the backend in use, and the saved items can be loaded on any backend with any precision, since the conversion is type-safe and done when `serialize()` and `deserialize()` are called.
All of the types are generated using simple derive macros without any conditional statements or complex syntax, as `Record` and `Module` are implemented for all primitive types.
This makes the code simple and easy to maintain.
In addition, you can extend the current system with your own `Recorder` and `RecordSettings` to control how your modules should be saved and loaded.
##### Pros
* All constraints are respected.
* The code is simple and easy to maintain, with very few conditional statements.
It is just recursive data structures, where all the complexity is handled by the framework in primitive implementations.
* The user API is simple and small, with only two derives (`Record` and `Module`) and no additional attributes.
* Users can create their own `Module` and `Record` primitive types, which gives them the flexibility to control how their data is serialized without having to fork the framework.
##### Cons
* There are more types, but most of them are automatically generated and single-purpose, so users don't need to interact with them for common use cases.
However, they can do so if necessary.
* When instantiating a new record manually, each field must be set to something, even if the type itself is `()`, which represents no value.
Since the code generation step uses associative types, it doesn't know that a field type is actually nothing.
Creating a record manually without using the generated function `into_record` or loading it from a file is only useful to load a set of parameters into a module from an arbitrary source.
Using the record may not be the optimal solution to this problem, and another API could be created in the future.
##### Compatibility
Record may become incompatible with previous versions of Burn, depending on the chosen format.
The more compact format (bincode) store minimal information about the type, making it significantly smaller but less resilient to type changes such adding an optional field.
At some point, it might be necessary to provide a translation script that can translate a more resilient format from a previous version to a more compact one.

Binary file not shown.

After

Width:  |  Height:  |  Size: 109 KiB

View File

@ -0,0 +1,134 @@
<mxfile host="app.diagrams.net" modified="2023-04-23T15:16:56.674Z" agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" etag="yRaqMiYc5aCAWcy34Vtg" version="21.1.9" type="device">
<diagram name="Page-1" id="p9OtIezBOMlZGQ46mofZ">
<mxGraphModel dx="2181" dy="3049" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="0A-74KZi9Q9iTp0nixR_-37" value="" style="rounded=1;whiteSpace=wrap;html=1;strokeWidth=3;" vertex="1" parent="1">
<mxGeometry x="950" y="-670" width="260" height="270" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-9" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;strokeWidth=3;endArrow=open;endFill=0;endSize=14;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-1" target="0A-74KZi9Q9iTp0nixR_-3">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-1" value="Module" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="30" y="-1140" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-7" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;strokeWidth=3;endArrow=diamondThin;endFill=0;fontStyle=1;endSize=20;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-3" target="0A-74KZi9Q9iTp0nixR_-6">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-3" value="&lt;span style=&quot;font-size: 18px;&quot;&gt;Module&lt;br style=&quot;font-size: 18px;&quot;&gt;Trait&lt;br style=&quot;font-size: 18px;&quot;&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="40" y="-990" width="150" height="150" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-30" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;strokeWidth=3;endArrow=diamondThin;endFill=0;endSize=20;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-6" target="0A-74KZi9Q9iTp0nixR_-29">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-6" value="&lt;span style=&quot;font-size: 18px;&quot;&gt;Record&lt;br style=&quot;font-size: 18px;&quot;&gt;Trait&lt;br style=&quot;font-size: 18px;&quot;&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="290" y="-990" width="150" height="150" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-24" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;strokeWidth=3;endArrow=open;endFill=0;endSize=14;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-10" target="0A-74KZi9Q9iTp0nixR_-6">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-10" value="Module&lt;br style=&quot;font-size: 18px;&quot;&gt;Record" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="280" y="-1140" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-12" style="edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;strokeWidth=3;endArrow=diamondThin;endFill=0;fontStyle=1;endSize=20;" edge="1" parent="1" target="0A-74KZi9Q9iTp0nixR_-13">
<mxGeometry relative="1" as="geometry">
<mxPoint x="973" y="-585" as="sourcePoint" />
<mxPoint x="1053" y="-585" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-13" value="Associative type" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;spacingLeft=6;" vertex="1" parent="1">
<mxGeometry x="1040" y="-600" width="140" height="30" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-14" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;strokeWidth=3;endArrow=open;endFill=0;endSize=14;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" target="0A-74KZi9Q9iTp0nixR_-15">
<mxGeometry relative="1" as="geometry">
<mxPoint x="973" y="-540" as="sourcePoint" />
<mxPoint x="1023" y="-540.5" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-15" value="Implement trait" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;spacingLeft=6;" vertex="1" parent="1">
<mxGeometry x="1040" y="-555" width="140" height="30" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-16" value="&lt;span&gt;&lt;br&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="990" y="-508" width="30" height="30" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-17" value="Trait" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;spacingLeft=6;" vertex="1" parent="1">
<mxGeometry x="1040" y="-508" width="140" height="30" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-18" value="" style="rounded=1;whiteSpace=wrap;html=1;fontSize=22;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="985" y="-460" width="40" height="20" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-19" value="Type" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1;spacingLeft=6;" vertex="1" parent="1">
<mxGeometry x="1040" y="-465" width="140" height="30" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-26" value="&lt;span style=&quot;font-size: 18px;&quot;&gt;Serialize&lt;br style=&quot;font-size: 18px;&quot;&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="220" y="-545" width="120" height="120" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-27" value="&lt;span style=&quot;font-size: 18px;&quot;&gt;Deserialize&lt;br style=&quot;font-size: 18px;&quot;&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="390" y="-545" width="120" height="120" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-31" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;strokeWidth=3;endArrow=open;endFill=0;endSize=14;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-29" target="0A-74KZi9Q9iTp0nixR_-26">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-32" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;strokeWidth=3;endArrow=open;endFill=0;endSize=14;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-29" target="0A-74KZi9Q9iTp0nixR_-27">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-29" value="Record Item&lt;br style=&quot;font-size: 18px;&quot;&gt;&amp;lt;&lt;font color=&quot;#b85451&quot; style=&quot;font-size: 18px;&quot;&gt;RecordSettings&lt;/font&gt;&amp;gt;" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="240" y="-750" width="250" height="100" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-40" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;endArrow=diamondThin;endFill=0;endSize=20;strokeWidth=3;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-33" target="0A-74KZi9Q9iTp0nixR_-36">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-41" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;endArrow=diamondThin;endFill=0;endSize=20;strokeWidth=3;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-33" target="0A-74KZi9Q9iTp0nixR_-35">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-42" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;endArrow=diamondThin;endFill=0;endSize=20;strokeWidth=3;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-33" target="0A-74KZi9Q9iTp0nixR_-34">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-33" value="&lt;span style=&quot;font-size: 18px;&quot;&gt;Record&lt;br style=&quot;font-size: 18px;&quot;&gt;Settings&lt;br style=&quot;font-size: 18px;&quot;&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="520" y="-1150" width="150" height="150" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-34" value="Float Elem" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="670" y="-980" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-35" value="Int Elem" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="670" y="-890" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-36" value="Recorder" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="670" y="-800.5" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-39" value="&lt;b&gt;&lt;font style=&quot;font-size: 23px;&quot;&gt;Legend&lt;/font&gt;&lt;/b&gt;" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" vertex="1" parent="1">
<mxGeometry x="950" y="-650" width="260" height="30" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-44" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;endArrow=diamondThin;endFill=0;endSize=20;strokeWidth=3;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-47" target="0A-74KZi9Q9iTp0nixR_-50">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-45" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;endArrow=diamondThin;endFill=0;endSize=20;strokeWidth=3;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-47" target="0A-74KZi9Q9iTp0nixR_-49">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-46" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;endArrow=diamondThin;endFill=0;endSize=20;strokeWidth=3;fontSize=18;" edge="1" parent="1" source="0A-74KZi9Q9iTp0nixR_-47" target="0A-74KZi9Q9iTp0nixR_-48">
<mxGeometry relative="1" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-47" value="&lt;span style=&quot;font-size: 18px;&quot;&gt;Recorder&lt;br style=&quot;font-size: 18px;&quot;&gt;&lt;/span&gt;" style="rhombus;whiteSpace=wrap;html=1;strokeWidth=3;fontSize=18;fontStyle=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
<mxGeometry x="890" y="-1150" width="150" height="150" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-48" value="Record Args" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="1040" y="-980" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-49" value="Record Output" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="1040" y="-890" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-50" value="Load Args" style="rounded=1;whiteSpace=wrap;html=1;fontSize=18;fontStyle=1;strokeWidth=3;fillColor=#fff2cc;strokeColor=#d6b656;" vertex="1" parent="1">
<mxGeometry x="1040" y="-800.5" width="170" height="70" as="geometry" />
</mxCell>
<mxCell id="0A-74KZi9Q9iTp0nixR_-52" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;rounded=0;" edge="1" parent="1">
<mxGeometry width="80" relative="1" as="geometry">
<mxPoint x="-660" y="-1310" as="sourcePoint" />
<mxPoint x="-580" y="-1310" as="targetPoint" />
<Array as="points" />
</mxGeometry>
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>