Update docs for mode B!

Includes:
* new performance numbers. Very exciting. Very cool. Ambient lighting
helps!
* deprecating the old 8-color mode ("mode 8C" under the new naming
scheme). This simplifies things, and a newer, faster format is waiting
in the wings...
* misc updates based on what I've learned
This commit is contained in:
sz 2024-02-12 21:29:30 -06:00
parent 95afff4d9e
commit 390041eb70
No known key found for this signature in database
GPG Key ID: 056CD84A187C0C44
6 changed files with 53 additions and 27 deletions

View File

@ -84,7 +84,7 @@ These properties may appear to be magical as you consider them more, and they do
2. wirehair requires the file contents to be stored in RAM
* this relates to the size limit!
This constraint is less of an obstacle than it may seem -- the fountain codes are essentially being used as a wire format, and the encoder and decoder could agree on a scheme to split up, and then reassemble, larger files. Cimbar does not (yet?) implement this, however!
The size constraint is less of an obstacle than it may seem -- the fountain codes are essentially being used as a wire format, and the encoder and decoder could agree on a scheme to split up, and then reassemble, larger files. Cimbar does not (yet?) implement this, however!
## Implementation: Decoder

View File

@ -4,36 +4,48 @@
## Numbers of note
* The barcode is `1024x1024` pixels. The individual tiles are `8x8` in a `9x9` grid (there is an empty row/column of spacing on either side)
* **7500** or 8750 bytes per cimbar image, after error correction
* **7500** bytes per cimbar image, after error correction
* There are 16 possible symbols per tile, encoding 4 bits
* There are 4 or 8 possible colors, encoding an additional 2-3 bits per tile.
* These 6-7 bits per tile work out to a maximum of 9300-10850 bytes per barcode, though in practice this number is reduced by error correction.
* These 6 bits per tile work out to a maximum of 9300 bytes per barcode, though in practice this number is reduced by error correction.
* The default ecc setting is 30/155, which is how we go from 9300 -> 7500 bytes of real data for a 4-color cimbar image.
* Reed Solomon is not perfect for this use case -- specifically, it corrects byte errors, and cimbar errors tend to involve 1-3 bits at a time. However, since Reed Solomon implementations are ubiquitous, it is currently in use.
## Current sustained benchmark
* 4-color cimbar with ecc=30:
* 4,717,525 bytes (after compression) in 45s -> 838 kilobits/s (~104 KB/s)
* `mode B` (8x8 4-color) cimbar with ecc=30/155:
* 4,689,084 bytes (after compression) in 43s -> 872 kilobits/s (~109 KB/s)
* mode B was introduced in 0.6.0, and should work in a wide variety of scenarios
* 8-color cimbar with ecc=30:
* *legacy* `mode 4C` (8x8 4-color) cimbar with ecc=30/155:
* 4,717,525 bytes (after compression) in 45s -> 838 kilobits/s (~104 KB/s)
* the original configuration. Mostly replaced by mode B.
* *deprecated* `mode 8C` (8x8 8-color) cimbar with ecc=30/155:
* 4,717,525 bytes in 40s -> 943 kilobits/s (~118 KB/s)
* removed in 0.6.0. 8-color has always been inconsistent, and needs future research
* *beta* `mode S` (5x5 4-color) cimbar with ecc=40/216 (note: not finalized, and requires a special build)
* safely >1 Mbit/s
* format still a WIP. To be continued...
* details:
* cimbar has built-in compression using zstd. What's being measured here is bits over the wire, e.g. data after compression is applied.
* these numbers are using https://github.com/sz3/cfc, running with 4 CPU threads on a Qualcomm Snapdragon 625
* perhaps I will buy a new cell phone to inflate the benchmark numbers.
* the sender is the cimbar.org wasm implementation. An equivalent command line is `./cimbar_send /path/to/file -s`
* these numbers are using https://github.com/sz3/cfc, running with 4 CPU threads on a venerable Qualcomm Snapdragon 625
* more modern cell CPUs run the decoder more quickly, but it turns out that this does not benefit performance much: the camera is usually the bottleneck.
* the sender is the cimbar.org wasm implementation. An equivalent command line is `./cimbar_send /path/to/file`
* cimbar.org uses the `shakycam` option to allow the receiver to detect/discard "in between" frames as part of the scan step. This allows it to spend more processing time decoding real data.
* burst rate can be higher (or lower)
* to this end, lower ecc settings *can* provide better burst rates
* 4-color cimbar is currently preferred, and will give more consistent transfer speeds.
* 8-color cimbar should be considered a prototype within a prototype. It is considerably more sensitive to lighting conditions and color tints.
* to this end, lower ecc settings *can* provide better burst rates. I've aimed for a balance of performance and reliability.
* cimbar `mode B` is preferred, and should be the most reliable.
* The older `mode 4C` *may* give more consistent transfer speeds in certain scenarios, but is mostly included for backwards-compatibility reasons.
* other notes:
* having better lighting in the frame often leads to better results -- this is why cimbar.org has a (mostly) white background. cfc uses android's auto-exposure, auto-focus, etc (it's a very simple app). Good ambient light -- or a white background -- can lead to more consitent quality frame capture.
* having better lighting in the frame often leads to better results -- this is why cimbar.org has a (mostly) white background. cfc uses android's auto-exposure, auto-focus, etc (it's a demo app). Good ambient light -- or a white background -- can lead to more consitent quality frame capture.
* screen brightness on the sender is good, but ambient light is better.
* because of the lighting/exposure question, landscape *may* be better than portrait.
* cfc currently has a low resolution, so the cimbar frame should take up as much of the display as possible (trust the guide brackets)
* the cimbar frame should take up as much of the display as possible (trust the guide brackets)
* the format is designed to decode at resolutions as low as 700x700, but performance may suffer.
* similarly, it's best to keep the camera angle straight-on -- instead of at an angle -- to decode the whole image successfully. Decodes should still happen at higher angles, but the "smaller" part of the image may have more errors than the ECC can deal with.
* other things to be wary of:
* glare from light sources.

View File

@ -5,7 +5,7 @@
Behold: an experimental barcode format for air-gapped data transfer.
It can sustain speeds of 943+ kilobits/s (~118 KB/s) using just a computer monitor and a smartphone camera!
It can sustain speeds of 872 kilobits/s (~109 KB/s) using just a computer monitor and a smartphone camera!
<p align="center">
<img src="https://github.com/sz3/cimbar-samples/blob/v0.5/6bit/4cecc30f.png" width="70%" title="A non-animated cimbar code" >

13
TODO.md
View File

@ -10,6 +10,7 @@ Performance optimizations aside, there are a number of paths that might be inter
* proper metadata/header information?
* would be nice to be able to determine ecc/#colors/#symbols from the cimbar image itself?
* The bottom right corner is the obvious place to reclaim space to make this possible.
* this is complicated by potential aspect ratio changes for future cimbar modes.
* multi-frame decoding?
* when decoding a static cimbar image, it would be useful to be able to use prior (unsuccessful) decode attempts to inform a future decode, and -- hopefully -- increase the probability of success. Currently, all frames are decoded independently.
* there is already a granular confidence metric that could be reused -- the `distance` that's tracked when decoding symbol tiles...
@ -18,18 +19,18 @@ Performance optimizations aside, there are a number of paths that might be inter
* there is surely a more optimal set -- a more rigorous approach should yield lower error rates!
* but, more importantly, it may be possible to go up to 32 symbols, and encode 5 symbol bits per tile?
* optimal symbol size?
* the symbols that make up each cell on the cimbar grid are 8x8 (in a 9x9 grid).
* this is because imagehash was on 8x8 tiles!
* smaller sizes might also work?
* the symbols that make up each cell on the cimbar grid are 8x8 (in a 9x9 grid). this is because imagehash was on 8x8 tiles!
* smaller sizes might also work? I've been looking into 5x5 (in a 6x6 grid) as a starting point. It seems promising.
* the limiting factor is the hamming distance between each image hash "bucket", and the 9Xth percentile decoding errors.
* optimal color set?
* the 4-color (2 bit) pallettes seem reasonable. 8-color, perhaps less so?
* this may be a limitation of the algorithm/approach, however. Notably, since each symbol is drawn with one pallette color, all colors need sufficient contrast against the backdrop (#000 or #FFF, depending). This constrains the color space somewhat, and less distinct colors == more errors.
* in addition to contrast, there is interplay (that I don't currently understand) between the overall brightness of the image and the exposure time needed for high framerate capture. More clean frames == more troughput.
* in addition to contrast, there is interplay between the overall brightness of the image and the exposure time needed for high framerate capture. More clean frames == more troughput.
* the camera framerate in the CFC app is limited by auto-exposure and auto-focus behavior. A newer/better decoder app might be helpful.
* optimal grid size?
* 1024x1024 is a remnant of the early prototyping process. There is nothing inherently special about it (except that it fits on a 1920x1080 screen, which seems good)
* the tile grid itself is 1008x1008 (1008 == 9x112 -- there are 112 tile rows and columns)
* a smaller grid would be less information dense, but more resilient to errors. Probably.
* a smaller grid *could* be more resilient to errors, at the expense of data capacity.
* optimal grid shape?
* it's a square because QR codes are square. That's it. Should it be?
* I'm strongly considering 4:3 for the next revision.
@ -41,6 +42,8 @@ Performance optimizations aside, there are a number of paths that might be inter
* proper GPU support (OpenCV + openCL) on android?
* It *might* be useful. [CFC]((https://github.com/sz3/cfc) is the current test bed for this.
* wasm decoder?
* android is going to kick CFC out of the store! (testing requirement)
* so it might be time to write this...
* probably needs to use Web Workers
* in-browser GPGPU support would be interesting (but I'm not counting on it)
* ???

22
WASM.md
View File

@ -4,16 +4,30 @@
## Releases
wasm and asm.js releases are available [here](https://github.com/sz3/libcimbar/releases/latest). The wasm build is what cimbar.org uses. The asm.js build can be downloaded, extracted, and run in a local web browser.
wasm and asm.js releases are available [here](https://github.com/sz3/libcimbar/releases/latest). The wasm build is what cimbar.org uses. [cimbar_js.html](https://github.com/sz3/libcimbar/releases/latest/cimbar_js.html) can be downloaded and opened/run in a local web browser -- no install required.
## Build
To build opencv.js (and the static libraries we'll need to build against opencv)...
To build, use the `package-wasm.sh` script in a docker container:
```
docker run --mount type=bind,source="$(pwd)",target="/usr/src/app" -it emscripten/emsdk:3.1.39
```
Then, inside the container:
```
bash /usr/src/app/package-wasm.sh
```
## Alternative build for the adventurous
Alternatively, if you have a local emscripten setup, you can try to run the package-wasm.sh commands piecemeal:
To build opencv.js:
```
cd /path/to/opencv
mkdir opencv-build-wasm
cd opencv-build-wasm
python ../platforms/js/build_js.py build_wasm --build_wasm --emscripten_dir=/path/to/emscripten
python3 ../platforms/js/build_js.py build_wasm --build_wasm --emscripten_dir=/path/to/emscripten
```
With opencv.js built:
@ -22,7 +36,7 @@ mkdir build-wasm
cd build-wasm
source /path/to/emscripten/emsdk/emsdk_env.sh
emcmake cmake .. -DUSE_WASM=1 -DOPENCV_DIR=/path/to/opencv
make -j7 install
make -j5 install
```
(do `-DUSE_WASM=2` to use asm.js instead of wasm)

View File

@ -27,8 +27,5 @@ namespace cimbar
static constexpr unsigned cell_size = 8;
static constexpr unsigned cell_offset = 8;
static constexpr unsigned cells_per_col = 112;
static constexpr int interleave_partitions = 2;
static constexpr int fountain_chunks_per_frame = 10;
};
}