2015-10-29 04:09:43 +08:00
Work-in-progress benchmarks.
benchmarking: measure and analyze scripts
This uses wall-clock time (for now) so it's noisier than alternatives
(cachegrind, CPU perf-counters), but it's still valuable. In a future diff we
can make it use those.
`measure.py` outputs something that `analyze.py` can understand, but you can use `analyze.py` without `measure.py` too. The file format is simple:
```
$ cat measurements.txt
factory_ms_jsc_jit 13.580322265625
factory_ms_jsc_jit 13.659912109375
factory_ms_jsc_jit 13.67919921875
factory_ms_jsc_nojit 12.827880859375
factory_ms_jsc_nojit 13.105224609375
factory_ms_jsc_nojit 13.195068359375
factory_ms_node 40.4891400039196
factory_ms_node 40.6669420003891
factory_ms_node 43.52413299679756
ssr_pe_cold_ms_jsc_jit 43.06005859375
...
```
(The lines do not need to be sorted.)
Comparing 0.14.0 vs master:
```
$ ./measure.py react-0.14.0.min.js >014.txt
Measuring SSR for PE benchmark (30 trials)
..............................
Measuring SSR for PE with warm JIT (3 slow trials)
...
$ ./measure.py react.min.js >master.txt
Measuring SSR for PE benchmark (30 trials)
..............................
Measuring SSR for PE with warm JIT (3 slow trials)
...
$ ./analyze.py 014.txt master.txt
Comparing 014.txt (control) vs master.txt (test)
Significant differences marked by ***
% change from control to test, with 99% CIs:
* factory_ms_jsc_jit
% change: -0.56% [ -2.51%, +1.39%]
means: 14.037 (control), 13.9593 (test)
* factory_ms_jsc_nojit
% change: +1.23% [ -1.18%, +3.64%]
means: 13.2586 (control), 13.4223 (test)
* factory_ms_node
% change: +3.53% [ +0.29%, +6.77%] ***
means: 42.0529 (control), 43.54 (test)
* ssr_pe_cold_ms_jsc_jit
% change: -6.84% [ -9.04%, -4.65%] ***
means: 44.2444 (control), 41.2187 (test)
* ssr_pe_cold_ms_jsc_nojit
% change: -11.81% [-14.66%, -8.96%] ***
means: 52.9449 (control), 46.6953 (test)
* ssr_pe_cold_ms_node
% change: -2.70% [ -4.52%, -0.88%] ***
means: 96.8909 (control), 94.2741 (test)
* ssr_pe_warm_ms_jsc_jit
% change: -17.60% [-22.04%, -13.16%] ***
means: 13.763 (control), 11.3439 (test)
* ssr_pe_warm_ms_jsc_nojit
% change: -20.65% [-22.62%, -18.68%] ***
means: 30.8829 (control), 24.5074 (test)
* ssr_pe_warm_ms_node
% change: -8.76% [-13.48%, -4.03%] ***
means: 30.0193 (control), 27.3964 (test)
$
```
2015-11-19 08:16:34 +08:00
## Running the suite
```
$ ./measure.py react-a.min.js >a.txt
$ ./measure.py react-b.min.js >b.txt
$ ./analyze.py a.txt b.txt
```
2015-10-29 04:09:43 +08:00
## Running one
One thing you can do with them is benchmark initial render time for a realistic hierarchy:
```
$ which jsc
/System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/Resources/jsc
$ jsc react-0.14.0.min.js bench-pe-es5.js -e 'var START=Date.now(); React.renderToString(React.createElement(Benchmark)); var END=Date.now(); print(END-START);'
45
```
Substitute `js` or `v8` for `jsc` to use SpiderMonkey or V8, respectively, if you've installed them.
## Creating one
To create one, copy `extract-component.js` to your clipboard and paste it into the Chrome console on facebook.com, perhaps after changing the root ID if you don't want the tree with ID `.0` .
Then to convert it to ES5:
```
babel --whitelist react,react.displayName --compact false bench-pe.js >bench-pe-es5.js
```
