canvas-lms/gems/i18n_tasks
Aaron Ogata 0babdc39f1 fix webpack memory regression from locale change
refs DE-1022

There have been reports of a memory regression caused by 1e8f45e ("use per-language instead of per-scope translation files"). This is indeed the case from the profiling steps below.

```
apt update && apt install -y time
rm -rf public/javascripts/translations/* && bundle exec rake i18n:generate_js

for run in {1..5}; do
  rm -rf node_modules/.cache/* && rm -rf public/dist/webpack-dev/*
  (/usr/bin/time -v yarn run webpack-development) 2>&1 >/dev/null | grep "Maximum resident set size"
done
```

Before 1e8f45e:
Maximum resident set size (kbytes): 3785704
Maximum resident set size (kbytes): 3991196
Maximum resident set size (kbytes): 3894452
Maximum resident set size (kbytes): 3858516
Maximum resident set size (kbytes): 3768916

After 1e8f45e:
Maximum resident set size (kbytes): 4556988
Maximum resident set size (kbytes): 4711868
Maximum resident set size (kbytes): 4862212
Maximum resident set size (kbytes): 4763156
Maximum resident set size (kbytes): 4803300

With this change, the memory usage drops to below the previous value, which would be expected from this change.

Maximum resident set size (kbytes): 3627176
Maximum resident set size (kbytes): 3608840
Maximum resident set size (kbytes): 3961552
Maximum resident set size (kbytes): 3887148
Maximum resident set size (kbytes): 3863004

The root cause appears to be that webpack is using a significant amount of memory to translate the many setLazyTranslation calls for each scope.

For the quick fix, we choose to do the fastest fix of wrapping these calls in an IIFE. A longer term fix would be to re-write the mergeI18nTranslations API to require only one call. This is work already scheduled for this project that will be handled later as it requires consensus and testing on different file formats.

Additionally, we could also convert the file to a JSON file, but we would lose the ability to lazy load each translation scope, and would require further performance testing. If we chose to do that work, we may be able to have a memory usage like:

Maximum resident set size (kbytes): 3285508

Change-Id: I827643a6706c163ab9b0f2a8a597c4d668c36e32
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/285584
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
2022-02-22 14:08:53 +00:00
..
lib fix webpack memory regression from locale change 2022-02-22 14:08:53 +00:00
spec RuboCop: Style/StringLiterals, Style/StringLiteralsInInterpolation 2021-11-25 14:03:06 +00:00
.rspec raise bundler minimum requirement 2021-02-19 22:49:02 +00:00
Gemfile RuboCop: Style/StringLiterals, Style/StringLiteralsInInterpolation 2021-11-25 14:03:06 +00:00
Rakefile add frozen_string_literal comment to engines and gems 2021-03-30 18:14:15 +00:00
i18n_tasks.gemspec RuboCop: Style/StringLiterals, Style/StringLiteralsInInterpolation 2021-11-25 14:03:06 +00:00
test.sh simplify gem test harnesses 2016-01-19 17:52:58 +00:00