Merge branch 'rich-text-small-screen' of https://github.com/cryptpad/cryptpad into rich-text-small-screen

.github/ISSUE_TEMPLATE/bug_resolution.yml

@@ -85,6 +85,8 @@ body:
       label: Version
       description: What version of CryptPad are you running?
       options:
+        - 5.5.0
+        - 5.4.1
         - 5.4.0
         - 5.3.0
         - 5.2.1

.github/ISSUE_TEMPLATE/config.yml

@@ -9,6 +9,3 @@ contact_links:
   - name: CryptPad.fr flagship instance issue
     url: https://cryptpad.fr/support/#new
     about: Issue with an account or document on cryptpad.fr only
-  - name: Report a security vulnerability
-    url: https://ouvaton.link/pOgHev
-    about: Please give us appropriate time to verify, respond and fix before disclosure

CHANGELOG.md

@@ -1,3 +1,140 @@
+
+# 5.5.0
+
+## Features
+
+- Moderation and content deletion features [#1253](https://github.com/cryptpad/cryptpad/pull/1253)
+  * Moderation
+    * archive an entire account and its owned documents from its public key
+    * restore this entire account if necessary
+  * Placeholder
+    * unavailable documents now provide improved messages communicating the reason they are unavailable:
+      - Deleted by an owner
+      - Deleted by an admin + reason from admin team (user account or document)
+      - Deleted for inactivity (documents not stored in a user drive and inactive)
+      - Protected with a new password (user account or document)
+    * it is no longer possible to re-use an previous password for a password-protected document
+- Only Office upgrade to 7.3.3.60
+  - New version of x2t for document conversions
+
+## Improvements
+
+- Accessibility
+  - Add text labels to elements [#1163](https://github.com/cryptpad/cryptpad/issues/1163), [#1122](https://github.com/cryptpad/cryptpad/issues/1122), [#1123](https://github.com/cryptpad/cryptpad/issues/1123), [#1124](https://github.com/cryptpad/cryptpad/issues/1124), [#1128](https://github.com/cryptpad/cryptpad/issues/1128), [#1129](https://github.com/cryptpad/cryptpad/issues/1129), [#1131](https://github.com/cryptpad/cryptpad/issues/1131), [#1140](https://github.com/cryptpad/cryptpad/issues/1140), [#1150](https://github.com/cryptpad/cryptpad/issues/1150), [#1159](https://github.com/cryptpad/cryptpad/issues/1159), [#1195](https://github.com/cryptpad/cryptpad/issues/1195), [#1194](https://github.com/cryptpad/cryptpad/issues/1194)
+  - Enable zooming and scaling [#1130](https://github.com/cryptpad/cryptpad/issues/1130)
+  - Turn login error message into an instruction [#1207](https://github.com/cryptpad/cryptpad/issues/1207)
+- Mobile usage
+  - Fix the instance links layout on the home-page [#1085](https://github.com/cryptpad/cryptpad/issues/1085)
+  - Display full file upload progress modal [#1086](https://github.com/cryptpad/cryptpad/issues/1086)
+  - Add text to Teams buttons [#1093](https://github.com/cryptpad/cryptpad/issues/1093)
+  - Fix button spacings [#1104](https://github.com/cryptpad/cryptpad/issues/1104), [#1106](https://github.com/cryptpad/cryptpad/issues/1106)
+  - Add even space between category buttons [#1113](https://github.com/cryptpad/cryptpad/pull/1113) thanks to @lemondevxyz
+  - Allow the About panel to be closed [#1088](https://github.com/cryptpad/cryptpad/issues/1088)
+  - Calendar
+    - Display full event edit panel [#1094](https://github.com/cryptpad/cryptpad/issues/1094)
+    - Make menu usable [#971](https://github.com/cryptpad/cryptpad/issues/971)
+  - Kanban
+    - Hide markdown help button instead of breaking the layout [#1117](https://github.com/cryptpad/cryptpad/issues/1117)
+    - Added margin for horizontal scroll [#1039](https://github.com/cryptpad/cryptpad/issues/1039)
+    - Remove margin from cards and columns [#1120](https://github.com/cryptpad/cryptpad/issues/1120)
+
+- Instance admin
+  - Added a warning to `/admin/#stats` about a process that can crash the instance [#1176](https://github.com/cryptpad/cryptpad/issues/1176)
+  - Added a setting to display a status page for the instance [#1172](https://github.com/cryptpad/cryptpad/issues/1172)
+- Replace the "sign up" button on the log-in page with a link [#1164](https://github.com/cryptpad/cryptpad/issues/1164)
+- Add support for Webp images [#1008] thanks @lukasdotcom
+- improvements and bug fixes for the archival of inactive documents
+
+## Fixes
+
+- Revert a button spacing regression introduced with 5.4.0 [#1229](https://github.com/cryptpad/cryptpad/pull/1229)
+- Login bug on the new Safari following macOS/iPadOS 14 [#1257](https://github.com/cryptpad/cryptpad/issues/1257)
+- Mermaid diagrams were sometimes displayed over each other in Code documents [#1244](https://github.com/cryptpad/cryptpad/issues/1244)
+- Own responses to a form could not be deleted [#1239](https://github.com/cryptpad/cryptpad/issues/1239)
+- Timezone differences caused errors in Forms "date/time" polls
+- The large attachment button did not look consistent in Forms [#1237](https://github.com/cryptpad/cryptpad/issues/1237)
+- The recent tab in the drive was missing column titles [#1233](https://github.com/cryptpad/cryptpad/issues/1233)
+- An export file type dropdown was hidden inside a popup [#1241](https://github.com/cryptpad/cryptpad/issues/1241)
+- Guest emoji avatars were not displayed constistently [#1188](https://github.com/cryptpad/cryptpad/issues/1188)
+- "Early Access" apps were not shown on the instance home page even when active
+- OnlyOffice document conversions
+  - Fix PDF export from Presentation document [#913](https://github.com/cryptpad/cryptpad/issues/913)
+  - Print sheets with long links [#1032](https://github.com/cryptpad/cryptpad/issues/1032)
+  - Fix some .xlsx imports [#1240](https://github.com/cryptpad/cryptpad/issues/1240)
+
+## Dependencies
+
+- Pin CKEditor to 4.22.1 [#1248](https://github.com/cryptpad/cryptpad/issues/1248)
+- Prevent x2t from being cached [#1278](https://github.com/cryptpad/cryptpad/issues/1278)
+
+## Deployment
+
+We now support Nginx with two configurations (find more information in our [administrator guide](https://docs.cryptpad.org/en/admin_guide/installation.html#install-and-configure-nginx)):
+* New recommended "basic" nginx config for small instances: `example.nginx.conf`
+* Update to the old "advanced" config: `example-advanced.nginx.conf`
+  * Add 2 lines in the "blob|block" section
+```diff
+# Requests for blobs and blocks are now proxied to the API server
+# This simplifies NGINX path configuration in the event they are being hosted in a non-standard location
+# or with odd unexpected permissions. Serving blobs in this manner also means that it will be possible to
+# enforce access control for them, though this is not yet implemented.
+# Access control (via TOTP 2FA) has been added to blocks, so they can be handled with the same directives.
+location ~ ^/(blob|block)/.*$ {
+    if ($request_method = 'OPTIONS') {
+        add_header 'Access-Control-Allow-Origin' "${allowed_origins}";
+        add_header 'Access-Control-Allow-Credentials' true;
+        add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+        add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
+        add_header 'Access-Control-Max-Age' 1728000;
+        add_header 'Content-Type' 'application/octet-stream; charset=utf-8';
+        add_header 'Content-Length' 0;
+        return 204;
+    }
+    # Since we are proxying to the API server these headers can get duplicated
+    # so we hide them
+    proxy_hide_header 'X-Content-Type-Options';
+    proxy_hide_header 'Access-Control-Allow-Origin';
+    proxy_hide_header 'Permissions-Policy';
+    proxy_hide_header 'X-XSS-Protection';
++   proxy_hide_header 'Cross-Origin-Resource-Policy';
++   proxy_hide_header 'Cross-Origin-Embedder-Policy';
+    proxy_pass http://localhost:3000;
+}
+```
+  * Fix DrawIO hash not matching the latest version
+```diff
+    # draw.io uses inline script tags in it's index.html. The hashes are added here.
+    if ($uri ~ ^\/components\/drawio\/src\/main\/webapp\/index.html.*$) {
+-        set $scriptSrc "'self' 'sha256-6zAB96lsBZREqf0sT44BhH1T69sm7HrN34rpMOcWbNo=' 'sha256-6g514VrT/cZFZltSaKxIVNFF46+MFaTSDTPB8WfYK+c=' resource: https://${main_domain}";
++        set $scriptSrc "'self' 'sha256-dLMFD7ijAw6AVaqecS7kbPcFFzkxQ+yeZSsKpOdLxps=' 'sha256-6g514VrT/cZFZltSaKxIVNFF46+MFaTSDTPB8WfYK+c=' resource: https://${main_domain}";
+    }
+```
+
+## Upgrade notes
+
+If you are upgrading from a version older than `5.4.1` please read the upgrade notes of all versions between yours and `5.4.1` to avoid configuration issues.
+
+To upgrade:
+
+1. Stop your server
+2. Get the latest code with git
+
+```bash
+git fetch origin --tags
+git checkout 5.5.0
+```
+
+3. Update dependencies
+
+```bash
+npm ci
+npm run install:components
+```
+
+4. Restart your server
+
+5. Review your instance's checkup page to ensure that you are passing all tests
+
 # 5.4.1
 
 ## Goals

customize.dist/index.html

@@ -4,7 +4,7 @@
 <head>
     <title data-localization="main_title">CryptPad: Collaboration suite, encrypted and open-source</title>
     <meta content="text/html; charset=utf-8" http-equiv="content-type"/>
-    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
     <link rel="icon" type="image/png" href="/customize/favicon/main-favicon.png" id="favicon"/>
     <script src="/customize/pre-loading.js?ver=1.1"></script>
     <link href="/customize/src/pre-loading.css?ver=1.0" rel="stylesheet" type="text/css">

customize.dist/loading.js

@@ -8,9 +8,10 @@ define([
     var urlArgs = window.location.href.replace(/^.*\?([^\?]*)$/, function (all, x) { return x; });
     var elem = document.createElement('div');
     elem.setAttribute('id', 'cp-loading');
+
     elem.innerHTML = [
         '<div class="cp-loading-logo">',
-            '<img class="cp-loading-cryptofist" src="/customize/CryptPad_logo.svg?' + urlArgs + '">',
+            '<img class="cp-loading-cryptofist" src="/customize/CryptPad_logo.svg?' + urlArgs + '" alt="' + Messages.label_logo + '">',
         '</div>',
         '<div class="cp-loading-container">',
             '<div class="cp-loading-spinner-container">',

customize.dist/login.js

@@ -10,6 +10,7 @@ define([
     '/common/common-constants.js',
     '/common/common-interface.js',
     '/common/common-feedback.js',
+    '/common/hyperscript.js',
     '/common/outer/local-store.js',
     '/customize/messages.js',
     '/components/nthen/index.js',
@@ -20,7 +21,7 @@ define([
     '/components/tweetnacl/nacl-fast.min.js',
     '/components/scrypt-async/scrypt-async.min.js', // better load speed
 ], function ($, Listmap, Crypto, Util, NetConfig, Cred, ChainPad, Realtime, Constants, UI,
-            Feedback, LocalStore, Messages, nThen, Block, Hash, ServerCommand) {
+            Feedback, h, LocalStore, Messages, nThen, Block, Hash, ServerCommand) {
     var Exports = {
         Cred: Cred,
         Block: Block,
@@ -103,7 +104,8 @@ define([
         return opt;
     };
 
-    var loadUserObject = Exports.loadUserObject = function (opt, cb) {
+    var loadUserObject = Exports.loadUserObject = function (opt, _cb) {
+        var cb = Util.once(Util.mkAsync(_cb));
         var config = {
             websocketURL: NetConfig.getWebsocketURL(),
             channel: opt.channelHex,
@@ -121,6 +123,9 @@ define([
         .on('ready', function () {
             setTimeout(function () { cb(void 0, rt); });
         })
+        .on('error', function (info) {
+            cb(info.type, {reason: info.message});
+        })
         .on('disconnect', function (info) {
             cb('E_DISCONNECT', info);
         });
@@ -210,6 +215,16 @@ define([
                         return void console.log("Block requires 2FA");
                     }
 
+                    if (err === 404 && response && response.reason) {
+                        waitFor.abort();
+                        w.abort();
+                        /*
+                        // the following block prevent users from re-using an old password
+                        if (isRegister) { return void cb('HAS_PLACEHOLDER'); }
+                        */
+                        return void cb('DELETED_USER', response);
+                    }
+
                     // Some other error?
                     if (err) {
                         console.error(err);
@@ -291,6 +306,7 @@ define([
             loadUserObject(opt, waitFor(function (err, rt) {
                 if (err) {
                     waitFor.abort();
+                    if (err === 'EDELETED') { return void cb('DELETED_USER', rt); }
                     return void cb(err);
                 }
 
@@ -388,6 +404,7 @@ define([
             loadUserObject(opt, waitFor(function (err, rt) {
                 if (err) {
                     waitFor.abort();
+                    if (err === 'EDELETED') { return void cb('DELETED_USER', rt); }
                     return void cb('MODERN_REGISTRATION_INIT');
                 }
 
@@ -589,15 +606,26 @@ define([
                                 break;
                             case 'INVAL_USER':
                                 UI.removeLoadingScreen(function () {
-                                    UI.alert(Messages.login_invalUser, function () {
+                                    UI.alert(Messages.login_notFilledUser , function () {
                                         hashing = false;
                                         $('#password').focus();
                                     });
                                 });
                                 break;
+/*
+                            case 'HAS_PLACEHOLDER':
+                                UI.errorLoadingScreen('UNAVAILABLE', true, true);
+                                break;
+*/
+                            case 'DELETED_USER':
+                                UI.errorLoadingScreen(
+                                    UI.getDestroyedPlaceholder(result.reason, true), true, () => {
+                                        window.location.reload();
+                                    });
+                                break;
                             case 'INVAL_PASS':
                                 UI.removeLoadingScreen(function () {
-                                    UI.alert(Messages.login_invalPass, function () {
+                                    UI.alert(Messages.login_notFilledPass, function () {
                                         hashing = false;
                                         $('#password').focus();
                                     });

customize.dist/pages.js

@@ -56,16 +56,15 @@ define([
     };
 
     var languageSelector = function () {
-        var options = [];
         var languages = Msg._languages;
         var selected = Msg._languageUsed;
         var keys = Object.keys(languages).sort();
-        keys.forEach(function (l) {
+        var options = keys.map(function (l) {
-            var attr = { value: l, role: 'option'};
+            var attr = { value: l };
             if (selected === l) { attr.selected = 'selected'; }
-            options.push(h('option', attr, languages[l]));
+            return h('option', attr, languages[l]);
         });
-        var select = h('select', {role: 'listbox', 'label': 'language'}, options);
+        var select = h('select', { 'aria-label': Msg.selectLanguage }, options);
         $(select).change(function () {
             Language.setLanguage($(select).val() || '', null, function () {
                 window.location.reload();
@@ -95,7 +94,7 @@ define([
         return h('a', attrs, [icon, text]);
     };
 
-    Pages.versionString = "5.4.1";
+    Pages.versionString = "5.5.0";
 
     var customURLs = Pages.customURLs = {};
     (function () {
@@ -103,7 +102,7 @@ define([
             source: 'https://github.com/cryptpad/cryptpad',
         };
         var l = Msg._getLanguage();
-        ['imprint', 'privacy', 'terms', 'roadmap', 'source'].forEach(function (k) {
+        ['imprint', 'privacy', 'terms', 'status', 'roadmap', 'source'].forEach(function (k) {
             var value = AppConfig[k];
             //console.log('links', k, value);
             if (value === false) { return; }

customize.dist/pages/index.js

@@ -121,6 +121,7 @@ define([
         var imprintLink = fastLink('imprint');
         var privacyLink = fastLink('privacy');
         var termsLink = fastLink('terms');
+        var statusLink = fastLink('status');
 
         var notice;
 /*  Admins can specify a notice to display in application_config.js via the `homeNotice` attribute.
@@ -133,6 +134,7 @@ define([
         }
 
         // instance title
+
         var instanceTitle = h('h1.cp-instance-title', Pages.Instance.name);
 
         // instance location
@@ -181,7 +183,8 @@ define([
                                 termsLink,
                                 privacyLink,
                                 imprintLink,
-                                h('a', {href:"/contact.html"}, Msg.contact)
+                                h('a', {href:"/contact.html"}, Msg.contact),
+                                statusLink,
                             ])
                         ]),
                         h('div.cp-apps.col-lg-6', [

customize.dist/pages/login.js

@@ -15,28 +15,38 @@ define([
                     h('div.col-md-3'),
                     h('div#userForm.form-group.hidden.col-md-6', [
                         h('div.cp-login-instance', Msg._getKey('login_instance', [ Pages.Instance.name ])),
-                        h('input.form-control#name', {
+                        h('div.big-container', [
-                            name: 'name',
+                            h('div.input-container', [
-                            type: 'text',
+                                h('label.cp-default-label', { for: 'name' }, Msg.login_username),
-                            autocomplete: 'off',
+                                h('input.form-control#name', {
-                            autocorrect: 'off',
+                                    name: 'name',
-                            autocapitalize: 'off',
+                                    type: 'text',
-                            spellcheck: false,
+                                    autocomplete: 'off',
-                            placeholder: Msg.login_username,
+                                    autocorrect: 'off',
-                            autofocus: true,
+                                    autocapitalize: 'off',
-                        }),
+                                    spellcheck: false,
-                        h('input.form-control#password', {
+                                    placeholder: Msg.login_username,
-                            type: 'password',
+                                    autofocus: true,
-                            'name': 'password',
+                                }),
-                            placeholder: Msg.login_password,
+                            ]),
-                        }),
+                            h('div.input-container', [
+                                h('label.cp-default-label', { for: 'password' }, Msg.login_password),
+                                h('input.form-control#password', {
+                                    type: 'password',
+                                    'name': 'password',
+                                    placeholder: Msg.login_password,
+                                }),
+                            ]),
+                        ]),
                         h('div.checkbox-container', [
                             UI.createCheckbox('import-recent', Msg.register_importRecent),
                         ]),
                         h('div.extra', [
                             (Config.restrictRegistration?
                                 undefined:
-                                h('button#register.cp-secondary', Msg.login_register)
+                                h('a#register', {
+                                    href: "/register/",
+                                }, Msg.login_register)
                             ),
                             h('button.login', Msg.login_login)
                         ])
@@ -53,4 +63,3 @@ define([
         ])];
     };
 });
-

customize.dist/pages/recovery.js

@@ -9,7 +9,6 @@ define([
 
     return function () {
         document.title = Msg.recovery_header;
-
         var frame = function (content) {
             return [
                 h('div#cp-main', [
@@ -21,7 +20,6 @@ define([
                 ]),
             ];
         };
-
         return frame([
             h('div.row.cp-recovery-det', [
                 h('div.hidden.col-md-3'),
@@ -29,24 +27,32 @@ define([
                     h('div.cp-recovery-step.step1', [
                         h('p', Msg.recovery_mfa_description),
                         h('div.alert.alert-danger.wrong-cred.cp-hidden', Msg.login_noSuchUser),
-                        h('input.form-control#username', {
+                        h('div.big-container', [
-                            type: 'text',
+                            h('div.input-container', [
-                            autocomplete: 'off',
+                                h('label.cp-default-label', {for: 'username'}, Msg.login_username),
-                            autocorrect: 'off',
+                                h('input.form-control#username', {
-                            autocapitalize: 'off',
+                                    type: 'text',
-                            spellcheck: false,
+                                    autocomplete: 'off',
-                            placeholder: Msg.login_username,
+                                    autocorrect: 'off',
-                            autofocus: true,
+                                    autocapitalize: 'off',
-                        }),
+                                    spellcheck: false,
-                        h('input.form-control#password', {
+                                    placeholder: Msg.login_username,
-                            type: 'password',
+                                    autofocus: true,
-                            placeholder: Msg.login_password,
+                                }),
-                        }),
+                            ]),
+                            h('div.input-container', [
+                                h('label.cp-default-label', {for: 'password'}, Msg.login_password),
+                                h('input.form-control#password', {
+                                    type: 'password',
+                                    placeholder: Msg.login_password,
+                                }),
+                            ]),
+                        ]),
                         h('div.cp-recover-button',
                             h('button.btn.btn-primary#cp-recover-login', Msg.continue)
                         )
                     ]),
-                    h('div.cp-recovery-step.step2', { style: 'display: none;' }, [
+                    h('div.cp-recovery-step.step2', {style: 'display: none;'}, [
                         h('label', Msg.recovery_mfa_secret),
                         h('input.form-control#mfarecovery', {
                             type: 'text',
@@ -61,7 +67,7 @@ define([
                             h('i.fa.fa-caret-right'),
                             h('span', Msg.recovery_forgot)
                         ]),
-                        h('div.cp-recovery-alt', { style: 'display: none;' }, [
+                        h('div.cp-recovery-alt', {style: 'display: none;'}, [
                             UI.setHTML(h('div'),
                                 Msg._getKey('recovery_forgot_text', [Config.adminEmail || ''])),
                             h('textarea.cp-recover-email', {readonly: 'readonly'}),
@@ -71,8 +77,8 @@ define([
                             h('button.btn.btn-primary#cp-recover', Msg.mfa_disable)
                         )
                     ]),
-                    h('div.cp-recovery-step.step-info', { style: 'display: none;' }, [
+                    h('div.cp-recovery-step.step-info', {style: 'display: none;'}, [
-                        h('div.alert.alert-info.cp-hidden.disabled', Msg.recovery_mfa_disabled),
+                        h('div.cp-hidden.disabled', Msg.recovery_mfa_disabled),
                         h('div.alert.alert-danger.cp-hidden.unknown-error', Msg.recovery_mfa_error),
                     ]),
                 ]),

customize.dist/pages/register.js

@@ -8,7 +8,6 @@ define([
 ], function (Config, $, h, UI, Msg, Pages) {
     return function () {
         document.title = Msg.register_header;
-
         var tos = $(UI.createCheckbox('accept-terms')).find('.cp-checkmark-label').append(Msg.register_acceptTerms).parent()[0];
 
         var termsLink = Pages.customURLs.terms;
@@ -52,29 +51,40 @@ define([
                 h('div.cp-reg-form.col-md-6', [
                     h('div#userForm.form-group.hidden', [
                         h('div.cp-register-instance', [
-                            Msg._getKey('register_instance', [ Pages.Instance.name ]),
+                            Msg._getKey('register_instance', [Pages.Instance.name]),
                             h('br'),
                             h('a', {
                                 href: '/features.html'
                             }, Msg.register_whyRegister)
                         ]),
-                        h('input.form-control#username', {
+                        h('div.big-container', [
-                            type: 'text',
+                            h('div.input-container', [
-                            autocomplete: 'off',
+                                h('label.cp-register-label', { for: 'username' }, Msg.login_username),
-                            autocorrect: 'off',
+                                h('input.form-control#username', {
-                            autocapitalize: 'off',
+                                    type: 'text',
-                            spellcheck: false,
+                                    autocomplete: 'off',
-                            placeholder: Msg.login_username,
+                                    autocorrect: 'off',
-                            autofocus: true,
+                                    autocapitalize: 'off',
-                        }),
+                                    spellcheck: false,
-                        h('input.form-control#password', {
+                                    placeholder: Msg.login_username,
-                            type: 'password',
+                                    autofocus: true,
-                            placeholder: Msg.login_password,
+                                }),
-                        }),
+                            ]),
-                        h('input.form-control#password-confirm', {
+                            h('div.input-container', [
-                            type: 'password',
+                                h('label.cp-register-label', { for: 'password' }, Msg.login_password),
-                            placeholder: Msg.login_confirm,
+                                h('input.form-control#password', {
-                        }),
+                                    type: 'password',
+                                    placeholder: Msg.login_password,
+                                }),
+                            ]),
+                            h('div.input-container', [
+                                h('label.cp-register-label', { for: 'password-confirm' }, Msg.login_confirm),
+                                h('input.form-control#password-confirm', {
+                                    type: 'password',
+                                    placeholder: Msg.login_confirm,
+                                }),
+                            ]),
+                        ]),
                         h('div.checkbox-container', [
                             UI.createCheckbox('import-recent', Msg.register_importRecent, true)
                         ]),
@@ -85,6 +95,4 @@ define([
             ])
         ]);
     };
-
 });
-

customize.dist/pre-loading.js

@@ -8,7 +8,7 @@ var elem = document.createElement('div');
 elem.setAttribute('id', 'placeholder');
 elem.innerHTML = [
     '<div class="placeholder-logo-container">',
-        '<img class="placeholder-logo" src="' + logoPath + '">',
+        '<img class="placeholder-logo" alt="CryptPad Logo" src="' + logoPath + '">',
     '</div>',
     '<div class="placeholder-message-container">',
         '<p>Loading...</p>',

customize.dist/src/less2/include/alertify.less

@@ -144,7 +144,6 @@
             display: flex;
 
             > * {
-                overflow:auto;
                 width: 100%;
                 min-width: 260px;
                 max-width: 500px;
@@ -157,6 +156,10 @@
                 }
             }
 
+            .cp-alertify-scrollable {
+                overflow: auto;
+            }
+
             .msg {
                 padding: @alertify_padding-base;
                 margin-bottom: @alertify_padding-base;

customize.dist/src/less2/include/fileupload.less

@@ -13,7 +13,7 @@
         .modal_base();
         position: absolute;
         right: 10vw;
-        bottom: 10vh;
+        bottom: 100px;
         border-radius: @variables_radius;
         box-sizing: border-box;
         z-index: 100001; //Z file upload table container: just above the file picker
@@ -21,9 +21,9 @@
         color: @cp_upload-fg;
         max-height: 180px;
         overflow-y: auto;
-
+        overflow-x: hidden;
         @media screen and (max-width: @browser_media-medium-screen) {
-            left: 5vw; right: 5vw; bottom: 5vw;
+            left: 1vw; right: 1vw; bottom: 1vw;
         }
 
         .cp-fileupload-header {
@@ -48,16 +48,17 @@
         #cp-fileupload-table {
             width: 100%;
             @upload_pad_h: 0.25em;
-            @upload_pad_v: 0.5em;
+            @upload_pad_v: 0.25em;
 
             td {
                 padding: @upload_pad_h @upload_pad_v;
+                white-space: nowrap;
             }
             .cp-fileupload-table-link {
                 display: flex;
                 align-items: center;
                 white-space: nowrap;
-                max-width: 30vw;
+                max-width: 40vw;
                 margin: 0px @upload_pad_v;
                 .fa {
                     margin-top: 4px;
@@ -66,6 +67,12 @@
                 .cp-fileupload-table-name {
                     overflow: hidden;
                     text-overflow: ellipsis;
+                    min-width: 35vw;
+
+                @media screen and (max-width: 600px) {
+                    min-width: 40vw;
+                }
+
                 }
                 &[href]:hover {
                     text-decoration: none;
@@ -75,11 +82,11 @@
                 }
             }
             .cp-fileupload-table-progress {
-                min-width: 12em;
-                max-width: 16em;
                 position: relative;
                 text-align: center;
                 box-sizing: border-box;
+                min-width: 20vw;
+
             }
             .cp-fileupload-table-progress-container {
                 position: relative;
@@ -88,6 +95,7 @@
                 position: absolute;
                 width: 0px;
                 height: 100%;
+                padding: 2px;
                 background-color: @cp_upload-progress;
                 z-index: -1; //Z file upload progress container
                 border-radius: @variables_radius;

customize.dist/src/less2/include/infopages.less

@@ -123,11 +123,6 @@ body.html {
                 background-color: contrast(@cp_buttons-primary-text, darken(@cp_buttons-primary, 10%), lighten(@cp_buttons-primary, 10%));
             }
         }
-        button.cp-secondary {
-            background-color: @cp_buttons-cancel;
-            color: @cp_buttons-fg;
-            border: 1px solid @cp_buttons-fg;
-        }
 
         .form-group {
             // & > * {
@@ -147,6 +142,14 @@ body.html {
                     margin-right: 0.5em;
                 }
             }
+            .big-container {
+                width: 100%;
+            }
+            .input-container {
+                display: flex;
+                flex-direction: row;
+                align-items: center;
+            }
         }
     }
     footer {

customize.dist/src/less2/include/loading.less

@@ -103,6 +103,9 @@
         a {
             color: @cp_loading-link;
         }
+        p:last-child {
+            margin-bottom: 0;
+        }
     }
 
     #cp-loading-password-prompt {

customize.dist/src/less2/include/sidebar-layout.less

@@ -85,7 +85,7 @@
                 pre {
                     color: @cryptpad_text_col;
                 }
-                label:not(.noTitle), .label {
+                label:not(.noTitle), .cp-default-label {
                     display: block;
                     font-weight: bold;
                     margin-bottom: 0;
@@ -93,7 +93,7 @@
                 .cp-sidebarlayout-description {
                     display: block;
                     color: @cp_sidebar-hint;
-                    margin-bottom: 5px;
+                    margin-bottom: 0.5rem;
                     p {
                         margin-bottom: 0;
                     }

customize.dist/src/less2/pages/page-index.less

@@ -1,4 +1,5 @@
 @import (reference) "../include/infopages.less";
+@import (reference) "../include/browser.less";
 @import (reference) "../include/colortheme-all.less";
 
 &.cp-page-index {
@@ -31,17 +32,20 @@
         color: @cryptpad_text_col;
     }
 
-
     h1.cp-instance-title {
         text-align: center;
         font-family: "IBM Plex Mono";
         font-weight: 500;
         color: @cryptpad_color_brand;
-        font-size: 2.8rem;
+        font-size: 2.8rem; // Not used
-        overflow-wrap: break-word;
         word-wrap: break-word;
         word-break: break-word;
     }
+    @media screen and (max-width: @browser_media-medium-screen) {
+        &.html h1.cp-instance-title {
+            font-size: 2.5em;
+        }
+    }
 
     .cp-home-hero {
         width: 100%;
@@ -85,7 +89,9 @@
         .cp-instance-links {
             padding: 1.3rem;
             a {
-                margin: 1.3rem;
+                display: inline-block;
+                margin: 0.5em;
+                white-space: nowrap;
             }
         }
     }
@@ -118,7 +124,7 @@
         .cp-app-grid-apps {
             display: flex;
             flex-wrap: wrap;
-            justify-content: space-between;
+            justify-content: center;
             padding: 0 0.8rem;
         }
         a {

customize.dist/src/less2/pages/page-login.less

@@ -26,7 +26,7 @@
             margin: 0 10px 10px 10px;
             border-radius: @infopages-radius-L;
             .cp-login-instance {
-                margin-bottom: 10px;
+                margin-bottom: 5px;
             }
             .form-control {
                 border-radius: @infopages-radius;
@@ -52,6 +52,9 @@
                 margin-right: 0px;
             }
         }
+        .cp-default-label {
+            display: none;
+        }
     }
 
     .cp-password-form {

customize.dist/src/less2/pages/page-recovery.less

@@ -36,6 +36,9 @@
         .cp-hidden {
             display: none;
         }
+        .cp-default-label {
+            display: none;
+        }
     }
     .alertify {
         // workaround for alertify making empty p

customize.dist/src/less2/pages/page-register.less

@@ -26,6 +26,16 @@
                 min-width: 30%;
             }
         }
+        .input-container {
+            display: flex;
+            flex-direction: row;
+            align-items: center;
+            justify-content: center;
+            input {
+                margin-top: 5px;
+            }
+        }
+        .cp-register-label { display: none; }
         padding-bottom: 3em;
         min-height: 5vh;
     }
@@ -94,6 +104,7 @@
                 .tools_placeholder-color();
             }
             .checkbox-container {
+                margin-top: 0.5rem;
                 color: @cryptpad_text_col;
             }
             button#register {

customize.dist/translations/README.md

@@ -1,8 +1,8 @@
 # Translations
 
-Translations can now be made using [Weblate](https://weblate.cryptpad.fr). We may still accept PRs for the internal translation files, but we won't provide support for this. See the state of the translated languages:
+Translations can now be made using [Weblate](https://weblate.cryptpad.org). We may still accept PRs for the internal translation files, but we won't provide support for this. See the state of the translated languages:
 
-![](https://weblate.cryptpad.fr/widgets/cryptpad/-/app/multi-auto.svg)
+![](https://weblate.cryptpad.org/widgets/cryptpad/-/app/multi-auto.svg)
 
 ## Request a new language
 

docs/cryptpad.service

@@ -2,7 +2,7 @@
 Description=CryptPad API server
 
 [Service]
-ExecStart=/home/cryptpad/.nvm/versions/node/v19.8.1/bin/node /home/cryptpad/cryptpad/server.js
+ExecStart=/usr/bin/node /home/cryptpad/cryptpad/server.js
 # modify to match the location of your cryptpad repository
 WorkingDirectory=/home/cryptpad/cryptpad
 

docs/example-advanced.nginx.conf

@@ -164,7 +164,7 @@ server {
 
     # draw.io uses inline script tags in it's index.html. The hashes are added here.
     if ($uri ~ ^\/components\/drawio\/src\/main\/webapp\/index.html.*$) {
-        set $scriptSrc "'self' 'sha256-6zAB96lsBZREqf0sT44BhH1T69sm7HrN34rpMOcWbNo=' 'sha256-6g514VrT/cZFZltSaKxIVNFF46+MFaTSDTPB8WfYK+c=' resource: https://${main_domain}";
+        set $scriptSrc "'self' 'sha256-dLMFD7ijAw6AVaqecS7kbPcFFzkxQ+yeZSsKpOdLxps=' 'sha256-6g514VrT/cZFZltSaKxIVNFF46+MFaTSDTPB8WfYK+c=' resource: https://${main_domain}";
     }
 
     # privileged contexts allow a few more rights than unprivileged contexts, though limits are still applied

lib/archive-account.js

@@ -0,0 +1,326 @@
+/* jshint esversion: 6, node: true */
+const nThen = require('nthen');
+const Pins = require('./pins');
+const Util = require("./common-util");
+const Store = require('./storage/file.js');
+const BlobStore = require("./storage/blob");
+const BlockStore = require("./storage/block");
+const Core = require("./commands/core");
+const Metadata = require("./commands/metadata");
+const Meta = require("./metadata");
+const Logger = require("./log");
+
+const Path = require("path");
+const Fse = require("fs-extra");
+
+const { parentPort } = require('node:worker_threads');
+
+const COMMANDS = {};
+let Log;
+
+const mkReportPath = function (Env, safeKey) {
+    return Path.join(Env.paths.archive, 'accounts', safeKey);
+};
+const storeReport = (Env, report, cb) => {
+    let path = mkReportPath(Env, report.key);
+    let s_data;
+    try {
+        s_data = JSON.stringify(report);
+        Fse.outputFile(path, s_data, cb);
+    } catch (err) {
+        return void cb(err);
+    }
+};
+const readReport = (Env, key, cb) => {
+    let path = mkReportPath(Env, key);
+    Fse.readJson(path, cb);
+};
+const deleteReport = (Env, key, cb) => {
+    let path = mkReportPath(Env, key);
+    Fse.remove(path, cb);
+};
+
+const init = (cb) => {
+    const Environment = require("./env");
+    const config = require('./load-config');
+    const Env = Environment.create(config);
+    Env.computeMetadata = function (channel, cb) {
+        const ref = {};
+        const lineHandler = Meta.createLineHandler(ref, (err) => { console.log(err); });
+        return void Env.store.readChannelMetadata(channel, lineHandler, function (err) {
+            if (err) {
+                // stream errors?
+                return void cb(err);
+            }
+            cb(void 0, ref.meta);
+        });
+    };
+
+    nThen((waitFor) => {
+        Logger.create(config, waitFor(function (_) {
+            Log = Env.Log = _;
+        }));
+        Store.create(config, waitFor(function (err, _store) {
+            if (err) {
+                waitFor.abort();
+                return void cb(err);
+            }
+            Env.store = _store;
+        }));
+        Store.create({
+            filePath: config.pinPath,
+            archivePath: config.archivePath,
+            // archive pin logs to their own subpath
+            volumeId: 'pins',
+        }, waitFor(function (err, _) {
+            if (err) {
+                waitFor.abort();
+                throw err;
+            }
+            Env.pinStore = _;
+        }));
+        BlobStore.create({
+            blobPath: config.blobPath,
+            blobStagingPath: config.blobStagingPath,
+            archivePath: config.archivePath,
+            getSession: function () {},
+        }, waitFor(function (err, blob) {
+            if (err) {
+                waitFor.abort();
+                return void cb(err);
+            }
+            Env.blobStore = blob;
+        }));
+    }).nThen(() => {
+        cb(Env);
+    });
+};
+
+COMMANDS.start = (edPublic, blockId, reason) => {
+    const safeKey = Util.escapeKeyCharacters(edPublic);
+    const archiveReason = {
+        code: 'MODERATION_ACCOUNT',
+        txt: reason
+    };
+
+    let ref = {};
+    let blobsToArchive = [];
+    let channelsToArchive = [];
+    let deletedChannels = [];
+    let deletedBlobs = [];
+    let Env;
+    nThen((waitFor) => {
+        init(waitFor((_Env) => {
+            Env = _Env;
+        }));
+    }).nThen((waitFor) => {
+        let lineHandler = Pins.createLineHandler(ref, (err) => { console.log(err); });
+        Env.pinStore.readMessagesBin(safeKey, 0, (msgObj, readMore) => {
+            lineHandler(msgObj.buff.toString('utf8'));
+            readMore();
+        }, waitFor());
+    }).nThen((waitFor) => {
+        Log.info('MODERATION_ACCOUNT_ARCHIVAL_START', edPublic, waitFor());
+        var n = nThen;
+        Object.keys(ref.pins || {}).forEach((chanId) => {
+            n = n((w) => {
+                // Blobs
+                if (Env.blobStore.isFileId(chanId)) {
+                    return void Env.blobStore.isOwnedBy(safeKey, chanId, w((err, owned) => {
+                        if (err || !owned) { return; }
+                        blobsToArchive.push(chanId);
+                    }));
+                }
+                // Pads
+                Metadata.getMetadata(Env, chanId, w((err, metadata) => {
+                    if (err) { return; } // Can't read metadata? Don't archive
+                    if (!Core.hasOwners(metadata)) { return; } // No owner, don't archive
+                    if (Core.isOwner(metadata, edPublic) && metadata.owners.length === 1) {
+                        channelsToArchive.push(chanId); // Only owner: archive
+                    }
+                }));
+            }).nThen;
+        });
+        n(waitFor());
+    }).nThen((waitFor) => {
+        Log.info('MODERATION_ACCOUNT_ARCHIVAL_LISTED', JSON.stringify({
+            pads: channelsToArchive.length,
+            blobs: blobsToArchive.length
+        }), waitFor());
+
+        var n = nThen;
+        // Archive the pads
+        channelsToArchive.forEach((chanId) => {
+            n = n((w) => {
+                Env.store.archiveChannel(chanId, archiveReason, w(function (err) {
+                    if (err) {
+                        return Log.error('MODERATION_CHANNEL_ARCHIVAL_ERROR', {
+                            error: err,
+                            channel: chanId,
+                        }, w());
+                    }
+                    deletedChannels.push(chanId);
+                    Log.info('MODERATION_CHANNEL_ARCHIVAL', chanId, w());
+                }));
+            }).nThen;
+        });
+        // Archive the blobs
+        blobsToArchive.forEach((blobId) => {
+            n = n((w) => {
+                Env.blobStore.archive.blob(blobId, archiveReason, w(function (err) {
+                    if (err) {
+                        return Log.error('MODERATION_BLOB_ARCHIVAL_ERROR', {
+                            error: err,
+                            item: blobId,
+                        }, w());
+                    }
+                    deletedBlobs.push(blobId);
+                    Log.info('MODERATION_BLOB_ARCHIVAL', blobId, w());
+                }));
+            }).nThen;
+        });
+        n(waitFor(() => {
+            // Archive the pin log
+            Env.pinStore.archiveChannel(safeKey, undefined, waitFor(function (err) {
+                if (err) {
+                    return Log.error('MODERATION_ACCOUNT_PIN_LOG', err, waitFor());
+                }
+                Log.info('MODERATION_ACCOUNT_LOG', safeKey, waitFor());
+            }));
+            blockId = blockId || ref.block;
+            if (!blockId) { return; }
+            BlockStore.archive(Env, blockId, archiveReason, waitFor(function (err) {
+                if (err) {
+                    blockId = undefined;
+                    return Log.error('MODERATION_ACCOUNT_BLOCK', err, waitFor());
+                }
+                Log.info('MODERATION_ACCOUNT_BLOCK', safeKey, waitFor());
+            }));
+        }));
+    }).nThen((waitFor) => {
+        var report = {
+            key: safeKey,
+            channels: deletedChannels,
+            blobs: deletedBlobs,
+            blockId: blockId,
+            reason: reason
+        };
+        storeReport(Env, report, waitFor((err) => {
+            if (err) {
+                return Log.error('MODERATION_ACCOUNT_REPORT', report, waitFor());
+            }
+        }));
+    }).nThen(() => {
+        parentPort.postMessage(JSON.stringify(deletedChannels));
+        process.exit(0);
+    });
+};
+
+
+COMMANDS.restore = (edPublic) => {
+    const safeKey = Util.escapeKeyCharacters(edPublic);
+    let pads, blobs;
+    let blockId;
+    let errors = [];
+    let Env;
+    nThen((waitFor) => {
+        init(waitFor((_Env) => {
+            Env = _Env;
+        }));
+    }).nThen((waitFor) => {
+        Log.info('MODERATION_ACCOUNT_RESTORE_START', edPublic, waitFor());
+        readReport(Env, safeKey, waitFor((err, report) => {
+            if (err) { throw new Error(err); }
+            pads = report.channels;
+            blobs = report.blobs;
+            blockId = report.blockId;
+        }));
+    }).nThen((waitFor) => {
+        Log.info('MODERATION_ACCOUNT_RESTORE_LISTED', JSON.stringify({
+            pads: pads.length,
+            blobs: blobs.length
+        }), waitFor());
+        var n = nThen;
+        pads.forEach((chanId) => {
+            n = n((w) => {
+                Env.store.restoreArchivedChannel(chanId, w(function (err) {
+                    if (err) {
+                        errors.push(chanId);
+                        return Log.error('MODERATION_CHANNEL_RESTORE_ERROR', {
+                            error: err,
+                            channel: chanId,
+                        }, w());
+                    }
+                    Log.info('MODERATION_CHANNEL_RESTORE', chanId, w());
+                }));
+            }).nThen;
+        });
+        blobs.forEach((blobId) => {
+            n = n((w) => {
+                Env.blobStore.restore.blob(blobId, w(function (err) {
+                    if (err) {
+                        errors.push(blobId);
+                        return Log.error('MODERATION_BLOB_RESTORE_ERROR', {
+                            error: err,
+                            item: blobId,
+                        }, w());
+                    }
+                    Log.info('MODERATION_BLOB_RESTORE', blobId, w());
+                }));
+            }).nThen;
+        });
+        n(waitFor(() => {
+            // remove the pin logs of inactive accounts if inactive account removal is configured
+            Env.pinStore.restoreArchivedChannel(safeKey, waitFor(function (err) {
+                if (err) {
+                    return Log.error('MODERATION_ACCOUNT_PIN_LOG_RESTORE', err, waitFor());
+                }
+                Log.info('MODERATION_ACCOUNT_LOG_RESTORE', safeKey, waitFor());
+            }));
+            if (!blockId) { return; }
+            BlockStore.restore(Env, blockId, waitFor(function (err) {
+                if (err) {
+                    blockId = undefined;
+                    return Log.error('MODERATION_ACCOUNT_BLOCK_RESTORE', err, waitFor());
+                }
+                Log.info('MODERATION_ACCOUNT_BLOCK_RESTORE', safeKey, waitFor());
+            }));
+        }));
+    }).nThen((waitFor) => {
+        deleteReport(Env, safeKey, waitFor((err) => {
+            if (err) {
+                return Log.error('MODERATION_ACCOUNT_REPORT_DELETE', safeKey, waitFor());
+            }
+        }));
+    }).nThen(() => {
+        parentPort.postMessage(JSON.stringify(errors));
+        process.exit(0);
+    });
+
+};
+
+const getStatus = (Env, edPublic, cb) => {
+    const safeKey = Util.escapeKeyCharacters(edPublic);
+    readReport(Env, safeKey, (err, report) => {
+        if (err) { return void cb(err); }
+        cb(void 0, report);
+    });
+};
+
+if (parentPort) {
+    parentPort.on('message', (message) => {
+        let parsed = message; //JSON.parse(message);
+        let command = parsed.command;
+        let content = parsed.content;
+        let block = parsed.block;
+        let reason = parsed.reason;
+        COMMANDS[command](content, block, reason);
+    });
+
+    parentPort.postMessage('READY');
+} else {
+
+    module.exports = {
+        getStatus: getStatus
+    };
+}

lib/challenge-commands/base.js

@@ -69,8 +69,8 @@ const removeBlock = Commands.REMOVE_BLOCK = function (Env, body, cb) {
 };
 
 removeBlock.complete = function (Env, body, cb) {
-    const { publicKey } = body;
+    const { publicKey, reason } = body;
-    Block.removeLoginBlock(Env, publicKey, cb);
+    Block.removeLoginBlock(Env, publicKey, reason, cb);
 };
 
 

lib/challenge-commands/totp.js

@@ -483,10 +483,10 @@ const removeBlock = Commands.TOTP_REMOVE_BLOCK = function (Env, body, cb) {
 };
 
 removeBlock.complete = function (Env, body, cb) {
-    const { publicKey } = body;
+    const { publicKey, reason } = body;
     nThen(function (w) {
         // Remove the block
-        Block.removeLoginBlock(Env, publicKey, w((err) => {
+        Block.removeLoginBlock(Env, publicKey, reason, w((err) => {
             if (err) {
                 w.abort();
                 return void cb(err);

lib/commands/admin-rpc.js

@@ -10,6 +10,10 @@ const Core = require("./core");
 const Channel = require("./channel");
 const BlockStore = require("../storage/block");
 const MFA = require("../storage/mfa");
+const ArchiveAccount = require('../archive-account');
+/* jshint ignore:start */
+const { Worker } = require('node:worker_threads');
+/* jshint ignore:end */
 
 var Fs = require("fs");
 
@@ -172,20 +176,26 @@ var archiveDocument = function (Env, Server, cb, data) {
 
     if (typeof(id) !== 'string' || id.length < 32) { return void cb("EINVAL"); }
 
+    const archiveReason = {
+        code: 'MODERATION_PAD',
+        txt: reason
+    };
+    const reasonStr = `MODERATION_PAD:${reason}`;
+
     switch (id.length) {
         case 32:
-            return void Env.msgStore.archiveChannel(id, Util.both(cb, function (err) {
+            return void Env.msgStore.archiveChannel(id, archiveReason, Util.both(cb, function (err) {
                 Env.Log.info("ARCHIVAL_CHANNEL_BY_ADMIN_RPC", {
                     channelId: id,
                     reason: reason,
                     status: err? String(err): "SUCCESS",
                 });
-                Channel.disconnectChannelMembers(Env, Server, id, 'EDELETED', err => {
+                Channel.disconnectChannelMembers(Env, Server, id, 'EDELETED', reasonStr, err => {
                     if (err) { } // TODO
                 });
             }));
         case 48:
-            return void Env.blobStore.archive.blob(id, Util.both(cb, function (err) {
+            return void Env.blobStore.archive.blob(id, archiveReason, Util.both(cb, function (err) {
                 Env.Log.info("ARCHIVAL_BLOB_BY_ADMIN_RPC", {
                     id: id,
                     reason: reason,
@@ -210,7 +220,7 @@ var removeDocument = function (Env, Server, cb, data) {
         id = args;
     } else if (args && typeof(args) === 'object') {
         id = args.id;
-        reason = args.reason;
+        reason = `MODERATION_DESTROY:${args.reason}`;
     }
 
     if (typeof(id) !== 'string' || id.length < 32) { return void cb("EINVAL"); }
@@ -223,7 +233,7 @@ var removeDocument = function (Env, Server, cb, data) {
                     reason: reason,
                     status: err? String(err): "SUCCESS",
                 });
-                Channel.disconnectChannelMembers(Env, Server, id, 'EDELETED', err => {
+                Channel.disconnectChannelMembers(Env, Server, id, 'EDELETED', reason, err => {
                     if (err) { } // TODO
                 });
             }));
@@ -279,6 +289,81 @@ var restoreArchivedDocument = function (Env, Server, cb, data) {
     }
 };
 
+// CryptPad_AsyncStore.rpc.send('ADMIN', ['ARCHIVE_ACCOUNT', {key, block, reason}], console.log)
+var archiveAccount = function (Env, Server, _cb, data) {
+    const cb = Util.once(_cb);
+    const worker = new Worker('./lib/archive-account.js');
+    const args = Array.isArray(data) && data[1];
+    if (!args || typeof(args) !== 'object') { return void cb("EINVAL"); }
+    worker.on('message', message => {
+        if (message === 'READY') {
+            return worker.postMessage({
+                command: 'start',
+                content: args.key,
+                block: args.block, // optional, may be including in pin log
+                reason: args.reason
+            });
+        }
+
+        // DONE: disconnect all users from these channels
+        Env.Log.info('ARCHIVE_ACCOUNT_BY_ADMIN', {
+            safeKey: args.key,
+            reason: args.reason,
+        });
+        const reason = `MODERATION_ACCOUNT:${args.reason}`;
+        var deletedChannels = Util.tryParse(message);
+        if (Array.isArray(deletedChannels)) {
+            let n = nThen;
+            deletedChannels.forEach((chanId) => {
+                n = n((w) => {
+                    setTimeout(w(() => {
+                        Channel.disconnectChannelMembers(Env, Server, chanId, 'EDELETED', reason, () => {});
+                    }), 10);
+                }).nThen;
+            });
+        }
+        cb(void 0, { state: true });
+    });
+    worker.on('error', (err) => {
+        console.error(err);
+        cb(err);
+    });
+    worker.on('exit', () => { worker.unref(); });
+};
+var restoreAccount = function (Env, Server, _cb, data) {
+    const cb = Util.once(_cb);
+    const worker = new Worker('./lib/archive-account.js');
+    const args = Array.isArray(data) && data[1];
+    if (!args || typeof(args) !== 'object') { return void cb("EINVAL"); }
+    worker.on('message', message => {
+        if (message === 'READY') {
+            return worker.postMessage({
+                command: 'restore',
+                content: args.key
+            });
+        }
+        // Response
+        Env.Log.info('RESTORE_ACCOUNT_BY_ADMIN', {
+            safeKey: args.key,
+            reason: args.reason,
+        });
+        cb(void 0, {
+            state: true,
+            errors: Util.tryParse(message)
+        });
+    });
+    worker.on('error', (err) => {
+        console.error(err);
+        cb(err);
+    });
+    worker.on('exit', () => { worker.unref(); });
+};
+var getAccountArchiveStatus = function (Env, Server, cb, data) {
+    const args = Array.isArray(data) && data[1];
+    if (!args || typeof(args) !== 'object') { return void cb("EINVAL"); }
+    ArchiveAccount.getStatus(Env, args.key, cb);
+};
+
 // CryptPad_AsyncStore.rpc.send('ADMIN', ['CLEAR_CACHED_CHANNEL_INDEX', documentID], console.log)
 var clearChannelIndex = function (Env, Server, cb, data) {
     var id = Array.isArray(data) && data[1];
@@ -499,6 +584,10 @@ var getDocumentStatus = function (Env, Server, cb, data) {
                 }
                 response.archived = result;
             }));
+            BlockStore.readPlaceholder(Env, id, w((result) => {
+                if (!result) { return; }
+                response.placeholder = result;
+            }));
             MFA.read(Env, id, w(function (err, v) {
                 if (err === 'ENOENT') {
                     response.totp = 'DISABLED';
@@ -530,6 +619,10 @@ var getDocumentStatus = function (Env, Server, cb, data) {
                 }
                 response.archived = result;
             }));
+            Env.blobStore.getPlaceholder(id, w((result) => {
+                if (!result) { return; }
+                response.placeholder = result;
+            }));
         }).nThen(function () {
             cb(void 0, response);
         });
@@ -548,6 +641,10 @@ var getDocumentStatus = function (Env, Server, cb, data) {
             }
             response.archived = result;
         }));
+        Env.store.getPlaceholder(id, w((result) => {
+            if (!result) { return; }
+            response.placeholder = result;
+        }));
     }).nThen(function () {
         cb(void 0, response);
     });
@@ -578,6 +675,8 @@ var getPinHistory = function (Env, Server, cb, data) {
     cb("NOT_IMPLEMENTED");
 };
 
+/*
+// NOTE: Deprecated, archive whole account now
 var archivePinLog = function (Env, Server, cb, data) {
     var args = Array.isArray(data) && data[1];
     if (!args || typeof(args) !== 'object') { return void cb("EINVAL"); }
@@ -586,7 +685,7 @@ var archivePinLog = function (Env, Server, cb, data) {
     if (!isValidKey(key)) { return void cb("EINVAL"); }
     var safeKey = Util.escapeKeyCharacters(key);
 
-    Env.pinStore.archiveChannel(safeKey, function (err) {
+    Env.pinStore.archiveChannel(safeKey, undefined, function (err) {
         Core.expireSession(Env.Sessions, safeKey);
         if (err) {
             Env.Log.error('ARCHIVE_PIN_LOG_BY_ADMIN', {
@@ -603,6 +702,7 @@ var archivePinLog = function (Env, Server, cb, data) {
         cb(err);
     });
 };
+*/
 
 var archiveBlock = function (Env, Server, cb, data) {
     var args = Array.isArray(data) && data[1];
@@ -610,7 +710,11 @@ var archiveBlock = function (Env, Server, cb, data) {
     var key = args.key;
     var reason = args.reason;
     if (!isValidKey(key)) { return void cb("EINVAL"); }
-    BlockStore.archive(Env, key, err => {
+    const archiveReason = {
+        code: 'MODERATION_BLOCK',
+        txt: reason
+    };
+    BlockStore.archive(Env, key, archiveReason, err => {
         Env.Log.info("ARCHIVE_BLOCK_BY_ADMIN", {
             error: err,
             key: key,
@@ -636,6 +740,8 @@ var restoreArchivedBlock = function (Env, Server, cb, data) {
     });
 };
 
+/*
+// NOTE: Deprecated, archive whole account now
 var restoreArchivedPinLog = function (Env, Server, cb, data) {
     var args = Array.isArray(data) && data[1];
     if (!args || typeof(args) !== 'object') { return void cb("EINVAL"); }
@@ -660,6 +766,7 @@ var restoreArchivedPinLog = function (Env, Server, cb, data) {
         cb(err);
     });
 };
+*/
 
 var archiveOwnedDocuments = function (Env, Server, cb, data) {
     Env.Log.debug('ARCHIVE_OWNED_DOCUMENTS', data);
@@ -770,9 +877,9 @@ var commands = {
 
     GET_PIN_LIST: getPinList,
     GET_PIN_HISTORY: getPinHistory,
-    ARCHIVE_PIN_LOG: archivePinLog,
+    //ARCHIVE_PIN_LOG: archivePinLog,
     ARCHIVE_OWNED_DOCUMENTS: archiveOwnedDocuments,
-    RESTORE_ARCHIVED_PIN_LOG: restoreArchivedPinLog,
+    //RESTORE_ARCHIVED_PIN_LOG: restoreArchivedPinLog,
 
     ARCHIVE_BLOCK: archiveBlock,
     RESTORE_ARCHIVED_BLOCK: restoreArchivedBlock,
@@ -780,6 +887,10 @@ var commands = {
     ARCHIVE_DOCUMENT: archiveDocument,
     RESTORE_ARCHIVED_DOCUMENT: restoreArchivedDocument,
 
+    ARCHIVE_ACCOUNT: archiveAccount,
+    RESTORE_ACCOUNT: restoreAccount,
+    GET_ACCOUNT_ARCHIVE_STATUS: getAccountArchiveStatus,
+
     CLEAR_CACHED_CHANNEL_INDEX: clearChannelIndex,
     GET_CACHED_CHANNEL_INDEX: getChannelIndex,
     // TODO implement admin historyTrim

lib/commands/block.js

@@ -172,10 +172,10 @@ Block.writeLoginBlock = function (Env, msg, _cb) {
     information, we can just sign some constant and use that as proof.
 
 */
-Block.removeLoginBlock = function (Env, publicKey, _cb) {
+Block.removeLoginBlock = function (Env, publicKey, reason, _cb) {
     var cb = Util.once(Util.mkAsync(_cb));
 
-    BlockStore.archive(Env, publicKey, function (err) {
+    BlockStore.archive(Env, publicKey, reason, function (err) {
         Env.Log.info('ARCHIVAL_BLOCK_BY_OWNER_RPC', {
             publicKey: publicKey,
             status: err? String(err): 'SUCCESS',

lib/commands/channel.js

@@ -8,7 +8,7 @@ const Metadata = require("./metadata");
 const HK = require("../hk-util");
 const Nacl = require("tweetnacl/nacl-fast");
 
-Channel.disconnectChannelMembers = function (Env, Server, channelId, code, cb) {
+Channel.disconnectChannelMembers = function (Env, Server, channelId, code, reason, cb) {
     var done = Util.once(Util.mkAsync(cb));
     if (!Core.isValidId(channelId)) { return done('INVALID_ID'); }
 
@@ -39,6 +39,7 @@ Channel.disconnectChannelMembers = function (Env, Server, channelId, code, cb) {
                 userId,
                 JSON.stringify({
                     error: code, //'EDELETED',
+                    message: reason,
                     channel: channelId,
                 })
             ], w());
@@ -52,6 +53,7 @@ Channel.disconnectChannelMembers = function (Env, Server, channelId, code, cb) {
         Env.Log.warn('DISCONNECT_CHANNEL_MEMBERS_TIMEOUT', {
             channelId,
             code,
+            reason
         });
         clear();
         done();
@@ -105,9 +107,10 @@ Channel.clearOwnedChannel = function (Env, safeKey, channelId, cb, Server) {
     });
 };
 
-var archiveOwnedChannel = function (Env, safeKey, channelId, __cb, Server) {
+var archiveOwnedChannel = function (Env, safeKey, channelId, reason, __cb, Server) {
     var _cb = Util.once(Util.mkAsync(__cb));
     var unsafeKey = Util.unescapeKeyCharacters(safeKey);
+    reason = reason || 'ARCHIVE_OWNED';
     nThen(function (w) {
         // confirm that the channel exists before worrying about whether
         // we have permission to delete it.
@@ -130,7 +133,7 @@ var archiveOwnedChannel = function (Env, safeKey, channelId, __cb, Server) {
     }).nThen(function () {
         var cb = _cb;
         // temporarily archive the file
-        return void Env.msgStore.archiveChannel(channelId, function (e) {
+        return void Env.msgStore.archiveChannel(channelId, reason, function (e) {
             Env.Log.info('ARCHIVAL_CHANNEL_BY_OWNER_RPC', {
                 unsafeKey: unsafeKey,
                 channelId: channelId,
@@ -141,16 +144,19 @@ var archiveOwnedChannel = function (Env, safeKey, channelId, __cb, Server) {
             }
             cb(void 0, 'OK');
 
-            Channel.disconnectChannelMembers(Env, Server, channelId, 'EDELETED', err => {
+            Channel.disconnectChannelMembers(Env, Server, channelId, 'EDELETED', reason, err => {
                 if (err) { } // TODO
             });
         });
     });
 };
 
-Channel.removeOwnedChannel = function (Env, safeKey, channelId, __cb, Server) {
+Channel.removeOwnedChannel = function (Env, safeKey, obj, __cb, Server) {
     var _cb = Util.once(Util.mkAsync(__cb));
 
+    var channelId = obj.channel;
+    var reason = obj.reason;
+
     if (typeof(channelId) !== 'string' || !Core.isValidId(channelId)) {
         return _cb('INVALID_ARGUMENTS');
     }
@@ -160,9 +166,9 @@ Channel.removeOwnedChannel = function (Env, safeKey, channelId, __cb, Server) {
     Env.queueDeletes(safeKey, function (next) {
         var cb = Util.both(_cb, next);
         if (Env.blobStore.isFileId(channelId)) {
-            return void Env.removeOwnedBlob(channelId, safeKey, cb);
+            return void Env.removeOwnedBlob(channelId, safeKey, reason, cb);
         }
-        archiveOwnedChannel(Env, safeKey, channelId, cb, Server);
+        archiveOwnedChannel(Env, safeKey, channelId, reason, cb, Server);
     });
 };
 
@@ -248,27 +254,29 @@ var ARRAY_LINE = /^\[/;
     call back with true if the channel log has no content other than metadata
     otherwise false
 */
-Channel.isNewChannel = function (Env, channel, cb) {
+Channel.isNewChannel = function (Env, channel, _cb) {
+    var cb = Util.once(_cb);
     if (!Core.isValidId(channel)) { return void cb('INVALID_CHAN'); }
     if (channel.length !== HK.STANDARD_CHANNEL_LENGTH &&
         channel.length !== HK.ADMIN_CHANNEL_LENGTH) { return void cb('INVALID_CHAN'); }
 
-    // TODO replace with readMessagesBin
+    Env.msgStore.readMessagesBin(channel, 0, function (msgObj, readMore, abort) {
-    var done = false;
-    Env.msgStore.getMessages(channel, function (msg) {
-        if (done) { return; }
         try {
+            var msg = msgObj.buff.toString('utf8');
             if (typeof(msg) === 'string' && ARRAY_LINE.test(msg)) {
-                done = true;
+                abort();
-                return void cb(void 0, false);
+                return void cb(void 0, {isNew: false});
             }
         } catch (e) {
             Env.WARN('invalid message read from store', e);
         }
-    }, function () {
+        readMore();
-        if (done) { return; }
+    }, function (err, reason) {
         // no more messages...
-        cb(void 0, true);
+        cb(void 0, {
+            isNew: true,
+            reason: reason
+        });
     });
 };
 

lib/commands/pin-rpc.js

@@ -115,7 +115,7 @@ Pinning.getTotalSize = function (Env, safeKey, cb) {
 */
 Pinning.removePins = function (Env, safeKey, cb) {
     // FIXME respect the queue
-    Env.pinStore.archiveChannel(safeKey, function (err) {
+    Env.pinStore.archiveChannel(safeKey, undefined, function (err) {
         Core.expireSession(Env.Sessions, safeKey);
         Env.Log.info('ARCHIVAL_PIN_BY_OWNER_RPC', {
             safeKey: safeKey,
@@ -184,6 +184,7 @@ Pinning.pinChannel = function (Env, safeKey, channels, cb) {
                 Env.pinStore.message(safeKey, JSON.stringify(['PIN', toStore, +new Date()]),
                     function (e) {
                     if (e) { return void cb(e); }
+                    if (!session || !session.channels) { return; }
                     toStore.forEach(function (channel) {
                         session.channels[channel] = true;
                     });

lib/defaults.js

@@ -49,7 +49,7 @@ Default.padContentSecurity = function (Env) {
 };
 
 Default.diagramContentSecurity = function (Env) {
-    return (Default.commonCSP(Env).join('; ') + "script-src 'self' 'sha256-6zAB96lsBZREqf0sT44BhH1T69sm7HrN34rpMOcWbNo=' 'sha256-6g514VrT/cZFZltSaKxIVNFF46+MFaTSDTPB8WfYK+c=' resource: " + Env.httpUnsafeOrigin).replace(/\s+/g, ' ');
+    return (Default.commonCSP(Env).join('; ') + "script-src 'self' 'sha256-dLMFD7ijAw6AVaqecS7kbPcFFzkxQ+yeZSsKpOdLxps=' 'sha256-6g514VrT/cZFZltSaKxIVNFF46+MFaTSDTPB8WfYK+c=' resource: " + Env.httpUnsafeOrigin).replace(/\s+/g, ' ');
 };
 
 Default.httpHeaders = function (Env) {

lib/eviction.js

@@ -186,8 +186,7 @@ var evictArchived = function (Env, cb) {
 
         var handler = function (err, item, cb) {
             if (err) {
-                Log.error('EVICT_ARCHIVED_CHANNEL_ITERATION', err);
+                return Log.error('EVICT_ARCHIVED_CHANNEL_ITERATION', err, cb);
-                return void cb();
             }
             // don't mess with files that are freshly stored in cold storage
             // based on ctime because that's changed when the file is moved...
@@ -199,13 +198,11 @@ var evictArchived = function (Env, cb) {
             // expire it
             store.removeArchivedChannel(item.channel, w(function (err) {
                 if (err) {
-                    Log.error('EVICT_ARCHIVED_CHANNEL_REMOVAL_ERROR', {
+                    return Log.error('EVICT_ARCHIVED_CHANNEL_REMOVAL_ERROR', {
                         error: err,
                         channel: item.channel,
-                    });
+                    }, cb);
-                    return void cb();
                 }
-                Log.info('EVICT_ARCHIVED_CHANNEL_REMOVAL', item.channel);
 
                 if (item.channel.length === 32) {
                     removed++;
@@ -213,7 +210,7 @@ var evictArchived = function (Env, cb) {
                     accounts++;
                 }
 
-                cb();
+                Log.info('EVICT_ARCHIVED_CHANNEL_REMOVAL', item.channel, cb);
             }));
         };
 
@@ -399,8 +396,7 @@ module.exports = function (Env, cb) {
             }
 
             if (err) {
-                Log.error('EVICT_CHANNEL_CATEGORIZATION', err);
+                return Log.error('EVICT_CHANNEL_CATEGORIZATION', err, cb);
-                return void cb();
             }
 
             // if the channel has been modified recently
@@ -421,7 +417,7 @@ module.exports = function (Env, cb) {
             Log.info('EVICT_CHANNELS_CATEGORIZED', {
                 active: active,
                 channels: channels,
-            });
+            }, w());
         };
 
         Log.info('EVICT_CHANNEL_ACTIVITY_START', 'Assessing channel activity');
@@ -443,12 +439,10 @@ module.exports = function (Env, cb) {
             }
 
             if (err) {
-                Log.error("EVICT_BLOB_CATEGORIZATION", err);
+                return Log.error("EVICT_BLOB_CATEGORIZATION", err, next);
-                return void next();
             }
             if (!item) {
-                next();
+                return void Log.error("EVICT_BLOB_CATEGORIZATION_INVALID", item, next);
-                return void Log.error("EVICT_BLOB_CATEGORIZATION_INVALID", item);
             }
             if (item.mtime > inactiveTime) {
                 activeDocs.add(item.blobId);
@@ -462,7 +456,7 @@ module.exports = function (Env, cb) {
             Log.info('EVICT_BLOBS_CATEGORIZED', {
                 active: active,
                 blobs: n_blobs,
-            });
+            }, w());
         }));
     };
 
@@ -530,31 +524,27 @@ module.exports = function (Env, cb) {
             // we plan to delete them, because it may be interesting information
             inactive++;
             if (PRESERVE_INACTIVE_ACCOUNTS) {
-                Log.info('EVICT_INACTIVE_ACCOUNT_PRESERVED', {
+                pinAll(pinList);
+                return Log.info('EVICT_INACTIVE_ACCOUNT_PRESERVED', {
                     id: id,
                     mtime: mtime,
-                });
+                }, next);
-                pinAll(pinList);
-                return void next();
             }
 
             if (isPremiumAccount(id)) {
-                Log.info("EVICT_INACTIVE_PREMIUM_ACCOUNT", {
+                pinAll(pinList);
+                return Log.info("EVICT_INACTIVE_PREMIUM_ACCOUNT", {
                     id: id,
                     mtime: mtime,
-                });
+                }, next);
-                pinAll(pinList);
-                return void next();
             }
 
             // remove the pin logs of inactive accounts if inactive account removal is configured
-            pinStore.archiveChannel(id, function (err) {
+            pinStore.archiveChannel(id, undefined, function (err) {
                 if (err) {
-                    Log.error('EVICT_INACTIVE_ACCOUNT_PIN_LOG', err);
+                    return Log.error('EVICT_INACTIVE_ACCOUNT_PIN_LOG', err, next);
-                    return void next();
                 }
-                Log.info('EVICT_INACTIVE_ACCOUNT_LOG', id);
+                Log.info('EVICT_INACTIVE_ACCOUNT_LOG', id, next);
-                next();
             });
         };
 
@@ -588,12 +578,10 @@ module.exports = function (Env, cb) {
         blobs.list.blobs(function (err, item, next) {
             next = Util.mkAsync(next, THROTTLE_FACTOR);
             if (err) {
-                Log.error("EVICT_BLOB_LIST_BLOBS_ERROR", err);
+                return Log.error("EVICT_BLOB_LIST_BLOBS_ERROR", err, next);
-                return void next();
             }
             if (!item) {
-                next();
+                return void Log.error('EVICT_BLOB_LIST_BLOBS_NO_ITEM', item, next);
-                return void Log.error('EVICT_BLOB_LIST_BLOBS_NO_ITEM', item);
             }
             total++;
             if (total % PROGRESS_FACTOR === 0) {
@@ -611,23 +599,21 @@ module.exports = function (Env, cb) {
             if (item.mtime > inactiveTime) { return void next(); }
 
             removed++;
-            blobs.archive.blob(item.blobId, function (err) {
+            blobs.archive.blob(item.blobId, 'INACTIVE', function (err) {
                 if (err) {
-                    Log.error("EVICT_ARCHIVE_BLOB_ERROR", {
+                    return Log.error("EVICT_ARCHIVE_BLOB_ERROR", {
                         error: err,
                         item: item,
-                    });
+                    }, next);
-                    return void next();
                 }
                 Log.info("EVICT_ARCHIVE_BLOB", {
                     item: item,
-                });
+                }, next);
-                next();
             });
         }, w(function () {
             report.totalBlobs = total;
             report.activeBlobs = total - removed;
-            Log.info('EVICT_BLOBS_REMOVED', removed);
+            Log.info('EVICT_BLOBS_REMOVED', removed, w());
         }));
     };
 
@@ -641,12 +627,10 @@ module.exports = function (Env, cb) {
         blobs.list.proofs(function (err, item, next) {
             next = Util.mkAsync(next, THROTTLE_FACTOR);
             if (err) {
-                next();
+                return void Log.error("EVICT_BLOB_LIST_PROOFS_ERROR", err, next);
-                return void Log.error("EVICT_BLOB_LIST_PROOFS_ERROR", err);
             }
             if (!item) {
-                next();
+                return void Log.error('EVICT_BLOB_LIST_PROOFS_NO_ITEM', item, next);
-                return void Log.error('EVICT_BLOB_LIST_PROOFS_NO_ITEM', item);
             }
             total++;
 
@@ -662,8 +646,7 @@ module.exports = function (Env, cb) {
                 blobs.size(item.blobId, w(function (err, size) {
                     if (err) {
                         w.abort();
-                        next();
+                        return void Log.error("EVICT_BLOB_LIST_PROOFS_ERROR", err, next);
-                        return void Log.error("EVICT_BLOB_LIST_PROOFS_ERROR", err);
                     }
                     if (size !== 0) {
                         w.abort();
@@ -672,19 +655,18 @@ module.exports = function (Env, cb) {
                 }));
             }).nThen(function () {
                 blobs.remove.proof(item.safeKey, item.blobId, function (err) {
-                    next();
                     if (err) {
-                        return Log.error("EVICT_BLOB_PROOF_LONELY_ERROR", item);
+                        return Log.error("EVICT_BLOB_PROOF_LONELY_ERROR", item, next);
                     }
                     removed++;
-                    return Log.info("EVICT_BLOB_PROOF_LONELY", item);
+                    return Log.info("EVICT_BLOB_PROOF_LONELY", item, next);
                 });
             });
         }, w(function () {
             Log.info("EVICT_BLOB_PROOFS_REMOVED", {
                 removed,
                 total,
-            });
+            }, w());
         }));
     };
 
@@ -703,8 +685,7 @@ module.exports = function (Env, cb) {
             }
 
             if (err) {
-                Log.error('EVICT_CHANNEL_ITERATION', err);
+                return Log.error('EVICT_CHANNEL_ITERATION', err, cb);
-                return void cb();
             }
 
             // ignore the special admin broadcast channel
@@ -715,14 +696,12 @@ module.exports = function (Env, cb) {
             if (item.channel.length === 34) {
                 return void store.removeChannel(item.channel, w(function (err) {
                     if (err) {
-                        Log.error('EVICT_EPHEMERAL_CHANNEL_REMOVAL_ERROR', {
+                        return Log.error('EVICT_EPHEMERAL_CHANNEL_REMOVAL_ERROR', {
                             error: err,
                             channel: item.channel,
-                        });
+                        }, cb);
-                        return void cb();
                     }
-                    Log.info('EVICT_EPHEMERAL_CHANNEL_REMOVAL', item.channel);
+                    Log.info('EVICT_EPHEMERAL_CHANNEL_REMOVAL', item.channel, cb);
-                    cb();
                 }));
             }
 
@@ -744,20 +723,19 @@ module.exports = function (Env, cb) {
                     }
                     // else fall through to the archival
                 }));
-            }).nThen(function () {
+            }).nThen(function (w) {
-                return void store.archiveChannel(item.channel, w(function (err) {
+                return void store.archiveChannel(item.channel, 'INACTIVE', w(function (err) {
                     if (err) {
                         Log.error('EVICT_CHANNEL_ARCHIVAL_ERROR', {
                             error: err,
                             channel: item.channel,
-                        });
+                        }, w());
-                        return void cb();
+                        return;
                     }
-                    Log.info('EVICT_CHANNEL_ARCHIVAL', item.channel);
+                    Log.info('EVICT_CHANNEL_ARCHIVAL', item.channel, w());
                     archived++;
-                    cb();
                 }));
-            });
+            }).nThen(cb);
         };
 
         var done = function () {

lib/hk-util.js

@@ -124,7 +124,7 @@ var CHECKPOINT_PATTERN = /^cp\|(([A-Za-z0-9+\/=]+)\|)?/;
     but for some reason are still present
 */
 const expireChannel = HK.expireChannel = function (Env, channel) {
-    return void Env.store.archiveChannel(channel, function (err) {
+    return void Env.store.archiveChannel(channel, 'EXPIRED', function (err) {
         Env.Log.info("ARCHIVAL_CHANNEL_BY_HISTORY_KEEPER_EXPIRATION", {
             channelId: channel,
             status: err? String(err): "SUCCESS",
@@ -199,10 +199,7 @@ const getMetadata = HK.getMetadata = function (Env, channelName, _cb) {
     }
 
     MetaRPC.getMetadataRaw(Env, channelName, function (err, metadata) {
-        if (err) {
+        if (err) { return void cb(err); }
-            console.error(err);
-            return void cb(err);
-        }
         if (!(metadata && typeof(metadata.channel) === 'string' && metadata.channel.length === STANDARD_CHANNEL_LENGTH)) {
             return cb();
         }
@@ -526,7 +523,12 @@ const getHistoryAsync = (Env, channelName, lastKnownHash, beforeHash, handler, c
         getHistoryOffset(Env, channelName, lastKnownHash, waitFor((err, os) => {
             if (err) {
                 waitFor.abort();
-                return void cb(err);
+                var reason;
+                if (err && err.reason) {
+                    reason = err.reason;
+                    err = err.error;
+                }
+                return void cb(err, reason);
             }
             offset = os;
         }));
@@ -540,8 +542,8 @@ const getHistoryAsync = (Env, channelName, lastKnownHash, beforeHash, handler, c
             const parsed = tryParse(Env, msgObj.buff.toString('utf8'));
             if (!parsed) { return void readMore(); }
             handler(parsed, readMore);
-        }, waitFor(function (err) {
+        }, waitFor(function (err, reason) {
-            return void cb(err);
+            return void cb(err, reason);
         }));
     });
 };
@@ -578,6 +580,7 @@ const handleFirstMessage = function (Env, channelName, metadata) {
         // Set the selfdestruct flag to history keeper ID to handle server crash.
         metadata.selfdestruct = Env.id;
     }
+    delete metadata.forcePlaceholder;
     Env.store.writeMetadata(channelName, JSON.stringify(metadata), function (err) {
         if (err) {
             // FIXME tell the user that there was a channel error?
@@ -682,7 +685,7 @@ const handleGetHistory = function (Env, Server, seq, userId, parsed) {
             if (isMetadataMessage(msg) && metadata_cache[channelName]) { return readMore(); }
             if (txid) { msg[0] = txid; }
             Server.send(userId, [0, HISTORY_KEEPER_ID, 'MSG', userId, JSON.stringify(msg)], readMore);
-        }, (err) => {
+        }, (err, reason) => {
             // Any error but ENOENT: abort
             // ENOENT is allowed in case we want to create a new pad
             if (err && err.code !== 'ENOENT') {
@@ -704,6 +707,11 @@ const handleGetHistory = function (Env, Server, seq, userId, parsed) {
                 Server.send(userId, [0, HISTORY_KEEPER_ID, 'MSG', userId, JSON.stringify(parsedMsg)]);
                 return;
             }
+            if (err && err.code === 'ENOENT' && reason && !metadata.forcePlaceholder) {
+                const parsedMsg2 = {error:'EDELETED', message: reason, channel: channelName, txid: txid};
+                Server.send(userId, [0, HISTORY_KEEPER_ID, 'MSG', userId, JSON.stringify(parsedMsg2)]);
+                return;
+            }
 
             // If we're asking for a specific version (lastKnownHash) but we receive an
             // ENOENT, this is not a pad creation so we need to abort.

lib/http-worker.js

@@ -9,6 +9,8 @@ const Logger = require("./log");
 const AuthCommands = require("./http-commands");
 const MFA = require("./storage/mfa");
 const Sessions = require("./storage/sessions");
+const BlobStore = require("./storage/blob");
+const BlockStore = require("./storage/block");
 
 const DEFAULT_QUERY_TIMEOUT = 5000;
 const PID = process.pid;
@@ -198,6 +200,13 @@ app.use('/blob', function (req, res, next) {
 /*  Head requests are used to check the size of a blob.
     Clients can configure a maximum size to download automatically,
     and can manually click to download blobs which exceed that limit.  */
+    const url = req.url;
+    if (typeof(url) === "string" && Env.blobStore) {
+        const s = url.split('/');
+        if (s[1] && s[1].length === 2 && s[2] && s[2].length === Env.blobStore.BLOB_LENGTH) {
+            Env.blobStore.updateActivity(s[2], () => {});
+        }
+    }
     if (req.method === 'HEAD') {
         Express.static(Path.resolve(Env.paths.blob), {
             setHeaders: function (res /*, path, stat */) {
@@ -404,6 +413,23 @@ app.use('/block/', function (req, res, next) {
 app.use("/block", Express.static(Path.resolve(Env.paths.block), {
     maxAge: "0d",
 }));
+// In case of a 404 for the block, check if a placeholder exists
+// and provide the result if that's the case
+app.use("/block", (req, res, next) => {
+    const url = req.url;
+    if (typeof(url) === "string") {
+        const s = url.split('/');
+        if (s[1] && s[1].length === 2 && BlockStore.isValidKey(s[2])) {
+            return BlockStore.readPlaceholder(Env, s[2], (content) => {
+                res.status(404).json({
+                    reason: content,
+                    code: 404
+                });
+            });
+        }
+    }
+    next();
+});
 
 app.use("/customize", Express.static('customize'));
 app.use("/customize", Express.static('customize.dist'));
@@ -611,6 +637,17 @@ nThen(function (w) {
         // websocket traffic to the correct port (Env.websocketPort)
         wsProxy.upgrade(req, socket, head);
     });
+
+    var config = require("./load-config");
+    BlobStore.create({
+        blobPath: config.blobPath,
+        blobStagingPath: config.blobStagingPath,
+        archivePath: config.archivePath,
+        getSession: function () {},
+    }, w(function (err, blob) {
+        if (err) { return; }
+        Env.blobStore = blob;
+    }));
 }).nThen(function () {
     // TODO inform the parent process that this worker is ready
 

lib/log.js

@@ -1,5 +1,6 @@
 /*jshint esversion: 6 */
 var Store = require("./storage/file");
+var Util = require("./common-util");
 
 var Logger = module.exports;
 
@@ -15,9 +16,13 @@ var messageTemplate = function (type, time, tag, info) {
 
 var noop = function () {};
 
-var write = function (ctx, content) {
+var write = function (ctx, content, cb) {
-    if (!ctx.store) { return; }
+    if (typeof(cb) !== "function") { cb = noop; }
-    ctx.store.log(ctx.channelName, content, noop);
+    if (!ctx.store) {
+        cb = Util.mkAsync(cb);
+        return void cb();
+    }
+    ctx.store.log(ctx.channelName, content, cb);
 };
 
 // various degrees of logging
@@ -37,7 +42,7 @@ var createLogType = function (ctx, type) {
     if (logLevels.indexOf(type) < logLevels.indexOf(ctx.logLevel)) {
         return noop;
     }
-    return function (tag, info) {
+    return function (tag, info, cb) {
         if (ctx.shutdown) {
             throw new Error("Logger has been shut down!");
         }
@@ -51,7 +56,7 @@ var createLogType = function (ctx, type) {
         if (ctx.logToStdout && typeof(handlers[type]) === 'function') {
             handlers[type](ctx, content);
         }
-        write(ctx, content);
+        write(ctx, content, cb);
     };
 };
 

lib/pins.js

@@ -33,6 +33,29 @@ var createLineHandler = Pins.createLineHandler = function (ref, errorHandler) {
     ref.index = 0;
     ref.latest = 0; // the latest message (timestamp in ms)
     ref.surplus = 0; // how many lines exist behind a reset
+
+
+    // Extract metadata from the channel list (#block, #drive)
+    let sanitize = (id, isPin) => {
+        if (typeof(id) !== "string") { return; }
+        let idx = id.indexOf('#');
+        if (idx < 0) { return id; }
+
+        let type = id.slice(idx+1);
+        let sanitized = id.slice(0, idx);
+        if (!isPin) { return sanitized; }
+
+        if (type === 'block') { // Note: teams don't have a block
+            ref.block = sanitized;
+            return;
+        }
+        if (type === 'drive') {
+            ref.drive = sanitized;
+            return sanitized;
+        }
+        return sanitized;
+    };
+
     return function (line) {
         ref.index++;
         if (!Boolean(line)) { return; }
@@ -55,17 +78,31 @@ var createLineHandler = Pins.createLineHandler = function (ref, errorHandler) {
         switch (l[0]) {
             case 'RESET': {
                 pins = ref.pins = {};
-                if (l[1] && l[1].length) { l[1].forEach((x) => { ref.pins[x] = 1; }); }
+                if (l[1] && l[1].length) {
+                    l[1].forEach((x) => {
+                        x = sanitize(x, true);
+                        if (!x) { return; }
+                        ref.pins[x] = 1;
+                    });
+                }
                 ref.surplus = ref.index;
                 //jshint -W086
                 // fallthrough
             }
             case 'PIN': {
-                l[1].forEach((x) => { pins[x] = 1; });
+                l[1].forEach((x) => {
+                    x = sanitize(x, true);
+                    if (!x) { return; }
+                    pins[x] = 1;
+                });
                 break;
             }
             case 'UNPIN': {
-                l[1].forEach((x) => { delete pins[x]; });
+                l[1].forEach((x) => {
+                    x = sanitize(x, false);
+                    if (!x) { return; }
+                    delete pins[x];
+                });
                 break;
             }
             default:

lib/storage/blob.js

@@ -8,12 +8,14 @@ var nThen = require("nthen");
 var Semaphore = require("saferphore");
 var Util = require("../common-util");
 
+const BLOB_LENGTH = 48;
+
 var isValidSafeKey = function (safeKey) {
     return typeof(safeKey) === 'string' && !/\//.test(safeKey) && safeKey.length === 44;
 };
 
 var isValidId = function (id) {
-    return typeof(id) === 'string' && id.length === 48 && !/[^a-f0-9]/.test(id);
+    return typeof(id) === 'string' && id.length === BLOB_LENGTH && !/[^a-f0-9]/.test(id);
 };
 
 // helpers
@@ -31,6 +33,10 @@ var makeBlobPath = function (Env, blobId) {
     return Path.join(Env.blobPath, blobId.slice(0, 2), blobId);
 };
 
+var makeActivityPath = function (Env, blobId) {
+    return makeBlobPath(Env, blobId) + '.activity';
+};
+
 // /blobstate/<safeKeyPrefix>/<safeKey>
 var makeStagePath = function (Env, safeKey) {
     return Path.join(Env.blobStagingPath, safeKey.slice(0, 2), safeKey);
@@ -41,6 +47,10 @@ var makeProofPath = function (Env, safeKey, blobId) {
     return Path.join(Env.blobPath, safeKey.slice(0, 3), safeKey, blobId.slice(0, 2), blobId);
 };
 
+var mkPlaceholderPath = function (Env, blobId) {
+    return makeBlobPath(Env, blobId) + '.placeholder';
+};
+
 var parseProofPath = function (path) {
     var parts = path.split('/');
     return {
@@ -49,6 +59,26 @@ var parseProofPath = function (path) {
     };
 };
 
+// Placeholder for deleted files
+var addPlaceholder = function (Env, blobId, reason, cb) {
+    if (!reason) { return cb(); }
+    var path = mkPlaceholderPath(Env, blobId);
+    var s_data = typeof(reason) === "string" ? reason : `${reason.code}:${reason.txt}`;
+    Fs.writeFile(path, s_data, cb);
+};
+var clearPlaceholder = function (Env, blobId, cb) {
+    var path = mkPlaceholderPath(Env, blobId);
+    Fs.unlink(path, cb);
+};
+var readPlaceholder = function (Env, blobId, cb) {
+    var path = mkPlaceholderPath(Env, blobId);
+    Fs.readFile(path, function (err, content) {
+        if (err) { return void cb(); }
+        cb(content.toString('utf8'));
+    });
+};
+
+
 // getUploadSize: used by
     // getFileSize
 var getUploadSize = function (Env, blobId, cb) {
@@ -57,7 +87,16 @@ var getUploadSize = function (Env, blobId, cb) {
     Fs.stat(path, function (err, stats) {
         if (err) {
             // if a file was deleted, its size is 0 bytes
-            if (err.code === 'ENOENT') { return cb(void 0, 0); }
+            if (err.code === 'ENOENT') {
+                return readPlaceholder(Env, blobId, (content) => {
+                    if (!content) { return cb(void 0, 0); }
+                    cb({
+                        code: err.code,
+                        reason: content
+                    });
+
+                });
+            }
             return void cb(err.code);
         }
         cb(void 0, stats.size);
@@ -103,6 +142,47 @@ var makeFileStream = function (full, _cb) {
     });
 };
 
+var clearActivity = function (Env, blobId, cb) {
+    var path = makeActivityPath(Env, blobId);
+    // if we fail to delete the activity file, it can still be removed later by the eviction script
+    Fs.unlink(path, cb);
+};
+var updateActivity = function (Env, blobId, cb) {
+    var path = makeActivityPath(Env, blobId);
+    var s_data = String(+new Date());
+    Fs.writeFile(path, s_data, cb);
+};
+
+var archiveActivity = function (Env, blobId, cb) {
+    var path = makeActivityPath(Env, blobId);
+    var archivePath = prependArchive(Env, path);
+    // if we fail to delete the activity file, it can still be removed later by the eviction script
+    Fse.move(path, archivePath, { overwrite: true }, cb);
+};
+var removeArchivedActivity = function (Env, blobId, cb) {
+    var path = makeActivityPath(Env, blobId);
+    var archivePath = prependArchive(Env, path);
+    Fs.unlink(archivePath, cb);
+};
+var restoreActivity = function (Env, blobId, cb) {
+    var path = makeActivityPath(Env, blobId);
+    var archivePath = prependArchive(Env, path);
+    Fse.move(archivePath, path, cb);
+};
+
+var getActivity = function (Env, blobId, cb) {
+    var path = makeActivityPath(Env, blobId);
+    Fs.readFile(path, function (err, content) {
+        if (err) { return void cb(err); }
+        try {
+            var date = new Date(+content);
+            cb(void 0, date);
+        } catch (err2) {
+            cb(err2);
+        }
+    });
+};
+
 /**********  METHODS **************/
 
 var upload = function (Env, safeKey, content, cb) {
@@ -298,10 +378,12 @@ var owned_upload_complete = function (Env, safeKey, id, cb) {
     });
 };
 
+
 // removeBlob
 var remove = function (Env, blobId, cb) {
     var blobPath = makeBlobPath(Env, blobId);
     Fs.unlink(blobPath, cb);
+    clearActivity(Env, blobId, () => {});
 };
 
 // removeProof
@@ -318,15 +400,18 @@ var isOwnedBy = function (Env, safeKey, blobId, cb) {
 
 
 // archiveBlob
-var archiveBlob = function (Env, blobId, cb) {
+var archiveBlob = function (Env, blobId, reason, cb) {
     var blobPath = makeBlobPath(Env, blobId);
     var archivePath = prependArchive(Env, blobPath);
     Fse.move(blobPath, archivePath, { overwrite: true }, cb);
+    archiveActivity(Env, blobId, () => {});
+    addPlaceholder(Env, blobId, reason, () => {});
 };
 
 var removeArchivedBlob = function (Env, blobId, cb) {
     var archivePath = prependArchive(Env, makeBlobPath(Env, blobId));
     Fs.unlink(archivePath, cb);
+    removeArchivedActivity(Env, blobId, () => {});
 };
 
 // restoreBlob
@@ -334,6 +419,8 @@ var restoreBlob = function (Env, blobId, cb) {
     var blobPath = makeBlobPath(Env, blobId);
     var archivePath = prependArchive(Env, blobPath);
     Fse.move(archivePath, blobPath, cb);
+    restoreActivity(Env, blobId, () => {});
+    clearPlaceholder(Env, blobId, () => {});
 };
 
 // archiveProof
@@ -506,6 +593,7 @@ BlobStore.create = function (config, _cb) {
         Fse.writeFile(fullPath, 'PLACEHOLDER\n', w());
     }).nThen(function () {
         var methods = {
+            BLOB_LENGTH: BLOB_LENGTH,
             isFileId: isValidId,
             status: function (safeKey, _cb) {
                 // TODO check if the final destination is a file
@@ -562,10 +650,10 @@ BlobStore.create = function (config, _cb) {
             },
 
             archive: {
-                blob: function (blobId, _cb) {
+                blob: function (blobId, reason, _cb) {
                     var cb = Util.once(Util.mkAsync(_cb));
                     if (!isValidId(blobId)) { return void cb("INVALID_ID"); }
-                    archiveBlob(Env, blobId, cb);
+                    archiveBlob(Env, blobId, reason, cb);
                 },
                 proof: function (safeKey, blobId, _cb) {
                     var cb = Util.once(Util.mkAsync(_cb));
@@ -601,6 +689,9 @@ BlobStore.create = function (config, _cb) {
                 var path = prependArchive(Env, makeBlobPath(Env, blobId));
                 isFile(path, cb);
             },
+            getPlaceholder: function (blobId, cb) {
+                readPlaceholder(Env, blobId, cb);
+            },
 
             closeBlobstage: function (safeKey) {
                 closeBlobstage(Env, safeKey);
@@ -623,6 +714,18 @@ BlobStore.create = function (config, _cb) {
                 getUploadSize(Env, id, cb);
             },
 
+            // ACTIVITY
+            updateActivity: function (id, _cb) {
+                var cb = Util.once(Util.mkAsync(_cb));
+                if (!isValidId(id)) { return void cb("INVALID_ID"); }
+                updateActivity(Env, id, cb);
+            },
+            getActivity: function (id, _cb) {
+                var cb = Util.once(Util.mkAsync(_cb));
+                if (!isValidId(id)) { return void cb("INVALID_ID"); }
+                getActivity(Env, id, cb);
+            },
+
             list: {
                 blobs: function (handler, _cb) {
                     var cb = Util.once(Util.mkAsync(_cb));

lib/storage/block.js

@@ -32,7 +32,28 @@ Block.mkArchivePath = function (Env, publicKey) {
     return Path.join(Env.paths.archive, 'block', safeKey.slice(0, 2), safeKey);
 };
 
-Block.archive = function (Env, publicKey, _cb) {
+var mkPlaceholderPath = function (Env, publicKey) {
+    return Block.mkPath(Env, publicKey) + '.placeholder';
+};
+var addPlaceholder = function (Env, publicKey, reason, cb) {
+    if (!reason) { return cb(); }
+    var path = mkPlaceholderPath(Env, publicKey);
+    var s_data = typeof(reason) === "string" ? reason : `${reason.code}:${reason.txt}`;
+    Fs.writeFile(path, s_data, cb);
+};
+var clearPlaceholder = function (Env, publicKey, cb) {
+    var path = mkPlaceholderPath(Env, publicKey);
+    Fs.unlink(path, cb);
+};
+Block.readPlaceholder = function (Env, publicKey, cb) {
+    var path = mkPlaceholderPath(Env, publicKey);
+    Fs.readFile(path, function (err, content) {
+        if (err) { return void cb(); }
+        cb(content.toString('utf8'));
+    });
+};
+
+Block.archive = function (Env, publicKey, reason, _cb) {
     var cb = Util.once(Util.mkAsync(_cb));
 
     // derive the filepath
@@ -52,7 +73,10 @@ Block.archive = function (Env, publicKey, _cb) {
     // TODO Env.incrementBytesWritten
     Fse.move(currentPath, archivePath, {
         overwrite: true,
-    }, cb);
+    }, (err) => {
+        cb(err);
+        if (!err && reason) { addPlaceholder(Env, publicKey, reason, () => {}); }
+    });
 };
 
 Block.restore = function (Env, publicKey, _cb) {
@@ -75,10 +99,13 @@ Block.restore = function (Env, publicKey, _cb) {
     // TODO Env.incrementBytesWritten
     Fse.move(archivePath, livePath, {
         //overwrite: true,
-    }, cb);
+    }, (err) => {
+        cb(err);
+        if (!err) { clearPlaceholder(Env, publicKey, () => {}); }
+    });
 };
 
-var isValidKey = function (publicKey) {
+var isValidKey = Block.isValidKey = function (publicKey) {
     return typeof(publicKey) === 'string' && publicKey.length === 44;
 };
 
@@ -131,7 +158,7 @@ Block.write = function (Env, publicKey, buffer, _cb) {
             cb(err);
         }));
     }).nThen(function (w) {
-        Block.archive(Env, publicKey, w(function (/* err */) {
+        Block.archive(Env, publicKey, 'PASSWORD_CHANGE', w(function (/* err */) {
     /*
         we proceed even if there are errors.
         it might be ENOENT (there is no file to archive)

lib/storage/file.js

@@ -70,6 +70,10 @@ var mkOffsetPath = function (env, channelId) {
     return mkPath(env, channelId) + '.offset';
 };
 
+var mkPlaceholderPath = function (env, channelId) {
+    return mkPath(env, channelId) + '.placeholder';
+};
+
 // pass in the path so we can reuse the same function for archived files
 var channelExists = function (filepath, cb) {
     Fs.stat(filepath, function (err, stat) {
@@ -141,6 +145,25 @@ var isChannelArchived = function (env, channelName, cb) {
     });
 };
 
+var addPlaceholder = function (env, channelId, reason, cb) {
+    if (!reason) { return cb(); }
+    var path = mkPlaceholderPath(env, channelId);
+    var s_data = typeof(reason) === "string" ? reason : `${reason.code}:${reason.txt}`;
+    Fs.writeFile(path, s_data, cb);
+};
+var clearPlaceholder = function (env, channelId, cb) {
+    var path = mkPlaceholderPath(env, channelId);
+    Fs.unlink(path, cb);
+};
+var readPlaceholder = function (env, channelId, cb) {
+    var path = mkPlaceholderPath(env, channelId);
+    Fs.readFile(path, function (err, content) {
+        if (err) { return void cb(); }
+        cb(content.toString('utf8'));
+    });
+};
+
+
 const destroyStream = function (stream) {
     if (!stream) { return; }
     try {
@@ -190,7 +213,16 @@ const readMessagesBin = (env, id, start, msgHandler, cb) => {
     const stream = Fs.createReadStream(mkPath(env, id), { start: start });
     const collector = createIdleStreamCollector(stream);
     const handleMessageAndKeepStreamAlive = Util.both(msgHandler, collector.keepAlive);
-    const done = Util.both(cb, collector);
+    const done = Util.both((err) => {
+        if (err && err.code === 'ENOENT') {
+            // If the channel doesn't exists, look for a placeholder.
+            // If a placeholder exists, call back with its content in addition to the original error
+            return readPlaceholder(env, id, (content) => {
+                cb(err, content);
+            });
+        }
+        cb(err);
+    }, collector);
     return void readFileBin(stream, handleMessageAndKeepStreamAlive, done, {
         offset: start,
     });
@@ -360,7 +392,14 @@ var getDedicatedMetadata = function (env, channelId, handler, _cb) {
         readMore();
     }, function (err) {
         // ENOENT => there is no metadata log
-        if (!err || err.code === 'ENOENT') { return void cb(); }
+        if (!err || err.code === 'ENOENT') {
+            if (err && err.code === 'ENOENT') {
+                return readPlaceholder(env, channelId, (content) => {
+                    cb(content);
+                });
+            }
+            return void cb();
+        }
         // otherwise stream errors?
         cb(err);
     });
@@ -670,7 +709,7 @@ var listChannels = function (root, handler, cb, fast) {
 
 // move a channel's log file from its current location
 // to an equivalent location in the cold storage directory
-var archiveChannel = function (env, channelName, cb) {
+var archiveChannel = function (env, channelName, reason, cb) {
     // TODO close channels before archiving them?
 
     // ctime is the most reliable indicator of when a file was archived
@@ -700,6 +739,9 @@ var archiveChannel = function (env, channelName, cb) {
         }));
     }).nThen(function (w) {
         clearOffset(env, channelName, w());
+    }).nThen(function (w) {
+        if (!reason) { return; }
+        addPlaceholder(env, channelName, reason, w());
     }).nThen(function (w) {
         // archive the dedicated metadata channel
         var metadataPath = mkMetadataPath(env, channelName);
@@ -775,6 +817,8 @@ var unarchiveChannel = function (env, channelName, cb) {
                 return void CB(err);
             }
         }));
+    }).nThen(function (w) {
+        clearPlaceholder(env, channelName, w());
     }).nThen(function (w) {
         var archiveMetadataPath = mkArchiveMetadataPath(env, channelName);
         // TODO validate that it's ok to move metadata non-atomically
@@ -1290,12 +1334,12 @@ module.exports.create = function (conf, _cb) {
                 isChannelArchived(env, channelName, cb);
             },
             // move a channel from the database to the archive, along with its metadata
-            archiveChannel: function (channelName, cb) {
+            archiveChannel: function (channelName, reason, cb) {
                 if (!isValidChannelId(channelName)) { return void cb(new Error('EINVAL')); }
 // again, the semantics around archiving and appending are really muddy.
 // so I'm calling this 'unordered' again
                 schedule.unordered(channelName, function (next) {
-                    archiveChannel(env, channelName, Util.both(cb, next));
+                    archiveChannel(env, channelName, reason, Util.both(cb, next));
                 });
             },
             // restore a channel from the archive to the database, along with its metadata
@@ -1385,6 +1429,9 @@ module.exports.create = function (conf, _cb) {
                     channelBytes(env, channelName, Util.both(cb, next));
                 });
             },
+            getPlaceholder: function (channelName, cb) {
+                readPlaceholder(env, channelName, cb);
+            },
         // OTHER DATABASE FUNCTIONALITY
             // remove a particular channel from the cache
             closeChannel: function (channelName, cb) {

lib/storage/tasks.js

@@ -206,7 +206,7 @@ var expire = function (env, task, cb) {
     Log.info('ARCHIVAL_SCHEDULED_EXPIRATION', {
         task: task,
     });
-    env.store.archiveChannel(args[0], function (err) {
+    env.store.archiveChannel(args[0], 'EXPIRED', function (err) {
         if (err) {
             Log.error('ARCHIVE_SCHEDULED_EXPIRATION_ERROR', {
                 task: task,

lib/workers/db-worker.js

@@ -387,8 +387,10 @@ const getPinState = function (data, cb) {
     var lineHandler = Pins.createLineHandler(ref, Env.Log.error);
 
     // if channels aren't in memory. load them from disk
-    // TODO replace with readMessagesBin
+    pinStore.readMessagesBin(safeKey, 0, (msgObj, readMore) => {
-    pinStore.getMessages(safeKey, lineHandler, function () {
+        lineHandler(msgObj.buff.toString('utf8'));
+        readMore();
+    }, function () {
         cb(void 0, ref.pins); // FIXME no error handling?
     });
 };
@@ -497,8 +499,13 @@ const getHashOffset = function (data, cb) {
         }
         offset = msgObj.offset;
         abort();
-    }, function (err) {
+    }, function (err, reason) {
-        if (err) { return void cb(err); }
+        if (err) {
+            return void cb({
+                error: err,
+                reason: reason
+            });
+        }
         cb(void 0, offset);
     });
 };
@@ -508,6 +515,8 @@ const removeOwnedBlob = function (data, cb) {
     const blobId = data.blobId;
     const safeKey = Util.escapeKeyCharacters(data.safeKey);
 
+    const reason = data.reason || 'ARCHIVE_OWNED';
+
     nThen(function (w) {
         // check if you have permissions
         blobStore.isOwnedBy(safeKey, blobId, w(function (err, owned) {
@@ -518,7 +527,7 @@ const removeOwnedBlob = function (data, cb) {
         }));
     }).nThen(function (w) {
         // remove the blob
-        blobStore.archive.blob(blobId, w(function (err) {
+        blobStore.archive.blob(blobId, reason, w(function (err) {
             Env.Log.info('ARCHIVAL_OWNED_FILE_BY_OWNER_RPC', {
                 safeKey: safeKey,
                 blobId: blobId,
@@ -599,16 +608,18 @@ const getPinActivity = function (data, cb) {
     var safeKey = Util.escapeKeyCharacters(data.key);
     var first;
     var latest;
-    pinStore.getMessages(safeKey, line => {
+    pinStore.readMessagesBin(safeKey, 0, (msgObj, readMore) => {
-        if (!line || !line.trim()) { return; }
+        var line = msgObj.buff.toString('utf8');
+        if (!line || !line.trim()) { return readMore(); }
         try {
             var parsed = JSON.parse(line);
             var temp = parsed[parsed.length - 1];
-            if (!temp || typeof(temp) !== 'number') { return; }
+            if (!temp || typeof(temp) !== 'number') { return readMore(); }
             latest = temp;
-            if (first) { return; }
+            if (first) { return readMore(); }
             first = latest;
-        } catch (err) { }
+            readMore();
+        } catch (err) { readMore(); }
     }, function (err) {
         if (err) { return void cb(err); }
         cb(void 0, {

lib/workers/index.js

@@ -382,11 +382,12 @@ Workers.initialize = function (Env, config, _cb) {
             });
         };
 
-        Env.removeOwnedBlob = function (blobId, safeKey, cb) {
+        Env.removeOwnedBlob = function (blobId, safeKey, reason, cb) {
             sendCommand({
                 command: 'REMOVE_OWNED_BLOB',
                 blobId: blobId,
                 safeKey: safeKey,
+                reason: reason
             }, cb);
         };
 

package.json

@@ -1,7 +1,7 @@
 {
   "name": "cryptpad",
   "description": "realtime collaborative visual editor with zero knowlege server",
-  "version": "5.4.1",
+  "version": "5.5.0",
   "license": "AGPL-3.0+",
   "repository": {
     "type": "git",
@@ -39,7 +39,7 @@
     "chainpad": "^5.2.6",
     "chainpad-listmap": "^1.0.0",
     "chainpad-netflux": "^1.0.0",
-    "ckeditor": "npm:ckeditor4@^4.22.1",
+    "ckeditor": "npm:ckeditor4@~4.22.1",
     "codemirror": "^5.19.0",
     "components-font-awesome": "^4.6.3",
     "croppie": "^2.5.0",

readme.md

@@ -58,9 +58,9 @@ meet our strict criteria for safety.
 # Translations
 
 CryptPad can be translated with nothing more than a web browser via our
-[Weblate instance](https://weblate.cryptpad.fr/projects/cryptpad/app/). See the state of the translated languages:
+[Weblate instance](https://weblate.cryptpad.org/projects/cryptpad/app/). See the state of the translated languages:
 
-![](https://weblate.cryptpad.fr/widgets/cryptpad/-/app/multi-auto.svg)
+![](https://weblate.cryptpad.org/widgets/cryptpad/-/app/multi-auto.svg)
 
 More information about this can be found in [our translation guide](/customize.dist/translations/README.md).
 
@@ -68,10 +68,7 @@ More information about this can be found in [our translation guide](/customize.d
 
 The best places to reach the development team and the community are the [CryptPad Forum](https://forum.cryptpad.org) and the [Matrix chat](https://matrix.to/#/#cryptpad:matrix.xwiki.com)
 
-The team is also on social media:
+The team is also on the fediverse: [@cryptpad@fosstodon.org](https://fosstodon.org/@cryptpad)
-- Mastodon: [@cryptpad@fosstodon.org](https://fosstodon.org/@cryptpad)
-- Twitter: [@cryptpad](https://twitter.com/cryptpad)
-
 
 # Team
 

www/admin/app-admin.less

@@ -13,8 +13,13 @@
         display: none !important;
     }
 
+    label{
+        margin-top:0.5rem;
+    }
+
     display: flex;
     flex-flow: column;
+
     a {
         color: @cryptpad_color_link;
         text-decoration: underline;
@@ -35,7 +40,7 @@
         }
         nav {
             display: flex;
-            margin-top: 5px;
+            margin-top: 0.5rem;
         }
     }
     .cp-admin-setlimit-form,

www/admin/inner.js

@@ -127,7 +127,7 @@ define([
         // Convert to camlCase for translation keys
         var safeKey = keyToCamlCase(key);
         var $div = $('<div>', {'class': 'cp-admin-' + key + ' cp-sidebarlayout-element'});
-        $('<label>').text(Messages['admin_'+safeKey+'Title'] || key).appendTo($div);
+        $('<label>', {'id': 'cp-admin-' + key}).text(Messages['admin_'+safeKey+'Title'] || key).appendTo($div);
         $('<span>', {'class': 'cp-sidebarlayout-description'})
             .text(Messages['admin_'+safeKey+'Hint'] || 'Coming soon...').appendTo($div);
         if (addButton) {
@@ -226,6 +226,7 @@ define([
                 data.currentlyOnline = response[0];
             }));
         }).nThen(function (w) {
+            if (!data.first) { return; }
             sframeCommand('GET_USER_QUOTA', key, w((err, response) => {
                 if (err || !response) {
                     return void console.error('quota', err, response);
@@ -236,6 +237,7 @@ define([
                 }
             }));
         }).nThen(function (w) {
+            if (!data.first) { return; }
             // storage used
             sframeCommand('GET_USER_TOTAL_SIZE', key, w((err, response) => {
                 if (err || !Array.isArray(response)) {
@@ -246,6 +248,7 @@ define([
                 }
             }));
         }).nThen(function (w) {
+            if (!data.first) { return; }
             // channels pinned
             // files pinned
             sframeCommand('GET_USER_STORAGE_STATS', key, w((err, response) => {
@@ -268,6 +271,17 @@ define([
                     data.archived = response[0].archived;
                 }
             }));
+        }).nThen(function (w) {
+            if (data.first) { return; }
+            // Account is probably deleted
+            sframeCommand('GET_ACCOUNT_ARCHIVE_STATUS', {key}, w((err, response) => {
+                if (err || !Array.isArray(response) || !response[0]) {
+                    console.error('account status', err, response);
+                } else {
+                    console.info('account status', response);
+                    data.archiveReport = response[0];
+                }
+            }));
         }).nThen(function () {
             //console.log(data);
             try {
@@ -281,6 +295,12 @@ define([
                     if (typeof(val) !== 'number') { return; }
                     data[`${k}_formatted`] = getPrettySize(val);
                 });
+                if (data.archiveReport) {
+                    let formatted = Util.clone(data.archiveReport);
+                    formatted.channels = data.archiveReport.channels.length;
+                    formatted.blobs = data.archiveReport.blobs.length;
+                    data['archiveReport_formatted'] = JSON.stringify(formatted, 0, 2);
+                }
             } catch (err) {
                 console.error(err);
             }
@@ -391,14 +411,16 @@ define([
             ]));
         }
 
-        // First pin activity time
+        if (data.first || data.latest) {
-        row(Messages.admin_firstPinTime, maybeDate(data.first));
+            // First pin activity time
+            row(Messages.admin_firstPinTime, maybeDate(data.first));
 
-        // last pin activity time
+            // last pin activity time
-        row(Messages.admin_lastPinTime, maybeDate(data.latest));
+            row(Messages.admin_lastPinTime, maybeDate(data.latest));
+        }
 
         // currently online
-        row(Messages.admin_currentlyOnline, data.currentlyOnline);
+        row(Messages.admin_currentlyOnline, localizeState(data.currentlyOnline));
 
         // plan name
         row(Messages.admin_planName, data.plan || Messages.ui_none);
@@ -407,27 +429,46 @@ define([
         row(Messages.admin_note, data.note || Messages.ui_none);
 
         // storage limit
-        row(Messages.admin_planlimit, getPrettySize(data.limit));
+        if (data.limit) { row(Messages.admin_planlimit, getPrettySize(data.limit)); }
 
         // data stored
-        row(Messages.admin_storageUsage, getPrettySize(data.usage));
+        if (data.usage) { row(Messages.admin_storageUsage, getPrettySize(data.usage)); }
 
         // number of channels
-        row(Messages.admin_channelCount, data.channels);
+        if (typeof(data.channel) === "number") {
+            row(Messages.admin_channelCount, data.channels);
+        }
 
         // number of files pinned
-        row(Messages.admin_fileCount, data.files);
+        if (typeof(data.channel) === "number") {
+            row(Messages.admin_fileCount, data.files);
+        }
 
         row(Messages.admin_pinLogAvailable, localizeState(data.live));
 
         // pin log archived
         row(Messages.admin_pinLogArchived, localizeState(data.archived));
 
+        if (data.archiveReport) {
+            row(Messages.admin_accountSuspended, localizeState(Boolean(data.archiveReport)));
+        }
+        if (data.archiveReport_formatted) {
+            let button, pre;
+            row(Messages.admin_accountReport, h('div', [
+                pre = h('pre', data.archiveReport_formatted),
+                button = primary(Messages.admin_accountReportFull, () => {
+                    $(button).remove();
+                    $(pre).html(JSON.stringify(data.archiveReport, 0, 2));
+                })
+            ]));
+        }
+
+
     // actions
-        if (data.archived && data.live === false) {
+        if (data.archived && data.live === false && data.archiveReport) {
-            row(Messages.admin_restoreArchivedPins, primary(Messages.ui_restore, function () {
+            row(Messages.admin_restoreAccount, primary(Messages.ui_restore, function () {
                 justifyRestorationDialog('', reason => {
-                    sframeCommand('RESTORE_ARCHIVED_PIN_LOG', {
+                    sframeCommand('RESTORE_ACCOUNT', {
                         key: data.key,
                         reason: reason,
                     }, function (err) {
@@ -497,9 +538,10 @@ define([
 
             // archive pin log
             var archiveHandler = () => {
-                justifyArchivalDialog(Messages.admin_archivePinLogConfirm, reason => {
+                justifyArchivalDialog(Messages.admin_archiveAccountConfirm, reason => {
-                    sframeCommand('ARCHIVE_PIN_LOG', {
+                    sframeCommand('ARCHIVE_ACCOUNT', {
                         key: data.key,
+                        block: data.blockId,
                         reason: reason,
                     }, (err /*, response */) => {
                         console.error(err);
@@ -512,7 +554,12 @@ define([
                 });
             };
 
-            row(Messages.admin_archivePinLog, danger(Messages.admin_archiveButton, archiveHandler));
+            var archiveAccountLabel = h('span', [
+                Messages.admin_archiveAccount,
+                h('br'),
+                h('small', Messages.admin_archiveAccountInfo)
+            ]);
+            row(archiveAccountLabel, danger(Messages.admin_archiveButton, archiveHandler));
 
             // archive owned documents
 /* // TODO not implemented
@@ -678,6 +725,7 @@ define([
                 }
                 data.live = res[0].live;
                 data.archived = res[0].archived;
+                data.placeholder = res[0].placeholder;
                 //console.error("get channel status", err, res);
             }));
         }).nThen(function () {
@@ -696,7 +744,6 @@ define([
 
 /* FIXME
     Messages.admin_getFullPinHistory = 'Pin history';
-    Messages.admin_archivePinLogConfirm = "All content in this user's drive will be un-listed, meaning it may be deleted if it is not in any other drive.";
     Messages.admin_archiveOwnedAccountDocuments = "Archive this account's owned documents (not implemented)";
     Messages.admin_archiveOwnedDocumentsConfirm = "All content owned exclusively by this user will be archived. This means their documents, drive, and accounts will be made inaccessible.  This action cannot be undone. Please save the full pin list before proceeding to ensure individual documents can be restored.";
 */
@@ -791,6 +838,11 @@ define([
 
         }
 
+        if (data.placeholder) {
+            console.warn('Placeholder code', data.placeholder);
+            row(Messages.admin_channelPlaceholder, UI.getDestroyedPlaceholderMessage(data.placeholder));
+        }
+
         if (data.live && data.archived) {
             let disableButtons;
             let restoreButton = danger(Messages.admin_unarchiveButton, function () {
@@ -1075,6 +1127,7 @@ define([
                 data.live = res[0].live;
                 data.archived = res[0].archived;
                 data.totp = res[0].totp;
+                data.placeholder = res[0].placeholder;
             }));
         }).nThen(function () {
             try {
@@ -1098,8 +1151,8 @@ define([
         row(Messages.admin_blockAvailable, localizeState(data.live));
         row(Messages.admin_blockArchived, localizeState(data.archived));
 
-        row(Messages.admin_totpEnabled, localizeState(data.totp.enabled));
+        row(Messages.admin_totpEnabled, localizeState(Boolean(data.totp.enabled)));
-        row(Messages.admin_totpRecoveryMethod, data.totp.recovery); // XXX localize?
+        row(Messages.admin_totpRecoveryMethod, data.totp.recovery);
 
         if (data.live) {
             var archiveButton = danger(Messages.ui_archive, function () {
@@ -1120,6 +1173,10 @@ define([
             });
             row(Messages.admin_archiveBlock, archiveButton);
         }
+        if (data.placeholder) {
+            console.warn('Placeholder code', data.placeholder);
+            row(Messages.admin_channelPlaceholder, UI.getDestroyedPlaceholderMessage(data.placeholder, true));
+        }
         if (data.archived && !data.live) {
             var restoreButton = danger(Messages.ui_restore, function () {
                 justifyRestorationDialog('', reason => {
@@ -1251,7 +1308,7 @@ define([
 
         row(Messages.admin_totpEnabled, localizeState(Boolean(data.totp.enabled)));
         if (data.totp && data.totp.enabled) {
-            row(Messages.admin_totpRecoveryMethod, data.totp.recovery); // XXX localize?
+            row(Messages.admin_totpRecoveryMethod, data.totp.recovery);
         }
 
         if (!data.totpCheck || !data.totp.enabled) { return tableObj.table; }
@@ -1289,15 +1346,15 @@ define([
 
     create['totp-recovery'] = function () {
         var key = 'totp-recovery';
-        // XXX translation keys
         var $div = makeBlock(key, true); // Msg.admin_totpRecoveryHint.totpRecoveryTitle
 
-        var textarea = h('textarea', {});
+        var textarea = h('textarea', {
+            id: 'textarea-input',
+            'aria-labelledby': 'cp-admin-totp-recovery'
+        });
         var $input = $(textarea);
 
-        var box = h('div.cp-admin-setter', [
+        var box = h('div.cp-admin-setter', textarea);
-            textarea,
-        ]);
 
         $div.find('.cp-sidebarlayout-description').after(box);
 
@@ -1476,10 +1533,13 @@ Example
 
         var input = h('input', {
             type: 'email',
-            value: ApiConfig.adminEmail || ''
+            value: ApiConfig.adminEmail || '',
+            'aria-labelledby': 'cp-admin-email'
         });
         var $input = $(input);
+
         var innerDiv = h('div.cp-admin-setter.cp-admin-setlimit-form', input);
+
         var spinner = UI.makeSpinner($(innerDiv));
 
         $button.click(function () {
@@ -1527,6 +1587,7 @@ Example
             type: 'text',
             value: getInstanceString('instanceJurisdiction'),
             placeholder: Messages.owner_unknownUser || '',
+            'aria-labelledby': 'cp-admin-jurisdiction'
         });
         var $input = $(input);
         var innerDiv = h('div.cp-admin-setter', input);
@@ -1568,7 +1629,9 @@ Example
             type: 'text',
             value: getInstanceString('instanceNotice'),
             placeholder: '',
+            'aria-labelledby': 'cp-admin-notice'
         });
+
         var $input = $(input);
         var innerDiv = h('div.cp-admin-setter', input);
         var spinner = UI.makeSpinner($(innerDiv));
@@ -1617,6 +1680,7 @@ Example
             type: 'text',
             value: getInstanceString('instanceName') || ApiConfig.httpUnsafeOrigin || '',
             placeholder: ApiConfig.httpUnsafeOrigin,
+            'aria-labelledby': 'cp-admin-name'
         });
         var $input = $(input);
         var innerDiv = h('div.cp-admin-setter', input);
@@ -1653,6 +1717,7 @@ Example
 
         var textarea = h('textarea.cp-admin-description-text', {
             placeholder: Messages.home_host || '',
+            'aria-labelledby': 'cp-admin-description'
         }, getInstanceString('instanceDescription'));
 
         var $button = $div.find('button').text(Messages.settings_save);
@@ -1696,17 +1761,25 @@ Example
         var _limit = APP.instanceStatus.defaultStorageLimit;
         var _limitMB = Util.bytesToMegabytes(_limit);
         var limit = getPrettySize(_limit);
-        var newLimit = h('input', {type: 'number', min: 0, value: _limitMB});
+        var newLimit = h('input', {
+            type: 'number',
+            min: 0,
+            value: _limitMB,
+            'aria-labelledby': 'cp-admin-defaultlimit'
+        });
         var set = h('button.btn.btn-primary', Messages.admin_setlimitButton);
+
         $div.append(h('div', [
             h('span.cp-admin-defaultlimit-value', Messages._getKey('admin_limit', [limit])),
             h('div.cp-admin-setlimit-form', [
-                h('label', Messages.admin_defaultLimitMB),
                 newLimit,
                 h('nav', [set])
             ])
         ]));
 
+
+
+
         UI.confirmButton(set, {
             classes: 'btn-primary',
             multiple: true,
@@ -1815,21 +1888,23 @@ Example
         var key = 'setlimit';
         var $div = makeBlock(key); // Msg.admin_setlimitHint, .admin_setlimitTitle
 
-        var user = h('input.cp-setlimit-key');
+        var user = h('input.cp-setlimit-key', { id: 'user-input' });
         var $key = $(user);
-        var limit = h('input.cp-setlimit-quota', {type: 'number', min: 0, value: 0});
+        var limit = h('input.cp-setlimit-quota', { type: 'number', min: 0, value: 0, id: 'limit-input' });
-        var note = h('input.cp-setlimit-note');
+        var note = h('input.cp-setlimit-note', { id: 'note-input' });
         var remove = h('button.btn.btn-danger', Messages.fc_remove);
         var set = h('button.btn.btn-primary', Messages.admin_setlimitButton);
+
         var form = h('div.cp-admin-setlimit-form', [
-            h('label', Messages.admin_limitUser),
+            h('label', { for: 'user-input' }, Messages.admin_limitUser),
             user,
-            h('label', Messages.admin_limitMB),
+            h('label', { for: 'limit-input' }, Messages.admin_limitMB),
             limit,
-            h('label', Messages.admin_limitSetNote),
+            h('label', { for: 'note-input' }, Messages.admin_limitSetNote),
             note,
             h('nav', [set, remove])
         ]);
+
         var $note = $(note);
 
         var getValues = function () {
@@ -2016,11 +2091,15 @@ Example
         onRefreshStats.reg(onRefresh);
         return $div;
     };
+
     create['disk-usage'] = function () {
         var key = 'disk-usage';
         var $div = makeBlock(key, true); // Msg.admin_diskUsageHint, .admin_diskUsageTitle, .admin_diskUsageButton
         var called = false;
+
         $div.find('button').click(function () {
+        UI.confirm(Messages.admin_diskUsageWarning, function (yes) {
+            if (!yes) { return; }
             $div.find('button').hide();
             if (called) { return; }
             called = true;
@@ -2050,6 +2129,8 @@ Example
                 })));
             });
         });
+        });
+
         return $div;
     };
 
@@ -2643,10 +2724,14 @@ Example
                     'data-lang': l,
                     label: {class: 'noTitle'}
                 });
+
+                var label = h('label', { for: 'kanban-body' }, Messages.kanban_body);
+                var textarea = h('textarea', { id: 'kanban-body' });
+
                 $container.append(h('div.cp-broadcast-lang', { 'data-lang': l }, [
                     h('h4', languages[l]),
-                    h('label', Messages.kanban_body),
+                    label,
-                    h('textarea'),
+                    textarea,
                     radio,
                     preview
                 ]));
@@ -2781,8 +2866,8 @@ Example
             }
 
             // Start and end date pickers
-            var start = h('input');
+            var start = h('input#cp-admin-start-input');
-            var end = h('input');
+            var end = h('input#cp-admin-end-input');
             var $start = $(start);
             var $end = $(end);
             var is24h = UIElements.is24h();
@@ -2853,9 +2938,9 @@ Example
 
             $form.empty().append([
                 active,
-                h('label', Messages.broadcast_start),
+                h('label', { for: 'cp-admin-start-input' }, Messages.broadcast_start),
                 start,
-                h('label', Messages.broadcast_end),
+                h('label', { for: 'cp-admin-end-input' }, Messages.broadcast_end),
                 end,
                 h('br'),
                 h('div.cp-broadcast-form-submit', [
@@ -2901,8 +2986,8 @@ Example
             }
 
             // Survey form
-            var label = h('label', Messages.broadcast_surveyURL);
+            var label = h('label', { for: 'cp-admin-survey-url-input' }, Messages.broadcast_surveyURL);
-            var input = h('input');
+            var input = h('input#cp-admin-survey-url-input');
             var $input = $(input);
 
             // Extract form data
@@ -3084,16 +3169,20 @@ Example
 
         var duration = APP.instanceStatus.profilingWindow;
         if (!isPositiveInteger(duration)) { duration = 10000; }
-        var newDuration = h('input', {type: 'number', min: 0, value: duration});
+        var newDuration = h('input#cp-admin-duration-input', { type: 'number', min: 0, value: duration });
         var set = h('button.btn.btn-primary', Messages.admin_setDuration);
+
+        var label = h('label', { for: 'cp-admin-duration-input' }, Messages.ui_ms);
+
         $div.append(h('div', [
-            h('span.cp-admin-bytes-written-duration', Messages.ui_ms),
             h('div.cp-admin-setlimit-form', [
+                label,
                 newDuration,
                 h('nav', [set])
             ])
         ]));
 
+
         UI.confirmButton(set, {
             classes: 'btn-primary',
             multiple: true,

www/calendar/app-calendar.less

@@ -30,6 +30,17 @@
             display: none !important;
         }
 
+
+        .tui-full-calendar-confirm {
+            span, i {
+                padding: 0.5rem;
+            }
+            span {
+                padding-left: 0;
+            }
+            width: unset;
+        }
+
         .tui-full-calendar-layout {
             background-color: @cp_sidebar-right-bg !important;
             color: @cryptpad_text_col;
@@ -113,7 +124,10 @@
                 justify-content: center !important;
                 align-items: center !important;
                 .tui-full-calendar-popup {
-                    width: 540px !important;
+                    width: 540px;
+                    max-width: 100%;
+                    max-height: 100%;
+                    overflow-y: auto;
                 }
             }
             #tui-full-calendar-popup-arrow {
@@ -138,6 +152,7 @@
             border-radius: @variables_radius_L;
         }
         .tui-full-calendar-popup-container {
+            min-width: 100%;
             background: @cp_flatpickr-bg;
             color: @cryptpad_text_col;
             border-radius: @variables_radius_L;
@@ -186,6 +201,7 @@
                     overflow: hidden;
                     text-overflow: ellipsis;
                     font: @colortheme_app-font;
+                    padding: 0 10px;
                 }
                 input { flex: 1; }
             }
@@ -288,6 +304,7 @@
                 display: flex;
             }
             .cp-calendar-notif-list-container {
+                width: 100%;
                 margin-bottom: 10px;
                 .cp-notif-label {
                     margin-top: 0.5em;
@@ -296,25 +313,37 @@
             .cp-calendar-notif-list {
                 display: flex;
                 flex-flow: column;
+                flex: 1;
+                min-width: 150px;
+                &:empty {
+                    display: none;
+                }
                 .cp-notif-entry {
+                    display: flex;
                     margin-bottom: 2px;
                     border-radius: @variables_radius;
                     background-color: fade(@cryptpad_text_col, 10%);
                     padding: 0.25rem;
                     .cp-notif-value {
-                        width: 170px;
+                        width: 75%;
                         display: inline-flex;
                         line-height: 30px;
                         vertical-align: middle;
+                        flex-grow: 1;
                         .cp-before {
                             flex: 1;
                             min-width: 0;
                         }
                     }
                     span:not(:last-child) {
-                        margin: 0px 5px;
+                        margin: 0;
+                        margin-right: 0.3rem;
+                    }
+                    span:first-child {
+                        margin-left: 0.3rem;
                     }
                     .btn-danger-outline {
+                        display: block;
                         margin-right: 0px !important;
                         background-color: transparent;
                         color: @cryptpad_text_col;
@@ -340,10 +369,20 @@
             }
             .cp-calendar-notif-form {
                 align-items: center;
+                .cp-calendar-notif-form-buttons {
+                    display: inline-flex;
+                    align-items: center;
+                    & > input {
+                        height: 40px;
+                    }
+                }
                 // margin-bottom: 20px;
                 input {
                     width: 80px;
-                    margin-right: 5px;
+                    margin-right: 0.3rem;
+                }
+                .fa-plus{
+                    margin-left:0.3rem;
                 }
             }
         }
@@ -628,6 +667,10 @@
     }
 
     @media (max-width: @browser_media-medium-screen) {
+        .cp-calendar-notif-list-container, .cp-calendar-notif-form {
+            flex-direction: column;
+            align-items: unset !important;
+        }
         .cp-calendar-newevent {
             i {
                 margin: 0 !important;
@@ -664,5 +707,6 @@
             }
         }
     }
+
 }
 

www/calendar/inner.js

@@ -718,8 +718,9 @@ define([
             text: '',
             options: options, // Entries displayed in the menu
             common: common,
-            buttonCls: 'btn btn-default fa fa-gear small cp-calendar-actions'
+            buttonCls: 'btn btn-default fa fa-gear small cp-calendar-actions',
-        };
+            ariaLabel: Messages.calendar_settings,
+    };
         return UIElements.createDropdown(dropdownConfig)[0];
     };
     var makeCalendarEntry = function (id, teamId) {
@@ -827,8 +828,9 @@ define([
             if (myCalendars.length) {
                 var user = metadataMgr.getUserData();
                 var avatar = h('span.cp-avatar');
+                var uid = user.uid;
                 var name = user.name || Messages.anonymous;
-                common.displayAvatar($(avatar), user.avatar, name);
+                common.displayAvatar($(avatar), user.avatar, name, function(){}, uid);
                 APP.$calendars.append(h('div.cp-calendar-team', [
                     avatar,
                     h('span.cp-name', {title: name}, name)
@@ -1192,10 +1194,10 @@ ICS ==> create a new event with the same UID and a RECURRENCE-ID field (with a v
         }).appendTo(APP.toolbar.$bottomL);
 
         // Change page
-        var goLeft = h('button.fa.fa-chevron-left');
+        var goLeft = h('button.fa.fa-chevron-left',{'aria-label': Messages.goLeft});
-        var goRight = h('button.fa.fa-chevron-right');
+        var goRight = h('button.fa.fa-chevron-right', {'aria-label': Messages.goRight});
         var goToday = h('button', Messages.calendar_today);
-        var goDate = h('button.fa.fa-calendar');
+        var goDate = h('button.fa.fa-calendar',{'aria-label': Messages.date});
         $(goLeft).click(function () {
             cal.prev();
             updateDateRange();
@@ -1920,7 +1922,9 @@ APP.recurrenceRule = {
         };
         $(addNotif).click(function () {
             var unit = $block.getValue();
-            var value = $number.val();
+            var val = $number.val();
+            if (val === "") { return void UI.warn(Messages.error); }
+            var value = Number(val);
             addNotification(unit, value);
         });
         oldReminders.forEach(function (minutes) {
@@ -1931,9 +1935,11 @@ APP.recurrenceRule = {
             listContainer,
             h('div.cp-calendar-notif-form', [
                 h('span.cp-notif-label', Messages.calendar_addNotification),
-                number,
+                h('span.cp-calendar-notif-form-buttons', [
-                $block[0],
+                    number,
-                addNotif
+                    $block[0],
+                    addNotif
+                ])
             ])
         ]);
     };

www/checkup/main.js

@@ -320,7 +320,7 @@ define([
         var blockUrl = Login.Block.getBlockUrl(opt.blockKeys);
         console.warn('Testing block URL (%s). One 404 is normal.', blockUrl);
 
-        var userHash = '/2/drive/edit/000000000000000000000000';
+        var userHash = Hash.createRandomHash('drive');
         var secret = Hash.getSecrets('drive', userHash);
         opt.keys = secret.keys;
         opt.channelHex = secret.channel;

www/common/application_config_internal.js

@@ -113,6 +113,12 @@ define(function() {
      */
     AppConfig.roadmap = false;
 
+    /* If you have a status page for your instance, you may use the setting belox
+     *
+     * See the comments above for a description of possible configurations.
+     */
+    AppConfig.status = false;
+
     /* By default CryptPad instances display some text on the home page indicating that
      * they are an independent community instance of the software. You can provide customized messages
      * by filling in the following data structure with strings for each language you intend to support.

www/common/common-interface.js

@@ -536,7 +536,8 @@ define([
             message = dialog.message(msg);
         }
 
-        var frame = h('div', [
+        var cls = opt.scrollable ? '.cp-alertify-scrollable' : '';
+        var frame = h('div'+cls, [
             message,
             dialog.getButtons(opt.buttons, opt.onClose)
         ]);
@@ -1042,12 +1043,17 @@ define([
             window.parent.location = href;
         });
         if (exitable) {
+            // XXX if true or function, ALSO add a button to leave
             $(window).focus();
             $(window).keydown(function (e) { // XXX what if they don't have a keyboard?
                 if (e.which === 27) {
+                    e.preventDefault();
+                    e.stopPropagation();
+                    // Function: call the function (should be a redirect)
+                    if (typeof(exitable) === "function") { return void exitable(); }
+                    // Otherwise remove the loading screen
                     $loading.hide();
                     $('html').toggleClass('cp-loading-noscroll', false);
-                    if (typeof(exitable) === "function") { exitable(); }
                 }
             });
         }
@@ -1544,5 +1550,49 @@ define([
 
     };
 
+    UI.getDestroyedPlaceholderMessage = (code, isAccount, isTemplate) => {
+        var account = {
+            ARCHIVE_OWNED: Messages.dph_account_destroyed,
+            INACTIVE: Messages.dph_account_inactive,
+            MODERATION_ACCOUNT: Messages.dph_account_moderated,
+            MODERATION_BLOCK: Messages.dph_account_moderated,
+            PASSWORD_CHANGE: Messages.dph_account_pw,
+        };
+        var template = {
+            ARCHIVE_OWNED: Messages.dph_tmp_destroyed,
+            MODERATION_PAD: Messages.dph_tmp_moderated,
+            MODERATION_ACCOUNT: Messages.dph_tmp_moderated_account,
+            PASSWORD_CHANGE: Messages.dph_tmp_pw
+        };
+        var pad = {
+            ARCHIVE_OWNED: Messages.dph_pad_destroyed,
+            INACTIVE: Messages.dph_pad_inactive,
+            MODERATION_PAD: Messages.dph_pad_moderated,
+            MODERATION_DESTROY: Messages.dph_pad_moderated,
+            MODERATION_ACCOUNT: Messages.dph_pad_moderated_account,
+            PASSWORD_CHANGE: Messages.dph_pad_pw
+        };
+        var msg = pad[code];
+        if (isAccount) {
+            msg = account[code];
+        } else if (isTemplate) {
+            msg = template[code];
+        }
+        if (!msg) { msg = Messages.dph_default; }
+        return msg;
+    };
+    UI.getDestroyedPlaceholder = function (reason, isAccount) {
+        if (typeof(reason) !== "string") { return; }
+        var split = reason.split(':');
+        var code = split[0]; // Generated code
+        var input = split[1]; // User/admin manual input
+        var text = UI.getDestroyedPlaceholderMessage(code, isAccount);
+        var reasonBlock = input ? h('p', Messages._getKey('dph_reason', [input])) : undefined;
+        return h('div', [
+            h('p', text),
+            reasonBlock
+        ]);
+    };
+
     return UI;
 });

www/common/common-thumbnail.js

@@ -19,6 +19,7 @@ define([
         'image/png',
         'image/jpeg',
         'image/jpg',
+        'image/webp',
         'image/gif',
         'video/',
         'application/pdf'
@@ -239,7 +240,7 @@ define([
             });
         };
         if (window.html2canvas) { return void todo(); }
-        require(['/components/html2canvas/build/html2canvas.min.js'], todo);
+        require(['/components/html2canvas/dist/html2canvas.min.js'], todo);
     };
 
     Thumb.initPadThumbnails = function (common, opts) {
@@ -316,7 +317,7 @@ define([
             var key = secret.keys && secret.keys.cryptKey;
             MediaTag.fetchDecryptedMetadata(src, key, function (e, metadata) {
                 if (e) {
-                    if (e === 'XHR_ERROR') { return; }
+                    if (/^XHR_ERROR/.test(e)) { return; }
                     return console.error(e);
                 }
                 if (!metadata) { return console.error("NO_METADATA"); }

www/common/common-ui-elements.js

@@ -827,6 +827,7 @@ define([
                 }
                 button = $('<button>', {
                     title: Messages.historyButton,
+                    'aria-label': Messages.historyButton,
                     'class': "fa fa-history cp-toolbar-icon-history",
                 }).append($('<span>', {'class': 'cp-toolbar-drawer-element'}).text(Messages.historyText));
                 if (data.histConfig) {
@@ -1151,11 +1152,13 @@ define([
             editor.focus();
         };
         for (var k in actions) {
-            $('<button>', {
+            let $b = $('<button>', {
                 'data-type': k,
                 'class': 'pure-button fa ' + actions[k].icon,
                 title: Messages['mdToolbar_' + k] || k
-            }).click(onClick).appendTo($toolbar);
+            }).click(onClick);
+            if (k === "embed") { $toolbar.prepend($b); }
+            else { $toolbar.append($b); }
         }
         $('<button>', {
             'class': 'pure-button fa fa-question cp-markdown-help',
@@ -1488,15 +1491,18 @@ define([
         if (config.buttonContent) {
             $button = $(h('button', {
                 class: config.buttonCls || '',
+                'aria-label': config.ariaLabel || '',
             }, [
                 h('span.cp-dropdown-button-title', config.buttonContent),
             ]));
         } else {
             $button = $('<button>', {
-                'class': config.buttonCls || ''
+                'class': config.buttonCls || '',
+                'aria-label': config.ariaLabel || '',
             }).append($('<span>', {'class': 'cp-dropdown-button-title'}).text(config.text || ""));
         }
 
+
         if (config.caretDown) {
             $('<span>', {
                 'class': 'fa fa-caret-down',
@@ -1737,10 +1743,11 @@ define([
         var sourceLine = template(Messages.info_sourceFlavour, Pages.sourceLink);
 
         var content = h('div.cp-info-menu-container', [
-            h('div.logo-block', [
+                h('div.logo-block', [
-                h('img', {
+                    h('img', {
-                    src: '/customize/CryptPad_logo.svg?' + urlArgs
+                        src: '/customize/CryptPad_logo.svg?' + urlArgs,
-                }),
+                        alt: Messages.label_logo
+                    }),
                 h('h6', "CryptPad"),
                 h('span', Pages.versionString)
             ]),
@@ -1765,7 +1772,7 @@ define([
             },
         ];
 
-        var modal = UI.dialog.customModal(content, {buttons: buttons });
+        var modal = UI.dialog.customModal(content, {scrollable: true, buttons: buttons });
         UI.openCustomModal(modal);
     };
 
@@ -2858,8 +2865,10 @@ define([
     UIElements.onServerError = function (common, err, toolbar, cb) {
         //if (["EDELETED", "EEXPIRED", "ERESTRICTED"].indexOf(err.type) === -1) { return; }
         var priv = common.getMetadataMgr().getPrivateData();
+        var viewer = priv.readOnly || err.viewer;
         var sframeChan = common.getSframeChannel();
         var msg = err.type;
+        var exitable = Boolean(err.loaded);
         if (err.type === 'EEXPIRED') {
             msg = Messages.expiredError;
             if (err.loaded) {
@@ -2876,6 +2885,23 @@ define([
                 delete autoStoreModal[priv.channel];
             }
 
+            if (err.message && err.drive) {
+                let msg = UI.getDestroyedPlaceholder(err.message, true);
+                return UI.errorLoadingScreen(msg, false, () => {
+                    // When closing error screen
+                    if (err.message === 'PASSWORD_CHANGE') {
+                        return common.setLoginRedirect('login');
+                    }
+                    return common.setLoginRedirect('');
+                });
+            }
+            if (err.message && (err.message !== "PASSWORD_CHANGE" || viewer)) {
+                // XXX If readonly, tell the viewer that their link won't work with the new password
+                UI.errorLoadingScreen(UI.getDestroyedPlaceholder(err.message, false),
+                    exitable, exitable);
+                return;
+            }
+
             if (err.ownDeletion) {
                 if (toolbar && typeof toolbar.deleted === "function") { toolbar.deleted(); }
                 (cb || function () {})();
@@ -2884,7 +2910,7 @@ define([
 
             // View users have the wrong seed, thay can't retireve access directly
             // Version 1 hashes don't support passwords
-            if (!priv.readOnly && !priv.oldVersionHash) {
+            if (!viewer && !priv.oldVersionHash) {
                 sframeChan.event('EV_SHARE_OPEN', {hidden: true}); // Close share modal
                 UIElements.displayPasswordPrompt(common, {
                     fromServerError: true,
@@ -2919,9 +2945,17 @@ define([
 
     UIElements.displayPasswordPrompt = function (common, cfg, isError) {
         var error;
-        if (isError) { error = setHTML(h('p.cp-password-error'), Messages.password_error); }
+        if (isError) {
+            let msg = isError === 'PASSWORD_CHANGE' ? Messages.drive_sfPasswordError : Messages.password_error;
+            error = setHTML(h('p.cp-password-error'), msg);
+        }
 
-        var info = h('p.cp-password-info', Messages.password_info);
+        var pwMsg = UI.getDestroyedPlaceholderMessage('PASSWORD_CHANGE', false);
+        if (cfg.legacy) {
+            // Legacy mode: we don't know if the pad has been destroyed or its password has changed
+            pwMsg = Messages.password_info;
+        }
+        var info = h('p.cp-password-info', pwMsg);
         var info_loaded = setHTML(h('p.cp-password-info'), Messages.errorCopy);
 
         var password = UI.passwordInput({placeholder: Messages.password_placeholder});
@@ -2958,8 +2992,16 @@ define([
                 });
             }
             common.getSframeChannel().query('Q_PAD_PASSWORD_VALUE', value, function (err, data) {
-                if (!data) {
+                data = data || {};
-                    return void UIElements.displayPasswordPrompt(common, cfg, true);
+                if (!data.state && data.view && data.reason === "PASSWORD_CHANGE") {
+                    return UIElements.onServerError(common, {
+                        type: 'EDELETED',
+                        message: data.reason,
+                        viewer: data.view
+                    });
+                }
+                if (!data.state) {
+                    return void UIElements.displayPasswordPrompt(common, cfg, (data && data.reason) || 1);
                 }
             });
         };

www/common/common-util.js

@@ -334,11 +334,11 @@
                 // this is resulting in some code duplication
                 return void CB(void 0, response);
             }
-            if (response.status === 401) {
+            if (response.status === 401 || response.status === 404) {
                 response.json().then((data) => {
-                    CB(401, data);
+                    CB(response.status, data);
                 }).catch(() => {
-                    CB(401);
+                    CB(response.status);
                 });
 
                 return;

www/common/cryptget.js

@@ -193,6 +193,11 @@ define([
                 finish(Session, void 0, doc);
             });
         };
+
+        config.onChannelError = function (info) {
+            finish(Session, info);
+        };
+
         overwrite(config, opt);
 
         start(Session, config);

www/common/cryptpad-common.js

@@ -476,6 +476,7 @@ define([
     common.drive.onLog = Util.mkEvent();
     common.drive.onChange = Util.mkEvent();
     common.drive.onRemove = Util.mkEvent();
+    common.drive.onDeleted = Util.mkEvent();
     // Profile
     common.getProfileEditUrl = function (cb) {
         postMessage("GET", { key: ['profile', 'edit'] }, function (obj) {
@@ -649,15 +650,23 @@ define([
 
         }).nThen(function (waitFor) {
             // If it's not in the cache or it's not a blob, try to get the value from the server
-            postMessage("GET_FILE_SIZE", {channel:channel}, waitFor(function (obj) {
+            var getSize = () => {
-                if (obj && obj.error) {
+                postMessage("GET_FILE_SIZE", {channel:channel}, waitFor(function (obj) {
-                    // If disconnected, try to get the value from the channel cache (next nThen)
+                    if (obj && obj.error === "ANON_RPC_NOT_READY") { return void setTimeout(waitFor(getSize), 100); }
-                    error = obj.error;
+
-                    return;
+                    if (obj && obj.error && obj.error.code === 'ENOENT' && obj.error.reason) {
-                }
+                        waitFor.abort();
-                waitFor.abort();
+                        cb(obj.error.reason);
-                cb(undefined, obj.size);
+                    } else if (obj && obj.error) {
-            }));
+                        // If disconnected, try to get the value from the channel cache (next nThen)
+                        error = obj.error;
+                        return;
+                    }
+                    waitFor.abort();
+                    cb(undefined, obj.size);
+                }));
+            };
+            getSize();
         }).nThen(function () {
             Cache.getChannelCache(channel, function(err, data) {
                 if (err) { return void cb(error); }
@@ -681,7 +690,7 @@ define([
             var error = obj && obj.error;
             if (error) { return void cb(error); }
             if (!obj) { return void cb('ERROR'); }
-            cb (null, obj.isNew);
+            cb (null, obj.isNew, obj.reason);
         }, {timeout: -1});
     };
     // This function is used when we want to open a pad. We first need
@@ -711,7 +720,7 @@ define([
                     } else if (error) {
                         return void cb(error);
                     }
-                    cb(undefined, obj.isNew);
+                    cb(undefined, obj.isNew, obj.reason);
                 }, {timeout: -1});
             };
             isNew();
@@ -952,9 +961,9 @@ define([
                 optsPut.accessKeys = keys;
             }));
         }).nThen(function () {
-            Crypt.get(parsed.hash, function (err, val) {
+            Crypt.get(parsed.hash, function (err, val, errData) {
                 if (err) {
-                    return void cb(err);
+                    return void cb(err, errData);
                 }
                 if (!val) {
                     return void cb('ENOENT');
@@ -996,10 +1005,10 @@ define([
                         optsGet.accessKeys = keys;
                     }));
                 }).nThen(function () {
-                    Crypt.get(parsed.hash, function (err, _val) {
+                    Crypt.get(parsed.hash, function (err, _val, errData) {
                         if (err) {
                             _waitFor.abort();
-                            return void cb(err);
+                            return void cb(err, errData);
                         }
                         try {
                             val = JSON.parse(_val);
@@ -1443,8 +1452,10 @@ define([
         }).nThen(function (waitFor) {
             optsPut.metadata.restricted = oldMetadata.restricted;
             optsPut.metadata.allowed = oldMetadata.allowed;
+            if (!newPassword) { optsPut.metadata.forcePlaceholder = true; }
             Crypt.put(newHash, cryptgetVal, waitFor(function (err) {
                 if (err) {
+                    if (err === "EDELETED") { err = "PASSWORD_ALREADY_USED"; }
                     waitFor.abort();
                     return void cb({ error: err });
                 }
@@ -1484,7 +1495,8 @@ define([
             // delete the old pad
             common.removeOwnedChannel({
                 channel: oldChannel,
-                teamId: teamId
+                teamId: teamId,
+                reason: 'PASSWORD_CHANGE',
             }, waitFor(function (obj) {
                 if (obj && obj.error) {
                     waitFor.abort();
@@ -1620,7 +1632,8 @@ define([
             // delete the old pad
             common.removeOwnedChannel({
                 channel: oldChannel,
-                teamId: teamId
+                teamId: teamId,
+                reason: 'PASSWORD_CHANGE'
             }, waitFor(function (obj) {
                 if (obj && obj.error) {
                     waitFor.abort();
@@ -1777,8 +1790,8 @@ define([
             var newCrypto = Crypto.createEncryptor(newSecret.keys);
             var oldCrypto = Crypto.createEncryptor(oldSecret.keys);
             var cps = Util.find(cryptgetVal, ['content', 'hashes']);
-            var l = Object.keys(cps).length;
+            var cpLength = Object.keys(cps).length;
-            var lastCp = l ? cps[l] : {};
+            var lastCp = cpLength ? cps[cpLength] : {};
             cryptgetVal.content.hashes = {};
             common.getHistory({
                 channel: oldRtChannel,
@@ -1801,7 +1814,7 @@ define([
                     }
                 });
                 // Update last knwon hash in cryptgetVal
-                if (lastCp) {
+                if (cpLength && newHistory.length) {
                     lastCp.hash = newHistory[0].slice(0, 64);
                     lastCp.index = 50;
                     cryptgetVal.content.hashes[1] =  lastCp;
@@ -1826,6 +1839,7 @@ define([
             // The new rt channel is ready
             // The blob uses its own encryption and doesn't need to be reencrypted
             cryptgetVal.content.channel = newRtChannel;
+            if (!newPassword) { optsPut.metadata.forcePlaceholder = true; }
             Crypt.put(newHash, JSON.stringify(cryptgetVal), waitFor(function (err) {
                 if (err) {
                     waitFor.abort();
@@ -1941,8 +1955,9 @@ define([
             console.log("checking if old drive is owned");
             common.anonRpcMsg('GET_METADATA', secret.channel, waitFor(function (err, obj) {
                 if (err || obj.error) { return; }
-                if (obj.owners && Array.isArray(obj.owners) &&
+                var md = obj[0];
-                    obj.owners.indexOf(edPublic) !== -1) {
+                if (md && md.owners && Array.isArray(md.owners) &&
+                    md.owners.indexOf(edPublic) !== -1) {
                     oldIsOwned = true;
                 }
             }));
@@ -1957,6 +1972,42 @@ define([
                     return void cb({ error: 'INVALID_CODE' });
                 }
             }));
+        }).nThen(function (waitFor) {
+            var blockUrl = Block.getBlockUrl(blockKeys);
+            // Check whether there is a block at that new location
+            Util.getBlock(blockUrl, {}, waitFor(function (err, response) {
+                // If there is no block or the block is invalid, continue.
+                // error 401 means protected block
+
+                /*
+                // the following block prevent users from re-using an old password
+                if (err === 404 && response && response.reason) {
+                    waitFor.abort();
+                    return void cb({
+                        error: 'EDELELED',
+                        reason: response.reason
+                    });
+                }
+                */
+
+                if (err && err !== 401) {
+                    console.log("no block found");
+                    return;
+                }
+
+                response.arrayBuffer().then(waitFor(arraybuffer => {
+                    var block = new Uint8Array(arraybuffer);
+                    var decryptedBlock = Block.decrypt(block, blockKeys);
+                    if (!decryptedBlock) {
+                        console.error("Found a login block but failed to decrypt");
+                        return;
+                    }
+
+                    // If there is already a valid block, abort! We risk overriding another user's data
+                    waitFor.abort();
+                    cb({ error: 'EEXISTS' });
+                }));
+            }));
         }).nThen(function (waitFor) {
             // Create a new user hash
             // Get the current content, store it in the new user file
@@ -1985,27 +2036,6 @@ define([
                     }
                 }), optsPut);
             }));
-        }).nThen(function (waitFor) {
-            var blockUrl = Block.getBlockUrl(blockKeys);
-            // Check whether there is a block at that new location
-            Util.fetch(blockUrl, waitFor(function (err, block) {
-                // If there is no block or the block is invalid, continue.
-                // error 401 means protected block
-                if (err && err !== 401) {
-                    console.log("no block found");
-                    return;
-                }
-
-                var decryptedBlock = Block.decrypt(block, blockKeys);
-                if (!decryptedBlock) {
-                    console.error("Found a login block but failed to decrypt");
-                    return;
-                }
-
-                // If there is already a valid block, abort! We risk overriding another user's data
-                waitFor.abort();
-                cb({ error: 'EEXISTS' });
-            }));
         }).nThen(function (waitFor) {
             // Write the new login block
             var content = {
@@ -2054,10 +2084,12 @@ define([
             if (!blockHash) { return; }
             console.log('removing old login block');
             Block.removeLoginBlock({
+                reason: 'PASSWORD_CHANGE',
                 auth: auth,
                 blockKeys: oldBlockKeys,
             }, waitFor(function (err) {
                 if (err) { return void console.error(err); }
+                common.passwordUpdated = true;
             }));
         }).nThen(function (waitFor) {
             if (!oldIsOwned) { return; }
@@ -2065,16 +2097,15 @@ define([
             common.removeOwnedChannel({
                 channel: secret.channel,
                 teamId: null,
-                force: true
+                force: true,
+                reason: 'PASSWORD_CHANGE'
             }, waitFor(function (obj) {
                 if (obj && obj.error) {
                     // Deal with it as if it was not owned
                     oldIsOwned = false;
                     return;
                 }
-                common.logoutFromAll(waitFor(function () {
+                common.stopWorker();
-                    common.stopWorker();
-                }));
             }));
         }).nThen(function (waitFor) {
             if (oldIsOwned) { return; }
@@ -2087,9 +2118,7 @@ define([
                 if (obj && obj.error) {
                     console.error(obj.error);
                 }
-                common.logoutFromAll(waitFor(function () {
+                common.stopWorker();
-                    common.stopWorker();
-                }));
             }));
         }).nThen(function () {
             // We have the new drive, with the new login block
@@ -2266,6 +2295,14 @@ define([
         cb();
     };
 
+    common.storeLogout = function (data) {
+        if (common.passwordUpdated) { return; }
+        LocalStore.logout(function () {
+            common.stopWorker();
+            common.drive.onDeleted.fire(data.reason);
+        });
+    };
+
     var lastPing = +new Date();
     var onPing = function (data, cb) {
         lastPing = +new Date();
@@ -2342,8 +2379,10 @@ define([
         DRIVE_LOG: common.drive.onLog.fire,
         DRIVE_CHANGE: common.drive.onChange.fire,
         DRIVE_REMOVE: common.drive.onRemove.fire,
+        DRIVE_DELETED: common.drive.onDeleted.fire,
         // Account deletion
         DELETE_ACCOUNT: common.startAccountDeletion,
+        LOGOUT: common.storeLogout,
         // Loading
         LOADING_DRIVE: common.loading.onDriveEvent.fire,
         // AutoStore
@@ -2459,6 +2498,14 @@ define([
                         });
                     }
 
+                    if (err === 404) {
+                        // Not found: account deleted
+                        waitFor.abort();
+                        return LocalStore.logout(function () {
+                            f(response || err);
+                        });
+                    }
+
                     if (err) {
                         // TODO
                         // it seems wrong that errors here aren't reported or handled
@@ -2496,6 +2543,15 @@ define([
                 }));
             }
         }).nThen(function (waitFor) {
+            var blockHash = LocalStore.getBlockHash();
+            var blockId = '';
+            try {
+                var blockPath = (new URL(blockHash)).pathname;
+                var blockSplit = blockPath.split('/');
+                if (blockSplit[1] === 'block') {
+                    blockId = blockSplit[3];
+                }
+            } catch (e) { }
             var cfg = {
                 init: true,
                 userHash: userHash || LocalStore.getUserHash(),
@@ -2508,9 +2564,10 @@ define([
                 neverDrive: rdyCfg.neverDrive,
                 disableCache: localStorage['CRYPTPAD_STORE|disableCache'],
                 driveEvents: !rdyCfg.noDrive, //rdyCfg.driveEvents // Boolean
-                lastVisit: Number(localStorage.lastVisit) || undefined
+                lastVisit: Number(localStorage.lastVisit) || undefined,
+                blockId: blockId
             };
-            common.userHash = userHash;
+            common.userHash = userHash || LocalStore.getUserHash();
 
             // FIXME Backward compatibility
             if (sessionStorage.newPadFileData) {
@@ -2786,6 +2843,9 @@ define([
                     LocalStore.loginReload();
                 } else if (o && !n) {
                     LocalStore.logout();
+                } else if (o && n && o !== n) {
+                    common.passwordUpdated = true;
+                    window.location.reload();
                 }
             });
             LocalStore.onLogout(function () {

www/common/diffMarked.js

@@ -31,9 +31,15 @@ define([
         "g.grid g.tick line { opacity: 0.25; }" +
         "g.today line { stroke: red; stroke-width: 1; stroke-dasharray: 3; opacity: 0.5; }";
 
+    var onMermaidRunEvt = Util.mkEvent(true);
+    var onMermaidRun = () => {
+        onMermaidRunEvt.fire();
+        onMermaidRunEvt = Util.mkEvent(true);
+    };
     var Mermaid = {
         __stubbed: true,
-        init: function () {
+        run: function (cb) {
+            onMermaidRunEvt.reg(cb);
             require([
                 'mermaid',
                 '/lib/mermaid/mermaid-zenuml.esm.min.js',
@@ -47,6 +53,18 @@ define([
                         theme: (window.CryptPad_theme === 'dark') ? 'dark' : 'default',
                         "themeCSS": mermaidThemeCSS,
                     });
+                    onMermaidRun();
+
+                    var run = Mermaid.run;
+                    var to;
+                    Mermaid.run = (cb) => {
+                        clearTimeout(to);
+                        onMermaidRunEvt.reg(cb);
+                        to = setTimeout(() => {
+                            onMermaidRun();
+                            run();
+                        });
+                    };
                 }
 
                 Mermaid.registerExternalDiagrams([zenuml]).then(() => pluginLoaded.fire());
@@ -504,12 +522,13 @@ define([
         name: 'mermaid',
         attr: 'mermaid-source',
         render: function ($el) {
-            Mermaid.init(undefined, $el);
+            Mermaid.run(() => {
-            // clickable elements in mermaid don't work well with our sandboxing setup
+                // clickable elements in mermaid don't work well with our sandboxing setup
-            // the function below strips clickable elements but still leaves behind some artifacts
+                // the function below strips clickable elements but still leaves behind some artifacts
-            // tippy tooltips might still be useful, so they're not removed. It would be
+                // tippy tooltips might still be useful, so they're not removed. It would be
-            // preferable to just support links, but this covers up a rough edge in the meantime
+                // preferable to just support links, but this covers up a rough edge in the meantime
-            removeMermaidClickables($el);
+                removeMermaidClickables($el);
+            });
         }
     };
 

www/common/drive-ui.js

@@ -1152,10 +1152,36 @@ define([
 
         var FILTER_BY = "filterBy";
 
+        var refreshDeprecated = function () {
+            if (!APP.passwordModal) { return; }
+            var deprecated = files.sharedFoldersTemp;
+            if (JSONSortify(deprecated) === APP.deprecatedSF) { return; }
+            APP.deprecatedSF = JSONSortify(deprecated);
+            if (typeof (deprecated) === "object" && Object.keys(deprecated).length) {
+                var nt = nThen;
+                Object.keys(deprecated).forEach(function (fId) {
+                    var data = deprecated[fId];
+                    var sfId = manager.user.userObject.getSFIdFromHref(data.href);
+                    if (folders[fId] || sfId) { // This shared folder is already stored in the drive...
+                        return void manager.delete([['sharedFoldersTemp', fId]], function () { });
+                    }
+                    nt = nt(function (waitFor) {
+                        UI.openCustomModal(APP.passwordModal(fId, data, waitFor()));
+                    }).nThen;
+                });
+                nt(function () {
+                    APP.refresh();
+                });
+            }
+
+        };
         var refresh = APP.refresh = function (cb) {
             var type = APP.store[FILTER_BY];
             var path = type ? [FILTER, type, currentPath] : currentPath;
-            APP.displayDirectory(path, undefined, cb);
+            APP.displayDirectory(path, undefined, () => {
+                refreshDeprecated();
+                if (typeof(cb) === "function") { cb(); }
+            });
         };
 
         // `app`: true (force open wiht the app), false (force open in preview),
@@ -1490,6 +1516,7 @@ define([
                 $li = findDataHolder($tree.find('.cp-app-drive-element-active'));
             }
             var $button = $driveToolbar.find('#cp-app-drive-toolbar-context-mobile');
+            $button.attr('aria-label', Messages.context_menu);
             if ($button.length) { // mobile
                 if ($li.length !== 1
                     || !$._data($li[0], 'events').contextmenu
@@ -2693,6 +2720,7 @@ define([
                 gridIcon,
                 listIcon
             ]));
+            $button.attr('aria-label', Messages.label_viewMode);
             var $gridIcon = $(gridIcon);
             var $listIcon = $(listIcon);
             var showMode = function (mode) {
@@ -3868,6 +3896,8 @@ define([
                 });
             }
 
+            var $fileHeader = getFileListHeader(false);
+            $list.append($fileHeader);
             $list.append(h('li.cp-app-drive-element-separator', h('span', Messages.drive_active1Day)));
             filesList.some(function (arr) {
                 var id = arr[0];
@@ -5370,11 +5400,13 @@ define([
             });
         }
         */
-        var nt = nThen;
+        APP.passwordModal = function (fId, data, cb) {
-        var passwordModal = function (fId, data, cb) {
             var content = [];
-            var folderName = '<b>'+ (data.lastTitle || Messages.fm_newFolder) +'</b>';
+
-            content.push(UI.setHTML(h('p'), Messages._getKey('drive_sfPassword', [folderName])));
+            var legacy = data.legacy; // Legacy mode: we don't know if the sf has been destroyed or its password has changed
+            var folderName = '<b>'+ (Util.fixHTML(data.lastTitle) || Messages.fm_newFolder) +'</b>';
+            var pwMsg = legacy ? Messages._getKey('drive_sfPassword', [folderName]) : Messages._getKey('dph_sf_pw', [folderName]);
+            content.push(UI.setHTML(h('p'), pwMsg));
             var newPassword = UI.passwordInput({
                 id: 'cp-app-prop-change-password',
                 placeholder: Messages.settings_changePasswordNew,
@@ -5425,24 +5457,7 @@ define([
                 onClose: cb
             });
         };
-        onConnectEvt.reg(function () {
+        onConnectEvt.reg(refreshDeprecated);
-            var deprecated = files.sharedFoldersTemp;
-            if (typeof (deprecated) === "object" && Object.keys(deprecated).length) {
-                Object.keys(deprecated).forEach(function (fId) {
-                    var data = deprecated[fId];
-                    var sfId = manager.user.userObject.getSFIdFromHref(data.href);
-                    if (folders[fId] || sfId) { // This shared folder is already stored in the drive...
-                        return void manager.delete([['sharedFoldersTemp', fId]], function () { });
-                    }
-                    nt = nt(function (waitFor) {
-                        UI.openCustomModal(passwordModal(fId, data, waitFor()));
-                    }).nThen;
-                });
-                nt(function () {
-                    refresh();
-                });
-            }
-        });
 
         return {
             refresh: refresh,

www/common/inner/access.js

@@ -752,7 +752,8 @@ define([
                 _owners[ed] = {
                     //selected: true,
                     name: user.name,
-                    avatar: user.avatar
+                    avatar: user.avatar,
+                    uid: user.uid
                 };
                 return;
             }
@@ -939,8 +940,12 @@ define([
                         }, function (err, data) {
                             $(passwordOk).text(Messages.properties_changePasswordButton);
                             pLocked = false;
-                            if (err || data.error) {
+                            err = err || data.error;
-                                console.error(err || data.error);
+                            if (err) {
+                                if (err === "PASSWORD_ALREADY_USED") {
+                                    return void UI.alert(Messages.access_passwordUsed);
+                                }
+                                console.error(err);
                                 return void UI.alert(Messages.properties_passwordError);
                             }
                             UI.findOKButton().click();
@@ -982,11 +987,17 @@ define([
 
                             if (data.warning) {
                                 return void UI.alert(Messages.properties_passwordWarning, function () {
+                                    if (isNotStored) {
+                                        return sframeChan.query('Q_PASSWORD_CHECK', newPass, () => { common.gotoURL(_href);  });
+                                    }
                                     common.gotoURL(_href);
                                 }, {force: true});
                             }
                             return void UI.alert(UIElements.fixInlineBRs(Messages.properties_passwordSuccess), function () {
                                 if (!isSharedFolder) {
+                                    if (isNotStored) {
+                                        return sframeChan.query('Q_PASSWORD_CHECK', newPass, () => { common.gotoURL(_href);  });
+                                    }
                                     common.gotoURL(_href);
                                 }
                             });

www/common/inner/common-mediatag.js

@@ -237,6 +237,7 @@ define([
             'image/png',
             'image/jpeg',
             'image/jpg',
+            'image/webp',
             'image/gif',
         ];
         var fmConfig = {
@@ -246,7 +247,7 @@ define([
             onUploaded: cb
         };
         var FM = common.createFileManager(fmConfig);
-        var accepted = ".gif,.jpg,.jpeg,.png";
+        var accepted = ".gif,.jpg,.jpeg,.png,.webp";
         var data = {
             FM: FM,
             filter: function (file) {

www/common/inner/share.js

@@ -615,7 +615,7 @@ define([
                         label: {style: "display: none;"}
                     }) : undefined;
         var rights = h('div.msg.cp-inline-radio-group', [
-            h('label', Messages.share_linkAccess),
+            h('label',{ for: 'cp-share-editable-true' }, Messages.share_linkAccess),
             h('div.radio-group',[
             UI.createRadio('accessRights', 'cp-share-editable-false',
                             labelView, true, { mark: {tabindex:1} }),

www/common/media-tag.js

@@ -47,6 +47,7 @@ var factory = function () {
             'text/plain',
             'image/png',
             'image/jpeg',
+            'image/webp',
             'image/jpg',
             'image/gif',
             'audio/mpeg',
@@ -234,6 +235,22 @@ var factory = function () {
         config.Cache.setBlobCache(id, u8, cb);
     };
 
+    var headRequest = function (src, cb) {
+        var xhr = new XMLHttpRequest();
+        xhr.open("HEAD", src);
+        if (sendCredentials) { xhr.withCredentials = true; }
+        xhr.onerror = function () { return void cb("XHR_ERROR"); };
+        xhr.onreadystatechange = function() {
+            if (this.readyState === this.DONE) {
+                cb(null, Number(xhr.getResponseHeader("Content-Length")));
+            }
+        };
+        xhr.onload = function () {
+            if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
+        };
+        xhr.send();
+
+    };
     var getFileSize = function (src, _cb) {
         var cb = function (e, res) {
             _cb(e, res);
@@ -243,25 +260,14 @@ var factory = function () {
         var cacheKey = getCacheKey(src);
 
         var check = function () {
-            var xhr = new XMLHttpRequest();
+            headRequest(src, cb);
-            xhr.open("HEAD", src);
-            if (sendCredentials) { xhr.withCredentials = true; }
-            xhr.onerror = function () { return void cb("XHR_ERROR"); };
-            xhr.onreadystatechange = function() {
-                if (this.readyState === this.DONE) {
-                    cb(null, Number(xhr.getResponseHeader("Content-Length")));
-                }
-            };
-            xhr.onload = function () {
-                if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
-            };
-            xhr.send();
         };
 
         if (!cacheKey) { return void check(); }
 
         getBlobCache(cacheKey, function (err, u8) {
-            if (err || !u8) { return void check(); }
+            check(); // send the HEAD request to update the blob activity
+            if (err || !u8) { return; }
             cb(null, 0);
         });
     };
@@ -748,7 +754,11 @@ var factory = function () {
             });
         };
 
-        if (cfg.force) { dl(); return mediaObject; }
+        if (cfg.force) {
+            headRequest(src, function () {}); // Update activity
+            dl();
+            return mediaObject;
+        }
 
         var maxSize = typeof(config.maxDownloadSize) === "number" ? config.maxDownloadSize
                                 : (5 * 1024 * 1024);

www/common/notifications.js

@@ -407,6 +407,25 @@ define([
         }
     };
 
+    handlers['SF_DELETED'] = function(common, data) {
+        var content = data.content;
+        var msg = content.msg;
+
+        // Display the notification
+        var title = Util.fixHTML(msg.content.title);
+        var teamName = Util.fixHTML(msg.content.teamName);
+
+        content.getFormatText = function() {
+            if (teamName) {
+                return Messages._getKey('dph_sf_destroyed_team', [title, teamName]);
+            }
+            return Messages._getKey('dph_sf_destroyed', [title]);
+        };
+        if (!content.archived) {
+            content.dismissHandler = defaultDismiss(common, data);
+        }
+    };
+
     handlers['MOVE_TODO'] = function(common, data) {
         var content = data.content;
         var msg = content.msg;

www/common/onlyoffice/inner.js

@@ -61,7 +61,7 @@ define([
     var CHECKPOINT_INTERVAL = 100;
     var FORCE_CHECKPOINT_INTERVAL = 10000;
     var DISPLAY_RESTORE_BUTTON = false;
-    var NEW_VERSION = 6; // version of the .bin, patches and ChainPad formats
+    var NEW_VERSION = 7; // version of the .bin, patches and ChainPad formats
     var PENDING_TIMEOUT = 30000;
     var CURRENT_VERSION = X2T.CURRENT_VERSION;
 
@@ -1353,7 +1353,7 @@ define([
             var oMemory = new w.AscCommon.CMemory();
             var aRes = [];
             patches.forEach(function (item) {
-                editor.GetSheet(0).worksheet.workbook._SerializeHistoryBase64(oMemory, item, aRes);
+                editor.GetSheet(0).worksheet.workbook._SerializeHistory(oMemory, item, aRes);
             });
 
             // Make the patch
@@ -1363,6 +1363,13 @@ define([
                 changesIndex: ooChannel.cpIndex || 0,
                 startSaveChanges: true,
                 endSaveChanges: true,
+                isExcel: true,
+                deleteIndex: null,
+                excelAdditionalInfo: null,
+                unlock: false,
+                releaseLocks: true,
+                reSave: true,
+                isCoAuthoring: true,
                 locks: getUserLock(getId(), true),
                 excelAdditionalInfo: null
             };
@@ -2135,9 +2142,9 @@ Uncaught TypeError: Cannot read property 'calculatedType' of null
                 return void e.asc_Print({});
             }
             x2tConvertData(data, filename, extension, function (xlsData) {
+                UI.removeModals();
                 if (xlsData) {
                     var blob = new Blob([xlsData], {type: "application/bin;charset=utf-8"});
-                    UI.removeModals();
                     saveAs(blob, finalFilename);
                     return;
                 }
@@ -2974,7 +2981,22 @@ Uncaught TypeError: Cannot read property 'calculatedType' of null
                     }
                     readOnly = true;
                 }
-
+            } else if (content && content.version <= 6) {
+                version = 'v6/';
+                APP.migrate = true;
+                // Registedred ~~users~~ editors can start the migration
+                if (common.isLoggedIn() && !readOnly) {
+                    content.migration = true;
+                    APP.onLocal();
+                } else {
+                    msg = h('div.alert.alert-warning.cp-burn-after-reading', Messages.oo_sheetMigration_anonymousEditor);
+                    if (APP.helpMenu) {
+                        $(APP.helpMenu.menu).after(msg);
+                    } else {
+                        $('#cp-app-oo-editor').prepend(msg);
+                    }
+                    readOnly = true;
+                }
             }
             // NOTE: don't forget to also update the version in 'EV_OOIFRAME_REFRESH'
 

www/common/onlyoffice/v6/build.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env sh
+
+set -euxo pipefail
+
+BRANCH=cp7.1.0.219
+
+WORK_DIR=`mktemp -d -t build-oo.XXXXX`
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+cd $WORK_DIR
+
+git clone --depth 1 --branch $BRANCH https://github.com/cryptpad/sdkjs
+git clone --depth 1 --branch $BRANCH https://github.com/cryptpad/web-apps
+
+cd sdkjs
+make
+
+cd $SCRIPT_DIR
+rm -rf sdkjs web-apps
+
+cp -r \
+  $WORK_DIR/sdkjs/deploy/sdkjs \
+  $WORK_DIR/web-apps/deploy/web-apps \
+  $SCRIPT_DIR

www/common/onlyoffice/v6/readme.md

@@ -0,0 +1,5 @@
+# OnlyOffice build
+
+This is the prebuild OnlyOffice client build from https://github.com/cryptpad/sdkjs/tree/cp7.1.0.219 and https://github.com/cryptpad/web-apps/tree/cp7.1.0.219 . OnlyOffice is licensed under the GNU Affero General Public License.
+
+Use `./build.sh`, if you want to build OnlyOffice your self.

www/common/onlyoffice/v7/build.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env sh
+
+set -euxo pipefail
+
+BRANCH=cp7.3.3.60
+
+WORK_DIR=`mktemp -d -t build-oo.XXXXX`
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+cd $WORK_DIR
+
+git clone --depth 1 --branch $BRANCH https://github.com/cryptpad/sdkjs
+git clone --depth 1 --branch $BRANCH https://github.com/cryptpad/web-apps
+
+cd sdkjs
+make
+
+cd $SCRIPT_DIR
+rm -rf sdkjs web-apps
+
+cp -r \
+  $WORK_DIR/sdkjs/deploy/sdkjs \
+  $WORK_DIR/sdkjs/deploy/web-apps \
+  $SCRIPT_DIR