react/scripts/babel/__tests__/transform-test-gate-pragma-...

214 lines
5.0 KiB
JavaScript
Raw Normal View History

Add pragma for feature testing: @gate (#18581) * Add pragma for feature testing: @gate The `@gate` pragma declares under which conditions a test is expected to pass. If the gate condition passes, then the test runs normally (same as if there were no pragma). If the conditional fails, then the test runs and is *expected to fail*. An alternative to `it.experimental` and similar proposals. Examples -------- Basic: ```js // @gate enableBlocksAPI test('passes only if Blocks API is available', () => {/*...*/}) ``` Negation: ```js // @gate !disableLegacyContext test('depends on a deprecated feature', () => {/*...*/}) ``` Multiple flags: ```js // @gate enableNewReconciler // @gate experimental test('needs both useEvent and Blocks', () => {/*...*/}) ``` Logical operators (yes, I'm sorry): ```js // @gate experimental && (enableNewReconciler || disableSchedulerTimeoutBasedOnReactExpirationTime) test('concurrent mode, doesn\'t work in old fork unless Scheduler timeout flag is disabled', () => {/*...*/}) ``` Strings, and comparion operators No use case yet but I figure eventually we'd use this to gate on different release channels: ```js // @gate channel === "experimental" || channel === "modern" test('works in OSS experimental or www modern', () => {/*...*/}) ``` How does it work? I'm guessing those last two examples might be controversial. Supporting those cases did require implementing a mini-parser. The output of the transform is very straightforward, though. Input: ```js // @gate a && (b || c) test('some test', () => {/*...*/}) ``` Output: ```js _test_gate(ctx => ctx.a && (ctx.b || ctx.c, 'some test'), () => {/*...*/}); ``` It also works with `it`, `it.only`, and `fit`. It leaves `it.skip` and `xit` alone because those tests are disabled anyway. `_test_gate` is a global method that I set up in our Jest config. It works about the same as the existing `it.experimental` helper. The context (`ctx`) argument is whatever we want it to be. I set it up so that it throws if you try to access a flag that doesn't exist. I also added some shortcuts for common gating conditions, like `old` and `new`: ```js // @gate experimental test('experimental feature', () => {/*...*/}) // @gate new test('only passes in new reconciler', () => {/*...*/}) ``` Why implement this as a pragma instead of a runtime API? - Doesn't require monkey patching built-in Jest methods. Instead it compiles to a runtime function that composes Jest's API. - Will be easy to upgrade if Jest ever overhauls their API or we switch to a different testing framework (unlikely but who knows). - It feels lightweight so hopefully people won't feel gross using it. For example, adding or removing a gate pragma will never affect the indentation of the test, unlike if you wrapped the test in a conditional block. * Compatibility with console error/warning tracking We patch console.error and console.warning to track unexpected calls in our tests. If there's an unexpected call, we usually throw inside an `afterEach` hook. However, that's too late for tests that we expect to fail, because our `_test_gate` runtime can't capture the error. So I also check for unexpected calls inside `_test_gate`. * Move test flags to dedicated file Added some instructions for how the flags are set up and how to use them. * Add dynamic version of gate API Receives same flags as the pragma. If we ever decide to revert the pragma, we can codemod them to use this instead.
2020-04-14 01:14:34 +08:00
/**
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
'use strict';
describe('transform-test-gate-pragma', () => {
// Fake runtime
// eslint-disable-next-line no-unused-vars
const _test_gate = (gateFn, testName, cb) => {
test(testName, (...args) => {
shouldPass = gateFn(context);
return cb(...args);
});
};
// eslint-disable-next-line no-unused-vars
const _test_gate_focus = (gateFn, testName, cb) => {
// NOTE: Tests in this file are not actually focused because the calls to
// `test.only` and `fit` are compiled to `_test_gate_focus`. So if you want
// to focus something, swap the following `test` call for `test.only`.
test(testName, (...args) => {
shouldPass = gateFn(context);
isFocused = true;
return cb(...args);
});
};
// Feature flags, environment variables, etc. We can configure this in
// our test set up.
const context = {
flagThatIsOff: false,
flagThatIsOn: true,
environment: 'fake-environment',
};
let shouldPass;
let isFocused;
beforeEach(() => {
shouldPass = null;
isFocused = false;
});
test('no pragma', () => {
expect(shouldPass).toBe(null);
});
// unrelated comment
test('no pragma, unrelated comment', () => {
expect(shouldPass).toBe(null);
});
// @gate flagThatIsOn
test('basic positive test', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOff
test('basic negative test', () => {
expect(shouldPass).toBe(false);
});
// @gate flagThatIsOn
it('it method', () => {
expect(shouldPass).toBe(true);
});
/* eslint-disable jest/no-focused-tests */
// @gate flagThatIsOn
test.only('test.only', () => {
expect(isFocused).toBe(true);
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOff
it.only('it.only', () => {
expect(isFocused).toBe(true);
expect(shouldPass).toBe(false);
});
// @gate flagThatIsOn
fit('fit', () => {
expect(isFocused).toBe(true);
expect(shouldPass).toBe(true);
});
/* eslint-enable jest/no-focused-tests */
// @gate !flagThatIsOff
test('flag negation', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOn
// @gate !flagThatIsOff
test('multiple gates', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOn
// @gate flagThatIsOff
test('multiple gates 2', () => {
expect(shouldPass).toBe(false);
});
// @gate !flagThatIsOff && flagThatIsOn
test('&&', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOff || flagThatIsOn
test('||', () => {
expect(shouldPass).toBe(true);
});
// @gate (flagThatIsOn || flagThatIsOff) && flagThatIsOn
test('groups', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOn == !flagThatIsOff
test('==', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOn === !flagThatIsOff
test('===', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOn != !flagThatIsOff
test('!=', () => {
expect(shouldPass).toBe(false);
});
// @gate flagThatIsOn != !flagThatIsOff
test('!==', () => {
expect(shouldPass).toBe(false);
});
// @gate flagThatIsOn === true
test('true', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOff === false
test('false', () => {
expect(shouldPass).toBe(true);
});
// @gate environment === "fake-environment"
test('double quoted strings', () => {
expect(shouldPass).toBe(true);
});
// @gate environment === 'fake-environment'
test('single quoted strings', () => {
expect(shouldPass).toBe(true);
});
// @gate flagThatIsOn // This is a comment
test('line comment', () => {
expect(shouldPass).toBe(true);
});
Add pragma for feature testing: @gate (#18581) * Add pragma for feature testing: @gate The `@gate` pragma declares under which conditions a test is expected to pass. If the gate condition passes, then the test runs normally (same as if there were no pragma). If the conditional fails, then the test runs and is *expected to fail*. An alternative to `it.experimental` and similar proposals. Examples -------- Basic: ```js // @gate enableBlocksAPI test('passes only if Blocks API is available', () => {/*...*/}) ``` Negation: ```js // @gate !disableLegacyContext test('depends on a deprecated feature', () => {/*...*/}) ``` Multiple flags: ```js // @gate enableNewReconciler // @gate experimental test('needs both useEvent and Blocks', () => {/*...*/}) ``` Logical operators (yes, I'm sorry): ```js // @gate experimental && (enableNewReconciler || disableSchedulerTimeoutBasedOnReactExpirationTime) test('concurrent mode, doesn\'t work in old fork unless Scheduler timeout flag is disabled', () => {/*...*/}) ``` Strings, and comparion operators No use case yet but I figure eventually we'd use this to gate on different release channels: ```js // @gate channel === "experimental" || channel === "modern" test('works in OSS experimental or www modern', () => {/*...*/}) ``` How does it work? I'm guessing those last two examples might be controversial. Supporting those cases did require implementing a mini-parser. The output of the transform is very straightforward, though. Input: ```js // @gate a && (b || c) test('some test', () => {/*...*/}) ``` Output: ```js _test_gate(ctx => ctx.a && (ctx.b || ctx.c, 'some test'), () => {/*...*/}); ``` It also works with `it`, `it.only`, and `fit`. It leaves `it.skip` and `xit` alone because those tests are disabled anyway. `_test_gate` is a global method that I set up in our Jest config. It works about the same as the existing `it.experimental` helper. The context (`ctx`) argument is whatever we want it to be. I set it up so that it throws if you try to access a flag that doesn't exist. I also added some shortcuts for common gating conditions, like `old` and `new`: ```js // @gate experimental test('experimental feature', () => {/*...*/}) // @gate new test('only passes in new reconciler', () => {/*...*/}) ``` Why implement this as a pragma instead of a runtime API? - Doesn't require monkey patching built-in Jest methods. Instead it compiles to a runtime function that composes Jest's API. - Will be easy to upgrade if Jest ever overhauls their API or we switch to a different testing framework (unlikely but who knows). - It feels lightweight so hopefully people won't feel gross using it. For example, adding or removing a gate pragma will never affect the indentation of the test, unlike if you wrapped the test in a conditional block. * Compatibility with console error/warning tracking We patch console.error and console.warning to track unexpected calls in our tests. If there's an unexpected call, we usually throw inside an `afterEach` hook. However, that's too late for tests that we expect to fail, because our `_test_gate` runtime can't capture the error. So I also check for unexpected calls inside `_test_gate`. * Move test flags to dedicated file Added some instructions for how the flags are set up and how to use them. * Add dynamic version of gate API Receives same flags as the pragma. If we ever decide to revert the pragma, we can codemod them to use this instead.
2020-04-14 01:14:34 +08:00
});
describe('transform test-gate-pragma: actual runtime', () => {
// These tests use the actual gating runtime used by the rest of our
// test suite.
// @gate __DEV__
test('__DEV__', () => {
if (!__DEV__) {
throw Error("Doesn't work in production!");
}
});
// @gate build === "development"
test('strings', () => {
if (!__DEV__) {
throw Error("Doesn't work in production!");
}
});
Add pragma for feature testing: @gate (#18581) * Add pragma for feature testing: @gate The `@gate` pragma declares under which conditions a test is expected to pass. If the gate condition passes, then the test runs normally (same as if there were no pragma). If the conditional fails, then the test runs and is *expected to fail*. An alternative to `it.experimental` and similar proposals. Examples -------- Basic: ```js // @gate enableBlocksAPI test('passes only if Blocks API is available', () => {/*...*/}) ``` Negation: ```js // @gate !disableLegacyContext test('depends on a deprecated feature', () => {/*...*/}) ``` Multiple flags: ```js // @gate enableNewReconciler // @gate experimental test('needs both useEvent and Blocks', () => {/*...*/}) ``` Logical operators (yes, I'm sorry): ```js // @gate experimental && (enableNewReconciler || disableSchedulerTimeoutBasedOnReactExpirationTime) test('concurrent mode, doesn\'t work in old fork unless Scheduler timeout flag is disabled', () => {/*...*/}) ``` Strings, and comparion operators No use case yet but I figure eventually we'd use this to gate on different release channels: ```js // @gate channel === "experimental" || channel === "modern" test('works in OSS experimental or www modern', () => {/*...*/}) ``` How does it work? I'm guessing those last two examples might be controversial. Supporting those cases did require implementing a mini-parser. The output of the transform is very straightforward, though. Input: ```js // @gate a && (b || c) test('some test', () => {/*...*/}) ``` Output: ```js _test_gate(ctx => ctx.a && (ctx.b || ctx.c, 'some test'), () => {/*...*/}); ``` It also works with `it`, `it.only`, and `fit`. It leaves `it.skip` and `xit` alone because those tests are disabled anyway. `_test_gate` is a global method that I set up in our Jest config. It works about the same as the existing `it.experimental` helper. The context (`ctx`) argument is whatever we want it to be. I set it up so that it throws if you try to access a flag that doesn't exist. I also added some shortcuts for common gating conditions, like `old` and `new`: ```js // @gate experimental test('experimental feature', () => {/*...*/}) // @gate new test('only passes in new reconciler', () => {/*...*/}) ``` Why implement this as a pragma instead of a runtime API? - Doesn't require monkey patching built-in Jest methods. Instead it compiles to a runtime function that composes Jest's API. - Will be easy to upgrade if Jest ever overhauls their API or we switch to a different testing framework (unlikely but who knows). - It feels lightweight so hopefully people won't feel gross using it. For example, adding or removing a gate pragma will never affect the indentation of the test, unlike if you wrapped the test in a conditional block. * Compatibility with console error/warning tracking We patch console.error and console.warning to track unexpected calls in our tests. If there's an unexpected call, we usually throw inside an `afterEach` hook. However, that's too late for tests that we expect to fail, because our `_test_gate` runtime can't capture the error. So I also check for unexpected calls inside `_test_gate`. * Move test flags to dedicated file Added some instructions for how the flags are set up and how to use them. * Add dynamic version of gate API Receives same flags as the pragma. If we ever decide to revert the pragma, we can codemod them to use this instead.
2020-04-14 01:14:34 +08:00
// Always should fail because of the unguarded console.error
// @gate false
test('works with console.error tracking', () => {
console.error('Should cause test to fail');
});
// Always should fail because of the unguarded console.warn
// @gate false
test('works with console.warn tracking', () => {
console.warn('Should cause test to fail');
});
// @gate false
test('works with console tracking if error is thrown before end of test', () => {
console.warn('Please stop that!');
console.error('Stop that!');
throw Error('I told you to stop!');
});
});
describe('dynamic gate method', () => {
// @gate experimental && __DEV__
test('returns same conditions as pragma', () => {
expect(gate(ctx => ctx.experimental && ctx.__DEV__)).toBe(true);
});
});