Commit Graph

8 Commits

Author SHA1 Message Date
Cody Cutrer f552f4fc00 simplify gem test harnesses
rely on test_all_gems.sh to output header and trailer,
and use `set -e` in each test.sh to simplify passing
through errors

Change-Id: I3ba724ad2539ddfe31195394c43f646acfc73920
Reviewed-on: https://gerrit.instructure.com/70469
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
2016-01-19 17:52:58 +00:00
Jon Jensen dfff1c06bd spec: ensure changes to test gems trigger a full run
Change-Id: Ic770eba54f21682479042f9a3d3155fa11f2885b
Reviewed-on: https://gerrit.instructure.com/70177
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
2016-01-13 18:59:28 +00:00
Jon Jensen 2b806932bc spec: don't improperly selinimize changes to the tinymce bundle
compiled/tinymce is special-cased in config/build.js.erb and friends; since
it's special/separate like the common bundle, we need to test all the
things when it changes

test plan:
1. edit a tinymce file, like public/javascripts/tinymce_plugins/instructure_equation/plugin.js
2. run `rake canvas:compile_assets`
3. run `gems/selinimum/bin/selinimize any list of spec files`
4. the last line of output should be that same list of spec files

Change-Id: I14d52003dd095769967053022bc2433999d23418
Reviewed-on: https://gerrit.instructure.com/69886
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
2016-01-07 21:39:48 +00:00
Shawn Meredith 384e22382c spec: selinimum yaml require
error seen in this build for docker
rspect-plugins-ruby21-docker/532/console

Change-Id: Ieaaaedf35925f68dccc17b28d259d85ddf963c6b
Reviewed-on: https://gerrit.instructure.com/69357
Reviewed-by: Jon Jensen <jon@instructure.com>
Tested-by: Jenkins
Product-Review: Shawn Meredith <shawn@instructure.com>
QA-Review: Shawn Meredith <shawn@instructure.com>
2015-12-28 18:33:40 +00:00
Jon Jensen e3a79b619c spec: fix selinimum capture logging
we only capture data in the post-merge build; this broke in 56b526f,
since we do a /health_check before any examples run

test plan:
n/a, selenium verified selinimum capture locally

Change-Id: I2741e34d9c418380751b6fae7645875a20a01763
Reviewed-on: https://gerrit.instructure.com/69157
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Product-Review: Ryan Norton <rnorton@instructure.com>
QA-Review: Ryan Norton <rnorton@instructure.com>
Tested-by: Ryan Norton <rnorton@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
2015-12-17 23:19:51 +00:00
Jon Jensen 3c022771b0 selinimum: the saga continues
Now with full js/jsx/coffee/hbs/scss goodness \o/

If your commit ONLY touches js/jsx/coffee/hbs/scss/views/controllers or
specific whitelisted things, and doesn't touch global-y bundles or
layouts, then your commit may** be selinimized. That means it will only
run the selenium specs that care about those files.

If you touch lib, or gems, or models, or any global-y stuff, all the specs
will still run.

Also add the ability to override what we selinimize against when testing
changes to selinimum.

**Note that actual selinimization depends on a post-merge build having
succeeded for the parent (or a near ancestor). If the post-merge build
failed, or your build starts before it finishes, or there are
unselinimizable changes between your commit and the nearest ancestor with
selinimum stats, then your commit will not be selinimized :'(

See 4a83f6b6 for the full story.

Test Plan:
* Specs should pass on this commit
* It should not be selinimized because we touch package.json
* See dependent test commits where selinimization happens

Change-Id: I62c78c316876aaff07514a0f460ff4d608dac494
Reviewed-on: https://gerrit.instructure.com/65082
Reviewed-by: Clay Diffrient <cdiffrient@instructure.com>
Tested-by: Jenkins
Reviewed-by: Derek Bender <djbender@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
2015-10-14 18:58:49 +00:00
Jon Jensen 0f85ad522d spec: add _core_en.js and selinimum files to whitelist
rails4 modifies _core_en.js while running compile_assets (presumably due
to new formats), this will allow the rails4 build to be selinimized

also whitelist selinimum's own files, since changes to them don't affect
the actual selenium tests

test plan:
1. rails3 and rails4 builds of this commit should get selinimized

Change-Id: I0f012f8e900f44290f258a791c96d5ab6e4cc095
Reviewed-on: https://gerrit.instructure.com/60010
Tested-by: Jenkins
Reviewed-by: James Williams  <jamesw@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
2015-08-05 14:46:58 +00:00
Jon Jensen 4a83f6b6a2 selinimum: run the minimum selenium necessary for your commit
What is this?
=============

Selinimum speeds up selenium by only running the specs it needs to for
your commit (iff it can safely determine that). For the purposes of the
initial implementation, that means that if your commit ONLY affects:

1. controllers (but not ApplicationController)
2. views (but not shared / layouts)
3. misc whitelisted stuff (images, .md files, etc.)

then it will only run the selenium specs that actually exercise those.
If your commit touches ANYTHING else (models, lib, etc.), all selenium
specs will run.

But wait, there's more!
=======================

Very soon selinimum will also handle:

1. js/coffee/jsx/hbs (except in the common bundle)
2. scss (except in the common bundle)

We already capture which bundles get used by each spec (see Capture), we
just need to correlate that with the individual files via a dependency
graph (probably using madge and sass-graph)

How does it work?
=================

The new post-merge selenium build will run all the specs with selinimum
capturing enabled. This records any controllers/views/js_bundle/css_bundle
that gets used in the course of each selenium spec, and stores a bunch of
data in S3.

Then when your patchset build runs, it will run Selinimum.minimize (via
corresponding tweak in rspect repo) on the list of spec files. If your
commit's dependents can be fully inferred and synthesized with the spec
dependency data from S3, only the relevant specs will actually be run.

Test Plan
=========

This commit doesn't actually cause selinimum to run on jenkins; that
requires some rspect changes and jenkins config. Refer to the test plan
here: https://gerrit.instructure.com/#/c/58088/

Change-Id: I991574c327a3a580c6fdc3ca3797dcfe0490a096
Reviewed-on: https://gerrit.instructure.com/58085
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Jon Jensen <jon@instructure.com>
QA-Review: Jon Jensen <jon@instructure.com>
2015-07-17 03:20:55 +00:00