Switches from standalone containers to explicit node+hub config
Selenium 4 has some differences in handling stale elements that we
should be aware of moving forward
closes OUT-4988
flag=none
[skip-stages=Flakey Spec Catcher]
Test-plan:
- make sure screenshots can happen for failures
- retrigger a few times and make sure things pass
- verify build summaries are intact
- verify FSC can still run seleniums
- verify local selenium running still works
- firefox / chrome / edge where applicable
- verify docker selenium running still works
- firefox / chrome / edge where applicable
Change-Id: I8f2fe5a34d712b5ccd7191bae7a9aeeb6f1f473d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284811
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Robin Kuss <rkuss@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
flag=none
Test-plan:
- run crystalball map, junit `Test Report` item should appear
- in crystalball_map job, the crystalball collator in the final
stage should report >= 40,000 specs
Change-Id: Id4f12671f1bc136c19d8d2f8e4fdcef844e1690c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/286485
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
refs DE-989
flag=none
TEST PLAN:
confirm https://code-coverage.inseng.net/ is updated with new report
Change-Id: I596ce0df04c920b096a875004bd765979dec344b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/285861
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Bobby Buten <bobby.buten@instructure.com>
Product-Review: Bobby Buten <bobby.buten@instructure.com>
flag=none
Test-plan:
- no unit tests should be run
Change-Id: Iaa5bc22a7cf36504a6a27b85f42ce6c673c24e94
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/285877
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
refs FOO-2720
flag = none
Short version: the "_core_en.js" file is no longer loaded in production
as its contents have been merged (there) with the super "core.js" file,
which is targeted for removal later in this series.
Change-Id: I05eb76a2f1cac12b4ab953bad0ca5a49b6ffd37b
---
The problem as I understand it: there are certain phrases that are
marked as "core" because they are used by shared/logic code like
DateTime formatters and are pre-defined in config/locales.yml. These
phrases were being supplied in two distinct files:
- _core_en.js, which includes only the phrases for the "en" language
- _core.js, which includes the phrases for the rest of the supported
locales
_core_en was split because it was deemed necessary to be loaded always,
regardless of the active locale, and that is - as it appears to me, at
least - because some code attempts to look up those phrases at the
time their modules are evaluated. This patch corrects those modules to
defer their lookups until the point where the translations are used,
and when the translations have become available -- just like the rest of
the codebase does.
But if this is true, this begs the question, how come those modules
weren't presenting bugs by using translations for "en" and not for the
target locale? My thinking is that it is only coincidental that they
weren't: should _any_ module that uses I18n be evaluated _before_ any of
those modules, the Webpack plugin will have already loaded the "core"
file, which includes the translations for those phrases in the target
locale. _core_en may not have been loaded by then, but that doesn't
matter because the resolver is gonna look for what's in _core first when
it's available, and it is.
---
What happens in this patch is a slight change to prepare for the full
removal of both _core and _core_en: _core_en is now loaded only in
builds that don't load actual translations because we need the default
values that that file provides. The alternative would've been to go to
each call-site that looks up the phrases provided in _core_en and have
them supply default values, but it's untenable at this point.
This is the list of call-sites and the phrases they look up:
ui/features/calendar/jquery/index.js: time.formats.tiny_on_the_hour
ui/features/quiz_statistics/util/parse_number.js: number.format.delimiter
ui/features/quiz_statistics/util/parse_number.js: number.format.separator
ui/shared/day-substitution/backbone/views/DaySubstitutionView.coffee: date.day_names
ui/shared/syllabus/jquery/calendar_move.js: date.month_names
ui/shared/datetime/jquery/DatetimeField.js: date.formats.medium
ui/shared/datetime/jquery/DatetimeField.js: date.abbr_month_names
ui/shared/datetime/jquery/DatetimeField.js: date.day_names
ui/shared/datetime/jquery/DatetimeField.js: date.abbr_day_names
ui/shared/datetime/jquery/DatetimeField.js: date.datepicker.column_headings
ui/shared/datetime/react/components/render-datepicker-time.js: datepicker.titles.hour
ui/shared/datetime/react/components/render-datepicker-time.js: datepicker.titles.minute
ui/shared/datetime/react/components/render-datepicker-time.js: datepicker.titles.am_pm
ui/shared/handlebars-helpers/dateSelect.js: date.order
ui/shared/handlebars-helpers/dateSelect.js: date.*
ui/shared/i18n/i18nObj.js: number.format
ui/shared/i18n/numberHelper.js: number.format.delimiter
ui/shared/i18n/numberHelper.js: number.format.separator
dateSelect.js is the gnarly one because it seems to be passing through
everything under date.* to God knows who.
The list above was generated with a command similar to this:
grep -rnP "I18n.(t|lookup)\(['\"](date|datetime|number|support|time)\S" ui
---
~ test plan ~
~~~~ ~~~~
- you can still activate a different locale and use something like the
datepicker to normal effect
[change-merged]
[build-registry-path=jenkins/canvas-lms/foo-2720-01]
Change-Id: Ifd5d2d888edc9b89a9930824f2c55fd9c275b03f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/284102
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
flag=none
Test-plan:
- unit tests have entries in the crystalball map
Change-Id: I9a05fd5c050f0b4142dfc19fb3b960fdb00f3584
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283875
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
flag=none
Test-plan:
- run a patchset with different value set for crystalball map
- previous timestamp should appear in #crystalball-noisy
Change-Id: I206e2c62bcfc00f0f8bc8c8ad0b1262d28caab4f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283794
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Change-Id: I4c2ad851e15b8c5bda80bbcdaa22a3481a030b73
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283463
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Brian Watson <bwatson@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
Change-Id: I1d8305402c73e677a66bae6bcc7343302a98510b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283137
Reviewed-by: Bobby Buten <bobby.buten@instructure.com>
Product-Review: Bobby Buten <bobby.buten@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Change-Id: Ia2d06a7d28f7937eb5a677c5acba098c655ad17c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/283060
Reviewed-by: Jeremy Stanley <jeremy@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs SEC-4437
flag = none
[pin-commit-respondus_lockdown_browser=8746ff2ccfd6e9ac1d7280687c88e9fa67c0f693]
[pin-commit-multiple_root_accounts=7b11533084764b3a34f390e9a75df26e9dd871f9]
rationale in the script file and more context in the accompanying ticket
~ test plan ~
edit package.json and modify one of the explicit workspace dependencies
such as @instructure/ready to have a specifier other than "*", run the
script and verify it exits with 1
yarn --silent workspaces info --json | \
node script/yarn-validate-workspace-deps.js 2>/dev/null
Change-Id: I6624ada67a21d433477a7ad4d36acf5801853b7a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282948
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ben Rinaca <brinaca@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
flag=none
Test-plan:
- tests should still use crystalball map correctly
- s3 map should have timestamp post merge
Change-Id: I563bbd32499b7d9424e1563c44f27c91e9357bb4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282795
Reviewed-by: Martin Yosifov <martin.yosifov@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
run crystalball build silently, in parallel to regular
builds.
flag = none
Test Plan:
-Jenkins passes
-proofs-of-concept/test-queue build runs same tests
-with ruby file change, proofs-of-concept/test-queue only
runs crystalball predictions for selenium
-regular test-queue still runs all tests
Change-Id: I243c096c9ac3fb2907d7b67272610663284aebc6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282684
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Brian Watson <bwatson@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Note: Ensure that istanbul is only enabled for crystalball before
this is merged
closes OUT-4918
flag=none
Test-plan:
- crystalball map should include JS files
- ensure that CRYSTALBALL_MAP isn't set to 1 in standard pre-merge
Change-Id: I5ae2f32177640e3caeb77871918644890eb4ae30
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/280813
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
This reverts commit 7dd287adea.
Reason for revert: try again
Change-Id: Ie0eededa8465a7fb79bbd221c777646ab2640fa7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282036
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs FOO-2520
flag = none
[pin-commit-multiple_root_accounts=2a9bf89895f38df6bf8f54828af66aced594abf0]
revisit the API for resolving asset names to their (real)path on disk,
because adding to the existing logic to support an alternative bundler
made things hard to understand.
This patch brings a new simplified interface Canvas::Cdn::Registry to
query assets and resolve their location.
- Registry#include?(path) tells whether a realpath points to a static
asset
- Registry#statics_available? tells whether static assets are available
- Registry#scripts_available? tells whether JS assets are available
- Registry#scripts_for(bundle) provides the realpaths to all the JS
files in the specified bundle
- Registry#url_for(name) provides the realpath to the static asset
The Registry is a good place to house the BrandableCSS resolving logic
in the future for even more consistency. It can also support an
alternative bundler internally without leaking. Eventually, it would be
nice to have it as a gem.
CHANGES
-------
- helper "font_url_for()" has been removed as it was a duplicate of
existing logic; instead use "font_path(...)" to achieve the correct
result. As a result, BrandableCSS is no longer querying Gulp's
manifest.
- preloaded fonts are now aware of the asset host and work for CDN
- InfoController uses the new Registry API to tell whether Gulp and
Webpack have produced their assets successfully
- ApplicationHelper no longer re-computes the base URL for JavaScripts,
now only the Registry is concerned with that
- ?optimized_js query parameter is no longer supported as it has no real
benefit now that we have access to sourcemaps on production
- ENV['USE_OPTIMIZED_JS'] is now more consistent as there is a single
source of truth for it. The Registry can be instantiated with
{environment: "production"} to point to the optimized version of the
scripts.
- "css:compile" task no longer writes BrandConfig records to the DB,
that is now done as part of the "compile_assets" task, which you can
opt out of doing by setting COMPILE_ASSETS_BRAND_CONFIGS=0
TEST PLAN
---- ----
- load your dashboard and verify all the assets are loaded correctly
- set up a CDN, restart your Rails server and reload the dashboard
- verify all assets are loaded from the CDN
- verify the Lato fonts are pre-loaded from the CDN
- (optional) add custom JS to a sub-account and visit it
- verify the custom JS is loaded and evaluated *after* Canvas's main
javascript bundles
Change-Id: I8198de747cdd5892d6a831cb6c61ba0ef9afa789
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276537
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Charley Kline <ckline@instructure.com>
Product-Review: Charley Kline <ckline@instructure.com>
This reverts commit 276eb49f8e.
Reason for revert: Something in here is failing deploys
Change-Id: I615a90e6a4edf743ba47a2673bc14483ad141b3d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282032
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Tested-by: James Butters <jbutters@instructure.com>
run rspecq/rspec tests in separate sub-build.
flag = none
refs: DE-929
Test Plan:
-Jenkins passes
-test count is consistent
Change-Id: Ia62730c32d1793d591555c0fd034538254efffc3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/282016
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
closes OUT-4926
flag=none
Test-plan:
- coverage should be complete
- coverage in latest jenkins build should match
what's visible in code-coverage.inseng.net
Change-Id: Ia64778433a698e7f2eb8bbd2a5e9621662eedb03
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/281119
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
refs DE-966
flag=none
TEST PLAN:
ensure build completes and all tests run successfully
Change-Id: I82ef402eb790fafce53393b17b4695c9151e1745
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/281489
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Ben Rinaca <brinaca@instructure.com>
QA-Review: Bobby Buten <bobby.buten@instructure.com>
Product-Review: Bobby Buten <bobby.buten@instructure.com>
(specifically canvas-rce)
closes MAT-393
flag=none
Test Plan
- Check out commit 111b6aa
(this commit has translations which are not synced to canvas-rce)
- Manually run `yarn wsrun --exclude-missing installTranslations`
- Manually run `yarn wsrun --exclude-missing commitTranslations`
- Observe that a patchset was created in Gerrit
Make sure to abandon the patchset
Change-Id: I33654c064ff8a22b5d78588d9946e94bb5460d18
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/280766
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Product-Review: Jon Scheiding <jon.scheiding@instructure.com>
Reviewed-by: Weston Dransfield <wdransfield@instructure.com>
QA-Review: Weston Dransfield <wdransfield@instructure.com>
[ignore-stage-results=Flakey Spec Catcher]
update chrome version to run tests against. A few small
fixes to work with new chromedriver.
closes: DE-930
Test Plan:
-Jenkins passes
-FSC build passes
-Run test local docker dev setup
-Run test local dev setup
Change-Id: Ie2d5dd918f85cbcc01d0f8b572efa8aab71c1803
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279959
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
closes OUT-4895
flag=none
Test-plan:
- after job builds, new map should be visible in instructure-canvas-ci
bucket
Change-Id: I3fa98f1da7e4b75f6b24dedff5a9cb35c3e08eb6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/280200
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
Also moves config/initializer/crystalball.rb to spec/support
to prevent it from running in prod
This reverts commit 9a9c68be6e.
Test-plan:
- passes cd to edge without failure
- generates map successfully in PoC build
Change-Id: I67c02ebaea06e11b3a3721aa49217da16e75bd32
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279848
Reviewed-by: Michael Hargiss <mhargiss@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
This reverts commit bd847c2d14.
Reason for revert: breaking cd
Change-Id: I85ed1dc69045d2cd984778da12c2e0b402b842ad
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279637
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
flag=none
Test-plan:
- #crystalball-noisy should link to the latest build AND the
latest map should be linked (stored as a build artifact)
- job should run nightly
- nightly alerts should promopt #crystalball-noisy posts
(but not gerrit manual triggers)
Change-Id: Ibbd45dfd8a9b3ed8f4c662b322214cbbc3dc99b4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277205
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Michael Hargiss <mhargiss@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
refs DE-545
When apt-get tries to pull a package from a server that is down or undergoing maintenance, builds that modify Gemfile can fail due to not being able to pull packages. By splitting the package installation into its own layer, external packages no longer need to be reinstalled when gems are updated.
Test Plan
1. Cache builds correctly from scratch, uploads all relevant images.
2. Cache is correctly reused when no gems are updated
3. Base cache image is reused when gems are updated, and all other images are rebuilt
4. Pre-merge build passes and uses cache as appropriate
5. Updating Dockerfile.jenkins re-builds base image
[build-registry-path=jenkins/canvas-lms/de-545-1]
[change-merged]
Change-Id: Ifd76064892817abb80dd0daebe8c2189c0338d78
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/279697
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs FOO-2371
flag = none
test plan: jenkins does what the title says (on post-merge)
Change-Id: Id17a194c1c96ade4d21cea0a6ed97205a8eb2754
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275751
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: August Thornton <august@instructure.com>
QA-Review: Ahmad Amireh <ahmad@instructure.com>
Product-Review: Ahmad Amireh <ahmad@instructure.com>
add back only archive test results if axe build
Change-Id: I9ae9bd1dd57d3ea6bcb1cc941c7a942d94d6f5a4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277946
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
only report the initial spec run, not the rerun.
Change-Id: I7a52f5b90d92c8dac30da8578a6f209a0e8de0b6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277873
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
do not alert slack if the build aborted due to new patchset
or user aborted.
Change-Id: I3074a23c7979254ee5798b6edb9bde31c07b675c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/277626
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
alert slack channel canvas_builds-noisy on node
failure and do not fail build for rspecq node
failure.
flag = none
Test Plan:
-Jenkins passes normally
-When queue is depleted and node acquired late
-Node will release before any steps are ran
-An RSpecQ node throws an exception during steps
such as can't pull image.
-The stage will get marked as UNSTABLE, following
stages will get skipped.
-Overall build result not affected by this failing
node.
-If tearDownNode stage fails, overall build will be affected.
-RSpecQ Reporter failures do affect overall build result
Change-Id: Iccbb66cad0a988a26bf17c8332a20d8d7d55ee5b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276927
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
sometimes the queueUnprocessed will happen so fast rspecq
hasn't had a chance to populate so now we'll also check
queueProcessed to see if any tests have been ran and if
there are any more.
Test Plan:
-Normal Jenkins build passes
-Add a sleep to make node wait for empty queue
-node will release
Change-Id: I2f18b6dd4011c2a331e675ff02672cc8bf95a74e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276872
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
[use-rspecq]
Test Plan:
-Jenkins passes both with and without rspecq
-add sleep to one node, wait for queue to empty
-node will not run any stages since queue is empty
-add sleep to one node, wait but still have tests in queue
-node will still run stages and tests
Change-Id: Iace45f456c19284c6a338f5537a367917fd4f9c0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275325
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
closes OUT-4802
flag=none
Test-plan:
- test with a custom_reports PS referencing this
- see comments for expectations of FSC re-runs, compare against
actual
Change-Id: I94ead7b61e37512231b558b16b57a8bfb2cff4c8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/276101
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
closes OUT-4802
flag=none
Test-plan:
- in PS1 builds,
- examine changed specs and match it up with FSC queued up tests
- in Jenkinsfile.selenium.flakey_spec_catcher, examine output, the new
"raw result from catcher" and "raw result from splitter" should output
similar lists
- examine any differences and whether they are noteworthy or not
- in PS2 builds,
- verify that FSC sub-build still functions after removing splitter
Change-Id: Icae697f1cedca8ab9a775a1f394891d4ca2007a3
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275765
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
[use-rspecq]
threading in the groovy script causes each process to act as a stage
and severely slows down each step. Move the threading into a bash
script similar to how rspec/selenium are currently ran.
flag = none
Test Plan:
-Jenkins Passes
-Step timings are faster and each process is not separate stage
Change-Id: I520c5c6ceecb717d64eee75e02913cf8ebfede00
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275903
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs FOO-2423
flag=none
TEST PLAN:
1) run pulsar specs lots of times with many seeds
2) they don't fail.
Change-Id: Ide7fbaebee5ddac37bf2db03d8699f32d1b57d56
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275315
Reviewed-by: Rob Orton <rob@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
[use-rspecq]
flag = none
Change-Id: I2e2a8ed8c878c0da4ac04f62b321ee740410fac9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/275093
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
it was just too confusing on which one an editor is using, double comments
in jenkins, etc.
this is accomplished by several things:
* required cops are just marked as severe, instead of using a separate
config for them, and failing if anything shows up from that config
* get rid of all the logic to only include certain directories for
certain cops. turns out it's not _that_ ominous to correct errors
across the entire repository before marking a cop as required.
* but still auto-generate config to turn _off_ autocorrect for
non-severe cops. this is important because auto-correct must run
for entire files, and we don't want it auto-correcting optional
things that you didn't touch.
* update gergich to get more details from the parsed comments.
this plus the prior point means we _don't_ have to have heavy mode when
in autocorrecting, but we still display out-of-context lines that were
autocorrected
this also makes it so we can use per-dir .rubocop.yml files again, so
take some of the exceptions out of the root and put them in their own
directory
Change-Id: Ie936d1a9920b68910acd250ba817c7b4a670b958
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/274394
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Simon Williams <simon@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
* remove spurious .rubocop.yml override files
* split the configuration into an enforced and optional
* run both configurations in jenkins (may result in some duplicate
comments at different levels)
* auto-correct the enforced configuration in the pre-commit hook
* fix comments for Gemfile.d and the root dir; enforced configuration
is only applied to that directory for now
Change-Id: I8da21073d74e19138b1b580d66c7aae6465348d4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273898
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
[canvas-builds-refspec=2450c492e6342da15bdbabbe79bf6a21edc987df]
Test Plan:
- Jenkins passes with the corresponding canvas-builds commit
- Jenkins passes with canvas-builds commit and rspecQ
Change-Id: I45374a43a58b4240d862b2afb9e3cea9f67efcae
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273441
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
each post-merge build will run update-timings to keep our
timings current. Also set file split threshold to high number
so we get accurate timings per file.
[use-rspecq]
Test Plan:
- Jenkins passes without rspecq
- Jenkins passes with rspecq
- pre-merge:
- RSPECQ_FILE_SPLIT_THRESHOLD equals 150
- RSPECQ_UPDATE_TIMINGS equals 0
- post-merge:
- RSPECQ_FILE_SPLIT_THRESHOLD equals 999
- RSPECQ_UPDATE_TIMINGS equals 1
Change-Id: I7c5e57fa6576c8b0cd1d724ae0b6958ceeb2707b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273267
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
[pin-commit-multiple_root_accounts=b29762c8bf01c2ffd52494f7e61df26aab6af921]
this adds the ability to run rspecq based on percentage of
times executed. this defaults to 10% and can be modified using
the env variable RSPECQ_EXPERIMENT_PERCENTAGE
test plan:
-in approximately 10 builds, rspecq executes at least once.
Change-Id: I7b6709c28b2f0344dfa253cb006ad779d673ccdc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273048
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
[pin-commit-multiple_root_accounts=eeb2c3571469a46b5bec97cd789ae5f86c0cd241]
Test Plan:
- Jenkins passes normal build
- tests are not skipped specific to this gerrit
- Jenkins passes RSpecQ build
- tests are skipped
Change-Id: Ibbec7b21f601fe242dd683e72cd3721d8a8a9eb7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/273134
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
RspecQ can now be used to run tests using the
'use-rspecq' commit message feature flag
test plan:
- build runs as normal without use-rspecq enabled
- enabling use-rspecq commit flag runs tests using rspecq,
no tests run via old rspec/slenium groups
- test numbers match between differing builds
- build artifacts are created with or without feature flag
- build summary report works with or without feature
flag
Change-Id: Ibaf32177a8ec28a89278eaa4277bd7a752cbfc58
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272130
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
gergich will process it and -1 it instead
Change-Id: I554ee1e63586bf27238b3cf0fc7fb70f1d2fe2c7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/272252
Tested-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
so that we can reference only that fragment from our binstub
(which is de-springified), making it run significantly faster
Change-Id: I4f602e6c4d1feecf74eccd66e610781c76756ffc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271957
Tested-by: Cody Cutrer <cody@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Cody Cutrer <cody@instructure.com>
Product-Review: Cody Cutrer <cody@instructure.com>
this adds some details around who and what is changing when
a user modifies a feature flag
flag=none
Change-Id: I36f6107aa1a0509243c2b169d017cd037451f51a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/271559
Tested-by: James Butters <jbutters@instructure.com>
QA-Review: Ryan Norton <rnorton@instructure.com>
Product-Review: Ryan Norton <rnorton@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
refs FOO-2198
flag=none
update AUA log compaction specs
to allow threaded queue to drain
TEST PLAN:
1) be sending messages over pulsar in AUA
2) cause some timeouts somehow
3) pulsar messages back up in memory
but canvas requests don't slow down.
Change-Id: I311f9d30699d6d07a58a80065a0d939472a2178e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/269906
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Cody Cutrer <cody@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
we had a typo in call function
test plan:
- dev builds rebase successfully
Change-Id: I81c1334f05e2f6083d5e1d83147187877a547357
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268469
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Since we are switching the main build to ruby 2.7 from 2.6
we want to rebuild the docker images and the migrations.
In order to do that we need to update the CACHE_VERSION var.
refs DE-745
test-plan:
- Jenkins build passes
Change-Id: Ia910603d14d7e9a5eacca7ead79a3e64ac822def
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/268795
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-719
withStarlordDockerLogin was needed for EKS, but with EC2
no docker login is needed.
test plan:
- EC2 builds continue to work as expected
Change-Id: Icbd3f28bf2ec6ec52c429c1eaf8fc6ce7bddd46b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267714
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
linters on plugin builds are currently failing because we are trying
to run git commands from /usr/src/app rather than the plugin
directory.
flag = none
closes: DE-713
Test Plan:
-Build passes
-plugin build with changes passes
Change-Id: Iad5b6275845e47159910e964993c71f40bf19672
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/267100
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs DE-707
Test Plan
1. Test count aligns with current master
2. Outcomes publish API step doesn’t publish on pre-merge and does publish on post-merge
Change-Id: Ic69f9b74137ffd36fc5a0964e5880a59457fca36
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266983
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Ryan Norton <rnorton@instructure.com>
Test Plan
1. Linters works on EKS
2. Linters works on EC2
3. Linkers runs appropriate stages
4. Yarn changes run appropriate stages
5. Groovy changes run appropriate stages
6. Linters reports back to patchset via gergich
Change-Id: I6c50ab5a6be2e1b7031b56c70ee6e2727e92580e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266509
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-706
In order to access the buildConfig variables without duplicating constants in other shared library files, restructure this stage to be in its own files, and provide helper methods for getting / setting changed file state.
Test Plan
1. Normal build runs appropriate stages.
2. Local docker dev changes runs appropriate stages.
3. Groovy linter changes runs appropriate stages.
4. Migration changes run appropriate stages.
5. Spec changes run appropriate stages.
6. Yarn changes run appropriate stages.
Change-Id: I72e35d2b45d4842e195cc27351327a6907b073b7
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266957
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-706
The snykSecurity() plugin doesn’t seem to work on EKS, fortunately we can convert it to using the npm package instead which also reduces complexity.
Test Plan
1. Dependency check runs and reports results to snyk on EC2
2. Dependency check runs and reports results to snyk on EKS
Change-Id: I317ffe8c90cda6754a2ad5df7227224450b86546
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/266779
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-687
Convert jobs over to EKS-compatible syntax in order to perform AB testing on EKS builds.
Test Plan
1. EC2 builds JS job with correct number of test results
2. EKS builds JS job with correct number of test results
[canvas-builds-refspec=3f9a35ef55d38ac885c3a4daef03e7e19cd8f4e7]
Change-Id: I350f2983eebeda4aa634daba0579389b3b987c6b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265580
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-687
Test Plan
1. All JS test counts align with latest post-merge
Change-Id: Id803db3f13999e149ecb2dfed32efc66dac61c76
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265419
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-687
Test Plan
1. All test counts align with master for JS jobs
Change-Id: Iffe8ca4f873429b258269d371da3c67c1363a0e4
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265395
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-687
As part of our work on EKS, we need to align the EC2 / EKS paths to pull artifacts from the same path on the container and host. As a step towards that, ensure that all containers output their test files to the same directory.
Change-Id: I312fb5f8505122c68ba5aaba30730aa5c1887177
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265370
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-690
flag = none
test-plan:
- Build passes
- Build trigger groovy linter stage if a change to Jenkinsfile is
made
- Build fails if a Jenkinsfile has linter issues
Change-Id: Ifdabb4f28f6124afcb6621844375a8de4b35d453
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265279
Reviewed-by: James Butters <jbutters@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
flag=none
closes OUT-4438
Test-plan:
- see axe build corresponding to latest PS
- in Build Artifacts under tmp, there should be axe_results.csv
compare it against Test Results and verify that specs are mapped
to specific a11y violations
- in Build Artifacts, verify that rspec_results with nested xml files
are archived for axe build but not for main build
Change-Id: Ic8c9ccbe9350bde0d0fd6914f16330ca5155a95b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264673
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Pat Renner <prenner@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Brian Watson <bwatson@instructure.com>
Product-Review: Brian Watson <bwatson@instructure.com>
refs DE-643
flag = none
Add a linter stage for groovy to the main build:
- Build passes
- Build passes and trigger groovy linter stage if no
linter errors are present
- Build fails on groovy linter stage if a linter issues
is found
Change-Id: I8f3c2f2f577227854e19b1af83400018a4c792db
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265024
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-687
Test Plan
1. Each JS stage executes all tests
2. Each JS stage build report summary works for failures
Change-Id: I2f9c9347ff6caac22ef3785e6a1a298870961f79
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265049
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-644
[canvas-builds-refspec=c30dc5a5768e38eb7ef483a6b8b5e7c30af8a25d]
Test Plan
1. RSpec / Selenium use correct number of nodes and run correct number of tests
2. RSpec / Selenium retry according to RERUNS_RETRY value
3. Build Summary Report generates correctly
Change-Id: I886e41a8712bda5a98f58eee0ba42d034a08db25
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265014
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-444
Test Plan
1. RSpec / Selenium use correct number of nodes and run correct number of tests
Change-Id: I8d6524620e2407ee5a58cb3edb6d7a691ab92763
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265008
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
Test Plan
1. RSpec / Selenium use correct number of nodes and run correct number of tests
2. Build Summary Report shows test results + HTML page
Change-Id: Icf39240c8f26e78024f4211ed7b12d3c61426b70
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265005
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-444
Test Plan
1. RSpec / Selenium use correct number of nodes and run correct number of tests
Change-Id: Ie95d0a504ae52ffa8e3540f3bc4facbed9d4fcad
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/265000
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
Test Plan
1. RSpec / Selenium use correct number of nodes and run correct number of tests
Change-Id: I6a58419513becc2cf587b65e05df3ed75ff6fe43
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264996
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
Test Plan
1. RSpec / Selenium obey force failure flag
2. RSpec / Selenium retry once upon test failure
3. RSpec / Selenium use correct number of nodes and run correct number of tests
4. Parallel log build works
[canvas-builds-refspec=5d46b7af0f1d5beb710b02863cf7b98689d0c332]
Change-Id: I7dbc2548f949ec307dec8e7d73d041ab24256072
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264978
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
Test Plan
1. Rspec / Selenium use appropriate number of nodes
2. Rspec / Selenium retry only once
3. Force failure flag works
Change-Id: I2fde387b410b895f9cf0cade9e5986ee34dd8391
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264904
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
This function is only used in one place, and we need to remove this file, so just integrate it into the calling job itself.
Change-Id: Ibda768b6672e1090947e72b1dd820662d895ab7b
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264902
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
This extra parameter isn’t used anywhere, so remove it.
Change-Id: I487e7a6d73fdd218f87263fdf1a8970057ae3ef8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264900
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-444
Modify the generation of these stages to align with our new extendedStage() conventions in preparation for a refactor of how the stage internals are implemented.
Test Plan
1. All RSpec stages are queued and run the correct tests
2. All Selenium stages are queued and run the correct tests
3. Parallel log build works correctly
Change-Id: Id0a35f84ddf8471f1f06eede608d6ad4dc964dea
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264870
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-444
This extra parameter isn’t needed, the cases that use it already have access to env.PATCHSET_TAG.
Change-Id: I60778b9efaccb49c4a2ca64ab0cf37c2f3f83c1e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264874
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-462
As part of the EKS transition, we need to restructure the JS stages. The coverage code is currently not working and will get in the way of this transition. Remove it for now and refs the coverage restoration ticket so when we bring back the coverage build the code is here for viewing.
Change-Id: I7437584107ac13a79000d43f5c14baa31dac5027
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264864
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-462
As part of the EKS transition, we need to restructure the rspec / selenium stages. The coverage code is currently not working and will get in the way of this transition. Remove it for now and refs the coverage restoration ticket so when we bring back the coverage build the code is here for viewing.
Change-Id: I137e7c20674542aac31184c6fd7aa648b6656362
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264859
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-467
it is no longer possible to spin up multiple docker containers
ona single rspec or selenium agent when running tests. parallelization
is now exclusively done through multple rspec processes on
a single container.
test plan:
- setting docker_processes ENV variable has no effect
- settting rspec_processes still runs multiple test threads
- tests continue to run as expected given that docker_processes has
been set to 1 for multiple months.
Change-Id: I604d953a71ad220373b6a71d558f46bff29f820e
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263701
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: James Butters <jbutters@instructure.com>
refs DE-467
this change will require a followup to remove
DOCKER_PROCESSES.
test plan:
-run FSC with a changed spec.
-FSC runs the modified spec 30 times, 10 times in each thread.
-run FSC with multiple chnaged specs
Change-Id: I640c7b87e439db3c7efb42ac4e7f81e10dcbc5c9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264703
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
refs DE-467
this change will require a followup to remove
DOCKER_PROCESSES.
test plan:
-run FSC with a changed spec.
-FSC runs the modified spec 30 times, 10 times in each thread.
Change-Id: I26755f6b9c3bf63267eb9d1ac648901d67ee41df
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264129
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
refs DE-670
For the EKS transition, we need to ensure that the Linters stage doesn’t run on the EC2 builder node and instead properly uses EKS containers. As part of this work, transition the dependency check stage to running as part of the Linters set of stages.
Test Plan
1. Dependency Check runs correctly on post-merge build
2. Linters runs correctly on pre-merge build
Change-Id: I16d9fb63b40b6eb9f8e24b4f5c4253538e91bbf0
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264653
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
refs DE-670
For the EKS transition, we need to ensure that the Linters stage doesn’t run on the EC2 builder node and instead properly uses EKS containers. As part of this work, transition the dependency check stage to running as part of the Linters set of stages.
Test Plan
1. Dependency Check runs correctly on post-merge build
Change-Id: Ifc9790e481e7d6cf7ba81e9f78b804a955f266b8
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264589
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-670
By pulling the image as part of setupNode, we get flakey network protection and also only pull the image once per build, instead of once per parallel stage.
Change-Id: I77b2d63a3ab971067604c30001387bbf67ad6c4a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264590
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-670
For the EKS transition, we need to ensure that the Linters stage doesn’t run on the EC2 builder node and instead properly uses EKS containers. As part of this work, transition linter stages into their own node as a sibling stage to Builder.
Test Plan
1. Linters run correctly
2. allow-stages directive works with Linters
3. allow-stages directive works without Linters
Change-Id: Ieae121ae3e79b5e30864f9060042e4453a8093e6
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264569
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-467
this change will require a followup to remove
DOCKER_PROCESSES
test plan:
-run linters
-linters use expected db on a per thread basis
Change-Id: Ie0fee7d22a335899e6dac5490f4cadf567e4e32f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264294
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
refs DE-670
Remove scripts that are part of an old iteration of Linters that is no longer in use.
Test Plan
1. Linters passes and runs all appropriate linters
Change-Id: I773bc090c4d028695f81ff9982ccedab66c5228c
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264562
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-670
For the EKS transition, we need to ensure that the Linters stage doesn’t run on the EC2 builder node and instead properly uses EKS containers. As part of this work, transition Master Bouncer to using the Linters image since it uses Git.
Test Plan
1. Master Bouncer runs correctly
Change-Id: I8aa6d2f589326c4f88e313d423f023ec9af8a79f
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264555
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-670
refs DE-500
For the EKS transition, we need to ensure that the Linters stage doesn’t run on the EC2 builder node and instead properly uses EKS containers. To do this, the Linters stage needs to always upload its image so that EKS can pull it.
Test Plan
1. Linters properly reports to the correct gerrit for main builds
2. Linters does not become the runtime bottleneck
Change-Id: Ibe28d47c13f965ee468b49ce6a74e3d08b75715d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264549
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Andrea Cirulli <andrea.cirulli@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
refs DE-670
For the EKS transition, we need to ensure that the Linters stage doesn’t run on the EC2 builder node and instead properly uses EKS containers. To do this, the Linters stage needs to not depend on having the .git directory on the host machine.
Test Plan
1. Linters properly reports to the correct gerrit for main builds
2. Linters properly reports to the correct gerrit for plugin builds
3. Linters does not become the runtime bottleneck
4. Linters yarn.lock diff check works
Change-Id: I49650bfb3feece9614a667d3af5ec9dba8c2e838
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264539
Reviewed-by: Kyle Rosenbaum <krosenbaum@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Aaron Ogata <aogata@instructure.com>
Product-Review: Aaron Ogata <aogata@instructure.com>
Fix the linter issues and restore the previous behaviour of codeNarc
for final variables
refs DE-643
flag = none
test-plan:
- build passes
Change-Id: I16f5e388e32fe0aec6789f1f87b0da03b1eacc83
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/264362
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
Product-Review: Andrea Cirulli <andrea.cirulli@instructure.com>
closes FOO-1835
closes FOO-1837
closes FOO-1838
flag=none
TEST PLAN:
1) enable aua on pulsar
2) all aua log messages that go to the
db also get sent to a pulsar topic
Change-Id: I0f90ab7c8aa32376a122e30a1c104a1651a507bf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263930
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
and that we have a vault token
refs FOO-1842
flag=none
If a cert url is in the config, we need
to fetch and write it to disk before trying
to open a connection to pulsar or we'll
get failures.
Also add some config caching so
we aren't re-parsing the same YAML
blob over and over.
TEST PLAN:
1) specs should pass
2) be in a pulsar-enabled environment
with a cert configured
3) when you build a message_bus producer,
the cert is fetched and written to
disk at the configured path.
Change-Id: I2fb61badd05aedadb6d07475875692b30dd30db9
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263687
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Jacob Burroughs <jburroughs@instructure.com>
QA-Review: Ethan Vizitei <evizitei@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>
refs DE-558, DE-582
[use-kubernetes]
[canvas-builds-refspec=24d3d83e8034ac6ad5380936498e1dcf01ab50eb]
modify the existing nodeRequirements to add podTemplate and
container definitions. initially, the podTemplate to be used
is docker-in-docker. the "use-kubernetes" flag must be set
as well.
test plan:
- build executes on k8s nodes in EKS when "use-kubernetes"
flag is set.
- build executes on EC2 when "use-kubernetes" flag is not set.
Change-Id: I13c68e5420bbf414532518eaf69ac82f9310bfbf
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/263257
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Reviewed-by: Aaron Ogata <aogata@instructure.com>
QA-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Product-Review: Kyle Rosenbaum <krosenbaum@instructure.com>
Aaron Ogata