Closes: CCI-261
The mdillon repo was an unofficial postgis repo, there is now and
official postgis/postgis repo!
https://registry.hub.docker.com/r/postgis/postgis/https://github.com/postgis/docker-postgis
Test Plan:
- First, take notice if you currently have pg_collkey installed:
$ docker-compose run --rm postgres psql -x \
-h postgres
-U postgres
-d canvas
-c "SELECT * FROM pg_available_extensions WHERE \
pg_available_extensions.name = 'pg_collkey';"
If `installed_version` is blank, then the extension is not isntalled
into the `canvas` database.
- take down any running environment:
$ docker-compose stop && docker-compose down
- copy (or update) new settings from docker-compose.override.yml:
# manual diff
$ diff config/docker-compose.override.yml \
docker-compose.override.yml
# overwrite any existing config
$ cp config/docker-compose.override.yml .
- copy (or update) new settings from database.yml:
# manual diff
$ diff docker-compose/config/database.yml config/database.yml
# overwrite any existing config
$ cp docker-compose/config/database.yml config/
- rebuild and startup the postgres image:
$ docker-compose up --build postgres
- the postgres logs do not mention
ERROR: could not access file "$libdir/collkey_icu.so":
No such file or directory
- install the extension if it wasn't previously present:
$ docker-compose run --rm postgres psql -x \
-h postgres \
-U postgres \
-d canvas \
-c "CREATE EXTENSION IF NOT EXISTS pg_trgm SCHEMA public;
CREATE EXTENSION IF NOT EXISTS postgis SCHEMA public;
CREATE EXTENSION IF NOT EXISTS pg_collkey SCHEMA public;"
$ docker-compose run --rm postgres psql -x \
-h postgres \
-U postgres \
-d canvas_test \
-c "CREATE EXTENSION IF NOT EXISTS pg_trgm SCHEMA public; \
CREATE EXTENSION IF NOT EXISTS postgis SCHEMA public; \
CREATE EXTENSION IF NOT EXISTS pg_collkey SCHEMA public;"
- pg_collkey is now present:
$ docker-compose run --rm postgres psql -x \
-h postgres
-U postgres
-d canvas
-c "SELECT * FROM pg_available_extensions WHERE \
pg_available_extensions.name = 'pg_collkey';"
$ docker-compose run --rm postgres psql -x \
-h postgres
-U postgres
-d canvas_test
-c "SELECT * FROM pg_available_extensions WHERE \
pg_available_extensions.name = 'pg_collkey';"
- Optionally, Build the postgres container with fresh volumes:
$ COMPOSE_PROJECT_NAME=<a-unique-name> docker-compose --build \
up postgres
- The postgres logs show no errors related to collkey_icu.so
(like above)
Change-Id: I4f0e027cd908fee937b6204cd3a20e5ecf998021
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/228190
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Currently it appears pg_collkey throws an error in the postgres
container, althought it doesn't appear to halt the process. Adding this
missing file resolves this problem
Fixes: CCI-252
Test plan:
- docker-compose build postgres
- docker-compose up -d
- docker-compose logs -f --tail=100 postgres
- visit http://canvas.docker/ in the browser
- the postgres logs should no longer contain a message like:
ERROR: could not access file "$libdir/collkey_icu.so": No such file or
directory
feature_flag = none
Change-Id: Ib1483a24e705260b2eaeba81ec456ed9f6acf89d
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/227805
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
Reviewed-by: Jacob Powell <spowell@instructure.com>
test plan:
This enables vendored gems to propagate, since those will have been
fixed by the first part of this commit which is ticket CCI-96
Loading migrated images is also enabled again in this commit, since they
will be correct after merging the CCI-96 patchset (they will be stored
on merge of that commit)
This follow up commit allows for a two-step verification that the new
cassandra changes actually work, and don't fully rebuild cassandra but
pull the base image and configure it instead. This will also fix
vendored gems from failing due to having to build cassandra, since it
will be using an existing image, therefore propagate will be turned back
on
fixes: CCI-101
Change-Id: I64803acb324132784e8caf5eb3957e094dbf7adc
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/217637
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: S. Jacob Powell <spowell@instructure.com>
test plan:
This now uses the starlord image of cassandra and passes in
a configuration that changes the data directory so that the commited
image includes the database and any migrations.
I tested this locally by building the cassandra image, checking that the
cassandra data files were indeed saved into that directory, saving
a custom file in the data directory, committing the container, stopping
the container, running the committed image, and verifying that the file
that was in the data directory is indeed still there (if it was in
a directory that was VOLUME'd, it would not have shown up there)
This also has been tested/will be tested by disabling loading the cached
migration images, forcing a rebuild of cassandra to see if it works
correctly.
A follow up commit enabling loading cache after this one will then load
the newly built cassandra image (and other database images), since it
will have been saved on the merge of this commit.
The follow up commit will enable vendored gems propagation again,
which depended on
a prebuilt cassandra so that it isn't so flaky.
(follow up ticket: CCI-101)
fixes: CCI-96
Change-Id: I4ce101f119ff6bdfd70bbdaeb4733e431184c1ee
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/217622
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: James Butters <jbutters@instructure.com>
QA-Review: James Butters <jbutters@instructure.com>
Product-Review: S. Jacob Powell <spowell@instructure.com>
Overview:
This patchset is intended to save time by saving and loading migrated
containers and reusing the image.
It checks to see if all three database images were loaded (the cached
ones) and if so, then it uses those instead of migrating from scratch.
If one of them is not loaded, it performs the regular process.
The storing of the migrated images will only happen on merge.
It does run migrate on the cached images, but it correctly skips
anything that was already done.
This pattern will be repeated in each sub build, which should save a lot
of time not having to redo the migrations.
Test plan:
View the build and confirm that it passes when loading cached migrations
Should be about a minute or more faster
Fixes: CCI-30
Change-Id: Iba878bb646fd976deab5d624e8960cac63d09677
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/213268
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
Reviewed-by: Rex Fleischer <rfleischer@instructure.com>
QA-Review: S. Jacob Powell <spowell@instructure.com>
Product-Review: S. Jacob Powell <spowell@instructure.com>
Closes: CCI-2
Test Plan:
- specs that require pg_collkey are ran in new jenkins
Change-Id: I530373f9855d124dfc3c302e0d340f6217b8f33a
Reviewed-on: https://gerrit.instructure.com/c/canvas-lms/+/215422
Reviewed-by: S. Jacob Powell <spowell@instructure.com>
Tested-by: Service Cloud Jenkins <svc.cloudjenkins@instructure.com>
Tested-by: Jenkins
QA-Review: Derek Bender <djbender@instructure.com>
Product-Review: Derek Bender <djbender@instructure.com>
Also print out debug info while it's trying to pull archives
from AWS S3.
Change-Id: I41dec09de8c67652b4900255a2e53f1655dfab5d
Reviewed-on: https://gerrit.instructure.com/196747
Tested-by: Jenkins
Reviewed-by: Robert Lamb <rlamb@instructure.com>
QA-Review: Tucker Mcknight <tmcknight@instructure.com>
Product-Review: Tucker Mcknight <tmcknight@instructure.com>
This might help mitigate the dreaded postgis error we've been seeing
(OperationalError: could not access file "$libdir/postgis-X.X) while
allowing us to continue to use our docker volume caching mechanism
in jenkins
Change-Id: I9ae8f660709ce4e261ac064ce294740264166882
Reviewed-on: https://gerrit.instructure.com/133015
Reviewed-by: Simon Williams <simon@instructure.com>
Tested-by: Jenkins
Product-Review: Shahbaz Javeed <sjaveed@instructure.com>
QA-Review: Shahbaz Javeed <sjaveed@instructure.com>
Refs: TS-3299
We think the problem is that we're using old data files with a newer
version of postgis, there is already a script to fix the problem we just
need to run it.
Change-Id: I666deb0efde95738aaa44a72a7656e1a4b287ac7
Reviewed-on: https://gerrit.instructure.com/132927
Reviewed-by: Shahbaz Javeed <sjaveed@instructure.com>
Tested-by: Jenkins
Reviewed-by: Simon Williams <simon@instructure.com>
Product-Review: Tyler Pickett <tpickett@instructure.com>
QA-Review: Tyler Pickett <tpickett@instructure.com>
Derek Bender