add contract testing readme

refs QUIZ-4175

Test Plan:
  - qa-cr

Change-Id: I8713c0dd007000037e774136ee37770eb90c9970
Reviewed-on: https://gerrit.instructure.com/149118
Tested-by: Jenkins
Reviewed-by: John Corrigan <jcorrigan@instructure.com>
Product-Review: John Corrigan <jcorrigan@instructure.com>
QA-Review: John Corrigan <jcorrigan@instructure.com>
This commit is contained in:
Michael Hargiss 2018-05-03 14:41:16 -04:00
parent 6c4ce74db4
commit 89f287444a
1 changed files with 95 additions and 0 deletions

95
spec/contracts/README.md Normal file
View File

@ -0,0 +1,95 @@
# Contract Tests
Canvas LMS has numerous API clients and Live Events subscribers. The purpose
behind contract tests is to prevent regressions in the API responses' and Live
Events messages' schemas for a given API request or Live Event subscription.
We follow a decoupled contract testing paradigm, as suggested by [Martin Fowler]
and the folks at Thoughtworks. This allows microservices to retain independent
CI and deployment pipelines. We use [Pact] to make this happen.
We use a Pact Broker web service to share Pact files between consumers and
providers. Ours is hosted internally at https://inst-pact-broker.inseng.net
Ask for credentials in the #test_advisory_board Slack channel.
## Running the tests locally
Running the contracts tests is a simple three step process:
1. Generate the contract(s)
2. Share the contract(s)
3. Verify the contract(s)
### Generate the Contracts
API clients and Live Events subscribers are responsible for defining,
generating, and publishing Pact files so that Canvas LMS can import and verify
them. Each Canvas LMS API client and Canvas LMS Live Event subscriber who's
integrated with Pact should have instructions in their own repo for generating
their Pact file(s).
If you'd like to see this in action, you're welcome to generate Quiz LTI's Pact
files:
```sh
cd path/to/quiz_lti # clone the repo first if necessary
bin/dev-setup
bin/contracts-generate
```
All specs should pass and Quiz LTI should generate the Pact files. The script
will fail, however, because it's trying to publish those Pact files to a Pact
Broker. Let's spin that up now.
### Share the Contracts
The Pact Broker is a simple web application with a database for storing Pact
files.
To use a Pact Broker on your computer:
1. Clone the `quiz_pact_broker` repo
2. `cd quiz_pact_broker`
3. Execute:
```sh
bin/dev-setup && open http://pact:broker@pact-broker.docker
```
Now you're ready to publish a Pact file to the Pact Broker. Let's run that Quiz
LTI script again. It should finish successfully this time.
```sh
cd path/to/quiz_lti
bin/contracts-generate
```
Now refresh the Pact Broker in your web browser. You should see at least two
Pact files there, one of which contains contracts between "Quiz LTI" and "Canvas
LMS Live Events". Click on the document icon to view the contract, if you like.
### Verify the Contracts
To verify the Live Events contracts for Canvas LMS, start Canvas on your
computer---either in docker or natively---then run:
```sh
bin/contract-verify-live-events
```
The spec(s) should pass. You're finished!
Note: As of 3 May 2018, no API clients have published a Pact file for Canvas to
verify. Once they do, we'll add a script to verify them also.
## More Info
If you'd like to learn more about contract testing and Pact, head over to the
[Test Advisory Board github repo]. There you'll find links to industry articles,
internal documentation, simple Pact code examples, and more.
You're also welcome to stop by the #test_advisory_board Slack channel!
[Martin Fowler]: https://martinfowler.com/articles/microservice-testing/#testing-contract-introduction
[Pact]: https://docs.pact.io/
[Test Advisory Board github repo]: https://github.com/instructure/test_advisory_board