Don't die if consul data is missing

closes CNVS-27672

This could stop all API requests from working
if consul suddenly catastrophically loses it's data.
With the patch in place, we'll still fail for
requests that need that data to decrypt (JWT), but regular
access tokens will still work ok.  This is better than
total system failure.

TEST PLAN:
 1) connect to consul, but put no data in it (particularly no
signing/encryption secrets)
 2) hit canvas with an API request using a regular access token
 3) it should not blow up

Change-Id: Ia435814929968373f22ab8d120153368267a3f32
Reviewed-on: https://gerrit.instructure.com/73492
Reviewed-by: Cody Cutrer <cody@instructure.com>
Tested-by: Jenkins
QA-Review: Jeremy Putnam <jeremyp@instructure.com>
Product-Review: Ethan Vizitei <evizitei@instructure.com>

lib/authentication_methods.rb

@@ -79,10 +79,17 @@ module AuthenticationMethods
     rescue JSON::JWT::InvalidFormat,             # definitely not a JWT
            Canvas::Security::TokenExpired,       # it could be a JWT, but it's expired if so
            Canvas::Security::InvalidToken,       # Looks like garbage
-           Canvas::DynamicSettings::ConsulError, # no config present for talking to consul
-           Faraday::ConnectionFailed             # consul config present, but couldn't connect
-      # could still be a regular access token
+           Canvas::DynamicSettings::ConsulError  # no config present for talking to consul
+      # these will happen for some configurations (no consul)
+      # and for some normal use cases (old token, access token),
+      # so we can return and move on
       return
+    rescue  Faraday::ConnectionFailed,            # consul config present, but couldn't connect
+            Faraday::ClientError,                 # connetion established, but something went wrong
+            Diplomat::KeyNotFound => exception    # talked to consul, but data missing
+      # these are indications of infrastructure of data problems
+      # so we should log them for resolution, but recover gracefully
+      Canvas::Errors.capture_exception(:jwt_check, exception)
     end
   end
 

spec/apis/auth_spec.rb

@@ -592,6 +592,12 @@ describe "API Authentication", type: :request do
       expect(JSON.parse(response.body).size).to eq 1
     end
 
+    it "recovers gracefully if consul is missing encryption data" do
+      Diplomat::Kv.stubs(:get).raises(Diplomat::KeyNotFound, "cannot find some secret")
+      check_used { get "/api/v1/courses", nil, { 'HTTP_AUTHORIZATION' => "Bearer #{@token.full_token}" } }
+      assert_status(200)
+    end
+
     it "should allow passing the access token in the post body" do
       @me = @user
       Account.default.account_users.create!(user: @user)