meta.auth_fail_reason present for 200 requests

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

In https://log.gprd.gitlab.net/app/r/s/6JlIj, we are seeing a number of 200 requests have a meta.auth_fail_reason mostly of insufficient_scope:

My guess is that validate_and_save_access_token! is being called with some other scope (e.g. k8s_proxy) and that fails for one set of scopes but succeeds for another.

We should probably log which scopes are being requested:

diff --git a/lib/gitlab/auth/auth_finders.rb b/lib/gitlab/auth/auth_finders.rb
index 7288245fc0a9..f4d5a91d431f 100644
--- a/lib/gitlab/auth/auth_finders.rb
+++ b/lib/gitlab/auth/auth_finders.rb
@@ -194,18 +194,18 @@ def validate_and_save_access_token!(scopes: [], save_auth_context: true)
 
         case AccessTokenValidationService.new(access_token, request: request).validate(scopes: scopes)
         when AccessTokenValidationService::INSUFFICIENT_SCOPE
-          save_auth_failure_in_application_context(access_token, :insufficient_scope) if save_auth_context
+          save_auth_failure_in_application_context(access_token, :insufficient_scope, scopes) if save_auth_context
           raise InsufficientScopeError, scopes
         when AccessTokenValidationService::EXPIRED
-          save_auth_failure_in_application_context(access_token, :token_expired) if save_auth_context
+          save_auth_failure_in_application_context(access_token, :token_expired, scopes) if save_auth_context
           raise ExpiredError
         when AccessTokenValidationService::REVOKED
-          save_auth_failure_in_application_context(access_token, :token_revoked) if save_auth_context
+          save_auth_failure_in_application_context(access_token, :token_revoked, scopes) if save_auth_context
           revoke_token_family(access_token)
 
           raise RevokedError
         when AccessTokenValidationService::IMPERSONATION_DISABLED
-          save_auth_failure_in_application_context(access_token, :impersonation_disabled) if save_auth_context
+          save_auth_failure_in_application_context(access_token, :impersonation_disabled, scopes) if save_auth_context
           raise ImpersonationDisabled
         end
 
@@ -224,10 +224,12 @@ def save_current_token_in_env
         request.env[API_TOKEN_ENV] = { token_id: access_token.id, token_type: access_token.class.to_s }
       end
 
-      def save_auth_failure_in_application_context(access_token, cause)
+      def save_auth_failure_in_application_context(access_token, cause, requested_scopes)
         Gitlab::ApplicationContext.push(
           auth_fail_reason: cause.to_s,
-          auth_fail_token_id: "#{access_token.class}/#{access_token.id}")
+          auth_fail_token_id: "#{access_token.class}/#{access_token.id}",
+          auth_fail_requested_scopes: requested_scopes
+        )
       end
 
       def find_user_from_job_bearer_token