Output go-redis internal log messages to JSON

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

The following discussion from !128204 (merged) should be addressed:

• @stanhu started a discussion: (+8 comments)

  On an Omnibus instance, I'm not seeing a SUBSCRIBE message when I ran redis-cli monitor on my Redis instance. Here are the Workhorse log messages:

  {"build_time":"20230821.060031","level":"info","msg":"Starting","time":"2023-08-21T06:10:46Z","version":"11-10-0cfa69752d8-74ffd66ae-ee-226562-gc5e1c2937ce6"}
  {"address":"localhost:9229","level":"info","msg":"Running metrics server","network":"tcp","time":"2023-08-21T06:10:46Z"}
  {"level":"info","msg":"Using redis/go-redis","time":"2023-08-21T06:10:46Z"}
  {"level":"info","msg":"keywatcher: starting process loop","time":"2023-08-21T06:10:46Z"}
  {"address":"0.0.0.0:1234","level":"info","msg":"Running upstream server","network":"tcp","time":"2023-08-21T06:10:46Z"}
  redis: 2023/08/21 06:11:47 pubsub.go:168: redis: discarding bad PubSub connection: EOF
  {"error":"keywatcher: pubsub receive: EOF","level":"error","msg":"","time":"2023-08-21T06:11:47Z"}

  Notice the redis error log message shows up not in JSON format.

  My key settings in /etc/gitlab/gitlab.rb:

  redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one
  redis['port'] = 6379
  
  ##! duration after which we timeout requests if they sit too long in the queue
  gitlab_workhorse['api_queue_duration'] = "30s"
  
  ##! Long polling duration for job requesting for runners
  gitlab_workhorse['api_ci_long_polling_duration'] = "60s"
  
  gitlab_workhorse['env'] = {
    'GITLAB_WORKHORSE_FF_GO_REDIS_ENABLED' => 'true'
  }

This can be a separate issue, but we should figure out how if it's possible to output the internal go-redis log messages to JSON. I see https://github.com/redis/go-redis/pull/2609, so it may not be possible yet.