Rouge::Lexer.guess can timeout and cause 500 errors

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Summary

When viewing certain files in the UI, or searching via Advanced Search, the Rouge::Lexer.guess can take a long time, leading to timeouts and 500 errors.

This was the case with a particular customer. Zendesk ticket refers (internal).

Steps to reproduce

1. Add bootstrap.bundle.js.map from Bootstrap to a repository
2. Enable Advanced Search indexing
3. View file in UI
4. Search for file in Advanced Search

Example Project

What is the current bug behavior?

Rack::Timeout::RequestTimeoutException returned. duration_s can reach 60 seconds, and cpu_s is the majority of this time. This results in 500 errors.

What is the expected correct behavior?

Improve performance of syntax highlighting to avoid timeouts

Relevant logs and/or screenshots

"exception.class":"Rack::Timeout::RequestTimeoutException","exception.message":"Request ran for longer than 60000ms ","exception.backtrace":["app/presenters/blob_presenter.rb:223:in `detect_language'","app/presenters/blob_presenter.rb:51:in `blob_language'","app/presenters/blob_presenter.rb:19:in `highlight_and_trim'","app/views/search/results/_blob_highlight.html.haml:7","app/views/search/results/_blob_data.html.haml:12","app/views/search/results/_blob.html.haml:8","app/helpers/application_helper.rb:14:in `render_if_exists'","app/views/search/_results_list.html.haml:18","app/views/search/_results.html.haml:9","app/views/search/show.html.haml:24","app/controllers/application_controller.rb:142:in `render'","app/controllers/application_controller.rb:573:in `block in allow_gitaly_ref_name_caching'","lib/gitlab/gitaly_client.rb:352:in `allow_ref_name_caching'","app/controllers/application_controller.rb:572:in `allow_gitaly_ref_name_caching'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/app/controllers/ee/application_controller.rb:46:in `set_current_ip_address'","app/controllers/application_controller.rb:524:in `set_current_admin'","lib/gitlab/session.rb:11:in `with_session'","app/controllers/application_controller.rb:515:in `set_session_storage'","lib/gitlab/i18n.rb:107:in `with_locale'","lib/gitlab/i18n.rb:113:in `with_user_locale'","app/controllers/application_controller.rb:506:in `set_locale'","app/controllers/application_controller.rb:499:in `set_current_context'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/memory_report.rb:13:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/web_transaction.rb:46:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/middleware/query_analyzer.rb:11:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/compressed_json.rb:37:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call'","lib/gitlab/middleware/release_env.rb:13:in `call'"]

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)

Workaround

Add the extension to .gitattributes file.

Example:

*.js.map gitlab-language=text

Possible fixes