Improve performance of diff_files action for Projects::CommitController under load into next tier

The TTFB (Time to First Byte) of the diff_files action for Projects::CommitController is over our performance target under load:

* Environment:                10k
* Environment Version:        16.1.0-pre `7b978d00a56`
* Option:                     60s_200rps
* Date:                       2023-06-13
* Run Time:                   1h 39m 22.73s (Start: 04:55:18 UTC, End: 06:34:40 UTC)
* GPT Version:                v2.13.0

NAME         | RPS   | RPS RESULT           | TTFB AVG  | TTFB P90              | REQ STATUS     | RESULT 
---------------------------------------------------------|-------|----------------------|-----------|-----------------------|----------------|--------
web_project_commit     | 20/s  | 18.96/s (>6.40/s)    | 619.99ms  | 1796.29ms (<3500ms)   | 100.00% (>99%) | Passed¹

 █ setup
     █ Web - Commit Details Page
     data_received.....................................................: 214 MB  3.5 MB/s
     data_sent.........................................................: 1.3 MB  21 kB/s
     group_duration....................................................: avg=3839.70ms min=2182.33ms med=3926.44ms max=5600.12ms p(90)=4169.76ms p(95)=4281.64ms
     http_req_blocked..................................................: avg=0.08ms    min=0.00ms    med=0.00ms    max=23.56ms   p(90)=0.00ms    p(95)=0.00ms   
     http_req_connecting...............................................: avg=0.01ms    min=0.00ms    med=0.00ms    max=0.91ms    p(90)=0.00ms    p(95)=0.00ms   
     http_req_duration.................................................: avg=673.60ms  min=83.97ms   med=419.76ms  max=3531.12ms p(90)=1956.43ms p(95)=2042.94ms
       { expected_response:true }......................................: avg=673.60ms  min=83.97ms   med=419.76ms  max=3531.12ms p(90)=1956.43ms p(95)=2042.94ms
     http_req_failed...................................................: 0.00%   ✓ 0         ✗ 1181
     http_req_receiving................................................: avg=1.59ms    min=0.03ms    med=0.65ms    max=51.73ms   p(90)=4.09ms    p(95)=4.69ms   
     http_req_sending..................................................: avg=0.11ms    min=0.04ms    med=0.11ms    max=0.27ms    p(90)=0.14ms    p(95)=0.15ms   
     http_req_tls_handshaking..........................................: avg=0.05ms    min=0.00ms    med=0.00ms    max=4.07ms    p(90)=0.00ms    p(95)=0.00ms   
     http_req_waiting..................................................: avg=671.90ms  min=83.78ms   med=418.50ms  max=3526.81ms p(90)=1950.60ms p(95)=2037.30ms
     ✓ { controller:Projects::CommitController#branches }..............: avg=119.24ms  min=83.78ms   med=102.07ms  max=1659.82ms p(90)=130.82ms  p(95)=142.56ms 
     ✓ { controller:Projects::CommitController#diff_files }............: avg=1938.81ms min=1694.05ms med=1885.03ms max=3526.81ms p(90)=2120.33ms p(95)=2291.37ms
     ✓ { controller:Projects::CommitController#merge_requests.json }...: avg=119.77ms  min=92.71ms   med=113.88ms  max=1317.26ms p(90)=134.03ms  p(95)=142.11ms 
     ✓ { controller:Projects::CommitController#show }..................: avg=509.77ms  min=409.92ms  med=484.08ms  max=1548.07ms p(90)=615.31ms  p(95)=644.09ms 
   ✓ http_reqs.........................................................: 1181    19.069136/s
     ✓ { controller:Projects::CommitController#branches }..............: 295     4.763247/s
     ✓ { controller:Projects::CommitController#diff_files }............: 295     4.763247/s
     ✓ { controller:Projects::CommitController#merge_requests.json }...: 295     4.763247/s
     ✓ { controller:Projects::CommitController#show }..................: 295     4.763247/s
     iteration_duration................................................: avg=3829.16ms min=707.99ms  med=3926.13ms max=5600.16ms p(90)=4169.36ms p(95)=4281.62ms
     iterations........................................................: 295     4.763247/s
   ✓ successful_requests...............................................: 100.00% ✓ 1180      ✗ 0   
     vus...............................................................: 6       min=2       max=20
     vus_max...........................................................: 20      min=20      max=20

Test Details

Testing was done on our 10k Reference Architecture environment with our lab condition GitLab Performance Tool pipeline. The project being tested is a copy of gitlabhq (tarball can be found here). GitLab Performance Tool tests information is listed at Current test details page.

The latest GitLab Performance pipeline results can always be found here. Through this page full Server Metrics can be found via the Metrics Dashboard link on that page.

As per our performance targets this endpoint's TTFB metric is above the target of 2000 ms which is severity2. Task is to improve the endpoint's performance into next tier.