Discussion: Consider performing diff highlighting on the frontend instead of backend

There are a lot of questions we should answer about our highlighting process for diffs.

Below is an unordered list of some of those questions:

• Why does Rouge (the current highlighter) not fail in the same ways that Highlight.js (our trial front end highlighter) fails?
  • Can we make up the difference with upstream patches?
  • One type of Highlight.js failure - not having the correct LoC-context, and so failing to properly parse it - is not intrinsic to Highlight.js. How does Rouge solve this?
• Can we run something like Rouge - or a faster (e.g. C++) highlighter - in the browser via WASM/WebAssembly?
• If a failure mode of front end highlighting is not having enough context, how can we deliver enough context to the front end without having to deliver the entire file?
• What is the overall performance of Highlight.js in comparison to Rouge
  • Since we can highlight only visible files on the front end (e.g. IntersectionObserver highlights the next file), how can we compare "on-demand" highlighting?
• Is Rouge the right back end highlighter?
• Is Highlight.js the right front end highlighter?
• In the future, we will want to treat lines as just one abstraction of a diff. How can our highlighting solution enable us to keep treating code as a cohesive object?
  • For example: If we want to eventually comment on part of some code AST (e.g. commenting on a function name, not the line it appears in), we need to be able to step outside of a fully highlighted object and interact directly with the code itself.