[go: up one dir, main page]
Skip to content

POC: investigate pre-installing GitLab Workflow in WebIDE

This issue describes the process for integrating GitLab Workflow in WebIDE. The introduction is meant for anyone interested in the topic. The collapsed sections contain technical details that will give you better insight into how we'll migrate Workflow to WebIDE, but are not essential for getting the big picture.

You can watch this video explaining the result of the POC. If you just want to see it working, jump to 7th minute of the video.

What is an extension?

VS Code editor's killer feature is extensions. VS Code makes it very easy to extend its functionality using extensions. Slightly simplified, an extension is a manifest file and JavaScript (or TypeScript) module that exports the activate() method. VS Code loads such a module and gives it access to the Extension API. This module can then control the editor.

Workflow, WebIDE and Code Suggestions

  • GitLab Workflow is the name of GitLab VS Code Desktop extension. You can install it in your Desktop VS Code. See a screenshot of the extension running in Desktop VS Code
    • Screenshot_2023-05-18_at_10.37.51
  • GitLab WebIDE is our fork of VS Code for the browser (think https://vscode.dev) with a few added extensions to make the vanilla VS Code understand GitLab. See the screenshot of the WebIDE running on the gitlab.com page:
    • Screenshot_2023-05-18_at_10.41.45
  • The Code Suggestions is a feature that is:
  • The Code Suggestions logic in the WebIDE and Workflow is 80% identical. The remaining 20% is mainly authentication.

Workflow is not currently compatible with WebIDE

  • Workflow is a desktop extension, meaning it runs in Node.js and has full access to the host system. It can do things that are impossible in the browser:
    • Make any HTTP calls (changing headers, ignoring CORS)
    • Creating and manipulating files on the file system
    • Has full access to the Git repository
    • Uses OS Keychain to store secrets
    • ...
  • It's also got some Desktop-specific concepts that are not relevant in WebIDE:
    • Support for multiple accounts (e.g. personal and work account on GitLab.com)
    • Support for multiple projects being opened at the same time (multiple microservices, backend and frontend projects)

Bringing Workflow to WebIDE

We have to abstract away the desktop-specific APIs (e.g. file system access) and the desktop-specific concepts (e.g. multiple projects). After that, most of the logic will be fully reusable between desktop and WebIDE (as long as it doesn't need platform-specific functionality like file system access)

Example 1: Code Suggestions

The code suggestions feature has three dependencies:

  • VS Code Extension API - it gets text from the current file and then suggests completion
  • GitLab API - it needs to authenticate to the GitLab instance and obtain JWT token
  • Model Gateway - it uses the JWT token to authenticate requests for suggestions

Only the access to GitLab API differs on the desktop and browser. The extension will use API abstraction that will be implemented differently in the WebIDE and Destkop VS Code.

Example 2: Show issues assigned to me:

The desktop extension can show you all issues assigned to you in the VS Code sidebar. It shows a list of all projects, and for each project, it can execute the "Issues assigned to me" query and show the resulting issues.

The desktop extension lists all Git repositories that are open, and then it tries to match the git remotes in these repositories with active accounts. It comes up with a list of valid projects, and for each, it will use the account's access token to execute the API query.

We will abstract the logic for obtaining valid projects, and in WebIDE, it will always return the one open project. Then we will abstract away the API calls, and instead of using the account token stored in the OS keychain, we'll use the X-CSRF-Token header and a cookie to authenticate the API requests.

Technical details

Technical details

Explain WebIDE

explain-web-ide

  • gitlab-web-ide-vscode-fork project is our fork of Microsoft's VS Code with a few changes made to the core VS Code
  • gitlab-web-ide several build scripts and TypeScript modules and extensions that change the vanilla VS Code behaviour to understand and work well with GitLab. Most of the WebIDE is implemented here
    • The `ai-assisted-code-suggestions extension for WebIDE is versioned in this repository
    • gitlab-web-ide project has a single output artifact - @gitlab/web-ide NPM module (roughly 50MB). The main GitLab app uses this module.

How WebIDE starts?

  1. Inside the main GitLab app, we create a <div> element and pass in all information needed for starting WebIDE as data- attributes.
  2. Then we import the web-ide module (roughly 50MB) and call start(div)
  3. From here, the web-ide module creates an iframe, establishes communication between the iframe and the rest of the page (e.g. for snowplow tracking) and inside of the iframe, starts the VS Code (starting VS Code is the responsibility of the vscode-bootstrap module)

How WebIDE features get init data and communicate with GitLab API

Explain the Workflow

explain-workflow

  • GitLab Workflow is a VS Code desktop extension that integrates VS Code editor with GitLab. Some of the main features are:
    • AI-Assisted Code Suggestions
    • MR Review
    • CI/CD status overview
  • The desktop environment for the extensions is much more powerful than the web environment. On the diagram, you can see that the extension can access the file system, run arbitrary commands and use Git.
  • Another key part is that in GitLab Workflow on the Desktop, you can have multiple accounts and multiple projects.

Share logic between Desktop and WebIDE

explain-sharing

  • We'll create two entry files, one for the browser and one for the desktop. We will create a common abstraction for executing API queries and getting the active project. Many features, including the Code Suggestions, can function only on this limited abstraction. We can then start changing existing features to use these new abstractions and release them individually.
  • On the diagram, we can see how each platform (browser & desktop) implements the API calls and getting current project separately.
POC code explained

POC code explained

The POC can be found in It can be found in the spike-integrate-workflow branch in the WebIDE project.

The POC is implemented in two projects, the WebIDE and the GitLab Workflow.

Workflow

Entry points

The POC implements a simple command called hello that will show the user the current opened project. Then the POC changes the Workflow build process to create two builds -- desktop and browser. These builds have different entry points:

These are used for building the final extension in these package.json scripts

Merging pakcage.json files

VS Code needs metadata stored in package.json to initialize the extension. The metadata contains information like:

  • what commands are available
  • extension icon
  • extension settings
  • what context menus and panels does the extension contribute

The desktop and browser versions of the extension won't have the same features (at the beginning, browser version will have a small subset of the features), and so we can't use one package.json for both.

The POC divides the metadata into 3 files:

We then use the merge_package_jsons.js script to create the final desktop or browser package.json

Main abstraction - ProjectManager

The extension uses ProjectManager abstraction, which let's you get the current project and execute API request:

export interface ProjectManager {
  project: GitLabProject;
  execute<T>(request: GraphQLQuery<T>): Promise<T>;
}

The shared logic then receives ProjectManager.

The project manager is implemented differently:

  • desktop

    const mg: MangerGetter = async () => {
  const projectInRepository = await getActiveProjectOrSelectOne();
  if (!projectInRepository) {
    throw new Error('no active project');
  }
  return {
    project: projectInRepository.project,
    execute: query => getGitLabService(projectInRepository).execute(query),
  };
};

  • browser

    const manager: projectmanager = {
  project,
  execute: async <t>(request: graphqlquery<t>): promise<t> =>
    request.processresult(await vscode.commands.executecommand(command_execute, request)),
} as unknown as projectmanager;

You can see that the desktop version uses existing logic to get the GitLab project and GitLabService and the browser version uses mediator commands to get the project and execute API requests.

WebIDE

In WebIDE, we added Git submodule with the Workflow extension.Then we changed the Makefile to include tasks to build and package the Workflow extension.

Supporting the ProjectManager abstraction.

To execute API requests, we had to introduce a new mediator command execute. Any extension running in the WebIDE can leverage this command by calling gitlab-web-ide.mediator.execute command

Issue breakdown

Issue breakdown

Investigation

  • WebIDE: Is Git submodule the best way to include Workflow Extension?
  • WebIDE & Workflow: Hot to pass feature flags to the extension?
  • Workflow: How to abstract away multiple projects and multiple accounts?
  • WebIDE: Spike replacing Mediator commands with trusted OAuth app
  • WebIDE: Debugging extensions that have sorucemaps included

Implementation

  • Workflow: Change the build process to produce two dist folders, one for browser, one for desktop.
  • Workflow: Add REST GET and POST requests to the GitLabService.execute() method
    • verifies the execute() interface design, then we can implement it in WebIDE
  • WebIDE: Implement the execute() mediator command and migrate the FETCH_PROJECT mediator command to use it
  • WebIDE: Add the workflow as a git submodule and change Makefile to include the GitLab Workflow extension
  • Workflow: Migrate Code Suggestions to use ProjectManager.
  • WebIDE: Deprecate Code Suggestions in favour of Workflow
Concerns

Concerns - what will be tricky when porting Workflow

Authentication

captured here

  • Workflow
    • The Workflow uses two ways of authenticating:
      • OAuth for GitLab.com
      • Personal Access Token (PAT) for any GitLab instance
    • The Workflow also supports multiple accounts
  • WebIDE
    • WebIDE runs in an already-authenticated GitLab page and uses mediator commands to make requests the same way as any other frontend JavaScript in GitLab does.
    • This approach will become an issue when we allow loading 3rd party extensions. Those extensions will also be able to invoke the mediator commands and invoke GitLab API.
    • We will have to either introduce some authentication to the mediator commands or use OAuth instead.

Injecting settings and feature flags

captured here

We don't have a good way of passing config and feature flags to WebIDE. We will need to design a unified approach that will work for browser and desktop.

Developer experience

The WebIDE developer experience is not optimal. We rely heavily on make and copying files around. The yarn multi-package workspace also bit us many times. Now we'll bring another project into the mix (GitLab Workflow). It's critical to integrate it in a way that will give contributors quick feedback and will work seamlessly.

Edited by Stanislav Lashmanov