[go: up one dir, main page]

POC: use gitlab-qa to create an omnibus with aigw and run VSCode tests against it

To verify whether it is feasible to run VSCode IDE tests in the gitlab monolith pipeline.

Answer
Manual smoke test - code suggestion responds with prompt
Manual smoke test - duo chat - responds with mock
Will tests need to be modified? See #520658 (comment 2369651502)
Will mocks need to be modified? I don't think so, perhaps for Duo Workflow if required, but this test only opens the webview currently
What token will be needed? We need a valid token for the GitLab instance under test, since the instance has access to Duo, any token should be OK
We would need to somehow obtain a PAT for a test user before kicking off the VSCode tests We can use the DEFAULT_ADMIN_API_TOKEN, since the AI gateway is mock response mode
How do we obtain the gitlab URL? The current omnibus tests use http://gitlab.test, but when running locally this doesn't work for me 🤔 However, http://localhost:32769 works locally. Perhaps this is a mac problem, we may just have to try it in CI https://stackoverflow.com/questions/40125571/docker-for-mac-hostname-setup-for-container
Limitations OAuth is not implemented for SM #509257 (comment 2371749445)

Steps to run VSCode extensions tests against a GitLab Self Managed omnibus instance (integrated with ai-gateway in mock response mode

From the monolith repo:

cd qa
bundle install
# the following uses the nightly image, but there are env variables to use different images/versions of gitlab
CHROME_DISABLE_DEV_SHM=true  DOCKER_DEFAULT_PLATFORM=linux/amd64 GITLAB_QA_ACCESS_TOKEN=<i think this is just an access token to download the docker images> GITLAB_LICENSE_MODE=test QA_EE_ACTIVATION_CODE=1Password bundle exec gitlab-qa Test::Integration::AiGateway EE --no-tests

From the vscode repo:

<from the vscode ext dir>
cd test/e2e
❯ E2E_GITLAB_HOST=http://localhost:32769 TEST_GITLAB_TOKEN=<see link below for token> npm run test:e2e

Token value

Some tests will fail, see: #520658 (comment 2369651502)

Summary

A subset of the VSCode E2E tests from https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/tree/main/test/e2e can run successfully against an Omnibus instance of GitLab configured to communicate to an ai gateway in mock response mode.

The same DEFAULT_ADMIN_API_TOKEN can be used for these tests, there is a question over which host we will need to use, locally I had to use http://localhost:32769 but in CI we may be able to use http://gitlab.test

OAuth will not work in this set-up as it is not implemented in the vscode extension for SM instances yet

Edited by Jay McCure