POC: use gitlab-qa to create an omnibus with aigw and run VSCode tests against it
To verify whether it is feasible to run VSCode IDE tests in the gitlab monolith pipeline.
| Answer | |
|---|---|
| Manual smoke test - code suggestion |
prompt
|
| Manual smoke test - duo chat |
mock
|
| Will tests need to be modified? | See #520658 (comment 2369651502) |
| Will mocks need to be modified? | I don't think so, perhaps for Duo Workflow if required, but this test only opens the webview currently |
| What token will be needed? | We need a valid token for the GitLab instance under test, since the instance has access to Duo, any token should be OK |
| We would need to somehow obtain a PAT for a test user before kicking off the VSCode tests | We can use the DEFAULT_ADMIN_API_TOKEN, since the AI gateway is mock response mode |
| How do we obtain the gitlab URL? | The current omnibus tests use http://gitlab.test, but when running locally this doesn't work for me |
| Limitations | OAuth is not implemented for SM #509257 (comment 2371749445) |
Steps to run VSCode extensions tests against a GitLab Self Managed omnibus instance (integrated with ai-gateway in mock response mode
From the monolith repo:
cd qa
bundle install
# the following uses the nightly image, but there are env variables to use different images/versions of gitlab
CHROME_DISABLE_DEV_SHM=true DOCKER_DEFAULT_PLATFORM=linux/amd64 GITLAB_QA_ACCESS_TOKEN=<i think this is just an access token to download the docker images> GITLAB_LICENSE_MODE=test QA_EE_ACTIVATION_CODE=1Password bundle exec gitlab-qa Test::Integration::AiGateway EE --no-tests
From the vscode repo:
<from the vscode ext dir>
cd test/e2e
❯ E2E_GITLAB_HOST=http://localhost:32769 TEST_GITLAB_TOKEN=<see link below for token> npm run test:e2e
Some tests will fail, see: #520658 (comment 2369651502)
Summary
A subset of the VSCode E2E tests from https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/tree/main/test/e2e can run successfully against an Omnibus instance of GitLab configured to communicate to an ai gateway in mock response mode.
The same DEFAULT_ADMIN_API_TOKEN can be used for these tests, there is a question over which host we will need to use, locally I had to use http://localhost:32769 but in CI we may be able to use http://gitlab.test
OAuth will not work in this set-up as it is not implemented in the vscode extension for SM instances yet
Edited by Jay McCure