Fix Bitbucket importer status page timing out and add pagination
What does this MR do and why?
This MR updates Bitbucket Importer status page and introduces infinite scroll, similar to github importer. This is needed because we have users who have extreme amounts of repositories (multiple thousands) and status page times out, which makes it impossible to migrate the projects.
It uses cursor based pagination, where we:
- Fetch the list of repositories, ordered by created_at
- Check
aftervalue in the response, which should include the next page of projects - As we scroll down, next page should be populated
References
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
- Setup bitbucket OAuth https://docs.gitlab.com/integration/bitbucket/
- Create 25+ projects on bitbucket so that pagination can be tested (or change this constant to a lower value)
- Go to http://localhost:3000/import/bitbucket/status and observe network tab. Requests to
status.jsonendpoint should be made withafterquery string containing a created_at cursor/date - Keep scrolling. New projects should keep appearing until all projects are listed
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by George Koltsov
