[go: up one dir, main page]

Skip to content

Log every time an import is stopped before finishing

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem

There are places in the Importer code (for GitHub and other import types) where we stop an import without any logging or error raising. We should make these places easier to track down for debugging purposes.

More context

Discovered while working on a POC. That POC contains comments where the import is silently stopped.

We shouldn't expect an import to stop if all is working correctly, so these places should be logged and raise errors.

Some examples of where imports stop silently:

  • app/workers/concerns/gitlab/github_import/stage_methods.rb : the find_project method returns nil if the project is not in an import state of started. Right now, the #perform method in this class does an early return if the project is not in an import state of started. This might be an expected scenario, but we should at least log if this happens. Done in !140385 (comment 1703281926).
  • app/workers/gitlab/import/advance_stage.rb: If the import state is nil the project may have been deleted or the import may have failed or been canceled. In that scenario, we tidy up the cache value and no longer attempt to advance to the next stage. This might be an expected scenario, but we should at least log if this happens.
  • lib/gitlab/github_import/parallel_scheduling.rb: import silently fails due to next unless page_counter.set(page.number). This will happen if the page counter cache was already set. This might be an expected scenario, but we should at least log if this happens.
  • app/services/projects/create_service.rb : import silently never starts if @project.import_state.schedule returns nil, which would happen if the project's import state is something other than none finished or failed. That state transition is what kicks off the project import to begin with, but if the state isn't correct then it fails silently. We should log and probably raise an error in this scenario, even though it is happening in the background.

As discussed in the refinement issue, we may want to find other examples of where an import can stop. This activity should be timeboxed.

Edited by 🤖 GitLab Bot 🤖