Apply import rate limits consistently to all importers and endpoints

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Problem

Project import/export and Direct Transfer have rate limits, but the others don't.

And they are inconsistently applied (group imports are only rate-limited in the UI and not through the API #285137).

Some of the rate limits are configurable in application settings but the Direct Transfer limit isn't.

Solution

We should:

• Apply rate limits consistently to all our importers (all endpoints)
• Allow all of those limits to be configurable through application settings (currently only file-based import and export rate limits are).

Questions

The existing rate limits configurable through application settings are group_import, group_export, project_import and project_export. These can be set to different values.

Do we want to continue to add new limits to application settings per importer, or should we consolidate towards a single rate limit that can be set to cover all importers?

File-based export endpoints are rate-limited, but do we want to do the same for Direct Transfer export endpoints, or will this likely cause failures in DT. I.e., should we skip adding rate limits for DT export endpoints?