Create file-based pipeline for BulkImports::Groups::Pipelines::SubgroupEntitiesPipeline
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
As part of leveraging our existing direct transfer service for offline transfers, we will need to add new file-based pipelines for pipelines in direct transfer that make API requests to the source instance during any part of the ETL pipeline. This new pipeline should not replace the existing pipeline and it should only be executed during offline transfer.
This issue is to create a pipeline compatible with offline transfer for BulkImports::Groups::Pipelines::SubgroupEntitiesPipeline. The current direct transfer pipeline uses a REST extractor to fetch subgroup entites. In offline transfer, entities will need to be fetched from an NDJSON file in object storage.