Create script and document how to create a Project Exports using Direct Transfer

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Work on this issue
• Close this issue

The Direct Transfer Export Relation API exports nearly all the files needed to create a Project Export archive. Additionally, because it exports files in smaller batches, it can successfully export large Projects that cannot be exported using a Project file base.

A script and documentation on how to use the Direct Transfer Export Relation API to create a Project Export archive can serve as an alternative for failed project exports.

Please note that the Direct Transfer Export Relation API does not export members, badges, wiki bundles, and snippet bundles. The script must manually export these relations using REST or GraphQL.

Overall script steps

1. Trigger relations export
2. Wait exports to complete
3. Download all relations files
4. For batched relations, concatenate them as a single ndjson file
5. Fetch members via API and create the project_members.ndjson file
6. Create metadata files (GITLAB_REVISION, GITLAB_VERSION, VERSION)
7. Fetch badges via Rest API and create badges.ndjson
8. Clone snippets and create bundle files
9. Clone wiki and create bundle file
10. Move all files to a temporary folder. The files should follow the Project archive folder structure:

GITLAB_REVISION
GITLAB_VERSION
VERSION
lfs-objects.json
lfs-objects/
project.bundle
project.wiki.bundle
snippets/
uploads/
tree
  project.json
  project
    auto_devops.ndjson 
    boards.ndjson
    ci_cd_settings.ndjson
    ci_pipelines_notes.ndjson
    ci_pipelines.ndjson
    container_expiration_policy.ndjson
    custom_attributes.ndjson
    error_tracking_setting.ndjson
    external_pull_requests.ndjson
    issues.ndjson
    labels.ndjson
    merge_requests.ndjson
    metrics_setting.ndjson
    milestones.ndjson
    pipeline_schedules.ndjson
    project_badges.ndjson
    project_feature.ndjson
    project_members.ndjson
    prometheus_metrics.ndjson
    protected_branches.ndjson
    protected_environments.ndjson
    protected_tags.ndjson
    push_rule.ndjson
    releases.ndjson
    security_setting.ndjson
    service_desk_setting.ndjson
    snippets.ndjson

11. Create the project archive

Documentation

Document it in user docs. Link in Importers FAQs. Inform PS and Support