[go: up one dir, main page]

Smaller manifests on remote when using replicate command

I have:

  • ([x] when completed)
  • searched https://gitlab.com/duplicity/duplicity/-/issues for similar issues. If you find a similar issue and the issue is still open, add a comment to the existing issue instead of opening a new one. If you find a Closed issue that seems like it is the same thing that you're experiencing, open a new issue and include a link to the original issue in the body of your new one.
  • searched https://bugs.launchpad.net/duplicity for similar issues. If you find a similar issue, open a new issue on here and include a link to the original issue in the body of your new one.
  • tested that this issue still occurs on the latest stable snap (install instructions: https://snapcraft.io/duplicity), please include the snap version (snap info duplicity | grep installed) output: installed: x.xx.xx (xx)
  • ideally, tested that this issue still occurs on the latest edge snap, if you can test without risking your data. Please include the snap version output: installed: x.xx.xx (xx)

Summary

I use the replicate command to replicate backup from file:///local_hard_drive to ftps://remote_ftps_server. It seems that the manifest are not valid on the remote_ftps_server. By looking in the manifests I noticed that the manifests on FTPS storage do not contain the Filelist listing the new, changed and deleted files, but only the volume list (and unsorted !). When I run a du -sh on my archive :

3.2G    standalone_hard-drive_archive_folder/duplicity-full-signatures.20211109T091646Z.sigtar.gz
64K     standalone_hard-drive_archive_folder/duplicity-inc.20211109T091646Z.to.20211115T102211Z.manifest
8.0K    standalone_hard-drive_archive_folder/duplicity-inc.20211115T102211Z.to.20211115T103552Z.manifest
16K     standalone_hard-drive_archive_folder/duplicity-inc.20211115T103552Z.to.20211115T112819Z.manifest
14M     standalone_hard-drive_archive_folder/duplicity-new-signatures.20211109T091646Z.to.20211115T102211Z.sigtar.gz
708K    standalone_hard-drive_archive_folder/duplicity-new-signatures.20211115T102211Z.to.20211115T103552Z.sigtar.gz
720K    standalone_hard-drive_archive_folder/duplicity-new-signatures.20211115T103552Z.to.20211115T112819Z.sigtar.gz
0       standalone_hard-drive_archive_folder/lockfile

16K     ftps_storage_archive_folder/duplicity-full.20211109T091646Z.manifest
3.2G    ftps_storage_archive_folder/duplicity-full-signatures.20211109T091646Z.sigtar.gz
4.0K    ftps_storage_archive_folder/duplicity-inc.20211109T091646Z.to.20211115T102211Z.manifest
4.0K    ftps_storage_archive_folder/duplicity-inc.20211115T102211Z.to.20211115T103552Z.manifest
4.0K    ftps_storage_archive_folder/duplicity-inc.20211115T103552Z.to.20211115T112819Z.manifest
14M     ftps_storage_archive_folder/duplicity-new-signatures.20211109T091646Z.to.20211115T102211Z.sigtar.gz
708K    ftps_storage_archive_folder/duplicity-new-signatures.20211115T102211Z.to.20211115T103552Z.sigtar.gz
720K    ftps_storage_archive_folder/duplicity-new-signatures.20211115T103552Z.to.20211115T112819Z.sigtar.gz
0       ftps_storage_archive_folder/lockfile

Environment

Debian 11.1 duplicity 0.8.17 Python 3.9.2

Steps to reproduce

env SIGN_PASSPHRASE=VERY_SECURE_PASSPHRASE PASSPHRASE=VERY_SECURE_PASSPHRASE /usr/bin/duplicity --no-compression --log-file dackup-worker_resource_to_backup_dup.log --archive-dir /var/duplicity/cache/resource_to_backup --volsize 10240 --tempdir /var/duplicity/temp/resource_to_backup --encrypt-key GPG_KEY_FINGERPRINT --sign-key GPG_KEY_FINGERPRINT --full-if-older-than 1M --exclude **/proc --exclude **/sys --exclude **/dev /mnt/lv/resource_to_backup file:///var/archives/duplicity/backup-storage/resource_to_backup

env SIGN_PASSPHRASE=VERY_SECURE_PASSPHRASE PASSPHRASE=VERY_SECURE_PASSPHRASE /usr/bin/duplicity remove-all-but-n-full 1 --log-file dackup-worker_resource_to_backup_dup.log --force --archive-dir /var/duplicity/cache/resource_to_backup --volsize 10240 --tempdir /var/duplicity/temp/resource_to_backup --encrypt-key GPG_KEY_FINGERPRINT --sign-key GPG_KEY_FINGERPRINT file:///var/archives/duplicity/backup-storage/resource_to_backup

env FTP_PASSWORD=VERY_SECURE_PASSWORD SIGN_PASSPHRASE=VERY_SECURE_PASSPHRASE PASSPHRASE=VERY_SECURE_PASSPHRASE /usr/bin/duplicity replicate --no-compression --log-file dackup-worker_resource_to_backup_dup.log --encrypt-key GPG_KEY_FINGERPRINT --sign-key GPG_KEY_FINGERPRINT --archive-dir /var/duplicity/cache/resource_to_backup --volsize 10240 --tempdir /var/duplicity/temp/resource_to_backup file:///var/archives/duplicity/backup-storage/resource_to_backup ftpes://ftp_user@ftp_server.com:ftp_port//backups/server/resource_to_backup

env FTP_PASSWORD=VERY_SECURE_PASSWORD SIGN_PASSPHRASE=VERY_SECURE_PASSPHRASE PASSPHRASE=VERY_SECURE_PASSPHRASE /usr/bin/duplicity remove-all-but-n-full 3 --log-file dackup-worker_resource_to_backup_dup.log --force --archive-dir /var/duplicity/cache/resource_to_backup --volsize 10240 --tempdir /var/duplicity/temp/resource_to_backup --encrypt-key GPG_KEY_FINGERPRINT --sign-key GPG_KEY_FINGERPRINT ftpes://ftp_user@ftp_server.com:ftp_port//backups/server/resource_to_backup

What is the current bug behaviour?

I cannot restore any file at any time. I end up with either :

var/log/syslog not found in archive - no files restored.

or

Error 'First patch in sequence [<duplicity.path.ROPath object at 0x7f2032186040>, <duplicity.path.ROPath object at 0x7f203211c880>, <duplicity.path.ROPath object at 0x7f203211ca60>] was a diff' patching .
var/log/syslog not found in archive - no files restored.

But the files are visible with the list-current-files command !

What is the expected correct behaviour?

I should be able to restore my files

Relevant logs and/or screenshots

/

Possible fixes

/

Edited by FirstN LastN