mondo-devel Mailing List for Mondo Rescue

Hello,
I have finally found some time to look into this again. After your 
response, I was confused. I kept looking into the code, and even after 
many more attempts, I still don't understand how this could work. Unless 
there is some hidden magic, we can't copy image of given size into the 
fs made in this image, it will never fit, regardless of how big it is. 
Even when it is a copy, the size is still the same.
So I made these changes into mindi on my system and backup now succeeds. 
I have no idea which files are needed for EFI boot, so I kept them all. 
I still need to test it actually boots, but at least it does not report 
any errors :-).

Best regards

Tomas

--- mindi    2020-12-29 17:57:44.623191813 +0100
+++ mindi-mine    2020-12-29 17:58:24.626430683 +0100
@@ -1874,6 +1874,13 @@
  if  [ "$target" = "ISO" ]; then
      cp $part $MINDI_CACHE
      if [ "$ARCH" = "ia64" ] || [ "$BOOT_TYPE" = "UEFI" ]; then
+
+        # TK hack
+        # create a new, bigger image instead of the original one
+        size2=$(($size*2))
+        dd if=/dev/zero of=$part bs=1k count=$size2 &> /dev/null || Die 
"Cannot dd blank $part"
+        mkfs.vfat $part 2>&1 >> $LOGFILE
+
          # Mount again !
          LogAll "INFO: Re-Mounting $part on $MINDI_TMP/mpt"
          mount -t vfat $mount_opt $part $MINDI_TMP/mpt 2>> $LOGFILE
@@ -1892,6 +1899,8 @@
          ls -lR $MINDI_TMP/mpt >> $LOGFILE
          LogFile "------------------------------------------"
          MakeISO
+        # TK hack
+        umount $MINDI_TMP/mpt
      fi
  else
      # USB


On 03/12/2020 18:45, Bruno Cornec wrote:
> Hello Tomas,
>
> Tomas Kopal said on Wed, Dec 02, 2020 at 03:52:15PM +0100:
>> Turns out only after upgrade I started doing backups as UEFI. Most 
>> issues were easy to resolve, but the last one.
>
> I'm interested by the issues youfixed, if they are of general insterest.
>
>> During boot disc creation, mindi reports an error:
>> '/tmp/mondo.tmp.xrh8ft/mpt/images/mindi-bootroot.img': No space left 
>> on device*
> [...]
>> But, for ISO image on UEFI system (my case), there is more code 
>> starting on line 1875 
>> (http://trac.mondorescue.org/browser/MondoRescue/branches/3.3/mindi/mindi#L1875), 
>> where the final partition image is copied to $MINDI_CACHE, then the 
>> original image is mounted to $MINDI_TMP/mpt (line 1879). Then the 
>> copy of the image from $MINDI_CACHE is mounted to $MINDI_TMP/mpt2 
>> (line 1882), and then *the whole partition image* (mindi-bootroot.img 
>> file) *is copied to the mounted partition image* folder via command 
>> 'cp -a $MINDI_TMP/mpt2/images/all.tar.gz 
>> $MINDI_CACHE/*mindi-bootroot.img* $MINDI_TMP/mpt/images' (line 1886).
>>
>> How on earth can partition image fit into itself?
>
> svn diff -r3532:3546 gives indeed the addition of that block, nearly 
> as it is now at the first addition.
> The comment associated with rev 3535 (from 2016) is (and corresponding 
> to the introduction of that block of code
> "Fix the creation of the bootable ISO for UEFI mode (needs kernel and 
> initrd at the root with EFI and images for the boot and restore part) 
> which can be created without making a FS for it BTW"
>
> Now as you mentionned I've done tests with RHEL since that time on 
> UEFI base systems with success, similarly for SLES. I have not done 
> these tests on Debian/Ubuntu by lack of time.
>
> If I try to comment the code (with ###):
>     cp $part $MINDI_CACHE
>     ### We first keep a copy of the boot+root disk created before 
> under MINDI_CACHE
>     ### part=$MINDI_TMP/mindi-bootroot.img previously in mindi
>     if [ "$ARCH" = "ia64" ] || [ "$BOOT_TYPE" = "UEFI" ]; then
>         # Mount again !
>         LogAll "INFO: Re-Mounting $part on $MINDI_TMP/mpt"
>         mount -t vfat $mount_opt $part $MINDI_TMP/mpt 2>> $LOGFILE
>         ### We now remount the file $part so under MINDI_TMP on the 
> mpt mount point also under MINDI_TMP
>         # We just need the EFI dir and the boot image for this case
>         mkdir -p $MINDI_TMP/mpt2 $MINDI_TMP/mpt/images 
> $MINDI_TMP/mpt/archives
>         mount -o loop $MINDI_CACHE/mindi-bootroot.img $MINDI_TMP/mpt2 
> 2>> $LOGFILE
>         ### Here the file we loopback mount is the one under 
> MINDI_CACHE under mpt2
>         # We need again the kernel+initrd at the root for UEFI boot at 
> least
>         cp -a $MINDI_TMP/mpt2/EFI $MINDI_TMP/mpt2/vmlinuz 
> $MINDI_TMP/mpt2/initrd.img $MINDI_TMP/mpt
>         ### we copy content from MINDI_CACHE image on MINDI_TMP image
>         # We need the boot image now and what mindi needs at restore 
> time as well
>         cp -a $MINDI_TMP/mpt2/images/all.tar.gz 
> $MINDI_CACHE/mindi-bootroot.img $MINDI_TMP/mpt/images
>         ### We continue to copy ontent from MINDI_CACHE image on 
> MINDI_TMP image.         ### The MINDI_CACHE/mindi-bootroot.img file 
> has not changed since we created the copy before the if block upper
>         # Avoids graphical tool to keep the FS mounted
>         sync
>         umount $MINDI_TMP/mpt2
>         LogFile "----------- target dir content -----------"
>         LogFile "------------------------------------------"
>         ls -lR $MINDI_TMP/mpt >> $LOGFILE
>         LogFile "------------------------------------------"
>         MakeISO
>     fi
>
> So I think this can work as indeed we are copying content of a loop 
> mounted file on itself but in another copy, because we need it.
> Now, Is that a working mechanism for Debian/Ubuntu is somehitng I have 
> never have time to explore, but would be happy to solve with your help !
> And maybe again we need to increase EXTRA_SPACE in your context, or 
> there is something behaving differently that explains that the copy 
> fails for you.
>
> Hope this clarifies,
> Bruno
> PS: It took me some time to realize that it was working indeed ;-) My 
> first mail was saying you were right !!
>
>
> _______________________________________________
> Mondo-devel mailing list
> Mon...@li...
> https://lists.sourceforge.net/lists/listinfo/mondo-devel

Hi!

I have some 10 year old backups i created years ago with mondo and had a
moment of panic where i couldn't retrieve the file list on a fresh install
of mondo yesterday.

After a bit of looking at the logs i can see that around version 3.0.4 the
config file name was changed from mondo-restore.cfg to mondorestore.cfg

all well and good, but may i suggest that newer versions of mondo do a 2nd
check for the old filename on a restore command?  I fixed it on my end by
installing an old version of mondo, but...

Thanks!

-- 
-----------------------------
Marc Swanson
M Swanson Consulting LLC

Hello Bruno,

thanks a lot for the fast answer (and for all your work on Mondo in the 
first place :-) ).

On 03/12/2020 18:45, Bruno Cornec wrote:
> Hello Tomas,
>
> Tomas Kopal said on Wed, Dec 02, 2020 at 03:52:15PM +0100:
>> Turns out only after upgrade I started doing backups as UEFI. Most 
>> issues were easy to resolve, but the last one.
>
> I'm interested by the issues youfixed, if they are of general insterest.

Well, one maybe interesting bit is that I had to update genisoimage to 
testing package (9:1.1.11-3.1) to get the -efi-boot option, the version 
in stable does not have it. Other issues were mostly my internal stuff, 
e.g. I am calling mondo in chroot environment to backup lvm snapshots, 
and I had to add bind mount to /run, as some sockets were not found.

One thing I noticed, which might be interesting for you is that the 4.19 
kernel I am using is compressed using xz compression, not gzip, so the 
check for initramfs fails. But the default is fine, so it does not 
matter. Still, if you are interested, I can try adding a specific check 
for xz compression and make the heuristic work again.

>> During boot disc creation, mindi reports an error:
>> '/tmp/mondo.tmp.xrh8ft/mpt/images/mindi-bootroot.img': No space left 
>> on device*
> [...]
>> But, for ISO image on UEFI system (my case), there is more code 
>> starting on line 1875 
>> (http://trac.mondorescue.org/browser/MondoRescue/branches/3.3/mindi/mindi#L1875), 
>> where the final partition image is copied to $MINDI_CACHE, then the 
>> original image is mounted to $MINDI_TMP/mpt (line 1879). Then the 
>> copy of the image from $MINDI_CACHE is mounted to $MINDI_TMP/mpt2 
>> (line 1882), and then *the whole partition image* (mindi-bootroot.img 
>> file) *is copied to the mounted partition image* folder via command 
>> 'cp -a $MINDI_TMP/mpt2/images/all.tar.gz 
>> $MINDI_CACHE/*mindi-bootroot.img* $MINDI_TMP/mpt/images' (line 1886).
>>
>> How on earth can partition image fit into itself?
>
> svn diff -r3532:3546 gives indeed the addition of that block, nearly 
> as it is now at the first addition.
> The comment associated with rev 3535 (from 2016) is (and corresponding 
> to the introduction of that block of code
> "Fix the creation of the bootable ISO for UEFI mode (needs kernel and 
> initrd at the root with EFI and images for the boot and restore part) 
> which can be created without making a FS for it BTW"
>
> Now as you mentionned I've done tests with RHEL since that time on 
> UEFI base systems with success, similarly for SLES. I have not done 
> these tests on Debian/Ubuntu by lack of time.
>
> If I try to comment the code (with ###):
>     cp $part $MINDI_CACHE
>     ### We first keep a copy of the boot+root disk created before 
> under MINDI_CACHE
>     ### part=$MINDI_TMP/mindi-bootroot.img previously in mindi
>     if [ "$ARCH" = "ia64" ] || [ "$BOOT_TYPE" = "UEFI" ]; then
>         # Mount again !
>         LogAll "INFO: Re-Mounting $part on $MINDI_TMP/mpt"
>         mount -t vfat $mount_opt $part $MINDI_TMP/mpt 2>> $LOGFILE
>         ### We now remount the file $part so under MINDI_TMP on the 
> mpt mount point also under MINDI_TMP
>         # We just need the EFI dir and the boot image for this case
>         mkdir -p $MINDI_TMP/mpt2 $MINDI_TMP/mpt/images 
> $MINDI_TMP/mpt/archives
>         mount -o loop $MINDI_CACHE/mindi-bootroot.img $MINDI_TMP/mpt2 
> 2>> $LOGFILE
>         ### Here the file we loopback mount is the one under 
> MINDI_CACHE under mpt2
>         # We need again the kernel+initrd at the root for UEFI boot at 
> least
>         cp -a $MINDI_TMP/mpt2/EFI $MINDI_TMP/mpt2/vmlinuz 
> $MINDI_TMP/mpt2/initrd.img $MINDI_TMP/mpt
>         ### we copy content from MINDI_CACHE image on MINDI_TMP image
>         # We need the boot image now and what mindi needs at restore 
> time as well
>         cp -a $MINDI_TMP/mpt2/images/all.tar.gz 
> $MINDI_CACHE/mindi-bootroot.img $MINDI_TMP/mpt/images

Well, I am far from being a shell guru, but as I understand the code, we 
copy the image itself, not the contents of the image in the line above. 
Or is there some behavior of cp I am missing? The source of the copy 
here are two files, first is all.tar.gz (from the mounted copy in 
MINDI_CACHE), and second is mindi-bootroot.img from MINDI_CACHE itself, 
right? The fact that it is mounted should not matter for this second 
param, right? What I am missing here?

>         ### We continue to copy ontent from MINDI_CACHE image on 
> MINDI_TMP image.         ### The MINDI_CACHE/mindi-bootroot.img file 
> has not changed since we created the copy before the if block upper
>         # Avoids graphical tool to keep the FS mounted
>         sync
>         umount $MINDI_TMP/mpt2
>         LogFile "----------- target dir content -----------"
>         LogFile "------------------------------------------"
>         ls -lR $MINDI_TMP/mpt >> $LOGFILE
>         LogFile "------------------------------------------"
>         MakeISO
>     fi
>
> So I think this can work as indeed we are copying content of a loop 
> mounted file on itself but in another copy, because we need it.
> Now, Is that a working mechanism for Debian/Ubuntu is somehitng I have 
> never have time to explore, but would be happy to solve with your help !
> And maybe again we need to increase EXTRA_SPACE in your context, or 
> there is something behaving differently that explains that the copy 
> fails for you.
>
> Hope this clarifies,

Not realy, I am afraid :-(. Apart from the point above, I also don't 
understand why we copy e.g. the all.tar.gz file, the partition image in 
MINDI_CACHE is the same as the one from MINDI_TMP (we did a copy), so if 
we mount both, the all.tar.gz file should be already there and 
identical? Looks like the images were meant to be different, but we are 
doing copy, so they are not?

I feel I am missing some important part of a big puzzle, but can't put 
my finger on it, so please have patience with me :-).

Increasing EXTRA_SPACE didn't seem to help during my first attempts, but 
I admit I tried only small values, I wanted to understand what should be 
the right value first, and that got me into this mess ;-).

> Bruno
> PS: It took me some time to realize that it was working indeed ;-) My 
> first mail was saying you were right !!
>
I hope I will have a chance to say the same soon :-). Thank you a lot 
for your support.

Best regards

Tomas

Hello Tomas,

Tomas Kopal said on Wed, Dec 02, 2020 at 03:52:15PM +0100:
>Turns out only after upgrade I started doing backups as UEFI. Most 
>issues were easy to resolve, but the last one.

I'm interested by the issues youfixed, if they are of general insterest.

>During boot disc creation, mindi reports an error:
>'/tmp/mondo.tmp.xrh8ft/mpt/images/mindi-bootroot.img': No space left 
>on device*
[...]
>But, for ISO image on UEFI system (my case), there is more code 
>starting on line 1875 (http://trac.mondorescue.org/browser/MondoRescue/branches/3.3/mindi/mindi#L1875), 
>where the final partition image is copied to $MINDI_CACHE, then the 
>original image is mounted to $MINDI_TMP/mpt (line 1879). Then the copy 
>of the image from $MINDI_CACHE is mounted to $MINDI_TMP/mpt2 (line 
>1882), and then *the whole partition image* (mindi-bootroot.img file) 
>*is copied to the mounted partition image* folder via command 'cp -a 
>$MINDI_TMP/mpt2/images/all.tar.gz $MINDI_CACHE/*mindi-bootroot.img* 
>$MINDI_TMP/mpt/images' (line 1886).
>
>How on earth can partition image fit into itself?

svn diff -r3532:3546 gives indeed the addition of that block, nearly as it is now at the first addition.
The comment associated with rev 3535 (from 2016) is (and corresponding to the introduction of that block of code
"Fix the creation of the bootable ISO for UEFI mode (needs kernel and initrd at the root with EFI and images for the boot and restore part) which can be created without making a FS for it BTW"

Now as you mentionned I've done tests with RHEL since that time on UEFI base systems with success, similarly for SLES. 
I have not done these tests on Debian/Ubuntu by lack of time.

If I try to comment the code (with ###):
	cp $part $MINDI_CACHE
	### We first keep a copy of the boot+root disk created before under MINDI_CACHE
	### part=$MINDI_TMP/mindi-bootroot.img previously in mindi
	if [ "$ARCH" = "ia64" ] || [ "$BOOT_TYPE" = "UEFI" ]; then
		# Mount again !
		LogAll "INFO: Re-Mounting $part on $MINDI_TMP/mpt"
		mount -t vfat $mount_opt $part $MINDI_TMP/mpt 2>> $LOGFILE
		### We now remount the file $part so under MINDI_TMP on the mpt mount point also under MINDI_TMP
		# We just need the EFI dir and the boot image for this case
		mkdir -p $MINDI_TMP/mpt2 $MINDI_TMP/mpt/images $MINDI_TMP/mpt/archives
		mount -o loop $MINDI_CACHE/mindi-bootroot.img $MINDI_TMP/mpt2 2>> $LOGFILE
		### Here the file we loopback mount is the one under MINDI_CACHE under mpt2
		# We need again the kernel+initrd at the root for UEFI boot at least
		cp -a $MINDI_TMP/mpt2/EFI $MINDI_TMP/mpt2/vmlinuz $MINDI_TMP/mpt2/initrd.img $MINDI_TMP/mpt
		### we copy content from MINDI_CACHE image on MINDI_TMP image
		# We need the boot image now and what mindi needs at restore time as well
		cp -a $MINDI_TMP/mpt2/images/all.tar.gz $MINDI_CACHE/mindi-bootroot.img $MINDI_TMP/mpt/images
		### We continue to copy ontent from MINDI_CACHE image on MINDI_TMP image. 
		### The MINDI_CACHE/mindi-bootroot.img file has not changed since we created the copy before the if block upper
		# Avoids graphical tool to keep the FS mounted
		sync
		umount $MINDI_TMP/mpt2
		LogFile "----------- target dir content -----------"
		LogFile "------------------------------------------"
		ls -lR $MINDI_TMP/mpt >> $LOGFILE
		LogFile "------------------------------------------"
		MakeISO
	fi

So I think this can work as indeed we are copying content of a loop mounted file on itself but in another copy, because we need it.
Now, Is that a working mechanism for Debian/Ubuntu is somehitng I have never have time to explore, but would be happy to solve with your help !
And maybe again we need to increase EXTRA_SPACE in your context, or there is something behaving differently that explains that the copy fails for you.

Hope this clarifies,
Bruno
PS: It took me some time to realize that it was working indeed ;-) My first mail was saying you were right !!
-- 
HPE WW FLOSS Technology Strategist      http://www.hpe.com/engage/opensource
Open Source Profession, WW Linux Community Lead    http://github.com/bcornec
FLOSS projects:     http://mondorescue.org        http://project-builder.org
Musique ancienne?   http://www.musique-ancienne.org  http://www.medieval.org

Hello Richard,

Well, you know, I can't really complain that people have work to do, when on my side I've been dalaying the release of MondoRescue for 4 years now !

Anyway, indeed ubuntu was not "supported" in UEFI mode up to now. So as it should work fine, I've just added it in mindi so you can test and report.
I've rebuild packages for ubuntu 18.04 that you'll find here:
ftp://ftp.mondorescue.org/test/ubuntu/18.04
(you just need to update the mindi package in your case.

And the nvme driver is now part of mindi ver listas well, so you should be able to go back to a std mindi.conf, but your modif doesn't hurt either.

Best regards,
bruno.

Richard Taylor said on Sun, Nov 29, 2020 at 11:34:37AM -0500:

> Bruno,
> 
> Sorry it has taken me so long to get around to working with this.  This is
> a work computer and I needed a quiet time where I could take a change on
> the 20.04 upgrade and have time to restore.
> 
> Today I tried to create my backup.  I modified the mindi.conf file to load
> the nvme driver (FORCE_MODS=nvme) on restore as that is needed for my main
> (SSD) drive.
> I got an error during my attempt.  This is what I am seeing is:
> 
-- 
HPE WW FLOSS Technology Strategist      http://www.hpe.com/engage/opensource
Open Source Profession, WW Linux Community Lead    http://github.com/bcornec
FLOSS projects:     http://mondorescue.org        http://project-builder.org
Musique ancienne?   http://www.musique-ancienne.org  http://www.medieval.org

Hi,

I am using MondoArchive on my Debian system for some time, and it's been 
always working great. However, after updating to Buster, I started 
receiving errors from mondoarchive (version v3.3.0-r3762). Turns out 
only after upgrade I started doing backups as UEFI. Most issues were 
easy to resolve, but the last one.

During boot disc creation, mindi reports an error:

INFO: Creating a vfat filesystem on /tmp/mondo.tmp.xrh8ft/mindi-bootroot.img
INFO: Mounting /tmp/mondo.tmp.xrh8ft/mindi-bootroot.img on 
/tmp/mondo.tmp.xrh8ft/mpt
INFO: Moving boot info on /tmp/mondo.tmp.xrh8ft/mpt
INFO: Re-Mounting /tmp/mondo.tmp.xrh8ft/mindi-bootroot.img on 
/tmp/mondo.tmp.xrh8ft/mpt
*cp: error writing 
'/tmp/mondo.tmp.xrh8ft/mpt/images/mindi-bootroot.img': No space left on 
device*
INFO: Generating ISO image.
INFO: Invoking /usr/bin/genisoimage -J -r -v -p Mindi -publisher 
http://www.mondorescue.org -A Mindi -V MindiCD -o 
/var/cache/mindi/mindi.iso  -b EFI/isolinux.bin -c EFI/boot.cat 
-no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot 
-efi-boot images/mindi-bootroot.img -no-emul-boot .
INFO: Created bootable ISO image at /var/cache/mindi/mindi.iso

After some naive attempts increasing the EXTRA_SPACE parameter, I 
started reading through the code to figure out what happens. And I got 
lost :-(.

Looking at mindi script source, I can see from line 1788 
(http://trac.mondorescue.org/browser/MondoRescue/branches/3.3/mindi/mindi#L1788), 
that $MINDI_TMP/mindi-bootroot.img is an image of vfat partition, 
populated with the contents of the target folder. So far so good.

But, for ISO image on UEFI system (my case), there is more code starting 
on line 1875 
(http://trac.mondorescue.org/browser/MondoRescue/branches/3.3/mindi/mindi#L1875), 
where the final partition image is copied to $MINDI_CACHE, then the 
original image is mounted to $MINDI_TMP/mpt (line 1879). Then the copy 
of the image from $MINDI_CACHE is mounted to $MINDI_TMP/mpt2 (line 
1882), and then *the whole partition image* (mindi-bootroot.img file) 
*is copied to the mounted partition image* folder via command 'cp -a 
$MINDI_TMP/mpt2/images/all.tar.gz $MINDI_CACHE/*mindi-bootroot.img* 
$MINDI_TMP/mpt/images' (line 1886).

How on earth can partition image fit into itself?

I am sure I am missing something, or reading some code somewhere wrong, 
as it looks the code has been there for ages and I found no reports that 
it does not work. Can someone, please, explain to me how this piece of 
code is actually meant to be working? AFAIK vfat can't do sparse files, 
there is no compression... I am lost.

Thank you in advance for any hints.

Tomas