#2660 PR merged: Cleanup USB format scripts¶
Labels: enhancement, cleanup, fixed / solved / done
jsmeix opened issue at 2021-07-20 15:43:¶
-
Type: Enhancement
-
Impact: Normal
-
Reference to related issue (URL):
https://github.com/rear/rear/pull/2656 -
How was this pull request tested?
See the comments below -
Brief description of the changes in this pull request:
Improved format/USB/default/300_format_usb_disk.sh
in particular regarding creating BIOS boot partition.
jsmeix commented at 2021-07-21 11:30:¶
First test on my x86_64 homeoffice laptop with UEFI firmware used in legacy BIOS mode:
My etc/rear/local.conf:
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
With current GitHub master code before this pull request:
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 12443 date 2021-07-21 13:19:00)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.sE2AXhLHqojokzx
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 61
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning '/dev/sdb'
Creating partition table of type 'msdos' on '/dev/sdb'
Creating BOOT partition with size 512 MiB aligned at 8 MiB on '/dev/sdb'
Setting 'boot' flag on
Creating ReaR data partition up to 10% of '/dev/sdb'
Creating ext2 filesystem with label 'REAR-BOOT' on '/dev/sdb1'
Creating ext3 filesystem with label 'REAR-000' on '/dev/sdb2'
Adjusting filesystem parameters on '/dev/sdb2'
Exiting rear format (PID 12443) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 520MiB 512MiB primary ext2 boot, type=83
2 520MiB 47694MiB 47174MiB primary ext3 type=83
With the changes in this pull request:
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 13108 date 2021-07-21 13:20:26)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.vBA3s4psvjFI8Ub
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type msdos on /dev/sdb
Creating boot partition /dev/sdb1 with size 512 MiB aligned at 8 MiB
Setting 'boot' flag on boot partition /dev/sdb1
Creating ReaR data partition /dev/sdb2 up to 10% of '/dev/sdb'
Creating ext2 filesystem with label 'REARBOOT' on boot partition /dev/sdb1
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb2
Adjusting filesystem parameters on ReaR data partition /dev/sdb2
Exiting rear format (PID 13108) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 520MiB 512MiB primary ext2 boot, type=83
2 520MiB 47694MiB 47174MiB primary ext3 type=83
OK - seems to be exactly the same partitioning in the default case.
jsmeix commented at 2021-07-21 11:47:¶
Second test on my x86_64 homeoffice laptop with UEFI firmware used in legacy BIOS mode:
My etc/rear/local.conf:
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
USB_DEVICE_PARTED_LABEL=gpt
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
With current GitHub master code before this pull request:
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 14111 date 2021-07-21 13:36:21)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.E3yq8SYRk3RsDEW
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 61
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning '/dev/sdb'
Creating partition table of type 'gpt' on '/dev/sdb'
Creating BIOS boot partition on '/dev/sdb' from sector 34 to 2047
Setting 'bios_grub' flag on /dev/sdb1
Creating BOOT partition with size 512 MiB aligned at 8 MiB on '/dev/sdb'
Setting 'legacy_boot' flag on
Creating ReaR data partition up to 10% of '/dev/sdb'
Creating ext2 filesystem with label 'REAR-BOOT' on '/dev/sdb2'
Creating ext3 filesystem with label 'REAR-000' on '/dev/sdb3'
Adjusting filesystem parameters on '/dev/sdb3'
Exiting rear format (PID 14111) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 0.02MiB 1.00MiB 0.98MiB primary bios_grub
2 8.00MiB 520MiB 512MiB ext2 primary legacy_boot
3 520MiB 47694MiB 47174MiB ext3 primary
# parted -s /dev/sdb unit B print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 500107859968B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 17408B 1048575B 1031168B primary bios_grub
2 8388608B 545259519B 536870912B ext2 primary legacy_boot
3 545259520B 50010783743B 49465524224B ext3 primary
With the changes in this pull request:
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 14839 date 2021-07-21 13:38:12)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.sGkbqScrXNmHktg
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type gpt on /dev/sdb
Creating BIOS boot partition /dev/sdb1
Setting 'bios_grub' flag on BIOS boot partition /dev/sdb1
Creating boot partition /dev/sdb2 with size 512 MiB aligned at 8 MiB
Setting 'legacy_boot' flag on boot partition /dev/sdb2
Creating ReaR data partition /dev/sdb3 up to 10% of /dev/sdb
Creating ext2 filesystem with label 'REARBOOT' on boot partition /dev/sdb2
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb3
Adjusting filesystem parameters on ReaR data partition /dev/sdb3
Exiting rear format (PID 14839) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 0.02MiB 8.00MiB 7.98MiB primary bios_grub
2 8.00MiB 520MiB 512MiB ext2 primary legacy_boot
3 520MiB 47694MiB 47174MiB ext3 primary
# parted -s /dev/sdb unit B print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 500107859968B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 24576B 8388607B 8364032B primary bios_grub
2 8388608B 545259519B 536870912B ext2 primary legacy_boot
3 545259520B 50010783743B 49465524224B ext3 primary
The BIOS boot partition is different as intended by the changes in this
pull request.
The other partitions are same so things look OK to me.
jsmeix commented at 2021-07-21 12:00:¶
Testing "rear mkbackup" with the changes in this pull request
with USB_DEVICE_PARTED_LABEL=gpt as in
https://github.com/rear/rear/pull/2660#issuecomment-884127834
# usr/sbin/rear -D mkbackup
Relax-and-Recover 2.6 / Git
Running rear mkbackup (PID 15612 date 2021-07-21 13:48:31)
Command line options: usr/sbin/rear -D mkbackup
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.utb9NMxpAqBu3FC
...
Created initrd.cgz with gzip default compression (68602370 bytes) in 13 seconds
Saved /root/rear.github.master/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210721.1348/rear-linux-h9wr.log
Writing MBR of type gpt to /dev/sdb
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.utb9NMxpAqBu3FC/outputfs/rear/linux-h9wr/20210721.1348/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 15612) and its descendant processes ...
# mount /dev/sdb2 /mnt
# find /mnt
/mnt
/mnt/lost+found
# umount /dev/sdb2
# mount /dev/sdb3 /mnt
# find /mnt
/mnt
/mnt/lost+found
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/message
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/lua.c32
/mnt/boot/syslinux/chain.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/poweroff.com
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/ls.c32
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/ldlinux.sys
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/config.c32
/mnt/rear
/mnt/rear/syslinux.cfg
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210721.1348
/mnt/rear/linux-h9wr/20210721.1348/kernel
/mnt/rear/linux-h9wr/20210721.1348/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210721.1348/syslinux.cfg
/mnt/rear/linux-h9wr/20210721.1348/backup.log
/mnt/rear/linux-h9wr/20210721.1348/initrd.cgz
/mnt/rear/linux-h9wr/20210721.1348/backup.tar.gz
/mnt/linux-h9wr
/mnt/linux-h9wr/.lockfile
Looks OK what there is on /dev/sdb
jsmeix commented at 2021-07-21 13:13:¶
But with what there is on /dev/sdb as shown in
https://github.com/rear/rear/pull/2660#issuecomment-884134008
my x86_64 homeoffice laptop with UEFI firmware used in legacy BIOS
mode
fails to boot from that USB disk.
In the firmware boot menue I can selevt to boot from USB disk
and then I see on the screen something like
Missing OS
directly followd by the usual initial GRUB2 boot messages
that show up when booting from the internal harddisk
and when I proceed it boots from the internal harddisk
so it seems the firmware fails to boot from that USB disk with
Missing OS
and because of that the firmware automatically continues
with the next bootable disk which is the internal harddisk.
So it seems I may also need GRUB2 as bootloader on the USB disk
to make my x86_64 homeoffice laptop with UEFI firmware used in legacy
BIOS mode
booting from the USB disk.
jsmeix commented at 2021-07-22 11:21:¶
Prior to all those recent changes with USB here things work for me.
When I go back to
git checkout cbd352d06facbd7d74c609382d7aa4652e9e07b1
with this etc/rear/local.conf
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
When I run usr/sbin/rear -D format /dev/sdb I get
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 47694MiB 47686MiB primary ext3 boot, type=83
# mount /dev/sdb1 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/lost+found
When I then run usr/sbin/rear -D mkbackup I get
# mount /dev/sdb1 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/ls.c32
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/lua.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/config.c32
/mnt/boot/syslinux/poweroff.com
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/message
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/ldlinux.sys
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/chain.c32
/mnt/lost+found
/mnt/rear
/mnt/rear/syslinux.cfg
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210722.1306
/mnt/rear/linux-h9wr/20210722.1306/syslinux.cfg
/mnt/rear/linux-h9wr/20210722.1306/backup.tar.gz
/mnt/rear/linux-h9wr/20210722.1306/initrd.cgz
/mnt/rear/linux-h9wr/20210722.1306/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210722.1306/backup.log
/mnt/rear/linux-h9wr/20210722.1306/kernel
/mnt/linux-h9wr
/mnt/linux-h9wr/.lockfile
which boots on another older x86_64 laptop with traditional BIOS
and there the ReaR recovery system starts up and is usable
(I cannot test an actual recovery on that laptop because I need is as
is).
jsmeix commented at 2021-07-22 11:32:¶
With current master code (i.e. commit
20e887f8ec3e30d577d1598f26ae09a62534b13a)
and same etc/rear/local.conf
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
when I run usr/sbin/rear -D format /dev/sdb I get
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 520MiB 512MiB primary ext2 boot, type=83
2 520MiB 47694MiB 47174MiB primary ext3 type=83
and then with # usr/sbin/rear -D mkbackup I get
# mount /dev/sdb1 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/lost+found
# mount /dev/sdb2 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/rear
/mnt/rear/syslinux.cfg
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210722.1327
/mnt/rear/linux-h9wr/20210722.1327/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210722.1327/backup.log
/mnt/rear/linux-h9wr/20210722.1327/syslinux.cfg
/mnt/rear/linux-h9wr/20210722.1327/backup.tar.gz
/mnt/rear/linux-h9wr/20210722.1327/initrd.cgz
/mnt/rear/linux-h9wr/20210722.1327/kernel
/mnt/lost+found
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/message
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/poweroff.com
/mnt/boot/syslinux/ls.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/lua.c32
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/config.c32
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/ldlinux.sys
/mnt/boot/syslinux/chain.c32
/mnt/linux-h9wr
/mnt/linux-h9wr/.lockfile
which fails to boot on my BIOS laptop with
Missing operating system
so with current master code (i.e. commit
20e887f8ec3e30d577d1598f26ae09a62534b13a)
OUTPUT=USB things do no longer work at least not with BIOS.
jsmeix commented at 2021-07-22 11:42:¶
The above described regression with OUTPUT=USB
in current master code can be manually fixed with
# parted -s /dev/sdb set 1 boot off
# parted -s /dev/sdb set 2 boot on
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 520MiB 512MiB primary ext2 type=83
2 520MiB 47694MiB 47174MiB primary ext3 boot, type=83
to flag the partition as boot that actually contains all stuff.
I.e. what is wrong is that in current master code with OUTPUT=USB
an additional boot partition is created in case of non-EFI (i.e. in case
of BIOS)
but nothing gets stored into that boot partition
i.e. all is still in the data partition
so the data partition needs to be flagged as boot.
DEvil0000 commented at 2021-07-22 12:36:¶
The "issue" is/was the kernel and initrd ended up on partition 2 instead
of partition 1.
This can be fixed by adding the following line to your config:
OUTPUT_URL=usb:///dev/disk/by-label/REAR-BOOT
Maybe this should be done shomewhat automatic with the new code.
Edit: Or the boot flag should be set on the partition the OUTPUT_URL points to.
jsmeix commented at 2021-07-22 13:19:¶
@DEvil0000
thank you for your info about setting a specific OUTPUT_URL
My first and foremost interest was that the "well known" standard case
as described in
http://relax-and-recover.org/documentation/getting-started
still works because otherwise it would be perceived as regression by our
users.
jsmeix commented at 2021-07-22 13:25:¶
With the recent changes in this pull request here
the standard case as described in
http://relax-and-recover.org/documentation/getting-started
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
works again for me. I use additionally
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
only to avoid that making the filesystem on my 466GiB USB disk
takes a longer time that I neither need nor want for my tests.
Now I get
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 11340 date 2021-07-22 15:06:11)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.uObqgOvIqxOAIwW
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type msdos on /dev/sdb
Creating ReaR data partition /dev/sdb1 up to 10% of /dev/sdb
Setting 'boot' flag on ReaR data partition /dev/sdb1
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb1
Adjusting filesystem parameters on ReaR data partition /dev/sdb1
Exiting rear format (PID 11340) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 47694MiB 47686MiB primary ext3 boot, type=83
and then with usr/sbin/rear -D mkbackup
that USB disk boots with syslinux
on my older x86_64 laptop with traditional BIOS
and the ReaR recovery system starts up and is usable.
jsmeix commented at 2021-07-22 13:28:¶
@DEvil0000 @rear/contributors
if there are no objections
I would like to merge this pull request tomorrow morning
to get the USB standard case as described in
http://relax-and-recover.org/documentation/getting-started
working again.
Then I will test step by step the more advanced cases
and do further adaptions, cleanups, and enhancements
if needed.
DEvil0000 commented at 2021-07-22 16:20:¶
I would like to see the boot partition in addition to the backup
partition instead of having only one partition. This way you have a
clean separation of kernel+initrd (system) and backups on the backup
partition. (For the msdos and gpt case). This would also align it mostly
with EFI where the kernel+initrd is stored on the EFI partition.
I missed in my original PR that I would break the default case when
OUTPUT_URL=usb:///dev/disk/by-label/REAR-BOOT is not set (- beside the
sector size thing).
I think it is now too many changes to tell just from looking at it if it
is working as intended. Testing however is very time consuming so I
would like to do a retest later when all changes are in.
No objections, go on as you think is best.
jsmeix commented at 2021-07-23 05:58:¶
@DEvil0000
of course I also want to have a separated boot partition in addition to
the backup data partition.
That's one of the main goals behind our recent enhancements for
OUTPUT=USB here
(the topmost goal is GRUB2 as USB bootloader).
But what I also have to do is keep things backward compatible.
We at ReaR upstream try hard to keep backward compatibility
as far as possible with reasonable effort.
Cf. the section "Maintain backward compatibility" in
https://github.com/rear/rear/wiki/Coding-Style
Therefore the behaviour for plain
OUTPUT=USB
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
should not change unless there is a really severe reason why it must change.
Today morning I will do a few tests with the current code of this pull
request
and if things behave sufficiently OK for me I merge this pull request
so that I could then further improve things based on it.
jsmeix commented at 2021-07-23 08:04:¶
The current code of this pull request does not behave sufficiently OK
for me
because it failed to make the USB disk bootable with syslinux/extlinux
when the booting related files are on a separated boot partition.
This should be fixed by my recent
https://github.com/rear/rear/commit/3218e36bbba1782ad8b9587b3fefe579eb3f9b9d
I will re-tests if now things behave sufficiently OK for me
with syslinux/extlinux with and witout a separated boot partition
and if yes I merge this pull request.
GRUB2 support comes afterwards via the separated pull request
https://github.com/rear/rear/pull/2659
jsmeix commented at 2021-07-23 10:31:¶
Tested that things behave sufficiently OK for me
with syslinux/extlinux without a separated boot partition.
With etc/rear/local.conf
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
I got
# usr/sbin/rear -D format /dev/sdb
...
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type msdos on /dev/sdb
Creating ReaR data partition /dev/sdb1 up to 10% of /dev/sdb
Setting 'boot' flag on ReaR data partition /dev/sdb1
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb1
Adjusting filesystem parameters on ReaR data partition /dev/sdb1
Exiting rear format (PID 19473) and its descendant processes ...
# usr/sbin/rear -D mkbackup
...
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (68665055 bytes) in 15 seconds
Saved /root/rear.github.master/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1015/rear-linux-h9wr.log
Making /dev/sdb bootable with syslinux/extlinux
Writing syslinux MBR /usr/share/syslinux/mbr.bin of type msdos to /dev/sdb
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.xZI0VIYHYBYGguS/outputfs/rear/linux-h9wr/20210723.1015/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 20117) and its descendant processes ...
# usr/sbin/rear -D mkbackup
...
Created initrd.cgz with gzip default compression (68663980 bytes) in 14 seconds
Saved /root/rear.github.master/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1018/rear-linux-h9wr.log
No need to update syslinux on /dev/sdb that has version 4.04
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.KXaE7iTTqPbzTLz/outputfs/rear/linux-h9wr/20210723.1018/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 12339) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 47694MiB 47686MiB primary ext3 boot, type=83
# mount /dev/sdb1 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/config.c32
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/lua.c32
/mnt/boot/syslinux/chain.c32
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/poweroff.com
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/message
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/ls.c32
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/ldlinux.sys
/mnt/lost+found
/mnt/linux-h9wr
/mnt/linux-h9wr/.lockfile
/mnt/rear
/mnt/rear/syslinux.cfg
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210723.1018
/mnt/rear/linux-h9wr/20210723.1018/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1018/kernel
/mnt/rear/linux-h9wr/20210723.1018/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1018/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1018/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1018/backup.log
/mnt/rear/linux-h9wr/20210723.1015
/mnt/rear/linux-h9wr/20210723.1015/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1015/kernel
/mnt/rear/linux-h9wr/20210723.1015/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1015/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1015/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1015/backup.log
that USB disk boots with syslinux
on my older x86_64 laptop with traditional BIOS
and the ReaR recovery system starts up and is usable
for both syslinux boot menue entries i.e.
for the 20210723 1015 backup and
for the 20210723 1018 backup
jsmeix commented at 2021-07-23 10:38:¶
Tested that things behave sufficiently OK for me
with syslinux/extlinux with a separated boot partition.
With etc/rear/local.conf
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
USB_BOOT_PART_SIZE=1024
OUTPUT_URL=usb:///dev/disk/by-label/REARBOOT
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
I got
# usr/sbin/rear -D format /dev/sdb
...
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type msdos on /dev/sdb
Creating boot partition /dev/sdb1 with size 1024 MiB aligned at 8 MiB
Setting 'boot' flag on boot partition /dev/sdb1
Creating ReaR data partition /dev/sdb2 up to 10% of /dev/sdb
Creating ext2 filesystem with label 'REARBOOT' on boot partition /dev/sdb1
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb2
Adjusting filesystem parameters on ReaR data partition /dev/sdb2
Exiting rear format (PID 5479) and its descendant processes ...
# usr/sbin/rear -D mkbackup
...
Created initrd.cgz with gzip default compression (68668484 bytes) in 14 seconds
Saved /root/rear.github.master/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1134/rear-linux-h9wr.log
Making /dev/sdb bootable with syslinux/extlinux
Writing syslinux MBR /usr/share/syslinux/mbr.bin of type msdos to /dev/sdb
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.wRFssqI1NeHFTGR/outputfs/rear/linux-h9wr/20210723.1134/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 6335) and its descendant processes ...
# usr/sbin/rear -D mkbackup
...
Created initrd.cgz with gzip default compression (68660335 bytes) in 14 seconds
Saved /root/rear.github.master/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1136/rear-linux-h9wr.log
No need to update syslinux on /dev/sdb that has version 4.04
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.C9Vjm2gzzjPR5aE/outputfs/rear/linux-h9wr/20210723.1136/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 30484) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 8.00MiB 1032MiB 1024MiB primary ext2 boot, type=83
2 1032MiB 47694MiB 46662MiB primary ext3 type=83
# mount /dev/sdb1 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/linux-h9wr
/mnt/rear
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210723.1134
/mnt/rear/linux-h9wr/20210723.1134/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1134/kernel
/mnt/rear/linux-h9wr/20210723.1134/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1134/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1136
/mnt/rear/linux-h9wr/20210723.1136/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1136/kernel
/mnt/rear/linux-h9wr/20210723.1136/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1136/rear-linux-h9wr.log
/mnt/rear/syslinux.cfg
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/ldlinux.sys
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/chain.c32
/mnt/boot/syslinux/lua.c32
/mnt/boot/syslinux/ls.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/message
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/config.c32
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/poweroff.com
/mnt/lost+found
# mount /dev/sdb2 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/rear
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210723.1136
/mnt/rear/linux-h9wr/20210723.1136/backup.log
/mnt/rear/linux-h9wr/20210723.1136/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1134
/mnt/rear/linux-h9wr/20210723.1134/backup.log
/mnt/rear/linux-h9wr/20210723.1134/backup.tar.gz
/mnt/lost+found
that USB disk boots with syslinux
on my older x86_64 laptop with traditional BIOS
and the ReaR recovery system starts up and is usable
for both syslinux boot menue entries i.e.
for the 20210723 1134 backup and
for the 20210723 1136 backup
I also tested that it still works to boot from the local harddisk
via the syslinux boot menue.
jsmeix commented at 2021-07-23 11:33:¶
First step done - phew!
jsmeix commented at 2021-07-23 11:39:¶
@DEvil0000
note that the default boot partition filesystem label is REARBOOT
(to be on the safe side it is only 8 characters long)
but you can specify what you want via
USB_DEVICE_BOOT_LABEL=...
jsmeix commented at 2021-07-23 11:45:¶
The new config variables USB_BOOT_PART_SIZE and
USB_DEVICE_BOOT_LABEL
are intentionally not yet documented in rear/conf/default.conf
because I may have to change their name or usage as needed
until I got a better overview about booting with OUTPUT=USB
Right now I see we have USB_DEVICE_FILESYSTEM_LABEL='REAR-000'
so the config variable for the boot partition filesystem label should
be
better something like
USB_DEVICE_BOOT_FILESYSTEM_LABEL='REARBOOT'
or without the meaningless _DEVICE as in other USB_ config variables
USB_BOOT_FILESYSTEM_LABEL='REARBOOT'
I wished I could change USB_DEVICE_FILESYSTEM_LABEL
to more meaningful USB_DATA_FILESYSTEM_LABEL
but I cannot because changing existing config variable names
would break backward compatibility
jsmeix commented at 2021-07-23 13:02:¶
Tested that things behave sufficiently OK for me
with syslinux/extlinux
without a separated boot partition
with GPT partitioning:
With etc/rear/local.conf
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
USB_DEVICE_PARTED_LABEL=gpt
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
I got
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 25712 date 2021-07-23 14:16:34)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master.with_pull_request_2660/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.cPB1O25rkcYPkxY
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master.with_pull_request_2660/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type gpt on /dev/sdb
Creating BIOS boot partition /dev/sdb1
Setting 'bios_grub' flag on BIOS boot partition /dev/sdb1
Creating ReaR data partition /dev/sdb2 up to 10% of /dev/sdb
Setting 'legacy_boot' flag on ReaR data partition /dev/sdb2
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb2
Adjusting filesystem parameters on ReaR data partition /dev/sdb2
Exiting rear format (PID 25712) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 0.02MiB 8.00MiB 7.98MiB primary bios_grub
2 8.00MiB 47694MiB 47686MiB ext3 primary legacy_boot
# usr/sbin/rear -D mkbackup
...
Created initrd.cgz with gzip default compression (68663760 bytes) in 14 seconds
Saved /root/rear.github.master.with_pull_request_2660/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1418/rear-linux-h9wr.log
Making /dev/sdb bootable with syslinux/extlinux
Writing syslinux MBR /usr/share/syslinux/gptmbr.bin of type gpt to /dev/sdb
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.3ePWn2ThWdxEgR2/outputfs/rear/linux-h9wr/20210723.1418/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 26347) and its descendant processes ...
# usr/sbin/rear -D mkbackup
...
Created initrd.cgz with gzip default compression (68668115 bytes) in 14 seconds
Saved /root/rear.github.master.with_pull_request_2660/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1425/rear-linux-h9wr.log
No need to update syslinux on /dev/sdb that has version 4.04
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.fo1DKTiBKiTD6IW/outputfs/rear/linux-h9wr/20210723.1425/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 18662) and its descendant processes ...
# mount /dev/sdb2 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/ldlinux.sys
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/message
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/chain.c32
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/lua.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/config.c32
/mnt/boot/syslinux/poweroff.com
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/ls.c32
/mnt/lost+found
/mnt/linux-h9wr
/mnt/linux-h9wr/.lockfile
/mnt/rear
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210723.1425
/mnt/rear/linux-h9wr/20210723.1425/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1425/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1425/kernel
/mnt/rear/linux-h9wr/20210723.1425/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1425/backup.log
/mnt/rear/linux-h9wr/20210723.1425/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1418
/mnt/rear/linux-h9wr/20210723.1418/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1418/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1418/kernel
/mnt/rear/linux-h9wr/20210723.1418/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1418/backup.log
/mnt/rear/linux-h9wr/20210723.1418/syslinux.cfg
/mnt/rear/syslinux.cfg
That USB disk boots with syslinux
on my older x86_64 laptop with traditional BIOS.
The ReaR recovery system starts up and is usable
for both syslinux boot menue entries i.e.
for the 20210723 1418 backup and
for the 20210723 1425 backup
I also tested that it still works to boot
from the built-in local harddisk
via the syslinux boot menue default entry.
jsmeix commented at 2021-07-23 13:35:¶
Tested that things behave sufficiently OK for me
with syslinux/extlinux
with a separated boot partition
with GPT partitioning:
With etc/rear/local.conf
OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
USB_BOOT_PART_SIZE=1024
USB_DEVICE_BOOT_LABEL=MY-BOOT
OUTPUT_URL=usb:///dev/disk/by-label/MY-BOOT
USB_DEVICE_PARTED_LABEL=gpt
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/REAR-000
I got
# usr/sbin/rear -D format /dev/sdb
Relax-and-Recover 2.6 / Git
Running rear format (PID 11468 date 2021-07-23 15:06:18)
Command line options: usr/sbin/rear -D format /dev/sdb
Using log file: /root/rear.github.master.with_pull_request_2660/var/log/rear/rear-linux-h9wr.log
Using build area: /tmp/rear.vz43Q29AaAFV8gh
Running workflow format on the normal/original system
USB or disk device /dev/sdb is not formatted with ext2/3/4 or btrfs filesystem
Formatting /dev/sdb will remove all currently existing data on that whole device
UserInput -I USB_DEVICE_CONFIRM_FORMAT needed in /root/rear.github.master.with_pull_request_2660/usr/share/rear/format/USB/default/200_check_usb_layout.sh line 62
Type exactly 'Yes' to format /dev/sdb with ext3 filesystem
(default 'No' timeout 300 seconds)
Yes
UserInput: No choices - result is 'Yes'
Repartitioning /dev/sdb
Creating partition table of type gpt on /dev/sdb
Creating BIOS boot partition /dev/sdb1
Setting 'bios_grub' flag on BIOS boot partition /dev/sdb1
Creating boot partition /dev/sdb2 with size 1024 MiB aligned at 8 MiB
Setting 'legacy_boot' flag on boot partition /dev/sdb2
Creating ReaR data partition /dev/sdb3 up to 10% of /dev/sdb
Creating ext2 filesystem with label 'MY-BOOT' on boot partition /dev/sdb2
Creating ext3 filesystem with label 'REAR-000' on ReaR data partition /dev/sdb3
Adjusting filesystem parameters on ReaR data partition /dev/sdb3
Exiting rear format (PID 11468) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 0.02MiB 8.00MiB 7.98MiB primary bios_grub
2 8.00MiB 1032MiB 1024MiB ext2 primary legacy_boot
3 1032MiB 47694MiB 46662MiB ext3 primary
# usr/sbin/rear -D mkbackup
...
Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression
Created initrd.cgz with gzip default compression (68665418 bytes) in 15 seconds
Saved /root/rear.github.master.with_pull_request_2660/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1507/rear-linux-h9wr.log
Making /dev/sdb bootable with syslinux/extlinux
Writing syslinux MBR /usr/share/syslinux/gptmbr.bin of type gpt to /dev/sdb
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.1r2LBsinngK0QOr/outputfs/rear/linux-h9wr/20210723.1507/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 12166) and its descendant processes ...
# usr/sbin/rear -D mkbackup
...
Created initrd.cgz with gzip default compression (68664065 bytes) in 15 seconds
Saved /root/rear.github.master.with_pull_request_2660/var/log/rear/rear-linux-h9wr.log as rear/linux-h9wr/20210723.1508/rear-linux-h9wr.log
No need to update syslinux on /dev/sdb that has version 4.04
Making backup (using backup method NETFS)
Creating tar archive '/tmp/rear.O7xzn8j1G7msalu/outputfs/rear/linux-h9wr/20210723.1508/backup.tar.gz'
Preparing archive operation
OK
Exiting rear mkbackup (PID 4010) and its descendant processes ...
# mount /dev/sdb2 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/boot
/mnt/boot/syslinux
/mnt/boot/syslinux/disk.c32
/mnt/boot/syslinux/cat.c32
/mnt/boot/syslinux/rear.help
/mnt/boot/syslinux/message
/mnt/boot/syslinux/cpuid.c32
/mnt/boot/syslinux/config.c32
/mnt/boot/syslinux/menu.c32
/mnt/boot/syslinux/sysdump.c32
/mnt/boot/syslinux/rosh.c32
/mnt/boot/syslinux/pci.ids
/mnt/boot/syslinux/chain.c32
/mnt/boot/syslinux/extlinux.conf
/mnt/boot/syslinux/ls.c32
/mnt/boot/syslinux/kbdmap.c32
/mnt/boot/syslinux/host.c32
/mnt/boot/syslinux/hdt.c32
/mnt/boot/syslinux/reboot.c32
/mnt/boot/syslinux/cmd.c32
/mnt/boot/syslinux/vesamenu.c32
/mnt/boot/syslinux/ldlinux.sys
/mnt/boot/syslinux/poweroff.com
/mnt/boot/syslinux/lua.c32
/mnt/lost+found
/mnt/rear
/mnt/rear/syslinux.cfg
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210723.1507
/mnt/rear/linux-h9wr/20210723.1507/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1507/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1507/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1507/kernel
/mnt/rear/linux-h9wr/20210723.1508
/mnt/rear/linux-h9wr/20210723.1508/initrd.cgz
/mnt/rear/linux-h9wr/20210723.1508/rear-linux-h9wr.log
/mnt/rear/linux-h9wr/20210723.1508/syslinux.cfg
/mnt/rear/linux-h9wr/20210723.1508/kernel
/mnt/linux-h9wr
# mount /dev/sdb3 /mnt ; find /mnt ; umount /mnt
/mnt
/mnt/lost+found
/mnt/rear
/mnt/rear/linux-h9wr
/mnt/rear/linux-h9wr/20210723.1508
/mnt/rear/linux-h9wr/20210723.1508/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1508/backup.log
/mnt/rear/linux-h9wr/20210723.1507
/mnt/rear/linux-h9wr/20210723.1507/backup.tar.gz
/mnt/rear/linux-h9wr/20210723.1507/backup.log
That USB disk boots with syslinux
on my older x86_64 laptop with traditional BIOS.
The ReaR recovery system starts up and is usable
for both syslinux boot menue entries i.e.
for the 20210723 1507 backup and
for the 20210723 1508 backup
I also tested that it still works to boot
from the built-in local harddisk
via the syslinux boot menue default entry.
jsmeix commented at 2021-07-23 13:55:¶
By the way FYI regarding partitioning alignment here
with default USB_PARTITION_ALIGN_BLOCK_SIZE="8"
and USB_DEVICE_FILESYSTEM_PERCENTAGE=10:
The alignment at 8 MiB byte values (8 MiB = 8388608 Bytes)
is only meant for the partition start values
but neither for partition sizes or partition end values.
Accordingly with USB_DEVICE_FILESYSTEM_PERCENTAGE=10
the data partition goes up to what parted does when 10%
is specified as end value for creating the partition because it is
called as
parted -s /dev/sdb unit B mkpart primary 1082130432 10%
The parted result is
# parted -s /dev/sdb unit B print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 500107859968B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 24576B 8388607B 8364032B primary bios_grub
2 8388608B 1082130431B 1073741824B ext2 primary legacy_boot
3 1082130432B 50010783743B 48928653312B ext3 primary
So the boot partition start value 8388608 is exactly 8 MiB
and the data partition start value 1082130432 is exactly 129 * 8 MiB
but the data partition end value 50010783743 is exactly 47694 MiB - 1
Byte
which is not a multiple of 8 MiB (50010783744 = 5961.75 * 8 MiB = 23847
* 2 MiB)
and the data partition size 48928653312 = 5832.75 * 8 MiB = 23331 * 2
MiB)
so the data partition size and end values are at least aligned to 2 MiB.
jsmeix commented at 2021-07-23 14:04:¶
@DEvil0000 @rear/contributors
enough for me for this week.
GRUB2 support comes next week via
https://github.com/rear/rear/pull/2659
I wish you a relaxed and recovering weekend!
@DEvil0000
many thanks for your contributions that make ReaR move forward.
Without that I would not have had a look at our old and somewhat messy
USB code.
I know since a long time it is messy and I tried to avoid it but now you
forced me ;-)
jsmeix commented at 2021-07-23 14:25:¶
This pull request is a part of https://github.com/rear/rear/issues/2648
DEvil0000 commented at 2021-07-23 15:35:¶
many thanks for your contributions that make ReaR move forward.
Without that I would not have had a look at our old and somewhat messy USB code.
I know since a long time it is messy and I tried to avoid it but now you forced me ;-)
It's a pleasure ;)
As a additional note on the label - this is not an optimal approach since it will not easily allow for two USB devices at the same time
jsmeix commented at 2021-07-26 06:34:¶
@DEvil0000
I do not understand what you mean with your "additional note on the
label"
why "it will not easily allow for two USB devices at the same time"?
Currently I use only one USB disk for my testing so I did not yet test
how more than one USB disk at the same time actually behaves.
I think because one can specify the labels in etc/rear/local.conf
one can "rear format" several USB devices as needed and
because "rear mkbackup" will use the USB device via the label
that is specified in etc/rear/local.conf one can use several USB
devices
but one must use each time the right etc/rear/local.conf as intended.
Because the used etc/rear/local.conf is copied into the ReaR recovery
system
the recovery system on a particular USB device has its matching label
settings
so that "rear recover" should work as expected for each particular USB
device.
E.g. when I specify in etc/rear/local.conf (excerpts only what is about the labels):
USB_DEVICE_FILESYSTEM_LABEL='MY-DATA'
USB_DEVICE_BOOT_LABEL=MY-BOOT
OUTPUT_URL=usb:///dev/disk/by-label/MY-BOOT
BACKUP_URL=usb:///dev/disk/by-label/MY-DATA
I get
# usr/sbin/rear -D format /dev/sdb
...
Repartitioning /dev/sdb
Creating partition table of type gpt on /dev/sdb
Creating BIOS boot partition /dev/sdb1
Setting 'bios_grub' flag on BIOS boot partition /dev/sdb1
Creating boot partition /dev/sdb2 with size 1024 MiB aligned at 8 MiB
Setting 'legacy_boot' flag on boot partition /dev/sdb2
Creating ReaR data partition /dev/sdb3 up to 10% of /dev/sdb
Creating ext2 filesystem with label 'MY-BOOT' on boot partition /dev/sdb2
Creating ext3 filesystem with label 'MY-DATA' on ReaR data partition /dev/sdb3
Adjusting filesystem parameters on ReaR data partition /dev/sdb3
Exiting rear format (PID 6856) and its descendant processes ...
# parted -s /dev/sdb unit MiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 476940MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 0.02MiB 8.00MiB 7.98MiB primary bios_grub
2 8.00MiB 1032MiB 1024MiB ext2 primary legacy_boot
3 1032MiB 47694MiB 46662MiB ext3 primary
# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,LABEL /dev/sdb
NAME TRAN TYPE FSTYPE SIZE LABEL
/dev/sdb usb disk 465.8G
|-/dev/sdb1 part 8M
|-/dev/sdb2 part ext2 1G MY-BOOT
`-/dev/sdb3 part ext3 45.6G MY-DATA
# find /dev/disk/by-*/ -ls | grep 'MY-'
... /dev/disk/by-label/MY-DATA -> ../../sdb3
... /dev/disk/by-label/MY-BOOT -> ../../sdb2
# usr/sbin/rear -D mkbackup
...
# grep 'MY-' var/log/rear/rear-linux-h9wr.log | grep '+ mount '
+++ mount -v -o rw,noatime /dev/disk/by-label/MY-DATA /tmp/rear.ZPx348oTd5BfuAj/outputfs
+++ mount -v -o rw,noatime /dev/disk/by-label/MY-BOOT /tmp/rear.ZPx348oTd5BfuAj/outputfs
# grep 'MY-' /tmp/rear.ZPx348oTd5BfuAj/rootfs/etc/rear/local.conf
USB_DEVICE_FILESYSTEM_LABEL='MY-DATA'
USB_DEVICE_BOOT_LABEL=MY-BOOT
OUTPUT_URL=usb:///dev/disk/by-label/MY-BOOT
BACKUP_URL=usb:///dev/disk/by-label/MY-DATA
DEvil0000 commented at 2021-07-26 16:15:¶
Yes, this is what I ment. One needs to changge the config (label config) when creating different usb sticks.
jsmeix commented at 2021-07-27 06:35:¶
@DEvil0000
could you file a new separated issue via
https://github.com/rear/rear/issues/new
where you describe your use case with different USB disks
so that we could better understand what you need?
Currently I do not understand how using different USB disks
could work without having different configs for them.
One can use different USB disks with the same config (i.e. same
filesystem labels)
when one likes to have redundancy for USB disks e.g. using two USB
disks
interchangeably to have another USB disk if one breaks but then those
two USB disks are not meant to be different but clones of each other.
Such USB disk clones would be same from ReaR's point of view
(in particular both disks would behave same for "rear recover")
but they could be made "looking different" for humans e.g. by
marking them differently with different physical paper labels
(e.g. "clone 1" and "clone 2" written on them)
that get glued on the case of the disk.
jsmeix commented at 2022-06-02 11:31:¶
With
https://github.com/rear/rear/pull/2815
merged
the new user config variables
USB_BOOT_PART_SIZE and USB_DEVICE_BOOT_LABEL
are documented in default.conf
In particular see my above
https://github.com/rear/rear/pull/2660#issuecomment-885582656
Because I did not find really better names I keep the
currently implemented user config variable names
USB_BOOT_PART_SIZE and USB_DEVICE_BOOT_LABEL
[Export of Github issue for rear/rear.]