#2699 PR merged: Overhauled serial console support code

Labels: enhancement, cleanup, fixed / solved / done, minor bug

jsmeix opened issue at 2021-10-19 10:51:

jsmeix commented at 2021-10-19 11:13:

@DEvil0000
could you test if my overhauled serial console code
in this pull request still works for you?

I found out why only a serial device was added
to kernel commandline and recovery system bootloaders
if there was a "speed" value for that serial device:
See my comment for the new get_serial_device_speed function in
https://github.com/rear/rear/blob/093bd1bc58bf68819f731ce374469147e06c2f15/usr/share/rear/lib/serial-functions.sh

jsmeix commented at 2021-10-19 13:29:

@DEvil0000
with my recent commits here I enhanced how
SERIAL_CONSOLE_DEVICE_GRUB can be used.
Now the user can alternatively specify a whole
GRUB 'serial' command that is used as specified, e.g.
SERIAL_CONSOLE_DEVICE_GRUB="serial --unit=0 --speed=9600"
See my updated description for it in default.conf.

jsmeix commented at 2021-10-19 13:43:

@DEvil0000
if my current SERIAL_CONSOLE_DEVICE_GRUB implementation
works well for you (or if you find bugs that can be easily corrected)
i.e. when you agree with the intent of my current
SERIAL_CONSOLE_DEVICE_GRUB implementation
then I could try to implement the same kind also for
SERIAL_CONSOLE_DEVICE_SYSLINUX

DEvil0000 commented at 2021-10-19 17:36:

I found out why only a serial device was added

makes sense. That was my guess, too

Now the user can alternatively specify a whole GRUB 'serial' command

sounds good but not sure if that much control was needed

could you test if my overhauled serial console code

will do as soon as I find the time for it. maybe tomorrow - hopefully this week.

jsmeix commented at 2021-10-20 09:38:

That SERIAL_CONSOLE_DEVICE_GRUB can be a whole GRUB 'serial' command
was more a nice side effect of my added safeguard condition
if test -c $SERIAL_CONSOLE_DEVICE_GRUB ; then
because that made me think about the else case.
Instead of just show an error for that else case
I found it more helpful to support a whole GRUB 'serial' command.
Of course then SERIAL_CONSOLE_DEVICE_GRUB="/dev/sda1" results
a broken GRUB config but that is what I mean with "final power to the user".
When the user manually specifies bad things then bad things will happen.
ReaR runs as 'root' and ReaR is meant for experienced admins.

jsmeix commented at 2021-10-20 12:44:

I cannot really test because my homeoffice laptop has no
real serial hardware so that for all its serial device nodes
stty -F /dev/ttyS* results "Input/output error" so not any 'speed' value
so there is not any (semi)-automated serial console setup.

The only test I could do so far is with

OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
USB_DEVICE_FILESYSTEM_LABEL='MY-DATA'
USB_BOOT_PART_SIZE=1024
USB_BOOTLOADER="grub"
USB_DEVICE_BOOT_LABEL=MY-BOOT
OUTPUT_URL=usb:///dev/disk/by-label/MY-BOOT
USB_DEVICE_PARTED_LABEL=gpt
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/MY-DATA
USE_SERIAL_CONSOLE="yes"
SERIAL_CONSOLE_DEVICE_GRUB="serial --unit=4 --speed=9600"
KERNEL_CMDLINE+=" console=ttyS4,9600 "

which results on my USB disk

# parted -s /dev/sdb unit GiB print
Model: TOSHIBA External USB 3.0 (scsi)
Disk /dev/sdb: 466GiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 
Number  Start    End      Size     File system  Name     Flags
 1      0.00GiB  0.01GiB  0.01GiB               primary  bios_grub
 2      0.01GiB  1.01GiB  1.00GiB  ext2         primary  legacy_boot
 3      1.01GiB  46.6GiB  45.6GiB  ext3         primary

# lsblk -ipo NAME,TRAN,TYPE,FSTYPE,SIZE,LABEL /dev/sdb
NAME        TRAN TYPE FSTYPE   SIZE LABEL
/dev/sdb    usb  disk        465.8G 
|-/dev/sdb1      part            8M 
|-/dev/sdb2      part ext2       1G MY-BOOT
`-/dev/sdb3      part ext3    45.6G MY-DATA

this boot/grub2/grub.cfg file on sdb2

search --no-floppy --set=root --label MY-BOOT
insmod all_video
set gfxpayload=keep
insmod part_gpt
insmod part_msdos
insmod ext2
serial --unit=4 --speed=9600
set timeout="300"
set default="chainloader"
set fallback="chainloader"
echo 'Switching to GRUB2 boot menu...'
sleep --verbose --interruptible 3
menuentry "Relax-and-Recover (BIOS or UEFI in legacy BIOS mode)" --id=rear {
    insmod gzio
    echo 'Loading kernel /rear/linux-h9wr/20211020.1437/kernel ...'
    linux /rear/linux-h9wr/20211020.1437/kernel root=UUID=f05af948-6075-40a3-9191-354b0a0a9afc  selinux=0 console=tty0 
    echo 'Loading initial ramdisk /rear/linux-h9wr/20211020.1437/initrd.cgz ...'
    initrd /rear/linux-h9wr/20211020.1437/initrd.cgz
}
menuentry "Boot from second disk hd1 (usually the original system disk)" --id=chainloader {
    insmod chain
    set root=(hd1)
    chainloader +1
}
menuentry "Reboot" --id=reboot {
    reboot
}
menuentry "Exit (possibly continue bootchain)" --id=exit {
    exit
}

with fallback KERNEL_CMDLINE=' selinux=0 console=tty0 '
in contrast to what I had specified in local.conf
KERNEL_CMDLINE+=" console=ttyS4,9600 ".
This is because cmdline_add_console() in usr/share/rear/lib/serial-functions.sh
strips all existing 'console=...' parameters from the kernel commandline
that code was before in rescue/GNU/Linux/400_use_serial_console.sh
cf. https://github.com/rear/rear/commit/009b1b43eb18df78fc5c4dde1e9268376d948bb4
but if no serial device node has a 'speed' value it adds the fallback 'console=tty0'
so I wonder how I could provide same final power to the user
also for serial console setup of his kernel commandline
as he has for his serial console setup for SYSLINUX and GRUB.

jsmeix commented at 2021-10-21 08:13:

With my recent enhancement SERIAL_CONSOLE_DEVICES_KERNEL
I can specify serial consoles for the kernel of the recovery system
as I like:

OUTPUT=USB
USB_DEVICE_FILESYSTEM_PERCENTAGE=10
USB_DEVICE_FILESYSTEM_LABEL='MY-DATA'
USB_BOOT_PART_SIZE=1024
USB_BOOTLOADER="grub"
USB_DEVICE_BOOT_LABEL=MY-BOOT
OUTPUT_URL=usb:///dev/disk/by-label/MY-BOOT
USB_DEVICE_PARTED_LABEL=gpt
BACKUP=NETFS
BACKUP_URL=usb:///dev/disk/by-label/MY-DATA
USE_SERIAL_CONSOLE="yes"
SERIAL_CONSOLE_DEVICE_GRUB="serial --unit=4 --speed=9600"
SERIAL_CONSOLE_DEVICES_KERNEL="/dev/ttyS7 console=ttyS8,9600 console=tty0"

results this boot/grub2/grub.cfg file on my sdb2 USB boot partition

search --no-floppy --set=root --label MY-BOOT
insmod all_video
set gfxpayload=keep
insmod part_gpt
insmod part_msdos
insmod ext2
serial --unit=4 --speed=9600
set timeout="300"
set default="chainloader"
set fallback="chainloader"
echo 'Switching to GRUB2 boot menu...'
sleep --verbose --interruptible 3
menuentry "Relax-and-Recover (BIOS or UEFI in legacy BIOS mode)" --id=rear {
    insmod gzio
    echo 'Loading kernel /rear/linux-h9wr/20211021.0955/kernel ...'
    linux /rear/linux-h9wr/20211021.0955/kernel root=UUID=f05af948-6075-40a3-9191-354b0a0a9afc  selinux=0 console=ttyS7 console=ttyS8,9600 console=tty0 
    echo 'Loading initial ramdisk /rear/linux-h9wr/20211021.0955/initrd.cgz ...'
    initrd /rear/linux-h9wr/20211021.0955/initrd.cgz
}
menuentry "Boot from second disk hd1 (usually the original system disk)" --id=chainloader {
    insmod chain
    set root=(hd1)
    chainloader +1
}
menuentry "Reboot" --id=reboot {
    reboot
}
menuentry "Exit (possibly continue bootchain)" --id=exit {
    exit
}

Booting from that USB disk works well for me
on my older x86_64 laptop with traditional BIOS
that has no serial hardware ("stty -F /dev/ttyS..." show "Input/output error")
so I see GRUB error messages about unavailable serial things
(because of "sleep --verbose --interruptible 3" one can see such messages)
but GRUB is sufficiently fail-safe and shows its menu on the VGA console
and same for the kernel that also uses the VGA console.
In the booted recovery system "dmesg" shows
"... console=ttyS7 console=ttyS8,9600 console=tty0"
in the kernel command line.

jsmeix commented at 2021-10-21 10:17:

@DEvil0000
because my above little test without real serial hardware
https://github.com/rear/rear/pull/2699#issuecomment-948367891
worked at least for me I think it this pull request is now in a state
where you could also test it - preferably with a real serial console.
If you find issues I will fix them (as good as I can).

jsmeix commented at 2021-10-21 10:36:

I wonder why we explicitly set hardcoded GRUB serial options

--word=8 --parity=no --stop=1

when those are the defaults in GRUB according to
https://www.gnu.org/software/grub/manual/grub/grub.html#serial
that reads (excerpt)

word and stop are the number of data bits and stop bits.
Data bits must be in the range 5-8 and stop bits must be 1 or 2.
Default is 8 data bits and one stop bit.
parity is one of ‘no’, ‘odd’, ‘even’ and defaults to ‘no’.

I think it is better to not explicitly enforce default values
because that is against what is meant with defaults.

With SERIAL_CONSOLE_DEVICE_GRUB the user can specify
a whole GRUB 'serial' command with all options
when he needs non-default options like

SERIAL_CONSOLE_DEVICE_GRUB="serial --unit=2 --speed=115200 --word=7 --stop=2 --parity=odd"

jsmeix commented at 2021-10-26 12:16:

@rear/contributors
I would like to merge it tomorrow afternoon
unless there are objections.

DEvil0000 commented at 2021-10-26 16:33:

I was a bit busy the last days. Will try to test it tomorrow

jsmeix commented at 2021-10-27 05:38:

@DEvil0000
OK - I will wait some more days so you could test it.

DEvil0000 commented at 2021-10-27 12:34:

base config used:

OUTPUT=USB
BACKUP=BORG
USB_DEVICE=/dev/disk/by-label/REAR-000
USB_DEVICE_FILESYSTEM_LABEL=REAR-000
USB_DEVICE_PARTED_LABEL=gpt
USB_DEVICE_FILESYSTEM=ext4
USB_UEFI_PART_SIZE="2048"
CLONE_USERS=(root)
CLONE_GROUPS=(root)
CLONE_ALL_USERS_GROUPS="false"
BORGBACKUP_REPO="/borg"
BORGBACKUP_UMASK="0002"
BORGBACKUP_ENC_TYPE="repokey"
export BORG_RELOCATED_REPO_ACCESS_IS_OK="yes"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK="yes"
export TMPDIR="/tmp/"
USE_RESOLV_CONF="no"
USE_DHCLIENT=no
USE_STATIC_NETWORKING=1
USE_SERIAL_CONSOLE=y
COPY_KERNEL_PARAMETERS=( 'net.ifnames' 'biosdevname', 'console' )
USING_UEFI_BOOTLOADER=0
USB_BOOTLOADER=grub
GRUB2_DEFAULT_BOOT="1"
#speedup testing:
REAR_INITRD_COMPRESSION="fast"
MODULES=( 'no_modules' )
FIRMWARE_FILES=( 'no' )
EXCLUDE_RUNTIME_LOGFILE="yes"

added for boot partition test as needed:

USB_BOOT_PART_SIZE="2048"
OUTPUT_URL=usb:///dev/disk/by-label/REARBOOT

I tested in addition the following serial config combinations:

SERIAL_CONSOLE_DEVICES="/dev/ttyS0"

SERIAL_CONSOLE_DEVICES=
SERIAL_CONSOLE_DEVICE_GRUB="serial --unit=1 --speed=115200"
SERIAL_CONSOLE_DEVICES_KERNEL="/dev/ttyS0"

SERIAL_CONSOLE_DEVICES=
SERIAL_CONSOLE_DEVICES_KERNEL="/dev/ttyS0"
SERIAL_CONSOLE_DEVICE_SYSLINUX="serial 1 115200"
USB_BOOTLOADER=syslinux

issues found:

  • USB_BOOT_PART_SIZE is not set by default causing a missing boot partition. a default of at least 1gb would make sense; min about 512mb; I like 2gb personally. I think it should be default to work well with all output modes. grub config is slightly wrong in this case and looking for label REARBOOT even it it does not exist but it still works.
  • if USB_BOOT_PART_SIZE is set the output URL default is wrong. it points to data not to boot. should be OUTPUT_URL=usb:///dev/disk/by-label/REARBOOT
  • mkrescue fails if data partition is mounted
  • default description is wrong: # SERIAL_CONSOLE_DEVICES_KERNEL="/dev/ttyS0 console=ttyS1,9600 console=tty0". the device should not be there but console= stuff should be

so the serial part looks good to me

jsmeix commented at 2021-10-28 06:45:

@DEvil0000
thank you for your review!
It helps us so much to have a review by one who actually has serial hardware.

Regarding USB_BOOT_PART_SIZE or other OUTPUT=USB issues:
I will do further enhancements and cleanup for OUTPUT=USB
see https://github.com/rear/rear/issues/2666
as time permits (but before ReaR 2.7 will be released).

Regarding SERIAL_CONSOLE_DEVICES_KERNEL description in default.conf

SERIAL_CONSOLE_DEVICES_KERNEL="/dev/ttyS0 console=ttyS1,9600 console=tty0"

I think it is right because I had tested

SERIAL_CONSOLE_DEVICES_KERNEL="/dev/ttyS7 console=ttyS8,9600 console=tty0"

and got in the kernel command line

console=ttyS7 console=ttyS8,9600 console=tty0

see https://github.com/rear/rear/pull/2699#issuecomment-948367891

jsmeix commented at 2021-10-28 06:46:

I would like to merge it today afternoon
unless there are objections.

If the SERIAL_CONSOLE_DEVICES_KERNEL description in default.conf
is really wrong I would fix it afterwards.

[Export of Github issue for rear/rear.]