#878 Issue `closed`: Can't see login prompt on serial console¶

Labels: enhancement, support / question, fixed / solved / done

bryan06 opened issue at 2016-06-15 09:05:¶

rear version (/usr/sbin/rear -V): Relax-and-Recover 1.18 / Git
OS version (lsb_release -a):

LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: RedHatEnterpriseServer
Description: Red Hat Enterprise Linux Server release 6.4 (Santiago)
Release: 6.4
Codename: Santiago

rear configuration files (cat /etc/rear/site.conf or cat /etc/rear/local.conf):
OUTPUT=PXE
BACKUP=NETFS
OUTPUT_URL=nfs://nfsserver/srv/tftp
BACKUP_URL=nfs://nfsserver/home/nfs/backups
BACKUP_TYPE=incremental
PXE_CONFIG_PATH=/srv/tftp/pxelinux.cfg
Brief description of the issue
I did a rescue image of a High Performance Computer, everything went well. Connected on serial console (ttyS1) I tried to restore the system using PXE boot method, the kernel and initrd are loaded and then everything seems to work normally but the process stops at : * * * Rescue Image is Ready * * *
and I never see the login prompt.. I don't know if it is a serial console issue or something else
Work-around, if any: I deleted the option vga=normal (there is no VGA card on the machine) in the kernel boot parameters and I have added the line co:23:respawn:/sbin/agetty ttyS1 115200 vt100-nav
in
/usr/share/rear/skel/default/etc/inittab
and
/usr/share/rear/skel/Linux-ppc64/etc/inittab

gdha commented at 2016-06-15 10:59:¶

Try adding USE_SERIAL_CONSOLE=y in /etc/rear/local.conf file

bryan06 commented at 2016-06-15 11:09:¶

Thank you for your quick reply, unfortunately I am a student doing an internship so I won't be able to test until tomorrow, I am sorry.. But I will keep you informed and tell you the result tomorrow as soon as I will be at work.

bryan06 commented at 2016-06-16 07:25:¶

Adding the line USE_SERIAL_CONSOLE=y did not work.. My kernel boot parameters look like this :
root=/dev/ram0 rd.debug rw selinux=0 console=ttyS1,115200

gdha commented at 2016-06-17 14:35:¶

@bryan06 In recover mode the script /usr/share/rear/skel/default/etc/scripts/system-setup.d/45-serial-console.sh should add an entry to the /etc/inittab during boot up time. However, it check if stty is present. Could you verify if that is the case?

bryan06 commented at 2016-06-17 15:36:¶

@gdha I have verified the script execution during boot up time and stty does find ttyS0 and ttyS1, the script adds the entries in /etc/inittab and lauches init q. The only error I get is that it could not find the file /etc/init.conf.
P.S : I am using IPMI

gdha commented at 2016-06-20 08:56:¶

@bryan06 The message init: /etc/init.conf: Unable to load configuration: No such file or directory is normal as there is no /etc/init.conf present on most systems.
However, if you say ttyS1 has been added the the /etc/inittab why you didn't get a prompt?

bryan06 commented at 2016-06-20 09:02:¶

@gdha I don't know ... There is a blank line after * * * Rescue Image is Ready * * * and I can't do anything else. I tried to boot the rescue image from a Virtualbox VM and it worked correctly, so I think it is a console issue but I don't have any idea of what could be misconfigured..

gdha commented at 2016-12-31 09:57:¶

@bryan06 According http://ibiblio.org/gferg/ldp/IPMI_on_Debian.html it IPMI uses a serial console, or did you drop ReaR altogether ?

gdha commented at 2017-01-17 12:45:¶

No response - close it

didacog commented at 2017-11-20 09:59:¶

Hi @gdha,

I'm just having same problem with rear 2.2-5 rhel7.2 in a Physical Host with VGA and Serial.

USE_SERIAL_CONSOLE=y
/etc/inittab with the console configured in rescue image, but no prompt in serial, and correct prompt in VGA.

If I run agetty ttyS0 115200 ... from the VGA prompt then I loss the VGA prompt and the serial start working.

I'm trying to understand what is happening, but any help will be appreciated.

Kind regards,
Didac

gdha commented at 2017-11-20 10:46:¶

I guess on RHEL7 the /etc/inittab file is not read anymore. I found the following at RH:

Edit /etc/default/grub file. Find the clause GRUB_CMDLINE_LINUX and append the clause console=ttyS0,115200. For example:

GRUB_CMDLINE_LINUX="rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb video=640x400 quiet console=ttyS0,115200"

And then, generate a fresh GRUB configuration:

# grub2-mkconfig -o /etc/grub2.cfg

However, we need to figure out how to do this with our rescue image - hope it helps?

didacog commented at 2017-11-20 11:45:¶

Yes, I've added these config in the GRUB config:

GRUB_CMDLINE_LINUX="ipv6.disable=1 crashkernel=auto rd.lvm.lv=vgroot/lv_root rd.lvm.lv=vgroot/lv_swap nomodeset LANG=en_US.UTF-8 KEYBOARDTYPE=pc KEYTABLE=es transparent_hugepage=never intel_idle.max_cstate=1 processor.max_cstate=1 elevator=noop vga=normal earlyconn=uart8250,io,0x3f8,115200n8 console=ttyS0,115200"

To configure it OK in ReaR I needed to set:
KERNEL_CMDLINE="earlyconn=uart8250,io,0x3f8,115200n8"

But at the end the serial console hang without prompt.
I'm looking at it and I will resolve this problem soon :P I guess :D

I will update the issue with my results and questions.

didacog commented at 2017-11-23 16:33:¶

Hi,

Well... I found some interesting things :-P

When ReaR detects serial it configures it in inittab, and this was working until systemd.

With systemd there are some strange things in the skel that I had to understand how they work (I'm not systemd expert :P)

The resulting KERNEL_CMDLINE with serial found in ReaR is: ....console=ttyS0,115200 console=tty0

the serial-getty@.service is not configured, or at least does not work, the service file exist, but seems no one is using it. and it has a typo:
is:
ExecStart=-/bin/agetty -s %I 115200,38400,9600
should be:
ExecStart=-/sbin/agetty -s %I 115200,38400,9600

But with this changed, still does not work...

I found a way to make it working, but I don't know if is the best option, following the changes I made to an installed system with rear:


cd /usr/share/rear/skel/default/usr/lib/systemd/system/

vi serial-getty@.service
---
ExecStart=-/sbin/agetty -s %I 115200,38400,9600
...
...
[Install]
Alias=serial-getty.target.wants/serial-getty@ttyS0.service
---

vi getty@.service
------
...
...
[Install]
Alias=getty.target.wants/getty@tty0.service
------

vi serial-getty.target
------
#  This file is part of systemd.
#

[Unit]
Description= Serial Login Prompts
-------

mkdir serial-getty.target.wants
cd serial-getty.target.wants/
ln -s ../serial-getty@.service serial-getty@ttyS0.service
cd multi-user.target.wants/
ln -s ../serial-getty.target serial-getty.target

cd getty.target.wants/
rm getty@tty{1,2,3,4}.service
ln -s ../getty@.service getty@tty0.service

I've removed the tty1-4 services, because at login ig tty1 and tty0 are present, both services get started and the graph console prompt is impossible to enter commands. and if I stop any of them from the serial console then works, but if I keep tty1 does not work at startup. :P

Let me know if this solution works also for you please, if is the case, I will send a PR asap.

I've tested this with RHEL 7.2 and HP SuperdomeX servers. using serial console connection and ILO console.

Regards,
Didac

didacog commented at 2017-11-24 11:50:¶

Hi, I'm testing with other changes that seems to work better, when the tests were finished I will update the issue with the final proposal, I guess :P

didacog commented at 2017-11-24 15:58:¶

@gdha

I found a better solution. We tested it with RHEL7.2 on HP supoerdomeX hardware with ILO console and serial console and with Debian 9 on libvirt with and without serial port attached.

These changes worked in our tests:

usr/share/rear/skel/default/usr/lib/systemd/system/getty@.service

#  This file is part of systemd.
#

[Unit]
Description=Getty on %I
Documentation=man:agetty(8)
After=systemd-user-sessions.service plymouth-quit-wait.service
After=sysinit.service

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target
IgnoreOnIsolate=yes

# On systems without virtual consoles, don't start any getty. (Note
# that serial gettys are covered by serial-getty@.service, not this
# unit
#ConditionPathExists=/dev/tty0

[Service]
Environment=TERM=linux
ExecStart=-/sbin/agetty %I 38400
Restart=always
RestartSec=0
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
TTYVTDisallocate=yes
KillMode=process
IgnoreSIGPIPE=no

# Unset locale for the console getty since the console has problems
# displaying some internationalized messages.
Environment=LANG= LANGUAGE= LC_CTYPE= LC_NUMERIC= LC_TIME= LC_COLLATE= LC_MONETARY= LC_MESSAGES= LC_PAPER= LC_NAME= LC_ADDRESS= LC_TELEPHONE= LC_MEASUREMENT= LC_IDENTIFICATION=

# Some login implementations ignore SIGTERM, so we send SIGHUP
# instead, to ensure that login terminates cleanly.
KillSignal=SIGHUP

[Install]
WantedBy=getty.target
DefaultInstance=tty0

usr/share/rear/skel/default/usr/lib/systemd/system/serial-getty@.service

#  This file is part of systemd.
#
[Unit]
Description=Serial Getty on %I
BindTo=dev-%i.device
After=dev-%i.device

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target

[Service]
Environment=TERM=vt100
ExecStart=-/sbin/agetty -s %I 115200,38400,9600
Restart=always
RestartSec=0
UtmpIdentifier=%I
KillMode=process

# Some login implementations ignore SIGTERM, so we send SIGHUP
# instead, to ensure that login terminates cleanly.
KillSignal=SIGHUP

[Install]
WantedBy=getty.target

IN usr/share/rear/skel/default/usr/lib/systemd/system/getty.target.wants/

rm getty@tty{1,2,3,4}.service
ln -s ../getty@.service getty@tty0.service
ln -s ../serial-getty@.service serial-getty@ttyS0.service

Let me know if this seems ok for you, and if you can test it. I will send a PR if seems OK for you.