Playing with serial consoles in Linux
We all administer our homelab servers by ssh’ing to them; and probably these servers are also headless. What happens if, in this scenario, you botch your networking? How do you access them to restore connectivity without using a crash cart? There is a way: connecting to a TTY via serial port.
If the computer doesn’t have a native serial port, it is possible to connect to one with an usb-serial adapter.
Activation con be done by bringing up the proper systemd service (serial-getty or getty), on the right port. Also it can be configured as a kernel parameter (which later will bring up the systemd daemon). The second way has the advantage that you can potentially view your server boot up messages.
So far, I’ve experimented with the following:
- ttyUSB0 on Proxmox, Fedora 38 and Debian 11.
- ttyS0 on Proxmox.
Wherever configurable, I’d rather use 115200 as baud speed.
Console on serial port (console ttySx)
The first serial port appears in /dev
as ttyS0, to to activate the serial console:
systemctl enable serial-getty@ttyS0.service
systemctl start serial-getty@ttyS0.service
The other way is modifying the /etc/kernel/cmdline
(if your system uses systemd-boot, if it uses grub it has to be modified in grub.conf
): append console=tty0 console=ttyS0,115200n8
to the existing config (if it exists) and then proxmox-boot-tool refresh
Console on serial port port USB connection (ttyUSBx)
In Fedora 38 it’s neccesary to deactivate SELinux for the serial-console@ttyUSB0.service
can open the serial port for the tty.
Getting ttyUSB0 to show boot time messages is more difficult, because the kernel has to be compiled with support for it. From my investigations, some distros -like Fedora- come with most neccesary modules compiled into the kernel, being only necessary loading the serial cable module in initramfs.
TO-DO:
- Learn differences between serial-getty and getty.
- Try to observe the boot process via ttyUSB0.
- Investigate why in Proxmox when the system boots, the baud rate gets downgraded from 115200 to 9600.
- Issues with h2’s Dell and HP.
Resources
- https://unix.stackexchange.com/questions/242778/what-is-the-easiest-way-to-configure-serial-port-on-linux
- http://0pointer.de/blog/projects/serial-console.html
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=868352
- https://www.coreboot.org/GRUB2#On_a_USB_serial_or_USB_debug_adapter
- https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdSerialPortsFixedBaud
- https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html
- https://src.fedoraproject.org/rpms/kernel/blob/rawhide/f/kernel-x86_64-fedora.config
- https://pve.proxmox.com/wiki/Host_Bootloader#sysboot_edit_kernel_cmdline
- https://wiki.archlinux.org/title/working_with_the_serial_console