Dissection of live boot rescue images

My arch install broke because of a module rename. All I would see was something like

Failed to open lvm (initramfs) Cannot open access to console, the root account is locked. See sulogin(8) man page for more details. See journal for logs - yada yada

Since root was unable to be mounted, no journal persisted. Hence, I cannot see the journal logs, can I ?

Anyway, the fix was simple. Get a rescue boot media like grml and regenerate initramfs using lvm2 instead of sd-lvm2.

Now, I was able to get grml on a pen drive to work on this machine but I’m not always this lucky (all my usb ports are bricked and only work when the shrodinger’s cat meows).
Also, I’m too lazy to plug a pen drive next time. Since I just used grml via a pendrive, can I have it boot from the hard disk too ?

grml’s website says: no

Is it possible to install Grml to harddisk?

No. If you want to install a Debian system take a look at grml-debootstrap (or use the Debian Installer instead).

Well, not every no is a no.

grml uses grub as bootloader with the following entry:

It sets kernel parameters to enable live boot via boot=live but no such kernel parameter exists.
It turns out, init programs inside the initramfs can look for these parms and set the system for bootup.

One such program is live-boot by debian.

live-boot is a collection of scripts providing hooks for the initramfs-tools, used to generate an initramfs capable of booting live systems, such as those created by live-build. This includes the live system ISOs, netboot tarballs, and USB stick images.

Arch docs also mention how to have grml directly on my esp (/boot).

Anyway, I don’t have enough space on the esp to store entire grml (~500M) and it doesn’t mention how can I point the live-media-path to a different partition. So, lets see how live-boot works.

live-boot⌗

live-boot project on commit ff8867c does the following:

1. Try 60 times, 1 per second, to find a live-bootable file system

   sh9990-main.sh

   # Scan local devices for the image i=0 while [ "$i" -lt 60 ] do livefs_root=$(find_livefs ${i}) if [ -n "${livefs_root}" ] then break fi sleep 1 i=$((i + 1)) done

   yes, you’ve 60 seconds to make your livefs available

2. Unless LIVE_MEDIA is specified on command line, check removable then non removable devices

   sh9990-misc-helpers.sh

   find_livefs () { ... devices_to_scan="$(removable_dev 'sys') $(non_removable_dev 'sys')" ... for dev in $(subdevices "${sysblock}") do if check_dev "${dev}" then return 0 fi done ... }

3. Mount whatever is mountable (also tries lvm, iso etc) and check inside

   sh9990-misc-helpers.sh

   check_dev () { ... mount -t ${fstype} -o ro,noatime "${devname}" ${mountpoint} || continue ... if is_live_path ${mountpoint} && \ ([ "${skip_uuid_check}" ] || matches_uuid ${mountpoint}) then echo ${mountpoint} return 0 else umount ${mountpoint} 2>/dev/null fi ... }

4. If a file with .squashfs or other fs extension exist inside a directory called live/ (LIVE_MEDIA_PATH="live" in 0001-init-vars.sh) boot from here.

   bash9990-misc-helpers.sh

   is_live_path() { DIRECTORY="${1}/${LIVE_MEDIA_PATH}" for FILESYSTEM in squashfs ext2 ext3 ext4 xfs dir jffs do if ls "${DIRECTORY}/"*.${FILESYSTEM} > /dev/null 2>&1 then return 0 fi done return 1 }

5. And finally mount the live filesystem’s root to /root via setup_unionfs.

   sh9990-main.sh

   if [ -n "${MODULETORAMFILE}" ] || [ -n "${PLAIN_ROOT}" ] then setup_unionfs "${livefs_root}" "${rootmnt?}" else mac="$(get_mac)" mac="$(echo "${mac}" | sed 's/-//g')" mount_images_in_directory "${livefs_root}" "${rootmnt}" "${mac}" fi

and btw, if you’re wondering (like I did) and greping the entire codebase (like I did) to find out where $rootmnt is set, you won’t find it. It is set in initramfs’s init script:

So, whenever an initramfs with live-boot loads, it checks everything it possibly can, starting with removable media, to find a /live directory and mounts file the ending with squashfs (or similar) to /root.

grml explicitly set their live-media-path=/live/grml-small-amd64/ so it doesn’t look for live/ but looks for this path instead.

The picture is clear, I just need to:

1. Boot a linux kernel with boot=live live-media-path=/live/grml-small-amd64/ parameters
2. Have an initramfs with live-boot package
3. Copy /live/grml-small-amd64/ from grml’s iso image to any of my partitions.

Encryption ?⌗

Warning: dead-end

live-boot seems to have some luks code as well, would it be able to detect a /live dir in my luks -> lvm -> xfs setup ?

At the top of live-boot script, it tries to decrypt partitions.

Let’s ensure if luks tools are present in grml’s initramfs. First extract it and then grep for luks

; cpio -idmv < boot/grmlsmallamd64/initrd.img ; grep -r 'luks'

Nothing. The directory structure also doesn’t make sense. Where’s the rest of the directories ?

; tree -L 6 . . └── usr └── lib └── modules └── 6.12.41+deb13-amd64 └── kernel ├── arch ├── crypto ├── drivers ├── fs ├── lib ├── net └── sound 13 directories, 0 files

Only kernel modules are here. It first I thought, maybe the cpio was not extracted properly. After several tries I observed compressed cpio size > extracted !? It doesn’t make sense

; file boot/grmlsmallamd64/initrd.img boot/grmlsmallamd64/initrd.img: ASCII cpio archive (SVR4 with no CRC) ; cpio -idmv < boot/grmlsmallamd64/initrd.img ; du -sh . 32M . ; ls -lh boot/grmlsmallamd64/initrd.img -rw-r--r-- 1 neo staff 48M Aug 15 23:06 boot/grmlsmallamd64/initrd.img

Something must be wrong with the img file, look deeper:

; binwalk -e boot/grmlsmallamd64/initrd.img ------------------------------------------------------------------------------------------------------------------------- DECIMAL HEXADECIMAL DESCRIPTION ------------------------------------------------------------------------------------------------------------------------- 0 0x0 CPIO ASCII archive, file count: 1296 30874112 0x1D71A00 XZ compressed data, total size: 18760580 bytes ------------------------------------------------------------------------------------------------------------------------- [+] Extraction of cpio data at offset 0x0 completed successfully [+] Extraction of xz data at offset 0x1D71A00 completed successfully ------------------------------------------------------------------------------------------------------------------------- Analyzed 1 file for 85 file signatures (187 magic patterns) in 1.5 seconds

So, there’s 2 concatenated archives present inside the same file, and kernel even extracts it properly !

There’s cpio inside the xz too:

; cd extractions/initrd.img.extracted/ ; file 1D71A00/decompressed.bin 1D71A00/decompressed.bin: ASCII cpio archive (SVR4 with no CRC)

Anyway, I found my missing files

; cpio -idmv < 1D71A00/decompressed.bin ; ls 0 1D71A00 bin conf cryptroot etc init lib lib64 run sbin scripts usr

Although, files in the initramfs I get dropped in after failed boot of grml (with only initramfs and no live-media-path) vs what I see here are different. Idk how.
In the failed boot terminal, there’s no /scripts/local-top/cryptroot but, in the extraction, there is. Maybe I’ll look at this in the future.

So, encryption -> no.

Let’s be a simple human being and just get a normal partition and have the /live/ directory there.

I’ll have to resize a vol, create a new partition and create a filesystem on it.
I’ve created one /dev/sdb4 and formatted it with ext4.
Then,

1. Copy the /live directory from the iso to the new partition

   [root@Journey hritik]# mount /dev/sdb4 /mnt [root@Journey hritik]# cp -r live /mnt/. [root@Journey hritik]# ls /mnt/ live lost+found [root@Journey hritik]# ls /mnt/live/ grml-small-amd64

2. Copy the kernel and initramfs to the esp (/boot)

   [root@Journey hritik]# cp boot/grmlsmallamd64/vmlinuz /boot/vmlinuz-rescue [root@Journey hritik]# cp boot/grmlsmallamd64/initrd.img /boot/initrd-rescue.img

3. Finally, create a bootloader entry for grml kernel and initramfs. I use systemd-boot, so, here’s how it looks:

   title Rescue (grml) linux /vmlinuz-rescue initrd /initrd-rescue.img options apm=power-off boot=live live-media-path=/live/grml-small-amd64/ nomce keyboard=dvorak grml2ram ssh=rescue

I’ve used a few extra parameters documented in the grml cheatcodes.

We’ve sucessfully installed grml on a hard disk, although this is non persistent (can be made persistent). So, exactly like a live boot but from a hard disk partition.

Notes⌗

Boot from grml in bootloader entry but with a pen drive plugged in⌗

This shows a warning that bootid.txt is present but not specified in the kernel parameters – indicating that it picked up the /live directory from the pendrive (as it’s a removeable media and was scanned first).

Although, after a successfull boot into grml, lsblk shows the internal disk partition (sdb4) is mounted for live media. I’m not sure when this pivot happened.

Does a lookup for /live across all devices make it possible to hijack live-boot ?⌗

Certainly.