2022-11-06

This week I went and reviewed the ldosboot boot sector loaders for a while, particularly the interaction between the FAT32 first stage and its associated second stage. This second stage is called FSIBOOT.

I found that the FSIBOOT interface, the current revision of which is known as FSIBOOT4, is very intimately interdependent. Because the IBMDOS load protocol (only used by PC-DOS 7.10 on FAT32) needs to look for a second file (IBMDOS.COM), this latest revision of FSIBOOT has a secondary entrypoint pointed to by the last word of the FSIBOOT payload. It expects the variables on the stack to be initialised like after the primary entrypoint has returned to the first stage loader. It is a function called dirsearch that searches the root directory for a file, the name of which is pointed to by an entry in the fsiboot_table (the dispatch/parameter table provided by the first stage). The dirsearch function is internally used by FSIBOOT to find the first kernel file. To note is that it will jump to the error handler in case of a missing file.

The old public domain sources of the ranish partition manager contain a FAT12/FAT16 boot sector loader, which I called ranboot. It has two special features:

First, it allows to search for multiple files, loading the first one found. As yet this is only used for searching WINBOOT.SYS followed by IO.SYS.

Second, it checks the MZ signature in the file of which it loaded the first four sectors: If present, it does an MS-DOS v7 style handoff, else an MS-DOS v6 style one.

Unfortunately the ranboot loader shares some deficits with Microsoft's loaders. For one, it expects that the sector size is at least 512 Bytes, and that the first 2 KiB of the file's data are consecutive. (Much like with the Microsoft loaders this avoids having to read the FAT.) For another, for the v6 protocol it will write the first two root directory entries to linear 00500h. Like the Microsoft loaders, it expects the first entry to be IO.SYS and the second entry to be MSDOS.SYS. But I believe the Microsoft loaders do verify this, whereas ranboot does not.

In any case, the ranboot example made me consider that the next FSIBOOT's secondary entrypoint to the dirsearch function could return a status as file not found instead of always jumping to the error handler. However, the only user of the secondary entrypoint is currently the IBMDOS load protocol, for which the second file is always required. Even if we implemented a case like ranboot's, we could call dirsearch after having detected that the second file is needed, leading once again to a case where the second file not being found is an error.

Searching for multiple names of the first file would require dirsearch either to return a file not found condition or for dirsearch to accept multiple filenames for the same search. In any case the FSIBOOT interface would need additional changes.

Another tidbit that I noticed is that after the fsiboot_table, which is word-aligned, we store the first filename in 8.3 format (thus needing 11 bytes). And then follows the FSIBOOT interface name, for 8 bytes. While we do use repe cmpsw to compare this, the source of the comparison is always misaligned. I believe this choice was made so as to put the filename first, so that instsect will properly detect the filename. If the signature was first then it could be misdetected as a filename. Perhaps we could put the signature *before* fsiboot_table in a subsequent interface revision? We would have to check the first bytes of the table to insure that the signature wouldn't be misdetected in that case either. (The last fsiboot_table entry is the load segment as a word, which we assume to be ⇐ 2000h. Therefore the last table byte is ⇐ 20h, which is not valid as the start of a filename. This is intentional so that the filename is not misdetected as starting earlier than it actually does.)

However, reserving 8 bytes in front of fsiboot_table would effectively enshrine the interface name as a part of the FSIBOOT interface. If one likes to live dangerously, one could want to omit the interface name from the first stage entirely. Either trusting that the FSIBOOT matches the expected interface, or perhaps computing a checksum of the interface name found in the FSIBOOT stage? In any case, there are possible reasons against putting the name first. Yet another possibility would be to put the interface name at the very end of the first stage. This would also allow it to be word-aligned (or, dword-aligned in fact).

In any case, today I noticed that the ever-crucial read_sector function of ldosboot and also lDebug could be optimised a bit. The savings were 3 bytes in the FAT12 and FAT16 and FAT32 loaders and 5 bytes in the initial loader. What actually made me stumble on this was a vaguely recalled comment that there are ROM-BIOS implementations of the LBA extensions that require es to be set to the buffer segment, even though for the extensions the buffer address is supposed to be passed in the disk access packet.

Cue my surprise at finding that setting es unconditionally, before the LBA or CHS dispatching, actually saved a few precious bytes! This is because we previously pushed the input bx (data target segment) onto the stack and later loaded that into es, whereas es was not used other than to hold that exact value. So early on in read_sector we now do mov es, bx and leave it at that.

During adaptation of this patch to the initial loader and the test status writer kernel we had to carefully consider the different paths. In addition to the plain LBA read and CHS read, both of them also have a "sector segment" assisted read (again LBA and CHS). That means when a data boundary error (code 09h) is returned constantly, the read is done into the aligned "sector segment" and then copied to the target. In addition to that, the test writer also has write functions for all four cases: LBA, LBA sectorseg, CHS, and CHS sectorseg.

Finally we also adapted the patch to lDebug, which has its own branch of the entire sector read/write handling. While based on the same original read_sector it had some different choices later. Like the test writer this comes in read and write flavour, LBA and CHS, and also normal and sectorseg.

During testing lDDebug within qemu, tracing the sectorseg read/write LBA/CHS cases, it happened to be the case that sectorseg reading loaded a random value into cx to copy the data from the sector segment to the target. The desired value would have been 200h, or 512. This happened because lDebug's branch of the sector access handling uses ds to address its data segment with the BPB in use, including the word holding the sector size in bytes. During coding of its sectorseg read handling it wasn't noticed that ds was set to address the source for rep movsb (ie, the sector segment) before loading cx from the sector size variable using the ds. So it ended up accessing a random word somewhere in memory.

ranboot repo: https://hg.pushbx.org/ecm/ranboot

ldosboot read_sector optimisation/hardening patch: https://hg.pushbx.org/ecm/ldosboot/rev/7077e4a45bc8

Same patch for lDebug: https://hg.pushbx.org/ecm/ldebug/rev/4b92bc476a38

Bugfix patch for lDebug: https://hg.pushbx.org/ecm/ldebug/rev/71d71f45d21f

fsiboot_table and the FSIBOOT4 entrypoint as of today.

ldosboot documentation on the filename detection intended for the lDOS load protocol: https://pushbx.org/ecm/doc/ldosboot.htm#protocol-sector-iniload-loadname