2024-09-22

This week some interesting developments occurred.

This tool is a re-implementation of the ltools/fixupp.exe that was included in the OpenDOS 7.01 Machine Readable Sources package, as that was provided without sources.

It operated on RASM-86 output object files to make them compatible with the object files assembled by JWasm. The exact purpose perpetually escapes me.

However, I already had extended it a while back (2024 June) to emit lines listing the relocations in the object file, for use with our companion applications tractest/convlist.pl, TracList, and ident86. The RASM-86 assembler (also from the OpenDOS MRS package and without sources too) and the then current version of JWasm didn't mark relocations sufficiently in their listing files.

The last week's work on fixupp was mainly about changing PUBDEF records so they refer to a group if their segment belongs to a group. Records that lack the group reference are created by older Microsoft MASM versions (such as v1.10 and v5.10) which first bit me during porting MSDebug to NASM. At the time I reported the resulting incompatibility to the NASM bug tracker, as it leads to contradicting a statement in the NASM manual.

At the time I worked around this by first porting the files that declare variables to NASM. This may be not as readily possible as it was then for working with the MS-DOS v4 sources however. That's why I adapted the fixupp utility to add in the missing group reference. It was a fairly easy adjustment.

The most difficult part of setting up the fixupp repo was to extract the source text file and its history from the edrdos hg repo. I initially just pushed the entire edrdos repo and stripped the unrelated files. However, I soon replaced this by a brand new repo with only the fixupp.c file history. Aside from storage considerations, this also avoids pulling in the OpenDOS MSR and Enhanced DR-DOS source files, with their arguably worse licensing status.

The first change is to add a simple type heuristic to the application. This can correctly detect some different protocols such as FreeDOS loader with FreeDOS load segment, FreeDOS loader with Enhanced DR-DOS load segment, or lDOS loader. It re-uses the search code with possible wildcards from the LBA detection detection.

The subsequent changes concern detection and matching of the FSIBOOT protocol in FAT32 boot sector loaders.

The FSIBOOT revision needed by a first stage is detected by searching for a snippet of code. This is part of the check for the FSIBOOT signature, and matches FSIBOOT revisions all the way back to 2017-09-13, the initial revision. It is used to find the address of the FSIBOOT signature, which must be composed of 8 printable ASCII bytes or 7 of the same plus one NUL byte. (This latter case is for the initial revision again.)

The next step was to add a list of known revisions and display the date they were introduced. The comments in the macro file also refer to the lDOS boot repo changesets of these revisions.

After that the common patterns (FSIBOOTx or FSIBEXxx, where x is alphanumeric) were added to the detection, to indicate future revisions of FSIBOOT or experimental FSIBOOT.

Next, the FSINFO sector is read even if we do not have an FSIBOOT stage to install. This allows detecting the FSIBOOT revision that is provided, in case we install a loader that needs FSIBOOT from a single-sector file.

Afterwards the /SI and /SJ switches were added. /SI (the default) means to validate that the needed FSIBOOT matches that provided in the FSINFO sector. /SJ disables this new check.

Then the /SG= option was added. This allows to check for an explicitly specified FSIBOOT revision.

That's all the major steps made with instsect this week.

Add the /R switch to reserve memory so the debuggee will always have to load at a higher address. Notably, this is all done in the debugger application init so no resident code space is needed.

Store a chain of the reserved memory in run time variables of the debugger. This allows to detect what reserved memory is allocated.

Add a reserve Extension for lDebug. Its DISPLAY command will show the currently allocated reserved memory blocks.

The FREE command to reserve.eld will free the reserved memory. ALLOC will make sure to allocate all remaining gaps of free memory below the limit.

The GET command displays the current reserved memory limit, while SET allows to change it.

A problem is that when loading a process, DOS will allocate a certain amount of memory to the process's environment block first. So if you reserve /R=FFF then the environment block would end up at segment 1000h, not the process block.

The first step to fixing this was to add a Debugger Common Options 7 flag to allow the debugger to operate without allocating a new attached process after the prior one terminated.

This allowed to add the new reserve.eld actions QA and L. QA will inject a QA command to the debugger (terminate attached process), then return into the ELD for further processing. L will likewise inject an L command (load attached process). The QA action will also temporarily set the new DCO7 option so the ELD can operate on the freed memory in peace without the debugger immediately re-creating an empty process.

Rounding out the Extension for lDebug is the GETENV and SETENV actions, which allow to display and temporarily store an environment block size. (It is only stored until the ELD unloads from memory.) Along with the ALLOCENV and FREEENV actions, this allows to create a chain of commands that:

1. Loads a process, environment block and all
2. Stashes the size of the resulting environment block
3. Terminates the attached process
4. Frees the reserved memory
5. Allocates a block matching the environment size
6. Reserves the remaining free memory below the limit
7. Frees the environment block
8. Reloads the process

If there is enough space free below the reserved memory limit, this dance will ensure that the process (not the environment) is loaded exactly after the reserved memory limit. To simplify use of the ELD, there is an action called RELOAD that will in turn run each of the actions needed to do this.

The final three changesets concern:

1. A bugfix that the ELD wouldn't shrink the correct block if the debugger process block was left larger than needed to reserve memory.
2. The function doscall_dx_for_es will return the es value in dx regardless which mode it is called in.
3. reserve.eld is added to the ELD mak script and to the extlib and extpak ELDs.