2022-11-27

In the past days, lDebugX received some attention which was initiated by a bugfix in FreeDOS Debug/X. That bug was the fact that parsecm (the function to parse C and M commands) used getaddr (in lDebugX terms) twice. (The same function was briefly called getaddrW in Debug/X, though has been replaced by getaddr with explicit calls to IsWriteableBX now. In lDebugX the latter function is called verifysegm and it is still called by getaddr, whereas getaddrX does not use it.)

The problem is that IsWriteableBX or verifysegm may replace the selector passed into it in BX by the debugger's scratchsel, set up to mirror the input selector. This happens if the input selector is a code selector.

If two different addresses have different selectors, and both are replaced by the scratch selector, then the second selector wins out and accesses meant to go through the first selector will actually use the scratch selector set up to reproduce the second selector. So that's bad. So in parsecm only one address should use verifysegm.

However, there are more bugs to do with the scratch selector. The A, C, and E commands would set up the scratch selector for code selectors, and would then display the scratch selector value instead of their original selector. For C this was simply a matter of amending parsecm so it won't call down to verifysegm for the C command, as it doesn't need to write to the "destination" at all.

For A and E, the solution is to use getaddrX at first but then use verifysegm explicitly later to actually write to memory. This makes the display of both commands, as well as the contents of the AAS variable, depict the source selector in any case.

verifysegm had another problem: If it replaced the input selector by scratchsel, it would copy and modify the descriptor of the original selector. That means that the limit of scatchsel is modified to match the input selector's as well.

Other code however, such as the DM command, used scratchsel to access R86M segments. But they didn't reset the limit. So if verifysegm set up a very low limit (like 0 or 1 in my test cases) then a subsequent DM command will fault.

This bug exists in the current Debug/X revision as well. The fix is simple: All code that initialises and uses scratchsel must also set up an appropriate limit.

Open question: If verifysegm receives a read-only data selector, will it set up scratchsel to be a read-only data selector as well? This would make the update IISP header function in ints.asm fail if called in PM.

Edited to add: In https://hg.pushbx.org/ecm/ldebug/file/59230c07f5ce/source/debug.asm#l3716 it is clearly visible that verifysegm does *not* use verw but rather checks specifically for a code descriptor. Also, it always sets up the Expand Down bit as zero and the Write Enable bit as one. So no question there; it always sets the scratchsel descriptor to a writeable data descriptor.

Today during tests for the next section I found that recent dosemu2 revisions would not allow me to enter PM in native DPMI mode any longer. The dosemu2 maintainer pointed out that I should build dosemu2 either without the -d switch to default-configure or with another arcane switch to re-enable a certain signal handling workaround which our server apparently needs. If neither condition is met, any attempt to initialise PM operation (such as running the example program dpmimini) would terminate dosemu instead.

I initially believed this to be another dosemu2 problem, but it isn't actually. During testing the scratchsel limit problem, I happened to re-use a scriptlet running lDebugX running lCDebugX running one of the dpmitest programs. When gaining control of the lCDebugX command line, the dpmitest program would have already entered Protected Mode. So I entered a G command to the debugger to terminate the example, then n dpmimini.com followed by l and g and g 15D.

When causing the fault in lCDebugX, dosemu would then report an unhandled exception! What happened to the outer debugger?

The answer is simple: In my use case both debuggers uninstalled their DPMI entrypoint hooks upon the first example program entering Protected Mode. Then lCDebugX would later gain control in V86M again… and would promptly re-install its DPMI entrypoint hook. The outer debugger, however, never had an opportunity to install its hook again.

The simple workaround is to avoid this situation: Once the first example terminated and lCDebugX regains control in V86M, run the bu command to return into lDebugX. Then when the outer debugger gets another run command, it will install its hook.

Of course I wasn't satisfied with this workaround, albeit it is perfectly reasonable and could just become part of the knowledge required for proper operation of the debuggable lDebugX builds.

So I added a new function to lDebugX's AMIS interface: Function 31h will give the (outer) debugger the opportunity to install its hook. To do that, it will be called by debuggable builds of lDebugX whenever this (inner) debugger is about to install its own hook. So instead of needing to break upwards at the right time, it is now sufficient to install the outer debugger's AMIS interface beforehand (using r dco4 or= 8).

The most difficult part was to adapt the hook2F function so it can run, albeit restricted, without the SS register pointing to the debugger's data segment. This required a fair amount of spaghettification. Another part was to avoid recursion, and also due to the SS problem the outer debugger (even if it is lCDebugX) will not attempt to call yet another debugger's function 31h. That means this solution is limited to a two-debuggers setup, but hopefully no one will ever need more than two debuggers.

The RVM command was updated to also display the segment (and, in Protected Mode, the selector) of the second code segment if it is in use.

Several variables were added to the debugger. With the new isvariable data structures in place, it is very cheap to add a dozen variables or so. If no special handling is needed, then the variables only require space in the data tables of the debugger, in its process/data/entry/stack segment. The space pressure in that segment is much lower than in the first code segment.

Another bug that was fixed was specific to symbolic lDebugX with dual code segments: In the PM initialisation, the setup of the code2 selector would corrupt the limit value (0000_FFFFh) in the cx:dx register pair, overwriting it with the base address of the second code segment. After that it was used to set the client code segment limit. So that limit was wrong. The insidious part was that this wrong limit would generally be higher than FFFFh, so it wasn't obvious at any point that this happened. I noticed during testing of the scratchsel related bugs that the CS limit had a wrong value. And even so, it was random chance that I happened to be running a symbolic, dual-code build of lCDebugX.

Another small change is the default-off support for the "double dollar" segment address. If enabled, a single-dollar address in Protected Mode will explicitly instruct the DPMI host to set a limit for accessing 64 KiB. A double-dollar address will instead instruct the host to set a limit for accessing 4 GiB. This is somewhat of an abuse of the DPMI "segment to selector" service, as it is documented that we shouldn't modify the selector that it returns.

The BOOT command to load a kernel could fail when an ENTRY or BPB parameter was specified, in a non-PM build running on a 386. This was due to checking for a nonzero edxh on 386+ machines, even when not building an lDebugX (PM) build. As the non-PM getaddr never modifies or initialises edxh, it would contain an arbitrary value that could very well be nonzero. In that case, specifying such a parameter would fail with an error. I noticed this while testing and examining the debugger for the new BOOT command documentation in the manual.

Using a BOOT READ command with the HIDDEN= keyword would break down while trying to read the partition tables, if a partition number was specified. This was because the handling of the HIDDEN= keyword would overwrite the hidden sectors before the partition walking was called, which depends on the hidden sectors still being zero.

The fix was to defer the hidden sectors override to until after the partition has been found.

This presented a new choice: What to do with a HIDDEN= keyword if there is a partition specification, that is there already is a nonzero hidden sectors value? It was decided to replace the hidden sectors with the explicit value, making BOOT READ hda1 200 HIDDEN=0 0 and BOOT READ hda 200 0 effectively access the same sector.

However, to be versatile, the HIDDENADD= keyword was added. If this is used instead of HIDDEN=, then the number is added to the hidden sectors detected for the partition, rather than replacing them.

The BOOT commands gained a section in the command reference chapter of the manual. This contains more verbose descriptions of all the commands and most keywords, as opposed to the ?BOOT online help page. (However, it does not contain a list of the pre-defined load protocols yet.)

This was used to test the scratchsel limit bug of lDebugX and FreeDOS Debug/X. It will allocate a selector, then set up its limit as 1 (allowing to access the first two bytes) and the type as a code segment. Unlike the other DPMI tests, this is as yet supplied only as a Script for lDebug (.sld) file which assumes the client is already in Protected Mode, and it lacks error checks.

It may be used as the base for a dedicated test later. The open question about scratchsel being initialised to a read-only data selector may also be tested wih the assistance of a new test program.

Recently I tested transferring several files from the HP 95LX using PD ZModem, at 57600 baud. Starting without the power adapter connected, I had a much lower data rate than usual. Immediately upon connecting the power adapter, the data rate as much as tripled. So it is certain that running off batteries makes for a worse data rate all else being equal.