2026-03-08

This week I uploaded a new branch to the lDebug repo, updating the sources to build with a makefile that assembles multiple object files, then links them using the WarpLink linker.

• Increment ecm release to r19.
• Fix a wllist with 256 or more EXTDEF strings. Details in the changeset message.
• Catch an EXTDEF index value equal to zero as an out of bounds value. This would have caught the bug fixed by the prior changeset.
• Merge the two prior changes.
• Increment ecm release to r20.

The bug caused random corruption in the lDebug 386-related patch list. It didn't appear in prior testing because of the minimum amount of EXTDEF entries needed to activate it.

• Allow to disable the usesection detection for lDebug. This requires a NASM revision with the %note directive. If the disable note is found, it is assumed all usesection directives will emit a corresponding note. This is useful because a usesection directive in a disabled conditional assembly branch won't be misdetected as changing the section.

The older NASM revision that we use is 03490692b0082fe16a1936a5774f4326a55075e6, "2.16.02rc2", of 2023-10-11. It doesn't support the %note directive. In addition, it mistakenly silently ignored a doubled extern label reference (ie, mov dx, 2*N_LTABO-2), assembling it to the scalar value (-2) only without generating any fixups.

On 2026-03-06 I tried out the then most recent git revision, b1909cf39b8028b72cf527ba617a940e329ea05c, "3.02rc2", of 2026-03-04. However, I found that it misassembles instructions like cmp al, 4 to the three-byte machine code 80 F8 04 rather than the two-byte 3C 04 that I expected.

Next I tried an older revision, 9b18e9a9134fc5bb3ed0df631c1012c647d379e2, "3.00rc9", of 2025-09-19. However, it doesn't work to declare a label extern with this revision.

I finally settled on 755593b12844a3189dcab7faf3e9158da0877021, "3.01rc3", of 2025-10-06. This supports the %note directive and everything else needed for building lDebug. It does swap around some instances of multiple prefixes, eg a rep prefix + a32 or o32 + a32. For the doubled extern label reference, it correctly emits the error: invalid operand type.

• Add the markwlc.pl script. It accepts a WarpLink extended .map file, a binary file, and a prefix switch. The purpose is to change all bytes affected by wlcalc requests. The requests are detected by scanning the .map file for symbols that indicate a use of wlcalc.
• In markwlc.pl open the .map file explicitly.
• Parse the binary name.
• Complete the script.

The following changes are to the main ident86 tool:

• Accept an opcode dump from a multi-line disassembly by the debugger. This occurred in the Protected Mode code of the debugger, with multiple prefixes and a 32-bit immediate operand. Example of this is listed in a subsequent section of this post.
• Detect NASM structures to skip them during processing of .tls files.
• Support repeatdw mmacro as indicating dw (word size data).
• Support ascii, asciz, and counted mmacros to detect db (byte size data).
• In matchsize skip over a fill or fill_at_least mmacro use.
• In the -j (show diff) dump, write up to several prior lines if the exact .tls line contains only a hexdump (no source line that was assembled).
• Add the -n switch (indicate NASM source) and support backslash continuation in NASM source. This is emitted as one long line in the .lst file (and consequently in the .tls) but appears as multiple lines in the source .asm or .nas file. Figuring out the logic of this was hard, taking possibly as long as 4 hours to solve. The magic solution is to add 1 to listskipped for every .asm line, then add this variable to linenumbersource after processing the complete continued line. listskipped is reset to zero initially and after it is read.
• Add some debug flags to a few debugging prints.
• Add the -w switch (wlcalc replace) to edit wlcalc spots that are marked as unrelocated words or dwords in the .tls file. This expects a binary file that's marked by markwlc.pl to help it find the wlcalc spots.
• Add -W switch to speed up editing. After a successful edit this forces the maximum offset to "0" to immediately end the iteration. (Alternatives mentioned in the changeset message.)
• Add the immasm_special_dump and isvariablestruc mmacros to matchsize to detect as dw (word data).
• Add areadefine to matchsize to detect as dw.
• Do not match the "note:" part of a %note directive as a label in the .tls file. This part already specifically excluded lines with a string of asterisks in the .tls hexdump prefix, to not match %warning directive output. The %note directive emits a string of dashes instead.

Note that the -w switch implies that the file to "edit" should be the original binary that is recreated by the build scriptlet, and the "original" file should be the output of the markwlc.pl script. ident86 will check that the "edit" binary doesn't change from the build. The .tls file needs to be updated to allow the -w run to proceed. (I did this wrongly when actually using this feature, but it is a forgiving error.)

The purpose of the -w switch is to edit the sources of the program (lDebug in this case). It inserts a "relocatedzero +" term into the instruction that generated a wlcalc-affected word or dword field that isn't marked as relocated in the .tls file. We want these fields to be marked as relocated in order for TracList to match these instructions, despite the wlcalc request having changed the literal bytes of the instruction.

The actual use of the ident86 -w switch is documented in the lDebug changeset generated by it:

cp -a ../tmp/ddebugx.big edit.big &&
  ~/proj/ident86/markwlc.pl ../lst/ddebugx.map edit.big --prefix=256
~/proj/ident86/ident86.py ../tmp/ddebugx.big edit.big \
  ../lst/ddebugx.tls ../lst/ddebugx.map \
  -s -o -256 -S -e 2 -x -w -p '^.*\/::' -p '\.lst$::.asm' -n -E \
  -r -b 'make NASM=~/proj/nasm-755593b12844a3189dcab7faf3e9158da0877021/nasm' \
  -c cookie -W; beep

As noted in the changeset message as well, this ran for longer than 60 minutes. (Of course, that time would have been shorter if I had already used the quicker convert listing script.)

Assembling the ddebugx.big image _LINK build of the most recent lDebug revision, f39e3d0d4990 (of 2026-03-08), using first the more recent working NASM revision as in make CONVLIST=~/proj/tractest/clquick.pl NASM=~/proj/nasm-755593b12844a3189dcab7faf3e9158da0877021/nasm and copying the .big, .tls, and .map files to the t subdirectory. Then run touch *.asm and rerun make with the 03490692b0082fe16a1936a5774f4326a55075e6 NASM revision.

Next set up the older ident86:

hg clone ~/webrepos/ident86/
hg -R ident86/ up 8f7e
ln -s ~/proj/ident86/ldebug.com ident86/
ln -s ~/proj/ident86/ldebug.img ident86/
ln -s ~/proj/ident86/extlib.eld ident86/

Then run it: ./ident86/ident86.py -s t/ddebugx.big ../tmp/ddebugx.big t/ddebugx.tls t/ddebugx.map -o -256

(We're using the newer NASM's .tls file because that fixes the usesection directives in not taken conditional branches.)

After a moment, the run comes across the multi-line disassembly:

...
005DDC  first:67 != second:66
005DDD  first:66 != second:67
005DEB  first:67 != second:66
005DEC  first:66 != second:67
005DDC up to below 005DFC, first=005DDC last=005DEC
Repeating due to malformed line 1 "4B17:0100 6766C74530AC9C00"
Repeating due to malformed line 1 "4B17:0100 6766C74530AC9C00"

Finally, after repeating this error several times, the program throws an exception "LogicError: Repetition limit reached". Following the chain of the instruction: 005DDCh plus the .big file origin of 100h is 005EDCh. Running the following:

~/proj/tractest/sortmap.pl t/ddebugx.map –skip-empty

We find the relevant segpart:

...
LDEBUG_DATA_ENTRY DATAGROUP      s=05D70h l=0088h a=16  ddebugx.obj/pmdata
LDEBUG_DATA_ENTRY DATAGROUP      s=05DF8h l=01F0h a=4   ddebugx.obj/pmentry
LDEBUG_DATA_ENTRY DATAGROUP      s=05FE8h l=0181h a=2   ddebugx.obj/pminit
...

So we open t/ddebugx.tls (the correct, more recent NASM's .tls file) and skip to where it says "pmentry.lst", then skip to "5EDC ". This points to the following instruction: mov dword [ ebp + fr32_eip ], debuggerexception

Opening the .big image in the debugger allows us to look at the disassembly:

$ dos "ldebug /f /e t\\ddebugx.big"
About to Execute : ldebug /f /e t\ddebugx.big
lDebug: Warning, unknown filename extension specified!
lDebug (2026-02-01)
-u 5EDC
2B5F:5EDC 6766C74530AC9C00
          00                mov     dword [30+ebp], 00009CAC
2B5F:5EE5 660FB706600C      movzx   eax, word [0C60]
2B5F:5EEB 676689453C        mov     [3C+ebp], eax
2B5F:5EF0 67806539FE        and     byte [39+ebp], FE
2B5F:5EF5 EB42              jmp     5F39
2B5F:5EF7 C606600202        mov     byte [0260], 02
-
2B5F:5EFC 800ED90108        or      byte [01D9], 08
2B5F:5F01 6766C74530DF6500
          00                mov     dword [30+ebp], 000065DF
2B5F:5F0A 678C4D34          mov     [34+ebp], cs
2B5F:5F0E 67806539FC        and     byte [39+ebp], FC
2B5F:5F13 6766C7453CA80E00
          00                mov     dword [3C+ebp], 00000EA8
-

So the offending instruction is a32 o32 opcode modrm disp8 imm32, for a total of 9 bytes, just shy of fitting into the expected single-line disassembly. (As mentioned for the NASM revisions, the two prefixes are reversed between these two builds.)

• Initial commit of a NASM/8086/DOS port of patchqry. The intention is to provide a replacement for the patchqry (and eventually, patldos) tool when it is to be run on a DOS system. The assembly language port easily needs a fraction of the disk space of the gcc-ia16 built C language program. The C program will be retained as it can be used for other platforms as well.
• In patldos.c fix a comment missing a closing parenthesis.
• Fix to allow a 16-bit number in patldos.c (for SCANMODE). Bug caused by a prior changeset (on 2026-01-16) which extracted the function parsenumber. This also adds the bytes parameter to the function in both patchqry.c and patldos.c (always a constant 1 in patchqry).
• Add ?bytes parameter to the assembly language function parsenumber. Also use the lret mmacro instead of a retn instruction, this causes the function to discard its parameters from the stack upon returning.
• In patldos.c mark some labels for assembler conversion.
• In patldos.c do not re-open the file upon looping. (Incomplete: The call to patchqry code may re-open the file once again.)
• Initial commit of patldos.asm, the NASM 8086 DOS port of the patldos.c program. Doesn't integrate the patchqry.asm tool yet.

Apart from all the boring bits of the ports, I had to get the DOS application command line into a format that resembles an argv array expected by the C code, that is a list of ASCIZ strings. In both programs, an argv loop runs after the processing of the kernel pathname, and rewrites the command line to be a list of ASCIZ strings terminated by an empty string. In patldos.asm there is an additional function called cut, which creates a single ASCIZ string out of one parameter early on.

The link branch contains updates to allow building the .big image either using the original mak.sh script (as a single-file assembly language stream) or using a makefile that assembles multiple object files and links them to create the .big image.

Overview of some specific changes (not all):

• Extract the sections.mac file.
• Rename _LINK define to _ELDLINKINFO
• Move section_of list to sections.mac
• Add _LINK define based on the NASM output format.
• Add %include directives to include debug.mac from all files that are to be built as separate object files. Because this directive predates the setting of the _LINK define, it is conditional on the __OUTPUT_FORMAT__ define that's set by NASM to distinguish -f bin and -f obj format.
• Make many original %include directives conditional on !_LINK
• Rename the function called uppercase to capitalise. The "uppercase" name is used for an mmacro by the -f obj NASM output format. Also add an alias link to the ELD link info to allow ELDs to use either name for this function. (This changes the linkinfo mmacros to accept a quoted string for the name to enter into the tables.)
• Add wllist use for 386.mac
• Add extern directives to the extern.mac file, gradually building it up to include all needed labels. (Base scriptlet is included in the "generate extern.mac from expr.asm missing labels" changeset:)

nasm -f obj expr.asm -I ../../lmacros/ 2>&1 |
  perl -ne '
    if (/error: symbol .(\S+)'"'"' not defined/) {
      print "$1\n";
    }' | LC_ALL=C sort | LC_ALL=C uniq | perl -e '
      my $prefix = " extern ";
      my $counter = 1;
      my $length = 0;
      while (<<>>) {
        chomp;
        if (not ($counter % 6) or ($length + 2 + length($_) > 71)) {
          $length = 0;
          $counter = 1;
          $prefix = "\n extern ";
        };
        print "$prefix$_";
        $length += length("$prefix$_");
        $prefix = ", ";
        $counter += 1;
      };
      print "\n";'

• Move three sets of DATASTACK variables into debug.asm so that all nobits variables of this section are collected in a single object file. This is needed for the absolute $ trick to generate a nobits section (that overlaps a progbits section) despite the -f obj format.
• Do not generate length labels for bb id and bb when buffers, use the corresponding option defines.
• Move a number of equates into other files to share them between different modules.
• Instead of size labels composed of label delta arithmetic, use "behind" labels with appropriate wrt (with reference to) terms to calculate the sizes. Occasionally using NASM extern labels and wrt clauses, more often using wlcalc ext/minusext/extpara/minusextpara requests to compose the desired computations in the linker.
• Add a linker response template file and makefile.
• Add missing extern labels for DDebugX build to extern.mac (a new scriptlet listed in the changeset message:)

make 2>&1 | perl -ne '
    if (/error: symbol .(\S+)'"'"' not defined/) {
      print "$1\n";
    }' | LC_ALL=C sort | LC_ALL=C uniq | tee -a temp.txt | perl -e '
      my $prefix = "\n extern ";
      my $counter = 1;
      my $length = 0;
      while (<<>>) {
        chomp;
        if (not ($counter % 6) or ($length + 2 + length($_) > 71)) {
          $length = 0;
          $counter = 1;
          $prefix = "\n extern ";
        };
        print "$prefix$_";
        $length += length("$prefix$_");
        $prefix = ", ";
        $counter += 1;
      };
      print "\n";' >> temp.mac

• In wllist and wlcalc mmacros, include a filename in the distinguishing string to avoid cross-file collisions caused by using the same namespace with the NASM macro-local counter resetting for each file.
• Duplicate some equates (eg). NASM won't generate a definition in the object file when trying to define a global label in terms of an equate that references an extern label. (The fact that it silently fails is concerning.)
• Create a .tls file using convlist.pl
• Byte-align most sections by default (in sections.mac), relying on NASM sectalign to use larger alignment only as needed. debug.asm, as the first object module passed to the linker, paragraph-aligns all sections.
• Use the WarpLink r19 itoa wlcalc feature to expand the /Y= switch maximum value in the command help message.
• Fix boot message segment being placed too early so some of it was overwritten by datastack section.
• In uu.asm replace some doubling extern label references by wlcalc. A newer NASM revision than our default detected these as an error. The older revision silently ignored the references, generating wrong code.
• Add relocatedzero to many wlcalc operations otherwise not marked as relocated in the .tls file. This used the new -w switch to ident86 so as to do the required edits (minus the global/extern directives for relocatedzero).
• Make DDebug build.
• Make DDebug build with _DUALCODE enabled, this adds a new wllist patch list for the immediate far calls from one code section to the other. _DUALCODENEARDUAL thunks are duplicated into every object file needing them (about 50 errors of duplicate thunks if they're created with global labels). wllist mmacro accepts a filename parameter to avoid uses of the mmacro generating "debug.mac" as the filename.
• Emit thunk extern labels. This is purely to help enumerating the thunks created. To avoid dd.asm included files generating multiple thunks to the same function, the NASM command passes a SOURCEFILENAME define on the command line, which is used to generate the thunks. (This may be re-used later for wlcalc and wllist so as to avoid FILE that may contain path separators.)

The .big image generated by the makefile can be used by the original mak scripts by passing the variable use_build_compress_only=1 to the script.

The object files assemble very quickly. The linker needs about 6 seconds to run. Running both the linker and convlist.pl to generate the .tls file needs nearly 30 seconds, not much faster than the single-file build. I am considering optimisations to convlist.pl to speed it up. (ETA: Now landed in clquick.pl, I'll write about it in the next post.)