2023-09-24

Last week on Sunday I was occupied with something other than preparing a blog post. Therefore, today I will cover the changes since two weeks ago.

The toclip application got its final patch, for now, to append a NUL byte to the text it writes to the clipboard. It was indicated by dosemu2 developers that it should do this.

The application was also added to the current release builds built by the wwwecm scripts.

During an audit of the ported WarpLink sources, I noticed that two spots used two-byte text literals as instruction operands. These are interpreted differently by NASM and TASM. While adding the conversion to the fixmem.pl script would be ideal, for now I simply changed the two occurrences manually.

Another change concerns the even mmacro in progbits sections. It will now align with NUL bytes rather than NOPs. This helps identicalise more to the TASM output.

• Fix minor bug (no ill effects), initialise all of enlarged auxiliary buffer in init. (The conversion calculated words from bytes where the source actually specified paragraphs, not bytes.)
• Add /TX, /TY, and /TZ switches to retry relocation allocation in init up to three times. Each attempt can use a different allocation strategy and UMB link state.
• Add missing dualfunction mmacro to the FloatToStr function
• Fix InDOS check sense in cmd3_close_handle (only used if _TEST_HELP_FILE is enabled)
• In _int21 function fix bootloaded check to use an ss: segment override (only applicable to lDebugX?)
• Always set DIF6 flag for debuggable mode for the unconditionally debuggable build
• Add /A=DEFAULT switch, currently synonym for /A=MIN
• Help: Do not store double NUL byte terminator in discarded boot help message.
• Fix boot file read (Y or EXT command) to work when the read ends on both the EOF and a cluster boundary. The check to allow this had mixed up the low and high word of si:di. (Noticed because ldmem.eld happened to be exactly 2 KiB sized.)

The big change this week is basic support for loading plugin executables stored in Extension for lDebug (.ELD) files.

The ELD architecture supplies an ELD with three structures:

• The ELD instance part. This is typically initialised to the ELD code section. Currently stored in a separate segment by the debugger, Protected Mode operation provides both a code selector and a data selector for this segment.
• The ELD data block. This can be initialised to the ELD data section. It is allocated within the debugger data/entry section.
• The link info. This provides tables that associate label names of data items, entry code, code1, or code2 functions with the corresponding addresses.

The idea is that the ELD carries a linker, which can be discarded once it has successfully run. The linker uses tables in the ELD as well as the debugger's link info tables to fix up several different kinds of relocations:

• Internal code to code relocations (fix offsets in ELD code instance to refer to the code instance where it is actually loaded)
• Internal code to data relocations (fix offsets in ELD code instance to refer to the ELD data block)
• Internal data to code relocations (fix offsets in ELD data block to refer to the ELD code instance)
• Internal data to data relocations (fix offsets in ELD data block to refer to the data block)
• Data link relocations in ELD code instance (refer to debugger data label in ELD code)
• Data link relocations in ELD data block
• Code link relocations from ELD code instance to one of three debugger code sections

Data link relocations can be optional: If an optional data link relocation is not found, the linker fills it with a zero word instead.

Code link relocations can refer to the code1 section, the code2 section, or the entry section of the debugger. The linker expands a code link relocation into a near call into the LINKCALL default ELD, to an entrypoint which will handle the setup of a far call into the corresponding section. The called function gets a near call return address which leads to an ext call return handler. This is simply a retf instruction for non-_PM builds. Otherwise, it loads the ext segment's segment or CS selector value and then does a retf. (This is similar to the dual code segment nearcall mechanism, and just like it the called functions may not use stack parameters. Unlike nearcall the _PM handler for ext return only gets a near return address offset, as well as a placeholder space for the segment or selector. This placeholder is always set to address the ext segment.)

The ext (code) segment can be any size from 0 Bytes to 65_520 Bytes. The default is 8 KiB for application or device mode, and 16 KiB for bootloaded mode. Non-bootloaded modes allow specifying an /X= switch to change the size at init time. The initial memory allocation for application and device mode assumes the maximum ext segment size.

The ELD data blocks space currently defaults to 16 KiB. As it resides within the data entry section, it cannot be resized. (I am considering to move it back past the stack, so it would occupy the tail end of the data entry section. This would allow to resize it in the init.)

The ELD link info currently takes less than 2 KiB. It is stored in the debugger's message segment, avoiding to use up space in the data or code sections.

The ELD linker and associated data tables in the ldmem.eld example takes up about 1.1 KiB. It is important to recall that the linker must be included in any (nontrivial) ELD, so its size must be added to every ELD's own size. This approach was chosen to avoid a resident linker in the debugger code section, presumably saving some 500 to 1000 Bytes.

When an ELD unloads itself, it is responsible for decrementing the ELD code segment and ELD data block space "used" counters. If it cannot do so upon a linker failure or because the ELD's code or data are not at the very end of their spaces, then it must leave resident a skeleton code instance which refers to the used memory. The first byte of the identifier field of this instance must be set to a NUL byte to indicate it is available for reclamation. The reclaim.eld ELD has not yet been written.

Basic linking and use of debugger functions and data structures is implemented. This is enough to implement a number of "one shot" ELDs, such as the ldmem.eld which displays information on the debugger's memory use. More complex ELDs which stay resident and implement their own command extensions are not yet possible, as the ELD mechanisms to hook existing interfaces still need to be added. The goal is to allow implementing commands like DIL and DM completely in ELDs which can be installed residently for InDOS operation. (Application and device mode ELD loading require DOS, conversely bootloaded mode ELD loading requires the auxiliary buffer and needs to access the disk using int 13h.)

The first really useful ELD is ldmem.eld, the lDebug memory use display ELD. For now, it detects all sections used by the main debugger (not including symbol tables) as well as gaps between them in the debugger allocation. It also sorts all the sections in order of increasing addresses, a feature I have long missed in the RVM command.

This ELD is far from complete however.

For now, the ELD interfaces and buffers are not included in default builds of the debugger. The _EXTENSIONS build option needs to be specified for including ELD support. Presumably once the ELD infrastructure is more complete, and before the next release (release 7), the build option will be enabled by default.