pushbx wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Early August work on LBAcache & bugs ======

**2026-08-09**

Most of the work this week was on converting Eric Auer's LBAcache to provide and use an Alternate Multiplex Interrupt Specification (AMIS) multiplexer. (The use is in uncache, the transient part that works with previously installed instances rather than installing a new one.) The advantages are as follows:

  * Cooperates with other TSRs by providing an IBM Interrupt Sharing Protocol (IISP) header, and an interrupt list for the AMIS interface.
  * Decouples the hardcoded variable and entry layout required for prior versions.
  * Allows detecting resident instances without the error-prone memory scan.
  * Using the advanced deinstallation method the uninstaller can take advantage of other IISP headers and AMIS interrupt lists.
  * Multiple resident instances can now uninstall cleanly using a single STOP command.

The disadvantages are:

  * Need to hook interrupt 2Dh.
  * Several paragraphs of memory spent for the multiplexer-specific data structures and code handlers. (Due to the optimisations of the resident part, the updated LBAcache including the multiplexer still uses less memory (table @ 1050h) than Eric's last version (table @ 117Ch).)

I found that testing LBAcache works best with ''buffers=1'' in ldos.ini, or by adding ''append buffers=1'' to the kernel command line. That way, DOS cannot buffer as much as it would usually, hence more reads go to LBAcache's handler.


===== patchini =====

The patldos.asm assembly language program's matching of progress type keywords was severely broken. [[https://hg.pushbx.org/ecm/patchini/rev/9c31e776f365|The changeset message]] states:

<blockquote>**patldos.asm: fix match of PRO mode keywords**

Selecting "app" would indicate an error stating that
"pp" was an invalid progress mode. The cause of this
error was the stray ''mov dx,si'' (si is CHG after the call to
comparecaps).

Next, the description would be matched unintentionally.
This was due to not incrementing past it for the first
.loopinnerpro iteration. Solved by moving ''inc di'' twice to
that label.

Subsequent mode keywords wouldn't be matched because the
code at .nextouterpro popped di once rather than twice.

Finally, .gotpro doesn't use the stacked di pointers any
longer so discard them off the stack.</blockquote>


===== lDOS kernel =====

  * [[https://hg.pushbx.org/ecm/msdos4/rev/d4ce07bf298b|Fix a typo]] in the manual.
  * Empty changeset [[https://hg.pushbx.org/ecm/msdos4/rev/1f460b996667|to rebuild]] for the new patldos revision of patchini.


===== Bugtracker =====

[[https://hg.pushbx.org/ecm/bugtracker/shortlog/0ca5ed512dd7|Add new bugs]] recently found:

  * [[https://pushbx.org/ecm/bug/1012/|Build of small inicomp test program failed]].
  * [[https://pushbx.org/ecm/bug/1013/|Add the scanner mode]] to inicomp.
  * LZMA-lzip depacker [[https://pushbx.org/ecm/bug/1014/|did not work with small buffers]].
  * LZMA-lzip depacker [[https://pushbx.org/ecm/bug/1015/|didn't check window size properly]].
  * New NASM [[https://pushbx.org/ecm/bug/1016/|spams label changed during code generation warnings]].
  * [[https://pushbx.org/ecm/bug/1017/|Pass mak.sh parameters faithfully]] (including quoted blanks) from mak.sh bash to the assembler that's executed via the makefile.
  * [[https://pushbx.org/ecm/bug/1018/|Allow mak.sh variables for multiple options to quote]] so that parameters with blanks can be passed.
  * ident86 [[https://pushbx.org/ecm/bug/1019/|would misdetect instruction boundary]] if a difference is in a continued machine code dump listing line.


===== wwwecm scripts =====

  * [[https://hg.pushbx.org/ecm/wwwecm.scr/rev/ef33aa0779b3|Add LBAcache build]] to script.
  * [[https://hg.pushbx.org/ecm/wwwecm.scr/rev/d7568939bcd0|Add intlist documentation]] to script, older change that was present in the working directory but not yet checked in.

===== LBAcache =====

  * [[https://hg.pushbx.org/ecm/lbacache/rev/2672ee66af9e|Add empty file]] to track listing subdirectories.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/005f797605d7|Add mak.sh script]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/5af6d9933549|Add a .hgignore file]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/4d4b0059de58|Include the lmacros3.mac file]], identicalised to prior revision.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/a6f48789b982|Optimisations]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/4a2d458af961|Automate calculation of three defines]]. Identicalised.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/4d2e388044a1|Strip trailing whitespace]], using a perl scriptlet: ''perl -i -pe 's/[ \t]+([\r\n]*)$/$1/' *.* tools/*.*''
  * [[https://hg.pushbx.org/ecm/lbacache/rev/f4745ab8d86e|Move the table label]] into an ''[absolute $]'' section, so its alignment need not be emitted into the binary. Identicalised.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e8315b996c79|Delete the progbits alignment]] in hello message.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/48aa8e21211b|Update version]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/7bcceea825ad|Update all mentions of "GPL 2"]] to "GPL 2+".
  * [[https://hg.pushbx.org/ecm/lbacache/rev/b50a71598a7c|Update attribution]] to include ecm.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/fd8e6eed79e9|Avoid repeatedly writing to tableend]]. The tableend and bonustable variables already were using the same word in memory, yet the writers wrote using both labels.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/ffa74228ea49|Optimisations in binsel.asm]]
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e3a3894448a5|Check for an invalid S=0 coordinate]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/70b449f6a95c|Align table on an 8-byte boundary]] (rather than only 4-byte).
  * [[https://hg.pushbx.org/ecm/lbacache/rev/453382311d5e|Align several variables]] on word or dword boundaries.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e2db02b782d7|Drop tabs before help message db directives]], so the source text lines fit on the screen better.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/4ae4f655c895|More optimisations]], including to call old int 13h without a retn near return eating a stack slot.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/df1d4907a3da|Add amis.mac macro file]] from [[https://hg.pushbx.org/ecm/ldebug/file/ae6cbfef5f51/source/amis.mac|lDebug repo]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/904991c156cd|Use IISP header for int13new]]. Drop the magic signature used to find resident instances in the memory scan. This makes the current uncache component incompatible with resident instances of this revision.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/b50adb0b2c4b|Optimise the int 13h handler]] and jump to downlink.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/6e256d26b987|Add an int 2Dh handler]], not yet used.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/63e44f6a1c55|Fix so that XMS handle is freed]] on later installation errors.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/6d1bee7c2476|Check that int 2Dh and int 2Fh are valid]] before calling or hooking them in setup.asm
  * [[https://hg.pushbx.org/ecm/lbacache/rev/689d1e6a17af|Find a free AMIS multiplex number]] and patch it into the multiplexer's int 2Dh handler.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/ba993c7a70f6|Install the int 2Dh handler]]. This completes most of the resident part of the multiplexer.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/05c76d9e821c|Make checkhandlers a function]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/8d1e0520e938|Display the allocated multiplex number]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e3c23f16cc52|Parse MPX keyword]] for preferred multiplex number.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/cbde703027e4|Display hex number hexits in allcaps]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/5110c7a97697|Insert a cld]] in the NEWI13 handler.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/fac36cbd063b|Move strtty function]] from lbacache.asm to uncache.asm
  * [[https://hg.pushbx.org/ecm/lbacache/rev/0199ffa5d4dd|Setup for a stand-alone build of uncache]]. Doesn't build yet. The lbacache.com build is identicalised.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/5c812f34b9a1|Add CTRLx structures]] needed by uncache to access the multiplexer.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/fe7c971a4d4a|Call checkhandlers in uncache]], and use args instead of the separate unflag byte variable.
  * Fix [[https://hg.pushbx.org/ecm/lbacache/rev/a5d374acfe34|to actually call eject (13.46) with drive B:]] rather than calling it twice for drive A:
  * Change amissig.desc [[https://hg.pushbx.org/ecm/lbacache/rev/36a18da9c1ae|to refer to "disk read cache"]]. Description of the hgweb repo also changed.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/d56968f4a10a|Drop unused variable named suppzero]].
  * Add mak.sh [[https://hg.pushbx.org/ecm/lbacache/rev/cd9d43890388|lines separating the assembler commands]], to be able to tell at a glance whether an error corresponds to the lbacache.com or uncache.com build.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/670593b49810|Add the multiplexer detection loop]] to uncache. Uninstall is broken. Control variables are accessed using the hardcoded layout as yet. The stand-alone uncache.com build doesn't assemble yet.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/9479fee48bbd|Add uncache files]] to .hgignore file.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/2bf2c60f5a74|Convert the infolists]] to an expanded format, which controls formatting in more detail and makes use of the ctrlX variables and interrupt list as returned by the resident multiplexer.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/0f8f77c33c10|Add INFO display]] of new i13 handler, old and new i2D handlers, and the multiplex number.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/76ffe29dfe71|Mark some dispatch.asm variables as ctrl4]], and expose them from the multiplexer.
  * Fix [[https://hg.pushbx.org/ecm/lbacache/rev/a8ceb8a4ba6e|to reset the amisintrptr]] if a subsequent instance doesn't support int 2D.mm04 as expected (unlikely).
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e83569b5fe8a|Add CTRL4 structure and variables]] to uncache.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/7b4b2ad8b45e|Fix detection of MPX keyword]] in command line processing, the high byte of EAX must be cleared for a valid check.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/da876392c712|Use remaining ctrlX and intr pointers]] for uncache. The uncache.com stand-alone build assembles for the first time. Uninstalling is commented out and as the changeset message states, "ecm advanced deinstallation method is tbd".
  * [[https://hg.pushbx.org/ecm/lbacache/rev/55192f4ac9ad|Allow running 2D.mm02]] if XMS handle already freed.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/be8deb73a0ab|Implement STOP]] for the multiplexer. Picked lots of code [[https://hg.pushbx.org/ecm/tsr/file/3c25a002aa4b/transien.asm|from the lDOS TSR example]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/d49b5ed3f08c|Add a cld]] to int 2Dh handler.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e725a3b17dac|Drop unused variable currint13]], no longer read anywhere.
  * For STOP command, [[https://hg.pushbx.org/ecm/lbacache/rev/c65eebe935fa|disable resident instance and free XMS handle after the interrupt unhooking simulation succeeds]]. This makes it more likely to end up either with a completely uninstalled instance or one that is still operable.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/a8d7a76659a5|Add int 2Dh functions]] to flush all drives or one drive. This is required to properly implement disabling and enabling the resident instance; enabling needs to flush so no stale data is used.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/ade8f4401d03|Fix flush function]] to initialise EAX.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/6aa1611166c5|Add DISA and ENAB commands]] to disable or enable a resident instance.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/b144934108ca|Add ONCE command]] to operate on the first detected instance only, rather than on all instances that can be detected.

**ETA**:

  * [[https://hg.pushbx.org/ecm/lbacache/rev/cd965b749f33|Move the hello message]] into space re-used for the cache table, to save on resident space. The multiplexer already contains some identifying strings.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/ecb1b12d46c3|Optimise resinst entrypoint]] to just a call flush, increment running variable, and run int 21h to execute TSR. Saves some resident space.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/9e1bf39ca53b|Close all process handles]] on successful installation before TSR using int 21h function 31h. Fixes the System File Table (SFT) leak if run with I/O redirection.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/b72e2e0892d8|Add RUNNING_ equates]] and clean up uses of the running variable, introducing the states RUNNING_INIT (as before), RUNNING_ENABLED (as before), and the newly distinguished disabled states: RUNNING_DISA (disabled using DISA command), RUNNING_SHUTDOWN (resident instance shut itself down due to invalid sector addressed or fishyfishy handler entered), RUNNING_FREED (XMS handle has been freed and thus the instance must not be re-enabled without allocating an XMS block).
  * [[https://hg.pushbx.org/ecm/lbacache/rev/f02663b6494e|Fix MEMOIZE function]] to use the drive/unit number for hashing, rather than the LRU byte of the cache table entry.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/e633b619a602|Drop some cs: overrides]], callers of MEMOIZE already set up ds =%%%%> resident segment.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/4a73173e497f|Use all-ones cache table entries]] as uninitialised. This saves some code in the flush functions which can just init the affected entries using all-ones. The inverted mask of used sectors takes slightly more code. The unit number field is used the same as before. The most complex change is using 255 as a "not used" LRU code and move the "frozen" LRU code to 254.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/83712112f143|Fix REMEMOIZE code]] to mask away the BINMASK from sector number in EAX, both to run the memo hash and to compare the resulting cache table entry to the requested sector.
  * In setup [[https://hg.pushbx.org/ecm/lbacache/rev/4a7f9f3dc25d|use the preferamisnum variable]] to check for a free multiplex number. In command line processing, do not set amisnum variable, only preferamisnum. This is in preparation for the next changesets.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/88c5b167e3b9|Delete the magic number]] previously used to scan for resident instances.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/39184d803a7d|Apply a patch to PSP:60h area]] (which is [[https://retrocomputing.stackexchange.com/questions/20001/how-much-of-the-program-segment-prefix-area-can-be-reused-by-programs-with-impun|safe to re-use without interfering with DOS]] for the TSR), to stash part of the multiplexer (interrupt entries and int 2Dh handler) in the PSP trailer and in the spot used by the flat-format application entrypoint jump. Saves some resident space.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/b71109ed4340|Insure that there's enough stack space]] between the end of the tables and the stack used during TSR, so that DOS certainly won't corrupt the bonus table after the flush.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/64585ebdf5e1|Align the fbPOINT variable]] (find bin last used table entry pointer).
  * Optimise, [[https://hg.pushbx.org/ecm/lbacache/rev/6ba7b2f5339c|drop the default cs: override]] from MEMOHASH mmacro. All users have ds =%%%%> resident segment set up.
  * Optimise, [[https://hg.pushbx.org/ecm/lbacache/rev/ee779f382596|do not MEMOIZE after a REMEMOIZE match]], would just rewrite the same bonus table entry with the pointer to the same cache table entry.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/3578549d0d05|Disable the checks on bonus table entries]] being in range of the main cache table. This should always be true, since we only ever write valid pointers to the bonus table.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/6c274fbb57f3|Add ctrl5 control data structure]] with the point query, point hit, memo query, and memo hit counters (each a dword). Inclusion in the resident instance is controlled by a define (default enabled), whereas the transient uncache always includes support for these. Adds the new MEMO command to list these counters. (Note that point query minus point hit always should equal memo query.)
  * [[https://hg.pushbx.org/ecm/lbacache/rev/379694dd8733|Drop an obsolete comment]] on the data layout of the resident instance.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/c3222c19eb1d|Move telltabsize]] into setup.asm and inline it into the only caller. Saves a bunch of resident space and even some of the file size.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/462f08028ad0|Add the TABLEENTRY structure]] and use its labels throughout instead of hardcoding the entry field offsets.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/475792f02f0c|Align the nbwhere and hashwhere variables]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/40e5a12232e1|Swap the jumps in betterbin code]], so that the expected repeated jump instruction comes first.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/8a0093059fcd|Fix bestalloc]] to treat ''byte [teLRU]'' equal to 255 as an unused table entry.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/a0c2a7eb0226|Move the binsel settings]] into lbacache.asm, preparing for the next changeset.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/179f62bf24a3|Hardcode the BINTABFORM define into tabsz variable]] instead of initialising it with a separate instruction in setup.asm
  * [[https://hg.pushbx.org/ecm/lbacache/rev/f1405f192549|Add amounttableentries variable]] to do fewer resident calculations, saving some resident space. Also optimise the setup some more.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/19a95595d28d|Optimise zeroing of stats]] by using ''rep stosd''
  * For ZERO, [[https://hg.pushbx.org/ecm/lbacache/rev/52bd9b80253e|also clear the known ctrl5 counters]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/359d7a51582e|Add TABLE command]] which uses the newly expanded ctrl2 structure to count some values on the cache table use.
  * For TABLE command [[https://hg.pushbx.org/ecm/lbacache/rev/0d47f2e0a843|also display amount of total and likely used bonus entries]]. The heuristic used is that a bonus entry pointing to c2hint (ie, the very first cache table entry) is likely unused, otherwise it is likely used.

**ETA**:

  * [[https://hg.pushbx.org/ecm/lbacache/rev/59752a68c540|Free separate stack block]] on int 2Dh function 02h call (uninstall), if TUNS was used during installation.


==== Highlights ====

=== Eric's bugs ===

The veritable bugs in the old revision of Eric's LBAcache, mostly copying the entries from above that list the fixes:

  * [[https://hg.pushbx.org/ecm/lbacache/rev/e3a3894448a5|Check for an invalid S=0 coordinate]].
  * The memory scanning uninstaller didn't free all memory if multiple instances of LBAcache were installed. The multiplexer uninstaller naturally finds the downlink from one LBAcache instance to another and uses it to uninstall everything, regardless the order of the scan results.
  * The memory scanning didn't find an LBAcache instance in a certain UMB on dosemu2, due to the segment addresses that it chose to scan, specifically with the PSP at segment C1AAh.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/63e44f6a1c55|Fix so that XMS handle is freed]] on later installation errors.
  * Fix [[https://hg.pushbx.org/ecm/lbacache/rev/a5d374acfe34|to actually call eject (13.46) with drive B:]] rather than calling it twice for drive A:
  * [[https://hg.pushbx.org/ecm/lbacache/rev/9e1bf39ca53b|Close all process handles]] on successful installation before TSR using int 21h function 31h. Fixes the System File Table (SFT) leak if run with I/O redirection.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/f02663b6494e|Fix MEMOIZE function]] to use the drive/unit number for hashing, rather than the LRU byte of the cache table entry.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/83712112f143|Fix REMEMOIZE code]] to mask away the BINMASK from sector number in EAX, both to run the memo hash and to compare the resulting cache table entry to the requested sector.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/b71109ed4340|Insure that there's enough stack space]] between the end of the tables and the stack used during TSR, so that DOS certainly won't corrupt the bonus table after the flush.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/59752a68c540|Free separate stack block]] on int 2Dh function 02h call (uninstall), if TUNS was used during installation.


=== ecm's bugs ===

Bugs that I introduced and later fixed during my work on LBAcache:

  * [[https://hg.pushbx.org/ecm/lbacache/rev/7b4b2ad8b45e|Fix detection of MPX keyword]] in command line processing, the high byte of EAX must be cleared for a valid check. Introduced [[https://hg.pushbx.org/ecm/lbacache/rev/e3c23f16cc52#l1.9|in the change adding the MPX keyword]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/ade8f4401d03|Fix flush function]] to initialise EAX. Introduced [[https://hg.pushbx.org/ecm/lbacache/rev/ffa74228ea49#l1.156|in the binsel.asm optimisations]].
  * [[https://hg.pushbx.org/ecm/lbacache/rev/8a0093059fcd|Fix bestalloc]] to treat ''byte [teLRU]'' equal to 255 as an unused table entry. Introduced [[https://hg.pushbx.org/ecm/lbacache/rev/4a73173e497f|in the all-ones as uninitialised]] patch.


=== Optimisations ===

  * Aligning several variables.
  * Resident space saved.
  * Fixing the MEMOIZE and REMEMOIZE bugs (see above).
  * Optimise, [[https://hg.pushbx.org/ecm/lbacache/rev/ee779f382596|do not MEMOIZE after a REMEMOIZE match]], would just rewrite the same bonus table entry with the pointer to the same cache table entry.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/3578549d0d05|Disable the checks on bonus table entries]] being in range of the main cache table. This should always be true, since we only ever write valid pointers to the bonus table.
  * [[https://hg.pushbx.org/ecm/lbacache/rev/f1405f192549|Add amounttableentries variable]] to do fewer resident calculations, saving some resident space. Also optimise the setup some more.

{{tag>patchini ldos msdos4 bugtracker wwwecm.scr lbacache amis tsr}}


~~DISCUSSION~~