pushbx wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Early September work - MSDebug, toclip, lDebug ======


**2023-09-10**

The first thing of note is that the HP 95LX had been refusing to repeat keypresses when a key was held pressed down. This had been true for at least a week. The setup does not appear to have a setting for this. Rebooting with Ctrl-Alt-Del, then restarting the system software with a ''$sysmgr'' command, actually seems to have fixed this.

After the release 6, some work on lDebug happened this week. One change was made simultaneously in lDebug and MSDebug. On the suggestion of stsp, I worked a bit on [[https://hg.pushbx.org/ecm/toclip|toclip]], a tool for accessing the WinOldAp clipboard. I didn't get to test this yet, however, because dosemu2 for now only supports the clipboard when running in an SDL window. (On the server, I can only run dosemu2 in ''-dumb'' or ''-t'' mode.)


===== toclip =====

I [[https://hg.pushbx.org/ecm/toclip|created a repo]] based on [[https://hg.pushbx.org/ecm/toclip/rev/42ddd7cc9ddf|the exactly 25 year old release]] of this program. It noted that it was released under the "GNU Public License", which I interpret as GNU GPL, any version. I [[https://hg.pushbx.org/ecm/toclip/rev/8aee615680df|added a copy of the GNU GPL v2]] in the file ''copying'' to the repo.

I modified the program to compile with gcc ia16 along with libi86. I had to update the server's libi86 to gain the ''_binmode'' function, to replace the use of ''setmode''. Other than that the source was missing some headers, declarations, and return types.

It also wanted to define its own structures for SREGS and REGS. This was particularly confusing in that one error message from the compiler was along the lines of "expected STRUCT SREGS *, got STRUCT SREGS *". The meaning was there were two different types with the same name. The REGS structure was a plain structure in the application whereas the ''dos.h'' header defines a union. That means register accesses like ''r.ax'' and ''r.cflag'' had to be changed to ''r.x.ax'' and ''r.x.cflag''.


===== MSDebug and lDebug =====

The minimal size of an empty process to create was raised to 512 Bytes ([[https://hg.pushbx.org/ecm/msdebug/rev/71502112e197|1]], [[https://hg.pushbx.org/ecm/ldebug/rev/f07a9e8432d4|2]]). This insures that a simple DOS call can succeed without overwriting the instruction running it. In practice this doesn't matter much. The size was mentioned in MSDebug's manual, as well.


===== lDebug =====


==== Various ====

  * INSTALL [[https://hg.pushbx.org/ecm/ldebug/rev/7da0b3134d20|correctly skips an initial comma]], rather than any separator followed (but not preceeded) by whitespace
  * Check install flag [[https://hg.pushbx.org/ecm/ldebug/rev/feb24235c781|loop optimised]]
  * INSTALL AREAS properly [[https://hg.pushbx.org/ecm/ldebug/rev/72192bfede5c|preserves the CX register]], needed if another noun follows
  * The areas hook server's root structure [[https://hg.pushbx.org/ecm/ldebug/rev/52f27064a60c|contains some code in its entry]] that will return an error when called
  * The Q command [[https://hg.pushbx.org/ecm/ldebug/rev/f20e42a8524f|will check that an areas hook client's areas structure was uninstalled]] properly
  * The Q command [[https://hg.pushbx.org/ecm/ldebug/rev/485e3bdb8e00|will try to uninstall areas structures (both as client and server) earlier]], before unhooking mandatory interrupt hooks
  * The call to ''dd_get_one_line_range'' [[https://hg.pushbx.org/ecm/ldebug/rev/831e22d5c13d|was moved to reflect]] how commented out code is intended
  * [[https://hg.pushbx.org/ecm/ldebug/rev/f0b8201be637|DQ command]] (Dump as Qwords), disabled for now. This is nigh unreadable due to the lack of separators. Adding separators is not straightforward, and would in a certain case even overflow 79 columns.
  * [[https://hg.pushbx.org/ecm/ldebug/rev/023c14dac709|Spaghettify]] ''dd_display_offset'' to save on a jump
  * [[https://hg.pushbx.org/ecm/ldebug/rev/e240efec4e33|Optimise a check for two flags set]] by using two conditional jumps on the same ''test'' instruction result, including a Parity Flag conditional, to figure out whether both flags are set
  * [[https://hg.pushbx.org/ecm/ldebug/rev/4cbcaab42e97|Do not allow DI with two parameters]] where the second parameter is below the first
  * Fix display of an error message after 255 entries have been displayed for the same interrupt in DI command (without DIL flag) ([[https://hg.pushbx.org/ecm/ldebug/rev/abdc5e6bfc7d|1]], [[https://hg.pushbx.org/ecm/ldebug/rev/64f2f8207638|2]])
  * Don't [[https://hg.pushbx.org/ecm/ldebug/rev/27fdede27157|preserve CX unnecessarily]] across AMIS callout in DIL command
  * In DIL command [[https://hg.pushbx.org/ecm/ldebug/rev/32f2e24155d3|check for a valid int 2Dh handler]] before calling it
  * [[https://hg.pushbx.org/ecm/ldebug/rev/5778baa0918c|Optimise DX command]] building its GDT
  * Optimise ''getrange'' and ''getaddr_segment'' to not preserve SI that is not needed ([[https://hg.pushbx.org/ecm/ldebug/rev/576db359b9d7|getrange]], [[https://hg.pushbx.org/ecm/ldebug/rev/b6bc53d1b043|getaddr_segment]])
  * Replace some labels by more descriptive ones ([[https://hg.pushbx.org/ecm/ldebug/rev/083c1d113b89|getrange]], [[https://hg.pushbx.org/ecm/ldebug/rev/ac0405dfdeeb|getaddr]], [[https://hg.pushbx.org/ecm/ldebug/rev/76244cdc7a59|getstr]])
  * Compare the four ''(NOT)TAKEN'' keywords [[https://hg.pushbx.org/ecm/ldebug/rev/a81d3329bc1b|in a table loop]] instead of unrolled
  * Check ''getstr'' limits [[https://hg.pushbx.org/ecm/ldebug/rev/bc915b945143|using a new table]] holding the limit offsets, sharing the code previously in all six item handlers
  * Optimise length keyword detection loop ([[https://hg.pushbx.org/ecm/ldebug/rev/260597aaf99a|1]], [[https://hg.pushbx.org/ecm/ldebug/rev/fe839fdd5108|2]])
  * [[https://hg.pushbx.org/ecm/ldebug/rev/a2dff24107cc|Optimise calculation]] of ''isoperator?'' result index
  * Optimise operator dispatchers with two new tables that give [[https://hg.pushbx.org/ecm/ldebug/rev/3cc68f41b246|the default BX]] and [[https://hg.pushbx.org/ecm/ldebug/rev/f64a1a8e3039|DX inputs]], making many of the dispatchers trivial. This uses a branch technique I picked up from EDR-DOS, where [[https://hg.pushbx.org/ecm/ldebug/rev/7e7992e1e18b|a near function is called, it pushes the dispatch table entry]], and is then free to re-use the table offset register (BX) before running a ''retn''.
  * [[https://hg.pushbx.org/ecm/ldebug/rev/bc4953d1f46f|Fix a minor bug]] in HH twofold operation, it can run after a operator dispatcher returned failure, at which point it is allowed to have changed AL.
  * Trivial optimisation of operator functions for [[https://hg.pushbx.org/ecm/ldebug/rev/999eb05bd88f|multiply]] and [[https://hg.pushbx.org/ecm/ldebug/rev/10404ef44a5f|divide]]
  * [[https://hg.pushbx.org/ecm/ldebug/rev/5bb01e2fa0d6|Fix /A switch relocation]] if _DUALCODE and _PM enabled
  * [[https://hg.pushbx.org/ecm/ldebug/rev/3daba0a1b114|Fix code section truncation]] to apply to areas client structure if _DUALCODE=0


==== Fractional digit for size display ====

The size display is most well known for lDebug's extension to the FreeDOS Debug DM command (Dump Memory blocks). In this case, the unit used can only be Bytes or KiB. (Another use is the H AS BYTES command, also with other size keywords. Some BOOT commands also use the size display.)

Even so, it is useful not to skip from 2048 Bytes right to 2 KiB, but rather [[https://hg.pushbx.org/ecm/ldebug/rev/0de4bd6731a3|to display a fractional tenth digit]] for values between 2.0 KiB and 99.9 KiB. Displaying two fractional digits would be more complicated, so this code is limited to one for now. Also, the decimal point is hardcoded as a dot, for now.

It was a new experience to figure out how to calculate the fractional digit. After some false starts, I found that I simply had to multiply the last unit prefix division remainder (handily available) by 10 and then divide the result by the unit prefix divisor (1024 or 1000), yielding a value of 0 to 9.


==== Debugger relocation in init ====

Referring back to [[blog:pushbx:2023:0828_late_august_work#freedos_kernel|the FreeDOS kernel buffer underflow bug]] that randomly occurred when using WarpLink, there is obviously a use for installing the debugger in another spot than it would be loaded by FreeCOM by default. (LH would work, but the initial executable allocation of the debugger is much higher than its final allocation, so our UMBs do not have enough space for a direct installation there on dosemu2.)

Instead of juggling two debuggers, it was desirable [[https://hg.pushbx.org/ecm/ldebug/rev/3ef2c29eb6b3|to add an option to the debugger]] to install itself at another location. This, of course, requires the dreaded process relocation. It was a simple matter to adjust [[https://hg.pushbx.org/ecm/tsr/file/00a31da6be94/transien.asm#l1240|the code from the TSR example's process relocation]]. Setting the UMB link state and memory allocation strategy, as well as allocating memory for the process block and environment block, were also readily apparent.

Where it got more difficult was to adjust all segment references initialised by earlier stages of the init. The relocation happens before installing mandatory interrupt hooks in non-debuggable mode, but there are plenty of references already even so. What I did was to trace through all parts of the application mode init and note down any use of segment addresses.

Relocating so late in the init has the advantage that we know the final size of the debugger process block, after the auxiliary buffer has been expanded if selected by the /A switch. It also means that we can select the relocation using a switch of its own, the /T switch. This is highly desirable to keep things more compatible when the switch is not used.

Some parts of the /A switch handling were re-used for the /T switch relocation in fact, specifically for relocating the history buffer segment and the code sections. The auxiliary buffer, message segment, immediate assembly segment, and of course the entry segment all needed to be relocated additionally. Same as for the /A switch, let's hope we got all things that need to be relocated.


=== /T switch documentation ===

I opted not to document the /T switch just yet. Perhaps I will change this before the next release. While it is still undocumented as concerns the manual, here is a description of all switches:

  * /T or /T+: Enable relocation
  * /T-: Disable relocation (Default)
  * /TI: If memory cannot be allocated, ignore the relocation attempt rather than aborting the load
  * /TO: Only succeed loading if memory can be allocated (This is the default if the /T switch is enabled)
  * /TP: If relocation is enabled, run an ''int3'' instruction before attempting to relocate
  * /TL: Restrict relocation allocation to the DOS Low Memory Area
  * /TU: Enable relocation allocation to use the DOS Upper Memory Area (Default)
  * /T=number: Specify the memory allocation strategy to set up, in hexadecimal. (Default is 82h, Last fit High then low.)


=== /T switch bugs ===

Two bugs cropped up during testing of the new switch, before committing the first revision of the change.

The first bug was writing the /T switch flag to the wrong segment, corrupting a byte in the debugger entry segment. However, it was easily detectable that the /T switch had no effect.

The second byte occurred during setup of the stack trampoline that is used to free the original process block before branching to the installed debugger's ''initcont'' function. (The original block is left allocated until this point to avoid having to relocate the init section a second time.) I wrongly anticipated that the trampoline would need four more bytes of space allocated to it, and failed to fix this before testing. The only ill effect was the initial K command received the wrong pointer, leading to an error on being unable to find a file. This was also quickly found.

There were also some errors I figured out only later.

It is possible to relocate the debugger down from the segment it was initially loaded to. To test this, some trickery is required, such as loading three debuggers and instructing the first one with a TSR command early. (The first debugger can also use the /A switch to enlarge its memory footprint.) Then the first debugger can be unloaded by using a Q command, leaving a gap low in the Low Memory Area into which the third debugger can relocate.

During my tests I also used the /A switch on the third debugger to avoid it allocating a suitable UMB in our dosemu2 setup. This was before the /TL and /T=number switches were added. As that meant the allocation strategy was fixed to Last fit, I also used a small script called ''enlarge.sld'' which grows the process memory block to its maximum size. This left the third debugger no choice but to relocate down. This is the script:

<code>r sp -= 30
a ptr sssp
 push ax
 push bx
 push es
 mov ah, 51
 int 21
 mov es, bx
 mov bx, FFFF
 mov ah, 4A
 int 21
 pop es
 pop bx
 pop ax
 int3
 jmp (cs):(ip)
 .
g = ptr sssp
t
r sp += 30</code>

Within 16-bit arithmetic on segment values, it doesn't matter whether we relocate up or down, as a negative delta (relocating down) will result in the same target as overflowing the calculation through 65_536. However, not all of our references were using 16-bit segment arithmetic. In particular, the areas client structures uses 32-bit linear addresses. The relocations that can be done by the /A switch and the /T switch initially zero-extended the 16-bit segment adjustment value to 32 bits and added that to the prior value.

This was fine for the /A switch relocation of the code sections, as this relocation was always up. For the /T switch relocation however, the relocation could be down as mentioned. In this case, the overflow into the 17th segment bit was wrong. [[https://hg.pushbx.org/ecm/ldebug/rev/a57c47dd2d21|The fix was to detect the Carry Flag]] from calculating the segment delta, and use it to sign-extend the linear address to 32 bits using this carry. This allows the areas linear addresses to overflow their 32 bits correctly for relocating down.

[[https://hg.pushbx.org/ecm/ldebug/rev/5bb01e2fa0d6|The next bug was mentioned above already]], as it actually affected the /A switch relocation even before the addition of the /T switch. The code section relocation would leave a superfluous word on the stack if _DUALCODE and _PM were both enabled.

For /T, this would only lead to wrong restoration of the UMB link state, memory allocation strategy, and a failure in the initial K command (like the trampoline bug). For /A, a crash was likely, but only when the app mode layout 2 had been selected (and as mentioned both _DUALCODE and _PM were enabled). This is somewhat unlikely.

[[https://hg.pushbx.org/ecm/ldebug/rev/3daba0a1b114|The other bug was also listed before]], but I am including it here because I found it during testing the /T switch. It was that with code section truncation in non-bootloaded mode, and ''_DUALCODE=0'', and the areas client enabled (usually for _PM _DEBUG build), the areas client structure linear end address did not get truncated. This was a minor bug with no ill effect.


=== Additional choices ===

With the additional switches described above, it is possible to relocate the debugger [[https://hg.pushbx.org/ecm/ldebug/rev/1e78c7a4ad6d|only into the UMA]], or [[https://hg.pushbx.org/ecm/ldebug/rev/42888c3c5df0|only into the LMA]]. When the selected strategy does not yield enough memory for the relocation, the load can be aborted (original behaviour) or [[https://hg.pushbx.org/ecm/ldebug/rev/15d573461936|it can ignore the failure]] and skip relocation. No error conditions other than memory shortage exist, as no interrupts have been hooked yet at this point. (Process relocation of course has to work as expected to succeed, using the TSR's "terminate old process to new process" method.)

=== Conclusion ===

I am very happy with the results so far. Besides the versatility, all additions for the /T switch live purely in the init, so no additional resident memory is used. Unlike the hack of loading two debuggers to load the second one with Last fit strategy, the init of the relocated debugger does not leave an unavoidable gap above the relocated debugger's resident memory block.

Using First fit or Best fit strategy, a gap can turn up where the relocated debugger was originally loaded. Avoiding **that** would require relocating twice and also relocating the init section a second time. This is the problem for which the TSRs originally added process relocation. The debugger relocation has a bit of a reverse purpose, though. So this isn't much of a problem; if we want to allocate the debugger at the beginning of the Low Memory Area we can simply not relocate it at all.

{{tag>msdebug toclip ldebug}}


~~DISCUSSION~~
pushbx wiki

User Tools

Site Tools

Page Tools
