User Tools

Site Tools


blog:pushbx:2022:1126_ldebug_debug_x_2.00_nasm_lz4_depacker_for_vgapride

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

blog:pushbx:2022:1126_ldebug_debug_x_2.00_nasm_lz4_depacker_for_vgapride [2022-11-26 22:11:38 +0100 Nov Sat] (current)
ecm created
Line 1: Line 1:
 +====== lDebug, Debug/X 2.00, NASM, LZ4 depacker for VGAPride ======
 +
 +**2022-11-25**
 +
 +Recently I fixed a few bugs in lDebug and added two new address keywords. Other than that, the original Debug/X just jumped to version 2.00.
 +
 +
 +===== The bugs =====
 +
 +The first bug was that leading blanks in commands read from the /C= switch or from the kernel command line were included verbatim instead of trimmed. This had the effect of indenting these commands oddly. (At first I believed this was due to lCDebug's conditional tilde in the prompt because I noticed the bug with a kernel command line starting with the command ''r dco6 xor= 100''. Not so however.)
 +
 +The second bug was that in order to specify a double-slash to the BOOT PROTOCOL= command, which is to indicate not to search for an additional file, it was necessary to enable command line support and specify a non-empty command line. The use of the ''iseol?'' function in this was confused and wrong. The fix was to delete that and a few more instructions. It was a while ago that I noticed this bug first, while trying out the IBMDOS protocol.
 +
 +
 +===== The taken keywords =====
 +
 +During debugging of the double-slash problem I had to step into the BOOT command. In the various command, protocol, and protocol flags loops I repeatedly had to skip to the target of a conditional branch that did not yet jump, in order to skip past the numerous iterations of the loop.
 +
 +This gave me the idea for a new command to do just that: ''GT'' for "Go to Target", or "Go to Taken branch". A complementary ''GNT'' soon suggested itself, for "Go to Not Taken branch".
 +
 +I at first considered adding the new command as just that, a command. However, I thought of how one may want to set additional breakpoints, at which point the new command would have to share or copy a lot of the breakpoint handling from the normal ''G'' command. So I had the idea of parsing the ''T'' and ''NT'' specifiers as keywords in the normal ''G'' command's breakpoint parsing loop.
 +
 +Next, I thought you may want to set a permanent breakpoint to one of the taken addresses as well. So the handling of those keywords moved up into ''getlinearaddr'', the common function used by gg and bb commands to receive a breakpoint address.
 +
 +Yet again, I noticed that ''getlinearaddr'' will simply call ''getaddr'' if the ''@'' special case is not used. So I decided to actually put the taken keywords parsing into ''getaddr'', meaning that all callers of ''getaddr'' get to use the keywords. Particularly, the U command and D command can use them, and so can the P/T/G commands' equal sign parsing.
 +
 +Along the way I decided to do two forms of each of the two keywords: The short forms ''T'' and ''NT'' to effectively allow a ''GT'' command, as well as the long forms ''TAKEN'' and ''NOTTAKEN'' to allow more verbose uses either for the G command or any other use of the keywords.
 +
 +Aside from all things mentioned so far, I also noticed that I can abuse the ''TTEST'' command to set ''cs:ip'' without running anything. This command was actually intended as a test aid, but it works for this new purpose in a pinch.
 +
 +Finally, the implementation of the taken keywords copies and re-uses parts of the pp commands to scan the possible prefixes and instruction at the current ''cs:ip''. It allows ''jcc rel8'' instructions, the 386+ ''jcc rel16/rel32'', as well as all variants of ''jcxz'' and ''loop''. If no match is found then during parsing the taken keyword is rejected with an error.
 +
 +
 +===== Debug/X version 2.00 =====
 +
 +The original Debug/X project, which I tend to still call FreeDOS Debug, recently jumped to the version number 2.00. The primary changes are support for the special-purpose DebugB (bootable) and DebugR (Protected Mode Ring 0) variants. DebugR is supposed to debug JEMM or HDPMI, and there are framing programs now included with JEMM as well as a repo called Deb386 which will use a DebugR image.
 +
 +Other than the new Protected Mode support, three files for the different entrypoints were extracted from the main assembly source file. These are for 86 Mode, ring 0 Protected Mode, and DPMI Protected Mode.
 +
 +The workings of these entrypoints was also changed. The ''run_int'' variable is only modified for normal entries, not if a call is passed on or a debugger exception occurs. Also, the handlers will use byte values that present as displacements from a base to point to an entry type message. Other changes concern the Ring 0 PM entries, which may need to handle error codes and Ring 3 stack pointers on the exception stack.
 +
 +
 +===== R86M fault handlers =====
 +
 +I'm unsure when this was introduced, but Debug/X now also supports catching interrupts 7, 0Ch, and 0Dh in Real or Virtual 86 Mode. The stack fault and general protection fault handlers are intended for 286+ machines running in R86M, in which exceptions will call these interrupt handlers. Debug/X's handlers for these will check the primary PIC's In-Service Register to determine if the handlers were called due to IRQs or due to faults. Also, if a fault occurs while running Debug/X itself, the debugger exception handler is now used as well. Previously this was only used for DPMI exceptions. There is also a new switch, /M, which will disable the IRQ check in the interrupt 0Ch and 0Dh handling. It's supposed to be used if an OS sets the primary PIC base for interrupts to something other than 8.
 +
 +The three new interrupt handlers, including the IRQ checks and the debugger exception handling, as well as the /M switch, are now available in lDebug as well. However, all of those are made conditional at build time and disabled by default.
 +
 +One more change in lDebug is that the serial interrupt handler will do an In-Service Register check similar to the interrupt 0Ch handler's, seeing as it can be used for interrupt 0Ch (for the COM1 port usually). If the ISR indicates that no IRQ is in service then the handler will pass along the call to the next interrupt handler. This change was necessary to allow using the stack fault handler, as that one is installed earlier and the serial interrupt handler later, so the serial handler sees a fault call first. This change was made unconditional, that is it is included in the default build.
 +
 +
 +===== Another change in lDebug =====
 +
 +The isvariable structure format is now explained better in the comments of expr.asm and debug.mac, which among additions also fixes a comment that wrongly claimed the ivName field would point to an ASCIZ string. In truth, the length of that string is determined solely by a length field in the structure, and there are no terminators.
 +
 +Additionally, the name length limit for variables is checked at build time, at least for the simple cases (non-array variables, or variables that cannot be indexed). This limit is determined by the implementation and callers of the ''isvariable?'' function.
 +
 +
 +===== NASM updates =====
 +
 +In recent weeks NASM received a lot of updates. I can proudly say that the version 2.16 release candidate 10 fixes all the major bugs that I reported in the last two years. It is tested to work by assembling lDebug with its defaults. The fixes include the ''%00'' not working bug, ''%strcat'' not accepting commas, the bug with define names starting with a question mark, the shift direction bug in the preprocessor, the wrong line numbers in include files, and last but not least the big memory leak in ''%[...]'' construct expansion.
 +
 +
 +===== VGAPride LZ4 depacker replacement =====
 +
 +Finally, I went and built Foone's VGAPride to test building and running it with my LZ4 depacker replacement. The depacker it had was under "the Demoscene license", which I believe is incompatible with the GNU GPL v3-only usage conditions. I had suggested and prepared my replacement as early as 2022 June.
 +
 +It turns out that they used an old version of the ''lz4'' command that created output data in the "Legacy frame" format. This is incompatible with my depacker. With a current ''lz4'' command in the planize.py script, my depacker works perfectly. No changes were needed beyond that which I already prepared back then, without testing.
 +
 +As part of the preparations, I made a repo containing the three lmacros files as well as the LZ4 depacker of the inicomp repo. I added the parts of inicomp.asm needed by the depacker, and disabled several parts. I also set up a fixed size for the input and output buffers, both 64 KiB.
 +
 +Finally, a magic change was to not loop past the first frame upon finishing decompressing it. This is needed because VGAPride's ''display.cpp'' currently does not pass a size of the compressed data to the depacker. It turned out to be correct.
 +
 +The only downside to my depacker is that it is slower than the prior implementation. On a pcjs machine the decompression of each plane, with about 32 KiB of decompressed data each, takes about 1 second. It is much faster with the other depacker, although still perceptible.
 +
 +
 +{{tag>ldebug nasm inicomp}}
 +
 +
 +~~DISCUSSION~~
  
blog/pushbx/2022/1126_ldebug_debug_x_2.00_nasm_lz4_depacker_for_vgapride.txt ยท Last modified: 2022-11-26 22:11:38 +0100 Nov Sat by ecm