Table of Contents
Late mid November work on symbol file program
2026-11-22
This week I mainly worked on the symscr ("symbolic support scripts") repo, adding what is for now called processt.py (process trace listing file to generate a symbol file).
ident86
- Fix to escape percent sign in a message that refers to %note directive.
tractest/convlist.pl
- Add section first info (group if any, and start of the section). The start may be nonzero if the section is part of a group. It matches the vstart of the first byte within the section.
(The special dump contents have to be supported to properly resolve the offsets: they're otherwise not modified.)
scanptab
- Use the lDebug mmacro dynamiclabel, with a fallback when not used as an lDebug include file.
- Drop the .nolist from the %ifnmacro directive. This probably caused a warning in some settings.
lDebug (experimental)
- In sections.mac fix the sense of an %ifidni directive.
- Disable listing of options.mac if
-f objand not assembling debug.asm - Support BUILDNAME variable with slashes in makefile.
- Support build_name variable with slashes in mak.sh
- In scanptab use dynamiclabel mmacro.
- In path.asm ELD source text, add dynamiclabel mmacro to prepare for its use in isstring.asm
- Use dynamiclabel mmacro more, and add underscores to file thunks so as to make their label names more readable.
- Allow question marks in symbol names (in
issymbol?function). - Drop .nolist from %ifnmacro directive in scanptab
- Drop [list -] directive in asmtabs.asm so that the
mnemmacro is listed. - In extname.asm ELD do not warn for Y command with .SYM filename extension.
- In run.asm use dynamiclabel mmacro for a few labels.
- In asmtabs.asm list more, except for the %exitrep use (would unbalance the listed %if directives). The problem solved by this is that nolisted mmacros appear to hex dump in the prior section even if they contain
usesectionmmacros. - In msg.asm add a special defer symbol comment for a certain case. The
msg.boothelplabel ended up wrong otherwise due to the use ofabsolute $before it. - In ss.asm use mmacros for the reloc and reloc2 macros rather than smacros. This simplifies handling in processt.py to not misdetect
reloc dwas a label.
Symbolic support scripts
- Fix process.pl :addsymbols to not count the start symbol, as it isn't written in the same block.
The remaining changes are all about processt.py, the program to process trace listing files into symbolic lDebug symbol files.
- Copy insbound.py from ident86 a11d95dd7331 to processt.py
- Initial commit of processt.py
- For conditions of
%ifand%eliflevels, use rstrip rather than strip. This preserves the initial blank if any, so that the original directive including the condition can be extracted from the level. - Add -d 1 switch to emit labels in %if %else constructs where no common labels are found.
- Add -d 2 switch to emit unmatched labels in %if %else constructs where some common labels are found.
- Add the sureinactive variable to the
aleveltuple. - Process %rep constructs similarly to mmacros, as always inactive.
- Detect dynamiclabel mmacros and some number of instructions as indicating inactive legs if they don't produce the expected listings (a note or hex dump).
- Detect inactive legs from
%noteor%warningthat doesn't produce the expected listing. This has a bug if a%warningappears as the last line before an%endif. - Return labels from the handlepotentialactive function, rather than appending them to
levelstreedirectly. - Check for an %else as the last level in the handlepotentialactive function. (Last leg level is first level in the
leglisthere.) - Sort labels on linear address only once, then re-use the sorted list.
- Prepare to sort labels on the .tls line number before expanding local labels. Currently they are always in this order already, however this is about to be changed.
- Allow more than 2 hexits before <rep Xh> hex dump content. This wrongly assumed that only the last byte is repeated, which is possible but not the only possible case. This is in fact ambiguous in the listing file of current NASM versions.
- Handle the ambiguous <rep Xh> as deferring subsequent labels, to avoid getting the size wrong sometimes.
- Accept <rep Xh> with a relocated hex dump, eg
[0000]<rep Xh>. Also defer the subsequent label. - Primitive handling of a non-nolisted mmacro that contains %00 in its expansion. This is needed in lDebug msg.asm for externally built help pages.
- Support <rep Xh> hex dump contents without any hexits. This may contain a wrong offset.
- Add the special case for a defer indicator comment ("SYMBOLIC defer symbol"). This is needed for the
absolute $use in lDebug msg.asm so thatmsg.boothelpgets deferred. - On
%unmacro, discard from the zerozerommacros set. - Fix, if %if is the inactive leg then mark another leg of %if %else construct as active. Wrongly treated a certainly inactive
%ifas a candidate for active leg. - Don't add inactive conditions to extra[1] if that leg is certainly inactive itself.
- Fix, only add label for %00 if zerozerolabel is not None. (This probably just worked out fine anyway. The undefined label would be added to the current level, but if that was an mmacro then it was certainly inactive and thus the labels were never used.)
Conclusions
Yet to be done: The insbound.py base contains some code that isn't needed, such as keeping track of the size/type of instructions and the found labels. These could be deleted with a little effort.
In its current incarnation, the program finds more than 5300 labels for the -f obj build of lDDebug, all of which match labels found by process.pl for the -f bin build, except for a few labels that differ naturally. These include ..start and the various "behind" labels, as well as relocatedzero. The .start labels for the sections WLEMITALIGNINITEND and WLEMITALIGNRELOCZERO are unique too.
The letter caps of some section names differs as well. And the DATASTACKBASE section of -f obj corresponds to DATASTACK of -f bin.
The -f bin build finds more than 7600 labels, and in particular it includes equates (eg equ $ or equ anotherlabel) and the labels generated for @@ anonymous local labels (up to 1513 of which exist).
Scriptlets used to check the resulting symbol files
To rebuild lDDebug occasionally, both using the -f bin and -f obj builds:
use_build_make=0 ./makerd; ./makerd
To generate the -f obj symbol file and -d 3 -c outputs of processt.py (this takes a while, about 15s), and compare to a prior revision's output:
~/proj/symscr/processt.py ../lst/ddebugl.tls ddebugl12c.sym -o 256 -d 3 -c \ | perl -pe 's/ddebugl..?c\.sym/ddebugl5c.sym/' > 12c.tmp \ && (diff -u ddebugl11c.sym ddebugl12c.sym; diff -u 11c.tmp 12c.tmp)
To update the -f bin symbol file, and search for lines in the -f obj file not matching any in the -f bin symbol file. For the search, both files are forced to allcaps and DATASTACKBASE from the -f obj file is replaced by DATASTACK.
~/proj/symscr/process.pl ../lst/ddebug.map > ddebug.sym \ && cat ddebug.sym | tr 'a-z' 'A-Z' > caps.tmp \ && cat ddebugl12c.sym | tr 'a-z' 'A-Z' | perl -pe 's/(DATASTACK)BASE/$1/' > caps5.tmp \ && grep -F -x -v -f caps.tmp caps5.tmp | less
Lines found in -f obj symbol file but not in -f bin symbol file
Output of the last scriptlet on the latest revisions:
; 2 DEBUG.LST ; 11449 AA.LST ; 65812 AMIS.LST ; 68325 BB.LST ; 72733 CC.LST ; 74889 DD.LST ; 80945 EE.LST ; 83294 EXPR.LST ; 91972 FF.LST ; 94032 FPTOSTR.LST ; 96432 HELPDEPK.LST ; 99128 HH.LST ; 101424 IIOO.LST ; 103742 IMMASM.LST ; 107186 INSTALL.LST ; 109433 INTS.LST ; 112097 LINEIO.LST ; 120464 LL.LST ; 123080 MM.LST ; 126483 MSG.LST ; 133905 PMDATA.LST ; 135960 PMENTRY.LST ; 138299 PMINIT.LST ; 140905 RR.LST ; 145973 RUN.LST ; 155746 SERIALP.LST ; 158548 SS.LST ; 161854 SYMBOLS.LST ; 167041 UU.LST ; 173794 VV.LST ; 176130 WW.LST ; 178350 BOOT.LST ; 187611 INIT.LST Z ADD L=(01F5C0 + V1) O=3650 S='SECTION.WLEMITALIGNINITEND.START' F=0120 Z ADD L=(01F5F0 + V1) O=0000 S='SECTION.WLEMITALIGNRELOCZERO.START' F=0120 ; TOTAL 11 SECTIONS Z ADD L=(000000 + SL.SECTION.LDEBUG_DATA_ENTRY.START) O=0100 S='..START' ; 1222 SYMBOLS ; 22 SYMBOLS ; 1 SYMBOL Z ADD L=(000000 + SL.SECTION.DATASTACK.START) O=81F0 S='BEHIND_PROGBITS_DATAENTRY' ; 66 SYMBOLS ; 48 SYMBOLS Z ADD L=(000000 + SL.SECTION.LDEBUG_CODE.START) O=0000 S='BEHIND_MESSAGESEGMENT' ; 3400 SYMBOLS Z ADD L=(000000 + SL.SECTION.LDEBUG_CODE2.START) O=0000 S='BEHIND_CODE' ; 1 SYMBOL Z ADD L=(000000 + SL.SECTION.INIT.START) O=0000 S='BEHIND_CODE2' ; 529 SYMBOLS ; SECTION "WLEMITALIGNINITEND" Z ADD L=(000000 + SL.SECTION.WLEMITALIGNINITEND.START) O=3650 S='BEHIND_INIT' ; 1 SYMBOL ; 10 SYMBOLS ; SECTION "WLEMITALIGNRELOCZERO" Z ADD L=(000000 + SL.SECTION.WLEMITALIGNRELOCZERO.START) O=0000 S='BEHIND_DEVICESHIM' Z ADD L=(000000 + SL.SECTION.WLEMITALIGNRELOCZERO.START) O=0000 S='RELOCATEDZERO' ; TOTAL 5302 SYMBOLS
(The -f bin symbol file ends with "; total 7650 symbols".)
