"Fun with BIOS" Programming Thread

[cr1901]

So that means that I have to update the CRC every time I write to my data area... wouldn't that have a significant effect on speed?

One 'feature' I thought of was that if a data area was not found, warn the user, give an option to continue and construct a 'default' data area with IBM PC BIOS-compatible interrupt vector locations, since they are by convention the same between BIOSes.

This would also probably work if the data area was clobbered, if one was willing to 'sacrifice' the program which decided to overwrite the debugger's data area .

The old vector addresses are pretty much the only data that must remain constant between debugger invocations (and even then, I could hardcode the jumps into the ROM at the potential cost of compatibility to non-compliant BIOSes).

 

[Chuck(G)]

An arithmetic checksum might be okay, and you only need to update the CRC/Checksum when you relinquish control. I'm saying only that there's a certain amount of bulletproofing expected from a debugger.

 

[cr1901]

EDIT: If you read the post I had here, ignore it! Sorry. I realized I was testing that in a version of the emulator from early 2011 that had known problems. It's not the case in the current, it just calls function 1h then 0h and sits there.

It calls func 0h immediately after 1h without a key even being pressed though!

Well then this just doesn't make a lot of sense to me, it looks like it's up to the BIOS code. What in the would could cause this?

I figured out what was wrong... at least partially. The reason why the ROM worked in Bochs but not Fake86 was because apparently the Bochs BIOS reserves the last kB of conventional memory for itself (related to the EBDA, whatever that means: See this Google search).

I was not properly interpreting the value from int 0x12, which on Bochs returns 639kB... I thought this meant 639kB extra kB of memory, and 639 (0x2CF) was simply the highest value that would be returned by any BIOS.

Long story short, my ROM was trying to allocate memory over the 640kB mark in Fake86, b/c I deliberately used the original returned amount of memory as the "base kB" of memory to start allocation. In other words if the BIOS returned 639kB, I decremented to 638kB, stored the decremented value in the BDA, and then started allocating at 639kB.

I say partially because the debugger still crashes if int 0x16 is called... which is a problem... I haven't checked the Generic BIOS yet, but if int 0x16, subfuntion 0x00 is anything like IBM's genuine implementation (it just sits in a loop waiting for buffer_head to not equal buffer_tail in the keyboard buffer, enabling external interrupt processing only for one instruction- a 'nop'- within the loop), I don't see why it should be crashing.

My hooked int 0x09 does work, however, after bootstrap at least.

Honest, I am getting somewhere . Just taking me a while to write this since this isn't exactly trivial.

 

[cr1901]

Re: Lack of Activity

Re: Lack of Activity

I'm still disassembling the COMPAQ BIOS, as I've stated in the "Portables/Handhelds" subforum, and I'm a bit discouraged that I just simply can't get the debugger's keyboard interrupt to work properly on a 8088-emulator. The interrupt DOES work if I wait till after boot to invoke the debugger, but something is causing the debugger to lock up while it attempts to find it's own memory area if invoked during boot (after Option ROM init) on an 8088-PC emulator such as Fake86. To that end, I decided to take a step back for a few days to avoid fatiguing myself working on fixing bugs.

Since people have taken a genuine interest in this project, have offered me advice, and even given me some hardware to use as a 'workbench', I feel that it's fair to give updates, even if not much has been accomplished in a given time frame.

Also, as some of you younger members may know already, GTA V comes out at midnight tonight and gameplay footage has already been uploaded (the 'gold master' binary was given to DVD manufacturers on August 25th or so). While I don't own a PS3 or Xbox 360/am waiting for the PC version. the footage so far of people's playthroughs has certainly been VERY enjoyable to watch- so that's also been the focus of my attention.

 

[Mike Chambers]

GTA V looks fun, yeah. Luckily I own a 360, even if I don't use it much. I was going to rent it when it comes to Redbox. If it's good, I'll buy it later when they release it for PC. I've been playing GTA IV again for a few hours tonight. (HEY COUSIN, IT'S YOUR COUSIN... LET'S GO BOWLING!!) Personally, I'm really waiting for Watch Dogs.

 

[cr1901]

(HEY COUSIN, IT'S YOUR COUSIN... LET'S GO BOWLING!!)

I never owned GTA IV, but I've definitely heard that quote enough to KNOW that I don't miss it XD. Also, from what I hear, bowling is in fact a bitch in that game.

Last time I recall playing it was when I got the One Man Army achievement for a friend... yea, cops are brutal in that game (and seem to be in V as will).

In any case, back to work on figuring out why I fail at ROMable programming .

 

[cr1901]

Where am I?!

Where am I?!

The last few days were a bit busy, and I really wanted to work on the code yesterday and tonight (in addition to preparing a new project to help me generate FM synthesis samples* akin to the Genesis YM2612 for SNES music programming- something on my bucket list). Mentally, I'm as sharp as ever, but the past few days I've had non-negligible physical pain in my lower back and have not able to sit and code for long periods of time before needing to lie down.

I most likely irritated the pain tonight by doing some physical work around the house- even now, I can barely get up from my bed. Curiously, the pain is negligible when I'm coding anywhere BUT home (including doing my thesis work each day), so I think I need to replace my chair. In any case, I will see what I can do to clean up my current code and release a minimal-component version in the next few days.

*For those interested in some context, the Genesis uses a Yamaha FM synthesis chip similar to those in Sound Blaster cards, but with different algorithms and ROM tables for modulating waveforms. The SNES uses sample-based music that the programmer is expected to generate.

 

[cr1901]

Some progress tonight!!

Some progress tonight!!

Mike Chambers, if you're still following this thread... I figured out why the keyboard interrupt was failing in Fake86 by manually generating an int 0x09 in my software. Since that worked, I realized that interrupts weren't reaching the CPU at all!

I confirmed this by checking the source to the Generic XT BIOS. The Generic XT BIOS does not enable interrupts until after the ROM scan, while in Bochs apparently, interrupts are enabled even during ROM scan. Since I can't assume that the keyboard will be available during ROM scan, I force the 8259 PIC to respond to the keyboard interrupt at the end on my ROM init routine:

mov al, 0xFD ;All but keyboard disabled.
out 0x21, al

I consider it okay to bypass the BIOS POST routine 'guards' in this case, because if you're using the debugger, chances are you have some idea of what you're doing . Timer, keyboard, and floppy interrupts are enabled immediately after ROM scan in the XT BIOS.

To that end, I may have to enable the timer int as well so I can make my debug banner stay on screen for a set time interval; the Seagate hard disk BIOS probably does this with its banner. Interestingly enough, while the Seagate banner will display during ROM init, the Seagate BIOS waits till after POST is done to query a user to format a hard drive if it detects a new one present- I wonder if it waits for the same reason (that it cannot assume the keyboard is available)?

ROM BASIC runs just fine with the modified routine... the FREEDOS image crashes after a number of keystrokes, presumably due to stack overflow.

 

[cr1901]

Does anyone have any advice on how to implement the BNF grammar of MS-DOS debug using x86 assembly (i.e. parsing tokens to make sure they are well formed and storing the results)? I can't really think of an elegant way of doing it aside from parsing one character at a time- I don't know ahead of time how many characters to parse (i.e. near pointer vs far pointer for a DUMP command), so LOOPing is probably inappropriate.

To that end, ChuckG, do you have an equally crufty code snippet to convert BACK to hex from ASCII?

I finally am at the point where I am implementing features. If anyone is interested, I have a version that only implements the 'quit' command as well as the basic parser (invoke on CTRL+ALT). It works fine immediately after ROM init, even during the BIOS POST routine, such as during memory check! It also works during/after MSDOS bootstrap. Unfortunately, whether I can invoke MSDOS DEBUG after my ROM debugger has revectored int 0x01 and 0x03 is a crapshoot for reasons I haven't figured out yet . Sometimes MSDOS DEBUG works fine, others it crashes the machine before even reaching the '-' prompt!

 

[Plasma]

The book "Mastering Turbo Assembler" by Tom Swan has some useful string handling routines and ascii/hex number conversion routines. (It's also a great book in general if you use TASM...better than Borland's manuals.)

http://files.mpoli.fi/unpacked/software/programm/asm/tasmswan.zip/strings.asm
http://files.mpoli.fi/unpacked/software/programm/asm/tasmswan.zip/binasc.asm

 

[Chuck(G)]

No shortcut, really--one of the problems being that you have to range-check argument, which sort of negates any shortcut you might take with a bit a clever code.

@Plasma, if you want to see some interesting code, go up a few posts and see a different idea of BinToHex. No branches.

 

[cr1901]

I'm starting to think that writing the debugger will be harder than writing the BIOS itself (in some respects). Writing a BNF parser is not fun, and I have to do it in assembly language, within 7kB of ROM and 512 bytes of RAM* o.0;. I suppose if I can do it in x86 however, writing one in C in the future (writing an assembler is on my bucket list) should be a cinch.

The debugger, to the extent I can properly emulate the syntax (http://technet.microsoft.com/en-us/library/cc722863.aspx), will accept MS-DOS debug commands, but for now, "assemble" is not supported due to environment constraints.

ChuckG, have you written a compiler/assembler before? If so, any advice on how to write my data structures (favor small code size ...) to store the command tokens into an AST?

*Can be increased if necessary.

 

[Mike Chambers]

Mike Chambers, if you're still following this thread... I figured out why the keyboard interrupt was failing in Fake86 by manually generating an int 0x09 in my software. Since that worked, I realized that interrupts weren't reaching the CPU at all!

I confirmed this by checking the source to the Generic XT BIOS. The Generic XT BIOS does not enable interrupts until after the ROM scan, while in Bochs apparently, interrupts are enabled even during ROM scan. Since I can't assume that the keyboard will be available during ROM scan, I force the 8259 PIC to respond to the keyboard interrupt at the end on my ROM init routine:

mov al, 0xFD ;All but keyboard disabled.
out 0x21, al

I consider it okay to bypass the BIOS POST routine 'guards' in this case, because if you're using the debugger, chances are you have some idea of what you're doing . Timer, keyboard, and floppy interrupts are enabled immediately after ROM scan in the XT BIOS.

To that end, I may have to enable the timer int as well so I can make my debug banner stay on screen for a set time interval; the Seagate hard disk BIOS probably does this with its banner. Interestingly enough, while the Seagate banner will display during ROM init, the Seagate BIOS waits till after POST is done to query a user to format a hard drive if it detects a new one present- I wonder if it waits for the same reason (that it cannot assume the keyboard is available)?

ROM BASIC runs just fine with the modified routine... the FREEDOS image crashes after a number of keystrokes, presumably due to stack overflow.

Good detective work.

Have you tried creating your own stack and switching to it when your routine is entered? Would you even have space for one?

 

[Chuck(G)]

Not for an 8086--the path from the ASCII input in Intel command syntax to the binary representation is pretty long--and is not necessarily isomorphic--i.e., I've run into differences in code between the same syntax in MASM and DEBUG. It's been awhile, but I think the version of DEBUG with PC-DOS 1.x didn't even have an assembler (dis-assembler only).

Good luck.

 

[cr1901]

Good detective work.

Have you tried creating your own stack and switching to it when your routine is entered? Would you even have space for one?

Debugger always uses the caller's stack, without exception... though I suppose I could create one, per my original idea- use 512 bytes of the 1k allocated for stack space, if sp < 0xffff - 512, stack overflow occurred and terminate debugger. I even have (currently unused) variables to switch between stacks.

Debugger is going to need some rewriting/rearranging code to be more manageable- there IS a reason I haven't created a git repo yet, other than me loathing it - code is subject to change at any time!

 

[cr1901]

Not for an 8086--the path from the ASCII input in Intel command syntax to the binary representation is pretty long--and is not necessarily isomorphic--i.e., I've run into differences in code between the same syntax in MASM and DEBUG. It's been awhile, but I think the version of DEBUG with PC-DOS 1.x didn't even have an assembler (dis-assembler only).

Good luck.

Not for an 8086--the path from the ASCII input in Intel command syntax to the binary representation is pretty long--and is not necessarily isomorphic--i.e., I've run into differences in code between the same syntax in MASM and DEBUG. It's been awhile, but I think the version of DEBUG with PC-DOS 1.x didn't even have an assembler (dis-assembler only).

Good luck.

It's gonna be a few days of reading compiler-design literature before any meaningful progress is made...

I asked the wrong question actually. I'm using a BNF grammar to represent MSDOS debug command interpreter itself. The parser reads the command line input, creates an AST of some sort, makes sure the input is well-formed (i.e. creates a valid statement within the grammar), and uses the AST to define variables which are then passed in to a function representing the corresponding command. The problem I'm currently wrestling with is how to convert from the AST to variables to pass into a function, because each possible command has more than one valid syntax.

For example, consider "d" vs "d 0000:0000". The "d" is obvious to handle- I can use a jump table to jump to the 'dump' function in my code if the letter "d" is found. However, the former has no variables to pass into the DEBUG 'dump' function in my code, but the second syntax does- how do I handle the possible combinations of input arguments in a consistent manner (i.e. the function will work correctly as long as a valid set of variables is passed into it- so I DON'T have to create a function for each possible combination of input arguments)? I guess I can use VARARG functions if push comes to shove, but I wonder if I can create some uniform data structure that is passed into each function ("dump", "fill", "in", etc) instead, such that arguments to functions are handled the same way for all commands...

 

[Chuck(G)]

Well, you may be many bytes ahead of the game going to a simple procedural command-unique scanner. So your code for a dump would be "is the first letter D", then "Get an address", then "Get a length or terminal address". The idea is that many DEBUG commands tend to resemble one another.

Here's a little package that I used when I was debugging some DOS code in 1988 and I needed to see what was happening with INT 13. Maybe it'll give you some ideas--honestly, this is the first time that I've seen this code in 20 years, so I can't comment on the quality.

 

[cr1901]

Looks like an interesting little application to play with... I'll try it out on one of my vintage machines.

For those who are interested, this is a patch to compile it on WASM (i.e. a non-DOS assembler) and the corresponding COM file (trivial- just remove the parentheses around each "dw (?)"). Not exactly sure how to use it though .

Was it just the "coding style of the time" to put everything into one source file, Chuck? If so, you won't like my source tree too much if you decide to look (I have 8 source files for the debugger) !

 

[Chuck(G)]

Consider the size of the source file--it's completely self-contained. If this were a 50KB source file, I'd do it differently, but this is just a short file--not worth dividing things up. And after 25 years, I still have everything that I need--no lost included files.

FWIW, every time I tackle a new processor (mostly MCU nowadays) architecture, the first thing that I do is write a debug routine that lets me poke at things. It's a great sanity check to see if I understand the way things work. It also tells you if you understand the assembly syntax.

 

[cr1901]

I would like to add that while I'm not sure if you're supposed to run Chuck's little debugger on a clean boot of DOS, I can verify that attempting to load an EMS driver WILL crash a PC/AT after the debugger program requests a reboot.