• Please review our updated Terms and Rules here

"Fun with BIOS" Programming Thread

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
After a couple of decades, machines and assemblers look pretty much the same. The 8088 looked pretty much like a juiced-up 8085. Consider this: the DAA instruction dates from the Intel 4004.
 

Trixter

Veteran Member
Joined
Aug 31, 2006
Messages
7,413
Location
Chicagoland, Illinois, USA
1. I keep a small MASM-compatible assembler on my IBM PC for on-the-fly debugging, and it doesn't appear to support the memory models (Arrowsoft Assembler).

Why not just use actual MASM on your IBM PC? (I'm guessing you are using MASM 6.x directives/constructs and 6.x doesn't run on the IBM PC)
 

cr1901

Veteran Member
Joined
Dec 28, 2011
Messages
817
Location
NJ
Why not just use actual MASM on your IBM PC? (I'm guessing you are using MASM 6.x directives/constructs and 6.x doesn't run on the IBM PC)

Well actually I try not to use 6.x directives, but for this project I don't have much of a choice (at least EXTERNDEF).

I suppose I'll go fetch a download of 5.1 and 6.0/6.11. Why doesn't 6.0/6.11 work on DOS? I thought it was an MZ executable: http://www.masm32.com/history.htm
 

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
6.11 does work with DOS--there are actually 2 versions of 6.11--a Win32 version and a DOS version that uses the Phar Lap extender. Sadly, that leaves out the 8088. I haven't checked 5.x.

Note that, for the simplified model directives, the segment names are simple TEXTEQU assignments. AFAIK, it's perfectly possible to change them to something else.
 

yuhong

Experienced Member
Joined
Mar 2, 2010
Messages
337
6.11 does work with DOS--there are actually 2 versions of 6.11--a Win32 version and a DOS version that uses the Phar Lap extender. Sadly, that leaves out the 8088. I haven't checked 5.x.
There isn't two versions of 6.10 and 6.11. There is only one version based on Win32 + the DOS extender. You see, it uses the DOSXNT extender which implements a subset of the Win32 API.
 
Last edited:

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
Sigh. Nobody believes me any more.

Okay, the first MASM 6.11 (ML.EXE) that's included in DDKs and usually comes with just ML.ERR and runs only on Win32 systems:

Code:
000000  4D 5A 90 00 03 00 00 00-04 00 00 00 FF FF 00 00  MZ..............
000010  B8 00 00 00 00 00 00 00-40 00 00 00 00 00 00 00  ........@.......
000020  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................
000030  00 00 00 00 00 00 00 00-00 00 00 00 D0 00 00 00  ................
000040  0E 1F BA 0E 00 B4 09 CD-21 B8 01 4C CD 21 54 68  ........!..L.!Th
000050  69 73 20 70 72 6F 67 72-61 6D 20 63 61 6E 6E 6F  is program canno
000060  74 20 62 65 20 72 75 6E-20 69 6E 20 44 4F 53 20  t be run in DOS 
000070  6D 6F 64 65 2E 0D 0D 0A-24 00 00 00 00 00 00 00  mode....$.......
000080  F8 9E AC 02 BC FF C2 51-BC FF C2 51 BC FF C2 51  .......Q...Q...Q
000090  BC FF C3 51 FC FF C2 51-DE E0 D1 51 BF FF C2 51  ...Q...Q...Q...Q)
0000A0  54 E0 C9 51 BD FF C2 51-3F E3 CC 51 A8 FF C2 51  T..Q...Q?..Q...Q

Next, the MASM 6.11 (ML.EXE) that was offered as a standalone product. Runs fine under plain DOS under a 386+ system; windows not needed:

Code:
000000  4D 5A 80 00 33 00 0E 00-08 00 41 02 FF FF 61 06  MZ..3.....A...a.
000010  00 20 00 00 18 21 20 00-40 00 00 00 00 00 00 00  . ...! .@.......
000020  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................
000030  00 00 00 00 00 00 00 00-00 00 00 00 80 66 00 00  .............f..
000040  3C 00 00 00 40 00 00 00-44 00 00 00 48 00 00 00  <...@...D...H...
000050  4C 00 00 00 04 00 ED 04-ED 0D 20 00 6A 0D ED 04  L......... .j...
000060  6E 0D ED 04 18 00 05 06-26 21 20 00 FF 21 20 00  n.......&! ..! .
000070  00 00 E8 04 E3 0D ED 04-00 00 00 00 00 00 00 00  ................
000080  43 6F 70 79 72 69 67 68-74 20 28 43 29 20 31 39  Copyright (C) 19
000090  38 36 2D 31 39 39 31 20-50 68 61 72 20 4C 61 70  86-1991 Phar Lap
0000A0  20 53 6F 66 74 77 61 72-65 2C 20 49 6E 63 2E 00   Software, Inc..
0000B0  00 00 43 35 53 32 53 32-50 4D 00 00 00 00 DC 4F  ..C5S2S2PM.....O

Looks like two different versions to me. The core code may be the same, but they're not the same.
 

yuhong

Experienced Member
Joined
Mar 2, 2010
Messages
337
Sigh. Nobody believes me any more.

Okay, the first MASM 6.11 (ML.EXE) that's included in DDKs and usually comes with just ML.ERR and runs only on Win32 systems:

Code:
000000  4D 5A 90 00 03 00 00 00-04 00 00 00 FF FF 00 00  MZ..............
000010  B8 00 00 00 00 00 00 00-40 00 00 00 00 00 00 00  ........@.......
000020  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................
000030  00 00 00 00 00 00 00 00-00 00 00 00 D0 00 00 00  ................
000040  0E 1F BA 0E 00 B4 09 CD-21 B8 01 4C CD 21 54 68  ........!..L.!Th
000050  69 73 20 70 72 6F 67 72-61 6D 20 63 61 6E 6E 6F  is program canno
000060  74 20 62 65 20 72 75 6E-20 69 6E 20 44 4F 53 20  t be run in DOS 
000070  6D 6F 64 65 2E 0D 0D 0A-24 00 00 00 00 00 00 00  mode....$.......
000080  F8 9E AC 02 BC FF C2 51-BC FF C2 51 BC FF C2 51  .......Q...Q...Q
000090  BC FF C3 51 FC FF C2 51-DE E0 D1 51 BF FF C2 51  ...Q...Q...Q...Q)
0000A0  54 E0 C9 51 BD FF C2 51-3F E3 CC 51 A8 FF C2 51  T..Q...Q?..Q...Q

Next, the MASM 6.11 (ML.EXE) that was offered as a standalone product. Runs fine under plain DOS under a 386+ system; windows not needed:

Code:
000000  4D 5A 80 00 33 00 0E 00-08 00 41 02 FF FF 61 06  MZ..3.....A...a.
000010  00 20 00 00 18 21 20 00-40 00 00 00 00 00 00 00  . ...! .@.......
000020  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00  ................
000030  00 00 00 00 00 00 00 00-00 00 00 00 80 66 00 00  .............f..
000040  3C 00 00 00 40 00 00 00-44 00 00 00 48 00 00 00  <...@...D...H...
000050  4C 00 00 00 04 00 ED 04-ED 0D 20 00 6A 0D ED 04  L......... .j...
000060  6E 0D ED 04 18 00 05 06-26 21 20 00 FF 21 20 00  n.......&! ..! .
000070  00 00 E8 04 E3 0D ED 04-00 00 00 00 00 00 00 00  ................
000080  43 6F 70 79 72 69 67 68-74 20 28 43 29 20 31 39  Copyright (C) 19
000090  38 36 2D 31 39 39 31 20-50 68 61 72 20 4C 61 70  86-1991 Phar Lap
0000A0  20 53 6F 66 74 77 61 72-65 2C 20 49 6E 63 2E 00   Software, Inc..
0000B0  00 00 43 35 53 32 53 32-50 4D 00 00 00 00 DC 4F  ..C5S2S2PM.....O

Looks like two different versions to me. The core code may be the same, but they're not the same.
Ah, I suspect only the DOS stub is different and both are still PE executables. FYI, I checked the Win98 DDK version and it has the Phar Lap DOS extender stub.
 
Last edited:

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
Yup, I think the XP DDK has the the native Win32 executable. I think the core code is the same or very close for both. There are some differences in the ML.ERR files, but I suspect you could just replace the Phar Lap stuff with a DOS stub and run it under Win32.

This from WikiP:

"Versions 6.12 to 6.14 were implemented as patches for version 6.11. These patches changed the type of the binary to native PE format; version 6.11 is the last version of MASM that will run under MS-DOS."
 
Last edited:

yuhong

Experienced Member
Joined
Mar 2, 2010
Messages
337
Yup, I think the XP DDK has the the native Win32 executable. I think the core code is the same or very close for both. There are some differences in the ML.ERR files, but I suspect you could just replace the Phar Lap stuff with a DOS stub and run it under Win32.
You don't need to. Win32 will detect the PE header and run it natively regardless of the DOS stub used. And I think the XP DDK shipped with MASM 7.0, or did it ship with both?
 

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
I was wondering if the later 6.1x or 7.x would run with the Phar Lap stub. It's convenient when I'm running DOSEMU on *nix--I don't have to install a Win32 host.

I'll have to go back and check my DVDs--but I think that even the Vista DDK shipped with 6.mumble, but I'm not certain.
 

yuhong

Experienced Member
Joined
Mar 2, 2010
Messages
337
I was wondering if the later 6.1x or 7.x would run with the Phar Lap stub. It's convenient when I'm running DOSEMU on *nix--I don't have to install a Win32 host.
I think it is mentioned in the README that support ended with 6.12.
 

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
That's not my point--I wonder if sufficient back-compatibility with the Phar Lap extender was maintained to allow for adding it to 6.12+-7.10 and getting a product via a bit of binary hackery that will run under DOS without having to install the Windows API.
 

yuhong

Experienced Member
Joined
Mar 2, 2010
Messages
337
That's not my point--I wonder if sufficient back-compatibility with the Phar Lap extender was maintained to allow for adding it to 6.12+-7.10 and getting a product via a bit of binary hackery that will run under DOS without having to install the Windows API.
Well, the Phar Lap extender dates back to 1993 and implements only a subset of Win32 APIs. I don't think it is likely.
 

cr1901

Veteran Member
Joined
Dec 28, 2011
Messages
817
Location
NJ
Apologies for the lack of progress this week... I'm still trying to organize my build, but I'm getting closer to making building this software MUCH easier on me and people interested. Also, real life such as cleaning the house (which I sadly enough enjoy when I'm in the mood- just rarely happens), is also getting in the way.

Below is a list of prerequisites for building BIOS when I release a minimal version:
  • SCons version 2.2 or above
  • Python version 2.6 or above
  • OpenWATCOM C Compiler and assembler (either Windows or Linux should be fine)
  • M4 (needed to generate linker scripts... on Windows, you need to tell SCons where it is in a build settings file)
  • Python requests library (Optional: Needed to send test code to Reenigne's server)... built in Python HTTP library is abysmal.
 

cr1901

Veteran Member
Joined
Dec 28, 2011
Messages
817
Location
NJ
Version 0.9 or so of my Python script to upload to reenigne's server is complete... still a lot of debugging code in it though. I'm just putting it up because it works for those curious.

You can upload and receive text output from programs. Because requests is blocking, however, canceling programs is not currently possible. If I make the program multithreaded, I can work around this (one thread polls the keyboard, the other polls the network), but this is not something I'm prepared to do right at this moment.

I tried using a SIGINT for the keyboard, but that didn't work well... although Windows responds to them, the handler's static variables aren't updated. Ah well.

Reenigne, if you are reading this, feel free to upload this version, but I will replace it with a better version in the next few days.

Sample output:
Code:
submit.py  [email]email@example.com[/email] hello_bin.bin
The XT Server has received your file.
Your program is starting
Upload complete.
OK
Hello, World!
Program ended normally.
This concludes your XT server session.

Python code:
Code:
#!/usr/bin/python

import sys, requests, re, signal
from HTMLParser import HTMLParser

debug = False

#Probably overkill for this...
"""def handler(signum, frame):
	handler.cancel_vars = None
	print 'User request to cancel process...', signum
	if handler.cancel_vars is None:
		print 'Nothing to cancel!'
	else:
		print 'Cancel ID is' + handler.cancel_string
	#raise IOError("Couldn't open device!")"""


class MyHTMLParser(HTMLParser):
	def __init__(self):
		self.curr_tag = None
		self.desired_tag = False
		self.program_output = None
		self.cancel_string = None
		HTMLParser.__init__(self)
		
	def feed(self, data):
		HTMLParser.feed(self, data)
		#return self.program_output.group(1)
		
	def handle_starttag(self, tag, attrs):
		self.curr_tag = tag
		#if self.curr_tag in ['h1', 'p', 'pre']:
		if self.curr_tag in ['p', 'pre', 'input']:
			if debug:
				print "Encountered desired tag:", tag
			self.desired_tag = True
			
			if self.curr_tag == 'input':
				#handler.cancel_vars = attrs
				if debug:
					print 'PARSE-DEBUG: The Cancel ID comes from here...'
			
		elif debug:
			print "Encountered a start tag:", tag
			
	def handle_endtag(self, tag):
		if self.desired_tag:
			if debug:
				print "Encountered a desired end tag :", tag
			self.desired_tag = False
		elif debug:
			print "Encountered an end tag :", tag
	def handle_data(self, data):
		if self.desired_tag:
			if debug:
				print "Encountered some desired data  :", data
			else:
				if debug:
					print repr(data)
				else:
					print data
			if self.curr_tag == 'pre':
				if debug:
					print 'PARSE-DEBUG: This is where the XT output comes from...'
				#http://stackoverflow.com/questions/3368969/find-string-between-two-substrings	
				#self.program_output = re.search('OK\n\t\t(.*)\n\t\tProgram ended', data)
		elif debug:
			print "Encountered some data  :", data


def submit(email, binary):
	
	

# Set the signal handler and a 5-second alarm
	#signal.signal(signal.SIGINT, handler)
	
	data = """<!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Strict//EN' 'http://www.w3.org/TR/x
		html1/DTD/xhtml1-strict.dtd'>
		<html xmlns='http://www.w3.org/1999/xhtml' dir='ltr' lang='en-US'>
		<head>
		<meta http-equiv='Content-Type' content='text/html; charset=UTF-8' />
		<title>XT Server - result</title>
		</head>
		<body><h1>XT Server</h1>
		<p>The XT Server has received your file.</p>
		<form action='http://reenigne.dyndns.org/cgi-bin/xtcancel.exe' method='post'>
		<input type='hidden' name='secret' value='KLoRaPxQgzf_H5yl'/>
		<button type='submit'>Cancel</button>
		</form>
		<pre>Your program is starting
		Upload complete.
		OK
		(#??#??#??#
		Program ended normally.</pre>
		<p>This concludes your XT server session.</p>
		</body>
		</html>"""
	
	parser = MyHTMLParser()
	
	email_data = {'email': email}
	file_upload = {'binary': open(binary, 'rb')}
	
	r = requests.post('http://reenigne.dyndns.org/cgi-bin/xtserver.exe', data=email_data, files=file_upload, stream=True)
	
	for line in r.iter_lines():
		#print line
	#http://search.cpan.org/~gaas/HTTP-Message-6.06/lib/HTTP/Status.pm
	#http://www.python-requests.org/en/latest/user/advanced/#streaming-requests
		if (r.status_code < 200) and (r.status_code > 208):
			#This doesn't get executed for some reason...
			print 'Error: ' + r.status_code
			print r.headers
			print r.text.encode(sys.stdout.encoding, errors='replace')
			break
		else:
			#print line.encode(sys.stdout.encoding, errors='replace')
			parser.feed(line.encode(sys.stdout.encoding, errors='replace'))
	

if __name__ == '__main__':
	if ('-h' in sys.argv) or len(sys.argv) != 3:
		print 'Syntax: submit.py [email] [binary]'
	else:
		submit(sys.argv[1], sys.argv[2])
 

mmruzek

Experienced Member
Joined
Sep 28, 2011
Messages
213
Location
Michigan, USA
Here's a dirty little trick that I've grown accustomed to when making ROM images to burn:

Code:
	.model tiny,c			

ROM_PAGES equ 4
ROM_BYTES equ ROM_PAGES*512

video_seg    segment at 0b800h
video_seg     ends

	.code
code_start:

	dw 0AA55h
	db ROM_PAGES
 
main	proc
	mov ax, seg video_seg
	mov ds, ax
	mov si, 00h
	mov cx, 1000h
	mov ax, 0AA55h
loop_vid:
	mov word ptr ds:[si], ax
	inc si
	inc si
	loop loop_vid

	int 3
main endp

	db     (ROM_BYTES - ($-code_start) -32) dup (0) ; filler

; The following is entered with a G= in DEBUG.	It computes and stores
; the checksum. It's always located 20h bytes before the end of the ROM.
 
Checkit		proc
	mov	si,100h			; where files get loaded in DEBUG
	mov	di,ROM_BYTES-1
	xor	ax,ax
Check2:
	add	al,cs:[si]
	inc	si
	dec	di
	jnz	Check2
	neg	al
	mov	cs:[si],al	    ; stores result in "Checksum"
	int	3
Checkit	       endp


	db     (ROM_BYTES - ($-code_start) -1) dup (0)
Checksum db 0

	end

(Note that I've changed the segment to B800 and the length to 4K to give you a nice green screen. At any rate, run DEBUG on the .COM file and G=8E0 into the checksum routine. It'll compute and store the checksum and exit to DEBUG. Then issue a -W command to DEBUG and then a -Q. Presto--you have your checksummed image.

Here is a photo of what this code looks like on the screen when I compiled, linked and burnt it to a ROM located at F400 and displayed using a VGA card and monitor.

Also, to go with the discussion about adding a debugger or monitor in the empty ROM space, I just happened to find the original source code for DEBUG written for the S100 bus before it was picked up by IBM. This was courtesy of 'STARMAN' who pointed out that Paterson (Original Author) actually has this posted on his website! Really interesting stuff. From STARMANS's email message...

"I doubt it will be of much help to you, but you can find the source code in a PDF downloadable dirrctly from Tim's own web site here:

http://www.patersontech.com/dos/docs/Mon_86_1.4a.pdf

Remember, this was for an earlier s-100 bus PC; not the one that was copyrighted by IBM and Microsoft."

Michael
 

Attachments

  • rompix.jpg
    rompix.jpg
    34 KB · Views: 1

cr1901

Veteran Member
Joined
Dec 28, 2011
Messages
817
Location
NJ
Well that was certainly helpful... looking at the source tells me that I was more-or-less on the right track.

This is certainly a challenge, writing a ROMable debugger... I can't assume an OS is installed, and even when it is, I can't assume any memory other than possibly the top of memory is available...

Unlike DEBUG, since this can be invoked at any time, and all registers have to be preserved, I have to recalculate the position of the extra segment (which I use for RAM variables... ds points to constant data) and stack segment.

The minimum that the ROM debugger must do during ROM option boot is:
  • Hook the IVT. This part should be easy.
  • Find a place for a stack/extra segment, allocate memory (pretty damn difficult to do without an OS XD), and find some way to be able to return to the extra segment when the debugger is invoked, either by int 0x03, a keypress combination (int 0x09), from the command line (int 0x21?), or even from MS-DOS debug itself. This isn't trivial.

I can alter the invocation depending on which interrupt the debugger is called from. The only way I've figured out how to reliably calculate the extra/stack segment each time, without any dependence on register values is to check int 0x12 for the memory size and make sure that each time the debugger is invoked is to do the following:

  1. Push some scratch registers (ax, bx, cx) and es onto the current stack. This assumes the current SP is valid
  2. Invoke int 0x12, return value in ax, remaining registers preserved (?)
  3. Point es to 1024 bytes below the top of user memory (mov cl, 10... shl ax, cl.... sub ax, 0x40... mov es, ax).
  4. Using es:[bx] indirect addressing*, store the old es (in my case, popped into ax) at the first location.
  5. Pop the scratch registers
  6. Store the current ss sp and bp, from the instruction before the debugger was invoked into the next few memory locations
  7. Set ss to es and sp to 0x400 (top of user memory).
  8. Using es segment and the preserved registers, push all the registers associated with the old task onto the new stack

As long as I pop the registers in the correct order from the new stack, restoring program state should be easier... just make sure flags is at bottom of new stack, followed by cs:ip of old routine.

Because I cannot guarantee any registers will have correct values, I have to do this each time the debugger is invoked. This seems to be a bit unwieldy however, and perhaps there is a more appropriate way to do this for the 8088? Does anyone have any feedback on either how to make this routine better, have an alternate means to perform the task switch they'd like to share, or point out any way this method could possibly backfire on me? Thanks in advance!

*es:[label] should work... just haven't updated the code.
 
Last edited:

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
Well, remember that the area around address 500-5FF is used by BASIC, so it's probably safe to use for the debugger.

As far as memory size goes, you don't need to call INT 12H--just read location 00413-00414 for memory size in K.
 

cr1901

Veteran Member
Joined
Dec 28, 2011
Messages
817
Location
NJ
Well, remember that the area around address 500-5FF is used by BASIC, so it's probably safe to use for the debugger.

As far as memory size goes, you don't need to call INT 12H--just read location 00413-00414 for memory size in K.

The ROM debugger can be called at any time with a specific keypress- in theory, this includes BASIC. Is there an API call for allocating memory within ROM BASIC? I'm willing to drop this constraint if necessary, or at least forbid trying to call the debugger if we are "inBASIC" (akin to the inDOS and inCriticalSection flags). BASIC would be a good scratch area for the stack segment and pointer while I switch stacks.

As for the second point, I'd like the ROM debugger to work on not only 100% compatibles but those which still contain a BIOS, like the Tandy 1000 and especially 2000 (which still has a BIOS, but isn't PC-compatible). This is why my debugger uses the INT 0x10 and 0x12 calls instead of using the hardware directly.

As someone who has written 8088 code professionally, how would you personally perform such a context switch (basically what I'm doing, except it's only between two tasks... the ROM debugger, and everything else!)? I've never had a class in OS design... I just know that all the registers must be preserved and that the 8088 is guaranteed to finish executing the current instruction before servicing an interrupt, which makes it easier to save the processor state.

I think my own ROM debugger will become the debugger I use for my own BIOS... and perhaps the debugger for my own debugger :p.
 

Chuck(G)

25k Member
Joined
Jan 11, 2007
Messages
40,847
Location
Pacific Northwest, USA
Exactly what you want your debugger to do matters a lot.

Do you want to set breakpoints? If so, then you can't set breakpoints in ROM. Do you want to be able to breakpoint ISRs? That might mean that you have to handle all your own I/O outside of the BIOS. There are a lot of games and software protection schemes that disable INT 3 by plugging it or move/modify code before executing it. You might think that it's safe to grab the upper 1K or so of memory for debugger use, but there are some protected games that do their own (destructive) memory checking.

Not trying to be pedantic, but I'd like to know exactly what capabilities that you plan on implementing.
 
Top