• Please review our updated Terms and Rules here
  • Exhibitor application for VCF West 2022 is now open! If you are interested in exhibiting, please fill out the form here.
  • Here are the results of the VCF East 2022 Post Event Survey: Survey Results

Network-based boot disk driver for 8088+ PCs -- Would anyone find this useful?

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
Would anyone else find a piece of software like this useful? I've been experimenting with the idea of an int 13h network disk driver layer via UDP and am wondering if I'm the only one who would find it useful? I have more old PC's than I have hard drive controllers, plus there are old laptops that only have floppy drives but can take a parallel port Ethernet adapter like a Xircom.

Like, the concept I have is this:

1. Boot a DOS disk that contains the loader, a packet driver, and a small driver payload written in asm.
2. Reserve a little RAM at the very top of conventional memory.
3. Load the packet driver and payload into the top of RAM.
4. Update BIOS data area with new amount of available memory.
5. Transfer CPU control to driver payload.
6. Driver destroys DOS by overwriting int 21h vector, containing just enough emulated functionality to make your typical packet driver work.
7. Runs packet driver and registers a packet receive callback with it.
8. Hook int 13h and handle calls with DL=80h (drive select hard drive 1), passing calls meant for other drive numbers to the original int 13h handler.
9. Driver bootstraps remote hard disk image by reading the boot sector into 0000:7C00 and transferring CPU control there.
10. Disk image loads whatever OS is on it!

At this point, the payload's int 21h emulation isn't needed anymore so the remotely booted DOS can safely overwrite that vector again.

Then it can talk with a Windows/Linux/whatever server hosting a hard disk image file via a simple UDP protocol.

I've got this working already in preliminary form under DOSBox-X, and it's able to boot MS-DOS over the network and run software/games. No sector write support yet, but easy enough to do. I haven't done any testing beyond that yet. Need to add more features, make the code prettier and do a lot more testing.

Can anybody think of any possible pitfalls? I'm sure there are many. For example, it seems to me like having interrupts enabled during sector reads may not be a great idea, but I need it to receive packets and for timing packet resends lol. Is that a problem? So far it seems happy enough. I'm not sure what other int 13h drivers do regarding that.

But the nice thing about this is that it's at the int 13h level! Theoretically, the only software that wouldn't work with this type of driver would be specialized utilities that want to talk directly to a specific kind of hard disk controller or programs that truly need a full 640 KB of RAM.

This could also be made to be an add-in card with a ROM version of it, though you'd also need to include a little SRAM if you don't want to steal a bit of conventional memory. Or it could be loaded from a floppy disk into VGA memory at segment A000 and it would work, keeping 640 KB conventional available... as long as you don't run any software that actually uses VGA. ;)
 
Last edited:

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
This is working now, by the way.

https://imgur.com/a/SjodI4p

That's a V20 board with no physical hard disk booted over the LAN from a hard disk image, playing Ultima 6.

I ditched the DOS loader route and just made my own floppy boot sector to cram the driver into the top of conventional memory.

My 8086 assembly skills are pretty rusty after ~8 years of non-use, but it's coming back to me. Need to crush a bug or two and add some features, then it might be ready if anyone else is interested in such a driver. I could definitely use testers!

Feature-wise, I'm thinking it would be nice to be able to tell it to attach the network image as the second hard drive, so it would be able to play nicely with an existing disk. I have a few other ideas as well.
 
Last edited:

TheDrip

Experienced Member
Joined
May 13, 2019
Messages
345
Location
San Diego CA
Would it be possible to load this as a rom image, akin to xtide? So a machine could be booted with nothing more than a vga card and a NIC? I suppose there would be some amount of trickery involved to get the correct packet driver on that image
 

whartung

Veteran Member
Joined
Apr 23, 2020
Messages
553
Wasn't this already a thing? Booting over the network on diskless workstations? Even in to something like DOS? I seem to recall network cards with special PROMs in them.

Mind, I mean, it sounds interesting to me, but I don't have any legacy DOS stuff myself. But, just saying, I thought this was a something that could be done already from back in the day.

Yes, here it is. It used TFTP.

https://en.wikipedia.org/wiki/Preboot_Execution_Environment
 

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
Would it be possible to load this as a rom image, akin to xtide? So a machine could be booted with nothing more than a vga card and a NIC? I suppose there would be some amount of trickery involved to get the correct packet driver on that image

What gets tricky about that is that the packet driver will expect the segment it's executing in to be writable. My driver layer needs a small amount of RAM, too. If one were to create a custom ISA card with both ROM and RAM loaded above 640 KB, this could be done without eating any conventional memory. It would be a very simple card. The ROM code would need to copy the packet driver to the RAM at boot, and then start executing it from there.

Thankfully, a lot of common packet drivers are less than or around 10 KB, and my driver uses 4 KB so it's not stealing a huge portion of RAM even using the floppy boot method. I think I can still trim some fat, too.


Wasn't this already a thing? Booting over the network on diskless workstations? Even in to something like DOS? I seem to recall network cards with special PROMs in them.

Mind, I mean, it sounds interesting to me, but I don't have any legacy DOS stuff myself. But, just saying, I thought this was a something that could be done already from back in the day.

Yes, here it is. It used TFTP.

https://en.wikipedia.org/wiki/Preboo...on_Environment

Indeed, network boot has been possible in one form or another for a very long time, yeah. It definitely isn't a new concept, but I believe it would get complicated to PXE boot DOS on an 8088-type machine from a software perspective. I don't think it's common to find PXE boot ROMs in old ISA NICs, and then I'm not even sure where you'd begin setting up a complete server stack, or if there even is a program already that will let you just serve up a raw disk image as a network drive to these machines. Something that you could turn around and boot in an emulator or mount easily on a modern PC to make changes to. You'd also need a corresponding loader that would be retrieved from TFTP.

I was just trying to come up with a quick all-in-one way to do this at the int 13h BIOS level that didn't need special obscure chips or days of research and hunting down old software, if it even exists. You'd just need a floppy and a packet driver. Doing this at int 13h is great from a compatibility perspective too. In theory it should be able to boot alternative OSes as well, like say, MINIX 2. You should be able to bootstrap any disk image, and as long as the OS on it talks 13h, you should be good to go.

Maybe there's even some similar software out there already and I haven't heard of it, but even in that case, it can't hurt to have options. :)
 
Last edited:

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
OK this is working and it seems to be pretty stable, but I'm getting terrible speeds. I need to figure out what the deal is with this. Probably something stupid/simple...

This is a NEC V20 @ 4.77 MHz with a 3C509B NIC. I know that's a slow PC, but it seems like it should be able to do considerably better than this.

e6UIzs0.jpg - Click image for larger version  Name:	e6UIzs0.jpg Views:	0 Size:	81.0 KB ID:	1217010
 
Last edited:

mbbrutman

Associate Cat Herder
Staff member
Joined
May 3, 2003
Messages
6,236
So a few things to keep in mind or look for while figuring out the performance problem:
  • UDP is lighter than TCP, but the checksum process is still expensive. Writing that code in assembler and loop unrolling helps, but also remember that for UDP it is optional. I'd always have it turned on, but for finding your absolute best speed you can disable checksum computations on both the send and receive paths.
  • TCP throughput is usually based on full sized packets with a 1460 byte payload. I'm assuming you are doing 512 byte payloads to match the standard DOS sector size, so your packet processing overhead is higher than it would be with larger packets.
  • Interrupts enabled during packet processing is generally fine. You don't want to screw the system timer up. You disable interrupts only in limited situations, like handing a pointer to the packet driver or adjusting the head or tail of a buffer.
  • Doing a small amount of sector caching would make a big difference in real world workloads. DOS does that via BUFFERS. You can do something similar, but you do run the risk of data loss if something is not flushed in a reasonable amount of time.
  • Are you having packet drops? Do you have explicit acknowledge packets after each operation? You need these to detect when a packet has dropped.
I really like the idea of an INT 21 driver over UDP. I think it would just be fine as a regular network drive letter though; the boot process trickery is just extra work unless you really want to turn it into a ROM extension.
 

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
Hey Mike.

Yes, I actually didn't do the UDP checksum, only the IP header checksum. I'm calculating a simple checksum still while copying the data to/from the packet buffer. For example, here's when it puts the data in a "write sector(s)" packet:

Code:
  mov cksum,0
  xor ah,ah
copy_write:
  mov al,es:[di]
  mov [si],al
  add cksum,ax
  inc si
  inc di
  loop copy_write
  mov cx,cksum
  mov word ptr [si],cx

Excuse my rusty assembly skills!

I also combine two sectors in one packet when possible to help throughput.

I do have acknowledgements. I also have a retry timer where it sends the packet again after a short time without getting a reply. I keep a "packet ID" value and put at the beginning of the data in both request and reply so I make sure I'm only taking action on the reply I'm currently waiting for, just in case a retry causes a duplicate response after I've already moved in. The ID increments for each request.

I think the problem is that my code is taking a long time to notice that I've gotten a reply somehow, so I'm going to be digging into this. It's either that, or I'm hanging out in my wait loop even after I've already processed the packet, though I'm not sure why it would be doing it. I'm basing this off some Wireshark captures. The server replies basically instantly, but the next request from the client takes some time to happen.

I thought about an int 21h driver, but I went with int 13h for a few reasons. It doesn't care what the OS is. One needn't run DOS! It's also a very simple set of BIOS calls to implement, and I'm already pretty familiar with it.

Additionally, I ditched the idea of loading the driver from DOS. Now there's a utility that creates a boot disk for you from your packet driver, my driver payload, and a custom bootloader.

Sector caching would be a great idea, but I'm trying to minimize the amount of memory I have to steal, and if I can solve this issue it probably won't really be very necessary.
 

mbbrutman

Associate Cat Herder
Staff member
Joined
May 3, 2003
Messages
6,236
So when the machine receives a packet the packet driver should get a hardware interrupt so that it can copy the contents from whatever buffer the hardware is sitting in. Under that interrupt it will then do the first call to your receiver routine to report the size of the incoming buffer and ask for a place to store it. Your receiver routine should be minimal, so then the packet driver copies the data, and calls again to tell you it completed.

If you have interrupts disabled that's going to be delayed or it is not going to happen.

With your transaction id, are you noticing dropped packets that you are having to retry for? The packet driver also keeps stats on how many packets it drops on the floor because buffer space wasn't provided fast enough. (I find on a noisy network the interrupt load can really hurt performance.)
 

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
It actually does often retry after the first, which is where the throughput is getting crushed. It doesn't retry until the next system timer tick. That's a lot of waiting.

Funny enough though, the packet driver only reports one dropped packet out of 572 received when I tested that. So it's not that. This points to my code ignoring the packet for some reason, after receiving it. That gives me something to go on!

If I give the packet driver a 0000:0000 for the buffer destination on the first receiver call, telling it I don't want it, it would count that as a dropped packet, right?
 
Last edited:

mbbrutman

Associate Cat Herder
Staff member
Joined
May 3, 2003
Messages
6,236
According to the spec, if you give it a null pointer for the receiver buffer the packet driver will throw the packet away and not make the second call. However, how it gets counted is up to the packet driver writer. Their definition of "dropped packet" might be that they didn't have enough buffer space available for incoming packets from the wire, as opposed to you decided to toss a packet.

Are you ever giving the packet driver a null pointer? If so, whatever it had is going to get flushed.
 

Mike Chambers

Veteran Member
Joined
Sep 2, 2006
Messages
2,621
That's what I was expecting it to do. I give it a null pointer either if the packet is too large for my buffer, or I'm not currently expecting a packet.
 
Top