Comparison between mTCP's NetDrive and EtherDFS

[alejack12001]

I just wanted find out if any of the forum members have deployed M. Brutman's mTCP NetDrive and possibly compared it to EtherDFS. I have been using the EtherDFS for about 2 years and have had a number of problems with it's use. First of all. my old DOS systems refuse to upload files to the DFS server unless the drive isn't formatted as FAT16. So, I have to keep a USB drive mounted to the the Raspberry Pi and have it formatted to FAT16 to upload files. The second annoying problem is that files greater than 64KB stall and display access denied. I can retry to send again and at times the file will go through. So, I was just trying to find out anyone has had success with the NetDrive in terms of the stated problems with DFS.

The second question, could this NetDrive be used with the PXE to remote boot a DOS machine?

 

[mbbrutman]

EtherDFS and mTCP NetDrive are two very different things.

• EtherDFS works at the filesystem level, so your Linux machine and DOS machine both see a set of files, similar to what two Windows machines do when using file sharing.
• mTCP NetDrive is a block device driver that provides raw storage to DOS. It's like adding more storage by putting a removable hard drive in the machine, except that it happens over the network. Only one machine can use that storage at a time, so while the DOS machine is connected to the storage the Windows or Linux server should not touch it. (There is an exception; if the storage is marked read-only then it's safe for any number of machines to look at it at the same time.) DOS takes the storage and manages the FAT12 or FAT16 filesystem on top of it.

Other differences:

• The NetDrive server runs on Windows or Linux (x86 and Arm) and does not require special permissions.
• NetDrive can be used across wireless connections, or even across the Internet because it uses UDP.
• NetDrive has a single device driver that works on all versions of DOS that have installable device drivers. (Versions 2.0 and up.)
• Netdrive allows you to add up to 24 drives ... Yes, if you boot DOS 5 from a floppy-only system you can add 48GB of storage! (Drive letters C through Z, 2GB each.)
• NetDrive supports write with verify better than DOS does on real hardware. When DOS verify is on it will only verify that what was written is readable, not that it is what you wrote. NetDrive actually does a full read-back of the written data and makes sure the bytes are the same, and that only slows things down about 10%.

If you can live with block level storage instead of filesystem level sharing I think that mTCP Netdrive will work well for you. While I can't guarantee other people's hardware, I'm pretty sure the software is solid and you won't have strange glitches. (All of the problems I have seen so far are due to crap quality packet drivers. I've been beating the crap out of the code for months and I haven't found corruption yet.)

I'm working on an undo feature for NetDrive now. Right from DOS you will be able to send a command to the server saying "remember this point", go do something you might regret, decide to undo it, and your drive will be restored to that checkpoint. You don't even have to disconnect from the server to do it.


-Mike

 

[mateuszviste]

my old DOS systems refuse to upload files to the DFS server unless the drive isn't formatted as FAT16

Are you sure you use the original (my) version of EtherDFS? There is a bunch of hacked versions floating around that may or may not work properly.
EtherDFS should not refuse to upload anything, but using a case-sensitive filesystem may potentially lead to seemingly duplicated files if you have any lower-case preexisting files on your drive.

So, I have to keep a USB drive mounted to the the Raspberry Pi and have it formatted to FAT16 to upload files.

It would be much easier to use a loop-mounted FAT image instead. I have documented this years ago in ethersrv-linux.txt.

The second annoying problem is that files greater than 64KB stall and display access denied. I can retry to send again and at times the file will go through.

Sounds like a network transmission issue, but it would be nice to debug this further to understand what is actually happening there.

trying to find out anyone has had success with the NetDrive in terms of the stated problems with DFS.

As Michael stated already, both solutions are hardly comparable. EtherDFS is a filesystem-over-network (NAS-like) while NetDrive appears to be a disk-over-network (SAN-like), similar (in concept) to EthFlop. NetDrive seems to be more feature-packed, though and not limited to floppy-like disks. It also uses UDP which should make it work in more scenarios, although this also makes it incompatible with a network stack running at the same time.

 

[mbbrutman]

It also uses UDP which should make it work in more scenarios, although this also makes it incompatible with a network stack running at the same time.

While that's true today, I expect that to be a temporary problem. Later versions of the device driver will continue to use UDP but will also allow for other packet driver TCP/IP apps to run at the same time.

 

[mateuszviste]

Later versions of the device driver will continue to use UDP but will also allow for other packet driver TCP/IP apps to run at the same time.

Interesting. Do you plan to add some mTCP-specific "protocol dispatcher" overlay on top of the packet driver, so mTCP application would register (bind to) specific UDP/TCP ports and query ARP through a (new) dedicated API instead of talking to the packet driver, or do you have another trick in mind? It's a genuine question, as I was wondering about the same thing a couple of years ago, but failed to find a solution that would work with WatTCP / Watt-32 applications.

 

[mbbrutman]

I'm planning on just having the device driver pretend that it's a packet driver for the higher level programs. The UDP for the NetDrive specific port won't be passed through, but all of the other packets can be passed through. I don't think the other programs will be able to tell except for two small things:

• Performance will be slightly different because of the extra call overhead.
• ARP has to be handled by the device driver. But it can also be passed through to the higher level program too. (A redundant response won't be great, but it won't hurt anything.)

 

[alejack12001]

Are you sure you use the original (my) version of EtherDFS? There is a bunch of hacked versions floating around that may or may not work properly.
EtherDFS should not refuse to upload anything, but using a case-sensitive filesystem may potentially lead to seemingly duplicated files if you have any lower-case preexisting files on your drive.



It would be much easier to use a loop-mounted FAT image instead. I have documented this years ago in ethersrv-linux.txt.



Sounds like a network transmission issue, but it would be nice to debug this further to understand what is actually happening there.



As Michael stated already, both solutions are hardly comparable. EtherDFS is a filesystem-over-network (NAS-like) while NetDrive appears to be a disk-over-network (SAN-like), similar (in concept) to EthFlop. NetDrive seems to be more feature-packed, though and not limited to floppy-like disks. It also uses UDP which should make it work in more scenarios, although this also makes it incompatible with a network stack running at the same time.

Mateuszviste: I wouldn't know if I am using a fake or real version of DFS. I downloaded DFS from sourceforge's website in 2022. Al's Geeklab from YouTube was the introduction to the EtherDFS. I believe Al used a Raspberry Pi with an SD card and created a 1-GB image file using fallocate command followed by make file system (mkfs) for msdos to the image created. Al then mounted that files system using a -o with a loop to the image file and a mount point.

My server is a little different since I am using a Raspberry Pi 3B+ with 16-GB SD card using Debian buckeye with a 1-GB USB thumb drive housed to the USB 3.0 port. The USB thumb drive was formatted on the Pi for plain FAT (FAT16) and mounted the drive as FAT to the Pi. For the most part the EtherDFS recognizes the mounted drive as FAT drive. The vintage machine is a IBM PC XT/286 running MSDOS 6.22. I have a Artisoft Ethernet AE-2/C 16-bit ISA card with AUI Ethernet adapter attached to my wired network. The network is 100-Mbps, which supports the Pi and the IBM. I loaded the DFS part and execute the server first on the Pi and the DFS on the PC-XT with X drive as the place for the file storage. If the file being sent is less than 64-KB then the file will upload fairly quickly. However, if a 100-200-KB file will fail quoting access denied. I changed permissions for the USB thumb drive but that hasn’t corrected the problem. I can restart the file and it may load without incident and sometimes it will repeat the same error. Seems like a buffer is full someplace.

As for Mr Brutman’s NetDrive, I first found out that the binary was for a 64-bit platform. My Pi 3B+ is 32-bit so I switched to a Pi 4 which is running Debian 11, a 64-bit version. The netdrive binary will not execute and gives an error looking for a version of gnu/lib.so.6 version GLIBC_2.34 not found. Is this gcc special as I have gcc version 7 on the Pi? I tried the “netdrive connect brutman.com:2002 bigdisk.dsk d:netdrive”. I tried other commands include a help, same error.

 

[mbbrutman]

Well, that's disappointing - Go usually tries to statically link everything but in this case it's generating a dynamically linked executable that requires libc.

I'm on a new RaspPi 5 running Debian 12.4. gcc -v reports "gcc version 12.2.0 (Debian 12.2.0-14)".

I'll try to get my hands on an older RaspPi to compile with. Eventually you'll be able to compile on whatever you have, but I'm making a lot of changes to the code so I haven't released source yet.

 

[alejack12001]

I have ordered the Pi 5 and will get the Debian software. Please continue with your efforts to to develop the netDrive. When the Pi 5 is operational then I'll let you know if the netDrive works. Thank you for replying.

 

[acgs]

I seriously doubt the problem is caused by a Pi 4 and requires a Pi 5. I suggest putting Debian 12.4 (the one @mbbrutman is using) on a new sd card and seeing if that solves the problem. The Pi 4 should run the new OS version fine (it's not that slow), but the problem you're experiencing appears to be caused by a missing version of the library. If you upgrade to the new OS, the newer/required version of libc should be installed along with it and you'd be good to go.

 

[mbbrutman]

Alejack12001,

The NetDrive for ARM I made available looks like it needs things from GLIBC 2.34.

I remembered I had a Pi Zero 2W in the house which is running Debian 11.8. I installed the Go compiler there, rebuilt it and that binary only requires GLIBC 2.17, so it should be fine if you try it. Please download it from www.brutman.com/mTCP/download/netdrive_debian_11_arm.zip and I'd like to know how it works for you.

As an aside, the Pi Zero 2W is not a fast machine, but it runs NetDrive just fine. (I was getting read and write speeds of 400KB/sec from it when testing with DOS in a virtual machine on my big Xeon desktop. NetDrive is also only consuming 6MB of RAM, so it easily fits on the machine. For $15 it's a deal.


-Mike

 

[alejack12001]

I seriously doubt the problem is caused by a Pi 4 and requires a Pi 5. I suggest putting Debian 12.4 (the one @mbbrutman is using) on a new sd card and seeing if that solves the problem. The Pi 4 should run the new OS version fine (it's not that slow), but the problem you're experiencing appears to be caused by a missing version of the library. If you upgrade to the new OS, the newer/required version of libc should be installed along with it and you'd be good to go.

Okay worth a try. I'll keep you informed.

 

[alejack12001]

Alejack12001,

The NetDrive for ARM I made available looks like it needs things from GLIBC 2.34.

I remembered I had a Pi Zero 2W in the house which is running Debian 11.8. I installed the Go compiler there, rebuilt it and that binary only requires GLIBC 2.17, so it should be fine if you try it. Please download it from www.brutman.com/mTCP/download/netdrive_debian_11_arm.zip and I'd like to know how it works for you.

As an aside, the Pi Zero 2W is not a fast machine, but it runs NetDrive just fine. (I was getting read and write speeds of 400KB/sec from it when testing with DOS in a virtual machine on my big Xeon desktop. NetDrive is also only consuming 6MB of RAM, so it easily fits on the machine. For $15 it's a deal.


-Mike

Thank you for the link. I'll give this a try before trying upgrade to Debian 12.4.

 

[alejack12001]

Alejack12001,

The NetDrive for ARM I made available looks like it needs things from GLIBC 2.34.

I remembered I had a Pi Zero 2W in the house which is running Debian 11.8. I installed the Go compiler there, rebuilt it and that binary only requires GLIBC 2.17, so it should be fine if you try it. Please download it from www.brutman.com/mTCP/download/netdrive_debian_11_arm.zip and I'd like to know how it works for you.

As an aside, the Pi Zero 2W is not a fast machine, but it runs NetDrive just fine. (I was getting read and write speeds of 400KB/sec from it when testing with DOS in a virtual machine on my big Xeon desktop. NetDrive is also only consuming 6MB of RAM, so it easily fits on the machine. For $15 it's a deal.


-Mike

Mike: I just wanted to report that the binary you provided, works. Thank you.

 

[mbbrutman]

Thanks for testing it. I'll replace the current download with that binary so that more people can use it. And I wish I had thought of it earlier, but I totally forgot that the Pi Zero 2 is on an older release. It's also terribly slow for building code. ;-0 (It can run it reasonably well though.)

 

[mateuszviste]

I downloaded DFS from sourceforge's website in 2022.

Then it is EtherDFS v0.8.2. Have you tried upgrading to v0.8.3? (released in June, 2023)

If the file being sent is less than 64-KB then the file will upload fairly quickly. However, if a 100-200-KB file will fail quoting access denied. I changed permissions for the USB thumb drive but that hasn’t corrected the problem. I can restart the file and it may load without incident and sometimes it will repeat the same error. Seems like a buffer is full someplace.

Within the EtherSRV files there is one named "debug.h". In it, a line reads "#define DEBUG 0". Changing this to "#define DEBUG 1" and recompiling (make clean, make) will generate a debug-enabled ethersrv-linux binary that outputs to console many things. It might be interesting to know what it outputs when the "access denied" issue occurs.

 

[alejack12001]

Thanks for testing it. I'll replace the current download with that binary so that more people can use it. And I wish I had thought of it earlier, but I totally forgot that the Pi Zero 2 is on an older release. It's also terribly slow for building code. ;-0 (It can run it reasonably well though.)

When you setup the file on the remote raspberry pi, I wanted to know if there is a limit to the file size? For example, I was planning on using a 512MB file to house all my DOS platform machines software as backup in case I loose a hard drive.

 

[alejack12001]

Then it is EtherDFS v0.8.2. Have you tried upgrading to v0.8.3? (released in June, 2023)



Within the EtherSRV files there is one named "debug.h". In it, a line reads "#define DEBUG 0". Changing this to "#define DEBUG 1" and recompiling (make clean, make) will generate a debug-enabled ethersrv-linux binary that outputs to console many things. It might be interesting to know what it outputs when the "access denied" issue occurs.

Thank you for the new info on the DEBUG line. I'll try this out as well.

 

[mbbrutman]

When you setup the file on the remote raspberry pi, I wanted to know if there is a limit to the file size? For example, I was planning on using a 512MB file to house all my DOS platform machines software as backup in case I loose a hard drive.

It's all in the docs and it depends on the version of DOS you are using, but here is the short story:

• DOS 2.x: These use FAT12 and are limited to 32MB disk images
• DOS 3.x: Most of these can use either FAT12 or the first version of FAT16. Both are limited to 32MB disk images, but the first version of FAT16 is more efficient on the hard drive with larger images.
• DOS 4.x and up: These can use FAT12, FAT16 or FAT16B. FAT16B is what most people think of when they read FAT16, but it's actually the second revision of it. FAT16B is limited to 2GB when using standard 512 byte sectors.

In summary, FAT12 and early FAT16 images are limited to 32MB. FAT16B is good up to 2GB in a single image, and NetDrive will let you use multiples of those.

 

[alejack12001]

It's all in the docs and it depends on the version of DOS you are using, but here is the short story:

• DOS 2.x: These use FAT12 and are limited to 32MB disk images
• DOS 3.x: Most of these can use either FAT12 or the first version of FAT16. Both are limited to 32MB disk images, but the first version of FAT16 is more efficient on the hard drive with larger images.
• DOS 4.x and up: These can use FAT12, FAT16 or FAT16B. FAT16B is what most people think of when they read FAT16, but it's actually the second revision of it. FAT16B is limited to 2GB when using standard 512 byte sectors.

In summary, FAT12 and early FAT16 images are limited to 32MB. FAT16B is good up to 2GB in a single image, and NetDrive will let you use multiples of those.

I am running MSDOS 6.22. I think Modem7 told me that version 6.22 was good to 2-GB which must be FAT16B. Right now I have Debian 12.5.1 on a 64 GB SD card. The properties state the drive is UNIX file system. When you made your 200-MB file in your demonstration video, you were able to store files within the 200-MB file. So, did you have to format the file to FAT or FAT16 to receive the files from the DOS machines. Reason I ask is the EtherDFS required either a partition of FAT or FAT16 to mount and accept files from the DOS machines. I had a USB drive formatted to accept the files from say an XT type machine. It worked except of course for the access denied, which now can be fixed with the DEBUG correction.