EtherDFS - an ethernet drive for DOS

[alank2]

Wireshark uses winpcap/libpcap. I too would love to see a windows server for this. I'm not sure how difficult it would be to port.

 

[pearce_jj]

I still don’t understand why we don’t use ATAoE for this...

 

[mateuszviste]

I still don’t understand why we don’t use ATAoE for this...

ATAoE is about network-attached devices, while EtherDFS is about a network-attached filesystem. The difference may seem minor, but it's in fact a big deal. Typically, you cannot mount a single block device to multiple machines - will result in fs corruption, or worse.
Another thing is that EtherDFS uses its own protocol, specifically tailored for DOS needs. This allows for maximum memory savings.

All that being said, it's worth noting that creating an ATAoE TSR for DOS would be actually much simpler than what EtherDFS achieves. The reason is that it would only need to handle int13h requests about 'reading block' or 'writing block', without all the filesystem-related drama.

 

[pcdata76]

Dear mateuszviste, first of all thank you very much for the development of such a useful utility. Prior EtherDFS, I was using Microsoft Network Client with IPX protocol and use a Windows XP PC (or XP virtual machine) to share files between modern and vintage computers. It was not only inefficient (huge memory footprint in compared with etherDFS) but also much more difficult to setup (particularly the DOS side)

I was not much familiar to linux operating system, using it for particular purposes (like the controller of my small cnc milling machine which runs under LinuxCNC) but I was mostly following the predefined steps to install and use, without getting much into deep. Thanks to the work that I need to do in order to install and use etherDFS efficiently, I'm much more familiar to the operating system now

Anyway, I've decided to build a 7/24 online etherdfs server running on RaspberryPi but after doing a bit of research, I've realized that I can use my old android phone sitting in the junk pile (lg g2) for the purpose. I've rooted the phone, installed necessary tools, linux-deploy app and managed to install and configure a recent version of Ubuntu on it after many retries. I can now connect to the linux working on the phone over vnc or ssh, I can share folders using samba, set proper access rights etc...I also installed the most recent version of ubuntu as a virtual machine (VMware) in my daily windows 7 computer.

VM Linux version : Ubuntu 19.10, Kernel 5.3.0-29-generic, x86 architecture
Phone Linux version: Ubuntu 18.04 LTS Kernel 3.4.0-perf-g88e404c, armhf architecture

I'm getting no issues at all to operate ethersrv on ubuntu (VM). Ethersrv can share the mount point of the FAT image file correctly and I can map the share from my DOS computers as a drive without any problem. File copy operations, assigning DOS file attributes etc. are entirely perfect. In the ubuntu linux working on the lg g2 phone, ethersrv can also share the mount point of the FAT image, but always recognizes it as a non-fat volume (despite it is FAT) and disables DOS attributes. I can map the share in DOS, perform file operations but DOS attributes does not work as expected.

I've checked the file system mount points using mount command and verified that they are exactly the same (vfat) in both of the computers. To double check, I copied the perfectly working and guaranteed FAT image file from virtual machine to the phone and mounted it, but nothing changed. Ethersrv recognized the mount point again as non-fat and disabled DOS attributes. I can browse the mount point in linux and perform file operations without issues.

What can be the reason of ethersrv recognizing the FAT mount point as non-fat in linux on the phone?

 

[mateuszviste]

Hello pcdata76, I'm impressed by what you did - I'd never even think about running ethersrv on an android phone. Well done! Do you have some written tutorial about this by any chance, or a blog entry? I'd gladly link to it from etherdfs' homepage.

About the problem: the fs used by your android system may be FAT, but maybe the fs implementation is somewhat "simple" and does not, for example, support attributes? Here is what ethersrv does to decide whether or not attributes should be allowed:

  /* test if I can fetch the serial id through calling the FAT IOCTL API */
  fd = open(d, O_RDONLY);
  if (fd == -1) return(-1);
  if (ioctl(fd, FAT_IOCTL_GET_VOLUME_ID, &volid) < 0) {
    close(fd);
    return(-1);
  }

Since you say ethersrv considers the mount point to be non-FAT, then it is either unable to open() the requested path or the FAT_IOCTL_GET_VOLUME_ID call is not implemented under Android.

I am attaching a patched file where the fat detection is shorted. Could you test it? If it is only a mis-detection issue, then attributes will work now - but if not, then it must be some android limitation thing. Let me know the result, if this works, then I will have to figure out another way to test for the FATness of mountpoints.

 

[pcdata76]

Hello pcdata76, I'm impressed by what you did - I'd never even think about running ethersrv on an android phone. Well done! Do you have some written tutorial about this by any chance, or a blog entry? I'd gladly link to it from etherdfs' homepage.

About the problem: the fs used by your android system may be FAT, but maybe the fs implementation is somewhat "simple" and does not, for example, support attributes? Here is what ethersrv does to decide whether or not attributes should be allowed:

  /* test if I can fetch the serial id through calling the FAT IOCTL API */
  fd = open(d, O_RDONLY);
  if (fd == -1) return(-1);
  if (ioctl(fd, FAT_IOCTL_GET_VOLUME_ID, &volid) < 0) {
    close(fd);
    return(-1);
  }

Since you say ethersrv considers the mount point to be non-FAT, then it is either unable to open() the requested path or the FAT_IOCTL_GET_VOLUME_ID call is not implemented under Android.

I am attaching a patched file where the fat detection is shorted. Could you test it? If it is only a mis-detection issue, then attributes will work now - but if not, then it must be some android limitation thing. Let me know the result, if this works, then I will have to figure out another way to test for the FATness of mountpoints.

It is actually running in a separate (well, not thoroughly separate, shares the kernel and some mount points like /dev with the phone's android) installation in a image file located in phone's memory. I didn't fully understand the logic behind but it is neither an emulation nor a virtual machine environment. I haven't written a tutorial yet, but it is in my near future plans. I'll let you know after preparing a step-by-step tutorial. Devices like raspberry pi are not so expensive for this purpose, but giving a second life to the otherwise unusable old phones is better way at least for hobbyists like us. After debloating the phone, disabling everything not needed and turning the flight mode on (only WiFi is enabled), i can get almost a week long battery life with its 4 years old, near-dead battery. I could operate one of my vintage or older computers as a file server, but electricity consumption would be nowhere near that phone's consumption.

Anyway, back to the topic. I've recompiled ethersrv using the file that you've attached, and voila! It works! It just doesn't complain about non-fat volume, but i can also set DOS attributes without any issues as well. So the issue seems like a mis-detection.

 

[mateuszviste]

I've recompiled ethersrv using the file that you've attached, and voila! It works! It just doesn't complain about non-fat volume, but i can also set DOS attributes without any issues as well. So the issue seems like a mis-detection.

What version of Android are you running? Some google search suggests that the FAT_IOCTL_GET_VOLUME_ID call has been implemented somewhere between KitKat (2014) and LolliPop (2015). Is your OS that old? If yes, is there any chance you could possibly test on a newer one?

 

[pcdata76]

I'm using Lollipop. It is the latest official android provided by LG.

 

[mateuszviste]

I'm using Lollipop. It is the latest official android provided by LG.

Thanks for the clarification. Some extra research done:
IOCTL calls related to FAT attributes appeared in Linux 2.6.12
the IOCTL call related to volume ID reading (used by ehtersrv to detect FAT) appeared in Linux 3.11

This means that on kernels between 2.6.12 and 3.11 ethersrv won't detect FAT partitions, even though setting/getting FAT attributes could work anyway. I believe this is the case you are in.

Lollipop is supposed to use Linux kernel 3.16.1 by default, so in theory it should work fine with the FAT detection used by ethersrv... But apparently vendors often do not upgrade the Linux kernel on devices that migrate from one Android version to another - meaning that a phone initially released with Kit Kat (ie. kernel 3.10), may get a Lollipop upgrade but still keep its 3.10 kernel.

I must say that I am reluctant in working around this limitation, since it is a non-critical issue (disables attributes support on some devices) and it impacts a very limited subset of (very old, ie. 5 years+) devices. The work-around is also quite easy to put in place (use my hacked fs.c file). Last but not least, I do not really see any alternative way of testing for FAT volumes in a way that would be as elegant and generic as the current method...
_________________________________________________
references:
https://www.systutorials.com/docs/linux/man/2-ioctl_fat/
https://android.stackexchange.com/questions/51651/which-android-runs-which-linux-kernel

 

[pcdata76]

I can use it without any issues with your hacked fs.c file, so the issue can be accepted as solved for my case Thank you very much again.

 

[Peter z80.eu]

Btw. does anyone tried to compile the sources with the Cygwin compiler environment meanwhile ?

 

[mateuszviste]

Btw. does anyone tried to compile the sources with the Cygwin compiler environment meanwhile ?

I'd recommend using MinGW rather than Cygwin. ethersrv should compile just fine with MinGW, assuming one invests some time into creating a libpcap wrapper around the Linux-only raw socket I/O.

 

[JohanLu]

Hi, Mateus, great program! I'm no Linux guy so I've probably made some stupid mistake in setting things up. Used the image for RPI at https://lazybrowndog.net/freedos/ethersrv/ then added SAMBA share on the disk (stupid to do before testing the first step). I have freedos on the old IBM5170 (286) and brutemans programs (DHCP+FTP works great). Auto sensing the MAC adress did not work, but with manual setup it connected right away. My problem i get a file not found when trying to access the created drive, the files on the server cannot be seen when doing DIR. Any direction to point me in? Is there a logfile / error output I can find somewhere? Thanks...

 

[mateuszviste]

Used the image for RPI at https://lazybrowndog.net/freedos/ethersrv/ (...) My problem i get a file not found when trying to access the created drive, the files on the server cannot be seen when doing DIR. Any direction to point me in? Is there a logfile / error output I can find somewhere? Thanks...

The lazybrowndog does not keep its files up to date, so the ethersrv version you got is probably somewhat old (2017, I believe). Can you please test again with latest etherdfs + latest ethersrv and let me know if things improved? ethersrv logs out errors to console, so you may want to run it manually and watch out for any messages on your console while etherdfs performs operations.

 

[JohanLu]

Great with such quick feedback from the author! Sure, I will test this tonight, good with the info that it outputs to console, I will run it that way instead!

 

[JohanLu]

Got this error msg when compiling, then segmentation fault when running file... do not have the knowledge to understand what to do to make it work...

gcc ethersrv-linux.c fs.c lock.c -o ethersrv-linux -O2 -Wall -std=gnu89 -pedantic -Wextra -s -Wno-long-long -Wno-variadic-macros -Wformat-security -D_FORTIFY_SOURCE=1
ethersrv-linux.c: In function ‘main’:
ethersrv-linux.c:919:22: warning: passing argument 2 to restrict-qualified parameter aliases with argument 4 [-Wrestrict]
select(sock + 1, &fdset, NULL, &fdset, &stimeout);
^~~~~~ ~~~~~~
fs.c: In function ‘delfiles’:
fs.c:446:26: warning: ‘%s’ directive writing up to 255 bytes into a region of size between 0 and 511 [-Wformat-overflow=]
sprintf(fname, "%s/%s", dir, diridx->d_name);
^~
In file included from /usr/include/stdio.h:873,
from fs.c:13:
/usr/include/arm-linux-gnueabihf/bits/stdio2.h:36:10: note: ‘__builtin___sprintf_chk’ output between 2 and 768 bytes into a destination of size 512
return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
__bos (__s), __fmt, __va_arg_pack ());

 

[JohanLu]

Now I reinstalled a clean rasbian, compiled the files without problems, tried to set it up again, creating FAT image by doing

sudo dd=if/dev/zero of=fat.fs bs=1024 count=100000
sudo mkdir -p /media/fat
sudo mount -t msdos -o loop fat.fs /media/fat -o dmask=022,gid=1000,uid=1000
sudo ethersrv-linux -f eth0 /media/fat

On the dos machine, i still need to specify the MAC adress manually, autodetect does not work. The DOS machine is on the same logical network but it is through a wifi extender, I might move the RPI to the same physical network, if that is the problem (the dos machine is not welcome in the more comfortable place where I'm sitting right now, as it is kind of noicy and bulky).

Still, on the dos machine, file not found is given. BUT if I create a file from the dos machine, the file is created on the mounted drive on RPI but with 0 size.

I guess it is some problems with the rights or the dos attributes? As I'm not a linux guy, I'm not familiar with these commands, so I pretty much googled how to set it up...

I will keep trying to change some things, but if you see some obvious error in the above setting, please tell me.

 

[Eudimorphodon]

Now I reinstalled a clean rasbian, compiled the files without problems, tried to set it up again, creating FAT image by doing

sudo dd=if/dev/zero of=fat.fs bs=1024 count=100000
sudo mkdir -p /media/fat
sudo mount -t msdos -o loop fat.fs /media/fat -o dmask=022,gid=1000,uid=1000
sudo ethersrv-linux -f eth0 /media/fat

Did you leave out the part where you used mkfs to actually put a fat file system on your blank image file?

 

[JohanLu]

Ah, sorry, no, I just missed to include it here;

This is what it should be;

sudo dd=if/dev/zero of=fat.fs bs=1024 count=100000
sudo mkfs.vfat fat.fs
sudo mkdir -p /media/fat
sudo mount -t msdos -o loop fat.fs /media/fat -o dmask=022,gid=1000,uid=1000
sudo ethersrv-linux -f eth0 /media/fat

 

[JohanLu]

Did you leave out the part where you used mkfs to actually put a fat file system on your blank image file?

My reply dissapered...
no, I just missed to write it in the forum post...