mTCP 2011-10-01 Release

[mbbrutman]

That actually fits the bill I wouldn't dare to stop Mike from making an mTCP variant, but Mike, if you want to, grab the ETHTOOLS package since it comes with C source and might answer your promiscuous mode questions.

I had already pointed out Wiwa's code in an earlier reply in this thread ... next time I'll provide a direct link.

Regardless, after writing a TCP/IP stack from scratch I don't think I need too much help figuring out a packet dumper. Some quick experiments will clear up whether the cards are in promiscuous mode or not. My question to Russ was more along the lines of why a promiscuous mode query or setting API is not visible in the packet specification. (He does have some older source code that can query and set it, so what I really need to do is look at his source code.)

One thing to keep in mind is that if you are using a true switch and not just a hub you can be in promiscuous mode and never see anything that is not destined for your card. Switches are supposed do do that to cut down on packet looks and extraneous traffic. So even with the card in promiscuous mode, you will still only see broadcast traffic and traffic for your NIC.

 

[wiwa64]

Regardless, after writing a TCP/IP stack from scratch I don't think I need too much help figuring out a packet dumper.

I think so too

My question to Russ was more along the lines of why a promiscuous mode query or setting API is not visible in the packet specification.

Is is not? Sorry if i misunderstood you, but when looking into the specification for the packet driver API (which is avaliabe e.g. here: http://www.crynwr.com/packet_driver.html) i find in sections "6.12. set_rcv_mode()" and "6.13. get_rcv_mode()" instructions how to set and how to query the card's mode. (mode 6 = fully promiscuous mode). Or did you mean something different?

 

[mbbrutman]

I think so too


Is is not? Sorry if i misunderstood you, but when looking into the specification for the packet driver API (which is avaliabe e.g. here: http://www.crynwr.com/packet_driver.html) i find in sections "6.12. set_rcv_mode()" and "6.13. get_rcv_mode()" instructions how to set and how to query the card's mode. (mode 6 = fully promiscuous mode). Or did you mean something different?

Argh - I don't know how I missed that! Thanks for pointing it out ...


Mike

 

[Trixter]

(OT: The card I ordered isn't going to work either. Although it works fine in my XT, I saw this on the Intel website:

After going through 5 cards, I managed to find one that is working perfectly in the AT&T PC 6300: An SMC 8013 (specifically an "SMC8013EWC EtherCard PLUS Elite 16C Combo"). The card is soft-selectable, but I chose the jumper that forces port 280, irq 3, memory window at D8000 and it's working great. 7.16MHz NEC v30, 8K memory window at D8000, and the transfer rate is around 50KB/s.

I'm posting this info in this thread so that Mike can be sure just how many machines and people his code has touched. This 28-yr-old machine has been in my possession since it was built and it has never connected to the internet until last night. FTP.EXE gets daily use whenever my old machines are turned on -- so much more convenient than even dual 1.44MB floppies.

 

[Mike Chambers]

After going through 5 cards, I managed to find one that is working perfectly in the AT&T PC 6300: An SMC 8013 (specifically an "SMC8013EWC EtherCard PLUS Elite 16C Combo"). The card is soft-selectable, but I chose the jumper that forces port 280, irq 3, memory window at D8000 and it's working great. 7.16MHz NEC v30, 8K memory window at D8000, and the transfer rate is around 50KB/s.

I'm posting this info in this thread so that Mike can be sure just how many machines and people his code has touched. This 28-yr-old machine has been in my possession since it was built and it has never connected to the internet until last night. FTP.EXE gets daily use whenever my old machines are turned on -- so much more convenient than even dual 1.44MB floppies.

same here, i dont have a single vintage x86 (read: 386 and older) computer without mike's tools. even the ones without a network card have them, along with the xircom parallel adapter packet driver just in case. mTCP is a Godsend to us collectors.

 

[[UFO]]

I also use mTCP with my 486 machine equipped with a 3COM Etherlink III ISA card. It is great!

Keep the good job, guys!

 

[jonnymacuser]

I can't seem to get FTPSRV.EXE running on a 5150 with 256K of memory.

I'm booting DOS 3.30, and that leaves me with only 207K free conventional memory. Upon launching FTPSRV, I get an error "Init: Failed creating buffers".

From ftpsrv.txt, it looks like you need 240KB available RAM for 1 client.

Is there a way to get FTPSRV up on a machine with only 256K?

 

[mbbrutman]

If you cut the default buffer sizes from 8KB to 4KB you can get it down to 230KB. For one client there is 104kB of code, 3 sockets required, and 5 outgoing TCP buffers. The maximum number of receive buffers could be reduced from 40 to 20 with no ill-effects, but that is an option set at compile time. Each of those is 1.5KB, so that would be a 30KB savings.

So if you are desperate I can compile a version that will work in around 200KB. When you say that you had 207KB available, was that before or after you loaded the packet driver?

 

[jonnymacuser]

With FTPSRV_FILEBUFFER_SIZE and FTPSRV_TCPBUFFER_SIZE set to 4K, I still get "failed creating buffers".

207KB is available with a loaded packet driver.

I'd definitely like to test out a smaller version.

 

[Ole Juul]

I can't seem to get FTPSRV.EXE running on a 5150 with 256K of memory.

I'm booting DOS 3.30, . . .

I'm wondering if it wouldn't be an advantage to boot 2.0 (or 2.1) instead. I'm not sure how much less memory use those have, but they are quite functional and most programs will run on them so it could be worth a try.

 

[mbbrutman]

With FTPSRV_FILEBUFFER_SIZE and FTPSRV_TCPBUFFER_SIZE set to 4K, I still get "failed creating buffers".

207KB is available with a loaded packet driver.

I'd definitely like to test out a smaller version.

I made a test version that only had 20 buffers and the memory requirements dropped by close to 70KB instead of the 30KB that I had estimated. That made me *very* curious. So I started to experiment with malloc and found out that its behavior is what we call 'non-linear'. Let me illustrate it:

If you allocate 1 receive buffer (1514 bytes) for the program, the program loads and requires 132848 bytes of memory, including stack and heap.

If you allocate 38 buffers and change nothing else the program requires 190208 bytes. That is only 1342 bytes off of what I would have thought, so I'm not concerned.

If you allocate 39 buffers the program requires 232912 bytes. That is 42532 bytes more than it should require. That is a huge jump.

If you allocate 40 buffers, the program requires 234416 bytes. That is 42522 bytes more, which is slightly less than the 39 buffer case.


I graphed out the required memory and noticed a few smaller anomalies along the way. It looks like the memory allocator in Watcom grabs a little extra, and then starts eating into it. At certain points you cross a threshold where it grabs a lot more. The difference between 38 buffers and 39 buffers is enough to cause that jump.

Now if you allocate 38 buffers with one malloc call, and then add one extra buffer with a different malloc call, you don't get the non-linear behavior. That only requires 190224 bytes, which is only a few bytes more than just allocating 38 buffers with one malloc call. That tells me that the memory allocator is definitely "over-provisioning" the calls, probably to allow some room for realloc.

I'm going to study it a little more and then give you some trial code that allocates one buffer at a time. There might be other large allocations that have this non-linear behavior where I can save even more memory.

This has been very interesting - thanks for bringing it up!

 

[Trixter]

For the ultimate bare-bones memory allocation, you could instead allocate no memory at all and ask DOS for it. Each chunk of memory would need 4 bytes for a pointer, that's it: http://www.ctyme.com/intr/rb-2934.htm

To the person trying to get more memory out of a 256K machine: Run DOS 2.11 if you have no hard disk, as that will free up 16K or more over DOS 3.x. You can also set BUFFERS=1 to free up another ~5K by only allocating one 512-byte buffer, but programs that do a lot of file access will drag.

 

[mbbrutman]

For the ultimate bare-bones memory allocation, you could instead allocate no memory at all and ask DOS for it. Each chunk of memory would need 4 bytes for a pointer, that's it: http://www.ctyme.com/intr/rb-2934.htm

To the person trying to get more memory out of a 256K machine: Run DOS 2.11 if you have no hard disk, as that will free up 16K or more over DOS 3.x. You can also set BUFFERS=1 to free up another ~5K by only allocating one 512-byte buffer, but programs that do a lot of file access will drag.

I was allocating pools of objects with a single malloc to get rid of the overhead with smaller, individual allocations. In this case somebody in run-time land thinks they are smarter. I'm going to check in on the Open Watcom list to see if I'm missing something, and then into the source code to see if there is an explanation.

The DOS allocation calls might not work - a lot of language run-times assume they own all of the available memory, and you have to use the language runtime to get at it. I need to code some more experiments to see what OW is doing.

Either way, it's an interesting problem. And for this code, it means getting back a significant chunk of memory almost for free.

 

[Trixter]

The DOS allocation calls might not work - a lot of language run-times assume they own all of the available memory, and you have to use the language runtime to get at it. I need to code some more experiments to see what OW is doing.

I blindly assumed it was easy with Watcom, sorry. In Turbo Pascal, you can specify how much memory is used with a compiler directive in the main file. For example, if you give this directive "4096" then it only grabs 4096 bytes for its heap.

 

[mbbrutman]

jonnymacuser,

Try this version: http://www.brutman.com/mTCP/ftpsrv.exe

You should leave the buffer sizes at 8KB. Even with 8KB buffers the memory usage of this one is down to 168KB. That should fit comfortably on your system.

Let me know how it works.


Mike

 

[jonnymacuser]

Works! Got an iPhone browsing the 5150's floppy & hard drives.

Left the buffer sizes at 8KB too. This is with DOS 2.10 -- not sure exactly how much memory is being used.


DOS 3.30 supports FTPSRV too, doing a MEM.EXE shows 207K free memory with a loaded packet driver.

DOS 6.00 won't work with FTPSRV (but just the FTP client does work.) It looks like there is just not enough with 197K free memory.

DR-DOS 3.41 doesn't work either (184K free memory with a 3C503.COM loaded), and its not even close to working with the memory hogging FreeDOS (105K free).

 

[jonnymacuser]

I graphed out the required memory and noticed a few smaller anomalies along the way. It looks like the memory allocator in Watcom grabs a little extra, and then starts eating into it. At certain points you cross a threshold where it grabs a lot more. The difference between 38 buffers and 39 buffers is enough to cause that jump.

It would be interesting to see a graph of this, i.e., "buffers allocated" vs. "bytes of memory required," and truly see the non-linear behavior.


On another topic, its very fun to run the FTPSRV on and old machine and remotely "play" with it while you are away from home or at work. This got me thinking about more ways in which one could do this. For example, say you wanted to remotely run programs, or compile code on your vintage computer -- how could you do this?

One crude way came to mind (which may or may not work) but involves:

1) uploading a text source code file to your machine running FTPSRV from a remote location;

2) issuing a command to re-boot the machine;

3) having an AUTOEXEC.BAT which takes care of running a compiler on your source code, i.e., COBOL compiler, or MACRO ASSEMBLER;

4) downloading the finished executable program from the FTPSRV.


Of course remotely debugging and viewing error messages from the compiler might be tricky (could these errors be placed in a text file to be downloaded?).


Anyway, just a thought on a cool project that would be fun to work on.

 

[Trixter]

Already been done, and better: http://josh.com/tiny/

 

[mbbrutman]

I was going to suggest Tiny, but you beat me to it.

Josh uses the Novell TCP/IP stack, which is loaded as a TSR. Unfortunately, you can't mix and match TCP/IP stacks and programs. So running one program requires all of the others to be disabled.

 

[Ole Juul]

Already been done, and better: http://josh.com/tiny/

That's very cool. I can see some practical uses for that around here. However it suggests to set up DOS 6.22 with "PCTCP network stack" which I've Googeled and come up empty. I wonder what that is referring to and where it can be dug up.