• Please review our updated Terms and Rules here

Features for an FTP client

mbbrutman

Associate Cat Herder
Staff member
Joined
May 3, 2003
Messages
6,409
Having FTP on a machine is really useful. It is probably right up there with having network drives that you can access by drive letter.

So in my quest to fill out my TCP/IP apps, I'm naturally working on an FTP client. I'd like to hear from people who might use it to see what features they consider important. Here is what I have on the list so far:

  • Small size - Executable no more than 90KB in size and total memory usage should stay within 160KB.
  • High performance - the bulk of file transfer code is based on my netcat client, which is as fast as DOS TCP/IP file transfer comes. :)
  • Split screen (like my IRC client) - enables fast screen refreshes and backscroll capability. (Think about those long FTP directory listings scrolling by.)
  • PASV (passive) connections only. Most modern FTP clients default to PASV connections anyway because it is more compatible with firewalls. (A passive connection is where the FTP server listens for an incoming data connection instead of the client.)
  • ASCII and binary transfer modes - but no EBCDIC or 'structured' modes

The only part I'm waffling on is the split screen. Most people use FTP clients from a command line, and the split screen kind of ruins the command line look and feel. But the ability to backscroll is really handy, and I've got the code written already. ;-0

What other features should I have?
 
<snip>

i agree with keeping the command line feel, but yeah backscroll is a great feature for big directory listings. maybe an acceptable middle ground would be pause until a key once every full screen?

you know what would be VERY cool? a TSR for DOS that allows an FTP server to be used just like a local hard drive. drive letter and all.

:eek:
 
Last edited by a moderator:
I thought about the pause prompt on the long directory listings too ... that can be tedious every 20 to 45 lines or so. And on the longest directory listings, I'll need to provide that feature along with the backscroll buffer too!

As for the TSR - that's functionally equivalent to a network redirector, except you want FTP as the protocol underneath? I'm not going there .. Samba is good enough. ; - 0
 
The biggest feature I'd want to see is resumeable downloads. Also on misconfigured networks (or firewalling outbound traffic) I'd include active transfers as well. Hm.. besides that I guess just a progress bar for downloading, ETA is nice but with vintage stuff may be more processing than it's worth.
 
I'm excited about this!

I use FTP between my main Linux machine and my DOS machine which is what I write on. It is an extremely important program for me.

Your list looks about right for me. I've never tried the split screen thing and it sounds like it just adds another level of encumbrance (ala windows) to an otherwise straight forward and intuitive concept. To me FTP should be basically invisible, that is, I just substitute GET and PUT (or mget/mput) for CP. If the split screen thing screws up the ability to just type my wants and needs without paying particular attention to the screen - then I am strongly against it.

Regarding that last point. Although I am perfectly sighted, I use the command line because it is relaxing for the eyes since I don't have to use them much such as with a mousie proggie. People who are blind don't have so much choice and there might be quite a number of potential users there.

One thing that should be on the list for everyday practical use is the ability to script a login. It would be a drag to have to type in a username, then a long password, then cd to the appropiate directory (phew!) before typing in the name of the file or files one wants to copy. That could get tedious. :)

I am also wondering about the configuration. Is that done through a seperate file like WATTCP? If it, in fact, was a WATTCP compatible program then it would be handy for me because it would just be a drop in. WATTCP.CNF is set in my environment and all I'd have to do is add the usual batch file for running the program.

Just for reference, the FTP program I am now using is WFTP and it is 54995b. It would, however be nice to have something that was a bit better. It is very fast on all ascii files but for some binaries it is slow.... beats me.

Lastly. I assume this is 16 bit. :)
 
The biggest feature I'd want to see is resumeable downloads. Also on misconfigured networks (or firewalling outbound traffic) I'd include active transfers as well. Hm.. besides that I guess just a progress bar for downloading, ETA is nice but with vintage stuff may be more processing than it's worth.

The 'PASV' mode (passive mode) actually refers to the server side, and makes the client do an active open to the server for the data connection. In plain English, you want all of your connections to be in PASV mode for getting around firewalls. This is what I had planned.

Resumable downloads is trickier. It is in the FTP spec, but only in block mode transfers. Are there servers that implement block mode transfers?

Progress bar .. easy enough. (Possibly not a bar, but some indicator will be there.)
 
Your list looks about right for me. I've never tried the split screen thing and it sounds like it just adds another level of encumbrance (ala windows) to an otherwise straight forward and intuitive concept. To me FTP should be basically invisible, that is, I just substitute GET and PUT (or mget/mput) for CP. If the split screen thing screws up the ability to just type my wants and needs without paying particular attention to the screen - then I am strongly against it.

If you want to see the split screen mode I'm talking about, try my IRC client. Basically the top 21 lines of the screen are status and the bottom 3 lines are for user input.

The advantage of split screen is that I can very tightly control the screen updates, and do it using very fast memory operations. It also allows me to implement a backscroll buffer very easily.

No mouse is required. (Or supported, or needed!)

One thing that should be on the list for everyday practical use is the ability to script a login. It would be a drag to have to type in a username, then a long password, then cd to the appropiate directory (phew!) before typing in the name of the file or files one wants to copy. That could get tedious. :)

Well, that's kind of standard for FTP. At least userid and password could be easily scripted by providing them in a configuration file or as environment variables. As for the rest of it, if I don't do split-screen the FTP session can probably be scripted by redirecting STDIN. (Which is why I'm still debating doing the split-screen thing.)

I am also wondering about the configuration. Is that done through a seperate file like WATTCP? If it, in fact, was a WATTCP compatible program then it would be handy for me because it would just be a drop in. WATTCP.CNF is set in my environment and all I'd have to do is add the usual batch file for running the program.

It's my own TCP/IP with it's own configuration file. Configuration is minimal, as it supports DHCP.

Just for reference, the FTP program I am now using is WFTP and it is 54995b. It would, however be nice to have something that was a bit better. It is very fast on all ascii files but for some binaries it is slow.... beats me.

Lastly. I assume this is 16 bit. :)

What don't you like about your current program? Does WFTP have DNS resolution in it? Does it support mput and mget?


Mike
 
Thanks for the detailed response Mike. :)

mbbrutman: It's my own TCP/IP with it's own configuration file. Configuration is minimal, as it supports DHCP.

I'll have to have a closer look at your TCP/IP.

Certainly a scrollback buffer is useful. I'm assuming that it would be something like DOSKEY or whatever, so one could edit previous imputs. I can't imagine what else I'd need to see.

As for DHCP, it seems that it is suitable for administration of large networks. I'm not sure that this software needs that. My router is currently using it, and frankly it's a nuisance. Putting in the parameters by hand is much easier to manage for a small situation like mine. Remember that a small user is likely to have a number of machines off line or packed away, and when those machines are put back online the numbers may have changed. Unless those machines also run DHCP then there's trouble. How hard is it to type 192.168.1.101 on one machine and ...102 on another, and so forth? My feeling is that it is best not to confuse matters with unnecessary software and a more complicated setup.

mbbrutman: What don't you like about your current program? Does WFTP have DNS resolution in it? Does it support mput and mget?

No, WFTP does not have DNS resolution (see below) and yes, it supports mput and mget, as well as most of the normal commands.

The main problem is that there is a delay after the completion of binary tranfers. Although the transfer is very fast the negotiation delay after the fact (ie returning to the FTP> prompt), is quite short after get and quite long after put. Fortunately most binary transfers are a "get" and most "puts" are text, but it is somewhat inelegant as I prefer things to be instant. That is why I am using DOS in the first place. :) The only other thing which could be improved would be a scrollback of commands at the FTP> prompt.
 
If you want to assign static addresses to machines, that is fine - it is supported. I started with that approach and then went to DHCP because I think that more people will use DHCP, even at home. Just about everybody has a router at home that supports DHCP and the configuration is so simple.

I generally only use one or two older machines at a time, and they generally not used as servers. So knowing their specific IP address isn't too important for me. If you want to statically configure mTCP it will work just as well.
 
The 'PASV' mode (passive mode) actually refers to the server side, and makes the client do an active open to the server for the data connection. In plain English, you want all of your connections to be in PASV mode for getting around firewalls. This is what I had planned.

Resumable downloads is trickier. It is in the FTP spec, but only in block mode transfers. Are there servers that implement block mode transfers?

Progress bar .. easy enough. (Possibly not a bar, but some indicator will be there.)

You're right, I'm probably thinking more server side features although the client needs to understand some of them too. I know PASV mode is the most successful and common for behind a firewall, and you should be fine there. Where I am our firewall internally blocks PASV (kinda dumb) so if I'm transferring something from a server (and not using scp for some reason) I have to use active mode. Going to the internet or customers retrieving data of course have to use passive mode.

Proftp supports resumable downloads if you want to play with it. It's just an option you enable in the configuration file. You're right though, that may be tricky and I never really thought about the implementation part of that lol

Progress bar you could do well, or just like windows cli ftp you could just type a hash every so often so I know there's activity going on. Otherwise it's kinda like sitting there hoping it's working but not being sure.

I like the scripting idea, just a simple flag to read commands from a txt file would be simple enough maybe :)
 
Here is what I have so far:

  • Binary gets and puts only (No ASCII mode yet)
  • PASV mode only (this is better and should always work than the default)
  • A command line interface that can be scripted by redirecting from STDIN

And things to do (or think about)

  • ASCII mode transfers
  • Mput and mget (although this is difficult because it is a variable list of filenames)
  • Possible: Split screen view with backscroll - this version would not be scriptable
  • Some sort of progress indicator


The important part is that file transfer is rock solid - I'm checking the contents of every file send and receive and I've not botched one yet. It is definitely usable now, but Mput and Mget would be nice.

And aw crap, I just tested against the FTP server in NCSA telnet. Looks like I have to implement the default 'non passive' data socket connections. (NCSA Telnet doesn't support passive opens)

Informal speed test:

PCjr running at 4.77Mhz with a NEC V20, Xircom parallel-to-Ethernet adapter, and a SCSI hard drive on a parallel-to-SCSI adapter

NCSA FTP acting as a server sending a 3.3MB file: 212 seconds, or 15.3KB/sec
My FTP client sending the file: 164 seconds, or 19.7KB/sec

When you consider that when this machine was new it's best comm interface ran at 2400 BPS, I'm kind of happy getting close to 20KB/sec including real disk I/O. (With a less than optimal disk and ethernet setup too.)
 
Last edited:
A DOS shell out command (e.g. ! on *nix ftp) would be very useful. Cuts down on the number of "local" commands (e.g. lls, lcd, etc.) that you have to implement.
 
This is looking good. I'm looking forward to trying a new network program for DOS, there aren't enough of them.

I guess I don't have anything else to gripe about ... for now. :) Thanks Mike, for addressing my concerns. It looks like the DHCP thing will be fine and perhaps quite useful. I'm just a cantankerous type when it comes to changing anything. (sigh) BTW, regarding progress indicator: WFTP uses a % indicator which is handy if something is large and you don't know if things are going well, but otherwise is more detailed than really needed.

Regarding binary only transfer. I don't know how this works but isn't an ASCII file a subset of a BINARY file? I just sent a text file from WFTP to the proftpd server and it went fast. Then I did a "get" on the same file and it sat there for some seconds before it did it. But it did it, and yes, it was still readable. The delay would indicate that one of the programs had to think about it though.

Edit: I was just about to logout when I saw Chuck's suggestion (above) for a shellout command. YES!! that's a good one. :)
 
Chuck - ugh, I forgot about those local commands.

This is getting tedious. Time to take a break. :)
 
Ole Juul,

ASCII and BINARY are two different things in FTP. In BINARY mode, files are transfered exactly byte for byte, which is how you want to send a ZIP or a program. In ASCII mode the machines will convert between CR/LF and just CR, EOF markers in the file will be treated as EOF, etc.

I think that for most things people want BINARY. ASCII is just too damned flakey. The only use I can see for it is for interchanging text files between DOS and Unix system, which never seem to agree upon how a line is terminated.
 
Ole Juul,

ASCII and BINARY are two different things in FTP. In BINARY mode, files are transfered exactly byte for byte, which is how you want to send a ZIP or a program. In ASCII mode the machines will convert between CR/LF and just CR, EOF markers in the file will be treated as EOF, etc.

I think that for most things people want BINARY. ASCII is just too damned flakey. The only use I can see for it is for interchanging text files between DOS and Unix system, which never seem to agree upon how a line is terminated.
That's how I had understood it, but I'm confused about why there is an ASCII transfer mode at all. I mean, an EOF marker is whatever code it is (hex 1A?), regardless of what meaning it may have (or not) to some other program, so why does FTP need to know what's in the file? I'm guessing ASCII can use shorter words and transfer faster.

Anyway, this is getting OT and I'll leave my learning problems for another thread. :)

Cheers,
Ole
 
Ftp dates from before microprocessors or even TCPIP and was not necessarily tied to Unix.

Text representations could be more difference than CRLF versus LF line terminators. Systems could be EBCDIC, for example, with all interesting sorts of record structure.

ASCII mode was supposed to handle the representational differences.

See RFC 114 for the earliest ftp standard. In particular, see section 4.
 
Commands and directory listings are transfered in NVT (Network Virtual Terminal), which defines the end of line character to be CR followed by an LF. Works great for a DOS client, no translation is needed.
 
A DOS shell out command (e.g. ! on *nix ftp) would be very useful. Cuts down on the number of "local" commands (e.g. lls, lcd, etc.) that you have to implement.

It would be a nice feature, but it is a little risky to do
  • The TCP stack hooks the timer interrupt and that interrupt has to get let go cleanly or the system crashes.
  • Same with the interface to the packet driver - if it is not explicitly shut down, the packet driver will keep dropping packets in unallocated memory or worse, make a call into what used to be my memory allocator but is now data.
  • If the user spends more than a few seconds in DOS and data comes in, the packets will be buffered but not processed. That can lead to packet resends from the server side, which at best result in duplicate packets and at worst will kill the connection.

I do need to implement some local commands to make it more usable:
  • Change directory
  • DIR
  • rename
  • copy
  • move
  • delete

Mike
 
Back
Top