Ethernet tools . . .

[wiwa64]

Hello everybody!

After some time of absence i'm here again with some new programs i wrote.

As result of some experiments i made with packet drivers, i wrote four little utilities which you might like (or perhaps not? ). But look yourself:


ETHSEND

Is a program that sends a single raw ethernet packet out to the LAN. As such, it is not terribly usefull for everday (network-) life, but for experimental and testing purposes it can be quite handy. It helped me a lot in writing the other three programs.​

ETHDUMP

Is just another packet sniffer. This isn't terribly innovative as well, but for me it was a pre-requisite for writing the other two programs. As it therefore had to be done anyway, why not give it a decent user-interface and turn it into something usefull?

It basicly duplicates the functionality of the program ETHCAPT, written by Yusuf Motiwala, many years ago. However it allows to display the data already while recording, with two levels of verbosity (headers only or full data as raw hex dump). In addition it has some filtering capabilities which its predecessor hasn't got. It can optionally write the captured data to a binary file in the same format as ETHCAPT would do, so it can later be visualized by ETHVIEW (also by Yusuf).​

ETHWHO

Is a program that tries to generate a who-is-who of your ethernet LAN (with bus-topology). It lists in tabular form the source and destination ethernet adresses of the packets passing by and counts who is sending whom how many packets. The content of this table is optionally exported into a csv-file (character separated value) from where it can be further processed in what ever way, e.g. be imported into an excel table.​

ETHWHAT

Is a program that watches your (bus topology) ethernet and tries to figure out what game they are playing. It analyzes the packets passing by, according to frame-type and protocol used (IP, IPX, ...) and displays statistics about it. In addition, if it finds IP-traffic, it tries to guess the basic configuration parameters from the found data.​

All programs require a packet driver but no further configuration. They behave purely passive and do not appear themselves in the LAN (except ETHSEND of course). And sorry, there is no further documentation yet, but each program has a brief help-function, which can be accessed by the /? command-line option ( -h will do as well).

Everybody may feel free to download the attached archive ETHTOOLS.ZIP and try the included programs. And of course i would appreciate many comments.

 

[Raven]

These sound nice, especially ETHWHAT. Into the archives of programs and such they go..

 

[mbbrutman]

Wiwa64 -

Sorry about the delay in responding. I've been in telnet hell recently.

I've tried the Ethtools under DOSBox and they all work well. I will be moving them to my older machines later today.

I have a 'diagnostic' type utility on my to-do list as well - the ability to capture and trace raw Ethernet traffic is really important for debugging. For my TCP/IP development I used TCPDUMP on Linux because it does all of the packet parsing already. For debugging my applications on DOS I have a trace mode in all of the applications that can dump and interpret the packets my code is interested in, but nothing that serves as a packet sniffer.

Are you still coding using Watcom? I would like to see your approach to incoming and outgoing buffer management - Are you doing to post a copy of the source code? I posted my low level packet handling code for Turbo C++ here a while ago: http://www.brutman.com/mTCP/mTCP_tcpacket.html

Good work, and don't be such a stranger. If you want to compare notes on implementing higher level protocols like TCP let me know - mbbrutman@yahoo.com . There are not too many people venturing into this forest, and it is nice to have company.

 

[wiwa64]

Wiwa64 -
Are you still coding using Watcom? I would like to see your approach to incoming and outgoing buffer management - Are you doing to post a copy of the source code? I posted my low level packet handling code for Turbo C++ here a while ago: http://www.brutman.com/mTCP/mTCP_tcpacket.html

Actually i have never used Watcom. It's all BorlandC (3.1 in my case) and it works quite well, so far. I don't have general reservations against publishing the source code as well, but there is (was?) a limit in file sizes, so i had to make a choice.

As far as the buffer management is concerned, this is essentially how i did it:

#define MAXBUFS     8  	                /* maximum number of Ethernet buffers */
#define BPMASK      7
#define BUFSIZE     1520

typedef unsigned short word;            /* 16 bits */
typedef unsigned char  byte;            /*  8 bits */

typedef struct {
  word  pkt_len;
  byte  pkt_data[BUFSIZE];
  } pkt_buffer;

word  pkt_get           = 0;
word  pkt_put           = 0;
pkt_buffer pb[MAXBUFS];

/*----- put packet into buffer -----------------------*/

// compiled with Borland C++  3.1
// with "Compile via assembler" option
// in Options | Compiler | Code generation ...

// this routine is called from the packet driver
void far pkt_callback(void) {
  asm {
    pop  di; // compensate for compiler generated PUSH DI / POP DI
    push ds;                   // save driver's data segment
    mov  di,DGROUP;            // NOT in huge model !!!
    mov  ds,di;                // set C's data segment
    }
  disable();
  if(_AX) {
    pb[pkt_put & BPMASK].pkt_len |= 0X8000;
    pkt_put++;
    }
  else {
    if(pb[pkt_put & BPMASK].pkt_len || _CX >= BUFSIZE) {
      _ES = 0;
      _DI = 0;
      }
    else {
      pb[pkt_put & BPMASK].pkt_len = _CX;
      _ES = FP_SEG(pb[pkt_put & BPMASK].pkt_data);
      _DI = FP_OFF(pb[pkt_put & BPMASK].pkt_data);
      }
    }
  enable();
  asm {
    pop  ds; // restore driver's data segment
    push di; // compensate for compiler generated PUSH DI / POP DI
    }
  }

/*----- get packet from buffer -----------------------*/

  do {
    if(pb[pkt_get & BPMASK].pkt_len&0x8000) {
      process_packet(&pb[pkt_get & BPMASK]);
      pb[pkt_get & BPMASK].pkt_len = 0;
      pkt_get++;
      if(pkt_get >= pkt_put) {
	pkt_get &= BPMASK;
	pkt_put &= BPMASK;
	}
      }
    } while( . . . );

As you can see, it's mostly "C"-code with only very little in-line-assembler included. It took me some days to find out, how to do that, (and during theese days may cat had to listen to a lot of swearing ) but since i found out how to do it, it's no problem anymore. The actual code varies a bit from one program to the other, in ETHDUMP for example, i added some more instructions to precisely detect the position of lost packets in the data stream, while in the other programs i was only interested in their total number.

Sooner or later i will publish an updated version. In the meantime i made a number of minor improvements to most of the grograms and i wrote an additional one to browse the capture file(s) generated by ETHDUMP (or ETHCAPT).

 

[mbbrutman]

I take it that the cursing occurred while you were trying to figure out exactly what regs to push and pop to make the linkage work with the C code? That is where I had my trouble.

I'm curious about your procedure though - it is conceptually simpler to use the 'interrupt' keyword so that all regs are saved and restored by the compiler. I don't know the ins and outs of Turbo C++ 3.0 (my compiler), so figuring out the linkage was a nightmare. Your linkage is different - how did you come up with all of that?

Mike

 

[wiwa64]

I take it that the cursing occurred while you were trying to figure out exactly what regs to push and pop to make the linkage work with the C code?

That's correct!

... it is conceptually simpler to use the 'interrupt' keyword so that all regs are saved and restored by the compiler. ...

This is not a clever idea!

The interrupt keyword is intended for subroutines that are either invoked by a true hardware interrupt or by the machine instruction int nn (opcode 0xCD 0xnn). On entry such a routine expects three words on the stack: the processor status word, the current code segment register and the instruction pointer, in that order. Such a routine will be left by an IRET (interrupt return, opcode 0xCF) instruction, which pulls theese three words off the stack and restores them into the respective registers.

An ordinary subroutine, in contrast, is called by a near or far CALL instruction (various opcodes) and left by a near or far RET (return from subroutine, also various opcodes) instruction. These instructions only push and pull one (or two for the far variants) words onto (and off) the stack.

The packet driver calls his receiver routine through a far CALL to the address you passed it in registers ES:DI of the access_type() function and hence pushes two words onto the stack. Declaring such a routine as "INTERRUPT" to the compiler, will cause it to use the IRET instruction to leave the routine and hence pull three words off the stack. Trying it this way, you may almost certainly expect trouble and the misalignment of the stack will probably be the least one of it.

... how did you come up with all of that?

The BorlandC compiler has a feature which causes it to produce assembler code (upon request). This way one can easily inspect how the complier translates the various C-instructions into mashine language. Of course it is helpfull to understand assembler code to do this, at least to some extend. Other compilers may or may not have a similar feature.

 

[mbbrutman]

That's correct!

This is not a clever idea!

The interrupt keyword is intended for subroutines that are either invoked by a true hardware interrupt or by the machine instruction int nn (opcode 0xCD 0xnn). On entry such a routine expects three words on the stack: the processor status word, the current code segment register and the instruction pointer, in that order. Such a routine will be left by an IRET (interrupt return, opcode 0xCF) instruction, which pulls theese three words off the stack and restores them into the respective registers.

An ordinary subroutine, in contrast, is called by a near or far CALL instruction (various opcodes) and left by a near or far RET (return from subroutine, also various opcodes) instruction. These instructions only push and pull one (or two for the far variants) words onto (and off) the stack.

The packet driver calls his receiver routine through a far CALL to the address you passed it in registers ES:DI of the access_type() function and hence pushes two words onto the stack. Declaring such a routine as "INTERRUPT" to the compiler, will cause it to use the IRET instruction to leave the routine and hence pull three words off the stack. Trying it this way, you may almost certainly expect trouble and the misalignment of the stack will probably be the least one of it.

The BorlandC compiler has a feature which causes it to produce assembler code (upon request). This way one can easily inspect how the complier translates the various C-instructions into mashine language. Of course it is helpfull to understand assembler code to do this, at least to some extend. Other compilers may or may not have a similar feature.

My code actually works very well - it runs for over a week at a time under a decent load under a variety of packet drivers, so I am sure that I am not having stack alignment issues.

I used the interrupt keyword because it has the advantage of forcing the compiler to push all of the registers onto the stack for me, and restore them on exit. The packet driver is an unknown environment to me, so rather than try to guess which registers were safe to use I just save and restore them all. This isolates me from implementation differences in the packet drivers. And I didn't have to figure out how to get addressability to my data segment - the compiler did that for me.

The only problem I had with linkage was specific to a particular packet driver - the Xircom PE3-10BT. That particular packet driver was poorly implemented, and had problems with the iret. Most packet drivers can figure out if you used a retf or an iret, and adjust accordingly because the packet specification is not clear on how the call should be made. (Most programmers use the interrupt keyword when getting a callback from a packet driver.) Russell Nelson (from Crynwr, the originator of the packet specification) helped me get through the specific problem with the Xircom packet driver - the solution was to continue use the interrupt keyword, but to write my own epilog code that does everything except the iret.

Before talking to Russell my code worked perfectly with the Davicom NE2000 clone, but not the Xircom .. this was my clue that there were implementation differences in the packet drivers that were beyond my control. My usage of the interrupt keyword is correct and isolates me from any possible difference, including inadvertently stepping on registers that might be in use.

 

[wiwa64]

Well i just had a short look on your code, actually just at your reciever routine and of course you wont have stack problems, despite of using the INTERRUPT keyword. Simply because you circumvented the problem with the IRET instruction by using your own epilogue. Effectively you declared the routine as interrupt to the compiler and behind its back turned it into an ordinary subroutine. Ok, that's a trick. I used a different trick and i consider my solution a bit cleaner, but that may be a matter of taste.

Actually it isn't necessary to save all registers on the stack. Have a look on what the compiler made of the above C-code:

_pkt_callback   proc    far
        push    bp
        mov     bp,sp
        push    di
        pop     di
        push    ds
        mov     di,DGROUP
        mov     ds,di
        cli
        or      ax,ax
        je      short @1@198
        or      word ptr DGROUP:_pkt_len,-32768
        jmp     short @1@310
@1@198:
        cmp     word ptr DGROUP:_pkt_len,0
        jne     short @1@254
        cmp     cx,2100
        jb      short @1@282
@1@254:
        xor     ax,ax
        mov     es,ax
        xor     di,di
        jmp     short @1@310
@1@282:
        mov     word ptr DGROUP:_pkt_len,cx
        mov     ax,ds
        mov     es,ax
        mov     di,offset DGROUP:_pkt_data
@1@310:
        sti
        pop     ds
        push    di
        pop     di
        pop     bp
        ret     
_pkt_callback   endp

You see, the only registers that are modified are AX, DI and ES. Now, ES and DI take the buffer address as return values, so the packet driver expects them to be changed. Remains AX which turned out to be not critical as well and if it were, it would have been easy to save this single register on the stack and restore it on return.

I didn't even know, that there are packet drivers around that tolerate a misaligned stack or compensate for it by themselves, so my code doesn't rely on that feature. However i faced two other problems:

First, the packet driver doesn't know about the C-programs data space. Therefore the data segment register has to be set to the C-programs data segment, which is done by the pair of intructions "mov di,DGROUP / mov ds,di". On exit, of course, the packet drivers data segment has to be restored to its original value which was saved on the stack. This is the reason, why my code has to be compiled with the "compile through assembler" option because the C-compiler itself doesn't know about the symbol DGROUP. But this is no big deal, you just check a box in the options/compiler menu and that's it. Everything else works as usual. Nevertheless, your approach with the interrupt keyword avoids this little inconvenience.

Second, the complier wants to be clever and saves register DI on the stack and restores it on return. This may be a good idea in ordinary C-routines. In our case however, we want to use register DI as a return path therefore it is absolutely counter productive if the compiler restores DI to the value it had on entry. A pair of pop di / push di instructions compensates for that.

But ooops, i just recognize, that your code does the same thing. It also restores the ES and DI registers in the epilogue, thus overwriting the values written into theese registers in the body of the routine. Now i really wonder how your buffer address ever arrives at the packet driver. I might have to explore this by passing your code, at least the relevant fragment, through the compiler to see how it ist actually translated. But to do this, it ist definitely to late now. (It's one 'clock AM, local time) So maybe tomorrow, or next year . . .

 

[mbbrutman]

I had to go back and dig through my email .. and some of it is missing. But here is the background on the Xircom driver:

The Xircom driver expects an RETF in the receiver routine, not an IRET. My original code used the interrupt keyword without the custom epilog, so it crashed all of the time. The Davicom packet driver worked perfectly, so I suspected a problem with the Xircom packet driver.

The original packet spec expects a RETF. The current packet spec and reference code is flexible, and allows for either a RETF or an IRET - it can tell what the upcall routine did by checking the flags. The Xircom code, being older, was never updated with this ability. So technically it is correct, but just not as nice as the newer packet drivers. Too bad it absolutely does not work with the default code generated by the interrupt keyword. My custom epilog is essentially the same as the compiler generated one, except for the RETF instruction.

At the time I wrote my code I was an assembler neophyte so it was easier for me to use the interrupt keyword than to try to re-establish addressability to the data segment used by the compiler. My compiler (Turbo C++ 3.0, not Borland 3.1) doesn't have a "compile through assembler" option either, so it's not an option.

One important part of the semantics of the interrupt keyword and my function that you missed are that all of the registers that I used treated as parameters pushed on the stack, not live registers. The interrupt keyword generates the pushes in the same order that I have the parameters specified. In the body of the routine those register variables look like parameters passed on the stack, and if the values are changed they are changed on the stack. The epilog code will pop the correct values into the proper registers.

Interesting conversation ...

 

[wiwa64]

In the meantime i found some time, to pass the code through the compiler and look, what code it generates. Here is the result:

; static void far interrupt receiver( unsigned bp, unsigned di, unsigned si,
;                                     unsigned ds, unsigned es, unsigned dx,
;                                     unsigned cx, unsigned bx, unsigned ax ) {
        assume  cs:_TEXT
@receiver$quiuiuiuiuiuiuiuiui   proc    far
        push    ax
        push    bx
        push    cx
        push    dx
        push    es
        push    ds
        push    si
        push    di
        push    bp
        mov     bp,DGROUP
        mov     ds,bp
        mov     bp,sp
; if ( ax == 0 ) {
        cmp     word ptr [bp+16],0
        jne     short @8@198
; if ( (cx>1514) || (Buffer_fs_index == 0) ) {
        cmp     word ptr [bp+12],1514
        ja      short @8@114
        cmp     byte ptr DGROUP:_Buffer_fs_index,0
        jne     short @8@142
@8@114:
; es = di = 0;
        xor     ax,ax              ; this is the actual trick:   writing to the
        mov     word ptr [bp+2],ax ; saved register values on the stack instead 
        mov     word ptr [bp+8],ax ; of writing to the actual registers.
; Packets_dropped++;
        add     word ptr DGROUP:_Packets_dropped,1
        adc     word ptr DGROUP:_Packets_dropped+2,0
; }
        jmp     short @8@170
@8@142:
; else {
; Buffer_fs_index--;
        dec     byte ptr DGROUP:_Buffer_fs_index
; Buffer_packet_tmp = Buffer_fs[ Buffer_fs_index ];
        mov     al,byte ptr DGROUP:_Buffer_fs_index
        mov     ah,0
        mov     cl,2
        shl     ax,cl
        mov     bx,ax
        mov     ax,word ptr DGROUP:_Buffer_fs[bx+2]
        mov     dx,word ptr DGROUP:_Buffer_fs[bx]
        mov     word ptr DGROUP:_Buffer_packet_tmp+2,ax
        mov     word ptr DGROUP:_Buffer_packet_tmp,dx
; es = FP_SEG( Buffer_fs[ Buffer_fs_index ] );
        mov     al,byte ptr DGROUP:_Buffer_fs_index
        mov     ah,0
        mov     cl,2
        shl     ax,cl
        mov     bx,ax
        mov     ax,word ptr DGROUP:_Buffer_fs[bx+2]
        mov     word ptr [bp+8],ax
; di = FP_OFF( Buffer_fs[ Buffer_fs_index ] );
        mov     al,byte ptr DGROUP:_Buffer_fs_index
        mov     ah,0
        mov     cl,2
        shl     ax,cl
        mov     bx,ax
        mov     ax,word ptr DGROUP:_Buffer_fs[bx]
        mov     word ptr [bp+2],ax
@8@170:
; }
; }
        jmp     short @8@254
@8@198:
; else {
; Packets_received++;
        add     word ptr DGROUP:_Packets_received,1
        adc     word ptr DGROUP:_Packets_received+2,0
; Buffer[ Buffer_next ] = Buffer_packet_tmp;
        mov     al,byte ptr DGROUP:_Buffer_next
        mov     ah,0
        mov     cl,2
        shl     ax,cl
        mov     dx,word ptr DGROUP:_Buffer_packet_tmp+2
        mov     bx,word ptr DGROUP:_Buffer_packet_tmp
        mov     si,ax
        mov     word ptr DGROUP:_Buffer[si+2],dx
        mov     word ptr DGROUP:_Buffer[si],bx
; Buffer_len[Buffer_next] = cx;
        mov     al,byte ptr DGROUP:_Buffer_next
        mov     ah,0
        shl     ax,1
        mov     dx,word ptr [bp+12]
        mov     bx,ax
        mov     word ptr DGROUP:_Buffer_len[bx],dx
; Buffer_next = (Buffer_next + 1) % (PACKET_RB_SIZE);
        mov     al,byte ptr DGROUP:_Buffer_next
        mov     ah,0
        inc     ax
        mov     bx,21
        cwd     
        idiv    bx
        mov     byte ptr DGROUP:_Buffer_next,dl
; if (Buffer_lowFreeCount > Buffer_fs_index ) {
        mov     al,byte ptr DGROUP:_Buffer_lowFreeCount
        cmp     al,byte ptr DGROUP:_Buffer_fs_index
        jbe     short @8@254
; Buffer_lowFreeCount = Buffer_fs_index;
        mov     al,byte ptr DGROUP:_Buffer_fs_index
        mov     byte ptr DGROUP:_Buffer_lowFreeCount,al
@8@254:
; }
; }
; asm {
; pop bp
        pop      bp
; pop di
        pop      di
; pop si
        pop      si
; pop ds
        pop      ds
; pop es
        pop      es
; pop dx
        pop      dx
; pop cx
        pop      cx
;        pop bx
        pop      bx
; pop ax
        pop      ax
; retf
        retf    
; }
; }
        pop     bp
        pop     di
        pop     si
        pop     ds
        pop     es
        pop     dx
        pop     cx
        pop     bx
        pop     ax
        iret    
@receiver$quiuiuiuiuiuiuiuiui   endp

Now i understand, how it works and how the results find thier way back to the driver in this version.
Certainly this should work with my programs as well and it shouldn't even be to difficult to change.

On the other hand, i remember how often i already thought this . . .

So i will probably stay with my version, as it works fine too, but keep this approach in mind as an option for the future