	TPTEST - Network throughput measurement program
	-----------------------------------------------

TPTEST 3.0 differs from 2.x in several ways, as explained below.

1. The test logic has been separated from the rest of the code and
   placed into a test engine. The core of the test engine is platform-
   independent, while the I/O and UI functions need to be provided by
   the programmer for his/her particular platform.

2. The client-server protocol has been changed. Again. Now we do
   "COMMAND var1=x;var2=y;var3=z". Reply codes are the same and a
   reply can either be e.g. "200 Welcome" or "200 var1=x;var2=y"

3. The master server protocol has also been changed to be spoken
   in the above "var1=x;var2=y" format.

4. The TCP tests are no longer done on the TCP control ports but
   rather on a separate TCP data port that both client and server
   opens when performing a TCP test.

The changes will be discussed more thouroughly further below.


The source package:
-------------------

The source package consists of three main directories. They are:

apps/		This is contributed applications (clients & servers)
engine/		This is the platform-independent test engine logic
os-dep/		This is the platform-dependent I/O routines

The original distribution contains several simple applications:

apps/macos/client/		GUI MacOS 8.x/9.x/10 client
apps/unix/client/		Text-based Unix client (*)
apps/unix/server/		Server daemon
apps/unix/masterserver/		Master server daemon
apps/windows/clients/gui/	GUI Win32 client
apps/windows/clients/cmdline/	Text-based Win32 client (*)

(*) = same program

The test engine logic in the engine/ directory consists of the
following source files:

engine/tpclient.c	Client support routines
engine/tpcommon.c	Miscellaneous routines
engine/tpengine.c	The core engine routines

and some header files:

engine/tpio.h		Prototypes for the platform-dependent I/O functions
engine/tpclient.h	Prototypes & defines for tpclient.c
engine/tpcommon.h	Prototypes & defines for tpcommon.h
engine/tpengine.h	Prototypes & defines for tpengine.c

Of the above, tpengine.h is the most important for application programmers. 
This is the file where the TPEngine struct is declared. TPEngine is used by 
applications to communicate with the test engine.

tpio.h is a list of prototypes of all the I/O functions the engine uses
and which should be implemented in the platform-dependent I/O module
os-dep/xxx/tpio_xxx.c   (e.g. os-dep/win32/tpio_win32.c)

If you intend to port TPTEST to a new platform, or maybe just write a 
better I/O module for one of the existing platforms, be sure to read all
the comments in tpio.h. They describe the attributes of each I/O function -
what their names are, what arguments they take, what they do and what
return values they give - allowing you to write your own I/O package for
your particular platform.

The platform-dependent I/O packages are located in the os-dep/ directory:

os-dep/unix/		Solaris/BSD/Linux I/O routines
os-dep/win32/		Windows 95/98/NT/2000/XP I/O routines
os-dep/macos/		MacOS/Carbon I/O routines


Building an application:
------------------------

To build a 3.0 application you'll need the test engine files, the
platform-dependent ones, and the main code for your application. Say
you want to build the Unix client found in apps/unix/client/
You would then have to do something like:

mkdir tmp
cp engine/* tmp
cp os-dep/unix/* tmp
cp apps/unix/client/* tmp
cd tmp
vi Makefile
make

Edit the Makefile and make sure you #define the correct values for your
operating system (e.g. -DUNIX -DLINUX for a Linux machine).

If the above seems overly difficult, let us know and we'll provide you with
binaries.


The included applications:
--------------------------

The Win32 and MacOS GUI clients are fairly simple to use, and also have
integrated help so we won't go into them here.

The Win32/Unix text-based client is a simple application to demonstrate
how to write clients and servers using the test engine. It cannot do host
name lookups (the current tpio_unix.c does not support name lookup), it
parses command-line arguments in a very simple way, and will dump core if
you give it unexpected or syntactically faulty commands. It will, however,
do TCP and UDP tests vs a test server, auto-testing vs a test server,
and it can retrieve a server list from a master server (provided you know
the IP-address of the master). Start the program without any arguments to
get a list of available commands.

The Unix test server is a dedicated, forking server daemon application with
currently no configuration options. Another server, which supports all the
old TPTEST 2.x server options (rate limiting testers, banning IP-ranges, etc)
is on its way. The server takes only one argument and that is the TCP control
port number it should start listening to for incoming connections.

The master server is similar to the test server but it doesn't use the test
engine at all. It just accepts connections and wait for the client to send
either the "HELO vmajor=x;vminor=y" message or the "INFO" message. 

Following a HELO-request the server responds with the contents of the 
tptest.servers file. Look at that file for more info on what should be in it.
An INFO request is responded to with a few lines containing contact information
about the owners of the master server. This is to allow clients to always give
their users up-to-date contact information. The information just needs to be
updated on the master server. The contact info is currently hard-coded into
the master server so a recompile is needed when the info changes.


How to write a test application:
--------------------------------

main.c:
-------

 #include <stdio.h>
 #include "tpengine.h"
 #include "tpclient.h"

 main(int argc, char **argv) {
   TPEngine * engp;
   int msSend, msRecv;
   
   if (argc < 3) {
     printf("Usage: %s <server IP> <server control port>\n", argv[0]);
     exit(1);
   }

   // Initialize the test engine
   if ((engp = CreateContext()) == NULL) {
     printf("Failed to create engine context\n");
     exit(1);
   }

   // Set address to selected test server
   engp->hostIP = inet_addr(argv[1]);
   engp->hostCtrlPort = atoi(argv[2]);

   // This program only does UDP receive tests
   engp->tpMode = M_UDP_RECV;

   // ...five second long ones
   engp->sessionTime = 5;

   // at a rate of 300 kbit/s
   engp->bitsPerSecond = 300000;

   // Call the client support function RecalculatePPSSZ() to set the
   // packet size and packet rate values to something that fits our
   // selected bitrate
   RecalculatePPSSZ(engp);

   if ((StartClientContext(engp) != 0) {
     printf("StartClientContext() failed\n");
     exit(1);
   }

   // Do the actual test
   while (engp->state != CLSM_COMPLETE && engp->state != CLSM_FAILED) {
     RunClientContext(engp);
   }

   // Report the results

   printf("%d bytes sent, %d bytes received\n",
	engp->stats.BytesSent, engp->BytesRecvd);
   printf("%d packets sent, %d packets received\n",
	engp->stats.PktsSent, engp->stats.PktsRecvd);

   // Calculate the send time, in milliseconds
   msSend = (engp->stats.StopSend.tv_sec - engp->stats.StartSend.tv_sec) * 1000;
   msSend += ((engp->stats.StopSend.tv_usec - engp->stats.StartSend.tv_usec) / 1000);
   printf("Send time: %d ms      Send rate: %d bit/s", msSend,
	(engp->stats.BytesSent * 8 * 1000) / msSend);

   // Calculate the receive time, in milliseconds
   msRecv = (engp->stats.StopRecv.tv_sec - engp->stats.StartRecv.tv_sec) * 1000;
   msRecv += ((engp->stats.StopRecv.tv_usec - engp->stats.StartRecv.tv_usec) / 1000);
   printf("Receive time: %d ms      Receive rate (throughput speed): %d bit/s", msRecv,
	(engp->stats.BytesRecvd * 8 * 1000) / msRecv);

   // Note that the receive time is always the value used for calculating 
   // network throughput, in send as well as receive tests. The receive time tells 
   // you when packet #1 arrived at its destination (engp->stats.StartRecv) and
   // when the last packet arrived (engp->stats.StopRecv). The difference is the 
   // time the whole transfer took and that can be used to calculate the bandwidth.

   // Clean up
   DeleteContext(engp);

   exit(0);
 }


Compile and link with tpengine.o, tpclient.o and your choice of tpio_xxx.o



  /Ragnar Lonn <prl@gatorhole.se> 2002-09-29




-----------------






TPTEST 2.01 differs from 2.0 mainly in the fact that a TCP test has 
been introduced. The protocol has been somewhat extended to allow for
the TCP tests but 2.0 and 2.01 are compatible (except that a test
involving either a 2.0 client or server will not be able to perform
a TCP test of course, only UDP).

The protocol now looks like this:

   TEST Vmajor,Vminor,Testtype,UDPport,PPS,Packetsize,Packets,Time,TCPBytes

   Vmajor = version number, major (1)
   Vminor = version number, minor (0)
   Testtype = 0 (full duplex), 1 (send only) or 2 (receive only)
		6 (TCP send) and 8 (TCP receive)
   UDPport = client UDP port number
   PPS = Requested no of packets per second
   Packetsize = Requested packet size (bytes)
   Paket = No of packets to be sent (time * PPS)
   Time = time (in seconds) the test is to be run
   TCPBytes = the number of bytes to transfer in a TCP test

First of all, Testtype includes a couple of new test modes, namely no 6
and 8 (there are other modes also like server mode and some auto-test
modes the client uses but they are, as of now, internal to the servers 
and clients. See the source code and especially tptest.h for more details).

Second, there is an extra value added to the end of the TEST string -
TCPBytes. This is the number of bytes to transfer in a TCP send or receive
test. It is 0 (zero) for other tests. Note that a 2.0 server will accept
the whole TEST line and just ignore the last value. It will, however,
complain about invalid test mode if you try to do a TCP send or receive
test. A 2.01 server being contacted by a 2.0 client on the other hand,
will look for 9 values following the TEST command and when it doesn't
see 9 values it will scan just 8.

A slightly kludgy addition to the protocol, but one which I found no
way of avoiding without spending even more unpaid hours on the program,
was to add a "COOKIE" reply from client to server when a TCP Receive
test was initiated. An example:

1. The client connects to the server
2. The server says "200 Welcome"
3. The client says "TEST 2,1,8,3456,0,0,0,10,200000"

(to do a TCP Receive test with a timeout of 10 seconds)

4. The server says "210 7654 7655 0xf466c204"

(210 = OK response code, 7654/7655 = UDP ports (not used), 0xf466c204 = cookie)

5. The client repeats the cookie to the server using the "COOKIE" command:
   The client says "COOKIE 0xf466c204"

6. The server starts sending data on the TCP connection

The UDP receive and full duplex tests handle this by including the cookie
value in the NATOPEN packets sent by the client to the server to open any 
NAT gateways that may exist between the client and the server. In the TCP
tests there is no need to open any NATs so we have to get the cookie to
the server somehow. A better way might have been to have the server send
a cookie to any client that connects - include it in the welcome message -
and let the client repeat the cookie when requesting a service. Well, maybe
in some later version.

Other things:

- Cleaned up the Makefile a little and made it easier to follow (I hope)


  / Ragnar Lonn <prl@gatorhole.se> 2002-01-13




-----------------




TPTEST 2.0 differs from TPTEST 1.0 in a few ways, which are
described below.

- TPTEST 2.0 implements a feature that opens NAT gateways/firewalls
  for testing of incoming aswell as outgoing bandwidth. 

- TPTEST 2.0 also implements a Cookie feature to enhance security.

- This text file is only available in english as of now. It is too
  much work having to write everything down in more than one language
  and the people reading this are extremely likely to be used to
  reading technical documentation in english anyhow.



Changes to the client-server protocol
-------------------------------------

The text-based command protocol used between test client and server
now works like this (the reader is expected to be familiar with TCP-
based Internet application protocols like e.g. SMTP, NNTP) :

1. The client connects to a test server on a TCP port (default: 1632)


2. The server says "200 <welcome message>" to signal its status as
   available for testing.

   If the server is not available it can respond "4xx <message>" 
   (come back later) or "5xx <message>" (don't come back at all)


3. The client requests a test session by using the TEST command:

   TEST Vmajor,Vminor,Testtype,UDPport,PPS,Packetsize,Packets,Time

   Vmajor = version number, major (1)
   Vminor = version number, minor (0)
   Testtype = 0 (full duplex), 1 (send only) or 2 (receive only)
   UDPport = client UDP port number
   PPS = Requested no of packets per second
   Packetsize = Requested packet size (bytes)
   Paket = No of packets to be sent (time * PPS)
   Time = time (in seconds) the test is to be run

   (Note that the syntax here is identical to the one used in TPTEST 1.0)


4. If all is OK, the server responds with:

   210 ServerUDPReceivePort ServerUDPSendPort 0xCookie 

   ServerUDPReceivePort = The local port the server is expecting test packets on
   ServerUDPSendPort = The local port the server will use to send outgoing packets
   Cookie = A magic cookie the client has to include in all UDP packets it sends

   The client then knows what UDP port the server expects packet to arrive at and 
   also what UDP port the server will be using when sending its outgoing packets.
   The Cookie value is a random number created by the server that the client has
   to include in all UDP packets it sends to the server or the server will just
   ignore the packets as they arrive.

   A typical server response might look like this:

   210 3458 3459 0xbc568a3e

   This means that the server is using UDP port 3458 as the receive port for
   incoming (to the server) UDP packets, it is using UDP port 3459 when sending
   outgoing UDP packets, and it requires the client to include the value
   bc568a3e in the "Cookie" header field of all its outgoing UDP packets
   during the test.

5. The test starts, if it is a SEND test. The client then starts sending
   UDP packets to the server.

   ...or...

5b.If the test is a "receive test" or a "full duplex test", which means that
   the server has to send packets to the client, the server will not send any
   packets to the client until the client has first sent the server an UDP
   packet containing the right cookie value. The client uses this opportunity
   to send its "NATOPEN" packets which are fairly short (60 bytes) packets
   sent from the client's UDP *RECEIVE* port to the server's UDP *SEND* port.

   This is why the client needs to know the port the server will be using to
   send its packets. The client will send several packets until it receives
   a packet back from the server. When that happens, the client will enter
   the real test mode and start counting packets received and reception
   times. The server will always "ack" a NATOPEN packet, even when it is
   performing the actal test. An ACK packet that gets lost in transit means 
   the server might fire away the test but the client will then treat the 
   first received test packet as the ACK and so count one packet less during
   the test. If it doesn't get an ACK or a test packet back, the client will
   resend the NATOPEN packet a few times before giving up.

6. The server ACKs the NATOPEN packet (it just returns it on the same ports)

7. The server starts the test in case of a RECEIVE test or the client starts
   the test in case of a FULL DUPLEX test.

8. Statistics are exchanged between client and server using the STATS
   command on the control channel.


The STATS line is exactly the same as the one in TPTEST 1.0

Why not release a 1.1 version instead of 2.0?  Well, feature- and 
functionality-wise 1.1 seems more right but this version is *incompatible*
with version 1.0 due to the unavoidable changes to the client-server
protocol. It seemed more right to use the minor version number for 
feature changes that didn't affect compatibility and the major version
number for changes that did.


  /prl@gatorhole.se 2001-10-23


-------






TPTEST 1.0

New control protocol between client and server
----------------------------------------------

   The data structures that were sent on the control channel between
   client and server have been replaced by a text-based command
   protocol that works like this:

   (This text requires some knowledge of text-based Internet
    application protocols in order to be completely understandable)

   The client connects to the test server on the control (TCP) channel.

   The test server says "200 welcome" if it accepts any clients.
   Otherwise it says "4xx try again later" or "5xx go away"

   After a 200-response the client says:
   
   TEST Vmajor,Vminor,Testtype,UDPport,PPS,Packetsize,Packets,Time

   Vmajor = version number, major (1)
   Vminor = version number, minor (0)
   Testtype = 0 (full duplex), 1 (send only) or 2 (receive only)
   UDPport = client UDP port number
   PPS = Requested no of packets per second
   Packetsize = Requested packet size (bytes)
   Paket = No of packets to be sent (time * PPS)
   Time = time (in seconds) the test is to be run

   The server responds with "200 ServerUDPport" if all is OK. The client
   then knows what UDP port the server is expecting packets to arrive at.

   If something's wrong the server says "5xx error message" or if it is
   a temporary problem "4xx error message".

   After completed test the test results are sent using a STATS
   command on the control channel. In the case of a send-only or a
   full duplex test the client will be the one to initiate the 
   statistics-exchange but in the case of a receive-only test it is
   the server that first sends a STATS line. The STATS line looks
   like this:

   STATS Vmajor,Vminor,PS,PU,PR,TXB,RXB,MaxRTT,MinRTT,
         TX1s:TX1us,TX2s:TX2us,RX1s:RX1us,RX2s:RX2us

   (everything on a single line)

   PS = No of packets sent
   PU = No of packets unsent
   PR = No of packets received
   TXB = No of bytes sent (64-bit integer)
   RXB = No of bytes received (64-bit integer)
   MaxRTT = Max roundtrip time (full duplex test)
   MinRTT = Min roundtrip time (full duplex test)
   TX1s:TX1us = Time in seconds and microseconds when transmission started
   TX2s:TX2us = Time in seconds and microseconds when transmission ended
   RX1s:RX1us = Time in seconds and microseconds when reception started
   RX2s:RX2us = Time in seconds and microseconds when reception ended

   Note that The time values (TX1s:TX1us and the others) have two components 
   each - two 32-bit integers that together comprise a "timeval" value with
   seconds since 1970 and microseconds since last second. In textform they
   can look like this:

   1014336000:578459,1014336005:579388 (TX1s:TX1us,TX2s:TX2us of a test
   that lasted 5 seconds)


The server program has got overload protection
-------------------------------------------------

   In tptest.h there is a DENYFILE definition which is defined, by default,
   as "/tmp/tptest.deny". If this file exists, tptest will refuse new client
   connections. Connecting clients will receive a "4xx" answer when they try
   to connect.

   Read further down also, about the server config file.



The server program has got a config file
-------------------------------------------

   Using the -f flag it is possible to tell the server to read a config file. 
   The config file contains max values for various test parameters - it tells
   the server what types of test values to accept. It is possible to limit
   bitrate per client, packet size, packet rate and test time and also the
   number of simultaneous clients the server will accept. Finally, it is
   possible to deny service to certain IP-numbers or ranges of IP-numbers
   using regular expressions. Look at the supplied sample tptest.conf file
   for more info on how to do all this.

   Starting the server with the command line:

        tptest -m s -f ./tptest.conf

   Will make it read the config file on startup. If you have made changes in 
   the file and want tptest to reload its config you can send it a kill -HUP 
   signal and it will do a config reload.



The windows client and the master server
----------------------------------------

   A TPTEST client for Windows (Win32) has been written and in connection
   with this we have also invented a master server function that keeps track
   of available test servers. This is to make it easier for testers to find
   test servers they can use and also to make it easier to reach users with
   info about new versions of the client program.

   A protocol for communication between master server and client has been
   created and it works as follows:

   The client connects to the master server (usually on port 1632).

   The client says:

   HELO Vmajor,Vminor

   Whereupon the master server responds with something like:

   250-bleak.pdc.kth.se TCP 1632 "100Mbps, KTH, Stockholm, Sverige"
   250-croclist.gatorhole.com
   250 

   The 250 code means that all is OK and that the master server is sending
   a list of the known test servers in existence. If there is a minus sign
   directly after "250" it means that the current line is not the last line
   but will be followed by more lines. After "250" and possibly a minus
   sign follows the hostname of the test server. If there is nothing after
   that the client will assume that the test server will accept connections
   on TCP port 1632 which is default. If there is anything within quotation
   marks "" it is interpreted as a short description of the test server.
   The client may show this description to the user to help the user 
   select a test server. 

   If the client has an old version of the client software, the master server
   may answer:

   501 You need to upgrade. 

   The client can also request contact information from the master server
   using the INFO command. The syntax is:
   
   The client says:

   INFO

   And the master server replies, for instance:

   250-The Swedish ICT-commission network throughput test program
   250-Email: bandbreddstest@itkommissionen.se
   250-WWW: http://www.itkommissionen.se/
   250

   The reply, then, is a number of lines of text with contact information
   for those who want to know more or upgrade their client, etc.
   

   The master server:

   The current master server program is very simple. It only knows the
   INFO and HELO commands and when it gets a HELO command it just burps
   up the contents of a certain file to the connected client, line by
   line.

   By default, this file is "/etc/tptest.servers" but that can easily
   be changed in tptestmaster.c by changing the DATAFILE definition
   or by supplying the filename as the only commandline argument when
   starting tptestmaster:

     ./tptestmaster /etc/tptest.servers

   Tptestmaster also checks for a DENYFILE which is, by default,
   "/tmp/tptestmaster.deny" and if it exists it means that tptestmaster
   refuses new connections.

   Note that tptestmaster and the tptest servers normally use the same
   TCP port (1632) by default. This means that if you want to run both
   a master server and a test server on the same machine you have to use
   another port for one of them (it's easiest to let the test server
   use another port).



   The Windows client:

   The Windows client performs its tests in the same way the Unix client
   does but it includes some extra features to make things easier to the
   user, especially when starting a test. The Unix client does not, for
   example, support the master server protocol.

   The Windows client user may click directly on the big START button 
   and by doing that immediately start a test without having to decide
   what server to use, what packet size, data rate, etc. This auto-test
   procedure works by the client first connecting to the master server
   (the address of the default master server is compiled into the
    client program), fetching a list of the available test servers and
   then selecting the first server in the list that wants to be spoken
   to using a protocol the client knows (Currently, the only protocol
   available today is "TCP" but the client supports the use of 
   different TCP ports at least). The client then makes several
   5-second tests at first a really low speed and then increasing the
   speed a little for each test until the test result stops improving.
   When the test result (the throughput) doesn't get any better the
   client assumes it has found the maximum throughput and stops the
   test and displays the results to the user.

   The user may also click on the "Select test speed" button and choose
   a certain speed to be tested. If s/he does that the selected speed
   will be the only speed tested.

   If the user chooses to use the Advanced menu (Advanced mode) it has
   options to set what test server to use and also to control in detail
   the various parameters of the test, just like with the Unix client.


Other changes
-------------

   TPTEST 0.97 and earlier versions regarded 1 kbps as 1024 bits/second.
   There is some confusion about this but we have concluded that when
   talking about network bandwidth it is more common that 1000 bits
   equals 1 kilobit so from version 1.0 all TPTEST software considers
   1 kbps to be 1000 bits/s. 

   When someone goes shopping for network bandwidth they often buy 
   bandwidth at speeds that are multiples of 64 kbps and that means 
   that a connection that is labeled "1 Mbps" can often be 1024 kbps
   (16 x 64 kbps). TPTEST will in such a case correctly state the
   speed as 1.024 Mbps or 1024 kbps.

   A so-called T1-connection is 1536 kbps (24 x 64 kbps) and that is,
   then, not exactly 1.5 Mbps which one might think but rather 1.54 Mbps.

   1 kB = 1 kilobyte = 1024 bytes
   64 kB = 64 kilobyte = 65536 bytes
   1 MB = 1 megabyte = 1048576 bytes

   1 kbit = 1 kilobit = 1000 bits
   64 kbit = 64 kilobit = 64000 bits
   1 Mbit = 1 megabit = 1000000 bits

