Minor Editing and HTML Conversion at Merit Network, Inc on 27 October, 1994 by Ken Latta
________________________________________________________________________________________________________
1
Message-Id: <[email protected]>
From: Curtis Villamizar <[email protected] style="">
Reply-To: [email protected]
Subject: Re: Notes from the October NANOG meeting
Date: Wed, 26 Oct 1994 18:55:16 -0400
Stan,
Thanks for taking notes. It is a difficult job. There are some major ommisions.
Here are just a few that I think are important in that they deal with what might be a critical issue. I left the Cc on, in case someone remembers differently.
Curtis
Ameritec presented testing results of tests using TCP on their NAP. (Please get details from Ameritec). Ameritec claims that their switch performed well. It was pointed out that with no delay in the tests, the delay bandwidth product of the TCP flows was near zero and it was asserted (by me actually) that results from such testing is not useful since real TCP flows going through a NAP have considerable delays.
> PacBell NAP Status--Frank Liu
>
> [ ... ]
>
> Testing done by Bellcore and PB.
> [TTCP was used for testing. The data was put up and removed quickly, so
I
> did lose some in taking notes.]
> [ ... ]
>
> Conclusions
>
> Maximum througput is 33.6 Mbps for the 1:1 connection.
>
> Maximum througput will be higher when the DSU HSSI clock and data-rate
> mistmatch is corrected.
>
> Cell loss rate is low (.02% -- .69%).
>
> Througput degraded with the TCP window size is greater than 13000
bytes.
>
> Large switch buffers and router traffic shaping are needed.
>
> [The results appear to show TCP backing-off strategy engaging.]
Again, no delay was added. Measured delay (ping time) was said to be 3 msec (presumably due to switching or slow host response). Again - It was pointed out that with no delay in the tests, the delay bandwidth product of the TCP flows was near zero and asserted that results from such testing is not useful.
> ANS on performance --- Curtis Vallamizar
> There are two problems: aggregation of lower-speed TCP flows, support
for
> high speed elastic supercomputer application.
>
> RFC 1191 is very important as is RFC-1323 for these problems to be
addressed.
>
> The work that was done -- previous work showed that top speed for TCP
was 30M
> bs.
>
> The new work -- TCP Single Flow, TCP Multiple Flow
>
> Environment -- two different DS3 paths (NY->MICH: 20msec;
NY->TEXAS->MICH:
> 68msec), four different versions of the RS6000 router software and
Indy/SCs
>
> Conditions -- Two backround conditions (no backround traffic, reverse
TCP
> flow intended to achive 70-80% utilization)
> Differing numbers of TCP flows.
The difficulty in carrying TCP traffic is proportional to the delay bandwidth product of the traffic, not just the bandwidth. Adding delay makes the potential for bursts sustained over a longer period. Real networks have delay. US cross continent delay is 70 msec.
ANSNET results were given using improved software which improved buffer capacity, intentionally crippled software (artificially limited buffering), and software which included Random Early Detection (RED - described in a IEEE TON paper by Floyd and Jacobson). Sustained goodput rates of up to 40-41 Mb/s were acheived using ttcp and 1-8 TCP flows. Some pathelogical cases were demonstrated in which much worse performace was acheived. These case mostly involve too little buffering at the congestion point (intentionally crippled router code was used to demostrate this) or using a single TCP flow and setting the TCP window much too large (3-5 times the delay bandwidth product). The latter pathelogic case can be avoided if the routers implement RED. The conclusions were: 1) routers need buffer capacity as large as the delay bandwidth product and 2) routers should impement RED.
Only a 20 msec delay was added to the prototype NAP testing. Results with the prototype NAP and 20 msec delay were very poor compared to the performance of unchannelized DS3. Prototype NAP testing results were poor compared to Ameritec and Pacbell results due to the more realistic delay bandwidth product. Worse results can be expected with a 70 msec delay and may be better indications of actual performance when forwarding real traffic. More testing is needed after fixes to ADSUs are applied. A sufficient bottleneck can not be created at the switch until ADSU problems are addressed.
There was some discussion (at various times during the presentations) of what this all means for the NAPs. If I may summarize - On the positive side the Ameritec switch has more buffering than the Fore used in the Bellcore prototype NAP. On the negative side, Ameritec didn't include any delay in their testing. NAP testing results (both positive results from Amertic, mixed results from PacBel and negative results from ANS) are inconclusive so far.
Curtis
BTW - I can't see how anyone could have kept accurate notes since a lot was presented. Maybe it would be better to just collect the slides from the presenters. If you want an ascii version on mine try http://tweedledee.ans.net:8001/nap-testing/tcp-per... and save-as plain text. You'll get the slides minus the plots of results but with some additional annotation.
________________________________________________________________________________________________________
2
From: [email protected] (Stan Barber)
Date: Wed, 26 Oct 1994 19:14:45 CDT
In-Reply-To: Curtis Villamizar <[email protected] style="">
"Re: Notes from the October NANOG meeting" (Oct 26, 6:55pm)
X-Mailer: Mail User's Shell (7.2.5 10/14/92)
To: [email protected]
Subject: Re: Notes from the October NANOG meeting
I am anticpating that your slides will be put up on prdb.merit.edu, so I don't think I will need to get your slides since they will have them as well as my notes on their server.
I was going to followup my notes (which I was trying to keep opinion nutral) with another note with some of the various opinions expressed about what was said. You have done part of that for me with your analysis of the NAP talks.
Thanks for your comments.
--
Stan | Academ Consulting Services |internet: [email protected]
Olan | For more info on academ, see this |uucp: bcm!academ!sob
Barber | URL- http://www.academ.com/academ |Opinions expressed are only
mine.
________________________________________________________________________________________________________
3
Date: Thu, 27 Oct 94 15:13:37 -0400
From: "Matt Mathis" <[email protected] style="">
Thanks for the notes!
My comments are belated responses to the participants, since I was unable to be at the the meeting.
>Throughput degraded with the TCP window size is greater than 13000 bytes.
We never use a maximum window size this small. Our system default is 32k which is 1.5 times the actual pipe size for a T1 connected site at 120 mS. This is near optimal for typical users on the west coast, who are one or two T1 hops away from the current NSFnet. This is slightly too aggressive for typical users on the East coast. But it is an order of magnitude too small for many of our users in Boston, San Francisco, Champaign-Urbana, etc. furthermore, I believe that a number of vendors are shipping workstations with large default window sizes, including SGI IRIX_52 on all platforms and OSF/1 for the DEC Alpha. A 13000 byte maximum window size is insufficient.
I would like to "second" Curtis' remarks about the impact of round trip delay on traffic burstyness. The essence of the problem is that TCP controls the total amount of data out in the network, but has no control over the distribution of data within one round trip time. Slow start and the "turbulence" effects discussed in Lixia Zhang's paper on two way traffic (sigcomm'92) tend to maximize this burstyness.
I have recently become aware of a weaker criteria for success that should also be considered.
If you imagine a infinite bandwidth network with finite delay and loss rates, TCP will run at some finite rate determined by the delay, MTU and loss rate.
A quick, back of the envelope calculation (neglecting many possibly important terms) yields:
BW = MTU/RTT * sqrt(1.5/Loss)
Or for the BW to be congestion controlled
Loss < 1.5 * (MTU/RTT/BW)**2 (please excuse the fortran ;-)
So for Curtis to reach 40 Mb/s with a 4k MTU and 70 mS RTT, the TOTAL END-to-END loss must have been less than 0.02% of the packets. Since each packet would be about 1000 cells.....
To reach 10 Mb/s with a 1500 Byte MTU, the same path needs to have better than a 0.05% end-to-end loss rate. PSC did a demo with LBL at NET'91, on a two month old T3 NSFnet (actually running at half T3) where we achieved near these rates (the RTT was only 20 mS so the loss might have been as high as 0.12%.)
Practical experience suggests that this calculation is not pessimistic enough, and that actual loss rates must be significantly better. For one thing it assumes an absolutely state of the art TCP (taho is not good enough!), otherwise performance drops by at least an order of magnitude.
--MM--
________________________________________________________________________________________________________
Last Updated: 23 April 1995
Back to the main NANOG2 web page.
Media Releases
NANOG Meetings
