DragonFly users List (threaded) for 2006-10
Re: Network Slowdowns?
Matthew Dillon wrote:
Packet overhead for bulk data transfers is only around 3.4%. The
ethernet MAC is 12 bytes. The TCP/IP header is 40 bytes (20 bytes
for UDP/IP), plus additional bytes for options, and the data payload
is usually around 1448 bytes.
Ethernet framing overhead is harder to calculate. 10BaseT is easy,
it's just manchester encoded at 20MHz -- so 8 bits per byte at 10MHz,
basically. I think there is a short preamble and a CRC tacked on.
100BaseT encodes 4 bits into 5 over the wire (10 bits per byte), but
the line frequency is actually 125MHz, not 100MHz, so one byte of
data takes 10 bits @ 8ns/bit = 80ns == 12.5 MBytes/sec. If you
subtract the 3.4% packet overhead you get around 12 MBytes/sec. I
think there is also a premable and some other junk there that
reduces the effective bandwidth to 11 MBytes/sec with 1500 MTU packets.
GiGE also uses a line frequency of 125MHz, encoding 2 (or is it 3?)
bits into each 8ns period. But (and now I am looking at the Wikipedia
entry for GigE) it transmits in both directions on all four cable pairs.
So you get 125 x 2 x 4 = 1 GBit. Each 8 bit byte is converted to 10 bits.
It's all very confusing, actually. GigE requires a lot of signal
processing and uses echo cancellation to work.
CPU overhead is a different beast entirely. SSH has a lot of encryption
and decryption overhead... my transfers over localhost top out at
21.8 MBytes/sec on my test box. At 100BaseT speeds the packet rate
is only around 6000-8000 packets per second and cpu overhead for packet
processing is usually in the noise. At GiGE rates, however, the packet
rate is over 66,000 packets per second and cpu overhead can be
significant even if you discount interrupt and device access overheads.
Stuff all well and true - as far as it goes, i.e in the lab, and under
controlled conditions (no other daemons running, for starters).
Now .. put yourself in the shoes of a field-Engineer armed, in ascending order with:
- 'promiscuous' mode IP NIC settings:
"Gee - there are IP packets eating up b/w that aren't mine at all..."
- protocol analyzer:
"Hmm.. and ethernet traffic that isn't even IP protocol.."
Holy *shit* what fool routed the CAT5 next to the power cables for the
Used to laugh at the folks from our C&W 'sister company' on Ethernet frantically
shouting across the office and phoning each other 'shut down your PC!' around 4
in the afternoon as air-mass thunderstorms overtook Tyson's Corner, Virginia.
Our business unit kept right on working.
ARCnet/TCNS on coax basically ignore lightning strikes a mere 500 meters away.
Ethernet doesn't have that luxury.