Buffering and/or delay are just two sides of the same coin w/r/t your phone call. There could be network delays, delays in the transcoding servers, delays in the backend providers, delays in the telo, etc. Some of these delays might involve buffering while others might involve very little, if any buffering. The key is to isolate where the delays are happening and get the sum total within a range that is usable.
Using your numbers, you could have one server hop of 1ms buffering 10KB before sending anything and that could account for 400ms (1way) or you could have no buffering with 400 hops of 1ms each accounting 400ms delay. More likely you have some combination of buffering and network delays. In the end, it only matters that you get the sum total down to something reasonable by isolating the delays that are not normal. Clearly the design itself is capable of reasonable performance because at some point it was usable.
Good points. But Internet cannot be causing this amount of delay. Lets discuss your point "...one server hop of 1ms bufferi[s] 10KB before sending anything and that could account for 400ms…"
It would actually take much more than 10K to buffer 400ms. Remember, your packets are mixed in with a huge number of other packets. If an Ooma packet is delayed at a hop, so are all the others. Internet switches simply don't have enough memory to buffer 400ms of data. Here's why.
At each hop, an input fiber from a distant hop is connected to an input port on the switch. The input port contains a line card (LC) that converts optical data to electronic data. IOW, an optical packet streaming into an LC fills an electronic ("SRAM") buffer. The LC looks at the header and decides what to do with it, where to forward it, etc. This takes a very short amount of time. Meanwhile, as you state, the packet has to be buffered.
Assume the fiber speed is 2.5 Gbits/sec speed. If packets are held for 400ms, how much memory would the LC need to buffer this traffic: 125 Mbytes!
(2,500,000,000 bps x 0.4 sec / 8 bits/byte). Line cards absolutely don't have this much memory for buffering.
Let's look at it from a different angle and assume an LC does have this amount of buffering -- 125 MB. IP packet lengths vary from about 50 bytes to 1,500 bytes, with an average length of about 500 bytes (from Google). It would take 250,000 packets
to fill a 125MB buffer! There is absolutely no reason to buffer 250,000 packets before sending them on.
Each LC has to process more than 700,000 packets per second. The LC has to perform a lookup based on the destination IP address in the packet header to determine which output port/fiber to forward it. But this must be done, and is done, in sub-microsecond time.
Each switch wants to get rid of arriving packets as fast as it possibly can. It is very expensive to buffer packets that are arriving a gigabit/sec rates. No more than a handful of packets are buffered at the LC. The average time it's delayed inside a switch is much less than 1ms. It arrives and is gone -- BAM!
Your other point of "400 hops at 1ms each for a total of 400ms" is important. But a packet cannot make that many hops. There is a TTL (time to live) byte in the header. Every time the packet passes through a switch (hop) this value is decremented by one. If it reaches zero the packet is discarded (not forwarded). The max TTL is 255, which sets an absolute upper limit of 255 hops.
But in reality, packets never even go through half that many. Assume 50 hops at 0.5 ms each, that's 25ms
total delay caused by the switches.
There is also a speed-of-light delay that has to be added to the switching delay. Assuming 3,000 miles of fiber (worst case!) out and back, I get about 27ms
of delay. Adding these, 25ms + 27ms is about 50ms
But all these calculations are irrelevant. The actual delay can be measured.
The VoIP tests show that actual Internet-related delays
are typically less than 50-100 ms. These test results have been reported in the forum many times by many people, at different times and locations. I've run them at least a dozen times and get consistent results. (And they are consistent with the calculated numbers.)
The total delay that we have measured and reported here is 500-550ms. So we have to account for 400-500ms of delay that is external to the Internet: Ooma. We are promised an upgrade to the Telo which addresses this issue.