Stop Broadband Blindness

See last mile packet loss, latency, jitter and more

April 10, 2024

AHHHHHH!! NOT NOW!! Sound familiar? Perhaps those are the words you silently screamed when your Internet connection either went down or degraded so badly due to packet loss that your web browsing became erratic or your voice or video call started breaking up.

But how do you prevent the next occurrence? How much faith do you have that your ISP will actually do something when you call to complain?

The problem is pervasive – last-mile Internet circuits are getting more overloaded by the day, and network providers frequently don’t keep pace by provisioning more bandwidth. When problems occur, it’s not in their interest to notify their customers that circuits are down or congested, and without hard proof of the network issue, it’s a perpetual finger-pointing exercise as the ISP declares “it’s not us.” Companies can no longer afford to be afflicted by #BroadbandBlindness and Firebind can help.

What defines a good broadband connection? Many people first think of bandwidth since that is the most common metric by which network circuits are sold, such as a broadband 500 Mbps up / 500 Mbps down circuit. But bandwidth is only a small part of the equation. Latency is another factor. It’s basically the round-trip-time it takes for a packet of data to go from your machine to the remote destination and back. And jitter is simply the variation in the latency. But the metric that is the true scourge of the Internet is really packet loss.

Packet loss is the villain that causes your VoIP call to break up or your video to pixelate or stop working altogether. It can also be the cause of slow web browsing as the TCP transport layer that the HTTP protocol runs over tries to re-transmit dropped packets. There are 2 big problems when measuring packet loss. First, you want a solution that is always sending test packets because without continuous testing, you can’t see when there is a problem. If there is no train going down the track, you can’t tell the track is safe, not even by looking at it. This is the primary drawback of a passive only monitoring approach because you can’t monitor for something going missing when it was never there to begin with, just like you can’t monitor for dropped packets on a VoIP call at 3am if no one is making calls at 3am.

The second issue is sending the test traffic. Ping can be helpful, but at a rate of 1 per second, it simply doesn’t stress the network like a VoIP type call at 50 or 100 packets per second would. And to make matters more challenging, network switches and routers frequently de-prioritize ping. Iperf is another open source tool and while it can emulate the bit rate and payload size of voice and video calls, it’s highly manual and not designed to run continuously.

Enter Firebind Reflector. Reflector was designed from the ground up to be a rapidly deployable solution that can continuously stress network connections and alert when certain thresholds are either not met or are exceeded. Firebind Reflector can be installed at the remote site on virtually any device, including PCs, Macs, Linux servers, virtual machines, or even Raspberry Pis, then configured to continuously execute tests to various test points, including hub locations in the cloud.

In the image below, a Firebind Reflector UDP packet loss test was configured to emulate a typical VoIP call between the Firescan command line client running on a Raspberry Pi and a $5/month Ubuntu VM running on Hetnzer in Virginia.  By running the test every 5 minutes for 15 seconds in each direction, we can see frequent instances of upload packet loss that are severe enough to greatly impact call quality.

In the next image, a TCP bandwidth test was run every 5 minutes as part of the same test suite.  Since TCP bandwidth tests will generally try to “max out” the link, the Ubuntu tc (traffic control) application was used to limit the egress bandwidth for both the initiator (Raspberry Pi) and target (cloud VM) using 3 Mbps and 10 Mbps limits respectively.  When comparing the above image (UDP loss) with the below image (TCP bandwidth), it’s very easy to see how during times of severe packet loss, the TCP bandwidth was impacted to a similar degree.

The next image shows that despite the packet loss issues, the jitter values of 1 to 3 milliseconds were very good and didn’t indicate any sort of jitter issue on this network path.

The final image shows HTTP response time from the Raspberry Pi to www.google.com.  It’s important to note that the test device was a Pi model 2 therefore the typical response time of 2,000+ ms was longer than a typical PC would experience, but the goal here is to look for the deviation in response time.  We clearly see multiple instances of response time more than tripling that correspond with the most severe packet loss timeframe.

Legacy open-source and browser-based solutions provide only a partial, point-in-time view and therefore don’t provide the continuous testing needed to not only spot trends, but to have a historical view for when a transient issue has stopped occurring before the troubleshooting has begun.

With Firebind Reflector’s easy-to-deploy architecture, users can continuously and actively monitor all facets of network performance between the two configured test locations to achieve true 24×7 visibility without the expense of a costly enterprise solution.