Why end-to-end testing fails

January 13, 2012

Many folks they think that end-to-end testing will validate their VoIP environment and make them feel comfortable that a VoIP deployment will go smoothly.

I’m gong to go back to my tried and true analogy comparing a VoIP network to the freeway system.

If you measure how long it takes you to drive to work each day, you can get a baseline of your latency and jitter (variance).

The problem with end-to-end testing is that it’s like you’re driving a car with blacked-out windows: When you arrive at work, you’ll have no idea why your normal 30minute commute took 4 hours.

As a result, you’ll never be able to solve the problem (which is the desired result).

The other problem with end-to-end testing is that you typically have to set up a computer or program at the remote end.  This may be difficult or challenging because hardware needs to be shipped around, and deployed in each remote site.

A better solution would be to do what modern freeway system management does: Monitor every freeway link for performance.  If there’s a slowdown or traffic jam, it’s able to be pinpointed when, where, and why the problem is occurring so it can be remedied rapidly.

 

The channel is thirsty – for Total Network Visibility

March 24, 2011

It’s been a whirlwind month for the PathSolutions team with intense interest from nearly every corner of the IT and comm sectors.
As a network engineer, I’ve had my share of frustration with the time-consuming and tedious nature of troubleshooting network issues that impact VoIP call quality and application performance. In the past few weeks, I’ve seen tangible evidence that many others share my pain.  And when they see exactly what PathSolutions can do to resolve this frustration, their eyes light up like someone who’s lost in the desert and then finds a tall cool glass of water.
At the recent ASCII Success Summit, resellers and MSPs came to the PathSolutions booth in droves to learn more about automated network monitoring and root-cause analysis.  Whether their interest was in a managed service offering, network assessment, or enabling customers to achieve Total Network Visibility, the response was virtually unanimous – Wow! ASCII is the world’s largest organization of independent IT solution providers, integrators and value added resellers (VARs) in the world.  They must like what we’re doing– they selected PathSolutions v4.1 as 2011 Best New Product!
The conversations with these networking experts crossed a myriad of topics, but at the root of each was the lack of an easy-to-use visibility tool in the marketplace.  Maximizing resource efficiency is one way these independent businesspeople ensure customer satisfaction and maintain profitability.   A pesky intermittent network issue that causes degraded VoIP call quality could consume many months of profit using old-fashioned troubleshooting.  With PathSolutions’ Total Network Visibility suite, the prescription for resolving the issue is placed at the engineer’s fingertips shortly after the software is installed–and that in itself only takes 12 minutes.
The extraordinarily fast installation and nearly instantaneous availability of actionable solutions also raised some eyebrows at the Enterprise Connect conference – the premier VoIP and telecom show in the US.   With the steady and rapid migration to VoIP for mission-critical applications in the enterprise, network monitoring is garnering a lot of interest, and the immediate ROI of the PathSolutions products really stood out amongst the folks we spoke with.
And finally, at the Channel Partners conference, the communications industry’s only event exclusively for indirect sales organizations, we saw intense interest in how PathSolutions could help these resellers differentiate themselves from the crowd with fast and simple monitoring that offers a comfortable recurring revenue.
We are working hard to satisfy the thirst in the industry. And later this year, we’ll even add a few more ice cubes to the glass – so stay tuned for news about new Total Network Visibility features and capabilities.

IT Expo Summary

February 5, 2011

IT Expo turned out to be a lot busier than I expected — There definitely is an economic turnaround starting, at least in the cloud and hosted communities.

Here are some interesting points that were discovered during the show:

  • More and more Cloud-based services are rapidly coming to market, each solving a wide variety of problems in a very low-cost manner.
  • The cloud-based services are inherently stable and secure, but the networks that convey the services are not.
  • Companies don’t have good tools to troubleshoot their own internal networks, let alone extranets and the cloud.
  • Troubleshooting performance and stability issues take way too long, and are resulting in finger-pointing between the cloud services vendor, the network provider/carrier, and the local network and application teams.

The takeaway from this is that many enterprises that have converted to VoIP have call quality and stability problems within their own network.  If you add the cloud to this picture, it’ll only get worse.

Presenting at IT Expo East 2011

February 2, 2011

I’ve been invited to speak as a guest lecturer at IT Expo East 2011. It looks like a packed show with lots of network & telecom experts both attending and presenting — I’m honored to be associated with a group of folks like this.

Feel free to join my program if you’re in the area: Dark Clouds : Can you trust the cloud? Latency, Security and Availability (CL-06). The session is on Feb 4th at 10am.

Publishing

July 2, 2010

I’ve discovered a great way of getting the word out about discovering easier ways to find and resolve network problems: Getting published!

Telecom Reseller has picked me up as a columnist, so theres another place to learn about network monitoring best practices:

http://www.telecomreseller.com/category/writers/titus/

Global Connect 2010 Show Report

April 20, 2010

The show floor is vibrant and filled with folks wanting to add to their telecom environment. It’s really great because it reinforces my belief that the economy is on the mend.

What’s bad is that there’s a lot of FUD (Fear, Uncertainty, and Doubt) being presented to folks. I find it upsetting when vendors promise everything and tell folks that they cure the entire problem when in fact they only answer part of the problem.

I guess it just goes back to the “buyer beware” mode, and make sure you understand what you’re buying so you don’t get caught spending your budget on something that doesn’t solve what you wanted to solve.

Green-Running Software

January 15, 2010

Saving money by reducing the number of servers in your infrastructure makes natural sense.

The next step that needs to be considered is green-running software.  This is software that scales incredibly well for any size of organization, and also does not require “big iron” to run.

If you have software that requires a dual-processor environment with 2gigs of RAM and a large disk array, it will cost you twice as much to operate as one that fulfills the same needs yet only requires one low-powered CPU and a gig of RAM.

This becomes even more important if the software requires remote agents to be deployed across the network, each agent taking a measured amount of resources to operate (CPU, RAM, disk space, air-conditioning).

Strong consideration should be given to the solution that requires less maintenance, less hardware, and less footprint on the earth, as it will save far more than money in the long-term:  Your sanity.

Packet Analysis and VoIP: Useless?

December 23, 2009

Many people think you need to “look inside the packets” with a network analyzer to be able to debug & troubleshoot VoIP problems.  I would argue that this is rarely the case, as network analyzers are useful in a small handful of cases and most involve troubleshooting application configuration problems.

If you look at an overnight courier like UPS, their operating model has a lot of similarities to a large VoIP network.  Clients send & receive hundreds of thousands of packages across their network on a nightly basis.

If you look at how they operate, they don’t tear open packages to try to determine why a shipment went missing or why it is late. They learned a long time ago that they needed to “watch their entire network” and have tracking systems that can insure:

  1. That the packages make it to their destination (no packages lost)
  2. The packages all reach their destination on-time (low transit latency/delay)
  3. The packages reach their destination on-time as a regular occurrence (high degree of predictability with their service/low jitter)

When running a VoIP network, it’s rarely beneficial to “look inside the packets” to see the voice contents. What’s needed is the ability to “watch the entire network” so problems can be pinpointed when, where, and why they occur for fast remediation.

Flying Blind on Your Network

November 30, 2009

Many network administrators are satisfied with a limited view of their network.  They ping their routers and switches and configure utilization monitoring of their WAN links and that’s as far as they go.

What’s terrible is that they paid for some really nice enterprise-grade switches and routers that have some serious smarts built in.  Too bad they aren’t getting the full value out of these devices.

The problem lies in the fact that the “smarts” for these devices are trapped inside the SNMP agents and require a lot of programming & configuration to extract the information, then you have to do some manual analysis to make sense out of the information.

This is what frustrated me a long time ago.  I knew that the routers & switches in my network new where the problems were, but getting them to disclose this information took a lot of work and thinking.

If only there was a solution that would disclose the information in a useful, plain-English format…

Why Autonegotiate Fails

September 17, 2009

The IEEE wrote the 802.3u standard that defines auto-negotiation for Ethernet interfaces. The problem is that the standard was optional, and also open for interpretation.

As a result, many devices don’t correctly auto-negotiate and your network ends up with duplex mismatches, 10meg speeds that should be set to 100meg, and collisions when you should have none.

This problem continues to plague network administrators, as it can take many hours to track down exactly where the problem is,  only to discover a duplex mismatch that can be fixed in under 30 seconds.

Thus, finding the “needle in the haystack” in the first place is the difficult part of the process.

The user typically complains about “network slowness” or a “poor quality VoIP call”.  That leads to 4-5 hours of troubleshooting.  In many cases, the problem isn’t evident at the time the engineer checks due to the network being a very dynamic environment — the problem tends to go un-resolved and the user remains dis-satisfied.

What you really need to do is to be proactive and root out the duplex mismatches so they won’t cause problems down the road.