Here’s a great quote about performance metrics that has nothing to do with IT.
Writing in his “Baseball Abstract, 1985,” Bill James makes these points about statistics:
1. Baseball statistics have the ability to conjure images
2. Baseball statistics can tell stories.
3. Baseball statistics acquire from these other properties a powerful ability to delude us.
For those of you who aren’t baseball fans, James is the man largely responsible for a massive revolution of statistical analysis that has changed baseball. His overall point is that the numbers we use to gauge performance – and value – in baseball can easily mislead us into conclusions about certain players. This results in poor investments and badly constructed teams.
Pitchers with lofty win-loss records are often the beneficiaries of great run support. Hitters who pile up runs batted in (RBIs) are getting to the plate with a lot of people on base. Each of these statistics rewards a player for his teammates’ successes to an extent. As a result, we’ve seen those metrics devalued in favor of fielding-independent pitching (FIP), on-base percentage and a number of other stats that more accurately assess individual performance and value.
To understand the meaning behind any statistic, we have to understand the situations that produced the data. In enterprise IT, we hear a lot about certain performance metrics, but making decisions about workload placement, expenditures and other aspects of IT based on raw data can lead to investments in expensive infrastructure components or technologies that don’t really solve problems – the same way baseball teams that pay for pitchers with a lot of wins or hitters with 100-plus RBIs often find themselves spending millions on a player who isn’t delivering.
You can’t isolate IT performance and availability from the processes they’re completing. They depend largely on the type, volume and primary activities of the workloads in question. Just because a certain cloud service is an ideal fit for one application doesn’t mean it’s suited for all applications on which a company relies. Latency-sensitive processes, such as financial transactions or loading medical records, need to be housed in environments equipped to deliver consistently high performance levels just as pitchers who thrive on generating groundball outs need great infield defense behind them to be effective.
Like James said about baseball statistics, performance metrics take on the same properties of conjuring images, telling stories and deluding IT decision-makers into thinking a VM or server can handle a critical workload because it performs well in other circumstances. Performance analytics must be calculated with data from every part of the IT stack to accurately point decision-makers to the root causes of problems. If you neglect even one aspect of data or focus on broad, non-specific datasets, the result is a skewed understanding of IT processes.
James published his first abstract to little fanfare. However, his influence on the game gradually became significant enough to draw the interest of the Boston Red Sox. James has consulted for the team off and on since 2003, assisting in the construction of three championship teams. Baseball and enterprise IT may not, on the surface, have much in common, but the overarching theory behind performance analytics in any field is simply to understand processes as granularly as possible to make the best decisions about spending money. So, when it’s time to solve an infrastructure problem, don’t be fooled by some gaudy statistics that don’t tell the whole story. We’ve all heard the phrase “lies, damned lies and statistics,” but the numbers never actually lie. You just have to ask the right questions.
Does it seem like your infrastructure needs a new player? VirtualWisdom4 can show you exactly what you need.