Metrics, a Tool to Measure IT Operational Performance

A passion of mine outside of work is following Major League Baseball. While many aspects of the game have changed over the years, many fans in my age bracket agree that there is too much reliance on analytics, which diminishes the human element from the sport. I recognize the value of baseball analytics as a tool; however, using analytics as the primary or sole means to construct a lineup or to make in-game decisions may not produce the desired results. I have noticed similar parallels in the overuse of analytics in Major League Baseball and in how metrics are used in IT Service Management (ITSM).

Metrics can be useful took to measure IT Operational performance and servicing value, but much like analytics in baseball, if metrics are overused, the desired outcomes are not achieved, and IT could be misaligned with their customers. For example, in the past, I worked for an organization where, by my estimations, the patch compliance team was achieving a 98% or greater success rate over a period of several months. However, the organization used a strange method for calculations, which showed the patch compliance team as below their SLA of 95% or better measured on a monthly basis. I suggested to the team manager, they should revise how they were reporting, but I didn’t have the authority or influence to implement new approaches, and so nothing changed.

The reality was that most of the servers were compliant each month, but due to the way results were calculated and reported, the customers thought there were ongoing security issues and that the team was regularly failing.