When Internal Metrics Miss Their Mark

In a recent episode of the InformationWeek Podcast, two chief technology officers discussed how internal metrics can sometimes steer IT projects away from the outcomes they were meant to achieve.

CTOs weigh the risks of metric‑driven optimization

Jerry Shu, CTO and co‑founder of Daylit, and Aaron Harris, CTO at Sage, opened the conversation by noting that the drive for faster development and deployment often overlooks the context in which performance numbers are generated. “What looks good on a dashboard can hide the real story,” Shu said, emphasizing that metrics should be tied to business realities, not just technical targets.

The pair highlighted that many organizations rely heavily on IT teams to design and monitor internal metrics, but that approach can miss critical input from other departments. Harris argued that involving product managers, finance staff, and even frontline employees can provide a broader view of how a metric impacts day‑to‑day operations. “When you bring more voices to the table, you’re less likely to chase a number that ultimately hurts the customer experience,” he explained.

Both executives recounted instances where an optimization plan—intended to speed up software releases—ended up creating bottlenecks elsewhere. The episode highlighted that an over‑emphasis on speed can distort the balance between development velocity and system stability.

When a metric goes off course, how do leaders respond?

Shu and Harris described a structured approach to course correction. First, they recommend a quick audit of the metric’s source data to confirm its accuracy. If the numbers appear sound, the next step is to evaluate whether the metric aligns with the organization’s strategic goals.

They also introduced a “Questionable Ideas” tabletop exercise, using a fictional company plagued by resource choices that seemed beneficial but introduced hidden risks. The scenario featured whimsical elements—goblins, gremlins, and kobolds—to illustrate how seemingly harmless decisions can expose an organization to unforeseen trouble. By role‑playing the exercise, participants learn to spot red flags before implementing costly changes.

During the discussion, the hosts emphasized the importance of having a fallback plan. If an optimization effort fails to improve operations, a clear rollback strategy can prevent prolonged disruption. “You want to know ahead of time how you’ll unwind a change,” Shu noted, adding that documenting the decision‑making process helps teams learn from past missteps.

In practice, companies that have faced metric‑driven setbacks often turn to cross‑functional review boards. These groups assess the impact of key performance indicators on both technical and business outcomes, ensuring that any shift in strategy is vetted from multiple angles. Such oversight can mitigate the risk of “metric tunnel vision,” where teams focus narrowly on a single number at the expense of broader performance.

Comparing this to past efforts, many firms have historically leaned on IT alone to define success metrics, especially during early cloud adoption phases. Over time, the pattern has shifted toward more inclusive governance, reflecting a recognition that technology decisions reverberate throughout the entire organization. This evolution mirrors broader industry trends where collaboration across departments is becoming a norm rather than an exception.

The episode concluded with a call for listeners to share their own experiences when internal metrics led to unexpected results. The hosts invited feedback via a dedicated email address, indicating an openness to community input and a desire to refine the conversation around metric‑driven IT strategies.

Metrics matter.