KPI, Scorecard, Dashboard - what's in a name?

Is it just my imagination or are the words "KPI", "Scorecard" and "Dashboard" appearing in spec documents with increasing frequency and (sometimes) decreasing thought? Just because a requirements document states deliverables of KPIs, scorecards and dashboards does not necessarily mean that using PPS elements that happen to share the same name represent the most appropriate solution.

This is not a post about what I think KPIs, scorecards & dashboard are or aren't, neither is it a post about any shortcomings in PPS functionality. What I'm focusing on is the sometimes casual use of the words "KPI" "scorecard" and "dashboard" in requirements documents. When it comes to a (potential) PPS deployment this has can skew the solution that is eventually delivered and potentially limit it's future capabilities and flexibility as an organisation's BI maturity grows.

Scorecards

In the heady days of BSM I blogged that people often expected a scorecard to do the job of a report and were disappointed when the product did not deliver on their expectations. "I guess it's just not fully featured yet..." they'd say. I still maintain that this disappointment was misplaced as expectations were centred around their idea of what a scorecard is for when it came to BSM and, more recently, PPS.

The PPS team has done a great job in improving the flexibility and functionality available to us when it comes to building out scorecard elements. In many ways I consider this to be a double-edged sword. So much new layout, aggregation and formatting power has now been given to the scorecard element that many new users of the technology are once more looking to the scorecard to deliver functionality that it is not best suited for.

There are many things that PPS scorecards do well but these are the ones that I think are most relevant:

1. enable hierarchical arrangement & nesting of KPIs
2. manage KPI weights
3. normalise KPI scores
4. roll up disparate KPIs

To me these are the things that make a scorecard what it is. A scorecard is there to provide an overall insight to an organisation / business unit / business area. Either we're heading in the right direction, in line with the strategic objectives of business, or we're not. If not, why not? That overall insight is materialised by the scorecard using the features above.

Scorecards facilitate comparison and roll up of figures that are normally incomparable with a simple reporting tool. Try rolling up your gross profit, cost of sale % and the number of customer complaints last month. Adding $1,436,814, 7.54% and 36 is not going to give you a very useful figure. Nonetheless these are useful KPIs that can assist in measuring the overall health of a business. Creating a KPI for each and arranging them in a scorecard could be the genesis of the very powerful business tool. Why? Because the inner workings of the scorecard element do all the heavy lifting to normalise and roll these numbers up - they can now be compared in the context of the scorecard. It's the normalisation of KPI values and subsequent weighed roll up that is the killer feature of scorecards.

Often the output users want when they say "scorecard" is the next level down from a scorecard like the one depicted above; the more granular detail of what is causing our customer complaints to increase, what department are they increasing for? The high-level scorecard simply indicates that the level of customer complaints has breached it's defined threshold.

A scorecard is not a report. More often than not people use the "scorecard" to describe [what I would consider to be] a report that uses some form of visual indicator like a smiley face or traffic light to denote good or bad performance. Now this is not to say that they are wrong in their use of this term, the definition of a scorecard is not set in stone. Just be wary not to let someone's use of this word steer you inexorably towards the creation of a PPS scorecard element without considering the best way to deliver what's required. If a breakdown of customer complaints by department with subtotals is required, how about an SSRS report? Just because PPS scorecards do aggregations and support the arrangement of dimension members on row and column axes does not necessarily make them a replacement for Excel pivot tables or the data regions supported by SSRS. Don't try and push the data into a scorecard just because requirements ask for a series of green ticks or a red crosses and uses the word "scorecard" to describe what's needed.

KPIs

PPS KPIs are very flexible elements. Their real strength lies in the ability encapsulate logic centred around a particular metric and compare actual with desired performance. These KPI elements can be hierarchically arranged, weighted and rolled up in a scorecard. KPIs do their best work and provide the greatest reuse when the the majority of logic and data required for their definition is stored and managed in the back end. i.e. the definition built in Dashboard Designer is kept simple. Another feature of well-designed KPIs is a common structure and naming standard for Actual and Target metrics.

Overly complex and over-engineered KPIs (many targets, differing metric names, lots of MDX) can certainly lead to a solution but just how reusable are these elements? Can they be reused in other scorecards, or were they purpose built to "make scorecard X look like the picture in the spec"?

Just because someone uses the term "KPI" does not always mean the creation of a PPS KPI element, an SSAS KPI or MOSS KPI is necessarily the answer. Many immediately equate the use of a smiley face, traffic light or background colour formatting with a KPI and subsequently a scorecard. Often the requirement is as simple as emphasising a positive or negative delta to a budget or forecast. Does this kind of comparison need to be made into a physical KPI? Possibly, but not definitely. Consider the alternatives. Often this kind of requirement can be fulfilled simply and elegantly with an SSRS report. Because you paid $$ for an application doesn't mean you have to use it more than the tools that came bundled with the SQL licence. Don't fall into the "when the only tool you have is a hammer, everything looks like a nail" trap. Use the right tools for the right job.

Square peg? Round hole? No problem.

Bart, get me my sledgehammer!

Just because KPI, scorecard and dashboard are used when describing the solution does not mean you must use these PPS elements to expose the data. Doing so can lead to a top-heavy, logic-ridden BI presentation layer as a result of the ol' "make it look just like this spreadsheet, please". Please try and avoid this situation at all costs. The square peg, round hole approach will work if you bash at it hard enough, but just how reusable will your elements be? How sensitive to change will your solution be when users start to wake up to the potential of BI and come up with all sorts of new ideas? I like Bill Inmon's quote of BI end users: "Give me what I ask for, then I'll tell you what I really want".

Dashboards

Whether you agree with someone's use of this word is immaterial. Try to figure out what they really want when they say "dashboard" before diving into solution mode. Could it be that they're simply referring to a report with a few different data regions and some dynamic navigation interactivity? Maybe. Maybe not... but it's still worth thinking about for a bit.

The dashboard PPS element is a fantastic addition to the product and provides enormous potential to build great dynamic, interactive content that is accessible through WSS. Nonetheless don't let the use of "d" word can muddy the waters as to the best way to deliver the solution.

Keep it Simple

Simple, reusable, modular elements are the hallmark of a good M&A implementation. One of the design goals for the PPS team was to make the creation of elements simple and easy, another was to facilitate reusability of elements. Sure there are a ton of areas where you can add a little bit of extra code (particularly MDX) to create some really cool dynamic behaviour in KPIs, scorecards and dashboards. I'm a big fan of this functionality, so long as it's not overused. The code required should itself be as simple as possible. If you're beginning to see nested IIF statements housed anywhere within your BSWX file, stop - it's too complex. Let your ETL and multidimensional data source do as much of the work for you as possible. Don't tie up all your logic in the front end.

Consider this situation: you have filled your PPS elements to the brim with custom code to get a scorecard/dashboard/KPI looking "just the way it does in the picture" and then the user says "that's great, can you please reproduce that same logic in a separate report?" Where does that leave you? D'oh!

Creating a good dashboard (with any BI application) should be simple and quick. This simplicity and speed of creation is borne of a solid back end foundation, the use of the right tools for the right job and keeping it simple. If you are skirting the fringes of the application and using hard-coded hacks to make things look "the way the user wants them to", take a step back and reconsider your approach.

In Summary

Just because someone utters the words KPI, scorecard or dashboard doesn't necessarily mean that the end solution should be delivered using those precise elements. Again, please note that I am not stating that people are right or wrong in their use of these terms. My point is to be wary that just because someone uses these words to describe what they want doesn't mean you necessarily need to create PPS elements that share the same name.

More often than not users refer to a report with a couple of traffic lights on as a scorecard. Don't assume that everyone is thinking Kaplan-Norton, normalized score calculations, cascading KPIs and weighted rollups when they say "KPI" and "scorecard". They may not thinking along the visual and analytical lines of Stephen Few or Wayne Eckerson when they say "dashboard" either.

Analyse and understand what users want and deliver it with the most appropriate tools. Do not bash the virtual square peg into the round hole with a PerformancePoint golden hammer.