Business Scorecard Manager – What’s the Score?

I have been waiting for the day a client asks me how the score of a KPI is calculated. Deciding that I wanted to avoid resorting to the “dunno” response I started searching but couldn’t find much in the way of a formula or other decent explanation for this mysterious number. I posed this question on the BSM newsgroup in an attempt to find out more. When I got a response from "Neil I" I fired a quick email to BSM PM Ian Tien to see if he had anything further. He sent me some documentation from the original Business Scorecard Accelerator. I certainly cannot be considered a mathematician or statistician, so for those of you who are please excuse me if I use the incorrect terminology. Here we go…

What is the KPI Score?

From Original Business Scorecard Accelerator doco:

“In balanced and general scorecards, one or more KPI values are rolled up to determine the value of the objective to which they belong. Because KPIs can measure different kinds of data, which are often quantified differently (currency vs. integers, for example), Business Scorecards uses a calculation method to determine KPI values on an evenly distributed scale (also called a "score"). After the individual KPI values are placed in the context of the evenly distributed scale, a value for the objective is calculated. Similarly, objective values are used to determine the value of a perspective.”

NB The quote above refers to Objectives and Perspectives which were explicit containers in the original Scorecard Accelerator. BSM gives us more flexibility as to how we group our KPIs in each individual scorecard but the purpose of score remains pretty much the same.

Where do I find / display the score?

A score is always calculated for each individual KPI. You can make the score of a KPI visible in the Specify Actual and Targets dialog box when configuring a scorecard view.
















If you choose not to explicitly display the score on the scorecard it will still be visible as a tooltip when hovering over the indicator of the KPI itself. Many people mistake this number for the figure which drives the indicator image – this is wrong.












The comparison of actual to target (depending on the banding method used) is the value which determines the colour/shape etc. of the indicator but the score does not have any effect on what KPI icon is displayed at the leaf level of a scorecard. Score does have an effect on the displayed indicator as soon as we begin to roll the leaf-level KPIs up.

How do we get an evenly distributed score?

If you do not normalize to a linear range according to consistent thresholds, you can’t roll up KPI values. In order to make the scores evenly distributed there needs to be something consistent underlying all score calculations.

BSM finds out in between which of the KPI target’s user-defined target Boundaries the (Actual or Normalised) value lies. Once it knows that the value falls between BoundaryX and BoundaryY it then determines what the linear range upper and lower thresholds to use in the calculation by determining what the values for BoundaryX and BoundaryY are for an n-level indicator. BSM uses a formula along the lines of

(1 / # of Indicator levels) * Boundary number

For example, my KPI target uses a 3-level indicator and my value (11) falls between the target Boundary1 (3) and Boundary2 (15) values I defined. Now that I know this I can use the table below to establish that the BSM score calculation will use

(1 / 3) * 1 = 0.333
and
(1 / 3) * 2 = 0.667










So what’s the formula?

Example 1

Your KPI target is based on a 3-level indicator, using Band by Numeric Value of Actual and Increasing is Better. You set the target Boundary1 = 5 and Boundary2 = 10. Your actual value is 7. Because the actual value (7) falls between your defined Boundary1 (5) and Boundary2 (10) the score will use linear threshold values of 0.333 and 0.667 for its calculations i.e. Boundary1 & Boundary2 for 3-level indicators.















The score would be calculated thus:

(7 - 5) / (10 - 5) * (0.667 - 0.333) + 0.333 = 0.467 (46.7%)

(Actual - Target Lower Threshold) / (Target Upper Threshold - Target Lower Threshold)
*
(Linear Upper Threshold - Linear Lower Threshold) + Linear Lower Threshold

Example 2

Your KPI target is based on a 5-level indicator, using Banded by Normalized Value and Decreasing is Better. You set the target Boundary2 at 25% and Boundary3 at 50%.















Your actual is 260 and your target is 200. Remember that you also have a Worst Value to adjust if need be. You set this to 400. These numbers result in a normalized value of 0.3 (30%) the calculation being:

1 – ((260 – 400) / (200 - 400)) = 0.3 (30%)

1 – ((Distance from Actual to Worst) / (Distance from Target to Worst))

OK, so 30% sits between your defined Boundary2 (25%) and Boundary3 (50%) therefore the linear thresholds for these for a 5-level indicator will be 0.4 and 0.6 (refer to reference table used earlier)

So to work out the score:

(0.3 - 0.25) / (0.5 – 0.25) * ((1 - 0.6) - (1 - 0.4)) + (1 - 0.4) = 0.56 (56%)

NB because we are using Decreasing is Better we have (1 – threshold value) for each of the internal boundaries in the second half of the formula.

The Rollup

So as a result of the two examples above your rollup score would be calculated

(0.467 + 0.56) / 2 = 0.5135 (51.3%)










So now, if and when a client asks me how KPI the score is calculated I can tell them. Now all I have to do is wait to be asked…

UPDATE (11 Feb 07): A few slight technical adjustments