| |
Lagging or Leading
7 March 2014 • Ype Wijnia
program management
One of the difficult problems in setting up an asset management organization is always the choice of indicators by which performance can be measured. The most obvious indicators are of course the business values which are captured in the risk matrix. If you base the decisions on investments and maintenance on these values, then it can be measured what the score is on these values. Otherwise, you can never determine whether your implemented plans have been effective. The big question is whether this is enough, and if not, what performance indicators should be added. In this column, we will try to shed some light on this subject. First we look at the problems you can have if you only look at the business values alone, then we make some suggestions for additions.
One of the major problems of measuring the actual performance is that it was a measurement afterwards. So it tells you how the performance was in the past, not necessarily how it is now. Think of the Titanic: until it ran into an iceberg, the performance was perfect. But then things went wrong. Once it was clear that it would go wrong, nothing could be done anymore. In terms of control, performance information of the past is not always meaningful.
This is somewhat dependent on the situation: if the information is measured short-cycled in a stable environment, this does contain useful information. But the longer the period of measurement is in respect to the changes in the environment, the less the signals contain control information from the past. Compare it with steering a car. The speed is basically a performance indicator of the past: how many kilometers are traveled in the last hour, though the actual measurement will probably on a second basis. If you drive at a relatively constant speed, the information is useful enough to avoid speeding tickets. It will be different if you are driving too fast and brake very hard just before the speed trap. The speed will change so rapidly that you cannot say what it is exactly. It is then to wait for mail including the speeding ticket from the authorities to determine your actual speed, but that takes quite a few days.
A slightly longer time scale is in the information that you get through the mirrors. Although the mirrors do not tell you whether someone is driving in front of you, on a highway you probably succeed to stay on track purely with the mirrors. In such an experiment you will notice that the further the mirror is looking back, the harder it is to keep track if there is a turn[1]. In the rear mirror you can see the road 100 meters behind you, so some 4 seconds ago. But with the right mirror adjusted strongly downward, this allows you to see the road directly behind you or even next to the car. By keeping a certain distance from the road mark you should still be able to keep on track reasonably in a curve.
Another problem with performance indicators is that the measurement value may not be only influenced by the selected measures, but also external factors play a role. Perhaps the best example is the Energy Agreement that was signed last year in the Netherlands[2]. One of the themes had to do with energy consumption at home. The energy consumption of households is 32% of the total, thus substantial savings can be achieved there. In order to measure the effectiveness of the energy consumption, we compare the consumption before and after the agreement with each other. And what happened? The year following on the agreement significantly less energy was used, focusing on the winter 's gas consumption it even dropped with 20%[3]. I think none of the parties to the agreement had expected such a great result. The initiator of the energy agreement should therefore immediately get a medal, plus the Nobel Peace Prize (for securing the future of the earth), Physics (because of the unique insulating properties of the document), Economics (because of the invention of the desire-driven economy instead of the market) and Literature (apparently because of the great inspirational texts in the agreement). Or do we simply fall into the Post Hoc, Ergo Propter Hoc pitfall? I do not want to spoil the Nobel party, but might it be, as a purely theoretical exercise of course, without trying to draw the eternal equal of the Energy Agreement in question, there is a very small chance that this energy saving may be the result of a natural swing, let's say (purely as a thought experiment) exceptionally mild winter of 2013/2014?
Of course, everyone knows that the energy saving of last winter is a random effect, because everyone knows that you have to heat more when it is cold, and less when it is hot. The relationship between these two can well be made quantitatively with the so-called degree days. If we now use the gas consumption per degree day as a benchmark, something sensible can be said about the effectiveness of energy savings, even if the actual energy consumption differs significantly.
But if the relationship is not so simple, it can be difficult to recognize the trap and thus all kinds of people get the space to claim that they have done so great, while it was purely a coincidental effect. The best examples are to be found in the stock exchange trading. With exceptions, many traders try to say something about the future price of a stock based on the shape of the performance line in the past. Find yourself on Technical Analysis with terms like Cup and Handle, Support line, Resistance line and Head and Shoulder. In addition, there are many enthusiasts who claim their prosperity to the analysis of stock market patterns. By contrast the Efficient Market Hypothesis argues that the historical prices do not include any indicator of future prices. To relativize this fierce debate, it is perhaps useful to think of gorilla Jacko from Berlin who takes a banana from a basket sometimes, after which purchases and sales are made on the stock market on basis of the chosen bananas. It is claimed that with the random selection Jacko performs all better than most years the Dutch AEX index. Without analysis it apparently goes well.
The bigger question is how you can sensibly deal with these problems of looking back and randomness? To begin with the last, you off course can apply corrections to filter random effects out. In the example of the Energy Agreement that is the gas consumption per degree day. But that only tells you what the actual trend is at most, and not necessarily where you will end. For this leading indicators are required. These indicate something on the extent to which a change is going to take place before it is visible in the actual performance. For the Energy Arrangement this could for example be the number of houses with additional insulation. By examining how much is saved you can even calculate what target you have set for the number of houses to achieve the desired energy saving. That is all SMART (Specific, Measurable, Achievable, Relevant and Time-bound). The corrected measurement of past performance can be used to check whether your expectations are correct. To include that in Evaluation and Review makes SMARTER .
To get back to the driving: the best way to ensure that you stay on the road is looking forward. There is a reason there is a windshield. But an occasional glance in the rearview mirror will not hurt. Then you can check whether the driver of the car behind you looks ahead. Even if it is a Smart.
[1] On a straight road that does not matter.
Ype Wijnia is partner at AssetResolutions BV, a company he co-founded with John de Croon. In turn, they give their vision on an aspect of asset management in a biweekly column. The columns are published on the website of AssetResolutions, http://www.assetresolutions.nl/en/column
<< back to overview
|
