Avoiding analysis paralysis and delusion

The consideration of Return on Investment (ROI) understandably increased during the downturn. In line with this, IT vendors have spent a lot more time talking about it and some even provide financial models to help customers assess it. While all this seems eminently sensible, however, we must be careful not to get too carried away, as coming up with valid and meaningful ROI cases is more difficult than many people acknowledge.

The usual interpretation of ROI revolves around time and money. One way of thinking about it is to consider the amount you will ultimately get back by investing a certain amount now. If you spend £20,000 on an IT system, for example, you might calculate that it will return £60,000 worth of benefit over its anticipated 10 year lifetime in the form of cost savings and/or increased revenue performance.

The first and most obvious problem with this approach is that the majority of organisations don’t have a good enough handle on the current costs and the current contribution of existing systems to feed accurate and precise information into the calculations. Assumptions and estimations therefore need to be made which are difficult to verify. The result is often an unknown level of error in the calculations, but more significant is that the process is open to manipulation to support any pre-existing agenda. Nudge a couple of assumptions in one direction and the investment case can be made; nudge them the other way and the proposal is canned.

The other big problem with basing assessments on the ultimate return is that a lot can happen over the lifetime of an investment. This is especially true of IT systems, as today’s leading edge technology providing competitive advantage or operational efficiency is tomorrow’s legacy holding the business back and sucking in excessive resource to keep it running. The inevitable changing of both the business and IT landscapes over time can render initial assumptions invalid and even core functionality less relevant or appropriate.

One way of tightening things up is to consider a shorter period. You may judge the context and environment to be relatively predictable over the medium term, for example, and therefore look to calculate a return over, say, a three year period. The other common approach is to look at payback time or time to ‘break even’. This is based on considering how long it will be before the amount of money spent is recouped in either cost savings or enabled improvements in revenue/profit generation. Of course both of these approaches, or any combination of measures you choose to use, are still open to manipulation, but assuming the right spirit, discipline and motivation, the impact of inaccuracies in your assumptions and estimations is less if a shorter period is being considered.

Having said all this, though, we still need to be realistic and appreciate the limitations of ROI assessments in the real world. While the creation of comprehensive ROI cases using elaborate models and complex spreadsheets can look very scientific and create the impression of precision and accuracy, the reality is often different. At best, an ROI model is a decision making aid that helps you to lock down as many of the known variables as possible in a structured manner, but it still requires good business sense and value judgements to work around the inevitable gaps, e.g. using ‘what if?’ style analysis. At worst, an ROI model can be a political tool used to create the illusion of rigour, and designed to precipitate a predefined outcome on the basis that no one is likely to come back down the line and check the numbers anyway.

Even if you approach ROI responsibly, however, it is important not to let it get in the way of doing the right thing in some circumstances. Back in the 90’s, for example, the justification for a lot of investment in ERP was simply to prevent business operations from collapsing. With so many organisations reliant on a fragile patchwork of highly customised point solutions, and/or considering the uncertainty of Y2K, they had little choice but to invest. What were they going to do if your financial ROI calculations came out unfavourably, just leave things as they were and wait for the inevitable catastrophe? That’s a bit like taking time out to calculate the cost justification for fixing faulty brakes on your car then questioning the result. Sometimes you just have to spend money to manage risks and keep things viable, and what you need to spend that money on is pretty obvious.

But some go too far the other way, and always just spend money on the obvious. If you are running out of disk space in your IT infrastructure then the obvious thing to do is buy more storage, right? Well not necessarily. You might find that investment in better information management software and/or storage virtualisation means you can free up space through safe deletion, archiving, etc, and make use of orphaned storage that was previously unutilised. Solving the immediate problem in this way might cost more initially, but has a much better payback over time, through cost savings on hardware, reduced administrative overhead, and the freeing up of information so it can be properly exploited for business benefit.

If you look across IT, there are lots of examples like this where the first decision to make is not which product or service to buy, but which approach to take to solving the problem, which brings us back to the question of RIO, and specifically the precision of ROI analyses. While the concept of return in investment in a generic sense is critical to understanding the significance of going down one route or another, because the impact of alternative approaches can be very different, you often don’t need a meticulous detailed analysis to tell you what you need to know. The above storage example is a good indication of this as, for instance, would be considering whether to implement a like for like replacement of an aging telephony system or an IP based alternative that opens the door to full unified communications.

Rough cut analysis is often adequate for this kind of direction setting and as a tip, rather than trying to express all elements of business value in strict monetary equivalents, and use arbitrary assumptions to do this, it is sometimes more meaningful to use more qualitative scoring, rating and ranking mechanisms. Taking our unified communications example, trying to put a monetary value on the benefit of streamlined communications, which in turn (amongst other things) enables more informed and efficient day to day decision making, is actually very hard. One clinched deal or avoided disaster as a result of key people being able to collaborate quickly and effectively and the investment may have paid for itself through a single event, but that is incredibly difficult to predict. What you can do, however, is rate different options as having a high, medium or low impact on operational or commercial decision making efficiency and effectiveness, which might be relatively crude, but at least good enough to give you a steer.

Downstream from this, once you have decided on an approach and are simply considering which product or service to buy, another trick is to focus on the cost element. While you might not have all of the metrics and data necessary for a robust financial ROI calculation, you might have enough for a reasonably accurate assessment of the Total Cost of Ownership (TCO) of the solutions being considered. Assuming they are functionally equivalent, this might be an appropriate measure to decide between one product and another.

In conclusion, while assessing the return on significant investments is clearly a part of running IT in a business-like manner, it is important not to be a slave to detailed financial modelling. Good decision making is enabled by the right amount of analysis, conducted at the right level, based on appropriate measures, backed up with an objective spirit and a healthy dose of common business sense.

By Dale Vile, research director at Freeform Dynamics.