The crippling costs of IT project rework

With so much time, effort and resource going into IT project rework, few software development projects are delivered on time and within budget. By Jessica Twentyman.

In the engineering, manufacturing and construction industries, rework – the process of making changes to original project plans – is something to be avoided at all costs. Few companies embark on building an aeroplane or a skyscraper without mapping out the requirements of the project in meticulous detail and sticking to them as far as possible, in order to deliver the project on time and within budget.

If only the same were true of building software. When it comes to developing the systems that will deliver mission-critical information and documents to employees, customers and trading partners, it seems that companies are only too willing to tolerate rework. Software code in development is subject to constant change, frequently revisited and frequently amended in order to fix bugs, to address gaps in function and to incorporate changing business requirements.

With so much effort going into rework, it is no wonder that few software development projects run smoothly. According to 2004 figures from market research company the Standish Group, fewer than a third (29 per cent) of all software development projects are successful; that is, they are delivered on time, on budget and with all the required features and functions. Over half (53 per cent) are “challenged” – they run late, exceed budget or deliver less than is required in terms of features and functions. Even worse, 18 per cent fail outright – they are cancelled prior to completion or delivered and never used.

Project failure by rework is well documented and all too common. “Rework is the crazy aunt in the basement that no one talks about,” says Mitch Bishop, chief marketing officer at software visualization tools company iRise. “We know it’s out there and we know it’s expensive, but many companies seem to accept it as a normal part of software development projects,” he says. In fact, says Bishop, the majority of companies actually budget for a hefty degree of rework when mapping out a development project.

“There is a real complacency about rework,” says David Oates, vice president of international operations at Primavera, a developer of project-portfolio-management software. “In my opinion, that’s a hangover from the 1980s where resources were thrown liberally in the direction of software-development projects. But in a tough environment where budgets are a lot tighter than they were, companies can no longer afford to be so tolerant,” he says.

In fact, if they took a closer look at what rework was costing them, many organisations would be shocked. In fiscal year 2003, for example, the US Department of Defence spent around $8bn, or 40% of its $21bn budget, to rework software because of quality-related issues, according to figures prepared for the US Congress by the General Accounting Office. That may be an extreme example, but according to analysts at IT market research company Forrester Research, rework typically constitutes as much as 30 per cent of the cost of developing any piece of software.

More worrying still, little of that is spent on correcting faulty code. In fact, say analysts at Meta Group, around 70 per cent of rework can be attributed to software that fails to fulfil end-user requirements. That underlines one of IT’s toughest challenges when it comes to implementing information-management systems: analysing and understanding what the business actually needs, cited by a fifth of information-management professionals as their toughest project challenge in a recent survey conducted by industry association AIIM and management consultancy PricewaterhouseCoopers.

Addressing re-work

It is safe to assume, then, that IT project rework is costing companies worldwide millions of dollars a year – and some are now seeking to address that.

“Tackling rework is one of the most significant steps a company can take to reduce their software-development costs,” says Martin Wischhusen, director of ebusiness at Agilent Technologies, a US manufacturer of testing and analysis equipment. “We know that we need to be more formal about how we tackle rework and have put in place a range of measures to track it.” That, he says, enables the company to eliminate unnecessary reworking, as well as allowing developers to identify where rework is necessary.

That is important, agrees Charlie Mayes, deputy managing director of DAV Management, a specialist project-management company that has worked with Monarch Airlines and Nationwide Building Society: “Some rework is inevitable and an element of rework needs to be built into every schedule. If you want to deliver a top-quality software project, you have to allow for contingencies and accept that you can’t possibly plan for everything.”

Putting effective measures in place, then, enables companies to differentiate between beneficial and unnecessary rework. Measuring rework for that analysis can be relatively simple: it involves tracking and quantifying the time and cost devoted over the course of a software-development project to finding and correcting problems before and after release. End-user verification and validation are typically excluded from these calculations, but any debugging effort during integration and system testing is included.

“At its simplest, rework is defined as any time allocated by programmers to a project beyond unit test. If changes are considered defects, rework calculations might also include the amount of time recorded to change requirements. It could also include retest time by business testers,” says Matthew Hotle, an analyst with Gartner. In order to provide direct comparisons between different IT development efforts, rework effort is ‘normalised’ by calculating it as a percentage of overall project time and cost.

Most organisations use simple spreadsheets to make these calculations. Others are taking more sophisticated approaches: developers at US telco Sprint, for example, are able to reach into the company’s cost-accounting systems and allocate cost codes to rework, according to its underlying causes, says Bishop.

With those calculations in hand, organisations are better prepared to tackle causes of unnecessary reworking. Agilent, for example, found that a common cause of rework was a proliferation of paper-based, text-heavy reports surrounding the requirements-capture process. “Text alone is not sufficient to define complex applications. Large, text-based documents passed between end users and developers quickly become too dense and too difficult to understand,” he says.

Instead, Agilent has replaced paper-based reports with software visualization tools from iRise that enable a system simulation to be produced and presented to end users for feedback before a single line of code is written. Most recently, the company used these tools in a project designed to redevelop the Agilent website to improve customer satisfaction rates. “In the past, that would have been a lengthy process but, by building a simulation using iRise, we got buy-in from business users in a single one-hour meeting,” he says.

In situations such as this, says Bishop, the simulation immediately becomes a visual contract between the business and IT that is unambiguous and enforceable. Fifty of the Fortune 100 companies use iRise to map out the functions and features of proposed systems to end users, he claims, and on average, they have managed to reduce rework costs by around 70 per cent as a result.

An added bonus of this approach for Agilent has been better communication between the business and IT. “When a lot of rework is required, IT is often identified by business managers as a group that is simply unable to get things right the first time,” says Wischhusen. In fact, that is sometimes unfair, he adds: “End users often find it hard to articulate their requirements in a clear way and many people don’t actually know what they want until they see it – or at least until they see something they don’t want.”

An iterative approach

Some form of prototyping, then, is a valuable weapon in tackling unnecessary rework by better matching development work to user needs. Software visualization tools, however, are a costly option and many companies are still required by budget limitations to rely on the traditional tools of whiteboards, spreadsheets and Word documents in order to create prototypes.

These can still be valuable, say experts, if used in conjunction with a more iterative approach to software development, which involves presenting small modules of completed software to end users as they are developed (focusing on areas of highest-priority function first) and giving users the opportunity to feedback – thus spreading the rework effort over the course of the project.

“With many projects I see the problem is not so much rework, but late rework – making changes at the end of a project. That’s where the real pain and cost is and that’s what should be avoided using an iterative approach,” says Anthony Kesterton, a technical consultant with IBM Rational.

This is in stark contrast with traditional development methodologies that aim to plan out an entire development project in great detail before coding begins, says Eugene Blaine of project-portfolio-management company Atlantic Global. “If you’re taking a traditional approach, then you’re making end users commit to specifications way in advance of any software actually being delivered – maybe as much as 18 months in advance. That may be good for developers, but it’s certainly not good for business, because requirements do change over the course of a project – that’s just a fact of life,” he says.

In particular, iterative rework is an important element of Agile programming methodologies, which have been used by specialist application development companies for over 15 years but have only gained popularity in in-house development teams over the past two to three years. Focusing heavily on user and development team feedback and communication, Agile programming methodologies aim to deliver software far faster than traditional methods allow and to eliminate late reworking.

“Rework isn’t something that ought to be avoided in all cases. In fact, it’s essential that it takes place in order to deliver a system that fulfils user requirements. But with Agile programming, we minimise the impact of those essential changes,” says Dave Farley, director of innovation at ThoughtWorks, an application-development services company that has used Agile programming methods to build information-management systems on behalf of companies such as retailer Dixons and US bank Washington Mutual.

Scope creep

An iterative approach, however, can lead to another problem that necessitates still further rework: ‘scope creep’. This term is used to refer to the way in which end users tend to introduce new requirements during consultation.

“Many companies try prototyping as a way to combat scope creep by demonstrating [to end users] how targeted functionality will deliver a comprehensive business process. However, seeing an application evolve can work against a team, too, because it can increase the temptation [for end users] to gold-plate an application,” says Forrester analyst Margo Visitacion. That leads to more iterations than expected, and because these additional iterations are not factored into initial budgets, budgets inflate.

“Logically, customers and stakeholders realise that initial estimates are rarely little more than educated guesses, but when the stakeholders don’t understand the effort required for rework, reactive and uncontrolled changes can drive projects to run over budget by more than 40 per cent,” she says.

For that reason, prototyping needs to be carefully controlled, says Visitacion, and there are several ways to enforce that. “Agree to deliver in order of priority and supply a cut-off date for prototyping in each cycle. Convey the message that development will not start until after the prototyping for that iteration is complete, and then involve the end user in user acceptance testing for each deliverable,” she says. Often, she adds, when the client sees the actual delivery of the most important functionality within budget, it helps to control budget variance and ‘gold-plating’ later in the project.

In addition, she says, development teams should demand that each request for rework made by the business be accompanied by a justification, a risk assessment and an updated budget figure. That may sound harsh, but it may be necessary if companies are truly determined to halt the already-spiralling costs of IT project rework.

Causes of rework:

Poor communication between business and IT;
Text-based specifications too complicated to understand;
User requirements misinterpreted by developers;
Bias by outsourcers towards ‘vanilla’ templates;
Lack of end-user involvement and acceptance;
Unrealistic expectation setting;
Users don’t know what they want until they see it;
Requirements change during project.