Government IT projects are prone to failure for 6 very predictable reasons
Published in HBR - by Rita McGrath | 1:29 PM October 10, 2008
In the course of doing research for our forthcoming book, “Discovery Driven Growth: A Breakthrough Process to Reduce Risk and Seize Opportunity,” we have spent a lot of time researching disappointments, redirections, and outright flops. Many of these involve IT in some substantive way.
I was recently asked by reporter to comment on why IT projects, particularly government-sponsored IT projects, are so prone to failure, and it got me to thinking about the lessons we’ve learned from other kinds of failures. With today’s headlines screaming about how government is taking a much larger role in our economy — and by extension in our lives — I thought it would be helpful to consider what we know about why governmental projects tend to flop and what we might want to consider doing about it. Many of these lessons are equally applicable to IT flops of other kinds, but since almost every major program involves a technological component, they are particularly of concern for the guys who are now going to have billions of dollars to spend pretty much as they see fit.
Government IT projects are prone to failure for 6 very predictable reasons:
1. Lack of good governance structures – often, large projects fall between agencies or between units of agencies and it is hard to achieve consensus on what each wants. What often happens is that decisions then get made by committees and there is little accountability. This leaves the teams building the systems to figure things out on their own without a clear decision structure, which can lead to inevitable problems.
The solution? Make sure that there is a clear set of rules for who is in charge, what results are desired, and how decision-making is going to take place. I, for one, would feel a lot more comfortable if Treasury and the other agencies involved in the financial bail-out were very transparent about these issues.
2. The “waterfall” method of systems design – in this approach, the system is scoped out in terms of desired outcomes, users or analysts prepare specifications, and these are then given to the programming team who create the code, which is then tested by a testing team and put into production. You can see that this is a relatively linear process. It creates all kinds of problems, beginning with the fact that often users and analysts really don’t know what a system should do until they get the chance to play with it, coders operate on specifications which may not lead to a desired result and the project can be way down the road to development before people realize it should have been done differently.
A far better approach is to iterate – mock up the functionality to get a basic process flow, and then build it out little by little, working intimately with future users so that when the system is finally delivered, it’s exactly what was wanted. The problem with this for governments is that it is almost impossible to write an RFP or requirements document that allows for iteration and non-specified deliverables. It’s an exact replication of the process we propose for any kind of uncertain new project in the new book. You don’t want to be basing plans on assumptions – far better to spend a little to convert assumptions to facts and go on from there.
3. “All at once” funding – in any large-scale project, you’re better off getting just enough funds to get to the next major milestone, at which time a mini-post mortem can reveal problems before a lot of expense has been incurred, and the project redirected or stopped while it is still inexpensive. With government projects, however, it is such a pain to get funding approval that the temptation to get funding all at once is irresistible – what that means, unfortunately, is that projects can get very large and very expensive before anyone has a good hard look at them.
The solution is to allocate the money, but don’t allow it to flow freely until certain conditions – milestones or checkpoints – have been accomplished. While it does increase the review burden, it dramatically decreases the program’s risk. Another reason I’m a little uneasy about $700 billion being laid out as available in one fell swoop.
4. Scope creep – there is a tendency in government to want any new system to meet all possible needs with people thinking that if they don’t get their feature on board with this system, it will never happen. So the scope of projects gets larger and larger, and if you don’t have good governance to put a stop to that, it can cause a project to become extremely costly and unwieldy.
A better approach is to plan projects as a series of additional functionalities, so that people don’t feel they have to get their feature in the first release – think of it like trains leaving a station rather than one big bang.
5. Automating before redesigning – a common problem with government projects is that the system is designed around processes as they are, rather than around processes as they should be. So essentially, you end up automating a dumb way of doing things without making any real operational improvements. This frequently happens because people are unwilling to put the serious design effort in up front and want to get right to the programming. It is really worth the time to go through the processes that you use to try to understand why each step is done as it is. What you will often find is that many steps are historical relics and are no longer needed; or that there are dramatically simpler ways to do things once you have the power of an integrated information system in place. I like to start with the end result the process is supposed to accomplish, and work backward into the minimum number of steps that are necessary to achieve that result.
6. Finally, failing to budget for ‘cleanup.’ This usually occurs in the implementation part of the process, when people suddenly realize that the data they are working with has to be moved from the old system to the new system and that it can’t be done easily. I estimate that a good 40% of an IT project’s budget could go into just making sure the information in the system is clean, accurate, and representative of what it is supposed to do. The first thing to do is to make sure you have given some thought to the state of your data before you throw the switch on a new system. Even better, try to build successive ‘clean ups’ into the ongoing project – don’t leave it until the very end and get surprised. What some people do to minimize problems is to move the data to an interim ‘holding pen’, then get it straightened up, and only then move it to the new system.
In terms of whether governments fail more than the private sector, there are plenty of stories of failed IT projects in the private sector as well. In fact, there are entire companies who have blamed poor performance or even bankruptcy on failed projects. The difficulties are the same – but in government there is often much less accountability so the losses pile up and are more visible.
To use these ideas pragmatically, if you are facing a large and ambitious IT project, you can simply ask yourself the following questions, giving yourself an agree/disagree score of for each set of risks. Any “agree” scores should prompt immediate attention on your part.
- Our project team reports to a committee with representatives from different parts of the company; the committee makes most operational decisions.
- We try to have complete specifications from the business owners of the project before we do any programming; changes to the specifications need to go through a controlled change process
- We believe in providing all the funds a project needs at the point that it is launched
- This project is so substantial and important that we will probably never do something like it again
- We don’t have time to do a complete business process redesign before getting the project going
- We’ve allocated 10% or less of our total budget to data quality and conversion issues
August 14, 2012 — Bruce McGraw
I saw an interesting story on the news today. Jason Wheeler from my local TV station did a story on “Texas Taxpayers Paying Millions More Than Planned For Computer Upgrades.” Great story Jason!
He brought out some interesting facts about the state’s IT projects:
“…many of those IT projects ran millions of dollars over budget. The pattern became so glaring that the State Auditor’s Office just issued a scathing 33 page report. …Of the projects reviewed by the auditor, 67 percent went beyond their estimated completion deadlines and 73 percent went over budget.”
I happen to agree with Karen Robinson, Executive Director of the Texas Department of Information Resources when she states that this is not unique to Texas and “Major IT project management remains a national challenge”.
If you have not worked on government projects (At any level – local, state or federal) then you might not understand some of the rules and contract vehicles that can lead to project failures. I have been called in many times in my career to “turn around” failed government projects. I have seen several patterns over the years that contribute to failed projects. (My definition of a failed project is when either cost, schedule or quality are far out of bounds from the original estimates.
The first pattern I have seen is what I call the “process” of doing procurement. This process is partly to blame for failed projects. The standard procurement process has not changed significantly in over a decade and has some severe limitations in how it sets the vendor and the government up for failure. The basic process is:
- A government agency advertises some type of Request for Proposal (RFP), a Statement of Work (SOW) and some guidelines for proposing
- Bidders are then allowed to ask questions and receive answers
- The agency then receives the proposals
- Through some set of criteria they determine the winner
While this process may seem straight forward and fair, it actually leads to poor requirements and too low a cost. The problem is that when the process leads to driving cost as the primary factor and not quality or flexibility then we get poor project performance and scope dis-agreements. In turn – these lead to problems in cost and schedule.
Because the procurement process assumes that the government knows exactly what they want, that is exactly what the companies propose. Where is the innovation in that? If the project is large and will take many months (or years), how could an SOW possibly accommodate the technology or process changes that are needed in advance?
And with the scrutiny that is placed on the procurement shops, they try to make all procurements just a “commodity” purchase. This makes it easier and takes away any chance of a challenge to the process. So here is my beef – how can you make developing a solution or system with people a commodity? That is like saying I want to hire a painter to paint a picture of a horse and there is little difference in the talent or end product that the painters will propose. Those of us who have been PMs for many years know that any 2 companies can propose on a project two different solutions that appear to give the government the some product. And if the process says that both solutions will yield the same product then price is the final determinant. And there starts the problem.
Ann All wrote a wonderful post on the subject and stated that “Government procurement appears to be a broken process, one in which agencies hamstring even their usual vendors from suggesting different ways of solving problems or achieving goals.”
The second pattern of problems I have seen is in project cost estimating. I wrote another post on that last year (Planning and Estimating Government projects) where I talk about how both government and corporate management want a “low cost” solution.
The third pattern I see in government projects is the lack of collaboration. After the contract is awarded you would assume that all parties (Users, PM, vendors, stakeholders) know exactly what is supposed to be delivered. Wrong – have you ever seen some of the 500 or more page proposals that the government receives? I have often thought that no human could understand what is supposed to be delivered on one of these huge IT projects. In my opinion, too often the lack of communication and collaboration between clients and vendors is the leading cause of project failure. The government wants exactly what the SOW and proposal stated even if there is a better solution and the vendor wants to only deliver the minimally acceptable solution in order to make money on a low-balled estimate. I would love to see procurement try some collaborative contract vehicles. Make the project a teaming agreement with the outcome of the project being aligned with the outcomes desired.
What if an IT system project had the structure and basic outcomes of the project defined but set aside the details of the user interface or process as a bounded effort to be defined? Kind of like building a house where you are given a budget for the floors, lights, paint, etc. You know the basic layout of the house, but can choose the details as you go along. I guess this would equate to more of an Agile type project – and that seems to be very hard for the government to do.
March 31, 2011 — Bruce McGraw
Academic papers in computer science and systems engineering as well as newspaper exposés frequently cite horrific examples of cost overruns on software projects. When I see a headline like, “GAO Highlights DoD Cost Overruns” I shrug and say to myself, “here we go again.” What got me going down this thought path today was a recent headline in our local, “Austin Business Journal” titled State lacks sufficient IT planning.
What followed in the article – sorry the full article requires a subscription – chronicles 48 technology projects undertaken by the State of Texas, of which 32 were late and/or over budget. You can read the report, “2010 Quality Assurance Team Annual Report” in its entirety, if you are so inclined. The article suggests that the blame for recurrent failures falls on the “state employees tasked with planning and budgeting new technology projects.” The implication being that if the state employees managing the projects just knew how to estimate software development cost and schedule, then the projects would successfully complete on time and within budget.
In my opinion, it is not that simple.
Software Cost Estimating Theory
Back in December 2010, I talked about Estimating Project Costs. In theory, a project manager uses customer requirements, resource availability and required milestones to construct a realistic project schedule and budget. In complex projects, components may be estimated separately and then the estimates are combined or rolled-up with integration estimates to produce cost and schedule for the total project. Sophisticated cost estimating tools help project managers construct a cost and schedule using parametric models and lessons learned from previous, similar projects.
Software Cost Estimating Reality
First, let me say that no one wants to know how much it will cost to get a software solution to meet their needs because the number will be too high. Senior management wants a software cost and schedule that allows them to win a competition based on lowest bidder. Government agencies want a solution that fits within the budget that was already proposed and approved before the RFP was announced.
Getting from point A to point B often involves a complex dance using multiple contractors, combinations of off-the-shelf and developed software and workers located anywhere around the world working for small or medium sized companies or as individual contractors. The set of vendors working together comprise the “project team” each of whom was pressured to commit to a cost and schedule that meets the target established by the customer.
I live in a large state with a state government budget that rivals many countries. To meet budget shortfalls, most states – including mine — have gone to a state run outsourcing model that encourages independent contractors and staffing firms to deliver “lowest cost.” What is happening is that they are not getting the right skill set and there is no synergy or “team” concept in these IT programs. Instead of utilizing two or three firms, who specialize in a particular area, they are putting 10 to 20 independent contractors together and saying – here is the schedule, make it happen.
This situation is a project manager’s nightmare. I hate to say I see the same trend in corporations…. They have stopped using small, expert firms and have gone to a staffing model that brings in independent contractors to work individual positions and tasks. I am only glad I am not in charge of one of those projects!
I would love to say that I have THE SOLUTION. However, I don’t. I applaud government agencies and companies that accept “Best Value” rather than lowest bid – that is a start. I would like to see professional support for government agencies in creating realistic RFPs for software procurements. In addition, I think that a procurement model that involves selecting a trusted contract manager or contractor and then incrementally creating requirements and development cycles, rather than all or nothing development, has merit. Then I would like to see teams made up of the right professionals – especially Project Managers. We have to start acknowledging that not all people are good at everything and a good team is not a collection of random consultants.
Here’s a highlight of where they’ve been going wrong:
- Too many initiatives on the go at any one time
- Wasting money hiring external consultants
- Keeping alive failing projects which should be stopped early and cheaply
- Opposition to change, especially in the diversity of hiring, ex private sector employees have to “fight against the machine” to fit in the public sector world
- Translation of policies from Whitehall to the front line of project delivery – there needs to be an understanding of the problems of implementation
- Key delivery staff are not in the job long enough to see the project through to completion
- Delivery staff are not experienced enough – especially lacking are people who have been round the block a few times and the issue of succession planning when key staff move off projects
Value is not adequately defined from the customer’s perspective.
A lack of systems thinking throughout the program/project.
- Trust and communication breakdowns.
2 – Lack of understanding of effort and/or feasability
3 – Changing org interests – why this project may have been super important 3 months ago, now no longer interesting
Scoping the project too big
2. Not adopting Agile practices – Close user involvement in development; small, frequent capability deliveries; responsive to change; leverage portfolio level resources
3. Too much oversight, complex policies, and restrictive statues to comply with
“Fuzzy business objectives, out-of-sync stakeholders, and excessive rework” mean that 75% of project participants lack confidence that their projects will succeed.
A truly stunning 78% of respondents reported that the “Business is usually or always out of sync with project requirements”
IBM survey in the success / failure rates of “change” projects finds;
Only 40% of projects met schedule, budget and quality goals
Best organizations are 10 times more successful than worst organizations
Biggest barriers to success listed as people factors: Changing mindsets and attitudes – 58%. Corporate culture – 49%. Lack of senior management support – 32%.
Underestimation of complexity listed as a factor in 35% of projects
Here are Philip’s six reasons that government IT projects fail:
- Analysis of business needs, missing or wrong: failure to undertake a proper needs analysis. Failure to consult those in the front line of delivery, or in receipt of services, is endemic among those planning new policy initiatives or changes to existing systems.
- Needs change before implementation: the churn of ministers and political priorities is significantly faster than that of technology. It has been suggested that suppliers commonly bid low on the original specification to get the opportunity to make profits on the subsequent changes.
- Over-ambition: about what is achievable in practice, given the people, time and budgets available….
- Delay: particularly in agreeing priorities between conflicting objectives, leading to delay in planning and procurement….
- Lack of top customer management involvement: and lack of high-level skills, training or experience in planning, procurement or implementation. This lack of training and experience on the part of the customer causes and compounds the previous four problems….
- Supplier project or team management: usually because the ‘B’ team is trying to salvage an already doomed system, after the ‘A’ team has moved on to the next bid.
Government projects have so many problems, according to Philip, because “accountability structures” separate project requirements and goals from the real needs of end-user constituencies. As he says, “shifting political priorities, with neither consultation with the users nor consideration of the practicality of the consequent ‘ministerial’ demands for change” cause these projects to flame out.
If the root of government IT failure lies deep in the structure and relationship between political masters and humble IT servants, then spectacular public sector meltdowns are here to stay.
In a guest blog, corporate IT lawyer Alistair Maughan argues that Agile development is an evangelical fad ill-suited to government IT.
The Government ICT strategy had some good ideas. Agile project management isn’t one of them.
The Cabinet Office expects Agile will reduce the risk of ICT project failure. It’s a nice idea in theory. But it won’t work in government IT. It won’t work in the real world.
Two of the most cautionary examples of failed ICT projects in recent years demonstrated the drawbacks of Agile.
The court battles of BSkyB v EDS and De Beers v Atos Origin showed that when Agile projects go wrong, they can go spectacularly wrong.
The Agile methodology is meant to deliver IT projects flexibly, in iterations. It’s meant to involve customers more directly and adapt quickly to their changing needs. This means the final system only emerges gradually. It means customers don’t pay a fixed price for a complete project. They pay for a commitment of resources.
But the lack of clearly defined project roles and requirements is a problem for Agile.
Agile evangelists argue fiercely that the conventional waterfall development methodology is unrealistic. They say the sheer scope of work required by its pre-set deliverables often leaves it unable to fulfill expectations. They set themselves up to fail, say the evangelists, when they should be working collaboratively for success.
I’m prepared to accept on trust that, if all goes well, Agile may reduce the risk of project failure. But Agile simply won’t work in the real world of government ICT. We need a Richard Dawkins to bust the myth of the Agile gospel.
There are four clear reasons why Agile won’t work in government ICT. The most obvious is that government customers want to know up-front how much a system will cost. That’s not so unusual, is it?
Under Agile projects, you pay a given amount of money for a set amount of effort. But you can’t guarantee a specified outcome for a specific price.
This won’t work in government. Departmental budgets are managed very tightly, and they must be approved. Agile implies that charges for time & materials should be open ended. Government departments won’t accept that.
Government is also legally required to follow open procurement rules.
That means comparing different bidders on a like-for-like basis, and deciding on best value for money. Agile can’t give you a clear specification of outputs up-front. Nor can it give a definitive up-font price.
So how are government bodies supposed to make Agile comply with the legal requirement that public procurements are fair and open?
As if that isn’t problem enough, Agile offers insufficient means of remedy if things go wrong.
This is a particularly sensitive issue for government, where departments suffer public opprobrium if their project isn’t a resounding success. The press, the National Audit Office, and the Public Accounts committee (PAC) will give government a kicking if they can’t make suppliers pay for the damage they caused.
Agile makes it hard to apportion blame because the customer is intimately involved in the work. Since Agile contracts lack clear contractual delivery obligations or remedies, how do you enforce properly? How do you recover loss or damage if there’s a problem?
I wouldn’t want to be the first Permanent Secretary to admit before the PAC that his or her department has no real right of legal recovery from a failure.
Agile is fourthly not suited to public sector management structures.
Decision-making is centralised in government. Civil service structures ensure every important decision flows up to senior levels. The Cabinet Office has under the current government taken even greater power over ICT projects. But Agile decision-making (over requirements, for example) flows down. This is key, so small devolved teams can react quickly and adjust to new scenarios.
It is inevitable that Agile decisions will go through management hierarchies in central government. This will be like kryptonite to Agile projects.
Agile projects rely on decisions based on mutual trust. They are therefore well suited to in-house projects. But the faith they ask customers to have in service providers makes them ill-suited for external developments.
You can have an ICT project with a watertight contract, clear deliverables, openly and legally procured, with a fixed price and appropriate remedies if you don’t get what you want. Or you can have an Agile project. You can’t have both.
I do appreciate that as a lawyer specialising in large ICT projects, you may think, “Well, he would say that, wouldn’t he?”. But my job is the help create successful projects.
I’ve seen too many projects flounder for a lack of trust between customer and supplier to think the answer to government’s ICT problems is the Agile credo of, “Let’s trust each other some more”.