Two talks presented to the British Computer Society on February 5, 2013 at the Davidson Building, London

2: Talk 2:
Atlantic: Agile Government on both sides of the Atlantic: Story far The Story So far

Brian Wernham FBCS FAPM

Brian Wernham 2013 CC BY-NC-ND

Moving towards Agile Government

There have only been incomplete attempts to survey the progress of the adoption of Agile in government on both sides of the Atlantic. A torrent of reports have been issued in the last two years, but their conclusions are tentative. The general consensus is that although targets for a move to Agile are broadly set, specific targets and robust measurement are needed. And, there has been little analysis as to whether the targets are achievable. So - here are the key questions this second talk will attempt to answer:

Brian Wernham FBCS FAPM Author and Consultant Welcome back! In my first talk, before the break, I stressed the need for moving beyond: belief in the Agile approach to having proof of the Agile approach. How are UK Agile? adopting Agile? and an d US Governments

Are the ambitious targets being achieved? Here are the four points I will discuss for both the UK and US: Firstly - some background: what is driving the need for new approaches in each country? Secondly - What do the respective IT strategies promise? Thirdly - What actions are underway? And fourthly - I will assess the current status of Agile adoption

More than ten years on from the signing of the Agile Manifesto, we are seeing more than just an interest in Agile from leaders in Government, we have had clear statements of intent. Both the US and the UK governments have said that they want to be more Agile. BUT: What assurance is there that problems on individual, high-profile government projects do not set back the whole Agile agenda? Can the US and UK Governments learn from each others successes and mistakes in the adoption of Agile? Could both the private and public sectors adopt Agile at a larger scale, faster if we had more research evidence from Government available to us?

Agile Government in the UK

New National Audit Office Report on Agile in the UK Government The old headquarters of Imperial Airways in Buckingham Palace Road used to be the hub for air travel for the London elite.

modern staff gymnasium and exercise studio. As well as refurbishing their offices, the NAO has 'refurbished' its IT skills. Internal expertise has been strengthened, and now there is now a reduced reliance on external skills for checking IT projects. Part of this capacity building has been in developing an approach to auditing government technology projects that are claiming to be 'agile' in approach. Earlier last year the NAO ICT and Systems Analysis Team published a report on how the private sector is using 'agile project management'. In its early days, one was whisked away on a sea plane from the river Thames, and flew to Southampton to board a luxury P&O steamer to travel to other major cities such as New York, and to exotic destinations in the Orient. Glamorous tea dances used to be held in the basement of the Imperial Airways building. Refreshments were consumed mainly in the form of bracing 'salty-dog' and martini cocktails whilst passengers waited for their flights. Of course, the butler would have gone ahead on a train from nearby Victoria station with the heavy baggage and would have all one's clothes ironed, and hanging in the port side cabin ready for departure.

In October the same team issued a follow-up report, focused this time on providing a 'snapshot' of the use of Agile Project Management in the 17 central UK Government departments.

How things have changed. The National Audit Office (NAO) has moved in from its old, grimy 'good enough for public servants' accommodation into the building, now refurbished with a no-nonsense 'fit for purpose' office environment, complete with bright lighting, hot desks for flexible working, modern IT, and video conferencing facilities. That basement ballroom is now a

The new report aims to identify elements of agile practices that are being used in central government departments, rather than analyse VFM. 2

It is expressly NOT intended to analyse whether agile is 'Value for Money' - VFM. Such VFM reports from the NAO are larger, and usually focus on one project. This is just a warm-up for analytical VFM reports on Government technology projects which will, undoubtedly, follow in due course. Some of these projects will be using (or at least claiming to be using) agile project management. The report lays out five "characteristics for agile delivery". Citizens and business at the heart of delivery Service or business change is delivered quickly and continuously improved Full service is built from small independently usable releases Team is responsible for making decisions rapidly Team continually redirects resources to maximize the value it delivers

What is interesting in the report is that only one central Government department, the Cabinet Office has significant Agile experience to share in all five key 'Areas of Agile'. Only 4 other central government departments have any Agile experience: Department for Transport and its Agencies Department of Energy and Climate Change Department of Health Government Digital Service (as part of the Cabinet Office)

The NAO report found no significant Agile experience in any of the other 12 central government departments. The report also found no specific targets for Agile success, mechanisms for measuring or evaluating success, or plans for rolling it out. There are three key lessons to be learned from the report: Firstly: the drive for Agile was often 'bottomup', from programmers, rather than from senior management. Secondly: barriers to Agile are mainly cultural - large projects are seen as too difficult for Agile. So, how will Agile be rolled out for use in at least 50% of all large, mission-critical government programs by this April this year? The difficulty of incremental release (say weekly, or even monthly) into operational use is also perceived as the main barrier on mainstream systems. Operational releases of changes of government mainframe systems are typically on a half-yearly, or at best quarterly basis at large delivery departments such as DWP (pensions and benefits) and HMRC (personal and corporate taxes).

The report breaks down these five characteristics into lists of typical agile "behaviours and actions". These are based on what the NAO team observed in some existing agile projects in government that they inspected. These checklists will be an important input into the NAO's future consideration in evaluating successes and failures in government technology projects.

Thirdly: Plans for moving forward are unclear - once people had used Agile, they rarely wanted to go back to the old ways of working, but that there was no consensus up to October last year as to what the next step should be. The final part of the report outlines 12 case studies, and for 11 of these a description is given of the "Measures of Success" that the NAO perceive as applicable in each case.

adoption of Agile has improved morale and reduced staff turnover. Increased Output: At the Department for Transport defect rates have fallen from 30% to 5% due to the adoption of Agile. Business Effectiveness: The most important, but perhaps the most difficult measure of project effectiveness. At the Office of the Public Guardian - clear, business orientated targets have been agreed, such as fewer errors in the registration for powers of attorney and a reduction in the cost per transaction. But most other organizations just state the need for compliance to project business cases - just in terms of satisficing the stated objectives of time, cost and quality, without an attempt to show whether Agile has really made a difference. But we still wait for an audit that categorically measures whether the Agile approach is really being used everywhere it is claimed in government, and more than anecdotal proof of its the business.

There are 5 "Measures of Success":

Agile Adoption Measures: For example, at the Health & Social Care Information Centre. It assesses how 'Agile' each project it is running really is, and how effective the project governance arrangements are. For example, an assessment is made as to whether development is really test-driven and how much testing really takes place within each development iteration. Cost reduction: For example, at the Department for Energy and Climate Change (DECC) which measures whether Agile reduces delivery timescales and costs. Process improvement. For example, the ability to track progress with more granularity at user story level by the Cabinet Office on its Electoral Registration project. They stress the importance of weekly show and tell sessions. At Companies House, the

In the UK there is now a clear plan. A few months ago in November, the Cabinet Office announced a Digital Strategy - a timetable to move, and move quickly, to 'Digital, by Default'. The aim is for the UK government to deliver everything online that can be delivered online. How can the ambitious targets be achieved? The Cabinet Office expects one in four government transactions to become digital by 2017"Figure 10: Digital take-up curve, averaged across case study data"

(Slide - Digital Graph) Then there is an 'inflexion point' - a major acceleration - over half of the 4

remaining transactions are expected to be digitized in just three years How can the Government involve more medium sized companies in the supply chain? Are Government Departments ready to manage subcontractors more directly? The spend on smaller technology contracts is expected to more than double

projects. These projects are expected to cost one hundred million pounds over the next two years, and will be built using suppliers approved by the Neutral Vendor. Will this accelerate the use of Agile in the UK Government? Lets wait and see.

Agile Government in the USA

The US government has had its fair share of IT technology disasters. For example, the failure of a huge project that tried to integrate the personnel systems for the US Army, Air force and Navy. In 2010, the project was cancelled, after 10 years and 800 & 50 million dollars thrown away. The next step, announced just before Christmas, is the set-up of a Digital Procurement Framework to spend up to one hundred million pounds. This comprises a mega-contract between the Cabinet Office and one supplier, called the Neutral Vendor. This Neutral Vendor will not be allowed to bid for Agile development, instead it will act as an honest broker, pre-vetting Large, Medium and Small organisations, and even freelances. By June this year the company running this Neutral Vendor will be acting as a dating agency - introducing suppliers on the list to Government bodies as and when projects are kicked-off. We already know which projects are in the pipeline. The Cabinet Office have identified 674 government transactions that could be digitised. Of these, about 250 are major transactions, and 25 have been prioritised to be developed using Agile - as exemplar One of the problems is that so many regulations have built up over the years. These try to improve technical development in diverse Federal government bodies, but often they have just ended up stifling effectiveness. A good example of how regulation has not improved project management is the series of regulations created over the years by the US Department of Defense. For example, to try and improve project management the DoD published the infamous 2167 standard. This was widely interpreted as mandating a waterfall approach. The department then tried to stress that modular development and incremental delivery was the preferred approach - they issued the 2167A standard. But waterfall projects continued unabated. Efforts were made to sweep up all the regulations under one all-encompassing umbrella standard, DOD498. IT disasters continued to plague the department. All that was being produced, was the creation of a Mirage of control by 5

standards. So-Congress got involved. The Clinger-Cohen Act of 1996 begat the 5000 series of regulations and the enforced use of Earned Value Analysis. These were intended to encourage incremental, evolutionary development. But an inflexible approach to project management was the actual result. What the DOD had now, was the creation of a Mirage of control by legislation - a real flexibility killer! The story is broadly similar in technology projects across all US public bodies. Each major project failure results in more regulations - more audit - and more centralised standards. These mirages of apparent control over projects in the US have just increased the management apparatus surrounding each development team smothering rather than helping. In 2008, when Barack Obama put together his team to prepare for his transition into office, he appointed a 34 year-old geek as his technology advisor.

he was the man for the job. When Obama took office in 2009, Kundra was appointed as Chief Information Officer with the power to review and cancel any project in the Federal government. Kundra inherited a legacy of 27 billion dollars of failing IT projects. In the previous decade, IT spending had nearly doubled, growing at an annual rate of 7 per cent. So Kundra immediately capped the IT Budget - saving over 25 billion dollars a year. He forced change to the running of technology projects by holding deep-dive project reviews. These reviews, Kundra called Technical Status, or TechStat reviews. Each TechStat review entailed a long, detailed, face-to-face meeting to inspect each yellow or red status project. These reviews were intended to delve deep into each project with a relentless pursuit of oversight to reshape problem projects - or, ultimately, to halt and even - terminate them. To kick off the initiative, Kundra attended more than three of these meetings a week, publically issuing memos to agencies where problems were found. At the Environmental Protection Agency, for example, one IT project was found to be one year late and 30 million dollars over budget, so Kundra gave them a month to put a recovery strategy in place for the project. He was rolling up his shirt-sleeves and meeting each agency CIO in long and detailed meetings. If a project could not come up with a realistic improvement plan, it would be cancelled. Kundra reviewed 38 projects. He saved three billion dollars by cancelling four and drastically reducing the scope of 11 other. In 12 cases he found ways to cut the time for delivery by more than half, from two to three years down to an average of 8 months by 6

Vivek Kundra proposed a very different approach to running technology projects. Previously, there was little central oversight of IT - there was an "Administrator for EGovernment", but the position was little more than a placeholder. He convinced the president elect that a powerful executive role was needed - and that

adopting a more Agile approach. In 2010 he published a "25 Point Plan" intended to shock the system. It shook up the counterproductive processes that had led to so many project failures. Major initiatives were kicked off to put in place technologies to complement Agile approaches. For example: Upgrading project management skills to include Agile training Breaking down barriers to Agile by requiring integrated project teams Making sure that procurement professionals took Agile approaches Influencing Congress to change legislative frameworks such as the Clinger-Cohen Act that were antipatterns to Agile development.

is the US equivalent to the UK National Audit Office (NAO). The GAO released a report in July charting progress in the adoption of Agile in the US government. They found pockets of excellence at various agencies: The Department of Defense had a 190 million dollar project developing a Combat Support System using Scrum A new 150 million dollar project to improve the management of the registration of Patents at the Department of Commerce was also using Scrum A 44 million dollar Agile project to create a system to manage tax payments on Branded Prescription Drugs

So, what is the current status in the US? In my first talk, I outlined about the failure of the FBI Virtual Case File project, and how the subsequent Sentinel project initially faltered. It started with a massive Project Management Office (PMO). This goliath superstructure tried to control the supplier - over 100 million dollars was spent on management 1/4 of the whole budget - completely wasted until an Agile approach was adopted and the PMO was disbanded. The introduction of Agile at the FBI was initiated by Chad Fulgham, who came in from the private sector, reorganised the Sentinel project to use Agile and led the project to success. Fulgham left earlier this year to return to the private sector. And, last year, Vivek Kundra left his post as Federal CIO for a research position at Harvard. Whether the US government can maintain the momentum behind incremental, accelerated delivery and implement Agile remains to be seen. The Government Accountability Office (GAO),

However, despite the GAO's enthusiasm for Agile, the CIO Council, now without Vivek Kundra, has yet to supply any leadership with regard to the take-up of Agile specifically. The Council has released guidance on modular procurement and modular development but has not specifically addressed Agile practices. Vivek Kundra is a tough act to follow. We will have to wait to see whether the new Federal CIO, Steven Van Roekel, can keep up the pressure for reform.

Conclusions
What I have found fascinating in my research are the similarities and the differences between the US and the UK. Both the Obama and the Cameron administrations have similar aims with regard to flexible IT development, but they have taken different approaches towards making those changes. On the one hand the US IT Strategy has measurable targets, but they mainly relate to deadlines for the production of

yet more guidance material on modular development running training courses setting up clear project management career paths.

suggest, to assess how the switch to Agile in the US and UK governments is progressing. To realise this goal of proving the benefits of Agile to government, we need research over the next 3-5 years to show: How many projects are actually using Agile? Which strategies for making the switch have really worked? What evidence there is that a switch to Agile has brought an economic benefit?

On the other hand the UK IT Strategy has a vaguely defined target of half of major ICTenabled change programmes being Agile by April this year - a great intent, but one that is difficult to measure and looking increasingly over-optimistic. Certainly, even though both governments have the aim of moving away from Waterfall, the approach in each case is different. Vivek Kundra in the US used a hands-on approach to re-shape failing projects for faster, more incremental delivery. In the UK a consensual approach of management by committee has been adopted. At the recent RAISE conference, I presented to the Agile Research Network a medium term goal, over the next 3-5 years, to assess how the switch to Agile in government is progressing, and whether it is delivering to the citizen and reducing the cost of project failures. This research will be helpful in identifying where the roll-out of Agile is faltering and why. We need to understand what barriers there are to the use of Agile in government, and how to overcome those barriers. The follow-up survey by the NAO, as we have seen, found it impossible to identify a consistent list of the Agile projects that are underway. There is a window of opportunity here, I

This is a fertile area in need of more research - perhaps some collaborative, trans-Atlantic work. Will the US resolve and clarify how 'modular' approaches relate to Agile approaches - are these terms synonymous? Is the new US guidance really any more than a new set of regulations - how can the culture change that is needed be enacted? And will the vigorous and decisive leadership during Vivek Kundra's term as Federal CIO be sustained now that he has moved on to fresh pastures? So, finally - we have a unique opportunity to contribute to the roll-out of Agile by the existence of similar Agile strategies, in both the US and the UK. It's up to us to seize the opportunity and show the taxpayers of the world that government can be Agile!

you! Thank you !

Brian Wernham's new book, "Agile Project Management for Government" was published this summer by Maitland and Strong (ISBN 978-0-957-22340-0) brian.wernham@gmail.com
8

Blog: http://brianwernham/wordpress.com