Federal Agility: a Cultural Solution to a Technical Problem

TL;DR: Increasingly slim federal IT budgets are best optimized by a concerted effort at realigning organization values around more agile approaches to software development

6 minute read

With the talks surrounding the fiscal cliff, never before has it been so vital that the federal government do more with less. Across the District, government agencies are tightening their belts considerably, but the challenge is not simply about trimming budgets or spending less. In many cases, the problem is a matter of spending smarter.

While the federal IT infrastructure is beginning to show its age — losing its ability to serve federal employees and thus more broadly the American public — federal approaches to IT procurement and management are increasingly proving themselves to be equally anachronistic. Traditional heavyweight philosophies known most commonly as waterfall development simply move too slowly for today’s quickly changing federal IT landscape. By the time projects reach completion, all too often, the underlying technology has evolved or the customer’s needs have fundamentally changed. As a result, what is delivered often does not even resemble what is ultimately needed. If we wish to create the efficient government of the 21st century, we must jettison traditional approaches to IT project management and adopt a more agile philosophy.

Agile development has an established history in the private sector. Tracing back to Toyota’s lean engineering practices of the ’70s, agile is an essentially an incremental approach to software development that breaks a project into many independent, fungible tasks. Unlike traditional waterfall development, it does not seek to build software in a sequential, step-wise manner. Instead, it recognizes the reality that both requirements and technology are unpredictable and change over time. As a result, project plans are continuously inspected and adapted through iterative, rapid prototyping. Empirically, agile delivers software faster, cheaper, and with a greater emphasis on end user value.

Technology has Outpaced the Laws that Govern its Adoption

Yet despite these advantages, traditional government contracts have not been conducive to agile, and with good reason: As a matter of public policy, procurement regulations force agencies to prefer full and open competition over sound project deliverables. By purposeful design, federal procurement emphasizes the process surrounding the purchase, not on the purchase itself. To ensure bidders have equal footing, traditional contracting vehicles require that stable requirements be outlined up front and rigidly adhered to, regardless of advancements in technology or changes to the agency’s needs. From the agency’s perspective, contracting officers must often secure and commit funding well in advance, rendering adaptation impossible and Agile’s “I’ll know it when I see it” approach, somewhat counterintuitive. Making matters worse, due to this significant administrative burden, agencies rightly seeking to minimize transaction costs must approach projects in large, single-pass batch processes rather than addressing the constituent parts as needed, and thus are all-too-often forced into organization-wide transformations doomed for failure from the start. These procedural necessities — rarely present in the private sector — are not only toxic to an approach that thrives on small iterations, but more importantly generate significant lead times further hindering agencies’ ability to be responsive to its customers’ needs.

These potential stumbling blocks, however, do not necessarily prove fatal. This seemingly technological challenge is, most elementally, a human one. Policy my be relatively easy to change compared to calcified culture predominant in today’s IT shops. Along side any effort to rethink agencies’ approach to requirements at the time of solicitation, we must seek to grow a more agile culture within our existing workforce. Like any effort at organizational change, none of this would be possible unless it is communicated as an agency- and government-wide imperative, and ultimately seen to fruition on a project-by-project and task-by-task level by the IT professionals that will be its daily shepherds. Here are five steps that will lead in that direction:

1. Rethinking an innovation-averse culture

Government agencies embody an entrenched culture of rigidly defined parameters, documentation, and overall, a meticulously defined hierarchical process. Uncertainty, and thus risk, is mitigated through policy that, by its very nature, resists change at all levels, especially a potentially disruptive change like agile. Before any degree of agility can be injected into a stereotypically top-down environment, managers must seek to create a more egalitarian, startup culture within their own business units.

At the most basic level, it is not agencies that purchase the government’s IT solutions, but rather individuals on behalf of those agencies. Like their private-sector counterparts, care must be taken to ensure that these individuals’ incentives remain in line with those of their employer. Despite this, government budgeting continues to focus primarily on projects that fall within a single budget year, are controlled by a single budget program, and promise an easily predicted, low-risk return. Put another way, existing values structures discourage the early failure necessary to facilitate agile’s internal learning mechanisms.

For example, a contracting officer that adopts an innovative, but ultimately flawed approach may face reprimand or termination when presented with the technology’s underperformance, while without explicit financial incentives, gains little if the experiment proves beneficial. Put another way, federal employees are encouraged to avoid near-term risk and thus have a disincentive to improve well entrenched but outdated processes like waterfall. New ideas must compete against proven ones, and all-too-often known problems take precedence over potential improvements. High-value, but high-risk ideas like agile require the kind of no-fault support from upper management that only a cultural shift can bring before they can begin to take foothold in the federal workplace.

2. Infusing agile knowledge into an otherwise waterfall workforce

Across the board, federal employees, from contracting officers to in-house IT departments to agency CIOs, many of whom have risen through the ranks of primarily waterfall federal bureaucracies, are often not accustomed to managing an agile project, nor do they often find the necessary encouragement to take on such an endeavor. These federal employees, the front line of agile adoption, must be given the necessary toolset to properly contract and manage agile projects in an otherwise “over the fence” culture.

While many cities, Washington, D.C. especially, may offer agile training and certification, by-and-large federal employees seemingly do not take advantage of such opportunities. Like any industry, federal IT professionals at all levels must make a commitment to stay abreast of significant developments in their field. Project management has undergone a radical transformation in recent years. Whereas once mainframe computers and batch processing rendered change costly and thus an analogy to traditional engineering fields apropos, today cloud computing, commodity hardware, and the adoption of higher-level languages has created a more interactive development process that make mistakes made under an agile framework negligibly cheap. Yet this game-changing inflection point has yet to be fully appreciated by government, despite its widespread adoption in the private sector. Without the necessary knowledge that there’s a better alternative out there, project managers will continue failed practices, and thus efforts will remain simply top down. Instead, management must seek to infuse agility into the IT workforce either through a concerted effort at continued education and strategic hires at all levels.

3. Focusing on functional, rather than technical requirements

From a practical standpoint, to encourage agile adoption, solicitations and other formal documentation should outline functional, rather than technical requirements, and should procure the underlying talent necessary, rather than the technology itself. Instead of specifying that a system use a particular encryption algorithm, for example, and thus risk that technology being obsolete or no longer needed at the time of delivery, requirements documentation should outline the functional goal that the agency really is after — in this case, industry-standard security practices — and procure the services of the necessary security experts.

These requirements should be generated without the significant lead-time often seen in government procurement efforts and should involve all relevant stakeholders from the contracting officer to the technical point of contact and future system. Put another way, an agile contract cannot be written from a waterfall mindset. The statement of work should specify high-level goals or systems (rather than specific features) that will eventually become the framework that defines each subsequent sprint. If necessary, the contract could include specific, functional milestones and accompanying deadlines, which may be married to an award schedule.

4. Making innovation an imperative at all levels

Like any push for organizational change, none of this can happen without significant, top-down support. Federal opinion-leaders like the Office of Management and Budget must lean heavily on agencies to adopt more modern software development practices. Without significant political pressure both within an agency and between, reform will remain simply rhetorical. Tangible, specific guidance from the administration to agency CIOs, such as the recent modular contracting guidance is a necessary step, but so long as long as multi-year, bet the business procurements remain tolerated, we will fail to overcome significant organizational inertia.

Second, relevant stakeholders within an agency must be empowered to take risks with, what is by government standards, a radically unconventional approach. Hierarchical, seniority-driven status and an over reliance on forms and formal processes — two staples of government — are antithetical to agile’s reliance on face-to-face, informal communication. When responsibility lies in management, and thus is distinct from the actual development activity, such an approach inhibits the fast, authoritative decisions that are the hallmark of agile.

5. Applying agile-to-agile adoption

It would be foolish to suggest that a transition to agile can happen on a government-wide or even agency level. Instead, like the processes it espouses, the push for agile should be iterative and rely almost exclusively on successive prototyping across business units. Individual projects, or even segments of individual projects, should be attempted using a fully agile framework, and slowly iterate through the agency’s efforts as subsequent development efforts learn from the evaluation of past sprints.

Beyond Technology

Government agility is not important in so far as it helps agencies hit increasingly frugal budget targets, but rather because when federal projects are not delivered on time and on budget agency objectives suffer. IT programs that cost more than they should or ultimately fail to meet agency needs, means the Department of Education helps fewer students, the Department of Health fosters cures for fewer patients, and the Department of Commerce creates fewer opportunities for the unemployed.

Technology long-ago lapped the approach the government has traditionally taken to managing it. A highly entrenched, management-directed culture fatal to agile has taken foothold in the nation’s capital, and shows no signs of lessening. Agencies must seek to build a workforce open to embodying the agile philosophy not only on paper, but also in their day-to-day work. Functionality and the agency’s mission, not technology, should rule the workplace. None of this is possible, however, without significant pressure, both top down and bottom up, to make this cultural shift not only a priority, but a necessity of survival in today’s 21st century government. As the public’s scrutiny turns increasingly toward the federal budget, it is especially important that agencies are smarter about their development efforts and see them not as a line-item expense, but rather as a potential cost-saving innovation.

Originally adapted from, Towards a More Agile Government, 41 Pub. Cont. L.J. 149, The Public Contract Law Journal, Fall 2011. Significantly updated, December 2012.

Originally published November 1, 2011 | View revision history

If you enjoyed this post, you might also enjoy:

Ben Balter is the Director of Engineering Operations and Culture at GitHub, the world’s largest software development platform. Previously, as Chief of Staff for Security, he managed the office of the Chief Security Officer, improving overall business effectiveness of the Security organization through portfolio management, strategy, planning, culture, and values. As a Staff Technical Program manager for Enterprise and Compliance, Ben managed GitHub’s on-premises and SaaS enterprise offerings, and as the Senior Product Manager overseeing the platform’s Trust and Safety efforts, Ben shipped more than 500 features in support of community management, privacy, compliance, content moderation, product security, platform health, and open source workflows to ensure the GitHub community and platform remained safe, secure, and welcoming for all software developers. Before joining GitHub’s Product team, Ben served as GitHub’s Government Evangelist, leading the efforts to encourage more than 2,000 government organizations across 75 countries to adopt open source philosophies for code, data, and policy development. More about the author →

This page is open source. Please help improve it.

Edit