Why your agency’s first open source project is going to be a flop

TL;DR: Best practices suggest that you should prime organizational pistons with an open source exercise, before you embark on your first highly publicized open source effort

4 minute read

If you’re a government agency, your first opens source project is likely going to be a flop. In all fairness, your organization has been set up to fail. Participating in the open source community bears little relationship to the decade (or century) old processes your agency has established, like those required to publish a press release or respond to a FOIA request. It’s not an understatement to say that open source is an animal unlike any your agency has ever seen before, and one that will require the cooperation of multiple departments.

The typical startup hockey stick

In startup land, new products are brought to market with an adoption curve that somewhat resembles a hockey stick. At first only a handful of early adopters kick the tires, things grow in at a linear rate, once the product is solid, a critical mass is achieved, and growth explodes exponentially, often with no end in sight. You can see this phenomenon in the adoption of open source by government employees, but the examples are countless. Even when an established firm brings a new product (or even a feature) to market, it’s done in such a way to artificially emulate this same adoption curve (to hedge the risk of trivial issues becoming serious by virtue of their scale). You’ve likely seen startups launch a new product in “private beta”, but it also happens behind the scenes every day in ways you likely don’t know (by design), with hidden experiments and phased rollouts.

Government, on the other hand, almost always adopts a reverse hockey stick approach to new initiatives, technical or otherwise, opting instead for a baptism-by-fire rollout. Take the launch of healthcare.gov as an example. On day one, demand is seemingly endless, interest decays exponentially over the following months, followed by a long tail of linear decline as the application exists in perpetuity (or until replaced wholesale). If a problem is discovered when the first (or second) user signs up, it’s already too late to correct, because by the time a fix is in production, user 10,000 (or 100,000) has already experienced it. Embarking on a new open source project is not dissimilar, and the results are likely to be the same without a different strategy.

Software that’s too big to fail

When starting your agency’s first open source project, the process necessarily involves lots of firsts for lots of people. For developers, it may be the first time that users, especially technical users, will scrutinize their code. For project managers, it may be their first time adopting a more open, more iterative workflow. For lawyers it may be their first time advising the agency on an entire field of intellectual property law. For the security team it may be their first time participating in collaborative code reviews. And for the public affairs office, it may be their first time directly engaging the developer community, trolls and all. For your agency’s first open source project to be a success, a lot of one-in-a-million chances need to go right. In fact, they almost all do.

Almost without exception, up until this point, government has preferred to wait until something is absolutely perfect before it’s operationalized—public policy, tank, battleship, code, or otherwise. Think about it logically: if you’re writing a law, or a treaty, or building an aircraft carrier you’d better hope you got it right before others depend on it. Adding to the problem, government agencies are also incentivized to maximize batch size by virtue of the bureaucratic process (once again, encouraging “too big to fail” initiatives). If getting one line of code into production takes six weeks of change control board reviews and getting 10,000 lines of code into production takes the same six weeks, logic dictates that batches should be maximized to minimize meta-work.

Ship 0.1, not 1.0

The most effective way to hedge the risk of too-big-to fail software is through iteration. Lean startup. Fail fast. Validated assumptions and all that. In the open source world, that means that when starting something new, you should version 0.1 (the minimum viable version of the thing you’re trying to create), not 1.0 (what you’re hoping to eventually). So long as you communicate the projects status (for example, “it’s a prototype”, “it’s a proof-of-concept”), the community will respond accordingly. Once you’ve gotten real-world, production-grade feedback, you can add additional functionality in a 0.2, and eventually ship the 1.0 you originally set out to create.

The irony is that government agencies often initially ship software that would be called 0.1 any place outside of government, but they give it the self-proclaimed badge of 1.0 quality, both internally and externally. Despite expectations or up-front planning, no matter how good your developers, no matter how good the project management, no matter how good the idea, humans are inherently imperfect, and thus so too are the tools they create. To paraphrase internet wisdom, waterfall approaches assume you can predict the future, open source assumes no one can can. The process of creating, releasing, and maintaining open source software is more art than science, and the only way to learn is by doing, not by planning, no matter how hard you try.

Think small, then think even smaller

Successful forays into open source begin not with large, flagship initiatives, but with small, nobody-will-notice-if-they-fail efforts. In fact, the story often starts well before a single line of code leaves the firewall. As I’ve said before, the technology is the easy part. It’s not a question of if your agency can write the software, or if the agency can publish it.

For open source to work right, a lot of organizational pistons need to fire in perfect timing. It’s more of an effort in organizational change than anything else. Look at it this way: despite their best efforts at public engagement, you’d be hard-pressed to find a successful government open source project that’s open source publicly, but follows an entirely closed and waterfall process behind the firewall.

Instead, best practices suggest that you should prime those organizational pistons with an open source exercise first, before you embark on a highly publicized effort. Create a list of the best lunch spots near your office in a private repository. Have coworkers submit pull requests with their favorite chocolate chip cookie recipe. Whatever it is, there’s immense value in forcing the organizations to go through the motions of open source as part of a smaller-scale skirmish—procurement, security, legal, developers, public affairs—than learning those organizational skills in real time.

If you want your first foray into open source to be successful, then start small, facilitate the organizational muscle memory necessary to allow you to focus on engaging the open source community, not fighting the process, and if you’re going to stumble as you learn, then fail small and privately, not largely and publicly. Otherwise, your government agency’s first opens source project is almost guaranteed going to be a flop.

Originally published February 11, 2015 | View revision history

If you enjoyed this post, you might also enjoy:

Ben Balter is the Director of Engineering Operations and Culture at GitHub, the world’s largest software development platform. Previously, as Chief of Staff for Security, he managed the office of the Chief Security Officer, improving overall business effectiveness of the Security organization through portfolio management, strategy, planning, culture, and values. As a Staff Technical Program manager for Enterprise and Compliance, Ben managed GitHub’s on-premises and SaaS enterprise offerings, and as the Senior Product Manager overseeing the platform’s Trust and Safety efforts, Ben shipped more than 500 features in support of community management, privacy, compliance, content moderation, product security, platform health, and open source workflows to ensure the GitHub community and platform remained safe, secure, and welcoming for all software developers. Before joining GitHub’s Product team, Ben served as GitHub’s Government Evangelist, leading the efforts to encourage more than 2,000 government organizations across 75 countries to adopt open source philosophies for code, data, and policy development. More about the author →

This page is open source. Please help improve it.

Edit