4 Best Practices for Repeatable Software Delivery

What does software delivery really mean? Why does it matter? In the age of minimum viable product, how does the concept of a software delivery process fit in? These questions, and more, come up every time companies look to shake up their engineering organization. 

Throughout my career, I’ve worked in a variety of product and software delivery styles. There isn’t one true software delivery path; what works for a startup with seven people will not work for a larger corporation with 700 or 7,000 people. Likewise, software running on critical infrastructure requires a different delivery process than something running on a mobile phone. But just because there isn’t a singular path doesn’t mean there aren’t commonalities across all processes that lead to better software delivery. In fact, if you’ve read the tenets of the Agile Manifesto you probably have a good idea of what these commonalities are: 

• Software Process: How teams design, develop and deliver software solutions. 
• Minimum Viable Product (MVP): The bare minimum amount of work we can do to deliver some value to the market so we can better map the direction of our product to the needs of our customer. Typically we incur technical debt, which limits our product as we move past the MVP. 
• Agile: A methodology where teams continually learn and work to improve how they work together to deliver product. 
• Scrum: Scrum is a methodology designed to give teams the framework to reach the ideals of Agile. Scrum is not Agile. 
• SAFe: Scaled Agile Framework is designed to scale Scrum to larger corporations and map the predictability of short waterfalls to Scrum processes. See Boehm Spiral. 
• Software Development Life Cycle (SDLC): The manner in which your company chooses to manage how software is created and delivered. Many times the SDLC is written out and ignored, at other times it doesn’t exist and is talked about as a silver bullet.
• CI/CD: Continuous integration and continuous deployment are the cornerstones of any successful software process. If your MVP does not contain CI/CD, then your MVP will become too difficult to maintain.

Agile is a mindset that allows software delivery teams to have some say over how they operate and allows the company to take feedback/learnings and adjust. This is imperative and the most important point of this post: Teams Matter. 

There is a converse on what Agile isn’t: Agile isn’t a way to ensure *only* the work on the board gets done. Agile isn’t anti-waterfall either, there are many paths for software delivery teams to learn and function well together. 

Agile is a way to do just enough to ensure what you build meets the requirements so you can then reflect and adjust to the ever changing market landscape and lessons learned. Agile, when done correctly, allows for variability to enter into the product design in a controlled manner. 

The idea of MVP goes together, quite nicely, with Agile practices. This is not always a happy union though. In my previous post on the “filtration stack”, we went over how an appropriate software quality testing process helps build world class software. There is another way the filtration stack, when combined with Agile, also helps your MVP cover the oft forgotten requirements that make for a truly delightful product. For example, your filtration stack could have third party testing, build manifests, and compliance and governance verification at build and run time. This yields a much more robust and delightful product. Without the testing and verification, the team wouldn’t be able to focus on quality which would lead to a much less delightful customer experience. 

For the rest of this post, the discussion will focus on processes that tend to promote good software development. 

When a new product idea comes about, and is pressure tested enough to start development, we begin our SDLC journey. One of the first steps to a successful project is to start with why: why are we building this software? The “why” should demonstrate what problems the software solves and answer why it’s best for us to build it. Everyone wants to know the “why” and believe in the “why” so they can effectively work toward building the “what”. 

The “why” leads to software delivery requirements. One of the best things about Agile development ideas is the concept of “just enough.” This allows teams to define requirements and questions to answer which then drives the direction of the work. 

The questions to answer, in particular, are really the most important. The best teams look at the questions that most need answers and prioritize this work first. Typically, these are the hardest questions that could change the course of the project. 

Many years ago, I was on a team building a robot and we didn’t know if our ability to map and navigate in the environment would work. We prioritized the navigation work early, which clearly showed the problem was much harder than we anticipated. To compensate, we changed the physical design of the robot so errors in navigation wouldn’t lead to disaster. We tackled the hardest problem, took the information and adjusted our product. 

This brings us to another thing the best teams I’ve worked with have done: allow variability early in the software delivery process. One of the dangers of any software delivery management process is anchoring to the requirements and schedule too soon. So many software projects are late and over budget and one of the reasons for this is managing the requirements. Early on, there are so many unknowns about how the software will work and how developers will interact with it. Software applications tend to be complex systems, and complex systems are hard. When developers are given freedom to explore the design and development of the software, typically they’ll uncover new requirements and possibly more efficient paths to the desired outcome. This also ties back to the “why” as well, since knowing the “why” allows for new and different ways to reach the desired goal and not just match a requirement. 

At a previous company I worked at, a developer who reported to me kept getting in trouble with his team’s product owner because his work wasn’t on the “Jira board.” The problem was, he was exploring new ways to integrate a desired feature which, to do right, required extensive re-architecture of a critical component. To the product owner, this looked like wasted work as no progress was made on the tickets assigned for the feature work. He wouldn’t accept “exploring” and “spikes” as viable work since the stories could be done as-is. When the developer was done, the work he did reduced the complexity of the software 10x and led to a much more robust, maintainable and reliable solution that our customers were delighted with. We want engaged software developers who are craftspeople, so we need to have processes that treat them as such. 

That’s not to say we should never manage requirements, we just shouldn’t do it early in a new project unless that project is really simple. As projects cross a threshold where the work is fairly well known, and now it’s a matter of “just” delivery, the best teams will not only focus on what needs to get done, but they will also look at what they can remove so they can deliver a better experience to their customers. 

The second half of a product’s lifecycle isn’t just reducing the variability, it’s reducing the requirements as well. As we develop product, it’s important to know the costs involved in bringing requirements to market. Costs include the development, expanding the quality filtration stack, and supporting the feature. This leads to one of the other great software inventions that all teams should embrace: feature flags. 

One of the many failings of waterfall was the inability to ship “partial features” - which means companies would hold up releases until all the features were complete, or until access to incomplete features was removed. Building in feature flags from the start is great software practice, as a feature flag allows you to enable and disable a feature. 

Feature flags, controlling for variability and exploration at the right times, building your quality filtration stack and encouraging your teams to continually improve are all hallmarks of a strong engineering practice that can lead to amazing software. These aren’t enough, but they are necessary. 

This has been proven to me time and time again. A few years back, my team delivered on a product where we used feature flags to control when a feature was enabled and to manage the roll out to customers. We used the filtration stack to ensure we had a solid release that we felt would stand up in unique customer environments and we had an idea where we’d fall over so we could recruit companies willing to work with us to improve the product’s quality. By having this knowledge in what we built, we were better able to prepare our customers for what to expect and how we’d address the issues they found. The team took the issues found at the customer sites and rolled the fixes into the code while rolling new tests into the filtration stack so we could catch the issues, and ones like them, internally. The process worked. 

The term MVP took over the software world, and with good reason, as the idea of reducing the problem to the smallest solvable solution greatly reduced the complexity of the software. Sadly, we’ve learned over the past decade that MVPs also lead to shortcuts in software development that, at best, lead to tech debt, and at worst, lead to poorly built software that fails to delight a customer; or, worse, can cause them harm. 

In the era of Agile, MVP, and complex software, delivery teams often try to limit complexity by defining an MVP early on. Typically the MVP doesn’t take into consideration why software is complex because it is geared toward testing ideas in the market. Good design, scalability, compliance to internal standards, and even security can get overlooked for the sake of the MVP. Many ways to make an MVP more robust require external software to verify that it operates within the conditions laid out for your company. For example, if you’re building in the cloud there are open source tools, like the open source rules engine Cloud Custodian, that help and can lead to your MVP entering the world as a well behaved cloud citizen. If you are on a team that forgoes testing in favor of getting your MVP to market then you are in danger of building an MVP that will not delight your customers. 

Teams need a robust methodology to ensure what they build works and is reliable.  The quality filtration stack is part of it. Robust processes that are flexible enough to respond to the changing demands of the project stage are imperative. Teams that continually learn and know why they are building their product will inevitably build better products. When you combine CI/CD with a quality filtration stack and add robust, yet flexible software delivery processes you can create products that are more than an MVP, you can create a product that delights. 

If you are looking for more information on general software delivery theory, please check out the book “Principles of Product Development Flow" by Donald G. Reinertsen.