Lean Software Development

Introduction

Hello, developers! 🚀

Today we talk about one of my favorite stories: Lean Thinking was born in the 40s in Toyota in the attempt of the Japanese company to become a leader in the automotive world business. This attempt will be successful and will build the legend of the Toyota Production System and its inventor Taiichi Ohno.

Ohno realized that the main issue of Toyota was that they were used to a PUSH mass production approach because this was the successful approach they were used to when creating military means. The Japanese car market didn’t have the same size and requirements, of course, so he brought the idea to shift to a PULL production approach, driven by customer demand.

The Toyota Production System has become legendary and it’s still evolving and in use today, its main goal is to reduce waste over the entire flow of work, and its main characteristics are:

The right process to produce the right results: a pull system to avoid overproduction and balance workload, building a culture of stopping to fix problems and getting quality from the start
Add value to the organization by developing your people and partners: grow leaders who can then teach others, develop exceptional people and teams, and help your partners and suppliers improve
Continuously solving root problems drives organizational learning: go and see for yourself to understand the situation, make decisions by consensus considering all the options, and build a learning organization

What this means is that it is a system for thorough waste elimination. Here, waste refers to anything which does not advance the process, everything that does not increase added value. Many people settle for eliminating the waste that everyone recognizes as waste. But much remains that simply has not yet been recognized as waste or that people are willing to tolerate. [Wikipedia]

Lean Thinking is the methodology born from the Toyota Production System: since TPS worked well, it has been used as a model to build a framework to replicate the same principles in different types of companies, businesses, and contexts.

From manufacturing to software

In a progressive process, Lean Thinking was taken as inspiration and transposed from the manufacturing to the software world, because smart people started to notice that there is much that the software world can learn from Lean manufacturing - a good (invented) story about this is the book “The Phoenix Project”.

In 2003, Mary and Tom Poppendieck wrote the book “Lean Software Development”, where they formalized the idea of Lean applied to Software, describing the 7 principles they based it on:

Eliminate Waste
Amplify Learning
Decide as late as possible
Deliver as fast as possible
Empower the team
Build integrity in
See the whole

In a similar way to XP, since principles are pretty high-level, they went deeper with each of those to give some more low-level and practical description of what they meant.

Eliminate Waste

Eliminating waste is the most fundamental lean principles.

Software development has multiple types of waste, and the first step is to understand how to recognize them:

unreleased work
obsolete or unused features
defects and bugs
context switch
waiting time
handoffs
unnecessary processes or documentation
management activities

Eliminating waste is definitely the place to start, and the main activity to achieve this target is to measure the lead time: you should map the high-level process from the idea to giving value to the customer, and then measure the time of this process including both work and wait time; this is the lead time, and once you are measuring it your objective is to pick the biggest opportunities to increase flow and reduce the lead time (theory of constraints) and implement it - then measure again to confirm the improvement, and move on with the next bottleneck.

Amplify Learning

Generate the most knowledge at the least cost possible.

Software Development is most often a learning process, so we should embrace that it involves trial and error and handle it by consequence, for example:

make sure feedback loops are in place and at the highest speed possible
keep synchronization high: integrate code changes in small batches and make it fast to build and test the system (CI)
work in (short) iterations delivering the highest priority item first and then get feedback
Talk about constraints instead of solutions, and allow the obvious solution to emerge by getting feedback on multiple options

Decide as late as possible

The target is a robus, change-tolerant design for the system so that it can be readily adapted to most likely changes.

We need to avoid that changing code becomes always harder because of new code, and to do that we need to favor emerging design over time. To achieve this target, we want to decide as late as possible following 3 main principles:

Options Thinking: make decisions under the road, delaying irreversible decisions until the uncertainty is gone (or reduced to the minimum); be open to doing something new in the future without building it before it’s needed and improve your skills in understanding what options are worth to keep open
Last responsible moment: delay decisions until failing to make a decision removes important alternatives - to delay commitment, favor abstractions over specific representations, separate interfaces from implementations, and use feature flags; the only decisions to take early are those that facilitate design to emerge
Making decisions: plan work without predicting it, and use simple principles or guidelines to enable decision-making by the people closest to work.

Deliver as fast as possible

The faster is delivery, the longer decisions can be delayed.

Delivering fast is important to reduce the feedback loop and reduce the weight of reversible decisions; to optimize speed without dropping quality, there are three main pillars to follow:

Implement a pull system, where customers pull the work instead of a schedule pushing it; use Kanban to visualize the work so that everyone can see what’s going on.
Follow queuing theory to reduce cycle time: split incoming work into small batches, prioritize work regularly, make people focus on bottlenecks, and limit variability in work; the target must be to optimize for quicker cycle time instead of people utilization, so you should always leave enough slack in capacity and a parallel independent stream of work.
Delays have a cost so we want to reduce it: give the team a simple, economic model, empowering it to make tradeoff decisions by putting a price tag on delivering vs delay; avoid trades of time vs scope vs money by making it all about money.

Empower the team

Give people the power to decide for their work, but also invest in developing their capacity of making decisions.

Decisions should be at the lowest level possible to ensure they are contextual and fast. You should empower the team by helping people to develop some characteristics: the team should feel responsible for their own work (including processes, practices, quality, etc), while managers should listen to the teams and help them remove organizational blockers but also coach them.

Allowing the team to commit together to achieve their objectives, celebrating successes, and allowing for mistakes will make people more motivated and committed, which is one of the objectives of leadership: set direction and lead, don’t manage.

Finally, ensure that the expertise of the team always grows over time: encourage experiments and continuous training and learning, favor sharing sessions to spread the knowledge gained, favoring conversations over documentation) and build internal communities of expertise.

Build integrity in

Integrity is a key differentiator.

We need to build great product integrity that must be perceived in the eyes of the customers - this integrity is built by multiple pieces:

Clear communications with customers: we need to ensure that the communication flows between customers and the team in both directions; it must be easy to receive feedback about the last iteration, but also to keep customers updated on what’s new; customer’s problems should always be kept as a priority when making decisions, supporting the team in achieving and maintaining a deeper customer understanding.
Clear communications within the team: ensure that the core concepts of the product belong together, and enable a regular flow of information between everyone working on the product, making it easy to discuss and solve problems together.
Continuous refactoring: foster a clean code culture to reduce the cost of refactoring and enable refactoring code daily to maintain the codebase under control; allow design and architecture to evolve so that you can delay decisions also on them; to ensure you refactor only where it’s worth it, try to refactor a piece of code right before it needs to change.
Testing: help the team in improving their testing knowledge and skills to ensure they use all the best possible testing techniques, from unit to e2e tests; ensure testing is automated, and use acceptance tests as a tool to ensure customer needs are met.

See the whole

The ability of a system depends on how well the parts work together.

We should always strive to improve the whole system and not the single parts: the focus when removing constraints, addressing problems, and measuring results should be on the entire system and not on an individual level.

Avoid measuring locally, because you want to optimize the whole and not just a small part: measure group productivity, not individual one - we want to win a marathon, not only a section.

When writing contracts with supplies and vendors assume good intent, and limit damaging behavior through relationship, not the contract, favoring a kind of contract that shares goals instead of fixed price ones.

A successful model for businesses

After all this history about Lean, and considering the words I use to talk about it, it’s probably quite clear that I love the Lean approach: it’s just a perfect match with the Software Industry and it should be much more considered.

The first time I heard about the Toyota Production System and Lean was in the Phoenix Project book: a narrative/tech book that tells the story of Bill, an IT Manager at a company called Parts Unlimited. Bill is responsible for a project that is fundamental for the future of the company but it’s massively over budget and later (codename “Phoenix”): he has 90 days to make it successful, otherwise the entire department will be outsourced. Luckily, he will receive the help of Erik, a new board member who shares his knowledge of TPS, Lean, and DevOps with Bill.

Through Erik, the authors make this book much more than simply a fun narrative about very common historical issues of the Software industry: through the story, you can learn the basics of those methodologies - and this book does not only have a sequel (The Unicorn Project) but also a strictly related book named The DevOps Handbook, a more classical tech book from the same authors that deep dive into the methodologies used in the story.

I was already into Agile and XP at that point, and I knew the basic principles of DevOps - I was fascinated by the story of Toyota and the Lean methodology, especially because it’s pretty clear that Agile, XP, Lean, and DevOps share a lot: to be honest, I treat them as a family.

Once you deep dive into Lean Software Development, you can either be fascinated or horrified I guess - based on which kind of professional developer you are: to me, a born team player who LOVES the team dynamics of collaboration practices such as Pair/Mob Programming, the focus on the entire system and not on individuals not only makes a lot of sense (that should be objective…) but also motivate me a lot.

Thanks to Lean, I learned to focus on flow and became interested in topics such as reducing work in process, especially the “extreme” version of it: Single-Piece Flow, where we can only work on one thing at a time - I even dedicated an old issue of the newsletter and a talk to this topic that seems so revolutionary in a world where multi-tasking is considered somehow the standard, completely ignoring how our brain works.

The 7 Lean Software Development principles I described above describe the way I see software work in a way that is probably even better than what Agile and XP can do because they mostly focus on team and company as a system, and if there is one thing I can say is that if the system doesn’t work you will fail, no matter if you have the best people in the team: there’s no way out.

Also, the Toyota story is a good conversational tool when you are trying to sustain the introduction of practices: the idea that a big company such as Toyota thinks that it’s a good idea to introduce the Andon cord to allow anyone over the entire production line to stop the entire line if something breaks up is incredibly powerful; making people responsible for the quality, and when something happens everyone focuses on the problem, instead of continuing the work on the working parts, not only enable a quicker fix of the problem but also avoid to accumulate work in the broken part of the system that would cause a slower restart when the fix is done - therefore, stopping all the work is better because it reduces the overall waste.

The first time I understood this, it was mind-blowing.

Lean is even more fascinating to me than Agile and XP, and this was a surprise for me: now I think the reason is that XP and Agile were made by developers and only after reached also other businesses, so they might be perceived as biased, especially when trying to foster them with skeptics or non-technical people.

Lean, on the other hand, did the opposite way: it was born in the manufacturing world, and only after it reached the Software world. This has some positive side effects:

it’s far easier to have concrete examples that it works, and Toyota is just the biggest example of them all - this makes the conversation with non-technical people far easier
It has an intrinsic match with businesses (reducing waste, we reduce costs and will more likely be successful and obtain more profit) and therefore the Software implementation of Lean seems more understandable to business people

I strongly believe that Lean is an amazing methodology to use in both business and software, and I hope that Lean Startup and Lean Software Development methodologies will keep growing in popularity because this can only bring positive side effects to our world.

Until next time, happy coding! 🤓👩‍💻👨‍💻

All we are doing is looking at the timeline, from the moment the customer gives us an order to the point when we collect the cash. And we are reducing the timeline by reducing the non-value-adding wastes.

[Taiichi Ohno]

Go Deeper 🔎

📚 Books

The Toyota Way - A classic in the world of business management, The Toyota Way has been helping leaders create powerful strategies based on the automaker's legendary management principles and philosophy for nearly two decades.
Taiichi Ohnos Workplace Management - This unique volume delivers a clear, concise overview of the Toyota Production System and kaizen in the very words of the architect of both of these movements, Taiicho Ohno.
Lean Software Development - In Lean Software Development, Mary and Tom Poppendieck identify seven fundamental "lean" principles, adapt them to the world of software development, and show how they can serve as the foundation for agile development approaches that work. Along the way, they introduce 22 "thinking tools" that can help you customize the right agile practices for any environment.
Implementing Lean Software Development - While the previous book provides an introduction, theoretical advice, and a reference to Lean, this follow-up incorporates their gained knowledge and understanding of what works and goes steps further to provide hands-on guidance for implementing a Lean system.
The Phoenix Project - The CEO demands Bill must fix the mess in ninety days or else Bill’s entire department will be outsourced. In a fast-paced and entertaining style, three luminaries of the DevOps movement deliver a story that anyone who works in IT will recognize. Readers will not only learn how to improve their own IT organizations, they’ll never view IT the same way again.
The DevOps Handbook - Technology is now at the core of every company, no matter the business model or product. The theories and practices laid out in The DevOps Handbook are tools to be used by anyone from across the organization to create joy and succeed in the marketplace.

📩 Newsletter issues

📄 Blog posts