The Art of Lean Software Development

This is an admittedly short book at only 122 pages. The authors felt that there was a need to have an introductory offering into the world of Lean and Agile methodologies, and have done a great job of keeping the book very focused. They offer an introductory look at some foundational Lean and Agile concepts and provide a clear understanding of how Lean is not in itself a specific process or methodology, but is at it’s core a philosophy that draws on existing processes and methodologies, giving them true value to the business.

The Good

The core of this book discusses what the authors believe are the “five most important practices that you can adopt as you begin your Lean software development journey.” The practices that they outline are common practices that are found in many existing Agile methodologies, so they are likely to be familiar to those who have done any research or read any blogs on recent trends in software development.

They actually list six practices, but consider the first one to be ‘Practice 0’ – what you should be doing, period, even if you are not doing any form of Agile or Lean development. The full list of development practices that they list, are:

• Practice 0: Source Code Management And Scripted Builds
• Practice 1: Automated Testing
• Practice 2: Continuous Integration
• Practice 3: Less Code
• Practice 4: Short Iterations
• Practice 5: Customer Participation

Over-all, I’m happy with the practices that they outline. The practices they chose are very fundamental to most Agile methodologies, and they do a great job of explaining how each of the practices supports the lean philosophies that were discussed at the beginning of the book. Each of the practices is discussed in enough detail from the process standpoint, to make the reader want to know more. They avoid getting into too many technology and platform specific details. However, they often mention frameworks and tools that cover a wide range of platforms, giving the reader ample information to continue their research. The authors also manage to be very up to date in the processes that they discuss, and often mention newer, budding efforts such as Behavior Driven Development and applying the Theory Of Constraints to software development.

While the practices may be engineering focused, they present them as benefits to the business of software development, which I applaud greatly. Far too many Agile books and discussions are focused entirely inward toward the engineers and project managers, with little to no regard for the customer (other than the standard ‘co-location’ or ‘constant communication’ lines).

Throughout the book, including the final chapter on ‘What Next’, there are multiple references to existing literature on the various tools, technologies, philosophies and practices, including Lean resources. The last portion of the book is dedicated to listing these references in a well organized, categorical manner. This makes it very easy for the reader to find additional resources on the information provided in the book.

Personally, my favorite chapter of the book was chapter 1: ‘Why Lean’. This chapter introduces many of the problems that we have known about in software engineering. They go on to talk about the Agile Manifesto and mention many of the popular Agile methodologies. The real meat of the chapter, to me, is the explanations used to justify Agile and Lean to the business – actual studies and reports generated from real world projects reported under ‘The CHAOS Study’, with over 40,000 (yes, that’s forty-thousand) projects in the study. I find myself referencing this chapter and this data time and time again. It’s a very enlightening dataset.

The Bad

There are a few questionable items in the book, in my opinion. I would not say these items are show-stoppers to prevent people from buying the book, though.

Code Coverage

The first item that jumped out at me is in Practice 1: Automated Testing. At the bottom of Page 49, they say the following (emphasis is mine, to illustrate where I take issue):

It is unrealistic, and probably not worth the cost, to achieve 100% test coverage of your source code. For new code and projects that are new to automated testing, 60 to 80% code coverage for your tests is a reasonable target, although seasoned testers will strive for 80 to 95% coverage. There are tools that will examine your source code and tests, and then tell you what your test coverage is (for example, Emma and Cobertura for Java, and NCover for .NET).

I am disappointed that they would make such a blanket statement without any conditions or explanation. I think the notion that 100% code coverage is ‘unrealistic’ betrays the authors’ personal experience and technologies that they develop against. Yes, there are many cases where 100% code coverage is not reasonable (integration with Sharepoint, for example), but there are plenty of cases where 100% coverage is possible and should be expected. At the very minimum, I would prefer them to say that starting with 100% coverage should be the default, and then to discuss scenarios where this is not reasonable.

Continuous Integration

The second item that jumped out at me was the overall repetition provided in Practice 2: Continuous Integration. While the chapter itself is very valuable and provides a great amount of information, it is very repetitious. They seem to take the following approach throughout the chapter:

1. List the benefits
2. Describe the benefits in a little detail
3. Re-list the benefits and how they support Lean
4. Re-describe the benefits in a little more detail and repeat from #3

It got a little old hearing the same thing over and over and over with just a little more detail each time. That being said, the chapter is still worth reading and properly understanding. They do a great job of describing why CI is important – but do it very repetitively (much like my description, here. )

Summary and Recommendation

From the preface, page ix:

Who Should Read This Book?

This book is for software developers and managers of software developers who are new to Lean software development and, possibly, new to Agile software development. It is for those who want to quickly understand why Lean software development is important and what it can do for you.

This is purposefully a short book, with short chapters. We know that you are just as busy as we are, and we don't believe in padding our chapters with useless fluff. In every chapter we try to be as succinct and to-the-point as possible. Our goal is to introduce you to the important topics and resources so that you know where to go when you need more details.

The overall focus on the business value that Lean provides, while introducing the engineering practices makes for a great project management or senior software engineer / team lead level read. There is enough information in this book to hopefully garner the additional research of the readers. I believe the authors have appropriately stated their audience in the preface and have done a phenomenal job of keeping the book short, easy to read, and very informative.

Final Score

In light of the two issues that I listed above, I would give this book a 4 out of 5 and recommend it to the same audience that the authors recommend.

I heard the phrase “one team, one aim” while listening to the audio book version of “Extreme Toyota”. This is a phrase that marks part of the core philosophies of Toyota, according to the book. There are many different philosophies that underlie this specific phrase and I’m sure there is a very large amount of text that can be written about it. Rather than jumping off of that deep end without a greater understanding than I currently have, though, I wanted to point out one very specific word in this phrase: “aim”.

Wiktionary has a definition of aim that appears to fit very well in this phrase: “Intention; purpose; design; scheme.”

So, why use the word “aim”, here? Why not talk about goals? Why not deal with something more concrete? I think the answer to this is found in one of the core principles of Toyota: Continuous Improvement. A goal is a concrete, fixed point to be reached. Although a goal can change, once it does change, it once again becomes a fixed point to be reached. If Toyota talked about goals then they would inherently be limiting themselves to such fixed points of achievement. By using the word “aim” instead of “goal”, they are illustrating the true intentions of Continuous Improvement. That is, they are not setting their sights on attainable goals. Rather, they are taking a philosophical, yet very practical and daily routine of continuously improving in all aspects.

This does not mean Toyota works without goals. Quite the opposite, really. They create some of the most ambitious goals that anyone can imagine - “impossible goals” as talked about in the Extreme Toyota book. However, these goals are always aligned with the company’s vision, their aim. Toyota recognizes that a single goal is never a place to stop, having accomplished something, but is only a starting point allowing them to define the next goal and begin that leg of the journey.

As I’ve talked about before, we should be focused on the journey – the aim of where we are heading - constantly adjusting that aim as we obtain new information. We then set goals to help us move in that direction, knowing that we are truly on a journey, not looking for a place to stop after a single, fixed point of achievement.

In the manufacturing world, you would never find a company that assembles a bunch of parts into a final product before inspecting any of the individual parts, and they would not wait until the end of the assembly line to test for the quality of the product. The very notion of waiting until the product is “done”, to test it, would be appalling. How much time and money would be lost trying to figure out why something didn’t fit together properly, why it didn’t work, and why there was a quality issue with the final product? Rather, we see the manufacturing world taking an active role in preventing defects. Yes, they still do a final quality inspection, but the primary means of ensuring a quality product is delivered is not by waiting until the product is assembled to test it. They build quality in from the start and maintain that quality throughout the manufacturing process.

Prying Open The Case

Imagine the inner workings of the phone to the left. This is a very complex piece of technology. Do you think Cisco would wait until they have assembled this phone and then try to pry open the case so that they can insert a set of electrodes to test and see that the circuit board is connected correctly? I certainly hope they don’t. Instead, when a manufacturing company is building something – anything – they start with the idea of preventing defects, not waiting until they are identified and correcting them. Many companies have an active Zero Defects policy where defect prevention is paramount and quality inspection is almost just a verification of what they already know – that the product is defect free.

When a part is stamped out, formed, molded, or otherwise created, it is done so to an exacting specification. After the part has been created, the part is then tested against the same specification to which it was originally built. If the part does not fall within the tolerance and guidance of the specification, it is scrapped and a new one is made. If a series of parts are found to be out of specification, it’s usually a sign that something in the process, tooling, or other portion of the manufacturing process is not right. When this happens, they fix the cause of the problem – whether the machines need to be calibrated, the people running the machines need better instructions or whatever the cause is. In the end, the specifications for the parts were used to create the part, identify whether or not the part was up to standards, and decide whether or not to keep that part.

What’s more, the manufacturing company doesn’t wait until after they start creating parts to create the specification. Rather, they take the time to properly engineer the specifications up front. They take measurements, create prototypes with varying specifications to see what works best, record the success and failure rates of the various specifications that are tried, and use other design and engineering principles to scientifically calculate the exacting specifications that will be used to produce the parts. This occurs at all levels of the product’s design and creation. For every resistor, capacitor and microchip that builds a circuit board, each one of them has their own specifications that have been carefully engineered. If any single capacitor does not meet the specifications, it is not sent to Cisco with the hopes that it works anyways. Only when all of the specifications of each part are met will they solder the parts to the circuit board, creating a subassembly.

When a subassembly is created, it also has a specification to which it was built. The subassembly then undergoes the same quality assurance process – verification that it meets the specifications and operational requirements. The process continues from here – each subassembly gets connected to a larger system which is built to a set of specifications, with rigorous testing of the larger system as it is built, ensuring it meets the specifications. When the final phone is assembled, we don’t have to worry about whether or not a specific capacitor is soldered to the correct location – we don’t have pry open the case on this phone and insert a set of electrodes to see if the electrical current is flowing correctly. Instead, we only need to plug this phone into the correct connections (an Cisco IP phone system in this case) and verify that the phone actually performs all of it’s functions, according the functional specifications of the phone. There simply is no need to verify the capacitor that was used in the very first step. We know it works because it was built in a system that actively prevents defects.

The manufacturing world is obsessed with testing. They are willing to test from the lowest possible levels of the system, out to the end-product and the behavior that is expected. They do this because the consumers of manufactured products demand perfect. Why, then, are so many software development companies so willing to only test from one end of the process? To only test once, and only from the user interface, just before the product is shipped?

A Specification By Any Other Name

Unfortunately, the software development industry as a whole, is years behind the manufacturing industry. Our definition of quality and success are often skewed and we may consider fifty or more known bugs in a system of moderate to large size to be acceptable. It doesn’t have to be this way, though. We have the technical capabilities of following in the footsteps of the manufacturing industry, and we should.

I’m sure there would be no small number of people that would say we already employ the use of specifications in software development. After all, that’s what the requirements gathering phase is for, right? So many software development companies have put so much effort, time and money into the process of producing a piece of paper that can be understood by humans, and labeled this a specification. The problem we face with paper, though, is how to effectively verify the software against what the paper says. How do we verify that series of software lines and I/O statements that are understood by a computer have actually implemented the human readable text on the paper?

We are fortunate, actually. We don’t have to accept a Word document or a piece of paper as the specification to build to. We have the ability to create executable specifications! We can, and should, be creating specifications that can measure and verify our code. Most people call it test driven development (TDD). Some call it Behavior Driven Development (BDD). I like to think of it as Specification Driven Development (SDD? Not sure if that really exists. And really, do we need another xDD acronym?). We write code that exercises our code in the form of unit tests, integration tests, functional tests, acceptance tests, or whatever you want to call them.

We’re Not Just Stamping Out Parts

One of the major problems that I have with the manufacturing/software development analogy is the obvious statement that we don’t stamp out the same parts over and over again. In spite of my previous comments on this analogy, I now think that we are more analogous to a specific part of manufacturing than I had previously understood. A more accurate representation of software development in the manufacturing world is new product design and development. The parallels work quite well from this perspective. I am not going to expound on this in great detail at this point. It should suffice to say, for the moment, that the process of product development described in Wikipedia is a fairly accurate representation of what we go through for the average software development project.

When a manufacturing company is working on a new product, they once again don’t stamp out parts without knowing what they are doing. Many different parts may need to be tested, many different designs may need to be tried, but every one of these is still built to a specification. The major difference is that the specifications used are expected to change over time, until the final specification for the final pieces are accurate enough to produce a production-ready prototype.  The same notions can be applied to the software development processes in TDD, with some additional benefits.

Built To Specs, Regression Tests And Change

Change happens. It’s a simple fact of software development. A customer thought they wanted X, but in reality they needed X-1/B – not quite what we originally thought. When this happens, we once again have a significant benefit created by our executable specifications. We only need to identify those specifications that are now wrong, correct them, and change the affected portions of the system to match the new specifications.

Our ability to change is direct evidence to one of the many benefits of TDD: regression tests. Every specification that we write becomes a regression test the moment we fulfill that specification’s requirements. With this in mind, we can work with an almost reckless abandon, free to add features, remove features, fix bugs (because let’s face it – we’re still going to find some issues somewhere in the system) and refactor the system to a higher standard, all without worry of breaking the system. We can act with this level of confidence because we have a safety net in our regression tests. If (when) we do break something in our new efforts, we will be notified the moment we re-execute our specifications – that is, run our regression tests. A specification test will fail and we will have a clear indication of what failed and why. This deep insight into the system gives us even further confidence in correcting any issues that we introduce. When a failed specification test tells you exactly which value from which class is wrong, and the context in which that class was executed is known (the exact input and expected output), pinpointing the problem becomes a rote process. Fixing the issue becomes relatively simple, and we begin to see true productivity improvements in our processes.

Start With Quality, End With Quality

Our industry is currently suffering from a lack of quality. We ship horrendously bad user experiences in products that are late and well over budget, yet we call this a ‘success’. It doesn’t have to be this way. If we change our perspective and start to take some cues from the manufacturing and product design and development world, we can dramatically increase our effectiveness as software developers. We can create high quality, low cost solutions like the world expects from manufacturers. Built to specification is certainly not a silver bullet. It is, however, the definition of quality in a Zero Defect environment.

By employing a built to specification mindset in our software development efforts, we can start with quality and maintain that quality throughout the life of our projects. This is the same process that is undertaken when a manufacturing company is working on new product design and development. It works well, it’s a proven process, and most of all – it makes sense. Build your software to specifications. Just make sure they are executable specifications.

For so long there have been so many advocating the benefits of the various Agile, Lean, Iterative, or whatever-you-want-to-call-it-these-days methodologies. We, as software developers, seem to understand the benefits of these methods. So why, then, do we still have customers that insist on big design up front, waterfall based methodologies, and other practices that we know are wrong?

Some Ad-hoc Consulting

During my Christmas vacation, I had the opportunity to do some consulting / coaching for my dad’s current company. The employee list that I spoke to consisted of my dad, my aunt, and my cousin. It was a nice little family gathering, I got paid with some amazingly good barbeque, and it was a very enjoyable experience.

I’m a software developer working for a company that provides software development and related services. My dad’s company is a customer of a software development organization (not the one I work for), receiving some custom built software for a project. They have been running into some issues with quality and timeliness of delivery for the last year or so and were wanting to ask some very pointed questions about the state of the software development industry. Essentially, they wanted me to tell them whether or not the issues they were having are “normal” and / or “acceptable” when receiving custom software development services. My answer was an unfortunate “yes, this is normal”, but “no, this is not acceptable”.

Over the next few hours of our discussion, we touched on many different issues that they are having and how some of those issues can be addressed from their perspective and from the perspective of the software development supplier. In the end, I had successfully educated my dad’s company on a few key points, confirming their suspicions and observations, and giving some new directions to start heading with the company that is providing the development services. I can’t go into all of the detail, but the summary bullet points included the following:

• The customer won’t know what they want until they see something they don’t want
• The software provider should deliver a working, stable version of the system on a regular basis, allowing the customer to use it in it’s current form
• The customer sets the priority of features and functionality to be implementing, in coordination with the technical needs and pre-requisites of the software development organization
• The customer is allowed to say something is wrong, even when they receive what they asked for
• The relationship between the customer and service provider must be a whole-team view. That is, “we are one team. we succeed together and we fail together”

On top of these bullet points, I laid out a simplified Scrum process for them, based on the already stated 3 week development cycle that the provider prefers.

In the end, my dad’s company was shocked, astounded, amazed, and energized by everything they heard. The light bulbs were clicking in their eyes, and I could see the value that they received form the conversations. They have dramatically changed their understanding of what it means to be a good customer and what it means to be a good supplier.

A New Perspective

As much as I believe I helped my dad’s company in their understanding of what they should expect and how they should approach their issues and needs, I also learned a great deal from this experience. It was very enlightening for me to hear the issues that we have known for so long as software providers, but from the customer’s perspective. Nothing that they spoke of was unknown or unfamiliar to me. I have lived through all of the same pains and issues that they are going through – from both a software provider and a customer of software services.

What really stuck with me from this experience, though, is the idea that our perspective on how to improve our industry is likely wrong. Yes, we need to continue educating our own ranks on the benefits of better software development techniques and processes. However, it does no good for us to have the perfect pneumatic toolset when the customer is asking us to use a rusted hammer and a broken hacksaw.

What I’m getting at is that its not us, the software providers, that need to change so greatly. Rather, our customer base needs to be educated and change. We need to stop being so self-centered and introspective for a while so that we can educate our customer base on what it means to be a good customer; how they can accomplish their goal of receiving quality software that provides true business value for them, at an affordable price. Until we have a customer base that expects software to be high quality, the first time, every time; expects to be given working software on a regular basis; and expects to be able to recognize that what they thought they wanted is not what they really needed; we will have no chance of our inward looking self-improvements taking hold.

I have to say, the Agile Manifesto got it right from a values perspective. The values espoused there are what we need to be looking toward, but I think our perspective needs to be changed on how we enable those values to be reached.

Of course, if we do succeed in educating our customers, we need to be able to respond appropriately. So please, continue the education of our own industry. Create better development standards that increase our quality and productivity. If our customer expects a quality delivery every 2 weeks, we better be able to deliver a quality product every 2 weeks. Just don’t stop here. Educate your customers. Help them understand the financial impact of poor development standards and processes. Change the norms and improve our industry by creating a customer base that requires perfect. The manufacturing industry did it 20 to 30 years ago. It’s time for us to step up to that level.

Where Do We Go From Here?

So where do we go from here? How do we start educating our customers? How do we change the social norms of software development so that our customers expect us to behave more like professional craftsmen and engineers, delivering quality faster?

I don’t claim to have the answers and I don’t claim to be the first to encourage this change. I only hope that we as an industry will begin to understand that the customer is usually the one that sets the expectation for how often software is delivered, and that we need the customer to expect something better.

Imagine that there are three group of people going out for a hike. Within each group of people, we have a goal of everyone reaching a picnic table at the end of the hike (and no one can eat until everyone has arrived). One of the people in each group has a map and a compass, knows how to use them, and will be directing the group on where they need to be going (we’ll call that person “the Pathfinder”). Everyone else is just along for the hike and the picnic.

The Pathfinder in each group is given a specific set of instructions on how they are going to direct everyone else.

Group 1: Observe, Plan, Walk

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course in one hundred foot increments, with your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking. Give an updated direction to walk every one hundred feet, enduring the group can go around any obstacles such as large rocks, impassable tree groupings, and small creeks.
4. Once you have plotted the correct course, store the compass and map in your backpack with the understanding that you are not allowed to use them again, until you have reached the picnic area.
5. Have the entire group read the instructions, then set out on your hike.
6. You are only allowed to check that everyone is on course by re-reading the instructions that you wrote down, every 100 feet – when the team needs to change direction to avoid an obstacle.
7. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 2: Observe, Plan, Walk, Start Over, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet, ensuring everyone in the group understands the directions.
4. Place your compass and map on the rock, then hop down from the rock. Re-join your group and lead them in the right direction for one hundred feet
5. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
6. Travel the one hundred foot distance with your team, ensuring that they end up in the correct location.
7. At the end of the one hundred foot walk, have the group stop where they are.
8. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
9. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
10. With the help of the ropes to guide you, walk back to the rock at the beginning of the trails, by yourself, and climb back on to it.
11. Plot the next one hundred foot leg of the journey from where the group currently is, using your map and compass.
12. Repeat this process from step 3 through 11, until your entire group has reached the picnic area.
13. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 3: Observe, Plan, Walk, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area and general understanding of how to get where you are going, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet.
4. Place your compass and map in your backpack, then join your group.
5. Let the group know where they are heading (the end goal) and where they need to be in the next one hundred feet, ensuring everyone in the group understands the directions that you outlined.
6. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
7. Lead the group in the right direction for one hundred feet, ensuring that everyone is able to safely navigate any obstacles.
8. Verify that that your group has stopped in the correct location, by using your map and compass and any other landmarks or means needed.
9. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
10. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
11. Repeat this process from step 3 through 10, until your entire group has reached the picnic area.
12. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Predictions on Outcome?

• Which group do you think will make it to the picnic site first?
• Which group do you think will be the most happy with the way the group was organized and run?
• Which group of people do you think would be most likely to do the same process on another hike?

Observations of the Groups

Group 1 didn’t get to the picnic site very fast and wasn’t very happy about the experience. While everyone in the group was standing around waiting, the Pathfinder was sitting on top of a rock, writing things down in a notebook, trying to describe – from a very high level, where people were supposed to be walking. When the Pathfinder finally came down from the rock – possibly an hour or more after he climbed up there – the initial directions were good and people could start finding their way. As time went on, though, the detail of where they should be going became more and more unclear, because the Pathfinder could not nee the actual pathways below the tree tops and behind the large rocks. This caused Group 1 to constantly stumble and struggle, creating a situation where the team had to find paths around obstacles that they couldn’t previously see. Every now and then, they got lost because the description of the landmark was insufficient from the distance that it was recorded. They eventually got to the picnic site – well after everyone was hungry and tired. In the end, no one was happy with the experience and just wanted to go home.

Group 2 had an easier time finding their way. Since the Pathfinder and the group were able to see all of the obstacles within one hundred feet, the directions that the Pathfinder described was much more enjoyable. No one stumbled over anything or got lost. But it was a difficult journey, at times. The group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. The problem, though, is that the group took forever to get to the picnic area. At first, having the Pathfinder re-check the direction they were heading was great. Everyone knew that he was keeping the group on target. However, every time the group reached their next stopping point, the Pathfinder would gather information about the next hundred feet, across a wide arc since he didn’t know which direction to go next, with great certainty. Only then could he travel all the way back to the original rock to verify that everyone was on the right path, heading toward the picnic table. After a few iterations of hiking, the process of verifying the course and location took more time than the process of hiking. The Pathfinder was simply repeating the same hike over and over again, to verify that they were on the original path laid out. No one in this group was disappointed in the path that they took to get to the picnic table, but no one appreciated the standing around and waiting for hours on end – waiting for the Pathfinder to get the next one hundred feet plotted. When they finally reached the picnic table, group 2 was full of hungry, grumpy people who just wanted to go home.

Group 3 got to the picnic site in record time. It was a difficult journey, at times – the group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. With the ability to see all of the obstacles in the surrounding area, and the ability for the group to make decisions that would get them around obstacles while still making progress toward the picnic table, the group was successful in getting to the picnic table before lunchtime. Everyone on the group loved the trip and appreciated the ability have input in how they got to the picnic table. As long as the direction that they headed for the next one hundred feet did not cause a serious delay in reaching the picnic table, or was justifiable by avoid an obstacle, all input was considered before setting the new direction. Several of the people in group 3 decided that the would take a few more hours and head off into the next set of hiking trails, trying to reach another picnic table before dinner time. Other decided to bring their family out to the trail the next day, and re-walk the well defined path (noted by the ropes tied to the starting and ending points of each leg of the journey). Overall, everyone in group 3 enjoyed the experience.

The Real World

Now imagine that the Pathfinder is represented by some aspect of your organization. It may be the company or project leadership; it may be the customer; it may be an individual person or group of people within a project team. Your Pathfinder may not be a person or other sentient being, though. Perhaps its the Methodology (capital M, as defined in Peopleware) or political environment in your company.

Which Pathfinder(s) do you have in your company? Which Pathfinder would you rather work with? More importantly – if you are not working for / with the Pathfinder that you want, what steps are you taking to correct that? Be a catalyst for change – appropriately and professionally.

Continuously improve – always within the context of the goal.

Let's assume that we are doing the appropriate amount of testing during our development process. If we include TDD, test automation, test engineers and customer acceptance testing, we should find the majority of the bugs in our system before they are released. However, not every bug will be found. There will be some situation that no one thought about before. There will be some special circumstance on someone's computer that hasn't been accounted for. There will be some client data that does fit the expected variance, despite the data being valid. The point is, there will be something that breaks after we deliver the software. Worse yet - it doesn't even take delivery to find bugs. What happens when the software gets to customer acceptance and the customer says that something is wrong, broken or whatever? We simply must account for the inevitable bug fixes and emergency patches in our system.

Our current kanban board, with all it's pipelines and queues, hasn't addressed the need to address problems. But before we get into any changes to the board, we have to define and create the culture of quality that we need in our team.

A Culture Of Quality

There are many different aspects of quality and many different ways to view and interpret quality. The individual specifics, as always, come down to your team, the project and the needs of both. Even with these variances, though, there are a few key concepts that should be found in any culture of quality, including Whole Team / Collective Ownership and Zero Defects.

Collective Ownership

I've talked about this concept before (see above link) and would recommend reading those prior posts. One thing I would like to add, though is that collective ownership can only success when the team takes their individual egos out of the equation. We must be able to accept criticism as a way to improve ourselves and our team. When we let go of our ego, we can be honest with ourselves and others, enabling everyone to own every part of the system.

Zero Defects

This is a subject that I have not talked about before, but is prevalent throughout the continuous improvement philosophies that I subscribe to. Wikipedia lists four principles of Zero Defect methodologies, all of which are paramount to our culture of quality.

1. Quality is conformance to requirements
2. Defect prevention is preferable to quality inspection and correction
3. Zero Defects is the quality standard
4. Quality is measured in monetary terms – the Price of Nonconformance

Don't be confused by "requirements" in item number one, though. In the world of software development, the word "requirements" has many meanings - not simply the feature list or functional descriptions set forth by our customers. Our teams and processes have certain requirements that are imposed on top of, and into each of the business requirements for the software. These often include the use of source control systems, test driven development, principles like S.O.L.I.D., etc.

Quality Checks in Kanban

Collective ownership and zero defects are only the tip of the iceberg. There is so much more to a culture of quality and so many different aspects of quality to account for. However we define quality, though, our ability to create a culture of quality is necessary for our kanban system to work. Without it, we lose the continuous improvement and elimination of waste that defines Lean. With it, though, we can introduce some specific tools to our process and improve our kanban system by having it handle the errors that are found in our systems.

Andon

From Wikipedia:

"a system to notify management, maintenance, and other workers of a quality or process problem"

When an issue is found in our system, we need to notify the team immediately so that the problem can be taken care of. The idea in a manufacturing line is to allow a worker to stop the line when a problem is found. In our grocery store example, andon could be invoked by a shelf stocker noticing a loose nut or bolt on the shelf and then notifying a maintenance person. In software development, andon can take many different forms. We report our issues list during our daily standup. We create issue tickets in our issue management system. If we're lucky and work in a company that supports the idea of a team room, we may just need to look up or turn around and let the team know that we have problems. We could even place an actual red flag on the desk of our developers, testers, customer representatives, etc., and have them raise that flag when they see a problem. Any of these ideas, plus many many more, can all be our andon system.

Andon itself is not intended to be an all encompassing electronic system with metrics and reports and yadda yadda yadda. It needs to be simple. It needs to be easily employed by anyone on the team. And it needs to mean something to the team. If a team member throws their andon card out on the table, the culture of that team needs to be ingrained with the knowledge that work may stop until the problem is addressed.

There's so little to andon, yet so much more than what I've described. However andon is enabled, it is critical to our zero defect policy in software development.

Jidoka

AKA "Autonomation", AKA "Automation with a human touch", AKA "intelligent automation", AKA ...

From Wikepedia:

"Autonomation prevents the production of defective products, eliminates overproduction and focuses attention on understanding the problem and ensuring that it never recurs."

I've talked about Jidoka in previous posts and would recommend reading them, at this point.

• Jidoka - If it's broke, fix it now
• Jidoka - More than just 'if it's broke, fix it now" - fix it and inform the team

I don't have much else to add, other than to say that Jidoka and Andon go hand in hand. Automated build servers (such as CCNet, TeamCity and many others) can often combine Andon and Jidoka for us by giving us instant visual feedback when something is broken. I've also recently set up BigVisibleCruise in my team area, giving even the casual observer the knowledge of whether our builds are broken or not.

Applying Andon and Jidoka to Our Kanban Board

Once we have the concepts of Andon and Jidoka in our team and culture, we can use these tools to generate issue cards and then look at a few possible changes to our kanban board to account for them. The three basic methods that I have seen used include:

1. Creating an Emergency Fixes pipeline
2. Tacking a smaller bug notice onto an existing card
3. Putting a Bug card in the backlog

I'm sure there are other alternatives, too. In my current team, we use the first and third method and are considering the second one as well. 

Options two and three essentially require no change to our kanban board. Implementing option two is a distinctive way of visually attaching a bug notice to one of the cards in our system. This could be done with little bug stickers, little red cards, or any other visual indicator that the team agrees on. Option three also needs something distinctive. Since we are creating an entire card for the bug, though, the entire card should be distinctive. I would recommend using a card that is colored red to signify issues. I would also recommend prioritizing the bug to the top of the backlog queue, when using option three. This will ensure that the bug gets worked as quickly as possible.

Option number one can also make use of number two and/or three. When we move a card into the Emergency Fixes pipeline, we may want it to be distinguishable as an issue by tacking on our bug symbol or by using a colored card.

Emergency Fixes Pipeline

Depending on the complexity or severity of the bug, we may want to include Analysis and Customer Acceptance in our Emergency Fixes. With andon and jidoka in mind, we will want to ensure that we fix any emergency issue immediately. This is not always possible, however. The customer may decide to delay the fixing of an issue for whatever reason. This leads us to only needing a single pipeline for Emergency Fixes, letting us set our limit to one.

We can easily add an Emergency Fixes pipeline to our kanban system, placing it directly underneath our existing WIP pipeline. This special pipeline can be designated with a name, color code, or other marks as needed.

If you are supporting production releases, you may also need to include a Delivery queue specifically for emergency fixes. This would allow multiple fixes to be compiled into a single patch release. There are other configurations to this, of course. As always, you will need to find what your specific team needs and create your process to suit.

Where Do We Go From Here?

With an Emergency Fixes pipeline in place, our kanban system is now set up to handle just about every situation that we will encounter. However, this does not mean that our system is perfect or truly complete. No process, no matter how well defined it is, is worth anything if the people running the process do not believe in it. I've said it before and I'll continue to say it - never stop improving your process. Always be mindful of waste, friction, smells, problems or whatever you want to call it. Inspect, adapt and continuously improve your process. Perfection is a journey, not an end-goal.

Stay tuned to my Adventures in Lean series, as I continue to explore the various aspects of lean software development in my own team and company culture.

In part 2 of my Kanban in Software Development series, I talked about completing a kanban board with queues, order points and limits. We saw how to take a complete development pipeline and work with a team, its processes and its bottlenecks. In the end, we had a kanban board that could easily represent the processes of the fictional team that we outlined.

One of the questions that I've often asked about a kanban board is how anyone would know when work in one column is done and ready to be pulled into the next column. For example - if a kanban card is sitting in the Analysis column, how does a developer know when that card is done so that they can pull it into Development and start coding it? I found the answer to this question when I was at the Kaizen Conference in October. Jef Newsom did a workshop on kanban and we ended up with this same question, and a solution.

A Pipeline for Analysis - WIP and Done

To facilitate the visualization of the difference between work in progress and work that is ready to be pulled to the next column, we can use the concept of a pipeline and split our existing queues into a WIP and Done step. For example, we want developers to pull work from the Analysis queue into the Development pipeline. We can show which cards are ready to move by splitting Analysis into sub-columns of WIP and Done.

When an Analyst is ready for work, they would pull from the Backlog into the Analysis / WIP column. When the analysis work is done and the card is ready to go into development, the card would be moved into the Analysis / Done column. Since we are wanting to maintain the concept of a queue for the overall Analysis column, we have create the WIP and Done subdivisions as a pipeline (noted by the dashed line). This allows us to keep our order point (3) and limit (5) in place for Analysis, and know what work is ready to be pulled into Development.

Applying Pipeline per Queue Across The Board

Not every queue needs to be a pipeline.

Consider the Backlog - the customer is placing the prioritized list of features in this column. The cards in this column exist so that the analysts will know what work needs to be done - not because any work needs to be done in this column, explicitly.

The Delivery column may not need a pipeline, either. If the delivery process is composed of sending an installer package to the customer, then there is no real work to be done in this column aside from sending that package. However, if there is some specific integration work (say, changing a web.config file) that needs to be done, we could include a WIP and Done pipeline for delivery. For this simple example, we'll say that there is no configuration change needed. Let's assume that the installation package changes all the needed configuration files based on user input.

With all of this in mind, we can apply our WIP and Done pipeline to the Analysis and Customer Acceptance columns.

Alternative Visualizations

There are other ways of visualizing WIP vs Done cards for a given column, of course. For example, you could forgo the pipeline idea and simply add a queue between Analysis and Development. This may require you to adjust your order points and limits between analysis and development - at least, where those order points and limits apply (Analysis vs. the queue between Analysis and Development). You could also split the column horizontally instead of vertically, creating either the top or bottom half as the WIP with the other being Done. Or you could just note the status on the kanban card itself. I'm sure there are a number of other ways to show this information, as well. The point is that the pipeline visualization that I'm showing here is not the only way. Find what works for you.

Where Do We Go From Here?

We have another tool in our belt, now. If need be, we can visualize the WIP vs work Done for a given queue, allowing us to see when we can pull work forward. But, like all tools in our belt, it is not *the* answer and this additional visualization may not be necessary for your team.

If your team is small or has great communication, you may not need any special distinctions to show work in progress vs work ready to be pulled. You could rely on the daily standup for people to report when they are done, or do it ad-hoc - when someone finished a card, they announce it to the team immediately. In a larger team, though, it may become necessary to have some visual distinction between work in progress and work completed for a given column.

Small teams might get away without it; larger teams might need it; its up to your team to decide. The key is not to apply these ideas as blanket rules that must be followed, but to allow the individual project and team to decide what is right for them. And as always, allow the process to change when it needs to. Always inspect, adapt and continuously improve.

In Part 1 of Kanban in Software Development, I introduced the concepts of kanban boards and pipelines. I also showed a very simple example of creating a pipeline for our development process. However, there were some obvious limitations in what I showed. The reality of software development is much more complicated than three simple steps (Analysis, Development and Testing). A software development team or company is going to have more to do than just these things, and there is usually a need for some team members to be cross-functional. Some team members will have to do development and testing, or analysis and development, or documentation and delivery, or any other combination of steps involved in creating software. In this post, I'll address these issues, creating a more complete pipeline and finishing our kanban board.

Working In A More Complete Pipeline

To create a more complete kanban board, we need more than just a three step pipeline. We need to allow for a fully functional team - developers, analysts, testers, technical writers, and others. We also need to allow the different team members to work on different parts of the system, as work is needed. The end goal is to enable the system to flow through the process and to ensure the work is "done done" before it goes to the customers.

For a more complete software development pipeline, let's use the following columns: Backlog, Analysis, Development, Documentation, Testing, Customer Acceptance, and Delivery. We can put together a pipeline diagram that follows these steps.

Now, let's assume that we have the following team (ignoring project managers, customer representatives, etc, for now): 6 developers, 2 business analysts, 3 test automation engineers, and 3 technical writers. Given this team, we would not be able to make a feature flow through this pipeline. We would have developers sitting around, waiting for work from the analysts. And, our documentation writers and test lab people would probably be pulling his hair out from boredom then pulling their hair out from too much work, in an unbreakable cycle.

Fortunately, we can account for the team makeup and the potential bottlenecks by introducing the concept of order points and limits that we saw in the grocery store, and by creating queues and multiple pipelines to be worked.

The Real World - Multiple Pipelines and Limits

In most software development efforts, it's unreasonable to expect an entire team to work on only one feature at a time. Most development managers want to maximize the throughput of development by working on tasks in parallel. Given this desire, we'll divide up our developers into three teams of two developers each. This development team division should allow three features to be developed at the same time. Since we want to work on three features at a time, we will need three pipelines for work.

These three pipelines constitute our first limit - we can have a maximum of three features in development at any given time. This limit is noted by putting a "3" in the upper right hand corner of the development column header, as shown above.

Having three development pipelines puts us in a tricky situation, though. We have very few business analysts compared to developers - even with our developer teams, we have less than one analyst per developer team. Fortunately, development work often takes longer than analysis. We can use this knowledge to our advantage and let the analysts work slightly ahead of the developers, creating a queue of work to be done.

Handling Bottlenecks - Queues, Limits and Order Points

If all three development teams happen to finish at the same time and need more work, we would need a minimum of three features that are ready to be worked. However, since we know that requirements change regularly, we don't want to get too far ahead of the developers. With this in mind, we can introduce an Analysis order point of three (the number of development teams) and limit of five. This gives the analysts the ability to work ahead of the developers and also makes them responsible for keeping work available for developers.

For an order point we'll introduce the number into the upper left hand corner of a column header, and we'll continue using the upper right hand corner to specify our limits.

Since we are now dealing with potentially more work in analysis than development, it seems we would have to increase our pipelines to five. This is not desirable, though, since we only have three development teams. What we will do, instead, is restructure the pipeline and turn the analysis column into a queue.

Since Analysis is now a queue, there does not seems to be a need for Backlog to be a step in a pipeline, either. The backlog is simply the list of features that the customer is expecting to be done next. Therefore, we will also turn the backlog into a queue - this time, with a priority (top of the board is highest priority) allowing the customer to tell us which specific feature should be worked on next. We will also want to keep the backlog column limited, to prevent the team from having too much information to think about. Since we only have two analysts on our team, it seems appropriate to keep at least two features in the backlog column at all times. With this in mind, we can safely set a backlog order point of two and limit of five (enough to keep the analysis column full).

The changing of Backlog into a queue has no made our Development column the first step in our actual pipeline.

Completing The Pipeline - Aggregate Limits and Work Cells

The next few column - Documentation and Testing - both have an easy amount of team members to deal with. There are three technical writers on our team, and three testing personnel. This distribution lets us keep the pipeline in tact between Development, Documentation and Testing, allowing us to set the same limit as Development (three).

With all three of these columns having the same limits, and with each column being properly staffed so that three pipelines could run at the same time, it makes sense to combine the team members from these three columns into a single workcell. If we consider all four team members (2 developers, 1 tech writer, 1 tester) as a team and allow them to work together as a team, we can aggregate the limits of the columns in the pipeline. We can also create a consolidated name for this pipeline - work in progress.

This consolidation can be shown by creating a parent / child header with the limit shown in the parent. We will also use a dashed lines between the columns of the pipelines. The combination of these two details will show us that we are dealing with a pipeline, and how many pipelines are allowed to flow, simultaneously.

By aggregating the pipeline limits, we allow the workcell for a given feature to focus exclusively on that feature. The development staff, technical writers, and test lab personnel will all be working on the same feature, allowing them to more easily share information between themselves. This will prevent the team members from having to switch back and forth between subjects, reducing cognitive load and allowing for greater quality to be attained in the individual feature.

Completing The Kanban Board - Customer Acceptance and Delivery

If the customer is on-site (co-located with the team or close enough to meet every day), then it may be possible to aggregate the Customer Acceptance process into the Work In Progress pipeline. If you can do this, you should. Getting immediate feedback from the customer, while work is still fresh in you mind, is critical to the quality and success of that work. What's more - you may not need this explicit column if you have the customer working with you every day.

Unfortunately, we don't always have the luxury of on-site customers. We may only have the customer around at specific timer intervals or only on request. In either of these cases, we have to account for the bottleneck of Customer Acceptance.  Additionally, the customer may not want to get delivery of individual features. They may want to wait for a quarterly release, or some other regularly scheduled release. Once again, we have to account for this bottleneck. We can apply the same principles that we used for Backlog and Analysis here, and create some appropriate order points limits.

Let's say that our customer wants to do Customer Acceptance testing once a week and wants a Delivery once every calendar quarter. To accommodate this, we can set the Customer Acceptance column as a queue with a 1 week limit. We can also set the Delivery column as a queue with a 1 quarter limit. Setting these limits puts a direct responsibility on the customer. If they cannot pull the features through these columns on or before the specified limits, then the entire system could grind to a halt. It's important to educate the customer in what they are committing to, and ensure that they can fulfill their obligations. This fulfillment may mean that they assign more than one person or group of people to the task of Customer Acceptance and accepting Delivery. Whatever the solution is, the customer has responsibilities to meet.

With Customer Acceptance and Delivery specified, our completed kanban board would look like this:

Put It In Action - A Kanban Process Is Never Truly "Complete"

Now that we have clearly defined our Kanban process and visualized it through our Kanban board, the next steps is to start using it! Start working through this process, pull value from the end of the process and allow work to flow through the pipeline. Respect the order points and limits, and above all, look for problems in the process and fix the process when you need to.

Perhaps you have too few or too many items in one of your queues - adjust the limits and order points. Perhaps the test lab is now a bottleneck - take them out of the pipeline and change them to a queue. There are many possible problems and many possible solutions. The key is to be aware - inspect and adapt and hold regular retrospectives - kaizen your process and continuously improve.

Where Do We Go From Here?

No process is perfect. Anyone that tells you otherwise is selling something. A customer may find a bug during Customer Acceptance; an issue makes it past all of our QA processes and is found out in production; or there's a feature that has a sudden priority over any other feature currently being worked. But our current kanban process doesn't address the natural problems that occur during a software package's lifetime. To address this properly, we'll need to employ the lean tools of Andon and Jidoka, and we'll also need to decide how to visualize this on our kanban board. These issues, and possibly more, will be addressed in another installment of Kanban in Software Development.

In the world of Scrum, XP and other forms of Agile software development, many teams use visual control systems to outline the various steps that software goes through during development. These boards are known by various names -  Scrum boards, card-boards, task-boards, swim lanes, and probably a few other names that I'm not aware of at the moment. Whatever you call it and whatever agile creed you live by, the primary use of these boards is the same all around - information radiation. Anyone can look at the board at any given moment in time, and know where in the development process a given card is. If you're doing iterations or sprints, the board also tells you where you are in that time box - if you're ahead, behind, or on time. The far left column - the iteration backlog - will be filled with cards at the beginning of the iteration. Over the coarse of the iteration, cards will be pushed through each of the columns until they are in the done column. The goal is for every card to be "done done" by the end of the iteration, with a set of features that is now deliverable to the customer.

Kanban Boards

Visual management tools are very common in agile shops, and for good reason - they work very well. It should come as no surprise, then, that kanban software development also employs various visual management tools, including a kanban board.

On the surface, there isn't much difference between an average task board and a kanban board. Each of these boards has various columns that represent the stages that a card needs to go through before it is considered done. The real difference in a kanban board, is not the board itself. The board is just a visual indicator, the same as any task board, and the intention is still to get the cards to the "done done" state - that is, delivered to the customer so that they can use the features from that card. The real difference is how the process is approached - by pulling value through the system.

Kanban Pipelines

When dealing with a kanban process and visualizing it into a kanban board, the various steps that a card goes through is often called a pipeline. A single card starts at one end of the pipe and flows through to the other end. This flow is enabled via the pull system that happens at the end of the pipe.

In the grocery store example from my previous post on pulling value, the pipeline would would likely include the store shelf, the store back room, the warehouse, the supplier and the product creator.

A Simple Software Development Pipeline

A software development pipeline works the same way as the grocery store pipeline. In this case, though, the product flowing through the pipe is likely to be a feature of the software package.

Consider the following three columns in a simple pipeline for software development: Analysis, Development and Testing. When a customer requests a given feature for a software product, they want to pull that feature out of testing so that they can start using it.

Once that feature has been moved out of Testing and the customer is ready to pull the next feature out, there isn't anything to pull. At this point, the Testing people would then try to pull the next feature out of Development.

And the same pull happens from Analysis to Development.

In the end, we have created a pipeline for how our development process works. The work that is done flows through that pipeline based on how often the customer wants to pull features out. As one feature exits the pipeline, another feature can be added into the pipeline.

The key to all of this is, again, pulling the features through the system.

Where Do We Go From Here?

I've briefly shown how we can take three steps and produce a simple pipeline for work to flow through. There is more to software development than just Analysis, Development and Testing, though. We also have to consider team size and makeup, parallel work, and other constraints. In the next entry of Kanban in Software Development, we'll flesh out a more complete pipeline and take the next steps to show how to complete our Kanban board, enabling our customers to pull features out of our development process.

Before I start talking about how our team is going about our implementations of Lean and Kanban, I wanted to start by outlining my current understanding of what kanban is. I'm hoping that this will set the ground work for the rest of my Adventures in Lean series, and implementing Kanban in a Scrum shop.

Groceries, On And Off The Shelf

The next time you're in a grocery store looking at a shelf stocked full of products, take note of the little stickers that adorn the shelf just below where the product is placed. This sticker is usually crammed full of information - some of it useful to you, the consumer, and some of it not useful to you. So why is all of that information there? Who is getting any value out of that information? The answer is surprisingly simple - the people that stock the shelves. When the supply of good on the shelf is running low or is empty the information on those stickers tells the stock-persons what to put back on that shelf. This person is then able to create a pick-list of goods to retrieve from the back of the store so that they can restock the shelves.

Now consider an extended example of more than just the grocery store. When you, the consumer, take an item off the shelf, you have created room for one more item to be placed on the shelf. The stock-person sees the sticker in that space, goes to the back of the store to retrieve the item and places it on the shelf where the space is. At this point, there is another person paying attention to how much is actually in storage in the back of the store. When the stock-person grabs an item from the back of the store, the back-room person sees that space and determines whether or not they need to order more of those items from the warehouse. When the back-room person orders items from the warehouse, the warehouse workers check to see if they need order more of those items from the supplier. When the warehouse orders those items from the supplier, the supplier checks to see if ... and on and on until we finally get to the point where the products are being made - the manufacturer or farm or whatever the ultimate source of supply actually is.

What Is Kanban?

This simple act of seeing that a space on a shelf needs to be filled, and filling it, is the core of what kanban is. Quite literally, a kanban is a sign or signal. In more conventional terms, though, it is a sign for action that must be taken to replenish the empty slots on the shelf of the grocery store, the back room, the warehouse, etc.

From Wikipedia:

"The Japanese word kanban (pronounced [kambaɴ]) is a common everyday term meaning "signboard" or "billboard" and utterly lacks the specialized meaning that this loanword has acquired in English. According to Taiichi Ohno, the man credited with developing JIT, kanban is a means through which JIT is achieved."

When a consumer takes an item off the shelf, the process of pulling products - pulling value, really - from the upstream suppliers begins. It's the pull of kanban that makes it such a powerful system. We let the customer pull the items they want which drives the rest of the pull system. If no one takes the item off the shelf, there is no need for the stock-person to put another item on the shelf, and no need for the store to order more from the warehouse, etc. Think of it as a form of Demand-And-Supply instead of Supply-And-Demand. The customer demands a product, the grocery store supplies it. The store demands the product, the warehouse supplies it, and on and on up the stream.

What Is An Order Point?

When a space is empty on a shelf, a stock-person may not immediately fill it. In fact, the stock person may wait until there are several spaces on that part of the shelf before filling them. On this shelf, the maximum number of empty spaces allowed is the order point. For example, if the shelf has three empty spaces and the order point is set at three, the stock-person knows that they need to replace the items on the shelf soon. If the number of empty spaces goes up to four, the stock-person knows that they need to replace the items on the shelf now.

Order points can be specified in a few basic ways:

• Maximum number of empty spaces needing to be filled
• or Minimum number of products on the shelf

I'm sure there are other ways, as well. These are the two that I've seen the most often, though.

Each area of the grocery store is going to have it's own set of order points. The shelves that consumers pull from will probably have a very low order point - may two or three items missing. The storage area of the store will probably have a larger order point - two or three cases missing. On up the supply stream, the warehouse may have a larger order point of two or three pallets, and on and up the stream the order point may grow larger.

In the grocery store, the kanbans that adorn the shelves often have the order point printed directly on them. This gives a stock-person the knowledge they need, when they need it, and prevents them from having to remember the order points for all the products in the store.

What Is A Stock Limit?

When a shelf at the grocery store is full of products - there is no space left to put anything else - that shelf is full. However, the individual products on that shelf may not be at their stock limit. A stock limit is the maximum number of a given item that can be stocked at a given time - wether that stock is on the shelf or is in the storage area of the store, or wherever it is. If a shelf has a stock limit of five, for a given product, then there should be no more than five of that product on the shelf. Both the shelf and storage area have their own stock limits. It's these stock limits that prevent the store from being overrun with too many of one product, and not enough of another product. 

Like order points, the kanbans that adorn grocery store shelves often have the stock limit printed directly on them. This tells the stock-person when to stop putting items on the shelf without them having to remember the limits for all the products in the store.

How Does This Apply To Software?

I'm not going to give away the farm just yet. Moving forward, though, I will be using the terms Kanban, Order Point, and Stock Limit (or just Limit). I wanted to get these core terms defined up front, so that they will make sense in the context of what is still to come.

Stay tuned for the next entry in my Adventures In Lean!

In the last six months, my team has undergone some very radical changes and has turned into a full blown Agile team. I'm very happy with our success and I consider this team to be the shining example in our company, at the moment.

Now, in keeping with the tradition of this team in changing at least one thing every few weeks, we are about to embark on a new journey in our project management processes: Lean Software Development.

Starting at 1pm, Central Time, today, my team will be kicking off the following processes:

• Kanban
• Release Per Feature
• Just In Time Retrospectives

I am going to be posting a series of entries on each of these specific items over the next few weeks, and will most likely keep this post updated as the index of entries for a while. So stay tuned for a whirlwind of opinionated posts on our next great experiment!

Table Of Contents For Adventures In Lean

Here are the articles that I have written or am writing for my Adventures In Lean series:

Kanban

1. Kanban - Pulling Value From The Supplier - In this post, I am laying the foundation of what Kanban is along with a couple of other important terms that I will be using throughout the series. Kanban is only one of many tools, techniques and philosophies found in lean, though. Trying to sell kanban as lean would be like selling a steering wheel as an entire car - you're only getting part of what you really need.
   
2. Kanban in Software Development
   1. Part 1: Introducing Kanban Boards and Pipelines
   2. Part 2: Completing the Kanban Board with Queues, Order Points and Limits
   3. Part 2.5: A Variation on Queues - Pipelines for WIP and Done
   4. Part 3: Andon and Jidoka - Handling Bugs and Emergency Fixes in Kanban
3. Our Kanban Board and Process 
   
4. Release Per Feature - Delivering Value As Soon As Possible
   
5. Just In Time Retrospectives - Fixing Problems As Problems Occur

There are likely to be other articles added to this list as well. Please check back now and then to see what has been added!

I was reviewing my calendar for the upcoming month of October, and it looks like I'm going to have a crazy-fun month!

October 3rd & 4th: Pablo's Day(s) of TDD

This is going to be a great couple of days of learning, and hopefully contributing, to spread 'the good news' of TDD.

October 7th: AgileAustin montly meeting

All about test automation! A subject I'm very interested in, recently.

October 13th: Austin .NET User Group (ADNUG)

Presentation by ... ME!

I'll be giving my SOLID Software Principles presentation, talking about why we want SOLID software, as well as how to achieve it.

October 30th - November 2nd: KaizenConf: Continuous Improvement in Software Development

Otherwise known as an Alt.NET Conference. This is going to be a great weekend of learning from the best and brightest minds in the Alt.NET community, as well as Tom and Mary Poppendieck!

... October is going to be a whirlwind of learning, for sure.

I've talked about code coverage and software quality with a number of people on Twitter in the last month or so, including Jimmy Bogard, who has a nicely written post about Quality and code coverage.

It's no secret that I have some fairly radical opinions on code coverage and that I don't agree with Jimmy in these regards. I firmly believe that 100% code coverage, in whatever form we can get, is a reasonable goal and not past the point of diminishing returns. As such, I'd like to respond to some of what Jimmy is saying and expand on it with my own thoughts.

But first - I don't like to draw hard lines between "Unit" tests and "Integration" tests - that just blurs the problem even more because we can argue that you are covering code or not, in "unit" tests all day long, based on this blurry line. This is not the discussion I want to have, so I am lumping "Unit" and "Integration" tests into "Unit" tests as a whole.

Secondly, I'd like to say that code coverage is not always an indication of quality - in poorly written systems, it's merely a measure of code coverage. However, quality code is another subject of much debate.

On to the response!

.............................................

"The idea is that the team, all practicing TDD, should dutifully measure and add unit tests until they reach the assumed pinnacle of unit testing: 100% coverage."

There are several assumptions and statements in this that I don't believe are true.

For one, TDD and Unit Testing are not the same - no where near the same. Unit Testing lets you add tests as you see fit - after you write the code, while you write the code, whenever you want. You can even do test-first unit testing. True TDD, however, is a pure pragmatic approach to software development in that you never write code that does not have proof of need to exist - a test that says it needs to be there. 

The "pinnacle" of TDD and Unit Testing are not the same thing. Unit Testing - the act of adding unit tests when you see the need - may have a goal of 100% coverage. TDD, on the other hand, has a goal of only implementing the code that has been specified as needed. 100% coverage is the default in TDD because you never write code you don't need.

If you are going through the "measure and add unit tests" process, I would put money down to say that you are doing Unit Testing and very little TDD. The process should be something more like "measure to make sure we haven't added anything we don't need yet". 100% coverage is a side effect of TDD.

"The general motivation behind 100% coverage is that 100% coverage equals zero bugs."

Unfortunately, this is the motivation for many people who write unit tests - but it's the wrong motivation. You will never be 100% bug free just because you have 100% code coverage. There will always be some business case or external system factors that cause a bug now and then. That doesn't mean we should be ok with letting bugs into our system. On the contrary, one of our motivations (not the only one) should be to prevent as many bugs as possible from getting into the system.

"NCover is a powerful tool, but it still doesn't support all types of coverage.  Attaining 100% coverage in NCover still means there are paths that we haven't tested yet, which means there are still potential bugs in our code. "

I think the logical conclusion of this argument is that if we don't have a tool that supports true 100% coverage, then we shouldn't use that tool at all. I know I'm inserting my own conclusions into Jimmy's words - that's simply the conclusion that I came to from these statements. And this is a very bad conclusion. Do you walk around naked simply because wearing clothes would get them dirty and you'd have to wash them? Both of these are akin to the broken window syndrome. Use the tools that are available to the extent that they can operate, and find better tools when you can.

"If 100% coverage is a goal,"

100% coverage should NEVER be a goal - it's merely an indication that you are on the right path and working pragmatically.

"In recent projects where we measured coverage several months in to the project, we saw regular numbers of 90% coverage.  This was on a team doing 100% TDD." ... "So are we doing TDD wrong?"

So, are you doing TDD wrong? Yes. You are unit testing in the guise of Test-First-Development and not adhering to the spirit or pragmatism that TDD wants.

"Every test introduced covered behavior we considered interesting.  If behavior isn't interesting, we don't care about it. "

If you have code that is not interesting and you don't care about it - why is it in the system? If you don't care about the code, then it makes no difference whether or not that code works correctly. I would be appalled to hear that this is true. If it is true, then that code should not be in the system to begin with.

"If tests are a description of the behavior of the system, why fill it with all the boring, trivial parts?  The effort required to cover triviality is just too high compared to other ways we can increase value."

I think your definition of "behavior" is wrong. From wordnet.princeton.edu/perl/webwn:

Behavior: "the action or reaction of something (as a machine or substance) under specified circumstances;"

So, how is throwing an exception when a parameter is null, not behavior? If you are doing defensive programming to make sure you have all the data you need, then you are creating behavior - system behavior, not business process behavior. Behavior is still behavior, though. The only possibility of code not being behavior is a simple data point. If you have a data point in your code that is not covered by a unit test of behavior, then you don't need that data point. Yes, yes, I know - "integration", "nhibernate", "not my data source", "it has to be there for ...". If you argue that you need a property for some external portion of the system to operate correctly, then you should be proving that you need the property by covering it with a unit test that specifically says what it is for and shows how it is used. If you don't, then there's no way of knowing why the property exists and it may be deleted because it looks like nothing in the system uses it. Or, possibly worse, you end up with a lot of dead code in your system because you are afraid of deleting anything.

"Missing in the 100% coverage conversation is the effort required to get to 100%.  Attempting to get another 5% takes equal effort of the previous 10%.  The next 2% takes equal effort of the previous 15%, and so on."

I'd like to know where you got the math. If you are talking about unit testing - even test-first unit testing - i can understand the pain you're talking about, here. Why should I bother setting up another test fixture and getting all the state of my objects in place for a simple null reference check? blech - that totally sucks, is boring, is tedious, etc.

However, my typical process of achieving 100% is to only write code that is needed for the system to work, by specifying how the system will work through tests and then coding it. If I have a null reference check somewhere in my code it should be because I already have a test that specifies why it's needed.

"This is called the law of diminishing returns.  As we get closer and closer to 100%, it takes vastly more effort to get there.  At some point, you have to ask yourselves, is there value in this effort? "

Here's the real crux of the problem - you are assuming that you started with, or at some point has less than 100% coverage. It takes zero additional effort to stay at 100% coverage if you start at 100% coverage and maintain it through pragmatic TDD.

"Often, bending code to get 100% can decrease design quality, as you're now twisting the original intent solely for coverage concerns, not usability, readability or other concerns."

If you find yourself bending code to get 100% coverage, you have one of two situations (if not both): 1) poor design in your code, 2) poorly written tests. I would bet that you have both because they are a vicious circle.

"Measuring coverage is an interesting data point, as are other measures such as static analysis.  But in the end, it's only a measure, an indication. "

100% agreed!

"It's still up to the team to decide on the value of addressing missing areas,"

This implies that you don't have 100% coverage, already. In this situation, 100% agreed. I'm in this situation right now - it's hard to know what tests create the most value, at this point. However, any new code - even code changes to existing code - are done TDD style. So, while we don't yet have 100% coverage, we are increasing coverage all the time by never writing code without a test (that's part of the goal, anyway... no one is perfect, and it takes serious discipline to do this)

"with the full knowledge that they are still limited to what the tool measures."

Agreed, again. If you don't know the limitations of the tools you are using, you're in trouble.

.............................................

The boilerplate argument that I am making is that you should never write code that is not required by a customer / consumer. However, as I've Previously Talked About, there's a high likelihood that you have not identified all of your customers / consumers. Take NHibernate, for example. I have a project that needs about 30 different properties for a specific NHibernate query to be executed. The NHibernate query is the only place they are ever used - no code reads them, only a search screen writes to them, then the NHibernate query loads data from the table that they are mapped to. When I first wrote this code, my coverage was at around 30%. This situation is one of many that I've been in recently, that has lead me to believe that other code must be considered a customer / consumer of your code. If I deleted any of these properties, then the NHibernate query would fail. Therefore, the properties are valuable to NHiberate. Since these properties are valuable to at least one of the consumers of my code, I should prove that they need to exist by writing a test that proves they are needed and why.

There's been a lot of recent talk about what is "done" in the Lean / Agile development communities, and the primary focus of these discussions has been focused on individual features or stories and the need to get them completely "done" (dev, test, acceptance test, documentation, delivered to customer) before they can be considered done. I wholeheartedly agree with this philosophy, to the point where I active introduce painful elements of software development to my team members, because the work must be done before we can deliver to the customer.

All that being said - I think we're missing out on some potential benefit by not applying "done" at various and different levels of our software development projects. For example, the concept of "done" can be applied to an iteration. What constitutes an iteration being "done"? It's certainly not just the time-box of 2 weeks, 1 month, or whatever our iteration length is defined as. However, that time box is still important. We don't want arbitrary lengths of iterations.

So how do we know when an iteration is completely "done" versus just being over? My initial thoughts revolve around a pass/fail checklist, similar to the swim lanes (or kanban board, if you want to call it that) that our stories go through. At this point, I would likely include the following:

• Iteration length passed
• All stories in iteration "done"
• Code reviewed (possibly part of story being "done")
• Software is in a stable, working state
• Software acceptance tested by customer (or customer representative / SME)
• Retrospective held

There's probably some additional items to include here - this is just my initial idea list.

Like the user story "done" criteria, an iteration cannot be considered "done" until all of the items in this list have been checked off. We don't give partial credit for a story making it to "in test", when there are 2 more columns to move through - the story is not done until it's completely "done". The same should apply to the iteration.

Why bother making an official "done" list for iterations? For the same simple reason we do this for user stories - transparency and visibility into bottlenecks and problems. It's easy for a team to ignore problems like broken software at the end of an iteration - "oh, it's just XYZ... i'll take care of that tomorrow" - when we don't have the same accountability as we do in our stories. By making the iteration a pass/fail set, we expose those problems for the world to see. This exposure is a great motivational tool - who wants to be caught with an un-"done" iteration because of a "simple" bug?

Do you and your team have an iteration "done" list? Do you even have a user story "done" list? It's all about exposing weakness and waste, then eliminating it and creating the official iteration "done" list is just one more step along the path. Don't stop there. What other processes - higher or lower level - can we standardize and improve?

The one and only goal in software development is to provide useful, functional software that creates value for the customer or consumer. Period. End of story. Not open for discussion or opinions.

If you want to deliver software that the customer or consumer wants to use, likes to use, and ultimately does use to help solve problems and/or automate business process, etc, then you must know what the customer or consumer values. In other words - you can't identify your software's 'value' if you don't know who/what ALL of your customers are.

I'd put money down to say that most software developers - perhaps most software development companies - have no clue how many customers and consumers there really are for any given software project. The one consistent answer that I would expect from every developer or company is that the person, group, or company paying money for the software is the customer. This is 100%, absolutely true. In addition to the paying customer, though, there are many other customers or consumers that need to be identified.

As a starter, consider the following list for the customers / consumers of your software:

• The paying customer
• Integrated external systems
• API consuming systems
• Software Testers and Test Lab personnel
• Technical and documentation writers
• Software developers (the ones writing the code!)

I know, I know... how could a software developer possibly be considered a customer or consumer of the software that they are writing? The answer should be obvious - who reads and writes the code? who maintains the code and needs to understand how the code is structured so it can be maintained? If you don't believe that your software developers are first class consumers of the system that they are writing, I would bet that you have a horribly complex kludge of code that no one wants to work on. If your developers are considered first class consumers of the code they are writing, I would bet that your team is happy and is constantly working toward better code - simple, readable, maintainable systems that are fun to work on.

So, who are your customers or consumers? What value do they need in your system and from your system? And, how are you and your team responding to those value needs (if at all)?

The entire team – BAs, Devs, UX, Testers, Tech Writers, Customers and Management (especially customers and management) – all need to understand that the first few iterations of a team that is trying to convert to an agile methodology, will be slow and painful. You won’t get as much done as you think you should. You’ll run into constant problems, questions, points of clarification, and unknowns.

Iterative development processes expose every single problem that your team never knew it had, and more – and if you’re honest and working with integrity, you won’t whitewash the problems and try to hide them; rather, you’ll embrace the transparency that iterations create and use it to drive improvement in your team.

Converting to Agile is painful and it’s embarrassing at time. There's no question that it’s difficult to convert – especially if the team has any significant habits or experience from previous projects (and who doesn’t, aside from entry level people). Work through the pain, solve the problems one at a time and continue to improve the process. Eventually, the team will find it’s groove and productivity will increase. Over time, if the team truly is honest with itself and is continuously solving problem, productivity should skyrocket and you’ll wonder how you ever worked without iterative / agile methodologies.

The key is - don’t expect it to be perfect… ever… take one step at a time and never stop moving forward.

A Food Network Challenge

During this season's 'The Next Food Network Star', there was an episode where the contestants had to cook a dish and review it in front of a camera, describing it's visuals and taste. The twist to the challenge was that they were assigned someone else's dish to review, after everyone had already cooked. Once the assignments were made for who was going to describe who's dish, they were not allowed to see the dishes anymore. A given contestant was then put in front of a camera and had their assigned dish uncovered in front of them. From that point, they had 90 seconds to describe the dish that was in front of them - that included figuring out what it was, what it looks like, what it tasted like and what it smelled like. This was a rather tough challenge to watch, and a few of them didn't do so well.

A Code Review Challenge

What if we take the Next Food Network Star's challenge and apply it to code reviews? When we sit down in a code review session, rather than having the person who wrote the code doing the driving and telling us what is going on, have someone who has not seen the code do the driving - read the code, read the unit tests, etc - and describe to us what is going on. 90 seconds wouldn't be nearly enough time, though, so we'd have to change that time limit based on the size and complexity of the functionality being reviewed. Other than that, though, it seems like it would be a great way to test the team's ability to pick up an area of the system that they have not seen before and quickly learn it so that they can work in it.

Standardized Work - Judging Code And Code Reviewer

There are 2 real benefits to the code review challenge, as I see it:

1. Judging how well did the coder apply our coding and development standards
2. Judging how well the reviewer can recognize and describe the standards being applied

We are now building our system from User Stories, with Context / Specification style unit testing. One of the intents of this style of unit testing is to provide an easy-to-read and easy-to-learn set of examples for developers that need to know how to do something or how something works in the system. If the unit tests are properly written and expressive of the code's intent - it should be really easy to see what's going on. If we have good standards in place, a developer should be able to find the code in well organized areas, easily, and know where to look for the various components and parts.

It seems to me that the code review challenge would help to prove a number of these key factors in our projects and our teams:

• Coding and architecture standards
  • How well the standards are known
  • How well the team follows the standards
• Expressive Code
  • How easy is it to read the code
  • How easy is it to understand the code's intent
• Maintainable Code
  • How easily can the code be changed and extended
  • How quickly the team can socialize and collectively own the system

A Challenge In Name, A Learning Experience In Practice

I would avoid applying any sort of point system or other reward / incentive for the code reviews. It seems that this could easily backfire and cause junior team members (in skill or time on the team/project) to feel intimidated or feel that it's an unfair process comparing them to the senior level team members. The intention of the code review challenge is not to cause dismay in any team member, but to create a learning environment by continuously challenging every member of the team. Ultimately, the code review challenge should foster the socialization of the system and lead to a strengthened sense of collective ownership, resulting in a better system.

Nothing New Under The Sun?

With all that being said - I'm doubtful that I'm the first person to ever hold a code review like this. I'd be interested in knowing if there's any standard development practices (Agile or otherwise) that are heading in the same direction. Any advice or opinions of you, the reader, would also be very welcomed.

If you know of a resource that describes the same basic ideas or if you have any input to help me improve the idea, please leave a comment and let me know.

My team's first swim lanes!

From left to right: Backlog, In Dev, In Test, Docs, Done.

sweeeeet.

Scott Bellware (who finally started blogging again), is posting a series on Sustaining Capacity in Maturing Agile Software Teams. It's worth a read, and may be familiar territory to anyone who has done Agile and been reading about Lean.

In the 3rd Post - Recognizing Entropy - he talks specifically about synchronizing the development testing and QA testing efforts:

Development and QA Test Synchronization. Without greater synchronization between development and QA testing, valuable input often comes too late in a development or release cycle to be effective. Test design and test architecture are valuable inputs to development, and software design and architecture are valuable inputs to QA testing. Teams often loose unrealized capacity by not pursuing means to do more development and testing in parallel.

This one really hit home for me, and is a shining beacon to the problems that my current team is going to face, soon. We had our first iteration planning and kickoff meeting yesterday, and the whole team was involved - except the testers from the QA department. For reasons that I have no control over (and honestly - they are legitimate business reasons), the QA department is not coming on to the project for another 2 months (at least).  Today, the team leadership was discussing how the QA personnel would create their test scenarios, and I brought up the point that creating the test scenarios after-the-fact (after the iterations are done) is going to be very problematic.

If a dev team is working from User Stories, and those User Stories are divided along various aspects of behavior, it is very likely that a single screen, workflow, or 'function' of a system is going to be divided into multiple user stories. When the QA team is involved with testing in parallel with the development team, this is not an issue because the QA team will be updating their test scenarios in each iteration. This allows them to keep the scenarios for a given screen, workflow, or 'function' of the system, growing over time. However, since our QA team is going to come into the project at least 2 months after the development team has started coding and testing, the QA team would have to sift through all of the current and previous iteration stories, try and aggregate them down into the screen, workflow, or 'function' of the system, and build test scenarios from there. If there is even a modest number of stories to sift through, this would be a daunting task for the QA team - they would be in a 'catch-up' mode from the beginning of their QA cycle, likely through the end of the project.

As a result of the QA team being unable to join our project immediately, we are forced to modify our entire development process and account for the needs of the QA team, after the fact. Essentially, we are creating additional swim-lanes for our stories to travel through, during an iteration:

1. Backlog
2. In Development
3. Developer Testing
4. Ad-Hoc Testing (BA's, Developers, etc. testing the app)
5. QA Functional Spec Documentation
6. Done

Swim-lane #4 and #5 are directly affected by the QA team not being able to enter the project immediately, and are causing the rest of the team to lower it's capacity. We (developers and BA's) are now required to do our own ad-hoc testing during the iterations, instead of letting the QA team take care of that; and we (the developers and BA's) are now required to create functional specifications, based on the user stories that are completed, to hand to the QA team once they are on the project. The functional specification documents will need to be updated for any given User Story, before the ticket is considered complete (the functional spec updates have become an acceptance criteria on every User Story).

End result - because we cannot get the QA team on our project immediately, and ensure that they are testing in parallel to the developers, we as the development team are losing capacity and building entropy from the very first day of the very first iteration.

Anyone that has worked on a project with me is familiar with my usual "I just fixed … whatever" emails. For example - yesterday, I sent out this email to my team:

FYI – I just pulled down the latest code from subversion, with the new solution and project reorganizations… there were several project reference errors in the {our project} solution – many of the projects were trying to reference the \Trunk\Code\Library\ folder for the compiled output of a project that was in the same solution.

We need to make sure that projects within the same solution reference the project directly, not the compiled dll file. You can make sure this is correct by deleting everything out of the Library folder, than updating it from subversion (to make sure you have no {our project}.dll files in the Library) and then see if you get any missing file errors in visual studio, for the project references.

I’ve been sending out similar email for several years now, and I’ve always felt that it was important to do so, so that the entire team can gain new insights, move forward with better techniques, creating better software, etc.

Yesterday, I found An Article on the Lean Manufacturing concept of Jidoka and a few items stuck out in my mind and helped me to understand why I think that it’s important to call out these items. The core of jidoka goes well beyond the basic idea of "If it's broke, fix it now". It gets much more involved than just that. The article has a list of steps that should be taken whenever something is found to be 'broken' or 'abnormal':

1. Detect the abnormality
2. Stop
3. Fix or correct the immediate condition
4. Investigate the root cause and install a countermeasure

There's a lot more to the real implementation of jidoka than just fixing the problem - finding the root cause and installing countermeasures to prevent it from happening again is by far the most important step in this process. If you fix the problem in isolation and no one else knows that the problem occurred, then it's likely that the problem will happen again for the same reasons. This gets straight to the core of why I send out the “I just fixed …” emails. I found this quote in the article that further illustrates the point:

What I would not want to happen is to expect the team member who discovered the problem to try to fix it without telling anyone. But many times we expect them to do just that, and the rework becomes so deeply embedded into the routine that we can't even tell it is happening. It seems normal because it has never been flagged as abnormal.

I don’t send out those emails to chastise anyone, point out that someone broke something, or anything negative like that. Honestly – I don’t care who broke something; I only care that it is broken, it gets fixed, and we find the cause and do what we can to eliminate that cause. I send out those emails because when we (as a team – any of us; developers, testers, subject matter experts, etc) find a situation that is not right, we need to fix it – but more importantly, we need to notify the rest of the team so that we can learn as a team and find ways to not make the same mistake in the future.

I highly recommend that everyone go read that article and I also want to suggest that any time any of us runs into a situation where something is wrong – whether it’s code that is written or designed poorly, a references issue, the build being broken, or whatever it is – we not only fix the problem, but we inform the team of what the problem is/was and how to fix it and prevent it. We need to have this level of communication in the team, or we are not going to grow as a team or as individual developers, as fast as we could.

Yesterday, I posted a quick thought on code generation, and one of the statements I made is worth re-stating to stand on it's own.

In Agile/Lean software development, a single User Story is one piece flow when implemented via TDD, in a Workcell

In Lean Manufacturing, one piece flow is the idea that you do not produce anything in batches, but that you produce one product from start to finish per a customer's order. In Lean / Agile software development, this is analogous to an iteration with user stories. An iteration backlog is a customer order - a set of behaviors or features that are requested to be delivered in the current iteration. A user story is the single piece that we want to flow from beginning to end - from the start of coding all the way to acceptance via user testing.

In manufacturing, a Workcell is used to facilitate one piece flow through the manufacturing process. In software development, a Workcell is also used to facilitate the one piece flow. The Workcell may be 1 person, pair programming or a three person team. In order for it to truly be one piece flow, though, strict standards such as TDD and automated acceptance tests must be followed.

I'm not going to detail the benefits of one piece flow, here (not yet, anyway... another post for another time). I highly recommend that you read The Toyota Way and Lean Thinking for a good understanding of this concept and the benefits it provides.

Until around a year ago, I was an advocate of code generation via CodeSmith. Having marginal "success" with it in the 4 years I advocated for it, I'm now of a different opinion.

Code generators, such as CodeSmith, are automated overproduction machines that require prior overproduction, in the form of schema, to be used

Micro code generators like as Resharper, are much closer to JIT machines when lined up in one-piece-flow processes, such as Test Driven Development

...

If you had told me, 2 years ago, that I would make these statements today, I probably would have laughed at you.

There are a few examples in The Toyota Way, of American manufacturing companies that were thought to be world class Lean Manufacturers. When representatives from Toyota began working with these companies, though, the result was quite a shake-up. The companies that were thought to be lean, were merely implementing some of the surface level processes of lean manufacturing. They had not realized that there was a root cause of the process that they were implementing.

The Principles, Not The Process

A coworker recently asked me to help out with a presentation on Model-View-Presenter. He had some specific questions about why we would want to use MVP vs. MVC vs. any other UI pattern. Many of the questions centered around various benefits that have been promoted by myself and others out in the developer world via blogs and articles - items such as testability, changeability, flexibility, and other "ilities". My answers to some of these questions surprised me.

Model-View-Presenter is often thought of as a journey and an end in itself. When I first started learning MVP just over a year ago, this is exactly how I saw it. I started with the intention of learning how to separate my core process from the form implementation, with the intention of being able to unit test the core process. I was working under the impression that I could find a way to implement MVP that would truly allow me to swap out my UI - to the point of changing the workflow within the UI. I had the goal in the mind, of being able to write one presenter and have it support WinForms, WebForms, Web Services, and any other client-facing API.

One year and several projects later, I have some good lessons learned. I have implemented various forms of MVP - automagic view injection, manual everything, and a lot of ideas in between. My understanding of MVP is fairly solid, and I am capable of implementing it in WinForms, WebForms, Web Services, and other client-facing API's.

What I have realized recently, is that MVP is not (or, should not be) a journey and a goal in itself. MVP, MVC, and other UI related patterns are actually an effect of other underlying principles. The underlying principles of object oriented development - such as Separation of Concerns, Single Responsibility Principle, Dependency Inversion and others - are the real cause of MVP, MVC, etc. Testability, flexibility, and the other "ilities" are merely side-effects of good design and implementation. And while Testability is a valid goal in itself, seeking it as the primary or only goal will only lead you part way down the path. You will end up like the American manufacturers that thought they were world class lean companies, only to find out that they had barely scratched the surface of lean process.

The Journey, Not The Goal

Despite the success of Toyota as the creators of lean manufacturing, they believe that they are continuously learning about lean, so they can continuously improve. Similarly, despite my many years of software development experience, I now believe that I know little about software development. I have a good knowledge of .NET and general knowledge of Object Oriented Development, but I don't know much about other paradigms like Functional Languages or Duck-Typed languages such as Ruby.

In my professional growth as a developer, I have set goals for myself at various times. Most of these goals were set with the belief that they would make me a great developer. However, I've found that every time I have reached a goal, there is an entirely new world of possibilities ahead and what I thought was a great developer was merely the beginning of a journey. I've discovered that the journey itself is often far more important than what I believed the goal to be.

I'm not advocating that you forget about setting goals and let your career happen as it will - that's a great way to become burned-out, outdated, and irrelevant. What I am advocating is that when we have a goal in mind, we set out with the understanding that the goal itself may only be a small leg on a long journey. In this case, the journey is the process of learning. The journey itself should be viewed as the long term strategy, to the point where you allow the individual goals to come and go as needed. You may not accomplish a specific set goal - but the experience gained while working toward that goal may open up opportunity for other goals to be reached easily.

Learn The Principles Via The Process

All this talk about knowing the underlying principles is great - but how can we understand the principles and philosophies without knowing the process? Unfortunately, I don't know if that's possible for most of us. I'm sure that somewhere out there, someone has read SRP, DI, SoC, and other principles, and as a result began writing code that was modeled in a manner similar to MVP (Martin Fowler, for example). For the majority of us, though, learning the principles is done by first learning the process - study the implementation of the effect to learn the cause.

Conclusions

The journey itself is how we build our experience and understanding. For most of us, that journey begins with a goal of learning a new process. Find a goal or two that you believe will help you be a better developer and learn the processes. Along the journey of reaching for this goal, always keep in mind that there is an underlying principle or philosophy that enables the process. By learning the principles and philosophies behind the process, the process itself becomes a trivial matter - an implementation of the principles and philosophies, and not a goal in itself.

The Toyota Way mentions the concept of Jidoka in chapter 1 (and probably other places that I haven't read yet). On page 6, in the "4 P" diagram, jidoka is described as

"Stop when there is a quality problem"

Wikipedia calls the same concept "Autonomation" and says it may be described as

"intelligent automation" or "automation with a human touch"

and

"At Toyota this usually means that if an abnormal situation arises the machine stops and the worker will stop the production line."

We can apply the same principles in software development in many different ways. One of the more common implementations is the use of Continuous Integration and Automated Testing.

According to the notes I took during A Day Of Bellware, if our CI server says the build is broken, we need to immediately stop working and fix the problem. I've heard this before, and will likely hear it again. However, I never really understood why people would say this. After all, as long as the problem is fixed eventually, isn't that ok? In the training that day, Scott provided a excellent visualization of why we should fix it immediately.

Waste and Rework

To steal Scott's illustrations, consider the following image to be an example of "perfect" software. All of the edges of each module (block) are well defined and it's easy stack new blocks next to existing blocks.

Figure 1. "Perfect" software

Now let's assume that somehwere in the coding process, someone accidentally causes a defect in the software. That defect could be represented by a buldge in one of the lines - as if the module was doing more than it should.

Figure 2. A defect

The individual block that has the defect may not be that bad, at first glance - or when examined on it's own. And on it's own, the defect could be fixed. Jidoka would say that you need to stop immediately and fix the problem.

So, what happens when you don't fix the problem right and and you try to stack another block to the right of the defect? Suddenly you find yourself re-shaping the next block in order to account for the problems in the previous one. Eventually, you may be able to smooth out the issues. Depending on the size of the issue, though, it may take several new modules to completely normalize things.

Figure 3. The effect of a defect

Now, 3 day, 3 weeks, 3 months, or whatever period of time later, you have a significantly larger problem to deal with. Let's go back and fix the original cause of the problem, to start with. What happens to the rest of the code that was warped around the original defect? The warped code in the rest of the system also has to be fixed. Simply removing the original bulge does not mean that the rest of the warps will magically disappear as well.

Figure 4. Removing the original bulge

How much re-work will it take to fix the rest of the system that was warped around the original problem? How much time and effort will be wasted because the original defect was not addressed immediately? These questions can only be answered in the context of your problems. If this defect is one line of code in one module, maybe it's not so bad. But if this defect is an entire module used by many other modules, the time and cost could be huge.

Conclusion: Fix It Now

If the continuous integration server says that our build is broken; or if the customer says "the software does not break, but piece XYZ isn't correct"; or ... pick a problem in your system; we need to respond as quickly as possible, in order to prevent the rest of the system from being warped by the original problem.

Under the assumption that we want to fix a defect as soon as possible - how do we ensure that we know about the problem as soon as possible? Whatever your implementation of this solution is, the solution to this problem comes down to shortening the feedback loop. If you are notified of the problem in 20 minutes vs. 20 days, there will be significantly less damage to the overall system and significantly less re-work and waste caused by the defect.

Myself and 10 other developers in my company went through a day of BDD / TDD training, with Scott Bellware, yesterday. It was a lot of fun, very challenging at times, and covered a lot of topics including an overview of Agile and Behavior Driven Development, all the way down to writing Specification Tests, doing Test Driven Development  and refactoring the model to improve readability, maintainability, flexibility, etc. I took notes via index cards (love that cool-aid) and wanted to share. I don't expect these notes to make sense to everyone. Hopefully it will spark some dialog in someone's mind and cause them to dig further.

First off - the quote of the day.

"Can I be honest with you and say that I've been wanting to touch your keyboard, all day?"

Now for my notes.

User Stories

[Role], [Goal], [Motiviation]

• As a [role], I want to [goal], so that [motivation]
• Example: As a nurse, I want to record a patients vital signs, so that I can determine their medication and care needs
• Motivation is critical - it determines how the development team understands and implements the story. It determines the user experience, how things are integrated, how the software is designed, etc.

Acceptance Criteria

• Acceptance Criteria is used to drive code, not the story, directly
• may change at any point, up to implementation
• is used to drive code design, test design, implementations, etc.
• should be spoken in domain language
• may include non-functional, technical details such as database tables, infrastructure, performance, etc
• All acceptance criteria must be met and tested / verified before a story is considered done

Specification Tests

• Test Fixture per Class is an anti-pattern (on a personal note, this problem bothered me for months before I discovered BDD)
• Context Specification or Behavior Specification testing
• When [verb] then [verb]
  • "When [verb]" is the context
  • "Then [verb]" is an observation of the behavior
• Based on Acceptance Criteria, but not "code-gen'd" from acceptance criteria

Story Estimation

• Agile Poker: uses generalized Fibonacci sequence as order of complexity
  • "?", 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, infinite
• everyone throws their estimate at same time
• if estimates have significant outliers, discussion occurs to understand why, get more detail, etc. and re-throw may happen

Entity Data vs. Aggregate Data

• Entities should never contain aggregate data
• Aggregate data is for reporting and other aggregate needs
• If you need aggregate data to process something, write an SQL query, stored proc, etc. - don't use an ORM like NHibernate
• We don't want a "Customer" entity to need 10,000 "Order" entities, to aggregate data for processing; write a query to aggregate instead
• We don't want to persist data that can be calculated / aggregated, generally (performance issues may override this)

Domain Services

• Can have dependencies on external systems
• are part of domain logic, therefore are in domain model / assembly
• are "Doers" of process that don't fit into entity and entity logic, directly
• coordination of entity logic
• can include calls to data access, logging, etc.

Continuous Integration

• Not just continuous compilation of code
• Full end to end integration of all code, components, databases, services, etc
• Full suite of integration testing including database testing
• Do not allow commits if build is currently broken
• do not allow defects to live - fix immediately, to fix build
• "Defect" is broken software, "Bug" is functional but wrong

Daily Scrum

• 3 Questions everyone answers:
  • What did I do yesterday?
  • What am I doing today?
  • What issues am I having?
• Each person should answer quickly - 1 or 2 minutes, max
• further discussion happens outside of the Scrum meeting

Productivity of Dev Team

• RAD and other non-review, non-iterative based management causes problems and loss of productivity
• we need constant review of the design to ensure good design
• shorten the feedback loop and get constant review of the design, to always improve the design, via pair programming, work cells, retrospectives, etc.
• good design will cause productivity gains in the development team beyond the capabilities of any tools

Whiteboard Diagramming vs. Details Specs

• White board diagramming and human interaction is always better than detailed documents and specs in UML
• Human interaction leads to knowledge crunching and learning, not just reading a repeating
• Take pictures, don't re-draw in UML; don't waste time with it
• Video the entire conversation is even better, so others can learn from the knowledge crunching that occurs; capture the human interaction, body language, etc.

One of the primary goals of Lean Manufacturing in the elimination of waste. This may seem like an obvious goal - after all, who wants waste in their system? What's really interesting, though, is seeing where the term "waste" is actually applied and how Toyota redefined what waste really is.

Muda, the concept of the seven wastes in the Toyota Production System, is a very different concept of waste than most of us would consider. It talks about not only physical waste (trash), but also logical, time, and effort waste. These seven wastes include (from Wikipedia):

• Defects: Quality defects prevent the customers from accepting the product produced.
  
• Overproduction: Overproduction is the production or acquisition of items before they are actually required.
  
• Transportation: Each time a product is moved it stands the risk of being damaged, lost, delayed, etc. as well as being a cost for no added value.
  
• Waiting: Refers to both the time spent by the workers waiting for resources to arrive, the queue for their products to empty as well as the capital sunk in goods and services that are not yet delivered to the customer.
  
• Inventory: Inventory; be it in the form of Raw Materials, Work-In-Progress (WIP), or Finished Goods, represents a capital outlay that has not yet produced an income either by the producer or for the consumer.
  
• Motion: As compared to Transportation, Motion refers to the producer or worker or equipment.
  
• Overprocessing: Using a more expensive or otherwise valuable resource than is needed for the task or adding features that are designed in but unneeded by the customer.

In "Lean Software Development: An Agile Toolkit", Tom and Mary discuss The Seven Wastes and offer a great set of parallels in software development. Rather than copy & paste what they've already said, I wanted to offer my own interpretation of these wastes - some of which are the same as Tom and Mary's.

Defects

This is exactly what it sounds like - a bug, a piece of functionality that doesn't work right, or an exception that gets thrown in the system and causes problems. How many hours, days and weeks have you spent fixing defects in software, in your career? Personally, I can't even begin to accumulate the time spent - it's far too great.

Part of the problem here, and what really makes it wasteful, is when defects are found and fixed in most software development cycles. In Code Complete 2, McConnell talks about the cost of defects in software, relative to when the defect is found and corrected (pages 472 & 473). If the cost of fixing a defect goes up over time, then the amount of waste also goes up. In other words, the higher the cost of fixing a defect, the more waste that defect has generated.

With the increasing cost of defects in mind, we will want to reduce the time to find a defect so that the amount of waste generated can be mitigated. In Lean Manufacturing, this is achieved by only creating what is needed for an actual customer order, right now. If one part is produced with a defect, that one part is scrapped and a new part is created, correcting the defect - the waste is minimal. Imagine a mass-production system where parts are created in batches of ten thousand. What happens if there is a defect caused by the process? We end up with a significant amount of waste, because we will have to scrap the defective parts in batches of ten thousand - the waste here, is significant.

The same principle holds true in software development. If we are writing code for extensive periods of time, and that code is not being tested until the end of that period, then the cost of finding a defect in that code goes up. If we have ten lines of code that rely on the defect working the way it does, it is significantly cheaper to fix than if we have ten thousand lines of code relying on the way the defect works. The easiest way of reducing the number of lines that rely on a defect is by working in small iterations. If that defect goes into test after two weeks, it will be much cheaper to fix than if it had gone in to test after two months. Additionally, if automated tests are done and can be run against the system on an even shorter schedule, then the cost of finding the defect may be even less. If the automated tests are run every twenty or thirty minutes, you may only have two or three lines of code that are affected by the defect.

Overproduction

There are several common terms for overproduction, in software development - Over-Engineering and Coding For The Future, for example. Both of these problems result in the same thing - too many features and more functionality than is actually needed right now. Additionally, overproduction can be called technical debt. The idea is that there is code in our system that will need change - when this code needs to be changed, how difficult is it to change; what is the cost of the change (how much debt are we in).

If a developer is tasked with creating Feature-X, they should focus on the core business value of that feature when writing the code. It's very easy for that developer to think about "what-if" scenarios and "we may need to" future functionality. However, this type of thinking leads directly to overproduction. If the business has not identified the "what-if" scenario, then why should the developer assume that handling the scenario would add business value? If the business has not yet identified Feature-Y or specified any additional functionality in Feature-X, then why should the developer assume that coding for the possibility of Feature-Y is valuable? Take note of the pattern in those questions - assumption. Assumption on the developer's part leads directly to the overproduction of the system being built.

So how do we prevent this overproduction? First and foremost - stop assuming you know what's coming down the line, or what will provide business value. If you believe that the specs you are currently working with are not accounting for an actual problem, then the responsible action is to discuss that issue with the customer / business analysts. Secondly, if you find yourself wanting to code for the feature that may be coming down the line - stop. Don't do it. Even if you know that this feature or functionality is coming down the line, it may not provide any business value for the current feature(s). You may be causing detriments to the business value of what is currently being built. If you truly believe that the feature you want to code is required for the feature that you are coding, then you need to go back to the customer / business analyst again, and discuss it with them. Let the customer tell you whether or not there is value in adding that feature now vs. later (vs. if at all).

Transportation

Think of transportation in the context of knowledge. No one knows everything about a particular domain or project, and possibly an individual feature. We have business analysts and customers precisely because of this. Eric Evans discusses knowledge crunching (the process of learning and gaining an understand of the domain in question, to create a model) in Domain Driven Design, in the first chapter. Tom and Mary talk about the process of learning all throughout the "Amplify Learning" chapter of Lean Software Development. There are countless other books on requirements gathering, model creation and other learning processes for software development, as well.

The problem that we face in our learning is that the transportation of knowledge, from one person to another, will always introduce change. Every person in existence has their own unique way of understanding, based on their experience, existing knowledge and methods of knowledge retention. To illustrate this, look back at your childhood. Did you ever play the "telephone" game where one person whispers in the ear of the person next to them, then they whisper the same thing into the ear of the person next to them, and so-on until the last person in line? What are the typical results of this game? More often than not, the message had some fundamental change to it throughout the transmissions. As simple as the telephone game is, it's a perfect illustration for knowledge transportation that occurs in software development projects.

How many times have your received a functional specification or requirement for a feature, and had no clue what it was trying to convey or what the functionality was supposed to be? This happens on a very regular basis. The problem may not be that the documentation is inadequate, but that the transportation of knowledge from one person's mind onto the paper and into your mind is inadequate - what makes sense to the document writer(s) may not make sense to you, because your contextual knowledge of the subject is different. This problem is exacerbated by design-up-front project methods. The knowledge is often crunched from the customer's mind into the BA's mind, and then trampled down into a linear document or subset of knowledge in a documentation set where the developer has no direct access to the customer - the source of knowledge - for clarification and discernment of the details. Even if the customer is on-site sitting next to the developer throughout the creation of the documentation, there is still a transportation problem in design up front projects. Whether or not the developer understands the information during the conversations, there is little chance that the developer will remember every detail of the information when they finally get down to coding.

The solution for the transportation of knowledge is ensure that the knowledge is transported as little as possible. Rather than focusing the effort of documentation on every last possible detail - creating tomes of documentation that no one is really willing to read - we should focus on creating higher level guidance; enough knowledge to know the direction of the development effort and the features to be created, with the understanding that the final details will come from the customer just-in-time for the development effort. In the world of lean manufacturing, this is typically done through the Kanban system. In software development, we have adopted the same process through the use of User Stories being written on index cards. The stories are not the final word in the development effort - they are merely a placeholder for a conversation between the customer and the developer.

Waiting

This is likely the most obvious of the seven wastes, when applied to software development. Whether you are waiting for a customer or business analyst to be available for questions; waiting for a requirements document to be signed off; or waiting for your next assignment of work, the simple process of waiting creates lost time which directly translates to lost opportunity. In the extreme cases, the problem is compounded by context switching. If you are waiting for a question to be answered - whether this means waiting for a phone call, a document to be written, or just walking down the hallway - you can easily lose your focus or lose you place in the code or coding process. When this happens, most people take a while to restore that focus and get back into the mind set where the new knowledge is useful. In my experience (no real data, just my experience), the average time of context switching within the same project is as little as five to ten minutes for wind down, and at least ten or fifteen minutes for spin-up. If you are context switching between multiple projects, though, expect to spend much longer on the spin-up; hours, even days, depending on the project and time between wind down and spin-up.

If we can ensure that we never wait longer than it takes for us to wind down, then we should be able to eliminate the spin-up period; thus reducing the effort required for reestablishing our focus on the feature(s) at hand. However, the solution for waiting may not be as obvious as the problem. If you are waiting for documentation, or if you are waiting for the customer to sign off on a feature or change, the way to eliminate the waiting is to eliminate the cause. Don't rely on heavy documentation or require customer sign-off on features. If we are working with the customer every hour of every day, eliminating the knowledge transportation issues, then we can also eliminate much of the waiting game by having the customer available. When the customer is available immediately, we have a much greater chance of the conversation beginning within the wind down period. When this occurs, we can stay focused on the problem at hand, and mitigate the spin-up period when we return to the code.

Inventory

Inventory in a manufacturing or supply context is typically a product or component of a product that is in storage and waiting to be used; or "a list of goods and materials held available in stock by a business" (Wikipedia). In software development, we only need to translate "good and materials" into "code and resources":

A list of code and resources held available in stock by a business.

Think of software inventory as unfinished code (as defined in Lean Software Development) or as overproduction of code. If we have unfinished code in our system, this is like having a physical component for a product - the component by itself is not useful until it is in the completed product. Similarly, the unfinished code provides no value to the software until it is finished. Even if the code is complete, if it is not being used in the software then it is dead code and is inventory for the software.

Inventory, in software development, carries a significant technical debt - the unknown factor of whether or not the code actually needs to be in the system. The longer we let inventory live in our code, the more likely we are to forget whether or not that code needs to be there. This inventory also becomes a potential mine field for developers - what happens if a developer sees this dead code and assumes that it is still functional and valid? If we are lucky, the code will work as advertised. The longer the code sits in inventory, though, the more likely it is to be outdated, invalid and eventually non-functioning.

Motion

In addition to physical movement causing waste through waiting (walking down the hall to the BA's office), motion can also be applied to the processes of producing the executable software (not the source code and other resources). How much effort does it take for your team to compile the resources, create the executables, package it into an installation and deliver it to QA, the customer, etc? The goal should be to make this as simple as possible - as little movement as possible.

Are you able to click a button and have the build, package, and delivery processes handled for you automatically? Or does it take a team of developers, build engineers and other persons with a manual process and a checklist of things to do? If your process involves people performing manual tasks - even if that task is simply the order in which multiple buttons get clicked - then you are at risk of creating waste through defects. Every manual motion in your build process is another point at which mistakes can be made, allowing defects to be built or causing the build to fail completely.

Software production (again - speaking about production of executable software, not development of resources or code) is often a difficult, tedious chore, requiring many moving parts - there is no way around this. However, it does not have to be a manual process. Continuous Integration and Continuous Deployment practices are abundant in the world of software development, specifically to address this issue. If you can automate your build process and eliminate the manual movement involved, you can eliminate a huge source of potential waste.

Overprocessing

Overprocessing, or too much process, is abundant in software development. When a customer has a request for a feature, how much process does it take for that feature to become functionality? In design-up-front project life cycles, the process of adding or changing a feature can be very time consuming, involving many steps on both the requester and implementer side:

request; acknowledgement and change management / approval; design and documentation by implementation staff (BA's, developers, etc); sign off of design and documentation by customer; implementation in software; scheduled for testing; tested; and if any issues are found, back to design or implementation for corrections, repeating several steps.

Additionally, how much work does it take for you to get your job done, as a developer? Do you spend as much time updating task / issue management tickets and managing your manager, as you do coding?  Too much process in software development is as bad, if not worse, than no process at all. Unfortunately, the failures of too much process are often seen as an indication of the need for more process, causing further delays and waste in the system.

There are often situations where you cannot do anything about the process that is in place. Life-Critical systems (x-ray machines, air plane guidance systems, etc) are abundant with process for good reason - if anything fails, people could be injured or killed. In the world of business software, though, this is not usually the case. Most business has a level of fault tolerance built into it, and some waste is expected / accepted. This is not a get out of jail free card, though. The more significant the waste, the more repercussions there are - fix the defect; pay for lost revenue; lose your job; etc. If we want to eliminate waste through the simplification of process, then we need to ensure that defects are found and fixed as early as possible, to avoid creating waste for the business using the software.

Process simplification can be done many different ways. Start with the creation of a process flow char, or value-stream chart. Find out how much time and effort is wasted in your processes and look for the waste that can be easily eliminated. Simplifying a single process by removal of waste can lead to a revolution in your process. You may find that the removal of one process negates the need for another process. You may also find that a simple change to one process does the same. If you are able to bring your customer into the work area with your team, and work in short iterations where defects are found and fixed quickly, you will likely see many of your old processes quickly become obsolete.

Conclusions

No one wants waste in their systems. Waste is money lost and revenue not earned. Waste management, though, can be a difficult proposition. Many company are willing to accept waste as the cost of doing business and are not willing to spend money eliminating waste because they do not see the waste or do not believe it can be eliminated. If we are truly going to revolutionize the business we work in, we must eliminate waste. Start where you have direct control or influence. You'll quickly find others wanting to know how you are so productive and soon the entire business will be in full waste reduction cycles.

I've been hearing this a lot lately, and finally started reading up on the roots of the Agile movement. It all goes back to Lean Manufacturing. Oddly enough, I've had exposure to Lean Manufacturing at a previous job. So the concepts and names are all very familiar to me. Back in those days, I had heard of the concept of Lean Software Development, but it never really made any impact on how I wrote software. These days, though, I'm diving into as much Agile as possible and the further down the road that I travel, the more and more parallels I see, to the lean manufacturing world.

If you are at all interested in the roots of agile, here's some recommend reading (mostly Wikipedia) to get you started.

Wikipedia:

• Lean Manufacturing
• Lean Systems
• Lean Software Development
• Toyota Production System
• Kanban

Books:

• The Toyota Way
• Lean Software Development: An Agile Toolkit
• (anyone have other suggestions?)

I'm hoping to absorb more of this information, quickly. It seems the more I learn about lean manufacturing, the more I understand about software development. I certainly don't claim to be an expert on either subject... hopefully I can change that.