In the manufacturing world, you would never find a company that assembles a bunch of parts into a final product before inspecting any of the individual parts, and they would not wait until the end of the assembly line to test for the quality of the product. The very notion of waiting until the product is “done”, to test it, would be appalling. How much time and money would be lost trying to figure out why something didn’t fit together properly, why it didn’t work, and why there was a quality issue with the final product? Rather, we see the manufacturing world taking an active role in preventing defects. Yes, they still do a final quality inspection, but the primary means of ensuring a quality product is delivered is not by waiting until the product is assembled to test it. They build quality in from the start and maintain that quality throughout the manufacturing process.

Prying Open The Case

Imagine the inner workings of the phone to the left. This is a very complex piece of technology. Do you think Cisco would wait until they have assembled this phone and then try to pry open the case so that they can insert a set of electrodes to test and see that the circuit board is connected correctly? I certainly hope they don’t. Instead, when a manufacturing company is building something – anything – they start with the idea of preventing defects, not waiting until they are identified and correcting them. Many companies have an active Zero Defects policy where defect prevention is paramount and quality inspection is almost just a verification of what they already know – that the product is defect free.

When a part is stamped out, formed, molded, or otherwise created, it is done so to an exacting specification. After the part has been created, the part is then tested against the same specification to which it was originally built. If the part does not fall within the tolerance and guidance of the specification, it is scrapped and a new one is made. If a series of parts are found to be out of specification, it’s usually a sign that something in the process, tooling, or other portion of the manufacturing process is not right. When this happens, they fix the cause of the problem – whether the machines need to be calibrated, the people running the machines need better instructions or whatever the cause is. In the end, the specifications for the parts were used to create the part, identify whether or not the part was up to standards, and decide whether or not to keep that part.

What’s more, the manufacturing company doesn’t wait until after they start creating parts to create the specification. Rather, they take the time to properly engineer the specifications up front. They take measurements, create prototypes with varying specifications to see what works best, record the success and failure rates of the various specifications that are tried, and use other design and engineering principles to scientifically calculate the exacting specifications that will be used to produce the parts. This occurs at all levels of the product’s design and creation. For every resistor, capacitor and microchip that builds a circuit board, each one of them has their own specifications that have been carefully engineered. If any single capacitor does not meet the specifications, it is not sent to Cisco with the hopes that it works anyways. Only when all of the specifications of each part are met will they solder the parts to the circuit board, creating a subassembly.

When a subassembly is created, it also has a specification to which it was built. The subassembly then undergoes the same quality assurance process – verification that it meets the specifications and operational requirements. The process continues from here – each subassembly gets connected to a larger system which is built to a set of specifications, with rigorous testing of the larger system as it is built, ensuring it meets the specifications. When the final phone is assembled, we don’t have to worry about whether or not a specific capacitor is soldered to the correct location – we don’t have pry open the case on this phone and insert a set of electrodes to see if the electrical current is flowing correctly. Instead, we only need to plug this phone into the correct connections (an Cisco IP phone system in this case) and verify that the phone actually performs all of it’s functions, according the functional specifications of the phone. There simply is no need to verify the capacitor that was used in the very first step. We know it works because it was built in a system that actively prevents defects.

The manufacturing world is obsessed with testing. They are willing to test from the lowest possible levels of the system, out to the end-product and the behavior that is expected. They do this because the consumers of manufactured products demand perfect. Why, then, are so many software development companies so willing to only test from one end of the process? To only test once, and only from the user interface, just before the product is shipped?

A Specification By Any Other Name

Unfortunately, the software development industry as a whole, is years behind the manufacturing industry. Our definition of quality and success are often skewed and we may consider fifty or more known bugs in a system of moderate to large size to be acceptable. It doesn’t have to be this way, though. We have the technical capabilities of following in the footsteps of the manufacturing industry, and we should.

I’m sure there would be no small number of people that would say we already employ the use of specifications in software development. After all, that’s what the requirements gathering phase is for, right? So many software development companies have put so much effort, time and money into the process of producing a piece of paper that can be understood by humans, and labeled this a specification. The problem we face with paper, though, is how to effectively verify the software against what the paper says. How do we verify that series of software lines and I/O statements that are understood by a computer have actually implemented the human readable text on the paper?

We are fortunate, actually. We don’t have to accept a Word document or a piece of paper as the specification to build to. We have the ability to create executable specifications! We can, and should, be creating specifications that can measure and verify our code. Most people call it test driven development (TDD). Some call it Behavior Driven Development (BDD). I like to think of it as Specification Driven Development (SDD? Not sure if that really exists. And really, do we need another xDD acronym?). We write code that exercises our code in the form of unit tests, integration tests, functional tests, acceptance tests, or whatever you want to call them.

We’re Not Just Stamping Out Parts

One of the major problems that I have with the manufacturing/software development analogy is the obvious statement that we don’t stamp out the same parts over and over again. In spite of my previous comments on this analogy, I now think that we are more analogous to a specific part of manufacturing than I had previously understood. A more accurate representation of software development in the manufacturing world is new product design and development. The parallels work quite well from this perspective. I am not going to expound on this in great detail at this point. It should suffice to say, for the moment, that the process of product development described in Wikipedia is a fairly accurate representation of what we go through for the average software development project.

When a manufacturing company is working on a new product, they once again don’t stamp out parts without knowing what they are doing. Many different parts may need to be tested, many different designs may need to be tried, but every one of these is still built to a specification. The major difference is that the specifications used are expected to change over time, until the final specification for the final pieces are accurate enough to produce a production-ready prototype.  The same notions can be applied to the software development processes in TDD, with some additional benefits.

Built To Specs, Regression Tests And Change

Change happens. It’s a simple fact of software development. A customer thought they wanted X, but in reality they needed X-1/B – not quite what we originally thought. When this happens, we once again have a significant benefit created by our executable specifications. We only need to identify those specifications that are now wrong, correct them, and change the affected portions of the system to match the new specifications.

Our ability to change is direct evidence to one of the many benefits of TDD: regression tests. Every specification that we write becomes a regression test the moment we fulfill that specification’s requirements. With this in mind, we can work with an almost reckless abandon, free to add features, remove features, fix bugs (because let’s face it – we’re still going to find some issues somewhere in the system) and refactor the system to a higher standard, all without worry of breaking the system. We can act with this level of confidence because we have a safety net in our regression tests. If (when) we do break something in our new efforts, we will be notified the moment we re-execute our specifications – that is, run our regression tests. A specification test will fail and we will have a clear indication of what failed and why. This deep insight into the system gives us even further confidence in correcting any issues that we introduce. When a failed specification test tells you exactly which value from which class is wrong, and the context in which that class was executed is known (the exact input and expected output), pinpointing the problem becomes a rote process. Fixing the issue becomes relatively simple, and we begin to see true productivity improvements in our processes.

Start With Quality, End With Quality

Our industry is currently suffering from a lack of quality. We ship horrendously bad user experiences in products that are late and well over budget, yet we call this a ‘success’. It doesn’t have to be this way. If we change our perspective and start to take some cues from the manufacturing and product design and development world, we can dramatically increase our effectiveness as software developers. We can create high quality, low cost solutions like the world expects from manufacturers. Built to specification is certainly not a silver bullet. It is, however, the definition of quality in a Zero Defect environment.

By employing a built to specification mindset in our software development efforts, we can start with quality and maintain that quality throughout the life of our projects. This is the same process that is undertaken when a manufacturing company is working on new product design and development. It works well, it’s a proven process, and most of all – it makes sense. Build your software to specifications. Just make sure they are executable specifications.

I recently saw this question asked and answered in an after-project retrospective paper.

Was The Project A Technical Success?

While I am not trying to address who asked the question or what the answer was, the question on it’s on own, is flawed in my mind. Or, at least when taken out of context, it has become somewhat of a fallacy by standing on its own.

Not ‘Technical’, But ‘Operational’

The question we should be asking first is not about ‘technical’ success, but ‘operational’ or ‘business solutions’ success:

Was The Project An Operational Success?

That is to ask, did we effectively reduce or solve the stated business problems to an acceptable level, allowing the users of the application – the operational people - to be more accurate, efficient, productive, responsive, … etc, in their job? Only when we can answer “yes” to the project being an operational success, and provide sufficient evidence of this success, should we ask if the project was a technical success.

If we created the world’s most perfect architecture, design and implementation, but the project was a DOA failure because it did not solve or sufficiently reduce any stated business problems, resulting in no one to buy it or use it; what good would it be to have the perfect system from a technical standpoint?

The counter-perspective, though, is just as important. If we have managed to solve or sufficiently reduce the stated problems then we need to ask if we did so in a manner that allows us to maintain this system for the long-term life of it. Were we successful in our technical implementation of the business solution, thereby allowing us to continue solving additional / ancillary business problems, easily and efficiently?

Two Sides Of The Same Coin

We can’t sacrifice the needs of the business to create what we believe is a successful technical implementation. Yet, we can’t sacrifice the technical implementation just to satisfy the needs of the business. There is a marriage between the two – they are (or should be) two sides of the same coin. If we fail on either aspect and are not able to recover quickly, we will eventually fail as a project / product team.

Manage up. Lead out and down.

You ARE A Manager

If you work FOR anyone, in any context, you’re a manager. You mange your boss’ expectations of when you will have something completed. You manage your customer’s expectations of what the software will look and behave like, how it will perform, and when it will be ready for testing. You manage your own time and resources to make sure that you can get things done, prioritizing based on many factors (often external factors and influence). The point is – you ARE a manager. Period. All of you. End of discussion.

You SHOULD BE A Leader

If you work WITH anyone, in any context, you should be a leader. You should never manage those you work with or those that report to you in any capacity. You should lead them by enabling them and empowering them to get their job done. You don’t manage your peers – you lead them. You influence them by showing them new and better ways of working, discussing the pros and cons and coming to conclusions that benefit everyone involved. You lead your subordinates the same way, but with the option of overriding decisions for business and/or quality and technical reasons. You lead your subordinates by blocking the stupid and detrimental from them. You lead your subordinates by letting them get their job done while you head off the office politics.

Manager And/Or Leader?

Are you a manager? Yes. Are you a leader? You should be.

Manage up. Lead out and down.

You ARE A Manager

If you work FOR anyone, in any context, you’re a manager. You mange your boss’ expectations of when you will have something completed. You manage your customer’s expectations of what the software will look and behave like, how it will perform, and when it will be ready for testing. You manage your own time and resources to make sure that you can get things done, prioritizing based on many factors (often external factors and influence). The point is – you ARE a manager. Period. All of you. End of discussion.

You SHOULD BE A Leader

If you work WITH anyone, in any context, you should be a leader. You should never manage those you work with or those that report to you in any capacity. You should lead them by enabling them and empowering them to get their job done. You don’t manage your peers – you lead them. You influence them by showing them new and better ways of working, discussing the pros and cons and coming to conclusions that benefit everyone involved. You lead your subordinates the same way, but with the option of overriding decisions for business and/or quality and technical reasons. You lead your subordinates by blocking the stupid and detrimental from them. You lead your subordinates by letting them get their job done while you head off the office politics.

Manager Or Leader?

Are you a manager? Yes. Are you a leader? You should be.

For so long there have been so many advocating the benefits of the various Agile, Lean, Iterative, or whatever-you-want-to-call-it-these-days methodologies. We, as software developers, seem to understand the benefits of these methods. So why, then, do we still have customers that insist on big design up front, waterfall based methodologies, and other practices that we know are wrong?

Some Ad-hoc Consulting

During my Christmas vacation, I had the opportunity to do some consulting / coaching for my dad’s current company. The employee list that I spoke to consisted of my dad, my aunt, and my cousin. It was a nice little family gathering, I got paid with some amazingly good barbeque, and it was a very enjoyable experience.

I’m a software developer working for a company that provides software development and related services. My dad’s company is a customer of a software development organization (not the one I work for), receiving some custom built software for a project. They have been running into some issues with quality and timeliness of delivery for the last year or so and were wanting to ask some very pointed questions about the state of the software development industry. Essentially, they wanted me to tell them whether or not the issues they were having are “normal” and / or “acceptable” when receiving custom software development services. My answer was an unfortunate “yes, this is normal”, but “no, this is not acceptable”.

Over the next few hours of our discussion, we touched on many different issues that they are having and how some of those issues can be addressed from their perspective and from the perspective of the software development supplier. In the end, I had successfully educated my dad’s company on a few key points, confirming their suspicions and observations, and giving some new directions to start heading with the company that is providing the development services. I can’t go into all of the detail, but the summary bullet points included the following:

• The customer won’t know what they want until they see something they don’t want
• The software provider should deliver a working, stable version of the system on a regular basis, allowing the customer to use it in it’s current form
• The customer sets the priority of features and functionality to be implementing, in coordination with the technical needs and pre-requisites of the software development organization
• The customer is allowed to say something is wrong, even when they receive what they asked for
• The relationship between the customer and service provider must be a whole-team view. That is, “we are one team. we succeed together and we fail together”

On top of these bullet points, I laid out a simplified Scrum process for them, based on the already stated 3 week development cycle that the provider prefers.

In the end, my dad’s company was shocked, astounded, amazed, and energized by everything they heard. The light bulbs were clicking in their eyes, and I could see the value that they received form the conversations. They have dramatically changed their understanding of what it means to be a good customer and what it means to be a good supplier.

A New Perspective

As much as I believe I helped my dad’s company in their understanding of what they should expect and how they should approach their issues and needs, I also learned a great deal from this experience. It was very enlightening for me to hear the issues that we have known for so long as software providers, but from the customer’s perspective. Nothing that they spoke of was unknown or unfamiliar to me. I have lived through all of the same pains and issues that they are going through – from both a software provider and a customer of software services.

What really stuck with me from this experience, though, is the idea that our perspective on how to improve our industry is likely wrong. Yes, we need to continue educating our own ranks on the benefits of better software development techniques and processes. However, it does no good for us to have the perfect pneumatic toolset when the customer is asking us to use a rusted hammer and a broken hacksaw.

What I’m getting at is that its not us, the software providers, that need to change so greatly. Rather, our customer base needs to be educated and change. We need to stop being so self-centered and introspective for a while so that we can educate our customer base on what it means to be a good customer; how they can accomplish their goal of receiving quality software that provides true business value for them, at an affordable price. Until we have a customer base that expects software to be high quality, the first time, every time; expects to be given working software on a regular basis; and expects to be able to recognize that what they thought they wanted is not what they really needed; we will have no chance of our inward looking self-improvements taking hold.

I have to say, the Agile Manifesto got it right from a values perspective. The values espoused there are what we need to be looking toward, but I think our perspective needs to be changed on how we enable those values to be reached.

Of course, if we do succeed in educating our customers, we need to be able to respond appropriately. So please, continue the education of our own industry. Create better development standards that increase our quality and productivity. If our customer expects a quality delivery every 2 weeks, we better be able to deliver a quality product every 2 weeks. Just don’t stop here. Educate your customers. Help them understand the financial impact of poor development standards and processes. Change the norms and improve our industry by creating a customer base that requires perfect. The manufacturing industry did it 20 to 30 years ago. It’s time for us to step up to that level.

Where Do We Go From Here?

So where do we go from here? How do we start educating our customers? How do we change the social norms of software development so that our customers expect us to behave more like professional craftsmen and engineers, delivering quality faster?

I don’t claim to have the answers and I don’t claim to be the first to encourage this change. I only hope that we as an industry will begin to understand that the customer is usually the one that sets the expectation for how often software is delivered, and that we need the customer to expect something better.

Imagine that there are three group of people going out for a hike. Within each group of people, we have a goal of everyone reaching a picnic table at the end of the hike (and no one can eat until everyone has arrived). One of the people in each group has a map and a compass, knows how to use them, and will be directing the group on where they need to be going (we’ll call that person “the Pathfinder”). Everyone else is just along for the hike and the picnic.

The Pathfinder in each group is given a specific set of instructions on how they are going to direct everyone else.

Group 1: Observe, Plan, Walk

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course in one hundred foot increments, with your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking. Give an updated direction to walk every one hundred feet, enduring the group can go around any obstacles such as large rocks, impassable tree groupings, and small creeks.
4. Once you have plotted the correct course, store the compass and map in your backpack with the understanding that you are not allowed to use them again, until you have reached the picnic area.
5. Have the entire group read the instructions, then set out on your hike.
6. You are only allowed to check that everyone is on course by re-reading the instructions that you wrote down, every 100 feet – when the team needs to change direction to avoid an obstacle.
7. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 2: Observe, Plan, Walk, Start Over, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet, ensuring everyone in the group understands the directions.
4. Place your compass and map on the rock, then hop down from the rock. Re-join your group and lead them in the right direction for one hundred feet
5. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
6. Travel the one hundred foot distance with your team, ensuring that they end up in the correct location.
7. At the end of the one hundred foot walk, have the group stop where they are.
8. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
9. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
10. With the help of the ropes to guide you, walk back to the rock at the beginning of the trails, by yourself, and climb back on to it.
11. Plot the next one hundred foot leg of the journey from where the group currently is, using your map and compass.
12. Repeat this process from step 3 through 11, until your entire group has reached the picnic area.
13. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 3: Observe, Plan, Walk, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area and general understanding of how to get where you are going, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet.
4. Place your compass and map in your backpack, then join your group.
5. Let the group know where they are heading (the end goal) and where they need to be in the next one hundred feet, ensuring everyone in the group understands the directions that you outlined.
6. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
7. Lead the group in the right direction for one hundred feet, ensuring that everyone is able to safely navigate any obstacles.
8. Verify that that your group has stopped in the correct location, by using your map and compass and any other landmarks or means needed.
9. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
10. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
11. Repeat this process from step 3 through 10, until your entire group has reached the picnic area.
12. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Predictions on Outcome?

• Which group do you think will make it to the picnic site first?
• Which group do you think will be the most happy with the way the group was organized and run?
• Which group of people do you think would be most likely to do the same process on another hike?

Observations of the Groups

Group 1 didn’t get to the picnic site very fast and wasn’t very happy about the experience. While everyone in the group was standing around waiting, the Pathfinder was sitting on top of a rock, writing things down in a notebook, trying to describe – from a very high level, where people were supposed to be walking. When the Pathfinder finally came down from the rock – possibly an hour or more after he climbed up there – the initial directions were good and people could start finding their way. As time went on, though, the detail of where they should be going became more and more unclear, because the Pathfinder could not nee the actual pathways below the tree tops and behind the large rocks. This caused Group 1 to constantly stumble and struggle, creating a situation where the team had to find paths around obstacles that they couldn’t previously see. Every now and then, they got lost because the description of the landmark was insufficient from the distance that it was recorded. They eventually got to the picnic site – well after everyone was hungry and tired. In the end, no one was happy with the experience and just wanted to go home.

Group 2 had an easier time finding their way. Since the Pathfinder and the group were able to see all of the obstacles within one hundred feet, the directions that the Pathfinder described was much more enjoyable. No one stumbled over anything or got lost. But it was a difficult journey, at times. The group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. The problem, though, is that the group took forever to get to the picnic area. At first, having the Pathfinder re-check the direction they were heading was great. Everyone knew that he was keeping the group on target. However, every time the group reached their next stopping point, the Pathfinder would gather information about the next hundred feet, across a wide arc since he didn’t know which direction to go next, with great certainty. Only then could he travel all the way back to the original rock to verify that everyone was on the right path, heading toward the picnic table. After a few iterations of hiking, the process of verifying the course and location took more time than the process of hiking. The Pathfinder was simply repeating the same hike over and over again, to verify that they were on the original path laid out. No one in this group was disappointed in the path that they took to get to the picnic table, but no one appreciated the standing around and waiting for hours on end – waiting for the Pathfinder to get the next one hundred feet plotted. When they finally reached the picnic table, group 2 was full of hungry, grumpy people who just wanted to go home.

Group 3 got to the picnic site in record time. It was a difficult journey, at times – the group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. With the ability to see all of the obstacles in the surrounding area, and the ability for the group to make decisions that would get them around obstacles while still making progress toward the picnic table, the group was successful in getting to the picnic table before lunchtime. Everyone on the group loved the trip and appreciated the ability have input in how they got to the picnic table. As long as the direction that they headed for the next one hundred feet did not cause a serious delay in reaching the picnic table, or was justifiable by avoid an obstacle, all input was considered before setting the new direction. Several of the people in group 3 decided that the would take a few more hours and head off into the next set of hiking trails, trying to reach another picnic table before dinner time. Other decided to bring their family out to the trail the next day, and re-walk the well defined path (noted by the ropes tied to the starting and ending points of each leg of the journey). Overall, everyone in group 3 enjoyed the experience.

The Real World

Now imagine that the Pathfinder is represented by some aspect of your organization. It may be the company or project leadership; it may be the customer; it may be an individual person or group of people within a project team. Your Pathfinder may not be a person or other sentient being, though. Perhaps its the Methodology (capital M, as defined in Peopleware) or political environment in your company.

Which Pathfinder(s) do you have in your company? Which Pathfinder would you rather work with? More importantly – if you are not working for / with the Pathfinder that you want, what steps are you taking to correct that? Be a catalyst for change – appropriately and professionally.

Continuously improve – always within the context of the goal.

My coworkers really do love me. This was their Christmas present for me, complete with shiny bow.

I was told to use it only for good, not for evil.

And yes, that is me preaching from my soap box.

Let's assume that we are doing the appropriate amount of testing during our development process. If we include TDD, test automation, test engineers and customer acceptance testing, we should find the majority of the bugs in our system before they are released. However, not every bug will be found. There will be some situation that no one thought about before. There will be some special circumstance on someone's computer that hasn't been accounted for. There will be some client data that does fit the expected variance, despite the data being valid. The point is, there will be something that breaks after we deliver the software. Worse yet - it doesn't even take delivery to find bugs. What happens when the software gets to customer acceptance and the customer says that something is wrong, broken or whatever? We simply must account for the inevitable bug fixes and emergency patches in our system.

Our current kanban board, with all it's pipelines and queues, hasn't addressed the need to address problems. But before we get into any changes to the board, we have to define and create the culture of quality that we need in our team.

A Culture Of Quality

There are many different aspects of quality and many different ways to view and interpret quality. The individual specifics, as always, come down to your team, the project and the needs of both. Even with these variances, though, there are a few key concepts that should be found in any culture of quality, including Whole Team / Collective Ownership and Zero Defects.

Collective Ownership

I've talked about this concept before (see above link) and would recommend reading those prior posts. One thing I would like to add, though is that collective ownership can only success when the team takes their individual egos out of the equation. We must be able to accept criticism as a way to improve ourselves and our team. When we let go of our ego, we can be honest with ourselves and others, enabling everyone to own every part of the system.

Zero Defects

This is a subject that I have not talked about before, but is prevalent throughout the continuous improvement philosophies that I subscribe to. Wikipedia lists four principles of Zero Defect methodologies, all of which are paramount to our culture of quality.

1. Quality is conformance to requirements
2. Defect prevention is preferable to quality inspection and correction
3. Zero Defects is the quality standard
4. Quality is measured in monetary terms – the Price of Nonconformance

Don't be confused by "requirements" in item number one, though. In the world of software development, the word "requirements" has many meanings - not simply the feature list or functional descriptions set forth by our customers. Our teams and processes have certain requirements that are imposed on top of, and into each of the business requirements for the software. These often include the use of source control systems, test driven development, principles like S.O.L.I.D., etc.

Quality Checks in Kanban

Collective ownership and zero defects are only the tip of the iceberg. There is so much more to a culture of quality and so many different aspects of quality to account for. However we define quality, though, our ability to create a culture of quality is necessary for our kanban system to work. Without it, we lose the continuous improvement and elimination of waste that defines Lean. With it, though, we can introduce some specific tools to our process and improve our kanban system by having it handle the errors that are found in our systems.

Andon

From Wikipedia:

"a system to notify management, maintenance, and other workers of a quality or process problem"

When an issue is found in our system, we need to notify the team immediately so that the problem can be taken care of. The idea in a manufacturing line is to allow a worker to stop the line when a problem is found. In our grocery store example, andon could be invoked by a shelf stocker noticing a loose nut or bolt on the shelf and then notifying a maintenance person. In software development, andon can take many different forms. We report our issues list during our daily standup. We create issue tickets in our issue management system. If we're lucky and work in a company that supports the idea of a team room, we may just need to look up or turn around and let the team know that we have problems. We could even place an actual red flag on the desk of our developers, testers, customer representatives, etc., and have them raise that flag when they see a problem. Any of these ideas, plus many many more, can all be our andon system.

Andon itself is not intended to be an all encompassing electronic system with metrics and reports and yadda yadda yadda. It needs to be simple. It needs to be easily employed by anyone on the team. And it needs to mean something to the team. If a team member throws their andon card out on the table, the culture of that team needs to be ingrained with the knowledge that work may stop until the problem is addressed.

There's so little to andon, yet so much more than what I've described. However andon is enabled, it is critical to our zero defect policy in software development.

Jidoka

AKA "Autonomation", AKA "Automation with a human touch", AKA "intelligent automation", AKA ...

From Wikepedia:

"Autonomation prevents the production of defective products, eliminates overproduction and focuses attention on understanding the problem and ensuring that it never recurs."

I've talked about Jidoka in previous posts and would recommend reading them, at this point.

• Jidoka - If it's broke, fix it now
• Jidoka - More than just 'if it's broke, fix it now" - fix it and inform the team

I don't have much else to add, other than to say that Jidoka and Andon go hand in hand. Automated build servers (such as CCNet, TeamCity and many others) can often combine Andon and Jidoka for us by giving us instant visual feedback when something is broken. I've also recently set up BigVisibleCruise in my team area, giving even the casual observer the knowledge of whether our builds are broken or not.

Applying Andon and Jidoka to Our Kanban Board

Once we have the concepts of Andon and Jidoka in our team and culture, we can use these tools to generate issue cards and then look at a few possible changes to our kanban board to account for them. The three basic methods that I have seen used include:

1. Creating an Emergency Fixes pipeline
2. Tacking a smaller bug notice onto an existing card
3. Putting a Bug card in the backlog

I'm sure there are other alternatives, too. In my current team, we use the first and third method and are considering the second one as well. 

Options two and three essentially require no change to our kanban board. Implementing option two is a distinctive way of visually attaching a bug notice to one of the cards in our system. This could be done with little bug stickers, little red cards, or any other visual indicator that the team agrees on. Option three also needs something distinctive. Since we are creating an entire card for the bug, though, the entire card should be distinctive. I would recommend using a card that is colored red to signify issues. I would also recommend prioritizing the bug to the top of the backlog queue, when using option three. This will ensure that the bug gets worked as quickly as possible.

Option number one can also make use of number two and/or three. When we move a card into the Emergency Fixes pipeline, we may want it to be distinguishable as an issue by tacking on our bug symbol or by using a colored card.

Emergency Fixes Pipeline

Depending on the complexity or severity of the bug, we may want to include Analysis and Customer Acceptance in our Emergency Fixes. With andon and jidoka in mind, we will want to ensure that we fix any emergency issue immediately. This is not always possible, however. The customer may decide to delay the fixing of an issue for whatever reason. This leads us to only needing a single pipeline for Emergency Fixes, letting us set our limit to one.

We can easily add an Emergency Fixes pipeline to our kanban system, placing it directly underneath our existing WIP pipeline. This special pipeline can be designated with a name, color code, or other marks as needed.

If you are supporting production releases, you may also need to include a Delivery queue specifically for emergency fixes. This would allow multiple fixes to be compiled into a single patch release. There are other configurations to this, of course. As always, you will need to find what your specific team needs and create your process to suit.

Where Do We Go From Here?

With an Emergency Fixes pipeline in place, our kanban system is now set up to handle just about every situation that we will encounter. However, this does not mean that our system is perfect or truly complete. No process, no matter how well defined it is, is worth anything if the people running the process do not believe in it. I've said it before and I'll continue to say it - never stop improving your process. Always be mindful of waste, friction, smells, problems or whatever you want to call it. Inspect, adapt and continuously improve your process. Perfection is a journey, not an end-goal.

Stay tuned to my Adventures in Lean series, as I continue to explore the various aspects of lean software development in my own team and company culture.

In part 2 of my Kanban in Software Development series, I talked about completing a kanban board with queues, order points and limits. We saw how to take a complete development pipeline and work with a team, its processes and its bottlenecks. In the end, we had a kanban board that could easily represent the processes of the fictional team that we outlined.

One of the questions that I've often asked about a kanban board is how anyone would know when work in one column is done and ready to be pulled into the next column. For example - if a kanban card is sitting in the Analysis column, how does a developer know when that card is done so that they can pull it into Development and start coding it? I found the answer to this question when I was at the Kaizen Conference in October. Jef Newsom did a workshop on kanban and we ended up with this same question, and a solution.

A Pipeline for Analysis - WIP and Done

To facilitate the visualization of the difference between work in progress and work that is ready to be pulled to the next column, we can use the concept of a pipeline and split our existing queues into a WIP and Done step. For example, we want developers to pull work from the Analysis queue into the Development pipeline. We can show which cards are ready to move by splitting Analysis into sub-columns of WIP and Done.

When an Analyst is ready for work, they would pull from the Backlog into the Analysis / WIP column. When the analysis work is done and the card is ready to go into development, the card would be moved into the Analysis / Done column. Since we are wanting to maintain the concept of a queue for the overall Analysis column, we have create the WIP and Done subdivisions as a pipeline (noted by the dashed line). This allows us to keep our order point (3) and limit (5) in place for Analysis, and know what work is ready to be pulled into Development.

Applying Pipeline per Queue Across The Board

Not every queue needs to be a pipeline.

Consider the Backlog - the customer is placing the prioritized list of features in this column. The cards in this column exist so that the analysts will know what work needs to be done - not because any work needs to be done in this column, explicitly.

The Delivery column may not need a pipeline, either. If the delivery process is composed of sending an installer package to the customer, then there is no real work to be done in this column aside from sending that package. However, if there is some specific integration work (say, changing a web.config file) that needs to be done, we could include a WIP and Done pipeline for delivery. For this simple example, we'll say that there is no configuration change needed. Let's assume that the installation package changes all the needed configuration files based on user input.

With all of this in mind, we can apply our WIP and Done pipeline to the Analysis and Customer Acceptance columns.

Alternative Visualizations

There are other ways of visualizing WIP vs Done cards for a given column, of course. For example, you could forgo the pipeline idea and simply add a queue between Analysis and Development. This may require you to adjust your order points and limits between analysis and development - at least, where those order points and limits apply (Analysis vs. the queue between Analysis and Development). You could also split the column horizontally instead of vertically, creating either the top or bottom half as the WIP with the other being Done. Or you could just note the status on the kanban card itself. I'm sure there are a number of other ways to show this information, as well. The point is that the pipeline visualization that I'm showing here is not the only way. Find what works for you.

Where Do We Go From Here?

We have another tool in our belt, now. If need be, we can visualize the WIP vs work Done for a given queue, allowing us to see when we can pull work forward. But, like all tools in our belt, it is not *the* answer and this additional visualization may not be necessary for your team.

If your team is small or has great communication, you may not need any special distinctions to show work in progress vs work ready to be pulled. You could rely on the daily standup for people to report when they are done, or do it ad-hoc - when someone finished a card, they announce it to the team immediately. In a larger team, though, it may become necessary to have some visual distinction between work in progress and work completed for a given column.

Small teams might get away without it; larger teams might need it; its up to your team to decide. The key is not to apply these ideas as blanket rules that must be followed, but to allow the individual project and team to decide what is right for them. And as always, allow the process to change when it needs to. Always inspect, adapt and continuously improve.

In Part 1 of Kanban in Software Development, I introduced the concepts of kanban boards and pipelines. I also showed a very simple example of creating a pipeline for our development process. However, there were some obvious limitations in what I showed. The reality of software development is much more complicated than three simple steps (Analysis, Development and Testing). A software development team or company is going to have more to do than just these things, and there is usually a need for some team members to be cross-functional. Some team members will have to do development and testing, or analysis and development, or documentation and delivery, or any other combination of steps involved in creating software. In this post, I'll address these issues, creating a more complete pipeline and finishing our kanban board.

Working In A More Complete Pipeline

To create a more complete kanban board, we need more than just a three step pipeline. We need to allow for a fully functional team - developers, analysts, testers, technical writers, and others. We also need to allow the different team members to work on different parts of the system, as work is needed. The end goal is to enable the system to flow through the process and to ensure the work is "done done" before it goes to the customers.

For a more complete software development pipeline, let's use the following columns: Backlog, Analysis, Development, Documentation, Testing, Customer Acceptance, and Delivery. We can put together a pipeline diagram that follows these steps.

Now, let's assume that we have the following team (ignoring project managers, customer representatives, etc, for now): 6 developers, 2 business analysts, 3 test automation engineers, and 3 technical writers. Given this team, we would not be able to make a feature flow through this pipeline. We would have developers sitting around, waiting for work from the analysts. And, our documentation writers and test lab people would probably be pulling his hair out from boredom then pulling their hair out from too much work, in an unbreakable cycle.

Fortunately, we can account for the team makeup and the potential bottlenecks by introducing the concept of order points and limits that we saw in the grocery store, and by creating queues and multiple pipelines to be worked.

The Real World - Multiple Pipelines and Limits

In most software development efforts, it's unreasonable to expect an entire team to work on only one feature at a time. Most development managers want to maximize the throughput of development by working on tasks in parallel. Given this desire, we'll divide up our developers into three teams of two developers each. This development team division should allow three features to be developed at the same time. Since we want to work on three features at a time, we will need three pipelines for work.

These three pipelines constitute our first limit - we can have a maximum of three features in development at any given time. This limit is noted by putting a "3" in the upper right hand corner of the development column header, as shown above.

Having three development pipelines puts us in a tricky situation, though. We have very few business analysts compared to developers - even with our developer teams, we have less than one analyst per developer team. Fortunately, development work often takes longer than analysis. We can use this knowledge to our advantage and let the analysts work slightly ahead of the developers, creating a queue of work to be done.

Handling Bottlenecks - Queues, Limits and Order Points

If all three development teams happen to finish at the same time and need more work, we would need a minimum of three features that are ready to be worked. However, since we know that requirements change regularly, we don't want to get too far ahead of the developers. With this in mind, we can introduce an Analysis order point of three (the number of development teams) and limit of five. This gives the analysts the ability to work ahead of the developers and also makes them responsible for keeping work available for developers.

For an order point we'll introduce the number into the upper left hand corner of a column header, and we'll continue using the upper right hand corner to specify our limits.

Since we are now dealing with potentially more work in analysis than development, it seems we would have to increase our pipelines to five. This is not desirable, though, since we only have three development teams. What we will do, instead, is restructure the pipeline and turn the analysis column into a queue.

Since Analysis is now a queue, there does not seems to be a need for Backlog to be a step in a pipeline, either. The backlog is simply the list of features that the customer is expecting to be done next. Therefore, we will also turn the backlog into a queue - this time, with a priority (top of the board is highest priority) allowing the customer to tell us which specific feature should be worked on next. We will also want to keep the backlog column limited, to prevent the team from having too much information to think about. Since we only have two analysts on our team, it seems appropriate to keep at least two features in the backlog column at all times. With this in mind, we can safely set a backlog order point of two and limit of five (enough to keep the analysis column full).

The changing of Backlog into a queue has no made our Development column the first step in our actual pipeline.

Completing The Pipeline - Aggregate Limits and Work Cells

The next few column - Documentation and Testing - both have an easy amount of team members to deal with. There are three technical writers on our team, and three testing personnel. This distribution lets us keep the pipeline in tact between Development, Documentation and Testing, allowing us to set the same limit as Development (three).

With all three of these columns having the same limits, and with each column being properly staffed so that three pipelines could run at the same time, it makes sense to combine the team members from these three columns into a single workcell. If we consider all four team members (2 developers, 1 tech writer, 1 tester) as a team and allow them to work together as a team, we can aggregate the limits of the columns in the pipeline. We can also create a consolidated name for this pipeline - work in progress.

This consolidation can be shown by creating a parent / child header with the limit shown in the parent. We will also use a dashed lines between the columns of the pipelines. The combination of these two details will show us that we are dealing with a pipeline, and how many pipelines are allowed to flow, simultaneously.

By aggregating the pipeline limits, we allow the workcell for a given feature to focus exclusively on that feature. The development staff, technical writers, and test lab personnel will all be working on the same feature, allowing them to more easily share information between themselves. This will prevent the team members from having to switch back and forth between subjects, reducing cognitive load and allowing for greater quality to be attained in the individual feature.

Completing The Kanban Board - Customer Acceptance and Delivery

If the customer is on-site (co-located with the team or close enough to meet every day), then it may be possible to aggregate the Customer Acceptance process into the Work In Progress pipeline. If you can do this, you should. Getting immediate feedback from the customer, while work is still fresh in you mind, is critical to the quality and success of that work. What's more - you may not need this explicit column if you have the customer working with you every day.

Unfortunately, we don't always have the luxury of on-site customers. We may only have the customer around at specific timer intervals or only on request. In either of these cases, we have to account for the bottleneck of Customer Acceptance.  Additionally, the customer may not want to get delivery of individual features. They may want to wait for a quarterly release, or some other regularly scheduled release. Once again, we have to account for this bottleneck. We can apply the same principles that we used for Backlog and Analysis here, and create some appropriate order points limits.

Let's say that our customer wants to do Customer Acceptance testing once a week and wants a Delivery once every calendar quarter. To accommodate this, we can set the Customer Acceptance column as a queue with a 1 week limit. We can also set the Delivery column as a queue with a 1 quarter limit. Setting these limits puts a direct responsibility on the customer. If they cannot pull the features through these columns on or before the specified limits, then the entire system could grind to a halt. It's important to educate the customer in what they are committing to, and ensure that they can fulfill their obligations. This fulfillment may mean that they assign more than one person or group of people to the task of Customer Acceptance and accepting Delivery. Whatever the solution is, the customer has responsibilities to meet.

With Customer Acceptance and Delivery specified, our completed kanban board would look like this:

Put It In Action - A Kanban Process Is Never Truly "Complete"

Now that we have clearly defined our Kanban process and visualized it through our Kanban board, the next steps is to start using it! Start working through this process, pull value from the end of the process and allow work to flow through the pipeline. Respect the order points and limits, and above all, look for problems in the process and fix the process when you need to.

Perhaps you have too few or too many items in one of your queues - adjust the limits and order points. Perhaps the test lab is now a bottleneck - take them out of the pipeline and change them to a queue. There are many possible problems and many possible solutions. The key is to be aware - inspect and adapt and hold regular retrospectives - kaizen your process and continuously improve.

Where Do We Go From Here?

No process is perfect. Anyone that tells you otherwise is selling something. A customer may find a bug during Customer Acceptance; an issue makes it past all of our QA processes and is found out in production; or there's a feature that has a sudden priority over any other feature currently being worked. But our current kanban process doesn't address the natural problems that occur during a software package's lifetime. To address this properly, we'll need to employ the lean tools of Andon and Jidoka, and we'll also need to decide how to visualize this on our kanban board. These issues, and possibly more, will be addressed in another installment of Kanban in Software Development.