A Food Network Challenge

During this season's 'The Next Food Network Star', there was an episode where the contestants had to cook a dish and review it in front of a camera, describing it's visuals and taste. The twist to the challenge was that they were assigned someone else's dish to review, after everyone had already cooked. Once the assignments were made for who was going to describe who's dish, they were not allowed to see the dishes anymore. A given contestant was then put in front of a camera and had their assigned dish uncovered in front of them. From that point, they had 90 seconds to describe the dish that was in front of them - that included figuring out what it was, what it looks like, what it tasted like and what it smelled like. This was a rather tough challenge to watch, and a few of them didn't do so well.

A Code Review Challenge

What if we take the Next Food Network Star's challenge and apply it to code reviews? When we sit down in a code review session, rather than having the person who wrote the code doing the driving and telling us what is going on, have someone who has not seen the code do the driving - read the code, read the unit tests, etc - and describe to us what is going on. 90 seconds wouldn't be nearly enough time, though, so we'd have to change that time limit based on the size and complexity of the functionality being reviewed. Other than that, though, it seems like it would be a great way to test the team's ability to pick up an area of the system that they have not seen before and quickly learn it so that they can work in it.

Standardized Work - Judging Code And Code Reviewer

There are 2 real benefits to the code review challenge, as I see it:

1. Judging how well did the coder apply our coding and development standards
2. Judging how well the reviewer can recognize and describe the standards being applied

We are now building our system from User Stories, with Context / Specification style unit testing. One of the intents of this style of unit testing is to provide an easy-to-read and easy-to-learn set of examples for developers that need to know how to do something or how something works in the system. If the unit tests are properly written and expressive of the code's intent - it should be really easy to see what's going on. If we have good standards in place, a developer should be able to find the code in well organized areas, easily, and know where to look for the various components and parts.

It seems to me that the code review challenge would help to prove a number of these key factors in our projects and our teams:

• Coding and architecture standards
  • How well the standards are known
  • How well the team follows the standards
• Expressive Code
  • How easy is it to read the code
  • How easy is it to understand the code's intent
• Maintainable Code
  • How easily can the code be changed and extended
  • How quickly the team can socialize and collectively own the system

A Challenge In Name, A Learning Experience In Practice

I would avoid applying any sort of point system or other reward / incentive for the code reviews. It seems that this could easily backfire and cause junior team members (in skill or time on the team/project) to feel intimidated or feel that it's an unfair process comparing them to the senior level team members. The intention of the code review challenge is not to cause dismay in any team member, but to create a learning environment by continuously challenging every member of the team. Ultimately, the code review challenge should foster the socialization of the system and lead to a strengthened sense of collective ownership, resulting in a better system.

Nothing New Under The Sun?

With all that being said - I'm doubtful that I'm the first person to ever hold a code review like this. I'd be interested in knowing if there's any standard development practices (Agile or otherwise) that are heading in the same direction. Any advice or opinions of you, the reader, would also be very welcomed.

If you know of a resource that describes the same basic ideas or if you have any input to help me improve the idea, please leave a comment and let me know.

My team's first swim lanes!

From left to right: Backlog, In Dev, In Test, Docs, Done.

sweeeeet.

Scott Bellware (who finally started blogging again), is posting a series on Sustaining Capacity in Maturing Agile Software Teams. It's worth a read, and may be familiar territory to anyone who has done Agile and been reading about Lean.

In the 3rd Post - Recognizing Entropy - he talks specifically about synchronizing the development testing and QA testing efforts:

Development and QA Test Synchronization. Without greater synchronization between development and QA testing, valuable input often comes too late in a development or release cycle to be effective. Test design and test architecture are valuable inputs to development, and software design and architecture are valuable inputs to QA testing. Teams often loose unrealized capacity by not pursuing means to do more development and testing in parallel.

This one really hit home for me, and is a shining beacon to the problems that my current team is going to face, soon. We had our first iteration planning and kickoff meeting yesterday, and the whole team was involved - except the testers from the QA department. For reasons that I have no control over (and honestly - they are legitimate business reasons), the QA department is not coming on to the project for another 2 months (at least).  Today, the team leadership was discussing how the QA personnel would create their test scenarios, and I brought up the point that creating the test scenarios after-the-fact (after the iterations are done) is going to be very problematic.

If a dev team is working from User Stories, and those User Stories are divided along various aspects of behavior, it is very likely that a single screen, workflow, or 'function' of a system is going to be divided into multiple user stories. When the QA team is involved with testing in parallel with the development team, this is not an issue because the QA team will be updating their test scenarios in each iteration. This allows them to keep the scenarios for a given screen, workflow, or 'function' of the system, growing over time. However, since our QA team is going to come into the project at least 2 months after the development team has started coding and testing, the QA team would have to sift through all of the current and previous iteration stories, try and aggregate them down into the screen, workflow, or 'function' of the system, and build test scenarios from there. If there is even a modest number of stories to sift through, this would be a daunting task for the QA team - they would be in a 'catch-up' mode from the beginning of their QA cycle, likely through the end of the project.

As a result of the QA team being unable to join our project immediately, we are forced to modify our entire development process and account for the needs of the QA team, after the fact. Essentially, we are creating additional swim-lanes for our stories to travel through, during an iteration:

1. Backlog
2. In Development
3. Developer Testing
4. Ad-Hoc Testing (BA's, Developers, etc. testing the app)
5. QA Functional Spec Documentation
6. Done

Swim-lane #4 and #5 are directly affected by the QA team not being able to enter the project immediately, and are causing the rest of the team to lower it's capacity. We (developers and BA's) are now required to do our own ad-hoc testing during the iterations, instead of letting the QA team take care of that; and we (the developers and BA's) are now required to create functional specifications, based on the user stories that are completed, to hand to the QA team once they are on the project. The functional specification documents will need to be updated for any given User Story, before the ticket is considered complete (the functional spec updates have become an acceptance criteria on every User Story).

End result - because we cannot get the QA team on our project immediately, and ensure that they are testing in parallel to the developers, we as the development team are losing capacity and building entropy from the very first day of the very first iteration.

Anyone that has worked on a project with me is familiar with my usual "I just fixed … whatever" emails. For example - yesterday, I sent out this email to my team:

FYI – I just pulled down the latest code from subversion, with the new solution and project reorganizations… there were several project reference errors in the {our project} solution – many of the projects were trying to reference the \Trunk\Code\Library\ folder for the compiled output of a project that was in the same solution.

We need to make sure that projects within the same solution reference the project directly, not the compiled dll file. You can make sure this is correct by deleting everything out of the Library folder, than updating it from subversion (to make sure you have no {our project}.dll files in the Library) and then see if you get any missing file errors in visual studio, for the project references.

I’ve been sending out similar email for several years now, and I’ve always felt that it was important to do so, so that the entire team can gain new insights, move forward with better techniques, creating better software, etc.

Yesterday, I found An Article on the Lean Manufacturing concept of Jidoka and a few items stuck out in my mind and helped me to understand why I think that it’s important to call out these items. The core of jidoka goes well beyond the basic idea of "If it's broke, fix it now". It gets much more involved than just that. The article has a list of steps that should be taken whenever something is found to be 'broken' or 'abnormal':

1. Detect the abnormality
2. Stop
3. Fix or correct the immediate condition
4. Investigate the root cause and install a countermeasure

There's a lot more to the real implementation of jidoka than just fixing the problem - finding the root cause and installing countermeasures to prevent it from happening again is by far the most important step in this process. If you fix the problem in isolation and no one else knows that the problem occurred, then it's likely that the problem will happen again for the same reasons. This gets straight to the core of why I send out the “I just fixed …” emails. I found this quote in the article that further illustrates the point:

What I would not want to happen is to expect the team member who discovered the problem to try to fix it without telling anyone. But many times we expect them to do just that, and the rework becomes so deeply embedded into the routine that we can't even tell it is happening. It seems normal because it has never been flagged as abnormal.

I don’t send out those emails to chastise anyone, point out that someone broke something, or anything negative like that. Honestly – I don’t care who broke something; I only care that it is broken, it gets fixed, and we find the cause and do what we can to eliminate that cause. I send out those emails because when we (as a team – any of us; developers, testers, subject matter experts, etc) find a situation that is not right, we need to fix it – but more importantly, we need to notify the rest of the team so that we can learn as a team and find ways to not make the same mistake in the future.

I highly recommend that everyone go read that article and I also want to suggest that any time any of us runs into a situation where something is wrong – whether it’s code that is written or designed poorly, a references issue, the build being broken, or whatever it is – we not only fix the problem, but we inform the team of what the problem is/was and how to fix it and prevent it. We need to have this level of communication in the team, or we are not going to grow as a team or as individual developers, as fast as we could.

Yesterday, I posted a quick thought on code generation, and one of the statements I made is worth re-stating to stand on it's own.

In Agile/Lean software development, a single User Story is one piece flow when implemented via TDD, in a Workcell

In Lean Manufacturing, one piece flow is the idea that you do not produce anything in batches, but that you produce one product from start to finish per a customer's order. In Lean / Agile software development, this is analogous to an iteration with user stories. An iteration backlog is a customer order - a set of behaviors or features that are requested to be delivered in the current iteration. A user story is the single piece that we want to flow from beginning to end - from the start of coding all the way to acceptance via user testing.

In manufacturing, a Workcell is used to facilitate one piece flow through the manufacturing process. In software development, a Workcell is also used to facilitate the one piece flow. The Workcell may be 1 person, pair programming or a three person team. In order for it to truly be one piece flow, though, strict standards such as TDD and automated acceptance tests must be followed.

I'm not going to detail the benefits of one piece flow, here (not yet, anyway... another post for another time). I highly recommend that you read The Toyota Way and Lean Thinking for a good understanding of this concept and the benefits it provides.

Until around a year ago, I was an advocate of code generation via CodeSmith. Having marginal "success" with it in the 4 years I advocated for it, I'm now of a different opinion.

Code generators, such as CodeSmith, are automated overproduction machines that require prior overproduction, in the form of schema, to be used

Micro code generators like as Resharper, are much closer to JIT machines when lined up in one-piece-flow processes, such as Test Driven Development

...

If you had told me, 2 years ago, that I would make these statements today, I probably would have laughed at you.

There are a few examples in The Toyota Way, of American manufacturing companies that were thought to be world class Lean Manufacturers. When representatives from Toyota began working with these companies, though, the result was quite a shake-up. The companies that were thought to be lean, were merely implementing some of the surface level processes of lean manufacturing. They had not realized that there was a root cause of the process that they were implementing.

The Principles, Not The Process

A coworker recently asked me to help out with a presentation on Model-View-Presenter. He had some specific questions about why we would want to use MVP vs. MVC vs. any other UI pattern. Many of the questions centered around various benefits that have been promoted by myself and others out in the developer world via blogs and articles - items such as testability, changeability, flexibility, and other "ilities". My answers to some of these questions surprised me.

Model-View-Presenter is often thought of as a journey and an end in itself. When I first started learning MVP just over a year ago, this is exactly how I saw it. I started with the intention of learning how to separate my core process from the form implementation, with the intention of being able to unit test the core process. I was working under the impression that I could find a way to implement MVP that would truly allow me to swap out my UI - to the point of changing the workflow within the UI. I had the goal in the mind, of being able to write one presenter and have it support WinForms, WebForms, Web Services, and any other client-facing API.

One year and several projects later, I have some good lessons learned. I have implemented various forms of MVP - automagic view injection, manual everything, and a lot of ideas in between. My understanding of MVP is fairly solid, and I am capable of implementing it in WinForms, WebForms, Web Services, and other client-facing API's.

What I have realized recently, is that MVP is not (or, should not be) a journey and a goal in itself. MVP, MVC, and other UI related patterns are actually an effect of other underlying principles. The underlying principles of object oriented development - such as Separation of Concerns, Single Responsibility Principle, Dependency Inversion and others - are the real cause of MVP, MVC, etc. Testability, flexibility, and the other "ilities" are merely side-effects of good design and implementation. And while Testability is a valid goal in itself, seeking it as the primary or only goal will only lead you part way down the path. You will end up like the American manufacturers that thought they were world class lean companies, only to find out that they had barely scratched the surface of lean process.

The Journey, Not The Goal

Despite the success of Toyota as the creators of lean manufacturing, they believe that they are continuously learning about lean, so they can continuously improve. Similarly, despite my many years of software development experience, I now believe that I know little about software development. I have a good knowledge of .NET and general knowledge of Object Oriented Development, but I don't know much about other paradigms like Functional Languages or Duck-Typed languages such as Ruby.

In my professional growth as a developer, I have set goals for myself at various times. Most of these goals were set with the belief that they would make me a great developer. However, I've found that every time I have reached a goal, there is an entirely new world of possibilities ahead and what I thought was a great developer was merely the beginning of a journey. I've discovered that the journey itself is often far more important than what I believed the goal to be.

I'm not advocating that you forget about setting goals and let your career happen as it will - that's a great way to become burned-out, outdated, and irrelevant. What I am advocating is that when we have a goal in mind, we set out with the understanding that the goal itself may only be a small leg on a long journey. In this case, the journey is the process of learning. The journey itself should be viewed as the long term strategy, to the point where you allow the individual goals to come and go as needed. You may not accomplish a specific set goal - but the experience gained while working toward that goal may open up opportunity for other goals to be reached easily.

Learn The Principles Via The Process

All this talk about knowing the underlying principles is great - but how can we understand the principles and philosophies without knowing the process? Unfortunately, I don't know if that's possible for most of us. I'm sure that somewhere out there, someone has read SRP, DI, SoC, and other principles, and as a result began writing code that was modeled in a manner similar to MVP (Martin Fowler, for example). For the majority of us, though, learning the principles is done by first learning the process - study the implementation of the effect to learn the cause.

Conclusions

The journey itself is how we build our experience and understanding. For most of us, that journey begins with a goal of learning a new process. Find a goal or two that you believe will help you be a better developer and learn the processes. Along the journey of reaching for this goal, always keep in mind that there is an underlying principle or philosophy that enables the process. By learning the principles and philosophies behind the process, the process itself becomes a trivial matter - an implementation of the principles and philosophies, and not a goal in itself.

The Toyota Way mentions the concept of Jidoka in chapter 1 (and probably other places that I haven't read yet). On page 6, in the "4 P" diagram, jidoka is described as

"Stop when there is a quality problem"

Wikipedia calls the same concept "Autonomation" and says it may be described as

"intelligent automation" or "automation with a human touch"

and

"At Toyota this usually means that if an abnormal situation arises the machine stops and the worker will stop the production line."

We can apply the same principles in software development in many different ways. One of the more common implementations is the use of Continuous Integration and Automated Testing.

According to the notes I took during A Day Of Bellware, if our CI server says the build is broken, we need to immediately stop working and fix the problem. I've heard this before, and will likely hear it again. However, I never really understood why people would say this. After all, as long as the problem is fixed eventually, isn't that ok? In the training that day, Scott provided a excellent visualization of why we should fix it immediately.

Waste and Rework

To steal Scott's illustrations, consider the following image to be an example of "perfect" software. All of the edges of each module (block) are well defined and it's easy stack new blocks next to existing blocks.

Figure 1. "Perfect" software

Now let's assume that somehwere in the coding process, someone accidentally causes a defect in the software. That defect could be represented by a buldge in one of the lines - as if the module was doing more than it should.

Figure 2. A defect

The individual block that has the defect may not be that bad, at first glance - or when examined on it's own. And on it's own, the defect could be fixed. Jidoka would say that you need to stop immediately and fix the problem.

So, what happens when you don't fix the problem right and and you try to stack another block to the right of the defect? Suddenly you find yourself re-shaping the next block in order to account for the problems in the previous one. Eventually, you may be able to smooth out the issues. Depending on the size of the issue, though, it may take several new modules to completely normalize things.

Figure 3. The effect of a defect

Now, 3 day, 3 weeks, 3 months, or whatever period of time later, you have a significantly larger problem to deal with. Let's go back and fix the original cause of the problem, to start with. What happens to the rest of the code that was warped around the original defect? The warped code in the rest of the system also has to be fixed. Simply removing the original bulge does not mean that the rest of the warps will magically disappear as well.

Figure 4. Removing the original bulge

How much re-work will it take to fix the rest of the system that was warped around the original problem? How much time and effort will be wasted because the original defect was not addressed immediately? These questions can only be answered in the context of your problems. If this defect is one line of code in one module, maybe it's not so bad. But if this defect is an entire module used by many other modules, the time and cost could be huge.

Conclusion: Fix It Now

If the continuous integration server says that our build is broken; or if the customer says "the software does not break, but piece XYZ isn't correct"; or ... pick a problem in your system; we need to respond as quickly as possible, in order to prevent the rest of the system from being warped by the original problem.

Under the assumption that we want to fix a defect as soon as possible - how do we ensure that we know about the problem as soon as possible? Whatever your implementation of this solution is, the solution to this problem comes down to shortening the feedback loop. If you are notified of the problem in 20 minutes vs. 20 days, there will be significantly less damage to the overall system and significantly less re-work and waste caused by the defect.

Myself and 10 other developers in my company went through a day of BDD / TDD training, with Scott Bellware, yesterday. It was a lot of fun, very challenging at times, and covered a lot of topics including an overview of Agile and Behavior Driven Development, all the way down to writing Specification Tests, doing Test Driven Development  and refactoring the model to improve readability, maintainability, flexibility, etc. I took notes via index cards (love that cool-aid) and wanted to share. I don't expect these notes to make sense to everyone. Hopefully it will spark some dialog in someone's mind and cause them to dig further.

First off - the quote of the day.

"Can I be honest with you and say that I've been wanting to touch your keyboard, all day?"

Now for my notes.

User Stories

[Role], [Goal], [Motiviation]

• As a [role], I want to [goal], so that [motivation]
• Example: As a nurse, I want to record a patients vital signs, so that I can determine their medication and care needs
• Motivation is critical - it determines how the development team understands and implements the story. It determines the user experience, how things are integrated, how the software is designed, etc.

Acceptance Criteria

• Acceptance Criteria is used to drive code, not the story, directly
• may change at any point, up to implementation
• is used to drive code design, test design, implementations, etc.
• should be spoken in domain language
• may include non-functional, technical details such as database tables, infrastructure, performance, etc
• All acceptance criteria must be met and tested / verified before a story is considered done

Specification Tests

• Test Fixture per Class is an anti-pattern (on a personal note, this problem bothered me for months before I discovered BDD)
• Context Specification or Behavior Specification testing
• When [verb] then [verb]
  • "When [verb]" is the context
  • "Then [verb]" is an observation of the behavior
• Based on Acceptance Criteria, but not "code-gen'd" from acceptance criteria

Story Estimation

• Agile Poker: uses generalized Fibonacci sequence as order of complexity
  • "?", 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, infinite
• everyone throws their estimate at same time
• if estimates have significant outliers, discussion occurs to understand why, get more detail, etc. and re-throw may happen

Entity Data vs. Aggregate Data

• Entities should never contain aggregate data
• Aggregate data is for reporting and other aggregate needs
• If you need aggregate data to process something, write an SQL query, stored proc, etc. - don't use an ORM like NHibernate
• We don't want a "Customer" entity to need 10,000 "Order" entities, to aggregate data for processing; write a query to aggregate instead
• We don't want to persist data that can be calculated / aggregated, generally (performance issues may override this)

Domain Services

• Can have dependencies on external systems
• are part of domain logic, therefore are in domain model / assembly
• are "Doers" of process that don't fit into entity and entity logic, directly
• coordination of entity logic
• can include calls to data access, logging, etc.

Continuous Integration

• Not just continuous compilation of code
• Full end to end integration of all code, components, databases, services, etc
• Full suite of integration testing including database testing
• Do not allow commits if build is currently broken
• do not allow defects to live - fix immediately, to fix build
• "Defect" is broken software, "Bug" is functional but wrong

Daily Scrum

• 3 Questions everyone answers:
  • What did I do yesterday?
  • What am I doing today?
  • What issues am I having?
• Each person should answer quickly - 1 or 2 minutes, max
• further discussion happens outside of the Scrum meeting

Productivity of Dev Team

• RAD and other non-review, non-iterative based management causes problems and loss of productivity
• we need constant review of the design to ensure good design
• shorten the feedback loop and get constant review of the design, to always improve the design, via pair programming, work cells, retrospectives, etc.
• good design will cause productivity gains in the development team beyond the capabilities of any tools

Whiteboard Diagramming vs. Details Specs

• White board diagramming and human interaction is always better than detailed documents and specs in UML
• Human interaction leads to knowledge crunching and learning, not just reading a repeating
• Take pictures, don't re-draw in UML; don't waste time with it
• Video the entire conversation is even better, so others can learn from the knowledge crunching that occurs; capture the human interaction, body language, etc.

One of the primary goals of Lean Manufacturing in the elimination of waste. This may seem like an obvious goal - after all, who wants waste in their system? What's really interesting, though, is seeing where the term "waste" is actually applied and how Toyota redefined what waste really is.

Muda, the concept of the seven wastes in the Toyota Production System, is a very different concept of waste than most of us would consider. It talks about not only physical waste (trash), but also logical, time, and effort waste. These seven wastes include (from Wikipedia):

• Defects: Quality defects prevent the customers from accepting the product produced.
  
• Overproduction: Overproduction is the production or acquisition of items before they are actually required.
  
• Transportation: Each time a product is moved it stands the risk of being damaged, lost, delayed, etc. as well as being a cost for no added value.
  
• Waiting: Refers to both the time spent by the workers waiting for resources to arrive, the queue for their products to empty as well as the capital sunk in goods and services that are not yet delivered to the customer.
  
• Inventory: Inventory; be it in the form of Raw Materials, Work-In-Progress (WIP), or Finished Goods, represents a capital outlay that has not yet produced an income either by the producer or for the consumer.
  
• Motion: As compared to Transportation, Motion refers to the producer or worker or equipment.
  
• Overprocessing: Using a more expensive or otherwise valuable resource than is needed for the task or adding features that are designed in but unneeded by the customer.

In "Lean Software Development: An Agile Toolkit", Tom and Mary discuss The Seven Wastes and offer a great set of parallels in software development. Rather than copy & paste what they've already said, I wanted to offer my own interpretation of these wastes - some of which are the same as Tom and Mary's.

Defects

This is exactly what it sounds like - a bug, a piece of functionality that doesn't work right, or an exception that gets thrown in the system and causes problems. How many hours, days and weeks have you spent fixing defects in software, in your career? Personally, I can't even begin to accumulate the time spent - it's far too great.

Part of the problem here, and what really makes it wasteful, is when defects are found and fixed in most software development cycles. In Code Complete 2, McConnell talks about the cost of defects in software, relative to when the defect is found and corrected (pages 472 & 473). If the cost of fixing a defect goes up over time, then the amount of waste also goes up. In other words, the higher the cost of fixing a defect, the more waste that defect has generated.

With the increasing cost of defects in mind, we will want to reduce the time to find a defect so that the amount of waste generated can be mitigated. In Lean Manufacturing, this is achieved by only creating what is needed for an actual customer order, right now. If one part is produced with a defect, that one part is scrapped and a new part is created, correcting the defect - the waste is minimal. Imagine a mass-production system where parts are created in batches of ten thousand. What happens if there is a defect caused by the process? We end up with a significant amount of waste, because we will have to scrap the defective parts in batches of ten thousand - the waste here, is significant.

The same principle holds true in software development. If we are writing code for extensive periods of time, and that code is not being tested until the end of that period, then the cost of finding a defect in that code goes up. If we have ten lines of code that rely on the defect working the way it does, it is significantly cheaper to fix than if we have ten thousand lines of code relying on the way the defect works. The easiest way of reducing the number of lines that rely on a defect is by working in small iterations. If that defect goes into test after two weeks, it will be much cheaper to fix than if it had gone in to test after two months. Additionally, if automated tests are done and can be run against the system on an even shorter schedule, then the cost of finding the defect may be even less. If the automated tests are run every twenty or thirty minutes, you may only have two or three lines of code that are affected by the defect.

Overproduction

There are several common terms for overproduction, in software development - Over-Engineering and Coding For The Future, for example. Both of these problems result in the same thing - too many features and more functionality than is actually needed right now. Additionally, overproduction can be called technical debt. The idea is that there is code in our system that will need change - when this code needs to be changed, how difficult is it to change; what is the cost of the change (how much debt are we in).

If a developer is tasked with creating Feature-X, they should focus on the core business value of that feature when writing the code. It's very easy for that developer to think about "what-if" scenarios and "we may need to" future functionality. However, this type of thinking leads directly to overproduction. If the business has not identified the "what-if" scenario, then why should the developer assume that handling the scenario would add business value? If the business has not yet identified Feature-Y or specified any additional functionality in Feature-X, then why should the developer assume that coding for the possibility of Feature-Y is valuable? Take note of the pattern in those questions - assumption. Assumption on the developer's part leads directly to the overproduction of the system being built.

So how do we prevent this overproduction? First and foremost - stop assuming you know what's coming down the line, or what will provide business value. If you believe that the specs you are currently working with are not accounting for an actual problem, then the responsible action is to discuss that issue with the customer / business analysts. Secondly, if you find yourself wanting to code for the feature that may be coming down the line - stop. Don't do it. Even if you know that this feature or functionality is coming down the line, it may not provide any business value for the current feature(s). You may be causing detriments to the business value of what is currently being built. If you truly believe that the feature you want to code is required for the feature that you are coding, then you need to go back to the customer / business analyst again, and discuss it with them. Let the customer tell you whether or not there is value in adding that feature now vs. later (vs. if at all).

Transportation

Think of transportation in the context of knowledge. No one knows everything about a particular domain or project, and possibly an individual feature. We have business analysts and customers precisely because of this. Eric Evans discusses knowledge crunching (the process of learning and gaining an understand of the domain in question, to create a model) in Domain Driven Design, in the first chapter. Tom and Mary talk about the process of learning all throughout the "Amplify Learning" chapter of Lean Software Development. There are countless other books on requirements gathering, model creation and other learning processes for software development, as well.

The problem that we face in our learning is that the transportation of knowledge, from one person to another, will always introduce change. Every person in existence has their own unique way of understanding, based on their experience, existing knowledge and methods of knowledge retention. To illustrate this, look back at your childhood. Did you ever play the "telephone" game where one person whispers in the ear of the person next to them, then they whisper the same thing into the ear of the person next to them, and so-on until the last person in line? What are the typical results of this game? More often than not, the message had some fundamental change to it throughout the transmissions. As simple as the telephone game is, it's a perfect illustration for knowledge transportation that occurs in software development projects.

How many times have your received a functional specification or requirement for a feature, and had no clue what it was trying to convey or what the functionality was supposed to be? This happens on a very regular basis. The problem may not be that the documentation is inadequate, but that the transportation of knowledge from one person's mind onto the paper and into your mind is inadequate - what makes sense to the document writer(s) may not make sense to you, because your contextual knowledge of the subject is different. This problem is exacerbated by design-up-front project methods. The knowledge is often crunched from the customer's mind into the BA's mind, and then trampled down into a linear document or subset of knowledge in a documentation set where the developer has no direct access to the customer - the source of knowledge - for clarification and discernment of the details. Even if the customer is on-site sitting next to the developer throughout the creation of the documentation, there is still a transportation problem in design up front projects. Whether or not the developer understands the information during the conversations, there is little chance that the developer will remember every detail of the information when they finally get down to coding.

The solution for the transportation of knowledge is ensure that the knowledge is transported as little as possible. Rather than focusing the effort of documentation on every last possible detail - creating tomes of documentation that no one is really willing to read - we should focus on creating higher level guidance; enough knowledge to know the direction of the development effort and the features to be created, with the understanding that the final details will come from the customer just-in-time for the development effort. In the world of lean manufacturing, this is typically done through the Kanban system. In software development, we have adopted the same process through the use of User Stories being written on index cards. The stories are not the final word in the development effort - they are merely a placeholder for a conversation between the customer and the developer.

Waiting

This is likely the most obvious of the seven wastes, when applied to software development. Whether you are waiting for a customer or business analyst to be available for questions; waiting for a requirements document to be signed off; or waiting for your next assignment of work, the simple process of waiting creates lost time which directly translates to lost opportunity. In the extreme cases, the problem is compounded by context switching. If you are waiting for a question to be answered - whether this means waiting for a phone call, a document to be written, or just walking down the hallway - you can easily lose your focus or lose you place in the code or coding process. When this happens, most people take a while to restore that focus and get back into the mind set where the new knowledge is useful. In my experience (no real data, just my experience), the average time of context switching within the same project is as little as five to ten minutes for wind down, and at least ten or fifteen minutes for spin-up. If you are context switching between multiple projects, though, expect to spend much longer on the spin-up; hours, even days, depending on the project and time between wind down and spin-up.

If we can ensure that we never wait longer than it takes for us to wind down, then we should be able to eliminate the spin-up period; thus reducing the effort required for reestablishing our focus on the feature(s) at hand. However, the solution for waiting may not be as obvious as the problem. If you are waiting for documentation, or if you are waiting for the customer to sign off on a feature or change, the way to eliminate the waiting is to eliminate the cause. Don't rely on heavy documentation or require customer sign-off on features. If we are working with the customer every hour of every day, eliminating the knowledge transportation issues, then we can also eliminate much of the waiting game by having the customer available. When the customer is available immediately, we have a much greater chance of the conversation beginning within the wind down period. When this occurs, we can stay focused on the problem at hand, and mitigate the spin-up period when we return to the code.

Inventory

Inventory in a manufacturing or supply context is typically a product or component of a product that is in storage and waiting to be used; or "a list of goods and materials held available in stock by a business" (Wikipedia). In software development, we only need to translate "good and materials" into "code and resources":

A list of code and resources held available in stock by a business.

Think of software inventory as unfinished code (as defined in Lean Software Development) or as overproduction of code. If we have unfinished code in our system, this is like having a physical component for a product - the component by itself is not useful until it is in the completed product. Similarly, the unfinished code provides no value to the software until it is finished. Even if the code is complete, if it is not being used in the software then it is dead code and is inventory for the software.

Inventory, in software development, carries a significant technical debt - the unknown factor of whether or not the code actually needs to be in the system. The longer we let inventory live in our code, the more likely we are to forget whether or not that code needs to be there. This inventory also becomes a potential mine field for developers - what happens if a developer sees this dead code and assumes that it is still functional and valid? If we are lucky, the code will work as advertised. The longer the code sits in inventory, though, the more likely it is to be outdated, invalid and eventually non-functioning.

Motion

In addition to physical movement causing waste through waiting (walking down the hall to the BA's office), motion can also be applied to the processes of producing the executable software (not the source code and other resources). How much effort does it take for your team to compile the resources, create the executables, package it into an installation and deliver it to QA, the customer, etc? The goal should be to make this as simple as possible - as little movement as possible.

Are you able to click a button and have the build, package, and delivery processes handled for you automatically? Or does it take a team of developers, build engineers and other persons with a manual process and a checklist of things to do? If your process involves people performing manual tasks - even if that task is simply the order in which multiple buttons get clicked - then you are at risk of creating waste through defects. Every manual motion in your build process is another point at which mistakes can be made, allowing defects to be built or causing the build to fail completely.

Software production (again - speaking about production of executable software, not development of resources or code) is often a difficult, tedious chore, requiring many moving parts - there is no way around this. However, it does not have to be a manual process. Continuous Integration and Continuous Deployment practices are abundant in the world of software development, specifically to address this issue. If you can automate your build process and eliminate the manual movement involved, you can eliminate a huge source of potential waste.

Overprocessing

Overprocessing, or too much process, is abundant in software development. When a customer has a request for a feature, how much process does it take for that feature to become functionality? In design-up-front project life cycles, the process of adding or changing a feature can be very time consuming, involving many steps on both the requester and implementer side:

request; acknowledgement and change management / approval; design and documentation by implementation staff (BA's, developers, etc); sign off of design and documentation by customer; implementation in software; scheduled for testing; tested; and if any issues are found, back to design or implementation for corrections, repeating several steps.

Additionally, how much work does it take for you to get your job done, as a developer? Do you spend as much time updating task / issue management tickets and managing your manager, as you do coding?  Too much process in software development is as bad, if not worse, than no process at all. Unfortunately, the failures of too much process are often seen as an indication of the need for more process, causing further delays and waste in the system.

There are often situations where you cannot do anything about the process that is in place. Life-Critical systems (x-ray machines, air plane guidance systems, etc) are abundant with process for good reason - if anything fails, people could be injured or killed. In the world of business software, though, this is not usually the case. Most business has a level of fault tolerance built into it, and some waste is expected / accepted. This is not a get out of jail free card, though. The more significant the waste, the more repercussions there are - fix the defect; pay for lost revenue; lose your job; etc. If we want to eliminate waste through the simplification of process, then we need to ensure that defects are found and fixed as early as possible, to avoid creating waste for the business using the software.

Process simplification can be done many different ways. Start with the creation of a process flow char, or value-stream chart. Find out how much time and effort is wasted in your processes and look for the waste that can be easily eliminated. Simplifying a single process by removal of waste can lead to a revolution in your process. You may find that the removal of one process negates the need for another process. You may also find that a simple change to one process does the same. If you are able to bring your customer into the work area with your team, and work in short iterations where defects are found and fixed quickly, you will likely see many of your old processes quickly become obsolete.

Conclusions

No one wants waste in their systems. Waste is money lost and revenue not earned. Waste management, though, can be a difficult proposition. Many company are willing to accept waste as the cost of doing business and are not willing to spend money eliminating waste because they do not see the waste or do not believe it can be eliminated. If we are truly going to revolutionize the business we work in, we must eliminate waste. Start where you have direct control or influence. You'll quickly find others wanting to know how you are so productive and soon the entire business will be in full waste reduction cycles.

I've been hearing this a lot lately, and finally started reading up on the roots of the Agile movement. It all goes back to Lean Manufacturing. Oddly enough, I've had exposure to Lean Manufacturing at a previous job. So the concepts and names are all very familiar to me. Back in those days, I had heard of the concept of Lean Software Development, but it never really made any impact on how I wrote software. These days, though, I'm diving into as much Agile as possible and the further down the road that I travel, the more and more parallels I see, to the lean manufacturing world.

If you are at all interested in the roots of agile, here's some recommend reading (mostly Wikipedia) to get you started.

Wikipedia:

• Lean Manufacturing
• Lean Systems
• Lean Software Development
• Toyota Production System
• Kanban

Books:

• The Toyota Way
• Lean Software Development: An Agile Toolkit
• (anyone have other suggestions?)

I'm hoping to absorb more of this information, quickly. It seems the more I learn about lean manufacturing, the more I understand about software development. I certainly don't claim to be an expert on either subject... hopefully I can change that.