Quality must be built in, it cannot be added on. … well, ok. I’ll admit that it’s not entirely true. If you don’t mind spending exorbitant amounts of money doing rework, causing projects to be late and over budget, and possibly losing the business, then by all means, wait until after-the-fact to try and add quality onto a product. No really. It can be done. You just have to accept that the system will be rewritten. After all, isn’t that the way software development goes? Build something… try to add on to it… recognize the problems that prevent you from continuing and then justify starting the same vicious cycle over. (How arrogant are we, as software developers and development teams, to expect our customers to accept this? But, I digress…)

We already know that the real world builds quality into their products by favoring defect prevention over quality inspection. What we often don’t realize, though, is the real cost of not building in quality from the start. So, what are the costs and effects of trying to add-on quality after the fact? Before I go down that rabbit hole, though, I want you to understand how I define quality, at the moment. Without this understanding, the illustration is far less effective.

Defining Quality

I’ve run across a lot of definitions of ‘Quality’ over the years. Some of them good, some bad, some so far off from the truth that they would cause me to laugh if I didn’t know someone truly believed them. After all this time, I’ve begun to settle on my current favorite definition of Quality as: conformance to requirements.

Yes, I understand the somewhat naive focus that this definition of quality presents. However, I have yet to find as solid a foundation for quality as this. The Wikipedia entry for Zero Defect, from which I take that definition, continues to say this about quality as conformance to requirements:

“Every product or service has a requirement: a description of what the customer needs. When a particular product meets that requirement, it has achieved quality, provided that the requirement accurately describes what the enterprise and the customer actually need. This technical sense should not be confused with more common usages that indicate weight or goodness or precious materials or some absolute idealized standard. In common parlance, an inexpensive disposable pen is a lower-quality item than a gold-plated fountain pen. In the technical sense of Zero Defects, the inexpensive disposable pen is a quality product if it meets requirements: it writes, does not skip nor clog under normal use, and lasts the time specified.”

In summary - the customer is the one that defines quality. That’s a much less naive view than the definition betrays.

More Than Meets The Eye

Saying that the customer defines quality is still somewhat naive – at least on the surface. It is true that the customer is the final authority on whether or not the product is quality. However, the reality of quality in software development is that there’s more too it than just meeting the current requirements. As I’ve previously talked about, I think quality is more of a two-sided coin in software. On the one side – the primary side – you have conformance to requirements. That is, what the customer actually needs to help solve their business needs. On the other side of that coin is the technical quality, as defined by the known best practices and standards that we adhere to as individual developers, team members, companies and larger communities. (Side note: More often, I’m finding it easier to define what quality in software is not, rather than what it is.) That being said, I’ll leave the definition of quality in software engineering to those previous and future discussions.

Getting back to the original point I was making, I’d like to illustrate a situation that I’m very familiar with on the cost of not building quality in from the start.

Creating The ‘Perfect’ Failure

A long time ago, in a company far far away… [insert star wars theme]… there was a product under development. To protect the guilty innocent, we’ll call this project “ProjectX”. ProjectX’s customer was a very large entity, with lots of software products and projects that had been developed by many many software providers in the past. So, when ProjectX was first delivered to this customer and the customer said “It’s PERFECT!”, there was much rejoicing on the part of the software provider. In fact, ProjectX was deemed so perfect by the customer – having met every requirement, with less than 1% of the functionality being deemed ‘flawed, but still usable’ – that the customer declared ProjectX’s test results and acceptance as ‘the new standard’, across the board.

But there is more to ProjectX than the initial quality that the customer observed. In fact, ProjectX had a deep, dark, disturbing secret just waiting to rear it’s ugly head. That secret was the distinct lack of quality that the software provider had built into ProjectX. Everyone at the software provider knew the lack of quality – it was a necessary business decision to role ProjectX out the door as quickly as possible.

Fast forward 5 years, now. ProjectX is still a living, breathing, changing system. The customer, over the last 5 years, has requested numerous enhancements, requirements changes, and other stuff for ProjectX to do. While working on these new requirements and changes, the software provider found themselves in a sticky situation. Since they had set ‘the new standard’ with the customer – a standard so high, that they were now expected to deliver ‘on time, under budget, with perfect quality’ – the customer expected the software provider to continue meeting ‘the new standard’. Since the software provider knew that the original business decision to sacrifice quality for speed would come back to haunt them eventually, the software provider spent inordinate amounts of time trying to rectify the situation. They re-implemented parts of the system that were being modified, and introduced new architectures and standards when features needed them, and they worked 60 to 80 hour weeks for months – nay, years – on end, trying to introduce quality into ProjectX.

With each successive delivery of new features and functionality, though, the customer began to notice a distinct drop in quality. In fact, there was an alarming trend in the quality of the product, over time. The customer began to track the ratio of defective to non-defective functions, and they began to complain about the problems that ProjectX was having. Eventually, the customer’s user base refused to install the latest version of ProjectX because of the large number of known issues in ProjectX.

The customer became very upset at this new trend and many discussions with many sad users were had. And there was much sadness at the software provider. Questions were asked. Fingers were pointed. Many more discussions were had, and so-on and so-forth. The software provider, not willing to accept the new state of their software, decided to fix things… but that’s another story.

How ‘Perfect’ Failed

So how did a system that was deemed ‘perfect’ by the customer come to fail so miserably in the eyes of the same customer? The simple, honest, oh-so-hard-to-believe truth is that the software provider was unable to ‘add quality on’ after-the-fact. The software development team grew in their capabilities. They brought in new eyes and fresh expertise in development practices and architectures. They re-thought, re-structured, and re-worked so much of the system over such a long period of time. But it was never enough. There was simply too much of ProjectX that could not reasonably be changed and still meat the deadlines for new requirements, changes and enhancements.

Over time, the initial lack of quality that was built into ProjectX rotted the system, from the inside out. Because ProjectX was unable to modified to meet the new requirements without breaking the old requirements, the customer no longer saw ProjectX as a quality product. The cost of this lack of quality was tremendous – the software provider’s reputation had been tarnished, the customer did not trust the provider and did not want to use ProjectX anymore, and the software provider had a legitimate chance of failing as a company and shutting its doors.

Conclusion: Build Quality In Or Risk Losing Your Lunch Money

Creating a quality product really is a two sided coin. If you create the perfect system in the customer’s eyes, but you can’t maintain that system and add new features and functionality – where is the quality? You’ve effectively put lipstick on a pig and called it a supermodel. But if you have the ‘perfect’ engineering practices and the ‘perfect’ system that is ultra-easy to add features on to, and no one wants to buy the product because it does not solve any real business problems or provide anything that a customer actually wants or needs, then where’s the quality in that? You’ve effectively delivered a Lexus LS-400 when the customer asked for a ham sandwich.

It is imperative that quality be built in to a product from the start. We know how to do this. We are (or should be) standing on the shoulders of giants. We only need to keep in mind both sides of the coin when creating the needed quality. Customer quality first, with engineering equally as important after the customer says ‘PERFECT!’.

This is not just a scary bed time story and this is not just a bunch of ‘ivory tower’ fluff. “ProjectX”, despite a fictitious name, was a real project at a real company and the story I told is the boiled down version of really happened. In fact, it has happened at least once in every job I’ve had as a professional software developer. I know the cost of not building in quality from the start. I’ve seen my friends, coworkers and bosses lose their lunch money. It’s not a happy scene.

I heard the phrase “one team, one aim” while listening to the audio book version of “Extreme Toyota”. This is a phrase that marks part of the core philosophies of Toyota, according to the book. There are many different philosophies that underlie this specific phrase and I’m sure there is a very large amount of text that can be written about it. Rather than jumping off of that deep end without a greater understanding than I currently have, though, I wanted to point out one very specific word in this phrase: “aim”.

Wiktionary has a definition of aim that appears to fit very well in this phrase: “Intention; purpose; design; scheme.”

So, why use the word “aim”, here? Why not talk about goals? Why not deal with something more concrete? I think the answer to this is found in one of the core principles of Toyota: Continuous Improvement. A goal is a concrete, fixed point to be reached. Although a goal can change, once it does change, it once again becomes a fixed point to be reached. If Toyota talked about goals then they would inherently be limiting themselves to such fixed points of achievement. By using the word “aim” instead of “goal”, they are illustrating the true intentions of Continuous Improvement. That is, they are not setting their sights on attainable goals. Rather, they are taking a philosophical, yet very practical and daily routine of continuously improving in all aspects.

This does not mean Toyota works without goals. Quite the opposite, really. They create some of the most ambitious goals that anyone can imagine - “impossible goals” as talked about in the Extreme Toyota book. However, these goals are always aligned with the company’s vision, their aim. Toyota recognizes that a single goal is never a place to stop, having accomplished something, but is only a starting point allowing them to define the next goal and begin that leg of the journey.

As I’ve talked about before, we should be focused on the journey – the aim of where we are heading - constantly adjusting that aim as we obtain new information. We then set goals to help us move in that direction, knowing that we are truly on a journey, not looking for a place to stop after a single, fixed point of achievement.

In the manufacturing world, you would never find a company that assembles a bunch of parts into a final product before inspecting any of the individual parts, and they would not wait until the end of the assembly line to test for the quality of the product. The very notion of waiting until the product is “done”, to test it, would be appalling. How much time and money would be lost trying to figure out why something didn’t fit together properly, why it didn’t work, and why there was a quality issue with the final product? Rather, we see the manufacturing world taking an active role in preventing defects. Yes, they still do a final quality inspection, but the primary means of ensuring a quality product is delivered is not by waiting until the product is assembled to test it. They build quality in from the start and maintain that quality throughout the manufacturing process.

Prying Open The Case

Imagine the inner workings of the phone to the left. This is a very complex piece of technology. Do you think Cisco would wait until they have assembled this phone and then try to pry open the case so that they can insert a set of electrodes to test and see that the circuit board is connected correctly? I certainly hope they don’t. Instead, when a manufacturing company is building something – anything – they start with the idea of preventing defects, not waiting until they are identified and correcting them. Many companies have an active Zero Defects policy where defect prevention is paramount and quality inspection is almost just a verification of what they already know – that the product is defect free.

When a part is stamped out, formed, molded, or otherwise created, it is done so to an exacting specification. After the part has been created, the part is then tested against the same specification to which it was originally built. If the part does not fall within the tolerance and guidance of the specification, it is scrapped and a new one is made. If a series of parts are found to be out of specification, it’s usually a sign that something in the process, tooling, or other portion of the manufacturing process is not right. When this happens, they fix the cause of the problem – whether the machines need to be calibrated, the people running the machines need better instructions or whatever the cause is. In the end, the specifications for the parts were used to create the part, identify whether or not the part was up to standards, and decide whether or not to keep that part.

What’s more, the manufacturing company doesn’t wait until after they start creating parts to create the specification. Rather, they take the time to properly engineer the specifications up front. They take measurements, create prototypes with varying specifications to see what works best, record the success and failure rates of the various specifications that are tried, and use other design and engineering principles to scientifically calculate the exacting specifications that will be used to produce the parts. This occurs at all levels of the product’s design and creation. For every resistor, capacitor and microchip that builds a circuit board, each one of them has their own specifications that have been carefully engineered. If any single capacitor does not meet the specifications, it is not sent to Cisco with the hopes that it works anyways. Only when all of the specifications of each part are met will they solder the parts to the circuit board, creating a subassembly.

When a subassembly is created, it also has a specification to which it was built. The subassembly then undergoes the same quality assurance process – verification that it meets the specifications and operational requirements. The process continues from here – each subassembly gets connected to a larger system which is built to a set of specifications, with rigorous testing of the larger system as it is built, ensuring it meets the specifications. When the final phone is assembled, we don’t have to worry about whether or not a specific capacitor is soldered to the correct location – we don’t have pry open the case on this phone and insert a set of electrodes to see if the electrical current is flowing correctly. Instead, we only need to plug this phone into the correct connections (an Cisco IP phone system in this case) and verify that the phone actually performs all of it’s functions, according the functional specifications of the phone. There simply is no need to verify the capacitor that was used in the very first step. We know it works because it was built in a system that actively prevents defects.

The manufacturing world is obsessed with testing. They are willing to test from the lowest possible levels of the system, out to the end-product and the behavior that is expected. They do this because the consumers of manufactured products demand perfect. Why, then, are so many software development companies so willing to only test from one end of the process? To only test once, and only from the user interface, just before the product is shipped?

A Specification By Any Other Name

Unfortunately, the software development industry as a whole, is years behind the manufacturing industry. Our definition of quality and success are often skewed and we may consider fifty or more known bugs in a system of moderate to large size to be acceptable. It doesn’t have to be this way, though. We have the technical capabilities of following in the footsteps of the manufacturing industry, and we should.

I’m sure there would be no small number of people that would say we already employ the use of specifications in software development. After all, that’s what the requirements gathering phase is for, right? So many software development companies have put so much effort, time and money into the process of producing a piece of paper that can be understood by humans, and labeled this a specification. The problem we face with paper, though, is how to effectively verify the software against what the paper says. How do we verify that series of software lines and I/O statements that are understood by a computer have actually implemented the human readable text on the paper?

We are fortunate, actually. We don’t have to accept a Word document or a piece of paper as the specification to build to. We have the ability to create executable specifications! We can, and should, be creating specifications that can measure and verify our code. Most people call it test driven development (TDD). Some call it Behavior Driven Development (BDD). I like to think of it as Specification Driven Development (SDD? Not sure if that really exists. And really, do we need another xDD acronym?). We write code that exercises our code in the form of unit tests, integration tests, functional tests, acceptance tests, or whatever you want to call them.

We’re Not Just Stamping Out Parts

One of the major problems that I have with the manufacturing/software development analogy is the obvious statement that we don’t stamp out the same parts over and over again. In spite of my previous comments on this analogy, I now think that we are more analogous to a specific part of manufacturing than I had previously understood. A more accurate representation of software development in the manufacturing world is new product design and development. The parallels work quite well from this perspective. I am not going to expound on this in great detail at this point. It should suffice to say, for the moment, that the process of product development described in Wikipedia is a fairly accurate representation of what we go through for the average software development project.

When a manufacturing company is working on a new product, they once again don’t stamp out parts without knowing what they are doing. Many different parts may need to be tested, many different designs may need to be tried, but every one of these is still built to a specification. The major difference is that the specifications used are expected to change over time, until the final specification for the final pieces are accurate enough to produce a production-ready prototype.  The same notions can be applied to the software development processes in TDD, with some additional benefits.

Built To Specs, Regression Tests And Change

Change happens. It’s a simple fact of software development. A customer thought they wanted X, but in reality they needed X-1/B – not quite what we originally thought. When this happens, we once again have a significant benefit created by our executable specifications. We only need to identify those specifications that are now wrong, correct them, and change the affected portions of the system to match the new specifications.

Our ability to change is direct evidence to one of the many benefits of TDD: regression tests. Every specification that we write becomes a regression test the moment we fulfill that specification’s requirements. With this in mind, we can work with an almost reckless abandon, free to add features, remove features, fix bugs (because let’s face it – we’re still going to find some issues somewhere in the system) and refactor the system to a higher standard, all without worry of breaking the system. We can act with this level of confidence because we have a safety net in our regression tests. If (when) we do break something in our new efforts, we will be notified the moment we re-execute our specifications – that is, run our regression tests. A specification test will fail and we will have a clear indication of what failed and why. This deep insight into the system gives us even further confidence in correcting any issues that we introduce. When a failed specification test tells you exactly which value from which class is wrong, and the context in which that class was executed is known (the exact input and expected output), pinpointing the problem becomes a rote process. Fixing the issue becomes relatively simple, and we begin to see true productivity improvements in our processes.

Start With Quality, End With Quality

Our industry is currently suffering from a lack of quality. We ship horrendously bad user experiences in products that are late and well over budget, yet we call this a ‘success’. It doesn’t have to be this way. If we change our perspective and start to take some cues from the manufacturing and product design and development world, we can dramatically increase our effectiveness as software developers. We can create high quality, low cost solutions like the world expects from manufacturers. Built to specification is certainly not a silver bullet. It is, however, the definition of quality in a Zero Defect environment.

By employing a built to specification mindset in our software development efforts, we can start with quality and maintain that quality throughout the life of our projects. This is the same process that is undertaken when a manufacturing company is working on new product design and development. It works well, it’s a proven process, and most of all – it makes sense. Build your software to specifications. Just make sure they are executable specifications.

I recently saw this question asked and answered in an after-project retrospective paper.

Was The Project A Technical Success?

While I am not trying to address who asked the question or what the answer was, the question on it’s on own, is flawed in my mind. Or, at least when taken out of context, it has become somewhat of a fallacy by standing on its own.

Not ‘Technical’, But ‘Operational’

The question we should be asking first is not about ‘technical’ success, but ‘operational’ or ‘business solutions’ success:

Was The Project An Operational Success?

That is to ask, did we effectively reduce or solve the stated business problems to an acceptable level, allowing the users of the application – the operational people - to be more accurate, efficient, productive, responsive, … etc, in their job? Only when we can answer “yes” to the project being an operational success, and provide sufficient evidence of this success, should we ask if the project was a technical success.

If we created the world’s most perfect architecture, design and implementation, but the project was a DOA failure because it did not solve or sufficiently reduce any stated business problems, resulting in no one to buy it or use it; what good would it be to have the perfect system from a technical standpoint?

The counter-perspective, though, is just as important. If we have managed to solve or sufficiently reduce the stated problems then we need to ask if we did so in a manner that allows us to maintain this system for the long-term life of it. Were we successful in our technical implementation of the business solution, thereby allowing us to continue solving additional / ancillary business problems, easily and efficiently?

Two Sides Of The Same Coin

We can’t sacrifice the needs of the business to create what we believe is a successful technical implementation. Yet, we can’t sacrifice the technical implementation just to satisfy the needs of the business. There is a marriage between the two – they are (or should be) two sides of the same coin. If we fail on either aspect and are not able to recover quickly, we will eventually fail as a project / product team.

For so long there have been so many advocating the benefits of the various Agile, Lean, Iterative, or whatever-you-want-to-call-it-these-days methodologies. We, as software developers, seem to understand the benefits of these methods. So why, then, do we still have customers that insist on big design up front, waterfall based methodologies, and other practices that we know are wrong?

Some Ad-hoc Consulting

During my Christmas vacation, I had the opportunity to do some consulting / coaching for my dad’s current company. The employee list that I spoke to consisted of my dad, my aunt, and my cousin. It was a nice little family gathering, I got paid with some amazingly good barbeque, and it was a very enjoyable experience.

I’m a software developer working for a company that provides software development and related services. My dad’s company is a customer of a software development organization (not the one I work for), receiving some custom built software for a project. They have been running into some issues with quality and timeliness of delivery for the last year or so and were wanting to ask some very pointed questions about the state of the software development industry. Essentially, they wanted me to tell them whether or not the issues they were having are “normal” and / or “acceptable” when receiving custom software development services. My answer was an unfortunate “yes, this is normal”, but “no, this is not acceptable”.

Over the next few hours of our discussion, we touched on many different issues that they are having and how some of those issues can be addressed from their perspective and from the perspective of the software development supplier. In the end, I had successfully educated my dad’s company on a few key points, confirming their suspicions and observations, and giving some new directions to start heading with the company that is providing the development services. I can’t go into all of the detail, but the summary bullet points included the following:

• The customer won’t know what they want until they see something they don’t want
• The software provider should deliver a working, stable version of the system on a regular basis, allowing the customer to use it in it’s current form
• The customer sets the priority of features and functionality to be implementing, in coordination with the technical needs and pre-requisites of the software development organization
• The customer is allowed to say something is wrong, even when they receive what they asked for
• The relationship between the customer and service provider must be a whole-team view. That is, “we are one team. we succeed together and we fail together”

On top of these bullet points, I laid out a simplified Scrum process for them, based on the already stated 3 week development cycle that the provider prefers.

In the end, my dad’s company was shocked, astounded, amazed, and energized by everything they heard. The light bulbs were clicking in their eyes, and I could see the value that they received form the conversations. They have dramatically changed their understanding of what it means to be a good customer and what it means to be a good supplier.

A New Perspective

As much as I believe I helped my dad’s company in their understanding of what they should expect and how they should approach their issues and needs, I also learned a great deal from this experience. It was very enlightening for me to hear the issues that we have known for so long as software providers, but from the customer’s perspective. Nothing that they spoke of was unknown or unfamiliar to me. I have lived through all of the same pains and issues that they are going through – from both a software provider and a customer of software services.

What really stuck with me from this experience, though, is the idea that our perspective on how to improve our industry is likely wrong. Yes, we need to continue educating our own ranks on the benefits of better software development techniques and processes. However, it does no good for us to have the perfect pneumatic toolset when the customer is asking us to use a rusted hammer and a broken hacksaw.

What I’m getting at is that its not us, the software providers, that need to change so greatly. Rather, our customer base needs to be educated and change. We need to stop being so self-centered and introspective for a while so that we can educate our customer base on what it means to be a good customer; how they can accomplish their goal of receiving quality software that provides true business value for them, at an affordable price. Until we have a customer base that expects software to be high quality, the first time, every time; expects to be given working software on a regular basis; and expects to be able to recognize that what they thought they wanted is not what they really needed; we will have no chance of our inward looking self-improvements taking hold.

I have to say, the Agile Manifesto got it right from a values perspective. The values espoused there are what we need to be looking toward, but I think our perspective needs to be changed on how we enable those values to be reached.

Of course, if we do succeed in educating our customers, we need to be able to respond appropriately. So please, continue the education of our own industry. Create better development standards that increase our quality and productivity. If our customer expects a quality delivery every 2 weeks, we better be able to deliver a quality product every 2 weeks. Just don’t stop here. Educate your customers. Help them understand the financial impact of poor development standards and processes. Change the norms and improve our industry by creating a customer base that requires perfect. The manufacturing industry did it 20 to 30 years ago. It’s time for us to step up to that level.

Where Do We Go From Here?

So where do we go from here? How do we start educating our customers? How do we change the social norms of software development so that our customers expect us to behave more like professional craftsmen and engineers, delivering quality faster?

I don’t claim to have the answers and I don’t claim to be the first to encourage this change. I only hope that we as an industry will begin to understand that the customer is usually the one that sets the expectation for how often software is delivered, and that we need the customer to expect something better.

Imagine that there are three group of people going out for a hike. Within each group of people, we have a goal of everyone reaching a picnic table at the end of the hike (and no one can eat until everyone has arrived). One of the people in each group has a map and a compass, knows how to use them, and will be directing the group on where they need to be going (we’ll call that person “the Pathfinder”). Everyone else is just along for the hike and the picnic.

The Pathfinder in each group is given a specific set of instructions on how they are going to direct everyone else.

Group 1: Observe, Plan, Walk

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course in one hundred foot increments, with your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking. Give an updated direction to walk every one hundred feet, enduring the group can go around any obstacles such as large rocks, impassable tree groupings, and small creeks.
4. Once you have plotted the correct course, store the compass and map in your backpack with the understanding that you are not allowed to use them again, until you have reached the picnic area.
5. Have the entire group read the instructions, then set out on your hike.
6. You are only allowed to check that everyone is on course by re-reading the instructions that you wrote down, every 100 feet – when the team needs to change direction to avoid an obstacle.
7. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 2: Observe, Plan, Walk, Start Over, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet, ensuring everyone in the group understands the directions.
4. Place your compass and map on the rock, then hop down from the rock. Re-join your group and lead them in the right direction for one hundred feet
5. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
6. Travel the one hundred foot distance with your team, ensuring that they end up in the correct location.
7. At the end of the one hundred foot walk, have the group stop where they are.
8. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
9. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
10. With the help of the ropes to guide you, walk back to the rock at the beginning of the trails, by yourself, and climb back on to it.
11. Plot the next one hundred foot leg of the journey from where the group currently is, using your map and compass.
12. Repeat this process from step 3 through 11, until your entire group has reached the picnic area.
13. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Group 3: Observe, Plan, Walk, Verify; Repeat;

1. Stand on top of the large rock at the beginning of the hiking trail.
2. Once you have a clear view of the area and general understanding of how to get where you are going, plot a course of one hundred feet using your map and compass.
3. Using any means necessary (pencil and paper, verbal communication, etc), describe the direction that the group is supposed to be walking for the next one hundred feet.
4. Place your compass and map in your backpack, then join your group.
5. Let the group know where they are heading (the end goal) and where they need to be in the next one hundred feet, ensuring everyone in the group understands the directions that you outlined.
6. Tie one end of a rope to a tree or post at the beginning of the one hundred foot walk.
7. Lead the group in the right direction for one hundred feet, ensuring that everyone is able to safely navigate any obstacles.
8. Verify that that your group has stopped in the correct location, by using your map and compass and any other landmarks or means needed.
9. Tie the other end of the rope to a tree or post at the end of the one hundred foot walk.
10. Have you and your group observe the new surroundings and take note of any obstacles that were note previously seen, and offer suggestions on where to go next.
11. Repeat this process from step 3 through 10, until your entire group has reached the picnic area.
12. Once your group has reached the picnic table, record the time that it took for the hike to be completed by everyone – yourself included.

Predictions on Outcome?

• Which group do you think will make it to the picnic site first?
• Which group do you think will be the most happy with the way the group was organized and run?
• Which group of people do you think would be most likely to do the same process on another hike?

Observations of the Groups

Group 1 didn’t get to the picnic site very fast and wasn’t very happy about the experience. While everyone in the group was standing around waiting, the Pathfinder was sitting on top of a rock, writing things down in a notebook, trying to describe – from a very high level, where people were supposed to be walking. When the Pathfinder finally came down from the rock – possibly an hour or more after he climbed up there – the initial directions were good and people could start finding their way. As time went on, though, the detail of where they should be going became more and more unclear, because the Pathfinder could not nee the actual pathways below the tree tops and behind the large rocks. This caused Group 1 to constantly stumble and struggle, creating a situation where the team had to find paths around obstacles that they couldn’t previously see. Every now and then, they got lost because the description of the landmark was insufficient from the distance that it was recorded. They eventually got to the picnic site – well after everyone was hungry and tired. In the end, no one was happy with the experience and just wanted to go home.

Group 2 had an easier time finding their way. Since the Pathfinder and the group were able to see all of the obstacles within one hundred feet, the directions that the Pathfinder described was much more enjoyable. No one stumbled over anything or got lost. But it was a difficult journey, at times. The group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. The problem, though, is that the group took forever to get to the picnic area. At first, having the Pathfinder re-check the direction they were heading was great. Everyone knew that he was keeping the group on target. However, every time the group reached their next stopping point, the Pathfinder would gather information about the next hundred feet, across a wide arc since he didn’t know which direction to go next, with great certainty. Only then could he travel all the way back to the original rock to verify that everyone was on the right path, heading toward the picnic table. After a few iterations of hiking, the process of verifying the course and location took more time than the process of hiking. The Pathfinder was simply repeating the same hike over and over again, to verify that they were on the original path laid out. No one in this group was disappointed in the path that they took to get to the picnic table, but no one appreciated the standing around and waiting for hours on end – waiting for the Pathfinder to get the next one hundred feet plotted. When they finally reached the picnic table, group 2 was full of hungry, grumpy people who just wanted to go home.

Group 3 got to the picnic site in record time. It was a difficult journey, at times – the group had to veer off the original course many time, in order to avoid some previously unknown obstacles. In the end, the Pathfinder did a great job of keeping everyone focused on the goal – the picnic table and food. With the ability to see all of the obstacles in the surrounding area, and the ability for the group to make decisions that would get them around obstacles while still making progress toward the picnic table, the group was successful in getting to the picnic table before lunchtime. Everyone on the group loved the trip and appreciated the ability have input in how they got to the picnic table. As long as the direction that they headed for the next one hundred feet did not cause a serious delay in reaching the picnic table, or was justifiable by avoid an obstacle, all input was considered before setting the new direction. Several of the people in group 3 decided that the would take a few more hours and head off into the next set of hiking trails, trying to reach another picnic table before dinner time. Other decided to bring their family out to the trail the next day, and re-walk the well defined path (noted by the ropes tied to the starting and ending points of each leg of the journey). Overall, everyone in group 3 enjoyed the experience.

The Real World

Now imagine that the Pathfinder is represented by some aspect of your organization. It may be the company or project leadership; it may be the customer; it may be an individual person or group of people within a project team. Your Pathfinder may not be a person or other sentient being, though. Perhaps its the Methodology (capital M, as defined in Peopleware) or political environment in your company.

Which Pathfinder(s) do you have in your company? Which Pathfinder would you rather work with? More importantly – if you are not working for / with the Pathfinder that you want, what steps are you taking to correct that? Be a catalyst for change – appropriately and professionally.

Continuously improve – always within the context of the goal.

Let's assume that we are doing the appropriate amount of testing during our development process. If we include TDD, test automation, test engineers and customer acceptance testing, we should find the majority of the bugs in our system before they are released. However, not every bug will be found. There will be some situation that no one thought about before. There will be some special circumstance on someone's computer that hasn't been accounted for. There will be some client data that does fit the expected variance, despite the data being valid. The point is, there will be something that breaks after we deliver the software. Worse yet - it doesn't even take delivery to find bugs. What happens when the software gets to customer acceptance and the customer says that something is wrong, broken or whatever? We simply must account for the inevitable bug fixes and emergency patches in our system.

Our current kanban board, with all it's pipelines and queues, hasn't addressed the need to address problems. But before we get into any changes to the board, we have to define and create the culture of quality that we need in our team.

A Culture Of Quality

There are many different aspects of quality and many different ways to view and interpret quality. The individual specifics, as always, come down to your team, the project and the needs of both. Even with these variances, though, there are a few key concepts that should be found in any culture of quality, including Whole Team / Collective Ownership and Zero Defects.

Collective Ownership

I've talked about this concept before (see above link) and would recommend reading those prior posts. One thing I would like to add, though is that collective ownership can only success when the team takes their individual egos out of the equation. We must be able to accept criticism as a way to improve ourselves and our team. When we let go of our ego, we can be honest with ourselves and others, enabling everyone to own every part of the system.

Zero Defects

This is a subject that I have not talked about before, but is prevalent throughout the continuous improvement philosophies that I subscribe to. Wikipedia lists four principles of Zero Defect methodologies, all of which are paramount to our culture of quality.

1. Quality is conformance to requirements
2. Defect prevention is preferable to quality inspection and correction
3. Zero Defects is the quality standard
4. Quality is measured in monetary terms – the Price of Nonconformance

Don't be confused by "requirements" in item number one, though. In the world of software development, the word "requirements" has many meanings - not simply the feature list or functional descriptions set forth by our customers. Our teams and processes have certain requirements that are imposed on top of, and into each of the business requirements for the software. These often include the use of source control systems, test driven development, principles like S.O.L.I.D., etc.

Quality Checks in Kanban

Collective ownership and zero defects are only the tip of the iceberg. There is so much more to a culture of quality and so many different aspects of quality to account for. However we define quality, though, our ability to create a culture of quality is necessary for our kanban system to work. Without it, we lose the continuous improvement and elimination of waste that defines Lean. With it, though, we can introduce some specific tools to our process and improve our kanban system by having it handle the errors that are found in our systems.

Andon

From Wikipedia:

"a system to notify management, maintenance, and other workers of a quality or process problem"

When an issue is found in our system, we need to notify the team immediately so that the problem can be taken care of. The idea in a manufacturing line is to allow a worker to stop the line when a problem is found. In our grocery store example, andon could be invoked by a shelf stocker noticing a loose nut or bolt on the shelf and then notifying a maintenance person. In software development, andon can take many different forms. We report our issues list during our daily standup. We create issue tickets in our issue management system. If we're lucky and work in a company that supports the idea of a team room, we may just need to look up or turn around and let the team know that we have problems. We could even place an actual red flag on the desk of our developers, testers, customer representatives, etc., and have them raise that flag when they see a problem. Any of these ideas, plus many many more, can all be our andon system.

Andon itself is not intended to be an all encompassing electronic system with metrics and reports and yadda yadda yadda. It needs to be simple. It needs to be easily employed by anyone on the team. And it needs to mean something to the team. If a team member throws their andon card out on the table, the culture of that team needs to be ingrained with the knowledge that work may stop until the problem is addressed.

There's so little to andon, yet so much more than what I've described. However andon is enabled, it is critical to our zero defect policy in software development.

Jidoka

AKA "Autonomation", AKA "Automation with a human touch", AKA "intelligent automation", AKA ...

From Wikepedia:

"Autonomation prevents the production of defective products, eliminates overproduction and focuses attention on understanding the problem and ensuring that it never recurs."

I've talked about Jidoka in previous posts and would recommend reading them, at this point.

• Jidoka - If it's broke, fix it now
• Jidoka - More than just 'if it's broke, fix it now" - fix it and inform the team

I don't have much else to add, other than to say that Jidoka and Andon go hand in hand. Automated build servers (such as CCNet, TeamCity and many others) can often combine Andon and Jidoka for us by giving us instant visual feedback when something is broken. I've also recently set up BigVisibleCruise in my team area, giving even the casual observer the knowledge of whether our builds are broken or not.

Applying Andon and Jidoka to Our Kanban Board

Once we have the concepts of Andon and Jidoka in our team and culture, we can use these tools to generate issue cards and then look at a few possible changes to our kanban board to account for them. The three basic methods that I have seen used include:

1. Creating an Emergency Fixes pipeline
2. Tacking a smaller bug notice onto an existing card
3. Putting a Bug card in the backlog

I'm sure there are other alternatives, too. In my current team, we use the first and third method and are considering the second one as well. 

Options two and three essentially require no change to our kanban board. Implementing option two is a distinctive way of visually attaching a bug notice to one of the cards in our system. This could be done with little bug stickers, little red cards, or any other visual indicator that the team agrees on. Option three also needs something distinctive. Since we are creating an entire card for the bug, though, the entire card should be distinctive. I would recommend using a card that is colored red to signify issues. I would also recommend prioritizing the bug to the top of the backlog queue, when using option three. This will ensure that the bug gets worked as quickly as possible.

Option number one can also make use of number two and/or three. When we move a card into the Emergency Fixes pipeline, we may want it to be distinguishable as an issue by tacking on our bug symbol or by using a colored card.

Emergency Fixes Pipeline

Depending on the complexity or severity of the bug, we may want to include Analysis and Customer Acceptance in our Emergency Fixes. With andon and jidoka in mind, we will want to ensure that we fix any emergency issue immediately. This is not always possible, however. The customer may decide to delay the fixing of an issue for whatever reason. This leads us to only needing a single pipeline for Emergency Fixes, letting us set our limit to one.

We can easily add an Emergency Fixes pipeline to our kanban system, placing it directly underneath our existing WIP pipeline. This special pipeline can be designated with a name, color code, or other marks as needed.

If you are supporting production releases, you may also need to include a Delivery queue specifically for emergency fixes. This would allow multiple fixes to be compiled into a single patch release. There are other configurations to this, of course. As always, you will need to find what your specific team needs and create your process to suit.

Where Do We Go From Here?

With an Emergency Fixes pipeline in place, our kanban system is now set up to handle just about every situation that we will encounter. However, this does not mean that our system is perfect or truly complete. No process, no matter how well defined it is, is worth anything if the people running the process do not believe in it. I've said it before and I'll continue to say it - never stop improving your process. Always be mindful of waste, friction, smells, problems or whatever you want to call it. Inspect, adapt and continuously improve your process. Perfection is a journey, not an end-goal.

Stay tuned to my Adventures in Lean series, as I continue to explore the various aspects of lean software development in my own team and company culture.

In Part 1 of Kanban in Software Development, I introduced the concepts of kanban boards and pipelines. I also showed a very simple example of creating a pipeline for our development process. However, there were some obvious limitations in what I showed. The reality of software development is much more complicated than three simple steps (Analysis, Development and Testing). A software development team or company is going to have more to do than just these things, and there is usually a need for some team members to be cross-functional. Some team members will have to do development and testing, or analysis and development, or documentation and delivery, or any other combination of steps involved in creating software. In this post, I'll address these issues, creating a more complete pipeline and finishing our kanban board.

Working In A More Complete Pipeline

To create a more complete kanban board, we need more than just a three step pipeline. We need to allow for a fully functional team - developers, analysts, testers, technical writers, and others. We also need to allow the different team members to work on different parts of the system, as work is needed. The end goal is to enable the system to flow through the process and to ensure the work is "done done" before it goes to the customers.

For a more complete software development pipeline, let's use the following columns: Backlog, Analysis, Development, Documentation, Testing, Customer Acceptance, and Delivery. We can put together a pipeline diagram that follows these steps.

Now, let's assume that we have the following team (ignoring project managers, customer representatives, etc, for now): 6 developers, 2 business analysts, 3 test automation engineers, and 3 technical writers. Given this team, we would not be able to make a feature flow through this pipeline. We would have developers sitting around, waiting for work from the analysts. And, our documentation writers and test lab people would probably be pulling his hair out from boredom then pulling their hair out from too much work, in an unbreakable cycle.

Fortunately, we can account for the team makeup and the potential bottlenecks by introducing the concept of order points and limits that we saw in the grocery store, and by creating queues and multiple pipelines to be worked.

The Real World - Multiple Pipelines and Limits

In most software development efforts, it's unreasonable to expect an entire team to work on only one feature at a time. Most development managers want to maximize the throughput of development by working on tasks in parallel. Given this desire, we'll divide up our developers into three teams of two developers each. This development team division should allow three features to be developed at the same time. Since we want to work on three features at a time, we will need three pipelines for work.

These three pipelines constitute our first limit - we can have a maximum of three features in development at any given time. This limit is noted by putting a "3" in the upper right hand corner of the development column header, as shown above.

Having three development pipelines puts us in a tricky situation, though. We have very few business analysts compared to developers - even with our developer teams, we have less than one analyst per developer team. Fortunately, development work often takes longer than analysis. We can use this knowledge to our advantage and let the analysts work slightly ahead of the developers, creating a queue of work to be done.

Handling Bottlenecks - Queues, Limits and Order Points

If all three development teams happen to finish at the same time and need more work, we would need a minimum of three features that are ready to be worked. However, since we know that requirements change regularly, we don't want to get too far ahead of the developers. With this in mind, we can introduce an Analysis order point of three (the number of development teams) and limit of five. This gives the analysts the ability to work ahead of the developers and also makes them responsible for keeping work available for developers.

For an order point we'll introduce the number into the upper left hand corner of a column header, and we'll continue using the upper right hand corner to specify our limits.

Since we are now dealing with potentially more work in analysis than development, it seems we would have to increase our pipelines to five. This is not desirable, though, since we only have three development teams. What we will do, instead, is restructure the pipeline and turn the analysis column into a queue.

Since Analysis is now a queue, there does not seems to be a need for Backlog to be a step in a pipeline, either. The backlog is simply the list of features that the customer is expecting to be done next. Therefore, we will also turn the backlog into a queue - this time, with a priority (top of the board is highest priority) allowing the customer to tell us which specific feature should be worked on next. We will also want to keep the backlog column limited, to prevent the team from having too much information to think about. Since we only have two analysts on our team, it seems appropriate to keep at least two features in the backlog column at all times. With this in mind, we can safely set a backlog order point of two and limit of five (enough to keep the analysis column full).

The changing of Backlog into a queue has no made our Development column the first step in our actual pipeline.

Completing The Pipeline - Aggregate Limits and Work Cells

The next few column - Documentation and Testing - both have an easy amount of team members to deal with. There are three technical writers on our team, and three testing personnel. This distribution lets us keep the pipeline in tact between Development, Documentation and Testing, allowing us to set the same limit as Development (three).

With all three of these columns having the same limits, and with each column being properly staffed so that three pipelines could run at the same time, it makes sense to combine the team members from these three columns into a single workcell. If we consider all four team members (2 developers, 1 tech writer, 1 tester) as a team and allow them to work together as a team, we can aggregate the limits of the columns in the pipeline. We can also create a consolidated name for this pipeline - work in progress.

This consolidation can be shown by creating a parent / child header with the limit shown in the parent. We will also use a dashed lines between the columns of the pipelines. The combination of these two details will show us that we are dealing with a pipeline, and how many pipelines are allowed to flow, simultaneously.

By aggregating the pipeline limits, we allow the workcell for a given feature to focus exclusively on that feature. The development staff, technical writers, and test lab personnel will all be working on the same feature, allowing them to more easily share information between themselves. This will prevent the team members from having to switch back and forth between subjects, reducing cognitive load and allowing for greater quality to be attained in the individual feature.

Completing The Kanban Board - Customer Acceptance and Delivery

If the customer is on-site (co-located with the team or close enough to meet every day), then it may be possible to aggregate the Customer Acceptance process into the Work In Progress pipeline. If you can do this, you should. Getting immediate feedback from the customer, while work is still fresh in you mind, is critical to the quality and success of that work. What's more - you may not need this explicit column if you have the customer working with you every day.

Unfortunately, we don't always have the luxury of on-site customers. We may only have the customer around at specific timer intervals or only on request. In either of these cases, we have to account for the bottleneck of Customer Acceptance.  Additionally, the customer may not want to get delivery of individual features. They may want to wait for a quarterly release, or some other regularly scheduled release. Once again, we have to account for this bottleneck. We can apply the same principles that we used for Backlog and Analysis here, and create some appropriate order points limits.

Let's say that our customer wants to do Customer Acceptance testing once a week and wants a Delivery once every calendar quarter. To accommodate this, we can set the Customer Acceptance column as a queue with a 1 week limit. We can also set the Delivery column as a queue with a 1 quarter limit. Setting these limits puts a direct responsibility on the customer. If they cannot pull the features through these columns on or before the specified limits, then the entire system could grind to a halt. It's important to educate the customer in what they are committing to, and ensure that they can fulfill their obligations. This fulfillment may mean that they assign more than one person or group of people to the task of Customer Acceptance and accepting Delivery. Whatever the solution is, the customer has responsibilities to meet.

With Customer Acceptance and Delivery specified, our completed kanban board would look like this:

Put It In Action - A Kanban Process Is Never Truly "Complete"

Now that we have clearly defined our Kanban process and visualized it through our Kanban board, the next steps is to start using it! Start working through this process, pull value from the end of the process and allow work to flow through the pipeline. Respect the order points and limits, and above all, look for problems in the process and fix the process when you need to.

Perhaps you have too few or too many items in one of your queues - adjust the limits and order points. Perhaps the test lab is now a bottleneck - take them out of the pipeline and change them to a queue. There are many possible problems and many possible solutions. The key is to be aware - inspect and adapt and hold regular retrospectives - kaizen your process and continuously improve.

Where Do We Go From Here?

No process is perfect. Anyone that tells you otherwise is selling something. A customer may find a bug during Customer Acceptance; an issue makes it past all of our QA processes and is found out in production; or there's a feature that has a sudden priority over any other feature currently being worked. But our current kanban process doesn't address the natural problems that occur during a software package's lifetime. To address this properly, we'll need to employ the lean tools of Andon and Jidoka, and we'll also need to decide how to visualize this on our kanban board. These issues, and possibly more, will be addressed in another installment of Kanban in Software Development.

In the world of Scrum, XP and other forms of Agile software development, many teams use visual control systems to outline the various steps that software goes through during development. These boards are known by various names -  Scrum boards, card-boards, task-boards, swim lanes, and probably a few other names that I'm not aware of at the moment. Whatever you call it and whatever agile creed you live by, the primary use of these boards is the same all around - information radiation. Anyone can look at the board at any given moment in time, and know where in the development process a given card is. If you're doing iterations or sprints, the board also tells you where you are in that time box - if you're ahead, behind, or on time. The far left column - the iteration backlog - will be filled with cards at the beginning of the iteration. Over the coarse of the iteration, cards will be pushed through each of the columns until they are in the done column. The goal is for every card to be "done done" by the end of the iteration, with a set of features that is now deliverable to the customer.

Kanban Boards

Visual management tools are very common in agile shops, and for good reason - they work very well. It should come as no surprise, then, that kanban software development also employs various visual management tools, including a kanban board.

On the surface, there isn't much difference between an average task board and a kanban board. Each of these boards has various columns that represent the stages that a card needs to go through before it is considered done. The real difference in a kanban board, is not the board itself. The board is just a visual indicator, the same as any task board, and the intention is still to get the cards to the "done done" state - that is, delivered to the customer so that they can use the features from that card. The real difference is how the process is approached - by pulling value through the system.

Kanban Pipelines

When dealing with a kanban process and visualizing it into a kanban board, the various steps that a card goes through is often called a pipeline. A single card starts at one end of the pipe and flows through to the other end. This flow is enabled via the pull system that happens at the end of the pipe.

In the grocery store example from my previous post on pulling value, the pipeline would would likely include the store shelf, the store back room, the warehouse, the supplier and the product creator.

A Simple Software Development Pipeline

A software development pipeline works the same way as the grocery store pipeline. In this case, though, the product flowing through the pipe is likely to be a feature of the software package.

Consider the following three columns in a simple pipeline for software development: Analysis, Development and Testing. When a customer requests a given feature for a software product, they want to pull that feature out of testing so that they can start using it.

Once that feature has been moved out of Testing and the customer is ready to pull the next feature out, there isn't anything to pull. At this point, the Testing people would then try to pull the next feature out of Development.

And the same pull happens from Analysis to Development.

In the end, we have created a pipeline for how our development process works. The work that is done flows through that pipeline based on how often the customer wants to pull features out. As one feature exits the pipeline, another feature can be added into the pipeline.

The key to all of this is, again, pulling the features through the system.

Where Do We Go From Here?

I've briefly shown how we can take three steps and produce a simple pipeline for work to flow through. There is more to software development than just Analysis, Development and Testing, though. We also have to consider team size and makeup, parallel work, and other constraints. In the next entry of Kanban in Software Development, we'll flesh out a more complete pipeline and take the next steps to show how to complete our Kanban board, enabling our customers to pull features out of our development process.

For the last month or so, I've been having a serious problem with the apparent lack of professionalism in the software development industry as a whole. There are many factors contributing to this and many different aspects of the problem. I'm not going to go into any detail on the issues I'm seeing, at this point, though. It will be sufficient, for the moment, to simply state that there is a serious lack of professionalism in our industry.

I've been discussing this issue with my dad, recently, and this morning he sent me an email that I thought was worth sharing with the world.

Today’s thought is to remind the team that life in general is determined by the 2^nd Law of Thermal Dynamics. All matter (systems) go from a state of order to disorder with out any outside input.

[From Wikipedia]:

"The second law of thermodynamics is an expression of the universal law of increasing entropy, stating that the entropy of an isolated system which is not in equilibrium will tend to increase over time, approaching a maximum value at equilibrium.

The second law traces its origin to French physicist Sadi Carnot's 1824 paper Reflections on the Motive Power of Fire, which presented the view that motive power (work) is due to the fall of caloric (heat) from a hot to cold body (working substance). In simple terms, the second law is an expression of the fact that over time, ignoring the effects of self-gravity, differences in temperature, pressure, and density tend to even out in a physical system that is isolated from the outside world. Entropy is a measure of how far along this evening-out process has progressed.

There are many versions of the second law, but they all have the same effect, which is to explain the phenomenon of irreversibility in nature."

 

Bottom line is that the point of natural human equilibrium is far below the level of professionalism demanded by our responsibilities to work, self, family and others.

The Point … doing what’s natural or going with the flow points life in the downward direction every time.  Let’s tweak the line upward… put some thought and energy into it.

Robert D. Bailey   12-04-2008  

A coworker recently asked if we should always abstract every object into an interface in order to fulfill the Dependency Inversion Principle (DIP). The question stunned me at first, honestly. I knew in my head that this was a bad idea - abstracting into interfaces for the sake of abstraction leads down the path of needless complexity. However, I wasn't able to clearly answer his question with specific examples of when you would not want to do this, at the time. I've been thinking about this for a few days now and I think I have a good, albeit very long winded, answer.

Before the question is answered, though, we need to step back and look at what DIP is all about. I've previously shown how to implement DIP and talked about why it's beneficial, so I won't be repeating that here. Rather, I want to talk about the language that describes DIP and what it really means.

Robert Martin's original definition of DIP is this:

A. High level modules should not depend upon low level modules. Both should depend upon abstractions.
B. Abstractions should not depend upon details. Details should depend upon abstractions.

The word 'abstraction' is used three times in the definition for DIP. So, in order to understand DIP, we have to first understand some of the basics of Abstraction.

Some Background On Abstraction

From Wikipedia (emphasis mine):

"In computer science, abstraction is a mechanism and practice to reduce and factor out details so that one can focus on a few concepts at a time"

I can't say it any better than this.

Abstraction can very directly lead to a system that is more understandable by helping us ignore the detail and implementation specifics, allowing us to focus on something at a higher level. This reduction in cognitive load can benefit someone that is reading the code by not forcing them to know the detail immediately. Additionally, abstraction is a form of encapsulation or information hiding, which again helps us to reduce cognitive load and produce better systems. From Wikipedia's entry on Information Hiding:

"In computer science, the principle of information hiding is the hiding of design decisions in a computer program that are most likely to change, thus protecting other parts of the program from change if the design decision is changed."

At the heart of abstraction and information hiding, we find the ability to change the system. The ability to change is an absolute requirement in software development and produces good design that is easier to work with, modify, and put back together as needed. The inability to change is directly called "bad design" by Robert Martin, in the DIP article.

Applying Abstraction And Encapsulation To DIP

So how does our knowledge of abstraction and information hiding play into DIP?

First and foremost, DIP never states that we should depend on explicit interfaces. Yes, in C# we have an explicit Interface as a form of abstraction. It is a separation of the implementation detail from the publicly available methods, properties, etc, of a class. Some languages, such as C++, don't have an explicit construct for interfaces, though. From Robert Martin's original DIP article, again:

"In C++ however, there is no separation between interface and implementation. Rather, in C++, the separation is between the definition of the class and the definition of its member functions."

A String As An Abstraction

Abstraction does always mean explicit interface constructs, as evidenced in C++. Nor does it always mean an abstract base class, which we also have available in .NET. In fact, languages such as Ruby don't really need either of these constructs. The duck-type nature of Ruby allows an implementation to be replaced at any point, without any special constructs. In .NET, though, there are a number of abstraction forms that we can rely on, explicitly. We have the obvious interfaces and base classes (abstract or not) - but we also have constructs like delegates and lambda expressions, and even the simple types that are built into the base class library.

Let's look at a simple string to illustrate abstraction. As I said in my SOLID presentation at ADNUG, we can invert our dependency on database connection information.  Rather than putting a connection string directly into our code that calls the database, we can use the string as our dependency and our abstraction. All we need to do is follow the basic DIP principle and provide the string as a parameter to the class that calls the database. We certainly don't need (or want, for that matter) to introduce a new interface or base class at this point. Our abstraction is simple enough to use a common type found in the .NET framework.

Other Forms Of Abstraction

Even if we are talking about an object, who says that the interface we are depending on has to be an explicit interface construct or base class? When I write a Domain Service that uses an Entity from my Domain, I don't create an explicit interface for that Entity. Rather, I use the Entity's inherent interface - it's public methods, properties, etc.

I also use delegates on a regular basis. By specifying my abstraction as a delegate, I can further decouple the depending object from the dependant code that it needs to call. I'd be willing to bet you have used delegates as abstractions as well. Have you ever created an event handler for something like a button click? There's a delegate's abstraction at work.

Abstract Judgement

The point is, there is not always a need to introduce an explicit interface or base class when inverting our dependencies. We still need to apply dependency inversion and provide our implementation as a constructor parameter (or setter, though I don't like setter injection). But, that dependency doesn't have to be anything more than the interface inherent to the object, or a simple type found in .NET.

You do have to be careful when making the call to not use an explicit abstraction with DIP, though. You can quickly turn your system into a ball of mud if you rely on a concrete class that is not intention revealing or well encapsulated to begin with. At the same time, too many abstractions can lead to needless complexity and make it very difficult to see the big picture of a system. Too few abstractions, though, will certainly lead to a rigid, immobile design that is hard to change. All of these problems are equally vicious - and I've been bitten by all of them in recent months.  It takes good judgement calls to determine when you do and do not need an explicit abstraction for a dependency. Unfortunately, good judgement comes from experience and experience comes from bad judgement. Don't be afraid to make bad decisions - make a decision, just be sure you can reverse that decision as easily as possible.

After some additional Twitter discussion with Jimmy last night, I realized that my previous response had neglected to distinguish between two very important contexts - new code and legacy code.

Considering a legacy system (as defined by Michael Feathers), I'm 100% on board with everything Jimmy is talking about and all of the techniques, troubles, and diminishing returns that he identifies.

I still maintain that 100% coverage should be the default result of true TDD when writing new code - and I apply this philosophy when writing new code inside of a legacy system. Write a test to prove you need the new code that you are writing... unfortunately, the distinction between new and legacy code does get blurry and can be difficult to navigate when working in a legacy system.

For more info on legacy code, in this context, and how to deal with it:

• Working Effectively With Legacy Code - the book by Michael Feathers
• Working Effectively With Legacy Code - Article on ObjectMentor by Michael Feathers (consider it the cheat-sheet for the book)
• Jimmy Bogard - He's done a "Legacy Code" presentation several times, based on his own use of the Michael Feathers techniques.

I've talked about code coverage and software quality with a number of people on Twitter in the last month or so, including Jimmy Bogard, who has a nicely written post about Quality and code coverage.

It's no secret that I have some fairly radical opinions on code coverage and that I don't agree with Jimmy in these regards. I firmly believe that 100% code coverage, in whatever form we can get, is a reasonable goal and not past the point of diminishing returns. As such, I'd like to respond to some of what Jimmy is saying and expand on it with my own thoughts.

But first - I don't like to draw hard lines between "Unit" tests and "Integration" tests - that just blurs the problem even more because we can argue that you are covering code or not, in "unit" tests all day long, based on this blurry line. This is not the discussion I want to have, so I am lumping "Unit" and "Integration" tests into "Unit" tests as a whole.

Secondly, I'd like to say that code coverage is not always an indication of quality - in poorly written systems, it's merely a measure of code coverage. However, quality code is another subject of much debate.

On to the response!

.............................................

"The idea is that the team, all practicing TDD, should dutifully measure and add unit tests until they reach the assumed pinnacle of unit testing: 100% coverage."

There are several assumptions and statements in this that I don't believe are true.

For one, TDD and Unit Testing are not the same - no where near the same. Unit Testing lets you add tests as you see fit - after you write the code, while you write the code, whenever you want. You can even do test-first unit testing. True TDD, however, is a pure pragmatic approach to software development in that you never write code that does not have proof of need to exist - a test that says it needs to be there. 

The "pinnacle" of TDD and Unit Testing are not the same thing. Unit Testing - the act of adding unit tests when you see the need - may have a goal of 100% coverage. TDD, on the other hand, has a goal of only implementing the code that has been specified as needed. 100% coverage is the default in TDD because you never write code you don't need.

If you are going through the "measure and add unit tests" process, I would put money down to say that you are doing Unit Testing and very little TDD. The process should be something more like "measure to make sure we haven't added anything we don't need yet". 100% coverage is a side effect of TDD.

"The general motivation behind 100% coverage is that 100% coverage equals zero bugs."

Unfortunately, this is the motivation for many people who write unit tests - but it's the wrong motivation. You will never be 100% bug free just because you have 100% code coverage. There will always be some business case or external system factors that cause a bug now and then. That doesn't mean we should be ok with letting bugs into our system. On the contrary, one of our motivations (not the only one) should be to prevent as many bugs as possible from getting into the system.

"NCover is a powerful tool, but it still doesn't support all types of coverage.  Attaining 100% coverage in NCover still means there are paths that we haven't tested yet, which means there are still potential bugs in our code. "

I think the logical conclusion of this argument is that if we don't have a tool that supports true 100% coverage, then we shouldn't use that tool at all. I know I'm inserting my own conclusions into Jimmy's words - that's simply the conclusion that I came to from these statements. And this is a very bad conclusion. Do you walk around naked simply because wearing clothes would get them dirty and you'd have to wash them? Both of these are akin to the broken window syndrome. Use the tools that are available to the extent that they can operate, and find better tools when you can.

"If 100% coverage is a goal,"

100% coverage should NEVER be a goal - it's merely an indication that you are on the right path and working pragmatically.

"In recent projects where we measured coverage several months in to the project, we saw regular numbers of 90% coverage.  This was on a team doing 100% TDD." ... "So are we doing TDD wrong?"

So, are you doing TDD wrong? Yes. You are unit testing in the guise of Test-First-Development and not adhering to the spirit or pragmatism that TDD wants.

"Every test introduced covered behavior we considered interesting.  If behavior isn't interesting, we don't care about it. "

If you have code that is not interesting and you don't care about it - why is it in the system? If you don't care about the code, then it makes no difference whether or not that code works correctly. I would be appalled to hear that this is true. If it is true, then that code should not be in the system to begin with.

"If tests are a description of the behavior of the system, why fill it with all the boring, trivial parts?  The effort required to cover triviality is just too high compared to other ways we can increase value."

I think your definition of "behavior" is wrong. From wordnet.princeton.edu/perl/webwn:

Behavior: "the action or reaction of something (as a machine or substance) under specified circumstances;"

So, how is throwing an exception when a parameter is null, not behavior? If you are doing defensive programming to make sure you have all the data you need, then you are creating behavior - system behavior, not business process behavior. Behavior is still behavior, though. The only possibility of code not being behavior is a simple data point. If you have a data point in your code that is not covered by a unit test of behavior, then you don't need that data point. Yes, yes, I know - "integration", "nhibernate", "not my data source", "it has to be there for ...". If you argue that you need a property for some external portion of the system to operate correctly, then you should be proving that you need the property by covering it with a unit test that specifically says what it is for and shows how it is used. If you don't, then there's no way of knowing why the property exists and it may be deleted because it looks like nothing in the system uses it. Or, possibly worse, you end up with a lot of dead code in your system because you are afraid of deleting anything.

"Missing in the 100% coverage conversation is the effort required to get to 100%.  Attempting to get another 5% takes equal effort of the previous 10%.  The next 2% takes equal effort of the previous 15%, and so on."

I'd like to know where you got the math. If you are talking about unit testing - even test-first unit testing - i can understand the pain you're talking about, here. Why should I bother setting up another test fixture and getting all the state of my objects in place for a simple null reference check? blech - that totally sucks, is boring, is tedious, etc.

However, my typical process of achieving 100% is to only write code that is needed for the system to work, by specifying how the system will work through tests and then coding it. If I have a null reference check somewhere in my code it should be because I already have a test that specifies why it's needed.

"This is called the law of diminishing returns.  As we get closer and closer to 100%, it takes vastly more effort to get there.  At some point, you have to ask yourselves, is there value in this effort? "

Here's the real crux of the problem - you are assuming that you started with, or at some point has less than 100% coverage. It takes zero additional effort to stay at 100% coverage if you start at 100% coverage and maintain it through pragmatic TDD.

"Often, bending code to get 100% can decrease design quality, as you're now twisting the original intent solely for coverage concerns, not usability, readability or other concerns."

If you find yourself bending code to get 100% coverage, you have one of two situations (if not both): 1) poor design in your code, 2) poorly written tests. I would bet that you have both because they are a vicious circle.

"Measuring coverage is an interesting data point, as are other measures such as static analysis.  But in the end, it's only a measure, an indication. "

100% agreed!

"It's still up to the team to decide on the value of addressing missing areas,"

This implies that you don't have 100% coverage, already. In this situation, 100% agreed. I'm in this situation right now - it's hard to know what tests create the most value, at this point. However, any new code - even code changes to existing code - are done TDD style. So, while we don't yet have 100% coverage, we are increasing coverage all the time by never writing code without a test (that's part of the goal, anyway... no one is perfect, and it takes serious discipline to do this)

"with the full knowledge that they are still limited to what the tool measures."

Agreed, again. If you don't know the limitations of the tools you are using, you're in trouble.

.............................................

The boilerplate argument that I am making is that you should never write code that is not required by a customer / consumer. However, as I've Previously Talked About, there's a high likelihood that you have not identified all of your customers / consumers. Take NHibernate, for example. I have a project that needs about 30 different properties for a specific NHibernate query to be executed. The NHibernate query is the only place they are ever used - no code reads them, only a search screen writes to them, then the NHibernate query loads data from the table that they are mapped to. When I first wrote this code, my coverage was at around 30%. This situation is one of many that I've been in recently, that has lead me to believe that other code must be considered a customer / consumer of your code. If I deleted any of these properties, then the NHibernate query would fail. Therefore, the properties are valuable to NHiberate. Since these properties are valuable to at least one of the consumers of my code, I should prove that they need to exist by writing a test that proves they are needed and why.

There's been a lot of recent talk about what is "done" in the Lean / Agile development communities, and the primary focus of these discussions has been focused on individual features or stories and the need to get them completely "done" (dev, test, acceptance test, documentation, delivered to customer) before they can be considered done. I wholeheartedly agree with this philosophy, to the point where I active introduce painful elements of software development to my team members, because the work must be done before we can deliver to the customer.

All that being said - I think we're missing out on some potential benefit by not applying "done" at various and different levels of our software development projects. For example, the concept of "done" can be applied to an iteration. What constitutes an iteration being "done"? It's certainly not just the time-box of 2 weeks, 1 month, or whatever our iteration length is defined as. However, that time box is still important. We don't want arbitrary lengths of iterations.

So how do we know when an iteration is completely "done" versus just being over? My initial thoughts revolve around a pass/fail checklist, similar to the swim lanes (or kanban board, if you want to call it that) that our stories go through. At this point, I would likely include the following:

• Iteration length passed
• All stories in iteration "done"
• Code reviewed (possibly part of story being "done")
• Software is in a stable, working state
• Software acceptance tested by customer (or customer representative / SME)
• Retrospective held

There's probably some additional items to include here - this is just my initial idea list.

Like the user story "done" criteria, an iteration cannot be considered "done" until all of the items in this list have been checked off. We don't give partial credit for a story making it to "in test", when there are 2 more columns to move through - the story is not done until it's completely "done". The same should apply to the iteration.

Why bother making an official "done" list for iterations? For the same simple reason we do this for user stories - transparency and visibility into bottlenecks and problems. It's easy for a team to ignore problems like broken software at the end of an iteration - "oh, it's just XYZ... i'll take care of that tomorrow" - when we don't have the same accountability as we do in our stories. By making the iteration a pass/fail set, we expose those problems for the world to see. This exposure is a great motivational tool - who wants to be caught with an un-"done" iteration because of a "simple" bug?

Do you and your team have an iteration "done" list? Do you even have a user story "done" list? It's all about exposing weakness and waste, then eliminating it and creating the official iteration "done" list is just one more step along the path. Don't stop there. What other processes - higher or lower level - can we standardize and improve?

A Food Network Challenge

During this season's 'The Next Food Network Star', there was an episode where the contestants had to cook a dish and review it in front of a camera, describing it's visuals and taste. The twist to the challenge was that they were assigned someone else's dish to review, after everyone had already cooked. Once the assignments were made for who was going to describe who's dish, they were not allowed to see the dishes anymore. A given contestant was then put in front of a camera and had their assigned dish uncovered in front of them. From that point, they had 90 seconds to describe the dish that was in front of them - that included figuring out what it was, what it looks like, what it tasted like and what it smelled like. This was a rather tough challenge to watch, and a few of them didn't do so well.

A Code Review Challenge

What if we take the Next Food Network Star's challenge and apply it to code reviews? When we sit down in a code review session, rather than having the person who wrote the code doing the driving and telling us what is going on, have someone who has not seen the code do the driving - read the code, read the unit tests, etc - and describe to us what is going on. 90 seconds wouldn't be nearly enough time, though, so we'd have to change that time limit based on the size and complexity of the functionality being reviewed. Other than that, though, it seems like it would be a great way to test the team's ability to pick up an area of the system that they have not seen before and quickly learn it so that they can work in it.

Standardized Work - Judging Code And Code Reviewer

There are 2 real benefits to the code review challenge, as I see it:

1. Judging how well did the coder apply our coding and development standards
2. Judging how well the reviewer can recognize and describe the standards being applied

We are now building our system from User Stories, with Context / Specification style unit testing. One of the intents of this style of unit testing is to provide an easy-to-read and easy-to-learn set of examples for developers that need to know how to do something or how something works in the system. If the unit tests are properly written and expressive of the code's intent - it should be really easy to see what's going on. If we have good standards in place, a developer should be able to find the code in well organized areas, easily, and know where to look for the various components and parts.

It seems to me that the code review challenge would help to prove a number of these key factors in our projects and our teams:

• Coding and architecture standards
  • How well the standards are known
  • How well the team follows the standards
• Expressive Code
  • How easy is it to read the code
  • How easy is it to understand the code's intent
• Maintainable Code
  • How easily can the code be changed and extended
  • How quickly the team can socialize and collectively own the system

A Challenge In Name, A Learning Experience In Practice

I would avoid applying any sort of point system or other reward / incentive for the code reviews. It seems that this could easily backfire and cause junior team members (in skill or time on the team/project) to feel intimidated or feel that it's an unfair process comparing them to the senior level team members. The intention of the code review challenge is not to cause dismay in any team member, but to create a learning environment by continuously challenging every member of the team. Ultimately, the code review challenge should foster the socialization of the system and lead to a strengthened sense of collective ownership, resulting in a better system.

Nothing New Under The Sun?

With all that being said - I'm doubtful that I'm the first person to ever hold a code review like this. I'd be interested in knowing if there's any standard development practices (Agile or otherwise) that are heading in the same direction. Any advice or opinions of you, the reader, would also be very welcomed.

If you know of a resource that describes the same basic ideas or if you have any input to help me improve the idea, please leave a comment and let me know.

Collective Ownership

One of the principles that I’ve tried to instill in my team(s) is that we are a collective ownership team. I've talked about Collective Code Ownership before, and I still believe it is needed to have a solid team. My views have recently expanded a little to include more than just the software developers on the project team - crossing all functional boundaries in a given project team.

Collective Ownership means that there is no one person on the team – developer, ba, tester, etc. – that "owns", controls, or is solely responsible for any one portion of the system. Code, documentation, and all other artifacts are owned by the entire team, as we should be working as a team to complete the project.  Obviously there are people who specialize in one area or another, so they will have more influence in those areas. Depending on the corporate culture that you work in, there may also be some boundaries that we can’t cross due to politics, etc. At the very least, though, every last line of source code in a project is owned by the team and is the responsibility of the team. The moment any artifact (code, images, documents, etc) is committed to source control or any other common project storage, the person that wrote/created/modified it is no longer directly responsible for it – the entire team is, and anyone on the team that sees a problem is expected to fix it or call attention to it so it can be fixed.

This principle is key to the team being able to create great software. We don’t want people to say "my code is better, and my area of the application works, but your code is bad and doesn’t work". This kind of divisive thinking will create a poorly constructed system that doesn’t fit together very well. We need to ensure that the entire team is saying "our code is good", or "our code needs work", and "I can help improve the team’s code base".

Your Code Is Terrible Because My Code Is Terrible

I’ve heard several times that a code base is only as good as the weakest developer on the team. I believe this is true – especially in a collective ownership environment. The moment a developer commits code, it is available for any other team member to modify, and therefore the quality of the code is immediately suspect and subject to the standards of the weakest team member.  Despite the potential problems that collective ownership creates, its very nature should help to bring the skill level of every developer on the team up, alleviating the 'weakest code' problem.

When a 'strong' developer sees the code of a 'weak' developer (those terms being relative to a lot of factors, of course) the strong developer should instinctively want to help the weak developer so that they are both producing better code. When a weak developer sees code that they believe is strong, they should instinctively want to mimic the qualities of that code and either ask for assistance or work to understand how to produce it on their own. Though it does take time and a lot of hard work, the end result should be that the collective weakness of the team disappears. The 'weakest' developer on the team will become an unknown – even irrelevant. Everyone will have strengths and weaknesses, but the team will balance each other out and the code that is delivered will be good.

Socializing The Design

For collective ownership to work, though, we have to ensure that the entire team is comfortable with the entire system. Each team member must be able to understand, update, and "own" any part of the system that they are currently working on (ownership being transitive, and forfeit once they commit the change, of course). For an entire team to be comfortable with the entire system, we need to socialize the design - that is, ensure the team understands the architecture and design fundamentals through the social aspects of working on a team. When we do a good job of socializing the design, it becomes much more natural for the team to collectively own the code. A given developer may not know anything about a given area of the system, but because there are good standards in place and everyone is aware of the overall design of the system, everyone on the team should be able to dig into any given area and learn it quickly.

The real challenge in socializing a design is figuring out what works for your team. There are as many ways of socializing the design, as there are systems and teams - some of which include:

• Code reviews (formal and informal)
• Pair programming
• Email, Presentations, and other “here’s how you do xyz”
• Wiki's, Forums, and other ways of asking “what do you think about xyz or 123”
• Iteration retrospectives
• Etc, etc, etc

No two teams or situations are alike, either. If you are limiting yourself to just a few of these techniques (or only the techniques listed here), you may not reach the full potential of your team's understanding. Try different combinations of socialization techniques and see what works well for your team.

End Goal: Continuous Improvement

In the end, a socialized design and collective ownership enable teams to work more efficiently by preventing the bottlenecks that 'I maintain XYZ' will create.  Code quality will rise, the team work work together, and we can start to move forward into better standards, better practices, and continuous improvement.

Anyone that has worked on a project with me is familiar with my usual "I just fixed … whatever" emails. For example - yesterday, I sent out this email to my team:

FYI – I just pulled down the latest code from subversion, with the new solution and project reorganizations… there were several project reference errors in the {our project} solution – many of the projects were trying to reference the \Trunk\Code\Library\ folder for the compiled output of a project that was in the same solution.

We need to make sure that projects within the same solution reference the project directly, not the compiled dll file. You can make sure this is correct by deleting everything out of the Library folder, than updating it from subversion (to make sure you have no {our project}.dll files in the Library) and then see if you get any missing file errors in visual studio, for the project references.

I’ve been sending out similar email for several years now, and I’ve always felt that it was important to do so, so that the entire team can gain new insights, move forward with better techniques, creating better software, etc.

Yesterday, I found An Article on the Lean Manufacturing concept of Jidoka and a few items stuck out in my mind and helped me to understand why I think that it’s important to call out these items. The core of jidoka goes well beyond the basic idea of "If it's broke, fix it now". It gets much more involved than just that. The article has a list of steps that should be taken whenever something is found to be 'broken' or 'abnormal':

1. Detect the abnormality
2. Stop
3. Fix or correct the immediate condition
4. Investigate the root cause and install a countermeasure

There's a lot more to the real implementation of jidoka than just fixing the problem - finding the root cause and installing countermeasures to prevent it from happening again is by far the most important step in this process. If you fix the problem in isolation and no one else knows that the problem occurred, then it's likely that the problem will happen again for the same reasons. This gets straight to the core of why I send out the “I just fixed …” emails. I found this quote in the article that further illustrates the point:

What I would not want to happen is to expect the team member who discovered the problem to try to fix it without telling anyone. But many times we expect them to do just that, and the rework becomes so deeply embedded into the routine that we can't even tell it is happening. It seems normal because it has never been flagged as abnormal.

I don’t send out those emails to chastise anyone, point out that someone broke something, or anything negative like that. Honestly – I don’t care who broke something; I only care that it is broken, it gets fixed, and we find the cause and do what we can to eliminate that cause. I send out those emails because when we (as a team – any of us; developers, testers, subject matter experts, etc) find a situation that is not right, we need to fix it – but more importantly, we need to notify the rest of the team so that we can learn as a team and find ways to not make the same mistake in the future.

I highly recommend that everyone go read that article and I also want to suggest that any time any of us runs into a situation where something is wrong – whether it’s code that is written or designed poorly, a references issue, the build being broken, or whatever it is – we not only fix the problem, but we inform the team of what the problem is/was and how to fix it and prevent it. We need to have this level of communication in the team, or we are not going to grow as a team or as individual developers, as fast as we could.

Yesterday, I posted a quick thought on code generation, and one of the statements I made is worth re-stating to stand on it's own.

In Agile/Lean software development, a single User Story is one piece flow when implemented via TDD, in a Workcell

In Lean Manufacturing, one piece flow is the idea that you do not produce anything in batches, but that you produce one product from start to finish per a customer's order. In Lean / Agile software development, this is analogous to an iteration with user stories. An iteration backlog is a customer order - a set of behaviors or features that are requested to be delivered in the current iteration. A user story is the single piece that we want to flow from beginning to end - from the start of coding all the way to acceptance via user testing.

In manufacturing, a Workcell is used to facilitate one piece flow through the manufacturing process. In software development, a Workcell is also used to facilitate the one piece flow. The Workcell may be 1 person, pair programming or a three person team. In order for it to truly be one piece flow, though, strict standards such as TDD and automated acceptance tests must be followed.

I'm not going to detail the benefits of one piece flow, here (not yet, anyway... another post for another time). I highly recommend that you read The Toyota Way and Lean Thinking for a good understanding of this concept and the benefits it provides.

There are a few examples in The Toyota Way, of American manufacturing companies that were thought to be world class Lean Manufacturers. When representatives from Toyota began working with these companies, though, the result was quite a shake-up. The companies that were thought to be lean, were merely implementing some of the surface level processes of lean manufacturing. They had not realized that there was a root cause of the process that they were implementing.

The Principles, Not The Process

A coworker recently asked me to help out with a presentation on Model-View-Presenter. He had some specific questions about why we would want to use MVP vs. MVC vs. any other UI pattern. Many of the questions centered around various benefits that have been promoted by myself and others out in the developer world via blogs and articles - items such as testability, changeability, flexibility, and other "ilities". My answers to some of these questions surprised me.

Model-View-Presenter is often thought of as a journey and an end in itself. When I first started learning MVP just over a year ago, this is exactly how I saw it. I started with the intention of learning how to separate my core process from the form implementation, with the intention of being able to unit test the core process. I was working under the impression that I could find a way to implement MVP that would truly allow me to swap out my UI - to the point of changing the workflow within the UI. I had the goal in the mind, of being able to write one presenter and have it support WinForms, WebForms, Web Services, and any other client-facing API.

One year and several projects later, I have some good lessons learned. I have implemented various forms of MVP - automagic view injection, manual everything, and a lot of ideas in between. My understanding of MVP is fairly solid, and I am capable of implementing it in WinForms, WebForms, Web Services, and other client-facing API's.

What I have realized recently, is that MVP is not (or, should not be) a journey and a goal in itself. MVP, MVC, and other UI related patterns are actually an effect of other underlying principles. The underlying principles of object oriented development - such as Separation of Concerns, Single Responsibility Principle, Dependency Inversion and others - are the real cause of MVP, MVC, etc. Testability, flexibility, and the other "ilities" are merely side-effects of good design and implementation. And while Testability is a valid goal in itself, seeking it as the primary or only goal will only lead you part way down the path. You will end up like the American manufacturers that thought they were world class lean companies, only to find out that they had barely scratched the surface of lean process.

The Journey, Not The Goal

Despite the success of Toyota as the creators of lean manufacturing, they believe that they are continuously learning about lean, so they can continuously improve. Similarly, despite my many years of software development experience, I now believe that I know little about software development. I have a good knowledge of .NET and general knowledge of Object Oriented Development, but I don't know much about other paradigms like Functional Languages or Duck-Typed languages such as Ruby.

In my professional growth as a developer, I have set goals for myself at various times. Most of these goals were set with the belief that they would make me a great developer. However, I've found that every time I have reached a goal, there is an entirely new world of possibilities ahead and what I thought was a great developer was merely the beginning of a journey. I've discovered that the journey itself is often far more important than what I believed the goal to be.

I'm not advocating that you forget about setting goals and let your career happen as it will - that's a great way to become burned-out, outdated, and irrelevant. What I am advocating is that when we have a goal in mind, we set out with the understanding that the goal itself may only be a small leg on a long journey. In this case, the journey is the process of learning. The journey itself should be viewed as the long term strategy, to the point where you allow the individual goals to come and go as needed. You may not accomplish a specific set goal - but the experience gained while working toward that goal may open up opportunity for other goals to be reached easily.

Learn The Principles Via The Process

All this talk about knowing the underlying principles is great - but how can we understand the principles and philosophies without knowing the process? Unfortunately, I don't know if that's possible for most of us. I'm sure that somewhere out there, someone has read SRP, DI, SoC, and other principles, and as a result began writing code that was modeled in a manner similar to MVP (Martin Fowler, for example). For the majority of us, though, learning the principles is done by first learning the process - study the implementation of the effect to learn the cause.

Conclusions

The journey itself is how we build our experience and understanding. For most of us, that journey begins with a goal of learning a new process. Find a goal or two that you believe will help you be a better developer and learn the processes. Along the journey of reaching for this goal, always keep in mind that there is an underlying principle or philosophy that enables the process. By learning the principles and philosophies behind the process, the process itself becomes a trivial matter - an implementation of the principles and philosophies, and not a goal in itself.

I saw "The Forbidden Kingdom" this weekend. Overall it was a great movie - lots of good action, a good storyline, etc. But there was one scene that stuck in my mind specifically. Jackie Chan is pouring some drink into a cup while the person holding it is talking. The cup quickly overflows and the person holding it notes that the cup is full and now overflowing. Jackie Chan's response was something to the effect of

How can you fill your cup when it is already full? How can you learn anything when you 'know so much' already?

The point of this, of course, is that the student thought he knew so much already and wanted to get right to the advanced stuff rather than letting the teacher guide him through the basics into the advanced stuff. As silly as it is for a pop-culture movie to have deep wisdom, I really thought that this scene's message was applicable to more than just martial arts. The same can be said of any situation where knowledge, learning and growth are present.

One of my coworkers (Michael Adkins - who's blog is not publicly available, yet... *nudge nudge*) recently had a post that talked about the same thing, with a slightly different quote at the beginning. Rather than me re-hashing his post with my own interpretation, I'm re-posting his post in it's entirety (with his permission, of course).

So, you think you know everything about development eh?

A wise man once said, "He who knows everything, cannot be taught anything."  This is such a profound statement.  When it comes to learning, I attempt to look at the learning process as if I don't know any of the material (all things being equal of course).  This allows me to see things I did not see in the past.  It allows me to critically think about ways to solve problems with the new knowledge I have gained.

Sometimes when a person asks questions, it is perceived that the person doesn't have a clue as to what is going on with the process.  In my case, it is my acid or litmus test to locate a person; to see if they can articulate an answer that tells me if they are trying to help or build their ego.

Nevertheless, whether a person is helping or building their ego, they are still helping with all types of insight.  They just may not be aware of it.  This is something we all may be guilty of in some form or another.  But it just goes to show that we don't know everything.  It also goes to show that we continue to teach and instruct when we think we know everything.

This blog came about when I was going through a webcast that dealt with Delegates.

I couldn't have said it better. ... I just hope that I'm not the guy who's cup is already full.

I've been learning a lot lately - reading up on Agile, Lean Manufacturing, Lean Software Development; experimenting with NHibernate, exploring UnitOfWork concepts and generally trying to become a better software developer. In this process of learning, discussion with others, and application of knowledge to my environment, I've found that knowledge is very distinctly different than understanding.

Let's look at my use of NHibernate as an example. 2 years ago, I knew the basics of what NHibernate could do for me - I spent a few weeks learning the very basics to see how it worked and whether or not I wanted to use it. 1 year ago, I actually started using NHibernate for a project and my knowledge of it quickly grew. I knew how to create the appropriate mapping files, configure NHibernate, etc. As this knowledge grew, I became more and more interested in what NHibernate could do and how it could be applied to many different applications and situations. I knew a lot about how NHibernate was built, how it worked, and what it's capabilities were. However, all of this knowledge was not a substitute for a real understanding of NHibernate. My knowledge of how it worked led me to some conclusions that I don't think are correct anymore - trying to apply NHibernate in situations where it really was not the right answer. 

I've seen the same problem occur multiple times in the last 6 months, and in the last 11+ years of my career. There's almost a recognizable pattern to the learning process:

• Learning curve to gain working knowledge
• Knowledge growth and intimate familiarity
• Assumption that knowledge gained is a substitute for experience and understanding
• Attempts to apply knowledge incorrectly / in wrong situation
• Realization that knowledge is not understanding or experience
• Stepping back from knowledge to gain understanding of how/when to apply the knowledge

I've done this with Agile/Lean software development as well, recently. I've gained a significant academic knowledge of agile and it's processes and practices. Some of my knowledge is directly backed up by experience, so I believe I do have some understanding of the agile engineering practices. However, I've let my limited understanding mix itself into my academic knowledge a little too much. I've found myself in situations recently where I'm arguing a logical conclusion to a situation and applying it to another situation incorrectly.

One of my coworkers likes to apply this adage to situations like this:

If a hammer is the only tool you have, everything looks like a nail.

I need to remember to step back from my raw knowledge - be it academic or real world use - and let common sense and experience interweave into understanding or the realization that I don't understand. I need to understand that just because I am currently holding a hammer, and thinking about hammers, that doesn't mean that the problem in front of me is a nail that needs to be hammered.

The realization that knowledge is not a substitute for understanding can be a very distressing and/or humbling experience. The reality of learning, knowledge and understanding, though, is that I can have every last bit of knowledge on a subject - but without experience to guide me, I can't always understand the where/when/why/how of applying that knowledge appropriately.

One of the primary goals of Lean Manufacturing in the elimination of waste. This may seem like an obvious goal - after all, who wants waste in their system? What's really interesting, though, is seeing where the term "waste" is actually applied and how Toyota redefined what waste really is.

Muda, the concept of the seven wastes in the Toyota Production System, is a very different concept of waste than most of us would consider. It talks about not only physical waste (trash), but also logical, time, and effort waste. These seven wastes include (from Wikipedia):

• Defects: Quality defects prevent the customers from accepting the product produced.
  
• Overproduction: Overproduction is the production or acquisition of items before they are actually required.
  
• Transportation: Each time a product is moved it stands the risk of being damaged, lost, delayed, etc. as well as being a cost for no added value.
  
• Waiting: Refers to both the time spent by the workers waiting for resources to arrive, the queue for their products to empty as well as the capital sunk in goods and services that are not yet delivered to the customer.
  
• Inventory: Inventory; be it in the form of Raw Materials, Work-In-Progress (WIP), or Finished Goods, represents a capital outlay that has not yet produced an income either by the producer or for the consumer.
  
• Motion: As compared to Transportation, Motion refers to the producer or worker or equipment.
  
• Overprocessing: Using a more expensive or otherwise valuable resource than is needed for the task or adding features that are designed in but unneeded by the customer.

In "Lean Software Development: An Agile Toolkit", Tom and Mary discuss The Seven Wastes and offer a great set of parallels in software development. Rather than copy & paste what they've already said, I wanted to offer my own interpretation of these wastes - some of which are the same as Tom and Mary's.

Defects

This is exactly what it sounds like - a bug, a piece of functionality that doesn't work right, or an exception that gets thrown in the system and causes problems. How many hours, days and weeks have you spent fixing defects in software, in your career? Personally, I can't even begin to accumulate the time spent - it's far too great.

Part of the problem here, and what really makes it wasteful, is when defects are found and fixed in most software development cycles. In Code Complete 2, McConnell talks about the cost of defects in software, relative to when the defect is found and corrected (pages 472 & 473). If the cost of fixing a defect goes up over time, then the amount of waste also goes up. In other words, the higher the cost of fixing a defect, the more waste that defect has generated.

With the increasing cost of defects in mind, we will want to reduce the time to find a defect so that the amount of waste generated can be mitigated. In Lean Manufacturing, this is achieved by only creating what is needed for an actual customer order, right now. If one part is produced with a defect, that one part is scrapped and a new part is created, correcting the defect - the waste is minimal. Imagine a mass-production system where parts are created in batches of ten thousand. What happens if there is a defect caused by the process? We end up with a significant amount of waste, because we will have to scrap the defective parts in batches of ten thousand - the waste here, is significant.

The same principle holds true in software development. If we are writing code for extensive periods of time, and that code is not being tested until the end of that period, then the cost of finding a defect in that code goes up. If we have ten lines of code that rely on the defect working the way it does, it is significantly cheaper to fix than if we have ten thousand lines of code relying on the way the defect works. The easiest way of reducing the number of lines that rely on a defect is by working in small iterations. If that defect goes into test after two weeks, it will be much cheaper to fix than if it had gone in to test after two months. Additionally, if automated tests are done and can be run against the system on an even shorter schedule, then the cost of finding the defect may be even less. If the automated tests are run every twenty or thirty minutes, you may only have two or three lines of code that are affected by the defect.

Overproduction

There are several common terms for overproduction, in software development - Over-Engineering and Coding For The Future, for example. Both of these problems result in the same thing - too many features and more functionality than is actually needed right now. Additionally, overproduction can be called technical debt. The idea is that there is code in our system that will need change - when this code needs to be changed, how difficult is it to change; what is the cost of the change (how much debt are we in).

If a developer is tasked with creating Feature-X, they should focus on the core business value of that feature when writing the code. It's very easy for that developer to think about "what-if" scenarios and "we may need to" future functionality. However, this type of thinking leads directly to overproduction. If the business has not identified the "what-if" scenario, then why should the developer assume that handling the scenario would add business value? If the business has not yet identified Feature-Y or specified any additional functionality in Feature-X, then why should the developer assume that coding for the possibility of Feature-Y is valuable? Take note of the pattern in those questions - assumption. Assumption on the developer's part leads directly to the overproduction of the system being built.

So how do we prevent this overproduction? First and foremost - stop assuming you know what's coming down the line, or what will provide business value. If you believe that the specs you are currently working with are not accounting for an actual problem, then the responsible action is to discuss that issue with the customer / business analysts. Secondly, if you find yourself wanting to code for the feature that may be coming down the line - stop. Don't do it. Even if you know that this feature or functionality is coming down the line, it may not provide any business value for the current feature(s). You may be causing detriments to the business value of what is currently being built. If you truly believe that the feature you want to code is required for the feature that you are coding, then you need to go back to the customer / business analyst again, and discuss it with them. Let the customer tell you whether or not there is value in adding that feature now vs. later (vs. if at all).

Transportation

Think of transportation in the context of knowledge. No one knows everything about a particular domain or project, and possibly an individual feature. We have business analysts and customers precisely because of this. Eric Evans discusses knowledge crunching (the process of learning and gaining an understand of the domain in question, to create a model) in Domain Driven Design, in the first chapter. Tom and Mary talk about the process of learning all throughout the "Amplify Learning" chapter of Lean Software Development. There are countless other books on requirements gathering, model creation and other learning processes for software development, as well.

The problem that we face in our learning is that the transportation of knowledge, from one person to another, will always introduce change. Every person in existence has their own unique way of understanding, based on their experience, existing knowledge and methods of knowledge retention. To illustrate this, look back at your childhood. Did you ever play the "telephone" game where one person whispers in the ear of the person next to them, then they whisper the same thing into the ear of the person next to them, and so-on until the last person in line? What are the typical results of this game? More often than not, the message had some fundamental change to it throughout the transmissions. As simple as the telephone game is, it's a perfect illustration for knowledge transportation that occurs in software development projects.

How many times have your received a functional specification or requirement for a feature, and had no clue what it was trying to convey or what the functionality was supposed to be? This happens on a very regular basis. The problem may not be that the documentation is inadequate, but that the transportation of knowledge from one person's mind onto the paper and into your mind is inadequate - what makes sense to the document writer(s) may not make sense to you, because your contextual knowledge of the subject is different. This problem is exacerbated by design-up-front project methods. The knowledge is often crunched from the customer's mind into the BA's mind, and then trampled down into a linear document or subset of knowledge in a documentation set where the developer has no direct access to the customer - the source of knowledge - for clarification and discernment of the details. Even if the customer is on-site sitting next to the developer throughout the creation of the documentation, there is still a transportation problem in design up front projects. Whether or not the developer understands the information during the conversations, there is little chance that the developer will remember every detail of the information when they finally get down to coding.

The solution for the transportation of knowledge is ensure that the knowledge is transported as little as possible. Rather than focusing the effort of documentation on every last possible detail - creating tomes of documentation that no one is really willing to read - we should focus on creating higher level guidance; enough knowledge to know the direction of the development effort and the features to be created, with the understanding that the final details will come from the customer just-in-time for the development effort. In the world of lean manufacturing, this is typically done through the Kanban system. In software development, we have adopted the same process through the use of User Stories being written on index cards. The stories are not the final word in the development effort - they are merely a placeholder for a conversation between the customer and the developer.

Waiting

This is likely the most obvious of the seven wastes, when applied to software development. Whether you are waiting for a customer or business analyst to be available for questions; waiting for a requirements document to be signed off; or waiting for your next assignment of work, the simple process of waiting creates lost time which directly translates to lost opportunity. In the extreme cases, the problem is compounded by context switching. If you are waiting for a question to be answered - whether this means waiting for a phone call, a document to be written, or just walking down the hallway - you can easily lose your focus or lose you place in the code or coding process. When this happens, most people take a while to restore that focus and get back into the mind set where the new knowledge is useful. In my experience (no real data, just my experience), the average time of context switching within the same project is as little as five to ten minutes for wind down, and at least ten or fifteen minutes for spin-up. If you are context switching between multiple projects, though, expect to spend much longer on the spin-up; hours, even days, depending on the project and time between wind down and spin-up.

If we can ensure that we never wait longer than it takes for us to wind down, then we should be able to eliminate the spin-up period; thus reducing the effort required for reestablishing our focus on the feature(s) at hand. However, the solution for waiting may not be as obvious as the problem. If you are waiting for documentation, or if you are waiting for the customer to sign off on a feature or change, the way to eliminate the waiting is to eliminate the cause. Don't rely on heavy documentation or require customer sign-off on features. If we are working with the customer every hour of every day, eliminating the knowledge transportation issues, then we can also eliminate much of the waiting game by having the customer available. When the customer is available immediately, we have a much greater chance of the conversation beginning within the wind down period. When this occurs, we can stay focused on the problem at hand, and mitigate the spin-up period when we return to the code.

Inventory

Inventory in a manufacturing or supply context is typically a product or component of a product that is in storage and waiting to be used; or "a list of goods and materials held available in stock by a business" (Wikipedia). In software development, we only need to translate "good and materials" into "code and resources":

A list of code and resources held available in stock by a business.

Think of software inventory as unfinished code (as defined in Lean Software Development) or as overproduction of code. If we have unfinished code in our system, this is like having a physical component for a product - the component by itself is not useful until it is in the completed product. Similarly, the unfinished code provides no value to the software until it is finished. Even if the code is complete, if it is not being used in the software then it is dead code and is inventory for the software.

Inventory, in software development, carries a significant technical debt - the unknown factor of whether or not the code actually needs to be in the system. The longer we let inventory live in our code, the more likely we are to forget whether or not that code needs to be there. This inventory also becomes a potential mine field for developers - what happens if a developer sees this dead code and assumes that it is still functional and valid? If we are lucky, the code will work as advertised. The longer the code sits in inventory, though, the more likely it is to be outdated, invalid and eventually non-functioning.

Motion

In addition to physical movement causing waste through waiting (walking down the hall to the BA's office), motion can also be applied to the processes of producing the executable software (not the source code and other resources). How much effort does it take for your team to compile the resources, create the executables, package it into an installation and deliver it to QA, the customer, etc? The goal should be to make this as simple as possible - as little movement as possible.

Are you able to click a button and have the build, package, and delivery processes handled for you automatically? Or does it take a team of developers, build engineers and other persons with a manual process and a checklist of things to do? If your process involves people performing manual tasks - even if that task is simply the order in which multiple buttons get clicked - then you are at risk of creating waste through defects. Every manual motion in your build process is another point at which mistakes can be made, allowing defects to be built or causing the build to fail completely.

Software production (again - speaking about production of executable software, not development of resources or code) is often a difficult, tedious chore, requiring many moving parts - there is no way around this. However, it does not have to be a manual process. Continuous Integration and Continuous Deployment practices are abundant in the world of software development, specifically to address this issue. If you can automate your build process and eliminate the manual movement involved, you can eliminate a huge source of potential waste.

Overprocessing

Overprocessing, or too much process, is abundant in software development. When a customer has a request for a feature, how much process does it take for that feature to become functionality? In design-up-front project life cycles, the process of adding or changing a feature can be very time consuming, involving many steps on both the requester and implementer side:

request; acknowledgement and change management / approval; design and documentation by implementation staff (BA's, developers, etc); sign off of design and documentation by customer; implementation in software; scheduled for testing; tested; and if any issues are found, back to design or implementation for corrections, repeating several steps.

Additionally, how much work does it take for you to get your job done, as a developer? Do you spend as much time updating task / issue management tickets and managing your manager, as you do coding?  Too much process in software development is as bad, if not worse, than no process at all. Unfortunately, the failures of too much process are often seen as an indication of the need for more process, causing further delays and waste in the system.

There are often situations where you cannot do anything about the process that is in place. Life-Critical systems (x-ray machines, air plane guidance systems, etc) are abundant with process for good reason - if anything fails, people could be injured or killed. In the world of business software, though, this is not usually the case. Most business has a level of fault tolerance built into it, and some waste is expected / accepted. This is not a get out of jail free card, though. The more significant the waste, the more repercussions there are - fix the defect; pay for lost revenue; lose your job; etc. If we want to eliminate waste through the simplification of process, then we need to ensure that defects are found and fixed as early as possible, to avoid creating waste for the business using the software.

Process simplification can be done many different ways. Start with the creation of a process flow char, or value-stream chart. Find out how much time and effort is wasted in your processes and look for the waste that can be easily eliminated. Simplifying a single process by removal of waste can lead to a revolution in your process. You may find that the removal of one process negates the need for another process. You may also find that a simple change to one process does the same. If you are able to bring your customer into the work area with your team, and work in short iterations where defects are found and fixed quickly, you will likely see many of your old processes quickly become obsolete.

Conclusions

No one wants waste in their systems. Waste is money lost and revenue not earned. Waste management, though, can be a difficult proposition. Many company are willing to accept waste as the cost of doing business and are not willing to spend money eliminating waste because they do not see the waste or do not believe it can be eliminated. If we are truly going to revolutionize the business we work in, we must eliminate waste. Start where you have direct control or influence. You'll quickly find others wanting to know how you are so productive and soon the entire business will be in full waste reduction cycles.

I've been hearing this a lot lately, and finally started reading up on the roots of the Agile movement. It all goes back to Lean Manufacturing. Oddly enough, I've had exposure to Lean Manufacturing at a previous job. So the concepts and names are all very familiar to me. Back in those days, I had heard of the concept of Lean Software Development, but it never really made any impact on how I wrote software. These days, though, I'm diving into as much Agile as possible and the further down the road that I travel, the more and more parallels I see, to the lean manufacturing world.

If you are at all interested in the roots of agile, here's some recommend reading (mostly Wikipedia) to get you started.

Wikipedia:

• Lean Manufacturing
• Lean Systems
• Lean Software Development
• Toyota Production System
• Kanban

Books:

• The Toyota Way
• Lean Software Development: An Agile Toolkit
• (anyone have other suggestions?)

I'm hoping to absorb more of this information, quickly. It seems the more I learn about lean manufacturing, the more I understand about software development. I certainly don't claim to be an expert on either subject... hopefully I can change that.

Way back in the day (somewhere around 1994 or 1995, I think), I saw the movie, "Search for Bobby Fischer". One of the lines that has always stuck in my head is from the teacher, Vinnie:

He didn't teach you how to win, he taught you how not to lose.

At the time, I didn't like this line - I didn't understand why playing "not to lose" was different than playing to win. Over the years, I've seen variations of this statement and had a little more understanding of what it actually means. Last night, though, I was involved in various philosophy-of-software discussions with a friend and this quote popped up in my mind again, in the context of project failure and the fear of failure. I think I finally understand what this quote is saying, now - and it all comes down to the fear of failure vs. embracing failure as an opportunity to learn and find new ways to succeed.

Knowing Success Through Failure

Looking back at my life, I have two very distinct memories of catastrophic failure - the kind of failure where I was so emotionally invested in the project or goal, that the failure was truly devastating to the point where I would have a complete breakdown and be entirely lost for a while.

The first of these memories was back in College. I have an associates degree (AA) in music, from a junior college back in my hometown in Kansas. After graduating with that degree, I found another school in Denver that had a sound recording / engineering track in their music school and I signed up for it with all the passion and motivation that I now have for software development. When I moved on to campus at that school and started figuring out what classes I would be taking, etc., I found out that I had slipped through the cracks in the school of music. As it turns out, I was supposed to have interviewed for the school of music and done some performance in front of various professors, to be accepted. With that in mind, I worked with the director of the program and did my performances - and completely failed. As it turns out, I had a general skill with musical instruments and could play just about anything at a very mediocre level, but I was not terribly good at any instrument. The end is result is that I was told (in very kind, helpful terms) that I would need a few years of lessons before I could be accepted into the school of music. Needless to say, this was a very devastating experience for me. My goal and my dream, at the time, was to be a musician and work in a recording studio. Shortly after the realization that my goal was unattainable, I had various talks with various people close to me and some talks with the director of the music school. We quickly realized that I could still attain the primary goal in my dream - to work in a sound recording studio and be in the sound recording / engineering program at the school - but my specified major would not be music. I ended up in the Management Information System department, with that being my major, while still taking the sound recording / engineering classes.

The second of these memories is significantly less important in the grand scheme of my life. A few years ago, I was attempting to install a new car stereo system in my car. I had done stereo and speaker installation before, but had never done an amp and subwoofer. Over a 4 day weekend, I ended up with a car that had no working speakers, a blown amp, and a subwoofer with holes punched in it. At the time, this was another very devastating outcome for me - I had a very specific goal in mind and had spent a significant amount of time and money on the project, with no usable results. Two days later, I had my car in a professional car audio shop with professional audio installation personnel fixing the problems and installing a new amp and subwoofer. The end result was that I had a great system in my car.

Looking back at both of these experiences, I see a pattern: a project or goal, a significant personal investment, a catastrophic failure and finally, an outcome that still satisfied my criteria for success.

There's an old cliche line that I've heard many times:

You can't know true success unless you know true failure.

I believe I've been as close to true failure as I can get, without giving up on a project or goal. True failure, though, can only come from giving up.

Playing To Win vs. Playing Not To Lose

My career as a software developer has been very interesting, so far. I've had a lot of great success and a few fairly large failures. What's really interesting to me, though, is looking back at those failures and seeing the risk that I was willing take. I've jumped into some very difficult, new, demanding situations with little or no knowledge and experience in those situations. Many times, I was able to find the experience quickly and succeed, but some times I was not able to find the experience quickly enough and I failed. These successes and failures are why I believe I understand the issue of playing to win vs. playing not to lose, now. As I said before, it all comes down to the fear of failure vs. embracing failure as an opportunity to learn and find new ways to succeed. With this in mind, I have some questions that may help you understand playing to win vs. playing not to lose.

Are you willing to take risks in your project? Are you willing to put forth the effort involved in trying to create what could be a better you, or a better project, knowing that you could fail if you try? Are you willing to open yourself for failure so that you can learn, and have a greater chance of greater success?

Are you playing to win?

Or are you only willing to do the work that you are comfortable with now, and only willing to learn a little bit of something new, keeping a safety net around yourself? Are you willing to keep yourself from taking risk so that you don't have any significant chance of failure, and no significant growth? Are you afraid of change or afraid that you will fail?

Are you playing not to lose?

Good Judgement and Experience

Another tired old cliche that I've heard a thousand times:

Good judgement comes from experience. Experience comes from bad judgement.

I make a dozen bad decisions every day. I also make a dozen good decisions every day, because I am willing to honestly look at the decisions I've made and admit that they were bad decisions. By allowing myself to understand that these decisions were bad decisions, and admitting that to not only myself, but to my managers and team members as well, I set myself up for success. I'm able to look at these decisions, understand why there wrong, and correct the mistakes. And by doing this on a daily basis, if not more often, I can mitigate the failure that any one decision may cause.

Embracing Failure Without Fearing It

One of the reasons I am so in love with Test Driven Development and Behavior Driven Development is that it accelerates the failure-to-success process. I'm not unit testing to prove that I have not failed. I'm unit testing to prove that I have succeeded and created the API that I want.

I expect failure when I unit test. I expect to write horrible APIs that are unusable, ugly, stupid, and obnoxious. I WANT to realize that I am doing this, through my unit tests, because I can accelerate the process of realization and correction by using my unit tests as a design tool.

And I challenge you to do the same - to stop fearing failure and not be afraid to make mistakes. If you believe you have made a mistake, expose it for peer review. Ask for others to validate or invalidate that mistake and learn from the experience. Find the tools, techniques, coworkers and peers that will help you accelerate the realization of failure so that you can correct the failure and succeed, faster.

A Personal Rant: My Success As A Developer

I believe I am a very successful software developer and leader. I have a lot of experience under my belt and a lot of knowledge that I want to share with others.

So, why am I successful? Why was I responsible for the public face of a 300 million dollar multi-national corporation, through it's web sites, when I was 22 years old? Why was I made the lead developer in a team of 10, when I was the new guy in the office? Why am I now diving into the cutting edge of Behavior Driven Development and actively involved in the Agile community? Why am I making a name for myself not only in my office, but also in the development world outside of my office?

Because I take risk and embrace my failures. I openly admit my failures at the first possible moment, so that I can learn. I actively seek the voice of dissent and the views that I don't agree with, so that I can be proven to be wrong and learn how to be right.

Why am I successful? Because I play to win.